JP2015216647A - Display control device and display control device control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a display control device, method, and program for improving user's convenience relating to image reproduction.SOLUTION: The display control device comprises: first acquisition means for acquiring image data; second acquisition means for acquiring information corresponding to an analysis result of the image data; display control means which causes a display unit to display a bar B for allowing a user to designate the reproduction start point of the image data; and determination means which determines the length of a portion of the bar B corresponding to a divided period obtained by dividing a period relating to the reproduction of the image data acquired by the first acquisition means, on the basis of the information acquired by the second acquisition means.

Description

The present invention relates to a display control device and a control method for the display control device, and more particularly to a technique suitable for use in order to enable a user to easily search for an important scene in consideration of user preferences.

  In recent years, with the development of video analysis technology and voice recognition technology, videos with abundant metadata are increasing. Information that can be given as metadata includes, for example, information on who the person is in and information on where a child can laugh. There has been proposed a method that makes it possible to quickly find a desired scene by using metadata, or to view only a scene that matches an individual's preference (see Patent Documents 1 and 2).

  On the other hand, with the development of communication networks in recent years, a viewing mode in which a video stored in a video distribution device is streamed and played on a display device at hand by a plurality of users through the network is spreading. In the future, it is considered that the video distribution system will need to provide a user interface that can easily find a scene important for the user in consideration of each user's preference.

  In addition, it is convenient for the user to refer to a frame cut out from the video in order to search for an important scene for the user from the video. That is, the user moves the slider and a representative frame is displayed on the video display screen. In parallel with streaming reproduction, a method for acquiring a plurality of video frame data existing within a certain prefetch target period has been proposed (see Patent Document 3).

JP 2001-292402 A JP 2007-208651 A JP 2006-287875 A

  In an existing display device, a user interface called a seek bar is used. The seek bar shows the amount of playback of the entire video projected onto the length on the display screen. Further, when the user moves the slider on the seek bar to a certain position by some operation device, the video is reproduced from the time point corresponding to the position. That is, by moving the slider, the video can be reproduced from an arbitrary time point.

  Usually, since the passage of time of video and the position on the bar are associated with each other in a proportional relationship, it is possible that a scene important for the user is biased on the seek bar. Then, when moving the slider, it is necessary to move the slider largely in order to skip an area that is not important to the user. On the contrary, in order to finely seek an area where important scenes are gathered, a fine operation of the slider is required. In this case, there is a problem that convenience for video reproduction for the user is impaired.

  The present invention has been made in view of the above-described problems, and an object thereof is to improve convenience related to video reproduction for a user.

  The display control apparatus of the present invention allows a user to specify a first acquisition unit that acquires video data, a second acquisition unit that acquires information according to the analysis result of the video data, and a playback start point of the video data. Display control means for displaying a bar for display on the display unit, and a length of the bar portion corresponding to a divided period obtained by dividing a period related to reproduction of video data acquired by the first acquisition means. (2) determining means for determining based on information acquired by the acquiring means.

  According to the present invention, it is possible to improve convenience related to video reproduction for a user.

It is a figure explaining the specific example of the seek bar generally used. It is a figure which shows embodiment of this invention and demonstrates the specific example of metadata. It is a figure which shows embodiment of this invention and demonstrates the specific example of user preference data. It is a block diagram which shows the example of 1 structure of the video delivery system of embodiment of this invention. It is a sequence diagram which shows embodiment of this invention and shows an example of the whole flow. It is a figure explaining the specific example of the user preference data described by XML. It is a flowchart explaining the detail of the delivery preparation in FIG. It is a figure which shows embodiment of this invention and demonstrates the specific example of frame data. It is a figure which shows embodiment of this invention and demonstrates the specific example of a non-linear seek bar. It is a figure explaining the specific example of user interface data. It is a flowchart explaining the detail of the delivery in FIG. It is a figure which shows a mode that frame data are dynamically created from the periphery of a frame.

First, a general seek bar will be described. In an existing display device, a user interface called a seek bar is often used.
FIG. 1 is a diagram illustrating an example of a general seek bar. In this user interface, the bars B are arranged in a straight line. This bar B shows the amount of playback of the entire video projected onto the length on the display screen. Usually, the left end corresponds to the beginning of the video, the right end corresponds to the end of the video, and the passage of time of the video and the position on the bar are often correlated in a proportional relationship.

  When the user requests video playback, the video is displayed on the video display screen D. During the reproduction of the video, the slider SL above the bar B moves on the bar B so as to correspond to the time point being reproduced. Further, when the user moves the slider SL to a certain position by some operation device, the video is reproduced from the time point corresponding to the position. That is, by moving the slider SL, it is possible to reproduce the video from an arbitrary time point.

  Such a general seek bar has the following problems. For example, it is assumed that the user has a preference to see a scene in which the user is shown. At this time, when the scene in which the user is shown is projected onto the seek bar, it is assumed that the scenes are arranged as I1 to I10 in FIG. Usually, since the correspondence between the bar B and the time point is in a proportional relationship, it is possible that scenes important for the user are biased on the seek bar.

  Then, when the slider SL is moved, the slider SL must be largely moved in order to skip an area that is not important to the user. Conversely, in order to finely seek an area where important scenes are gathered, a fine operation of the slider SL is required. In this case, there is a problem that it is impossible to reproduce a video that is convenient for the user.

Hereinafter, embodiments of the present invention will be described.
First, terms used in the following description are defined.
“Metadata” is data that accompanies the video, and is data that indicates the characteristics of the video, such as what scenes exist at which time points in the video. Specifically, among face and object recognition information, landscape recognition information, zoom operation information applied to the video camera that captured the video, volume change information, audio recognition information, tagging information by the user himself Including at least one. For example, the metadata includes information that a child's face starts to appear at a time point of 0 minute 14 seconds 16 of the video. There are both cases where the metadata is embedded in the video and cases where the metadata is managed separately from the video. In the present embodiment, the latter case will be described. FIG. 2 shows an example of metadata.

  “User preference data” is a score given to various information given as metadata. This score represents the degree of preference the user has for each piece of metadata. FIG. 3 shows an example of user preference data.

  An “important scene” represents a time point of a video corresponding to a type of metadata having a high score in user preference data.

  FIG. 4 shows a configuration of a typical video distribution system in the embodiment of the present invention. In the present embodiment, the video distribution device 100 and one or more display devices 200 are used. The video distribution device 100 and the display device 200 are connected via a network 300. The user views the video stored in the video distribution device 100 via the network by operating the display device 200.

  The components of the video distribution device 100 will be described. The receiving unit 101 and the transmitting unit 109 exchange information with the display device 200. The video database 103 stores a video corresponding to the video ID. Here, the video ID is an ID for identifying a video. The metadata database 104 stores video metadata corresponding to the video ID.

  The user preference data database 105 stores user preference data transmitted and acquired from the display device 200. The database management unit 102 manages the video database 103, the metadata database 104, and the user preference data database 105. In the present embodiment, a method using user preference data transmitted from the display device 200 is used. However, the database management unit 102 may estimate user preferences and create user preference data.

  The important scene determination unit 106 creates important scene information of the video based on the metadata weighted by the user preference data. “Important scene information” is a collection of information on how important scenes there are at which time points in a video. The user interface data creation unit 107 creates user interface data customized so as to be suitable for searching for an important scene for the user based on the obtained important scene information. The frame data creation unit 108 creates frame data from the video based on the important scene information, video, and seek information. Here, the “frame data” includes a frame extracted from the video and data related to the frame.

Next, components of the display device 200 will be described.
The operation unit 201 is connected to the operation device 400 and receives an operation from the user. The operation unit 201 also creates seek information that is information indicating which frame in the video the user has selected. The display unit 203 outputs the video received from the video distribution apparatus 100 to the display device 500 for display.

  In addition, the display unit 203 receives user interface data from the video distribution apparatus 100, and creates and displays a user interface based on the user interface data. Further, the display unit 203 receives frame data from the video distribution device 100 and appropriately displays a frame when the user performs a seek operation. The transmission unit 202 and the reception unit 204 exchange information with the video distribution device.

Next, the flow of processing in this embodiment will be described with reference to FIG.
First, the display device 200 transmits user preference data of the user to the video distribution device 100 (S100). The transmitted user preference data is stored in the user preference data database 105 of FIG. 4 (S200). A specific example of user preference data to be transmitted will be described later.

  Next, the display device 200 transmits the ID of the desired video to the video distribution device 100 (S300). The video distribution apparatus 100 performs distribution preparation (S400) based on this, and transmits the created frame data and user interface data to the display apparatus 200. Details of the distribution preparation (S400) will be described later.

  Subsequently, the display apparatus 200 notifies the video distribution apparatus 100 of a distribution start request (S500). Based on this, the video distribution apparatus 100 distributes the video (S600) and transmits the video to the display apparatus 200. Details of distribution (S600) will be described later. The above is the flow of processing in this embodiment.

Next, a specific example of user preference data transmitted from the display device 200 to the video distribution device 100 will be described with reference to FIG. Here, the user preference data is expressed in the XML format (extensible markup language format) as a data format when received from the outside, or it is binarized, and the following contents are described. ing.
(1) ID for identifying the user, (2) metadata class and type, and (3) score given to the metadata.

  FIG. 6 shows an example in which a user having an ID “0000123” sets his / her preferences for four types of metadata. Here, the first description of the user preference data means that a score “5” is given to “night scene” belonging to the “landscape” class. The second description means that the score “4” is given to the ID “0000123” (that is, the user himself) belonging to the “face” class.

  The third description means that a score “4” is given to “favorites” belonging to the “user-defined” class. Here, the user-defined class is a class to which metadata tagged by the user himself belongs. The fourth description means that a score “−3” is given to “noise” belonging to the “sound” class. Here, a negative score means a preference of not wanting to watch this scene.

Next, a specific example of the delivery preparation (S400) method in FIG. 5 will be described in detail with reference to FIGS.
When the distribution preparation is started (S701), the video distribution apparatus 100 receives the video ID from the display apparatus 200 (S702). The database management unit 102 checks whether there is a video corresponding to this video ID and confirms the validity of the video ID (S703). If the video ID is not valid, an error is notified to the display device 200 (S704), and the preparation for distribution ends (S705). If the video ID is valid, the video corresponding to the video ID is read from the video database 103 (S706).

  Subsequently, the database management unit 102 checks whether or not the metadata given the score in the user preference data exists in the metadata database 104 (S707). If the corresponding metadata does not exist, it is checked whether the metadata can be created by the video distribution apparatus 100 (S708).

  If the metadata can be created, a metadata creation process for dynamically creating the metadata from the video is executed (S711), and the metadata database 104 is updated (S712). If the metadata cannot be created, an error is notified to the display device 200 (S709), and the reproduction preparation is terminated (S710). Of course, this method shown in FIG. 7 can be omitted, and when the metadata does not exist, the user preference data corresponding to the metadata can be ignored and the processing can be continued.

  In this way, after extracting the metadata (S713), the important scene determination unit 106 creates important scene information indicating which scene is important to whom at which time point of the video based on the extracted metadata. (S714). At this time, according to the score given in user preference data, the height of the importance of a scene can be weighted.

  Based on the important scene information, the frame data creation unit 108 creates frame data in an important scene (S715), and the user interface data creation unit 107 generates user interface data optimized for the display device 200. Create (S716). Then, the transmission unit 109 sends the frame data and the user interface data to the reception unit 204 of the external display device 200 connected via the network 300, the video distribution preparation is completed (S717), and the process is finished ( S718).

Next, a specific example of frame data transmitted from the video distribution device 100 to the display device 200 will be described.
As described above, the frame data includes a frame extracted from the video and data related to the frame. An example of the frame data is shown in FIG. The frame is extracted from the video and given an ID for identification. The data regarding the frame is expressed using, for example, the XML format of FIG. The XML in FIG. 8 describes, for each frame, information as to which time point of the video the frame was created from and the class and type of metadata that caused the frame to be extracted.

  In FIG. 8, a method of collecting all data related to a plurality of frames into one XML is used, but a method of preparing one XML for each frame may be used. By transmitting the frame data configured as described above to the display device 200, the frame can be arranged at a corresponding place on the seek bar on the display device 200 side. This frame data is transmitted when preparing for video distribution and when the user performs a seek operation (described later).

  FIG. 9 shows an example of a typical user interface in the present embodiment. The seek bar in FIG. 9 has a small time width corresponding to the moving distance of the slider in an area where many scenes important for the user are gathered. Conversely, for areas where there are few important scenes for the user, the time width corresponding to the moving distance of the slider is set large. Thus, by providing a user interface in the form of a seek bar that is non-linear with respect to time, it is possible to reproduce video according to user preferences.

Next, an example method for creating a non-linear seek bar will be described.
First, the total time of the video is equally divided by an appropriate number N. Next, for each divided section, an appropriate score is given with the number and types of important scenes existing in the section as parameters. Various methods can be used to give the score, for example, a method that gives a high score to scenes that are important to the user, and a method that gives a negative score to scenes that the user does not want to watch. Is possible.

Then, the seek bar is divided so as to be proportional to the score of each section and assigned to each section. In this way, a non-linear seek bar can be constructed.
When the division number N is 6 and the score calculation method is “add 1 to the number of important scenes existing in each section”, the seek bar shown in FIG. .

Next, a specific example of user interface data transmitted from the video distribution apparatus 100 to the display apparatus 200 will be described with reference to FIG.
In FIG. 10, the user interface data is expressed in the XML format, and the following contents are described.
(1) User interface class (here, seek bar), (2) total time of video, (3) correspondence between video time point and seek bar position. The display device 200 can receive such user interface data and create a user interface.

Next, the method of distribution (S600) in FIG. 5 will be described in detail with reference to FIGS.
As described above, the video distribution device 100 transmits frame data corresponding to an important scene to the display device 200 prior to video distribution.
When distribution is started (S1101), the video distribution apparatus 100 receives a distribution start request from the display apparatus 200 (S1102). Next, the video distribution apparatus 100 first sets the reproduction start position (S1103), and transmits the video to the display apparatus 200 (S1104).

  Here, it is assumed that the user enters a seek operation and selects one of the frames in the video (S1105). Information indicating which frame is selected is sent as seek information from the display device 200 to the frame data creation unit 108. The frame data creation unit 108 creates frame data at a time interval shorter than the time interval between the previously extracted frames within a certain range before and after the selected frame (S1106), and transmits the frame data to the display device 200. (S1107). It is determined whether the frame selection operation by the user has been completed (S1108). If completed, the playback start position is corrected (S1109), and the video is retransmitted to the display device 200 (S1110). If the user's frame selection operation is not completed in S1108, the process returns to S1105.

  In S1111, it is determined whether or not the reproduction is finished. If the reproduction is not completed, the process returns to S1105, and if the reproduction is completed, the process proceeds to S1112 to perform an end process.

Next, FIG. 12 shows an operation example when the user performs a seek operation.
The positions of frame data corresponding to important scenes sent to the display device 200 prior to video distribution are I1 to I10.

  FIG. 12 shows that when the user slides the slider SL to the position I6, frame data is dynamically created around the slider SL and arranged in J1 to J3 on the seek bar. As a result, the user can seek more finely.

  When the frame selection operation by the user is completed, the video distribution apparatus 100 corrects the reproduction start position (S1109) and redistributes the video (S1110). In this way, by transmitting frame data in stages, the user can easily seek scenes that are important to the user over the entire area of the video, and more finely without wasting communication bandwidth. Can seek.

Next, a method for extracting a frame to be sent when the user enters a seek operation will be described.
Several methods can be applied to create frame data to be sent when the user enters a seek operation. As a first method, there is a method of creating several pieces of frame data at a time point separated from a selected frame by a certain time. For example, when the selected frame is extracted from the 22.50 second of the video, from the time point separated by 2.00 seconds before and after 22.50 seconds and the time point separated by 1.00 seconds before and after. Create frame data.

  As a second method, there is a method of extracting a frame corresponding to the preference of a user other than the user who is currently viewing the video. Hereinafter, a user who wants to view a video is referred to as “viewing user”, and a user other than the viewing user is referred to as “other user”. As described above, the important scene determination unit 106 creates the important scene information of the video based on the user preference data and the metadata.

  At this time, not only important scene information for the viewing user is created from the viewing data of the viewing user, but also important scene information for the other user is created from the preference data of the other user. This makes it possible to create frame data corresponding to a scene important for other users. As a result, when the viewing user performs a seek operation, an effect of noticing a scene that the viewing user alone did not notice can be expected. However, from the viewpoint of personal information protection, it is desirable that the range of other users can be set in advance.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (computer program) that implements the functions of the above-described embodiments is supplied to a system or apparatus via a network or various computer-readable storage media. Then, the computer (or CPU, MPU, etc.) of the system or apparatus reads out and executes the program.

  DESCRIPTION OF SYMBOLS 100 Video delivery apparatus, 101 Receiving part, 102 Database management part, 103 Video database, 104 Metadata database, 105 User preference data database, 106 Important scene determination part, 107 User interface data creation part, 108 Frame data creation part, 109 Transmitter, 200 Display Device, 300 Network, 400 Operation Device, 500 Display Device

Claims (16)

First acquisition means for acquiring video data;
Second acquisition means for acquiring information according to the analysis result of the video data;
Display control means for causing a display unit to display a bar for allowing the user to specify the playback start point of the video data;
A determination unit that determines a length of the bar portion corresponding to a divided period obtained by dividing a period related to reproduction of the video data acquired by the first acquisition unit based on information acquired by the second acquisition unit. And a display control device.   The display control apparatus according to claim 1, wherein the information according to the analysis result includes information for specifying a predetermined scene in the video data.   The display control apparatus according to claim 2, wherein the predetermined scene is specified based on metadata including information related to video characteristics included in information according to the analysis result.   4. The display control apparatus according to claim 1, further comprising an analyzing unit that generates information corresponding to the analysis result from the video data by analyzing the video data. 5.   5. The display control apparatus according to claim 2, wherein the predetermined scene is specified based on information according to the analysis result and preference data relating to a user's preference. 6. A determination means for determining the importance of the scene specified based on the information according to the analysis result based on the preference data;
The display control apparatus according to claim 5, wherein the predetermined scene is a scene whose importance determined by the determination unit is a first predetermined value or more.   The said determination means determines the length of the part of the said bar corresponding to the said division | segmentation period based on the presence or absence of the said predetermined scene in the said division | segmentation period, The one of the Claims 2 thru | or 6 characterized by the above-mentioned. The display control apparatus according to 1.   The display control apparatus according to claim 6, wherein the determining unit determines the length of the bar portion corresponding to the divided period based on the importance of the predetermined scene in the divided period. .   The determining means makes the length of the bar portion corresponding to the divided period having the predetermined scene longer than the length of the bar portion corresponding to the divided period not having the predetermined scene. The display control device according to claim 2, wherein the display control device is determined as follows.   The determining means has a total importance of the predetermined scene in the first divided period equal to or greater than a second predetermined value, and a total importance of the predetermined scene in the second divided period is smaller than the second predetermined value. The length of the bar portion corresponding to the first divided period is determined to be longer than the length of the bar portion corresponding to the second divided period. 9. The display control device according to 8.   11. The display according to claim 1, wherein the determination unit determines a length of the bar portion corresponding to each of the plurality of divided periods having the same time width. Control device.   The information according to the analysis result includes recognition information of at least one of a face, an object, and a landscape, information about a zoom operation applied to a camera that has captured a video, information about a change in volume, audio recognition information, and a user The display control apparatus according to claim 1, wherein the display control apparatus includes at least one of the information added by the control unit.   The display control according to any one of claims 5, 6, 8, and 10, wherein the preference data includes a score given to information included in information according to the analysis result. apparatus. A first acquisition step of acquiring video data;
A second acquisition step of acquiring information according to the analysis result of the video data;
A display control step of causing a display unit to display a bar for allowing the user to specify the playback start point of the video data;
Based on the information acquired in the second acquisition step, the length of the bar portion corresponding to the divided period obtained by dividing the period of reproduction of the video data acquired in the first acquisition step is determined. A display control device control method.   The method according to claim 14, wherein the information according to the analysis result includes information for specifying a predetermined scene in the video data.   A program for causing a computer to operate as the display control apparatus according to any one of claims 1 to 13.

Images