JP2004110096A - User operation log visualizing method, user operation log visualizing device, program and its recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To visually recognize the click operation log of a user to an image. <P>SOLUTION: This user operation log visualizing device is provided with a means 201 for storing user operation log data; a means 202 for selecting necessary log data from the stored log data; a means 203 for shaping the log data to facilitate retrieval with a time position as a key; a means 207 for reproducing the image; a means 206 for acquiring the reproduced time position of the image; a means 204 for acquiring log data in the reproduced time position; and a means 205 for visualizing the log data and making a superposed display on an image screen. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an interactive video content providing technology using a user operation on each subject in a video provided as main content, and in particular, a method, apparatus, and program for superimposing and displaying user operation log information on a video as main content And its recording medium.
[0002]
[Prior art]
As an example of an application utilizing a user operation of designating a subject in a video (such as mouse click), there is an application that presents information related to a subject designated by a user (for example, Non-Patent Document 1, Non-Patent Document 1). Reference 2).
[0003]
FIG. 5 is an example of such an application, and shows an example of an electronic aquarium application. In FIG. 5, it is assumed that an image 501 of an aquarium tank is displayed on a screen 500, and a fish 502 appears as a subject in the image 501 of the aquarium. At this time, when an event occurs in which the user designates the fish 502, a text 503 that is a detailed description of the fish is presented.
[0004]
In such an application, a temporal and spatial partial region including a subject in a video is defined as an anchor, and the relationship between the anchor and another material is defined in advance. FIG. 6 shows an example in which an anchor area including a subject in a video is defined. In FIG. 6, a rectangle 601 is defined so as to include a subject (fish) 602 at a time position t1 of the video 600. Similarly, rectangles are defined at time positions t2 and t3 so as to include the subject. A hexahedron 603 having two rectangles defined at the time positions t1 and t2 is an anchor area between the time positions t1 and t2. Similarly, an anchor area of a hexahedron 604 having two rectangles at t2 and t3 is defined.
[0005]
The application obtains the time position and spatial coordinates in the video when the user clicks, determines whether or not the coordinates are included in the anchor area, and if so, preliminarily associates with the anchor area. Display the material. In such an application, it is possible to acquire a click operation log for a user's video by saving the time position and spatial coordinate data in the video when the user clicks.
[0006]
Further, by including the identifier of the anchor area in the user operation log, it is possible to identify which subject in the video the user has clicked. FIG. 7 shows a log output example of such a user operation. In the log of FIG. 7, in response to the user's operation of clicking the video, the video time position, the spatial coordinates in the video, the identifier of the anchor area, and the user ID are output. The example of the log 702 indicates that the user “tim” clicks the coordinates of (50, 50) in the 1000 ms frame from the beginning of the video. The coordinates of (50, 50) in the 1000 ms frame are included in the anchor area “fish-A”.
[0007]
In the interactive video content providing service, such a click operation log for the user's video is important for analyzing the user's interest, extracting scenes that are easily clicked by the user, and scenes that the user does not click often. Data.
[0008]
[Non-patent document 1]
Tetsuo Ikeda, Yasuhiro Hirano, Toru Yuguchi, Takashi Hoshi, "Consideration of Development Environment for Data Model of Multimedia Application", Transactions of Information Processing Society of Japan, Vol. 40, no. 1, pp. 2 to 12, 1999
[Non-patent document 2]
Toru Yuguchi, Tetsuo Ikeda, Takashi Hoshi, "Developing Video Hypermedia for PC", NTT Technical Journal, March 1999 [0009]
[Problems to be solved by the invention]
However, in the above application, when focusing on one user operation log data, it is possible to identify which subject the user clicked on by an anchor identifier in the log data. Conventionally, it has been difficult to know whether the user clicked in the scene or which part of the subject was clicked.
[0010]
In the example of the log 702 in FIG. 7, the spatial coordinates (50, 50) in the image clicked by the user “tim” cannot be determined from the log 702 only from which part of “fish-A”. . In addition, it is also possible to know what kind of scene the user clicked specifically, such as whether the “fish-A” is a scene reflected up, or how many other fish are simultaneously displayed. Can not.
[0011]
In order to know the specific scene clicked by the user, it is necessary to actually reproduce the video and adjust the reproduction time position to the scene of the time information described in the log data. In addition, it is necessary to determine which part of the scene has been clicked on the basis of correspondence with the spatial coordinates in the video.
[0012]
It is also conceivable that the user clicks on a location that is not defined in advance as a clickable area. In this case, since the clicked coordinates are not defined as the anchor area, the identifier does not exist and cannot be acquired. Therefore, the information that can be obtained is only the numerical data of the time coordinate and the space coordinate in the video when the user clicks, and it is more difficult to specify what the user clicked.
[0013]
The log 703 in FIG. 7 corresponds to this example. The user “joe” clicks the coordinates (150, 240) at the time position 1000 ms, but since the anchor area is not defined, it is not possible to know what was clicked only from the log information. .
[0014]
As described above, in the related art, there is a problem that a troublesome operation is required to specify which scene the user has clicked on.
[0015]
On the other hand, it has been conventionally possible to obtain distributions of a plurality of user operation logs on a time axis and a space axis. FIG. 8 shows the log data of FIG. 7 shaped based on the time axis. From the log data in FIG. 8, it can be seen that the clicks are concentrated from the time position 1000 ms to 1100 ms, and that the clicks are not performed until 1530 ms after 1100 ms.
[0016]
Also, by mapping the log data on the x-y two-dimensional space, it is possible to know the spatial coordinates of the image that have been frequently clicked and the spatial coordinates of the image that have been clicked less. However, the information that can be understood only by the distribution is that the click is concentrated at which spatial coordinate at which time and is dispersed, and the distribution is concentrated at the scene where what is reflected in what scene In order to know the fact, it is necessary to actually play back the video and check the target time coordinates and space coordinates, which is a very troublesome operation.
[0017]
In particular, in order to grasp what scenes the user clicked frequently, the temporal partial area, the space where the user did not click much, and the temporal partial area throughout the entire video, the playback scene It is necessary to create a distribution on the spatial coordinates in the reproduction scene of the video while gradually changing the distribution of the video, and compare the distribution with the video screen to confirm the distribution, which is not a very realistic operation.
[0018]
SUMMARY OF THE INVENTION An object of the present invention is to make it possible to easily identify what kind of scene it was and what click was made in a click operation log for a video of a user.
Another object of the present invention is to make it possible to actually see the distribution of a plurality of click operation logs in association with a video, and to easily grasp the tendency of the click operation.
[0019]
[Means for Solving the Problems]
In one embodiment of the present invention, only the necessary log data is selected from the accumulated user operation log data, and the data is formatted so that the data search using the time position as a key is easily performed, and while reproducing the video, The reproduction time position of the video is obtained, the log data at the reproduction time position is obtained, and the log data is visualized and superimposed on the video screen. As a result, it is possible to easily identify which scene and what user clicked. In addition, the distribution of the user's click operation log can be viewed in association with the actual video, and the tendency of the click operation can be easily grasped.
[0020]
BEST MODE FOR CARRYING OUT THE INVENTION
FIG. 1 shows a flowchart of an embodiment of a method for visualizing a user operation log according to the present invention.
Here, it is assumed that the user operation log data is already stored in a predetermined log data storage unit. First, only necessary log data, such as a desired user, is selectively extracted from the log data stored in the log data storage means (step 101), so that the extracted log data can be easily searched using a time position as a key. (Step 102). Then, the target video is reproduced (step 103). During the video playback, the playback time position of the video is obtained (step 104), the click log of the user at the playback time position is obtained (step 105), and the user clicks on the space coordinates in the video corresponding to the user click coordinates. The indicated mark is drawn so as to be superimposed on the reproduced video screen (step 106).
[0021]
The above steps 104 to 106 are repeated while the video is being reproduced (step 107), and a mark indicating the user's click is displayed superimposed on the video screen to be reproduced. Of course, when a subject in the video is clicked, information related to the subject is displayed as in the related art. Further, it is also possible to arbitrarily change the reproduction method (reproduction speed, reproduction image size, etc.) of the reproduction video according to a user's instruction.
[0022]
FIG. 2 shows a functional block diagram of one embodiment of the user operation log visualization device of the present invention. The user operation log visualization device includes a log data storage unit 201, a filtering unit 202, a data shaping unit 203, a reproduction time position log data acquisition unit 204, a log data drawing unit 205, a video reproduction time position acquisition unit 206, a video reproduction unit 207, It comprises a user interface 208 and a video storage means 209.
[0023]
The log data storage unit 201 is a database for storing user operation logs, and stores user operation log data as shown in FIG. Of course, the information shown in FIG. 7 is merely an example, and if there is any other necessary information, it is acquired and stored. In this embodiment, it is assumed that log data relating to one video is stored in one database, and the correspondence with the video is also stored in the log data storage unit 201. Of course, it is also possible to store log data relating to a plurality of videos in one database, but in that case, an identifier for identifying the videos is required.
[0024]
The filtering unit 202 filters the user operation log data stored in the log data storage unit 201, selects only target log data, and transfers the selected log data to the data shaping unit 203. For example, when a user ID is given to log data as shown in FIG. 7, it is possible to acquire click operation log data for each user by filtering by user ID. Of course, all the log data may be passed to the data shaping unit 203 without any filtering.
[0025]
The data shaping unit 203 arranges the three-dimensional t, x, and y axes so that the log data selected and extracted by the filtering unit 202 can be searched using time (t) as a key. FIG. 8 shows an example in which the data shaping unit 203 shapes the log data shown in FIG. In the example of FIG. 8, at the time position 1000 ms of the image, it is indicated that the spatial coordinates (50, 50) in the image are clicked by the user “tim” and (150, 240) are clicked by the user “joe”. . As described above, the log data is shaped so that the click coordinates (x, y) at a certain time position t can be easily obtained. Video storage means 209 stores video (content). Of course, the video storage means 209 may be on a network.
[0026]
The user interface 208 designates log data and a video to be clicked by the user based on a key input or a mouse operation of the user, and controls the video reproducing unit 207. The video reproducing unit 207 acquires and reproduces a video targeted for a user click from the video storage unit 209.
[0027]
The video reproduction time position obtaining means 206 obtains the reproduction time position of the video being reproduced by the video reproduction means 207 and passes it to the reproduction time position log data obtaining means 204. The reproduction time position log data obtaining means 204 searches the log data shaped by the data shaping means 203 for log data (click log data) at the time position obtained by the video reproduction time position obtaining means 206, Get coordinates. For example, in the example of FIG. 8, if the time position obtained by the video reproduction time position obtaining means 206 is 1000 ms, the space coordinates (50, 50) and (150, 240) in the video are obtained as the user click coordinates. .
[0028]
The log data drawing means 205 draws a mark indicating a click log at a video screen position corresponding to the spatial coordinates of the click log data acquired by the reproduction time position log data acquisition means 204 for a predetermined time. For example, when the reproduction time position is 1000 ms, a predetermined mark is drawn at the positions of the spatial coordinates (50, 50) and (150, 240) in the video.
[0029]
FIG. 3 shows a screen example in this case. In FIG. 3, a video 301 whose reproduction time position is 1000 ms is displayed on a screen 300, and a fish (fish-A) 302 appears as a subject. A mark 303 indicating the click log of the user “tim” is located at the position (upper left) of the spatial coordinates (50, 50) in the video 301, and the user “joe” is located at the position (lower center) of the spatial coordinates (150, 240). The mark 304 indicating the click log of "." The mark indicating the user click log may be different for each user, for example. In FIG. 3, a slider bar 305 indicating a playback position and playback, pause, and stop buttons 306, 307, and 308 are shown below the video 304 as an example of a user interface for controlling video playback. ing.
[0030]
Thus, the log analyst can easily recognize, for example, what kind of video scene the log 702 in FIG. 7 was clicked on and what part of the anchor area “fish-A” was clicked. Is possible. In addition, it is possible to easily recognize what the click log 702 at the place where the anchor area is not defined is clicked.
[0031]
Also, by repeating this operation, it is possible to easily recognize what kind of scene the scene from 1000 ms to 1100 ms where the user's clicks are concentrated and what kind of subject is often clicked. Also, it is possible to easily recognize what kind of scene was from 1100 ms to 1530 ms without a click.
[0032]
In addition, by making the user interface 208 have a function of designating an arbitrary time position of the reproduced video and changing the reproduction speed and the size of the reproduced video, the recognition can be further facilitated.
[0033]
FIG. 4 shows a hardware configuration diagram to which the user operation log visualization method and apparatus shown in FIGS. 1 and 2 are applied. In FIG. 4, an information processing device (CPU) 401 is a device that executes the processes of steps 101 to 106 in FIG. 1 or a device that realizes the processing functions of the units 202 to 207 in FIG. A display 402, a keyboard 403, and a mouse 404 are devices constituting a user interface. The hard disk 405 stores video and user operation log data. The video may be stored on another recording medium such as a CD-ROM, or may be stored on a network 406 such as the Internet. Good. Also, user operation log information may be accumulated on the network 406 and downloaded from now.
[0034]
The user operation log visualization method and apparatus can be applied to various interactive video content providing systems utilizing user operations on a subject, such as a system for providing related information by an instruction for a subject and a game system in which a score is added by an instruction for a subject. Can be applied to
[0035]
Note that a part or all of the processing functions of each unit in the apparatus shown in FIG. 2 are configured by a computer program, and the program can be executed using a computer to realize the present invention. Needless to say, the processing procedure shown can be configured by a computer program and the computer can execute the program. In addition, a program for realizing the processing function by the computer or a program for causing the computer to execute the processing procedure is stored in a computer-readable recording medium such as an FD, an MO, a ROM, a memory card, and a CD. , A DVD, a removable disk, etc., and can be stored or provided, and the program can be distributed through a network such as the Internet.
[0036]
【The invention's effect】
According to the present invention, a user's click operation log is visually recognized by displaying a user's click operation log superimposed on a video for an interactive video content providing service using a user operation for a user's video click. It is possible to easily identify what scene the user clicked and what the user clicked. In addition, the distribution of the plurality of click operation logs can be actually seen in association with the video, and the tendency of the click operation can be easily grasped.
[Brief description of the drawings]
FIG. 1 is a flowchart of an embodiment of a method for visualizing a user operation log according to the present invention.
FIG. 2 is a functional block diagram of an embodiment of a user operation log visualization device according to the present invention.
FIG. 3 is a diagram showing a screen display example according to the present invention.
FIG. 4 is a diagram showing an example of a hardware configuration to which the present invention is applied.
FIG. 5 is an example of an application that presents related information by clicking a subject in a video of a user.
FIG. 6 is an example of anchor definition of a temporal and spatial partial region including a subject in a video.
FIG. 7 is an example of operation log information of a user.
FIG. 8 is an example of shaping user operation log information.
[Explanation of symbols]
201 log data storage means 202 filtering means 203 data shaping means 204 reproduction time position log data acquisition means 205 log data drawing means 206 video reproduction time position acquisition means 207 video reproduction means 208 user interface 209 video storage means 300 screen 301 video 302 subject 303 , 304 user click log mark

Claims (10)

A user operation log visualization method for visualizing log information of a user operation on a video,
Reproducing a video; obtaining a reproduction time position of the reproduced video; obtaining log data relating to the reproduction time position from accumulated user operation log data; and displaying the log data on a video screen. And a step of visualizing and displaying the user operation log. 2. The method according to claim 1, further comprising the step of selecting only necessary log data from the accumulated user operation log data. 3. The user operation log visualization method according to claim 1, further comprising a step of shaping the accumulated user operation log data so as to be easily handled. 4. The user operation log visualization method according to claim 1, further comprising a step of arbitrarily changing a reproduction method of a reproduction video. A user operation log visualization device that visualizes log information of a user operation on a video,
Means for accumulating user operation log data, means for reproducing video, means for obtaining a reproduction time position of the reproduced video, and obtaining log data relating to the reproduction time position from the stored user operation log data And a means for visualizing and displaying log data on a video screen. 6. The user operation log visualization device according to claim 5, further comprising means for selecting only necessary log data from the stored user operation log data. 7. The user operation log visualization device according to claim 5, further comprising means for shaping the accumulated user operation log data so as to be easy to handle. 8. The user operation log visualization device according to claim 5, further comprising means for arbitrarily changing a reproduction method of a reproduction video. A program for causing a computer to execute each step of the user operation log visualization method according to claim 1, 2, 3, or 4, or each function of the user operation log visualization device according to claim 5, 6, 7, or 8. A program for causing a computer to execute each step of the user operation log visualization method according to claim 1, 2, 3, or 4 or each function of the user operation log visualization device according to claim 5, 6, 7, or 8 is recorded. recoding media.

Images