JP2003283981A - Method and system for inputting/displaying comment about video, client apparatus, program for inputting/ displaying comment about video, and storage medium thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system for inputting and displaying, which is used for making communication with respect to video via a network. <P>SOLUTION: This system has a video reproduction control means (1), input means (2) for inputting time information and space information for displaying a comment, comment inputting means (3), means (4, 5) for storing comment information including the time information and the space information, time information acquiring means (6) for video which is being reproduced, means (7) for acquiring the comment information corresponding to the time information of the video, means (8) for controlling the comment to be displayed, and means (9) for preparing a comment displaying region in response to the space information and displaying an image for instructing a position displayed in the space information from the comment displaying region. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention effectively displays the relationship between comments input to a video and makes clear the target of the comment when communicating via a video on a computer. The present invention relates to a method and system for inputting / displaying a video comment, which enables easier communication by instructing to.

[0002]

2. Description of the Related Art Conventionally, when a user wants to communicate via a video on a network, a user combines the means for viewing the video and the means for communicating to communicate.

For example, BROBA (http://www.broba.cc
In general video distribution sites such as /), video is viewed using video playback software, and communication related to video is performed using the bulletin board function and chat function. The bulletin board function is an information exchange system operated on a specific server, and is characterized by being able to write information at any time. The chat function is software using the communication function of the network, and is characterized by allowing multiple people to exchange information in real time.

There is also a means for communicating via video using dedicated software. Vi
deoClick (http://www.vsoft.com/products/)
Can place a bookmark anywhere on the video and write a comment there. Since the link information and the comment information can be disclosed to a plurality of people by using the communication function of the software, it is possible to communicate with the plurality of people through the comment information while watching the video (see FIG. 50). ).

[0005]

When the video viewing means and the video communication means are separated as in the above video distribution site, there arises a problem that it is not known which part of the video the comment for the video is talking about. . For example, if there is a comment saying "The scenery is beautiful," you don't know which scenery in the video. Also, in order to indicate where it is, it must be expressed as "The scenery of the escape scene in the middle of the story is beautiful," and even in that case, the scene cannot be clearly indicated. Furthermore, when browsing a certain scene, it is difficult to find a comment related to the scene, and thus it is difficult to efficiently communicate.

Further, in the bulletin board and the above-mentioned dedicated software, since the parts for inputting and displaying the comments are gathered together, various comments are uniformly presented to the user, and a comment in which the user is interested is searched for. Difficult to do.
In addition, it is difficult to find a comment that has been written with respect to the comment, which causes a problem that communication cannot be performed smoothly. FIG. 51 shows problems in the video communication method using such conventional dedicated software.

Therefore, according to the present invention, in the above-mentioned conventional means for communicating about a picture,
Indicate where the user wants to comment, or
You can easily browse the comments related to the place you care about, and you can easily grasp the relationship between the comments by effectively displaying the comments according to their contents. Video comment input
An object of the present invention is to provide a display system, a client device, a video comment input / display program, and a storage medium for the program.

[0008]

In order to achieve the above object, the present invention allows a user to specify a place to comment and input a comment when performing communication regarding video, and when displaying a comment. Proposes a video comment input / display method in which a comment is displayed for each designated location so as to point to the designated location when inputting.

FIG. 1 is a diagram for explaining the principle of the video comment input / display method of the present invention.

The invention according to claim 1 is a video comment input / display method for communicating video on a network, and a playback control procedure (step) in which a user performs playback control such as video playback and pause. 1)
And the procedure to specify the time information and space information of the place where you want to enter a comment in the playback-controlled video (step 2)
And the procedure for inputting a comment (step 3), the comment input by the user by the procedure for inputting the comment, and the position information designated by the procedure for designating time information and space information of the place where the comment is desired to be input. Procedure for accumulating (step 4), procedure for acquiring the time of the video whose playback is controlled (step 5), and procedure for acquiring the comment related to the acquired time from the accumulated comments (step 6) , The comment to be displayed, the place to display the comment, and the procedure for controlling the comment display by specifying the amount of the comment (step 7), and the comment display area is prepared according to the position information in which the comment is input. And a step (step 8) of displaying a comment display area on the screen so as to point to a place designated as a place to input.

With this, it is possible to point and comment on a place to comment, that is, the time information and the space information (points on the spatiotemporal axis), and it becomes clear which part of the image to comment. You can
Further, since the comment is displayed so as to indicate the place where the comment is written, it is possible to grasp at a glance what kind of place and how many comments are written when browsing the comment. Thereby, communication can be achieved more smoothly.

As a first aspect of the present invention, there is proposed a method of directly specifying a comment and adding the comment when a comment that has already been input is desired to be further commented.

Therefore, in the invention according to claim 2, in the procedure of designating the time information and the space information of the place where the comment is desired to be input in the reproduction-controlled video, the comment is written when the comment is written. Directly specify the displayed comment display area and perform additional writing.

According to this method, when it is desired to write a comment, it is possible not only to specify the location on the video but also to directly specify and write the comment written therein.

As a second aspect of the present invention, when accumulating a comment and position information, a method of accumulating position information given a predetermined allowable width is proposed.

Therefore, in the invention according to claim 3, the procedure for accumulating the comment entered by the user and the location information designated by the procedure for designating the time information and the space information of the place where the comment is desired to be entered is designated. A predetermined tolerance is given to the time information and / or the spatial information of the obtained position information, and the place obtained by this is stored together with the comment.

According to this method, when designating a place, it is not necessary to accurately designate a certain point. Also, when specifying the location of the comment to be displayed, it is not necessary to specify an exact point. Further, when displaying a comment, the comment can be displayed with a certain width on the spatiotemporal axis, instead of displaying the comment at a certain point.

As a third aspect of the present invention, there is proposed a method of dynamically switching the amount of comments to be displayed when all the comments cannot be displayed simultaneously when displaying the comments.

Therefore, in the invention according to claim 4, the procedure for controlling the comment display by designating the comment to be displayed, the place where the comment is to be displayed, and the comment amount is the total area of the comment displayable area, The amount of comments to be displayed at once and the display method thereof are dynamically changed from the amount of comments to be displayed and the minimum area of the comment display area.

According to this method, when all the comments cannot be displayed in the comment display area, the comment can be displayed easily by controlling the display amount at one time.

As a fourth aspect of the present invention, when displaying a comment, a method is proposed in which the area for displaying the comment is effectively utilized to make the comment easy to see.

Therefore, in the invention according to claim 5, the comment display area is prepared according to the position information in which the comment is input, and the procedure for displaying the comment on the screen so as to point to the place is as follows. The layout and display area of the comment display area are dynamically changed and displayed based on the total area, the amount of comments to be displayed simultaneously, and the minimum area of the comment display area.

According to this method, all the comments can be displayed when the comment amount to be displayed is small, and when the comment amount is large, the comments can be re-laid out and displayed so that the entire comment displayable region can be effectively utilized. Therefore, it is possible to present the comment easily according to the comment amount.

As a fifth aspect of the present invention, a method of effectively utilizing the comment display area by displaying the comments written in the same place together is proposed.

Therefore, in the invention according to claim 6, the procedure of displaying the comment so as to point to the position in accordance with the position information in which the comment is input is as follows. To display.

By this method, the comments commenting at the same place can be displayed in one area, so that the mutual relationship can be easily understood, and the display area can be effectively used to display the comments. it can.

According to a sixth aspect of the present invention, when all the comments cannot be displayed at the same time, the comments indicating the close places are combined into one comment.
We propose a method to effectively use the comment display area.

Therefore, in the invention according to claim 7, the procedure of displaying the comment so as to point to the place in accordance with the position information in which the comment is input is as follows. indicate.

According to this method, the number of comment display areas can be limited without changing the total amount of comments, so that a large number of comment display areas are not displayed on the screen, which hinders video viewing. You can browse the comments without.

FIG. 2 is a schematic block diagram of a video comment input / display device for implementing the video comment input / display method of the present invention. The video comment input / display device shown in the figure includes a video reproduction control means 11 for a user to perform reproduction control such as reproduction and pause of a video, and a reproduction-controlled video.
By the comment input location designating means 12 for designating time information and space information of a location where a comment is desired to be input, the comment inputting means 13 for entering a comment, the comment information DB 24 for holding the comment and position information, and the comment inputting means 13 described above. The comment information DB2 is the comment input by the user and the position information designated by the comment input location designating means 12 for designating the time information and the space information of the location where the comment is desired to be input.
4, the comment information accumulating means 23 for accumulating, and the video section acquiring means 15 for acquiring the time of the video whose reproduction is controlled.
And a comment acquisition unit 25 for acquiring comment information related to the time acquired by the video section acquisition unit 15 from the comment information accumulated in the comment information DB 24, a comment to be displayed, position information of the comment, And a comment display control means 16 for controlling the comment display by designating the amount of the comment, and a comment display area is prepared according to the position information of the comment, and the comment display area is displayed on the screen so as to indicate the place where the comment is desired to be displayed. And a comment display unit 17 for displaying.

The comment input location designating means 12 designates the time information of the video and the spatial information on the screen where the user wants to input a comment.

When writing to a comment, the comment input location designating means 12 can directly designate the display area in which the comment is displayed and additionally write.

The comment information storage means 23 stores the comments input by the user in the comment input means 13,
When inputting a comment, the comment input location designating means 12
The position information designated by is stored in the comment information DB 24.

The comment information accumulating means 23 can add a predetermined width to the time information and / or the space information of the designated position information and accumulate the comment information in the comment information DB 24.

Further, the comment display control means 16 specifies a comment that the user wants to display from the comments acquired by the comment acquisition means 25, a place where the user wants to see the comment, or a display. You can specify the amount of comments to make.

The comment display control means 16 dynamically changes the amount of comments displayed at one time from the total area of the comment displayable area, the amount of comments to be displayed simultaneously, and the minimum area of the comment display area. It is possible to

The comment display means 17 receives the comment information acquired by the comment acquisition means 25 and the control by the comment display control means 16, prepares a comment display area according to the position information in which the comment is input, and displays the comment. A comment display area is displayed on the screen so as to point to a place.

The comment display means 17 dynamically changes the layout and display area of the comment display area based on the total area of the comment displayable area, the amount of comments to be displayed simultaneously, and the minimum area of the comment display area. It is possible to display all of the comments pointing to the same place in one comment display area, and to display the comments pointing to a close location all in one comment display area.

[0039]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Next, embodiments of the present invention will be described with reference to the drawings.

In the description of the embodiments of the present invention, the video content is not limited to the content including the video information and the sound information, and any content including at least the image information is called the video content. The comment is not limited to text information, but includes information used for communication such as a URL pointing to content on the web and an image expressing emotion.

<First Embodiment> The video comment input / display system according to the first embodiment of the present invention displays a comment written by another user while watching a video stored on a server, or It is a system to write new impressions.

FIG. 3 is a system configuration diagram showing a first embodiment of the present invention.

The video comment input / display system includes a plurality of client devices 1 and a server device 2, and the client device 1 and the server device 2 are connected to each other via a network (not shown) such as the Internet. Connected through. Although there may be a plurality of client apparatuses, one client apparatus 1 will be described below for simplification of description.

Briefly explaining, by applying the first embodiment of the present invention to the conventional video comment input / display system as shown in FIG. 50, the comment is displayed in the comment input / display field in FIG. Make it clear where the comment refers to in the video so that you can easily find the comment written for a certain place. Also, it is possible to input a comment by pointing to an arbitrary place in the video.

The client device 1 has a video reproduction control unit 11 which receives the video contents from the video DB 21 on the server device 2 and controls the reproduction. Video playback controller 1
Reference numeral 1 is an interface for viewing a video and designating a certain place on the time axis, and reproduction control such as normal reproduction, double speed, slow motion, frame advance and rewind is possible.

The client device 1 can specify a position or a section on the time axis of the video as a place to write a comment in the video whose reproduction is controlled, and further, in the displayed screen space. It has a comment input location designating section 12 capable of designating a certain position or area.

In order to specify a certain position on the time axis, not only a numerical value is directly input, but, for example, as shown in FIG. 52, it is intuitive to move a bar on the time axis expressed in a band shape. It is possible to use an existing interface called a slider that specifies a position on the time axis. Further, by using the start button and the end button as shown in the figure together, the start point and the end point of the section can be input, and the section on the time axis can be designated.

In order to specify a certain position on the space, there is a method of directly specifying a certain position on the screen using a mouse cursor, in addition to inputting the coordinates on the screen numerically. When designating an area, the area can be easily designated by a drag operation using a mouse cursor.

In the present embodiment, a certain position on the time axis can be designated by using the video reproduction control section 11.
The video reproduction control unit 11 and the comment input place designating unit 12 are realized as an integrated interface, a time for inputting a comment can be designated, and a position on the screen can be designated by using a mouse cursor.

The client device 1 has means for inputting a comment to a certain position on the image designated by the comment input location designating unit 12, and the comment and the input position information in the spatiotemporal space are commented. As information
It has a comment input unit 13 to be sent to the comment information storage unit 23 on the server device 2. Specifically, the comment information of this embodiment includes a comment, a designated time, and coordinates on the screen. FIG. 4 is an explanatory diagram of the comment information in the first embodiment of the present invention.

The image display function control interface 14 is
This is an interface for selecting whether to display a comment or not. In the present embodiment, when the user presses the button, the video time acquisition unit 15 is controlled so as to acquire the video time when the button is pressed.

The client device 1 has a video time acquisition unit 15 that acquires the time of the currently displayed video and sends it to the comment acquisition unit 25 of the server device 2.

The client device 1 receives a plurality of comment information acquired by the comment acquisition unit 25 of the server device 2 and controls the comment display control unit 16 for controlling the number of comments displayed on the screen and the comments to be displayed. Have.

FIG. 5 shows an example of screen display on the client device in the first embodiment of the present invention. In the present embodiment, as shown in FIG. 5, by enumerating the acquired comments, enlarging / reducing the display portion, and selecting the comment to be displayed using the slider, the number of comments displayed on the screen and the display thereof are displayed. To control the comments. As a control method according to another embodiment other than this embodiment, a method of designating the maximum number of displayed comments as a numerical value, a method of selecting only the latest n comments, and the like can be considered. Further, in the first embodiment of the present invention, by designating a certain position on the video screen, it is possible to select a comment related to the position and control the comment to be displayed on the screen.

The client device 1 displays the comment controlled by the comment display control section 16 by pointing to a certain position in the space designated at the time of input by using the position information corresponding to the comment. It has a display unit 17.

As a method of displaying a comment so as to point to a certain place in the space, for example, one method for associating the comment with the place shown in FIG. 53, the object corresponding to each comment is designated. By placing the object on the screen, the comment and the place can be associated with each other. In the first embodiment of the present invention, an arrow is used to indicate the location on the screen corresponding to the comment.

FIG. 6 is an explanatory diagram of a display example generated on the screen on the client device according to the first embodiment of the present invention.

The server device 2 has a video DB 21 holding video content, a video section information DB 22 holding video section information, and comment information D holding comment information.
Has B24.

The video section stored in the video section information DB 22 is, for example, a scene section in which the same scenery is continuously shot, a section in which a certain person appears, a telop display section in which characters are displayed, and music flows. Represents a physical or semantic section on the time axis, such as a section. For the semantic section, the method of manually specifying the section is mainly used. Regarding the physical section, it is possible to obtain a scene switching point or a character display section by using video analysis technology. Regarding video analysis technology, Japanese Patent No. 2839
132, JP-A-9-238298, and
It is described in JP-A No. 11-178007 and the like. In this embodiment, in order to simplify the description, a scene section using a scene switching point is used as a video section.
FIG. 7 is an explanatory diagram of a video section using scene switching points in the first embodiment of the present invention.

The video section is stored in the video section information DB 22 as video section information as shown in FIG.

The server device 2 stores the comment information transmitted from the comment input unit 13 in the client device 1 in the comment information DB 24, the comment information storage unit 23.
Have.

When the comment information storage unit 23 stores the comment information in the comment information DB 24, the comment information storage unit 23 can give a predetermined width to the time when the comment is input and the specified position on the screen.

As a method of giving a predetermined width to time, for example, a method of giving a width of m seconds before and after (m is determined by the system) to a specified time, or a specified spatial information When a certain object is shown, there is a method of setting the period during which the object is displayed on the screen as the time width. In the first embodiment of the present invention, the video section information DB 2
Using the video section information stored in 2, the video section including the designated time is stored as the time when the comment was input.

As a method of giving a predetermined width to the space, for example, n pixels are arranged vertically and horizontally with respect to a specified point or area on the image (pixel is a pixel unit of the screen. N is determined by the system). There is a method of recognizing an object having the specified spatial information and setting the shape of the object as the specified area. In the first embodiment of the present invention, the screen is displayed horizontally.
Includes specified spatial information by dividing into i and vertical j
The area of the k-th column and the 1-th row is accumulated as the specified area. Figure 9
FIG. 7 is an explanatory diagram of a method of expanding designated coordinates to a region on a screen according to the first embodiment of the present invention.

As described above, the comment information is accumulated in the comment information DB 24 with a predetermined width for the time information and the space information. FIG. 10 is an explanatory diagram of the comment information accumulated in the comment information DB 24 according to the first embodiment of this invention.

The server device 2 uses the time information input from the video time acquisition unit 15 to take out the comment information related to the time information from the comment information DB 24 and sends it to the comment display control unit 16, the comment acquisition unit 25. Have. When there are a plurality of comment information related to time information, a plurality of comment information is transmitted. In the first embodiment of the present invention, when the time information of each entry in the comment information DB shown in FIG. 10 is a section ts ≦ t <te including the input time information t, it corresponds to that time section. Acquire and send comment information.

Next, the overall operation of the video comment input / display system according to the first embodiment of the present invention will be briefly described. FIG. 11 is an overall flowchart of the video comment input / display system in the first embodiment of the present invention.

Step 11: Video playback control such as video playback, rewinding, and pause is performed in response to a request.

Step 12: In the comment display function control interface 14 on the client device 1, it is judged whether or not the user has instructed to display the comment. Specifically, for example, it is determined whether the comment display button as shown in FIG. 6 has been pressed.

Step 13: It is judged whether or not the user has designated the comment input location using the comment input location designation section 12 on the client device 1. In particular,
For example, in FIG. 6, when a certain point on the video display area is designated by using the mouse cursor, it is determined that the coordinate on the screen space is designated as the space information. Also, when the mouse is clicked, the time of the video under playback control is acquired, and that time is acquired as time information on the video time axis.

Step 14: Determine whether the user has entered a comment. Specifically, when executing step 14, the comment input unit 1 as shown in FIG.
3 is displayed on the screen. User enters comment
When the K button is pressed, it is determined that a comment has been input. When a comment is input, the comment input unit 13 stores the coordinates in the screen space acquired in step 3, that is, the space information, the time information on the video time axis, and the input comment in the comment information storage unit 23. Send to.

Step 15: The space information, the time information, and the comment acquired in step 4 are accumulated in the comment information DB 24, and the process returns to step 11. The expansion processing of the spatial information information and the time information information performed when accumulating will be described later.

Step 16: Since the comment display button has been pressed during video reproduction, processing for acquiring the time of the video whose reproduction is controlled at that time is performed.

Step 17: Using the video time acquired in step 6, the comment information associated with the time information corresponding to the acquired video time is acquired from the comment information DB 24.

Step 18: It is judged whether or not the user enlarges or reduces the area of the comment display portion and selects the comment to be displayed.

Step 19: When the comment display part is changed in step 18, the comment display control unit 16 performs the comment display control process associated with the change.

Step 20: Prepare a comment display area for the comment selected according to the comment display control unit 16, display it on the screen so as to indicate the place, and return to step 11.

Next, the comment information accumulating process in step 15 of the overall operation flow will be described in detail.

FIG. 10 shows the first embodiment of the present invention.
It is explanatory drawing of the extended comment information accumulated on comment information DB24. In the comment information accumulation process in step 15, the comment information acquired in step 14 is stored in the comment information DB 24 as extended comment information, as shown in FIG. More specifically, the position information composed of time and space coordinates is accumulated as extended position information composed of a temporal video section and a spatial region.

FIG. 13 shows the first embodiment of the present invention.
6 is a detailed flowchart showing a method of accumulating comment information executed by a comment information accumulating unit 23.

Step 51: Acquire the vertical and horizontal sizes (w, h) of the displayed image.

Step 52: Acquire the number of vertical and horizontal divisions (i, j) of the video display area. In one embodiment, the value of (i, j) may be fixed by the system, for example.

Step 53: Acquire the position (k, l) on the divided area of the coordinate value (x, y) specified when the comment is input.
As shown in FIG. 9, the size (w, h) of the image acquired in step 51, the number of divisions (i, j) acquired in step 52, and
The position (k, l) on the divided area from the coordinate value when entering a comment
Is required. Specifically, for m satisfying 0 ≦ m <i, m satisfying (w × m ÷ i) ≦ x <(w × (m + 1) ÷ i) is k. Similarly, for n satisfying 0 ≦ n <j, n satisfying (h × n ÷ j) ≦ y <(h × (n + 1) ÷ j) is l.

Step 54: The video section including the comment input time t is acquired from the video section information DB 22. Since each video section of the present embodiment uses sections that do not overlap each other as video section information as shown in FIG. 8,
The n entries in the video section information DB 22 are sequentially scanned, and the start time Ts_i and the end time Te_ of the i-th entry satisfying Ts_i ≦ t <Te_i (0 ≦ i ≦ n) are satisfied.
i is used as extended time information information.

In the first embodiment of the present invention, the video section information is not limited to the format shown in FIG. 8. For example, as shown in FIG. They may be defined as overlapping sections. In that case, as shown in FIG. 14, the sum of all m video sections including the time t is calculated, and the minimum start time Ts_
The video section can be specified by using sum and the maximum end time Te_sum as the start time and the end time of the video section.

Finally, the area coordinates on the divided area acquired in step 53 and the video section acquired in step 54 are stored in the comment information DB 24 together with the corresponding comment as expanded comment information. Figure 15
FIG. 6 is a detailed explanatory diagram of extended comment information accumulated in the comment information DB according to the first embodiment of the present invention.

Next, the comment acquisition process in step 17 of the overall operation flow of FIG. 11 will be described in detail.

In step 15 and step 16, when the user presses the comment display button, the time t of the video whose reproduction is controlled is acquired. In step 7, a plurality of comment information related to this time t is acquired from the comment information DB 24.

FIG. 16 shows the first embodiment of the present invention.
7 is a detailed flowchart of comment acquisition processing executed by the comment acquisition unit 25 of the server device 2.

Step 71: In the comment information DB 24
To obtain the comment information related to the video time t when the comment information button is pressed from the n pieces of comment information, a temporary array show [n] is created.

Step 72: Initialize the index i for scanning comment information.

Step 73: The i-th comment information is acquired from the comment information DB 24 as shown in FIG. 15, and its start time Ts_i and end time Te_i are acquired.

Step 74: Start time Ts_i to end time
Determine whether the range up to Te_i includes time t.

Step 75: If time t is not included, sho
Set w [i] to false.

Step 76: If the time t is included, show [i]
To true.

Step 77: It is judged whether or not n pieces of comment information have been scanned.

Step 78: If n comment information has not been scanned, the index i is incremented so as to check the next comment information.

Step 79: Since n comment information items have been scanned, all the comment information items (comment i) corresponding to i for which show [i] is true are acquired, and the process ends. FIG. 17 is an explanatory diagram of the comment information acquired by the comment acquisition unit 25 in the first embodiment of the present invention in this way. The m pieces of comment information finally acquired include time t in the video section, as shown in FIG.

Next, the comment display control processing of step 19 in the overall operation flow of FIG. 11 will be described in detail.

The comment display control unit 16 proceeds to step 18
When the comment display area is changed or the comment to be displayed is selected, the comment display is controlled according to the change, and the comment to be displayed is selected in step 19.

FIG. 18 shows the first embodiment of the present invention.
7 is a detailed flowchart of comment display control processing executed by the comment display control unit 16 of the server device 2.

Step 91: All the comment information acquired by the comment acquisition unit in step 17 is set to the selected state.

Step 92: It is judged whether or not the user has designated a certain coordinate within the image display area by clicking with a mouse cursor or the like.

Step 93: When the coordinates are designated, display comment selection processing is performed to select the comment to be displayed.

Step 94: It is judged whether the comment display area has been enlarged or reduced by the user's operation or the like.

Step 95: When the comment display area is enlarged or reduced, the display area is enlarged or reduced, and the comment to be displayed is selected according to the enlarged or reduced display area.

Step 96: It is determined whether or not the comment in the comment display area has been scrolled by the user's operation.

Step 97: When the comment is scrolled, the display area is scrolled to select the comment to be displayed.

FIG. 19 is a detailed flowchart of the comment selection process of step 93 executed by the comment display control unit 16 of the client device 1 in the first embodiment of the present invention.

FIG. 20 is an explanatory diagram of the display comment selection processing in step 93. In the figure, a case where the coordinates (x, y) in the video display area are selected using the mouse cursor will be described as an example.

Step 9301: Obtain vertical and horizontal sizes (w, h) of the displayed image.

Step 9302: Acquire the number of vertical and horizontal divisions (i, j) of the video display area. In this embodiment, the value of (i, j) can be set to a predetermined value by the system.

Step 9303: Specified coordinate value (x,
The position (k, l) of y) on the divided area is acquired. Step 9
Image size (w, h) obtained in step 301, step 930
From the number of divisions (i, j) and coordinate values (x, y) acquired in 2 (k, l)
Is required. Specifically, for m satisfying 0 ≦ m <i, m satisfying (w × m ÷ i) ≦ x <(w × (m + 1) ÷ i) is k. Similarly, for n satisfying 0 ≦ n <j, n satisfying (h × n ÷ j) ≦ y <(h × (n + 1) ÷ j) is l. As a result, the area of the k-th column and the l-th row as shown in (1) of FIG.

Step 9304: The specified coordinates for the m pieces of comment information acquired by the comment acquisition unit 25
Scan for comments associated with (x, y). Create a temporary array select [m] to determine relevance.

Step 9305: Initialize the index i for scanning comment information.

Step 9306: The i-th comment information is acquired, and its spatial information information (ki column, li row) is acquired.

Step 9307: It is determined whether or not the area (k, l) including the designated coordinates matches the spatial information information (ki column, li row) acquired in step 9306.

Step 9308: If no match, sele
Set ct [i] to false.

Step 9309: If match, select
Set [i] to true.

Step 9310: It is judged whether or not m pieces of comment information have been scanned.

Step 9311: If m comment information items have not been scanned, the index i is incremented so as to check the next comment information item.

Step 9312: Since m comment information items have been scanned, all comment information items for which select [i] is true are acquired, and the process is terminated. As a result, FIG.
Comment information is acquired as indicated by 0 (2).

FIG. 21 is an explanatory diagram of the display area enlargement / reduction processing of step 95 which is executed by the comment display control unit 16 of the client device 1 in the first embodiment of the present invention. In the figure, a case where the display area is enlarged will be described as an example.

When the size of the display area is changed, first,
The minimum display area required for displaying each comment is calculated ((1) in FIG. 21). Specifically, the number of characters that can be displayed in one line is obtained from the width of the display area and the font size of each comment, and the number of lines required for display is calculated from the total number of characters in the comment.

Next, rearrange the comments in the display area,
A comment that can be displayed is selected ((2) in FIG. 21). Figure 21
In the example, since the display area is enlarged, the comments 5 and 6 are newly selected for display. Finally, the comments 1 to 6 are selected as the comments to be displayed.

FIG. 22 is an explanatory diagram of the display area scroll processing of step 97 which is executed by the comment display control section 16 of the client device 1 in the first embodiment of the present invention. In the figure, the case where the bar is moved downward and the display area is scrolled downward will be described as an example.

By scrolling the display area, a comment at the scroll destination is newly selected. In the example of FIG. 22, by scrolling the region by two comments, the comments 1 and 2 are deselected, and the comments 7 and 8 are newly selected for display. Finally, comment 3
Is selected as the comment to be displayed.

Further, the comment display processing in step 20 of the overall operation flow shown in FIG. 11 will be described in detail.

At step 20, the comment display control unit 1
The comment information selected by 6 is displayed in the comment display area so as to point to the place specified when the comment is input.

FIG. 23 shows the first embodiment of the present invention.
6 is an explanatory diagram of comment display processing executed by a comment display unit 17 of the client device 1. FIG.

In this embodiment, as shown in FIG. 23, the arrow is extended from each comment selected by the comment display control section 15. Each arrow points to the center of gravity of the pointing area by using the position information expanded by the comment information storage unit 23.

Thus, according to the first embodiment of the present invention, by using the video comment input / display method according to claim 1 or the video comment input / display system according to claim 8, a place where a comment is desired. You can point to and comment. Further, when a comment is browsed, by designating a place on the video display area, the comment can be displayed so that the user can grasp at a glance what place and how much comment is written. Figure 24
FIG. 7 is an explanatory diagram of a case where a comment is displayed by designating a location of a video area in the first embodiment of the present invention.

A method according to claim 3 or claim 10
By accumulating the comment information using the described system, it is not necessary to specify an exact point when pointing to the place where the comment is input or the place where the comment is desired to be displayed.

A method according to claim 4 or claim 11
By displaying comments using the described system, the amount of comments displayed at one time can be controlled, and the comments can be displayed in an easy-to-read manner.

<Embodiment 2> The video comment input / display system according to the second embodiment of the present invention uses the P2P (peer-to-peer) method to simultaneously watch video between clients while allowing the user to watch the video. It is a communication system that displays comments written by or writes new impressions. FIG. 25 is an explanatory diagram of such a communication system in the second embodiment of the present invention. In this embodiment, the client A that distributes video and stores and distributes comments from each user is called a server device, and the clients 1 to n that read and write comments while watching a video are called client devices.

FIG. 26 is a system configuration diagram showing a second embodiment of the present invention.

Video comment input according to the second embodiment of the present invention
The display system includes a plurality of client devices 21 and a server device 22, and the client device 21 and the server device 22 are connected via a network (not shown) such as the Internet, for example. Although there may be a plurality of client devices, one client device 21 will be described below for simplification of description.

[0138] Generally speaking, while the video is being viewed, the comment information accumulated on the server device 22 is sequentially displayed on the client device 21, and the comment input on the client device 21 is displayed on the server device 22. It is a system for sequentially accumulating and displaying on the client device 21. When entering a comment, it is possible to specify the location on the video and then enter it. Also, when displaying comments, the position and size of the display area are dynamically changed while considering the total amount of comments to be displayed and the size of the display area, and what kind of comment is displayed in what place. Make it easy to understand what is written.

The client device 21 is the server device 22.
It has a video reproduction interface 211 that receives a video from the above video DB 222. Video playback interface 21
1 receives playback control such as playback, rewind, stop, etc. from the video playback control unit 221 on the server device 22,
The video is reproduced in synchronization with the operation of the server device 22.

The client device 21 can specify a certain position or section on the time axis of the image as a place to write a comment in the image, and further, the coordinate or area on the displayed screen space. It has a comment input location designation unit 212 that can be designated.

In this embodiment, when the coordinates on the screen are designated by using the mouse cursor or the like, the time of the image and the designated coordinates are acquired as the position information. Further, the comment input location designating section 212 can directly specify the comment displayed on the screen to acquire the coordinates indicated by the comment, and can also acquire the time when the comment is specified. FIG. 27 is an explanatory diagram of coordinate acquisition by comment specification in the second embodiment of the present invention.

The client device 21 has a comment input section 213 for inputting a comment corresponding to the position information designated by the comment input location designation section 212. Further, the comment input unit 213 sends the comment and the specified position information as comment information to the comment information storage unit 224 on the server device 22. Similar to the comment information (FIG. 4) of the first embodiment of the present invention, the comment information of the present embodiment is a comment, a designated time, and coordinates on the screen.

The client device 21 has a video time acquisition unit 214 that sequentially acquires the video time and sends it to the comment acquisition unit 226 on the server device 22.

The client device 21 is the server device 22.
Of a plurality of comment information acquired by the comment acquisition unit 226 of, and the number of comments displayed on the screen,
Comment display control unit 2 for controlling the display state of comments
Have 15.

In this embodiment, when a certain place (time information and space information) on the video screen is designated, a comment related to that place is selected and displayed on the screen.

The client device 21 has a comment display section 216 for displaying the comment selected and controlled by the comment display control section 215 by using the comment information so as to point to the place designated at the time of input.

In this embodiment, using the coordinate information in the space designated at the time of input, a comment display area in the shape of a balloon is provided for each comment pointing to the same coordinate, and the comments are collectively displayed for each coordinate. FIG. 28 shows a comment display means using such a balloon-shaped display area in the second embodiment of the present invention. At the time of this comment display, the layout is dynamically changed based on the number of comment display areas displayed simultaneously and the minimum display area of each comment display area so that the comment display areas do not overlap each other. In addition, when a large number of comment display areas are displayed on the screen, the comments indicating the near places are collectively displayed in one comment display area, and the comments are displayed while limiting the number of comment display areas.

FIG. 29 shows a screen display example on the client device 22 in the second embodiment of the present invention obtained as described above.

The server device 22 has a video DB 222 for holding video contents, and a video reproduction control section 221 for selecting a video from the video DB 222 and performing video reproduction control such as video reproduction, rewinding and fast forward. The video reproduction control unit 221 controls the video reproduction of the client device 21 by sending a video reproduction control command to the plurality of client devices 21 together with the video content.

The server device 22 has a video area information DB 223 holding video area information and a comment information DB 225 holding comment information.

The image area information held in the image area information DB 223 is information indicating the area of each object imaged on the screen, such as the face of a person or a car. For the method of automatically extracting the area of an object, see "Akama, Mitsui, Kontani, Kushima," Image Retrieval System Using Automatic Extraction of Objects in Images ExSig
t ", DEWS'97 Proceedings, Ma
rch, 1997 ”and the like. Also, regarding the method of tracking an object that moves over time,
"Misu, Naemura, Sakaida, Chung, Kinji," Robust tracking method of soccer player by fusion of multiple information ", Communication method, MVE2
001-46, pp.23-30, 2001 ”and the like.

FIG. 30 is an explanatory diagram of the video information stored in the video area information DB 224 according to the second embodiment of the present invention. In this embodiment, as shown in FIG. 30, the area name and the area shape detected for each time are displayed in the video area information DB 223.
Accumulate on top. At this time, the same area name is given to the areas recognized as the same area at the time t and the time t + 1 and accumulated.
For example, in FIG. 30, a face area estimated as the same area at time t and time t + 1 is given the name of area 2. By using the area recognition and tracking technology as described above, the same name can be given to an area that moves and deforms with time, for example, an area n in FIG.

The server device 22 includes the comment input unit 213.
The comment information sent from the comment information DB 225
It has a comment information storage unit 224 that stores the comment information.

The comment information accumulating section 224 can give a predetermined width to the time when the comment is input and the designated coordinates when accumulating the comment information. In this embodiment, by using the video area information accumulated in the video area information DB 223, the area i including the designated coordinates is
Is the area on the space of the comment. Also, the region i
Accumulates the time section displayed on the screen as the section on the time axis. FIG. 31 shows an explanatory diagram of extended comment information accumulated in the comment information DB in the second embodiment of the present invention. By expanding in this way, for example, when a person's face is specified, the time interval and the display area in which the face is displayed on the screen are used as the comment position information.

The server device 22 includes the video time acquisition unit 214.
It has a comment acquisition unit 226 that fetches a plurality of comment information related to the time information from the comment information DB 225 and transmits the comment information to the comment display control unit 215 by using the time information input from. In the present embodiment, when the time information of each entry in the comment information DB shown in FIG. 31 is the section ts ≦ t <te including the input time information t, the comment information is acquired and transmitted.

Next, the overall operation of the video comment input / display system according to the second embodiment of the present invention will be briefly described. FIG. 32 is an overall flowchart of the video comment input / display system in the second embodiment of the present invention.

Step 21: The video reproduction control section 221 on the server device 22 performs video reproduction control such as video reproduction, rewinding, and pause.

Step 22: A picture is displayed on the client device 21 and it is judged whether or not a predetermined time has elapsed. For example, it is possible to measure the elapsed time by generating a signal at regular intervals by a timer process or the like.

Step 23: It is determined whether or not the user has designated the comment input location using the comment input location designation section 212 on the client device 21. Specifically, in FIG. 29, when the coordinates of a certain point on the video display area are designated using the mouse cursor, it is determined that the coordinates in the screen space have been designated. As shown in FIG. 27, when a comment in the comment display area is designated, the comment information of the designated comment is referred to, and the location (area i) pointed to by the comment is determined as the location on the screen space. Also, the time of the video under playback control is acquired when the mouse is clicked.

Step 24: Determine whether the user has entered a comment. Specifically, when the user inputs a comment and presses the OK button using the comment input unit shown in FIG. 29, it is determined that the comment has been input.
When a comment is input, the comment input unit 213 uses the coordinates and time in the screen space and the comment acquired in step 23 as the comment information to store the comment in the comment storage unit 22.
Send to 4.

Step 25: The coordinates and time on the screen space acquired in step 24 and the comment are accumulated in the comment information DB 225, and the process returns to step 21. At this time, the position information is expanded using the image area information accumulated in the image area information DB 223. Details of the expansion of the position information will be described later.

Step 26: Since a certain period of time has elapsed since the image was displayed, a process for obtaining the time of the image being reproduced is performed.

Step 27: The comment acquisition unit 226
Using the video time acquired in step 26, the comment information related to the video time is acquired from the comment information DB 225. The acquisition method is the same as the processing flow (FIG. 16) of the comment acquisition processing in the above-described first embodiment of the present invention, and therefore detailed description will be omitted.

Step 28: It is judged whether or not the user has selected a comment to be displayed.

Step 29: When the comment display is designated in step 28, the comment display control unit 215 performs the comment display control process associated with the change.

Step 210: Comment display control unit 21
5, the comment is displayed so as to indicate the place where the comment is input, and the process returns to step 21.

Next, the method of accumulating the comment information in step 25 of the above-mentioned overall operation flow will be described in detail.

FIG. 33 shows the second embodiment of the present invention.
It is explanatory drawing of the comment information acquired at step 24 and the extended comment information accumulated on comment information DB225. In step 25, the comment information acquired in step 24 is stored in the comment information DB 225 as extended comment information, as shown in FIG.

FIG. 34 shows a second embodiment of the present invention.
6 is a detailed flowchart showing a method of accumulating comment information executed by a comment information accumulating unit 224. Consider a case where the time of the comment information to be accumulated is t, and the comment information includes coordinate information (x, y) or area a as spatial information. The case where the area a is included is the case where the comment display area is designated as the comment input location in step 23.

Step 251: It is judged whether the spatial information of the comment information acquired in step 24 is a region.

Step 252: If the spatial information is a region, the spatial information to be extended (region t) is set to the region a and step 2
Proceed to 510.

Step 253: Let n be the total number of video areas in the video area information DB 223 detected at time t. Also, the index i for scanning the image area is initialized.

Step 254: The area of the i-th video area information is acquired.

Step 255: It is determined whether or not the area acquired in step 254 includes coordinates (x, y).

Step 256: If the area includes the coordinates (x, y), the area t is set as the area i and the process proceeds to step 2510.

Step 257: To scan the next video area information, the index i is incremented.

As a result, the area t is acquired.

Step 258: The video start time is Ts and the end time is Te. Start time ts and end time te of region t
To initialize. Also, the index t and the area display flag show are initialized.

Step 259: All areas detected at time t are acquired from the video area information DB 223.

Step 2510: It is determined whether or not the area t is included in the area acquired in step 259. If the area is not included, proceed to step 2514.

Step 2511: If the area acquired in step 259 includes the area t, the area t is displayed on the screen. Therefore, the area display flag sh
Determine ow.

Step 2512: If the flag show is false, it means that the area t has not been displayed before the time t, so t is substituted for the start time ts of the area t. Further, the flag show is set to true, and the process proceeds to step 2515.

Step 2513: If the flag show is true, the area t has already been displayed, so t is set at the end time te.
Is substituted.

Step 2514: It is determined whether or not all the images have been scanned from the start to the end, and if so, the process ends.

Step 2515: To scan the area at the next time, increment t.

As a result, the display section [Ts, Te] is acquired.

Next, the comment display control processing of step 29 in the overall operation flow of FIG. 32 will be described in detail.

The comment display control unit 215 executes the step 2
If the comment to be displayed in 8 is selected, the comment to be displayed is selected in step 29.

FIG. 35 shows the second embodiment of the present invention.
7 is a detailed flowchart of a comment display control process executed by a comment display control unit 215 of the client device 21.

Step 291: All the comment information acquired by the comment acquisition process of step 27 is set to the selected state.

Step 292: It is judged whether or not the user has designated specific coordinates in the video display area by clicking with the mouse cursor or the like.

Step 293: If coordinates are specified,
Select the comment to be displayed by performing the display comment selection process.

FIG. 36 shows the second embodiment of the present invention.
9 is a detailed flowchart of a display comment selection process executed by a comment display control unit 215 of the client device 21.

Step 2901: Let n be the total number of comment information acquired by the comment acquisition processing. Further, the index i for scanning comment information is initialized, and the array select [n] for determining comment selection is prepared.

Step 2902: Acquire the location information (area i) of the i-th comment information Step 2903: It is judged whether or not the area i includes the designated coordinates.

Step 2904: If no coordinates are included,
Set select [i] to false.

Step 2905: If coordinates are included, sele
Set ct [i] to true.

Step 2906: It is judged whether all the comment information has been scanned.

Step 2907: If not all scanned, index i to scan next comment information
Is incremented.

Step 2908: If all scans,
Get all comment information for which select [i] is true.

Further, the comment display processing of step 210 in the overall operation flow of FIG. 32 will be described in detail.

In step 210, the comment information selected by the comment display control unit 215 is displayed in a balloon-shaped comment display area for each comment pointing to the same location, and the comments are collectively displayed for each location. At this time, the layout and the area of the comment display areas are dynamically changed and laid out so that the comment display areas do not overlap each other based on the amount of comments displayed simultaneously and the minimum area of the comment display areas. In addition, when a large number of comment display areas are displayed on the screen, the comments indicating the near places are collected and displayed in one comment display area, thereby displaying the comments while limiting the number of comment display areas.

FIG. 37 shows a second embodiment of the present invention.
9 is a detailed flowchart of comment display processing executed by the comment display unit 216 of the client device 1.

Step 2101: The comment selected by the comment display control processing is classified into each area indicated by the comment. FIG. 38 shows an explanatory diagram of comment information classification.

Step 2102: A comment display area is prepared for each area and temporarily arranged on the screen. When temporarily arranging, each comment display area is limited to the minimum area necessary for display, and the comment display area is arranged in an area (displayable area) in which the comment display area can be arranged. FIG. 39 shows an explanatory diagram of a temporary placement method of the comment display area. When the comment display area is arranged on the video, it may hinder the viewing of the video. Therefore, the displayable area of the present embodiment is a square-shaped box with the video display area hollowed out as shown in FIG. The area. The detailed algorithm of the arrangement method will be described later.

Step 2103: When the comment display area is temporarily arranged, it is determined whether all the comment display areas can be displayed. If all the comment display areas can be displayed, the process proceeds to step 2105. If it cannot be displayed, the process proceeds to step 2104. FIG. 40 shows an example in which the display areas of all the comments cannot be arranged.

Step 2104: When it is impossible to display all the comment display areas, the comments indicating the close places are collected and displayed in one display area so that the comment display area can be displayed. To do,
Performs comment merge processing.

Step 2105: Display area layout processing is performed to rearrange the temporarily arranged comment display area so that the allocatable area is effectively used, and change the size of the comment display area to make the comment easy to see.

FIG. 41 shows the second embodiment of the present invention.
6 is a detailed flowchart showing a temporary placement algorithm used by the comment display unit 216 of the client device 1 to temporarily place an area.

Step 2121: Process i comment areas which are not displayed. Prepare a list of i comments.

Step 2122: It is judged whether a comment can be placed in the displayable area. If it cannot be displayed, the processing ends as an error.

Step 2123: The distance [i] is prepared and the index j is initialized.

Step 2124: It is determined whether or not a comment is to be placed on the screen for the first time (that is, whether or not there is a comment previously placed on the screen).

Step 2125: Distance between the center of the jth comment area and the center of gravity of the area pointed to by the comment
Place the jth comment area at an appropriate place in the displayable area so that [j] is the shortest.

Step 2126: If a comment has already been placed, it is adjacent to the already placed comment so that the distance [j] between the center of the comment area and the center of gravity of the area pointed to by the comment is the shortest. Place the jth comment area in place. FIG. 42 shows an explanatory diagram of the distance calculation algorithm of the comment display area.

Step 2127: It is judged whether or not the distance has been calculated for i comments.

Step 2128: Of the i comments, if there is a comment whose distance has not yet been calculated, the index j is incremented and the step 212 is executed.
Return to 3.

Step 2129: Among i comments, the area j having the shortest distance [j] is selected and placed in the displayable area. Figure 43. It is explanatory drawing of the temporary arrangement algorithm of a comment display area.

Step 21210: Comment j from list
, And decrement i.

Step 21211: It is checked whether or not all the comments are arranged, and if they are arranged, the processing is ended.

FIG. 44 shows the second embodiment of the present invention.
9 is a detailed flowchart of a comment merge process executed by the comment display unit 216 of the client device 21. Further, FIG. 45 is an explanatory diagram of the comment merge processing.

Step 2141: The total number of comment display areas is n, and the total number of undisplayable comment display areas is m.

Step 2142: It is judged whether or not m is 0. If it is 0, the process is terminated.

Step 2143: Variable r for area merge
To initialize.

Step 2144: A circle of radius r is drawn from the center of gravity of each area indicated by the n comment display areas. Figure 4
The radius r from the center of gravity of the area referenced by the comment in (1)
The circle drawn in is shown.

Step 2145: It is judged whether or not there are areas where the circles overlap.

Step 2146: If there is no overlapping area, increment r.

Step 2147: If there are areas where the circles overlap, the comments indicating all the overlapping areas are collected in one comment display area. In addition, the newly generated comment display area indicates the center of gravity of all areas where circles overlap. In (2) of FIG. 45,
The comment display areas of the comments 1, 2 and 3 merged in this way are shown.

Step 2148: The total number m of comment display areas is set to m = m- (number of areas where circles overlap) +1, and the process returns to step 2142.

FIG. 46 shows the second embodiment of the present invention.
9 is a detailed flowchart of a display area arrangement process executed by a comment display unit 216 of the client device 21.

Step 2151: The total number of comment display areas is set to n, and the comment display areas are sorted in the reverse order of temporary arrangement. Also, the index i for placement is initialized.

Step 2152: The comment display area i is moved to an empty area in the allocable area, and the comment display area i is arranged so that the distance from the center of gravity of the area pointed to by the comment display area is the shortest. FIG. 47 shows an explanatory diagram of the display area arrangement processing.

Step 2153: It is determined whether or not the n comment display areas have been moved.

Step 2154: The index i is incremented to move the next comment display area.

Step 2155: Initialize the index i.

Step 2156: The comment display area is enlarged by an amount necessary for the comment display in the empty area in the arrangeable area. FIG. 48 shows an explanatory diagram of the display area expansion processing.

Step 2157: It is judged whether or not the n comment display areas have been enlarged. When finished, the process ends.

Step 2158: The index i is incremented to enlarge the next comment display area.

As described above, according to the second embodiment of the present invention, the comment is input by using the video comment input / display method according to claim 1 or the video comment input / display system according to claim 8. You can point and comment where you want. Further, when a comment is browsed, by designating a place on the video display area, the comment can be displayed so that the user can grasp at a glance what place and how much comment is written. When writing a comment for a comment by specifying the comment input location using the method according to claim 2 or the system according to claim 9,
You can easily specify the destination you want to write to.

Further, by accumulating comments by using the method according to claim 3 or the system according to claim 10, it is possible to accurately indicate the place where the comment is input or the place where the comment is to be displayed. There is no need to specify a point in.

Further, by displaying a comment by using the method according to claim 5 or the system according to claim 12, it is possible to dynamically change the amount of the comment, the display place, and the area. The comment can be displayed easily.

Further, in the method according to claim 6 or the system according to claim 13, by displaying comments, it is possible to list the comments written to the same place.

Further, in the method according to claim 7 or the system according to claim 14, by displaying the comment, even if a large amount of the comment display area needs to be displayed on the screen, a fixed amount is displayed. Since the comments can be suppressed and the comments pointing to the close places are put together, the position information pointed to by the comments is not lost.

In the second embodiment of the present invention, the display position and size of the comment dynamically change depending on the position of the area to be commented and the number of comments displayed at the same time. Therefore, the content of the image changes with the passage of time. Even in this case, it is possible to provide an environment in which communication is easy. FIG. 49 shows an explanatory view of a change in comment display in the temporal transition of the video according to the second embodiment of the present invention.

In the second embodiment of the present invention, the method of communicating while watching video simultaneously in the client / server type system is presented as an example. However, the comment information is downloaded from the comment information DB of the server device and the video is displayed on the client. The present invention can also be applied to asynchronous communication in which comments are viewed or added while viewing. By uploading the newly input comment information to the server device side, it is possible to have a communication in which many people share their comment information.

The video comment input / display method according to the above-described embodiment of the present invention can be constructed by software (program), and the video according to the embodiment of the present invention can be executed by executing this program by the CPU of the computer. A comment input / display system can be realized. The constructed program is recorded in a disk device or the like and installed in a computer as necessary, and a flexible disk, a memory card, a CD-RO is installed.
It is stored in a portable recording medium such as M and installed in the computer as needed, or installed in the computer via a communication line or the like, and the CPU of the computer
Executed by

The representative embodiments of the present invention have been described above, but the present invention is not limited to the above embodiments, and various modifications and applications are possible within the scope of the claims.

[0248]

As described above, according to the present invention, when communication is performed via video on a network, the communication can be performed while accurately expressing or grasping a topic as compared with the conventional method. Can be achieved. More specifically, it is possible to write a comment on a video while pointing to a place to be talked about. Also, it becomes easy to understand what kind of comment is written in what place on the video.
It is also possible to easily find out the comment written for the subject of interest and to easily write the comment for the comment, which enables active communication.

[Brief description of drawings]

FIG. 1 is a diagram illustrating the principle of a video comment input / display method of the present invention.

FIG. 2 is a schematic configuration diagram of a video comment input / display device of the present invention.

FIG. 3 is a system configuration diagram in the first embodiment of the present invention.

FIG. 4 is an explanatory diagram of comment information according to the first embodiment of this invention.

FIG. 5 is an explanatory diagram of a comment display control method in the first embodiment of the present invention.

FIG. 6 is a screen display example on the client device according to the first embodiment of the present invention.

FIG. 7 is an explanatory diagram of a video section using scene switching points in the first embodiment of the present invention.

FIG. 8 is a video section information DB in the first embodiment of the present invention.
It is explanatory drawing of the storage method of the video area information on 22.

FIG. 9 is an explanatory diagram of a method of expanding designated coordinates to an area on a screen according to the first embodiment of this invention.

FIG. 10 is comment information D in the first embodiment of the present invention.
It is explanatory drawing of the comment information accumulated on B24.

FIG. 11 is an overall flowchart of the video comment input / display system in the first embodiment of the present invention.

FIG. 12 is an explanatory diagram of comment input means in the first embodiment of the present invention.

FIG. 13 is a detailed flowchart of comment information storage processing according to the first embodiment of this invention.

FIG. 14 is an explanatory diagram of a process of acquiring a video section from video sections that overlap each other in the first example of the present invention.

FIG. 15 is a detailed explanatory diagram of extended comment information accumulated in the first embodiment of the present invention.

FIG. 16 is a detailed flowchart of comment acquisition processing according to the first embodiment of this invention.

FIG. 17 is an explanatory diagram of comment information acquired in the first embodiment of this invention.

FIG. 18 is a detailed flowchart of comment display control processing according to the first embodiment of this invention.

FIG. 19 is a detailed flowchart of a display comment selection process in the first embodiment of the present invention.

FIG. 20 is an explanatory diagram of display comment selection processing according to the first embodiment of this invention.

FIG. 21 is an explanatory diagram of a display area enlarging / reducing process according to the first embodiment of the present invention.

FIG. 22 is an explanatory diagram of a display area scrolling process according to the first embodiment of this invention.

FIG. 23 is an explanatory diagram of comment display processing according to the first embodiment of this invention.

FIG. 24 is an explanatory diagram of an example of displaying a comment by designating a place on a video area in the first embodiment of the present invention.

FIG. 25 is an explanatory diagram of a communication system according to the second embodiment of the present invention.

FIG. 26 is a system configuration diagram in the second embodiment of the present invention.

FIG. 27 is an explanatory diagram of coordinate acquisition by comment specification in the second embodiment of the present invention.

FIG. 28 is an explanatory diagram of a comment display unit using a balloon-shaped display area according to the second embodiment of the present invention.

FIG. 29 is a screen display example on the client device according to the second embodiment of the present invention.

FIG. 30 is an explanatory diagram of video area information accumulated in the second embodiment of the present invention.

FIG. 31 is an explanatory diagram of extended comment information accumulated in the second embodiment of the present invention.

FIG. 32 is an overall flowchart of a video comment input / display system according to the second embodiment of the present invention.

FIG. 33 is an explanatory diagram of comment information accumulated in the second embodiment of the present invention.

FIG. 34 is a detailed flowchart of a comment information storage process according to the second embodiment of the present invention.

FIG. 35 is a detailed flowchart of comment display control processing in the second embodiment of the present invention.

FIG. 36 is a detailed flowchart of display comment selection processing in the first embodiment of the present invention.

FIG. 37 is a detailed flowchart of comment display processing according to the second embodiment of the present invention.

FIG. 38 is an explanatory diagram of comment information classification according to the second embodiment of the present invention.

FIG. 39 is an explanatory diagram of temporary placement processing of a comment display area according to the second embodiment of the present invention.

FIG. 40 is an explanatory diagram showing an example in which all comment display areas cannot be arranged in the second embodiment of the present invention.

FIG. 41 is a detailed flowchart of a temporary placement process of a comment display area according to the second embodiment of the present invention.

FIG. 42 is an explanatory diagram of a distance calculation algorithm for a comment display area according to the second embodiment of the present invention.

FIG. 43 is an explanatory diagram of a temporary placement algorithm of a comment display area according to the second embodiment of the present invention.

FIG. 44 is a detailed flowchart of comment merge processing according to the second embodiment of this invention.

FIG. 45 is an explanatory diagram of a comment merge process according to the second embodiment of this invention.

FIG. 46 is a detailed flowchart of a display area arranging process in the second embodiment of the present invention.

FIG. 47 is an explanatory diagram of a display area arranging process according to the second embodiment of the present invention.

FIG. 48 is an explanatory diagram of display area expansion processing according to the second embodiment of the present invention.

[Fig. 49] Fig. 49 is an explanatory diagram of a change in comment display in the temporal transition of a video according to the second example of the present invention.

FIG. 50 is an explanatory diagram of a video communication method using conventional dedicated software.

FIG. 51 is an explanatory diagram of problems in the conventional video communication method using dedicated software.

FIG. 52 is an explanatory diagram of an existing interface that specifies a certain place on the time axis.

FIG. 53 is an explanatory diagram of a method of associating a comment with a place.

[Explanation of symbols]

1 Client device 2 Server device 11 Video playback controller 12 Comment input location specification section 13 Comment input section 14 Comment display function control interface 15 Video time acquisition section 16 Comment display controller 17 Comment display area 21 Video DB 22 Video section information DB 23 Comment Information Storage 24 Comment information DB 25 Comment acquisition section

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Masashi Morimoto             2-3-1, Otemachi, Chiyoda-ku, Tokyo             Inside Telegraph and Telephone Corporation (72) Inventor Haruhiko Kojima             2-3-1, Otemachi, Chiyoda-ku, Tokyo             Inside Telegraph and Telephone Corporation F-term (reference) 5C052 AA01 AB02 CC01 DD04                 5C053 FA29 JA16 LA04 LA14

Claims (22)

[Claims] 1. A method of inputting and displaying a video comment for communication of video on a network, comprising a playback control procedure in which a user controls playback of a video such as playback and pause, and a playback control procedure. Then, the procedure to specify the time information and space information of the place where you want to enter the comment, the procedure to enter the comment, the comment entered by the user by the procedure to enter the comment above, and the time information of the place to enter the comment above The procedure for accumulating the position information specified by the procedure for specifying the spatial information, the procedure for acquiring the time of the video whose playback is controlled, and the comment related to the acquired time from the accumulated comments Comment display control by specifying the procedure, the comment to display, the place to display the comment, and the amount of comment Procedures and, in accordance with the position information input comment prepared comment display area, the image comment input and display method comprising a step of displaying to point to the location specified as the location you want to enter comments, that. 2. The procedure for designating the time information and the space information, when the comment is written, the comment display area in which the comment is displayed is directly designated to perform additional writing. Video comment input and display method. 3. The procedure of accumulating the comment and the position information gives a predetermined allowable width to the time information and / or the spatial information of the specified position information, and accumulates the obtained location together with the comment. The video comment input / display method according to claim 1 or 2. 4. The comment display control procedure includes the total area of the comment displayable area, the amount of comments to be displayed simultaneously, and the minimum area of the comment display area. 4. The video comment input / display method according to claim 1, wherein the display method is dynamically changed. 5. The procedure for preparing the comment display area and displaying it so as to point to a place is based on the total area of the comment displayable area, the amount of comments to be displayed simultaneously, and the minimum area of the comment display area. 2. The arrangement and display area of the comment display area are dynamically changed and displayed.
4. The video comment input / display method according to any one of 4 to 4. 6. The procedure for preparing the comment display area and displaying it so as to point to a place is to collectively display the comments pointing to the same position in one comment display area,
The video comment input / display method according to any one of claims 1 to 5. 7. The procedure for preparing the comment display area and displaying it so as to indicate a place is to collectively display the comments indicating a close place in one display area.
5. A method for inputting / displaying a video comment according to any one of 5 to 5. 8. A video comment input / display system for communicating video on a network, wherein a user controls playback of a video such as playback and pause, and playback control means for playing back the video. Then, the means to specify the time information and space information of the place where you want to enter the comment, the means to enter the comment, the comment entered by the user by the means to enter the comment, and the time information of the place to enter the comment and A means for accumulating the position information specified by the means for specifying the spatial information, a means for acquiring the time of the video whose playback is controlled, and a comment related to the acquired time from the accumulated comments Comment display by specifying the method, the comment to display, the place to display the comment, and the amount of comment And means for the control, in accordance with the position information you enter a comment to prepare the comment display area, the video comment input and display system having a means for displaying to point to the location specified as the location you want to enter a comment, the. 9. The means for designating the time information and the space information, when writing to a comment, directly designates a comment display area in which the comment is displayed to perform additional writing. Video comment input / display system. 10. The means for accumulating the comment and the position information gives a predetermined allowance to the time information and / or the spatial information of the specified position information, and accumulates the location obtained by this together with the comment. The video comment input / display system according to claim 8 or 9. 11. The means for controlling the comment display is
The amount of comments displayed at one time and the display system thereof are dynamically changed from the total area of the comment displayable area, the amount of comments to be displayed simultaneously, and the minimum area of the comment display area. 10. The video comment input / display system according to any one of 10. 12. The means for preparing the comment display area and displaying it so as to point to a place is a total area of a comment displayable area, an amount of comments to be displayed at the same time,
The video comment input / display system according to any one of claims 8 to 11, wherein the layout and the display area of the comment display area are dynamically changed and displayed from the minimum area of the comment display area. 13. The comment display area is prepared, and the means for displaying so as to indicate a place collectively displays the comments indicating the same position in one comment display area. Video comment input / display system described in section. 14. The comment display area is prepared, and the means for displaying so as to indicate a place collectively displays comments indicating a near place in one display area. Video comment input and display system described. 15. A method for inputting / displaying a video comment for performing communication relating to a video via a network, the procedure for playing the video, time information for designating a place for displaying the comment during the video playback, and A procedure for inputting the spatial information, a procedure for inputting the time information and a comment to be displayed related to the spatial information, a procedure for accumulating the time information, the spatial information, and the comment as comment information, and reproduction. The procedure for acquiring the time information of the video in progress, the procedure for acquiring the comment information whose time information corresponds to the time information for the video acquired above from the accumulated comment information, and the comment for the comment information acquired above Is displayed on the screen, the steps to control the comments displayed and the empty space associated with the comments displayed above. Comment display area was prepared, procedures and, the image comment input and display method with which to display the comment display area images that points to the positions indicated in the spatial information associated with comments that the display on the screen in accordance with the information. 16. A video comment input / display system for performing video-related communication via a network, comprising means for playing video, time information for designating a place for displaying a comment during video playback, and Means for inputting spatial information, means for inputting the time information and a comment to be displayed related to the space information, means for accumulating the time information, the space information, and the comment as comment information, reproduction Means for acquiring the time information of the video inside, and means for accumulating the comment information, means for acquiring comment information whose time information corresponds to the time information for the acquired video, and the comment acquired above. A means to control the comments displayed so that the information comments can be displayed on the screen, and the comments displayed above. Means for displaying on the screen an image indicating a position indicated by the spatial information related to the displayed comment from the comment display area, and providing a comment display area in accordance with the spatial information related to・ Display system. 17. A video comment input / display system including a server device for supplying video and a client device for controlling reproduction of the supplied video, and means for reproducing the video in a video comment input / display system for communicating the video via a network. , A means for inputting time information and space information for designating a place to display a comment during video playback, and inputting a comment to be displayed related to the time information and the space information, the time information, the above A means for transmitting the space information and the comment as comment information to the server device, a means for acquiring time information of a video being played back, and time information from the server device corresponding to the time information of the acquired video. The method of acquiring the comment information that is being displayed and the comment of the comment information acquired above can be displayed on the screen. As described above, a comment display area is prepared according to the means for controlling the displayed comments and the spatial information related to the displayed comments, and the comment display area is used to display the spatial information related to the displayed comments. Means for displaying on the screen an image indicating the specified position. 18. Communication relating to video via a network, comprising a server device for controlling video playback and accumulating comment information about the video, and a plurality of client devices for controlling playback of the same video in synchronization with the server device. In the video comment input / display system that performs A means for inputting the comment to be displayed related to the time information and the space information, and transmitting the time information, the space information, and the comment as comment information to the server device, and time information of a video being reproduced. And the time information from the server device corresponds to the time information of the acquired video. The means for acquiring the comment information, the means for controlling the displayed comment so that the comment of the acquired comment information can be displayed on the screen, and the comment according to the spatial information related to the displayed comment. Means for preparing a display area and displaying on the screen an image indicating a position indicated by the spatial information related to the displayed comment from the comment display area. 19. The means for controlling the displayed comment includes displaying the comment simultaneously based on the total area of the comment displayable area, the amount of the comment to be displayed simultaneously, and the minimum area of the comment display area. , The display format of the comment, the layout of the comment display area, and the area of the comment display area are dynamically changed, and the means for displaying the image indicating the above position on the screen is a method of collecting the comments indicating the close positions into one. 19. The client device according to claim 17, which is displayed in the comment display area. 20. A video comment input / display system comprising a server device for supplying a video image and a client device for controlling reproduction of the supplied video image, wherein a comment is input from the client device in a video comment input / display system. And time information for designating a place to display, means for receiving and accumulating, as comment information, a comment to be displayed related to the time information and the space information, and time information of a video being reproduced from the client device. Receiving, the time information to obtain comment information corresponding to the time information of the received video, a means for transmitting to the client device, and means for receiving and accumulating the comment information, The time information is accumulated as time section information having a predetermined width, and the comment information is The server device, wherein the comment information acquired by the means for acquiring and transmitting information and transmitted to the client device is comment information having time period information including time information of the video. 21. A video comment input which realizes a video comment input / display function for communicating the video to a client device connected to a server supplying the video via a network and controlling reproduction of the supplied video.・ It is a display program, and the above-mentioned function is a function to play back the video, a function to input the time information and the space information that specifies the place to display the comment during the playback of the video, and the time information and the space information. A function of inputting a comment to be displayed, which is related to the above, and transmitting the time information, the space information, and the comment as comment information to the server device; a function of acquiring time information of a video being reproduced; From the server device, the comment information whose time information corresponds to the time information of the acquired video is acquired. And a function to control the displayed comment so that the comment of the acquired comment information can be displayed on the screen, and a comment display area is prepared in accordance with the spatial information related to the comment displayed above. And a function of displaying an image indicating a position indicated by the spatial information related to the displayed comment on the screen from the display area, and the function of controlling the displayed comment is an area in which the comment can be displayed. Based on the total area of, the amount of comments that should be displayed at the same time, and the minimum area of the comment display area,
The function to dynamically change the amount of comments displayed at the same time, the display format of comments, the layout of the comment display area, and the area of the comment display area, and to display the image indicating the above position on the screen, displays the comment indicating a close position. Video comment input / display program that collectively displays in one comment display area. 22. Inputting a video comment according to claim 21.
A computer-readable recording medium recording a display program.