JP6718169B2 - Information presentation system, information presentation device and program - Google Patents

Description

The present invention relates to an information presentation system, and more particularly, to an information presentation system using an avatar.

In recent years, there has been provided a service in which a comment is received from a user who views a video distributed via the Internet, and the comment is overlaid on the video (for example, Patent Document 1).

In this service, the comment input by the user during the playback of the video is stored together with the commenting time (relative time from the start of the playback of the video), and when the video was played, the video was viewed in the past. By sequentially displaying the comments of the user according to the comment giving time, it is possible to create a sense of unity with other users as if they were watching the video together.

On the other hand, the non-patent document 1 describes Kinect (registered trademark), focusing on the current situation that most of the communication on the Internet depends on character information, although 65% of human communication is performed by non-verbal behavior. ) Is used to acquire skeleton information of a user who views a moving image, and a system for overlay-displaying a rod-shaped avatar generated based on the skeleton information on the moving image is disclosed.

However, Non-Patent Document 1 reports that in a system user questionnaire, many respondents did not feel a sense of unity with other users. The reason for this is that their avatar and others He cited that the avatar was difficult to identify, and that he could not express the interaction between his avatar and the avatars of others.

JP 2008-148071 A

Yuka Yoshida, Yoshiaki Miyashita, Sharing a sense of unity on moving images by superimposing body movements, IPSJ Interaction 2012, p527-523, 2012

The present invention has been made in view of the above-described problems in the conventional technology, and provides a novel information presentation system capable of producing a sense of unity with other users who watch a distributed video at different times. With the goal.

The present inventor has earnestly studied a new information presentation system capable of producing a sense of unity with other users who watched a distribution video at different times, and as a result, came up with the following configuration, and arrived at the present invention. It was.

That is, according to the present invention, there is provided an information presentation system in which a plurality of information presentation devices and an information distribution server are connected via a network, wherein the information presentation device first distributes a distribution moving image distributed via the network. No. 1 display area, a distribution motion image display section, a physical motion information acquisition section for acquiring physical motion information relating to a physical motion of a user who views the distribution video displayed in the first display area, and the physical motion. The information distribution server includes: an avatar generation unit that generates a first avatar that expresses the physical action of the user based on information; and an avatar display unit that displays the first avatar in a second display area. Is a database that stores avatar information including physical movement information related to physical movements when a user views a distribution video in the past, and the distribution in the past in response to a request from the information presentation device that displays the distribution video. An avatar information distribution unit that transmits the avatar information of another user who viewed the video, and the avatar generation unit is based on the physical motion information included in the avatar information received from the information distribution server, A second avatar that reproduces the physical action of the other user is generated, and the avatar display unit displays the first avatar and the second avatar together in the second display area. A system is provided.

As described above, according to the present invention, there is provided a novel information presentation system capable of producing a sense of unity with other users who watched a distribution video at different times.

The figure which shows the network structure of the information presentation system of 1st Embodiment. The figure which shows the viewing-and-listening environment of the delivery video in 1st Embodiment. The figure which shows the moving image viewing screen in 1st Embodiment. The functional block diagram which shows the function structure of each apparatus which comprises the information presentation system of 1st Embodiment. The figure which shows the condition setting screen in 1st Embodiment. The flowchart which shows the process which the information presentation system of 1st Embodiment performs. The figure which shows the moving image search screen in 1st Embodiment. The figure which shows the data structure of 1st Embodiment. The conceptual diagram for demonstrating the generation method of an avatar. The flowchart which shows the skeleton coordinate conversion process in 1st Embodiment. The figure which shows the avatar display area in the split display mode in 1st Embodiment. The flowchart which shows the skeleton coordinate conversion process in 1st Embodiment. The conceptual diagram for demonstrating the skeleton coordinate conversion process in 1st Embodiment. The figure which shows the avatar display area|region in the batch display division mode in 1st Embodiment. The flowchart which shows the avatar information generation process in 1st Embodiment. The flowchart which shows the comment information generation process in 1st Embodiment. The figure which shows the moving image viewing screen in the audio|voice reproduction mode and comment display mode in 1st Embodiment. The figure which shows the variation of the avatar expression in 2nd Embodiment. The functional block diagram which shows the functional structure of each apparatus which comprises the information presentation system of 2nd Embodiment. The figure which shows various tables. The figure which shows the data structure of the avatar information of 2nd Embodiment. The figure which shows the notification screen etc. of 2nd Embodiment. The figure which shows the moving image search screen etc. of 2nd Embodiment. The flowchart which shows the process which the information presentation system of 2nd Embodiment performs. The figure which shows the moving image viewing screen in 2nd Embodiment. The flowchart which shows the interactive information generation process in 2nd this embodiment. The flowchart which shows the process which the information presentation system of 2nd Embodiment performs. The figure which shows the moving image viewing screen in 2nd Embodiment. The flowchart which shows the vote acceptance process etc. in 2nd Embodiment. The figure which shows the message notification screen in 2nd Embodiment. The flowchart which shows the message notification screen generation process in 2nd Embodiment. The figure which shows the condition setting screen in 3rd Embodiment. The figure which shows the moving image viewing screen in 3rd Embodiment. The figure which shows the moving image viewing screen in 3rd Embodiment. The figure which shows the moving image viewing screen in 3rd Embodiment.

The present invention will be described below with reference to the embodiments shown in the drawings, but the present invention is not limited to the embodiments shown in the drawings. In each drawing referred to below, common elements are denoted by the same reference numerals, and description thereof will be appropriately omitted.

(First embodiment)
FIG. 1 shows a network configuration of an information presentation system 100 according to the first embodiment of the present invention. As shown in FIG. 1, the information presentation system 100 of the present embodiment includes a plurality of information presentation devices 10, a moving image distribution server 30, and an avatar information distribution server 20.

The information presenting apparatus 10 can be referred to as a personal computer used for viewing distribution videos, and can communicate with each of the video distribution server 30 and the avatar information distribution server 20 via a network 50 referred to as the Internet. It is connected. In the following, the information presentation device 10 will be referred to as a PC 10.

The moving image distribution server 30 reads a moving image file (hereinafter, simply referred to as moving image) from the database and delivers it in response to a request from the PC 10.

On the other hand, the avatar information distribution server 20 reads out and distributes the avatar information from the database in response to the request from the PC 10. The details of the avatar information will be described later.

Next, the viewing environment of the distributed moving image in this embodiment will be described based on FIG. As shown in FIG. 2, a display device 12, a motion capture device 14, a microphone 16, and a speaker 18 are connected to the PC 10 used for viewing the distributed moving image. Here, the motion capture device 14 is a means for three-dimensionally measuring the physical movement of the user 40 who views the distribution video, and is fixed at an appropriate position.

In the present embodiment, it is premised that the user 40 takes a desired action (physical action/speech or utterance) in accordance with the content while viewing the distribution video displayed on the display device 12, and the motion capture device 14 Tracks the physical movement of the user 40 who views the distributed video, and the microphone 16 records the utterance and utterance of the user 40 during that time.

Kinect (registered trademark, the same applies below) from Microsoft Corporation can be cited as a motion capture device 14 suitable for this embodiment. Since Kinect has a built-in microphone, when using Kinect as the motion capture device 14, the microphone 16 can be omitted.

Here, the PC 10 of the present embodiment generates a moving image viewing screen for simultaneously viewing the distributed moving image and the avatar, and displays the moving image viewing screen on the display device 12.

FIG. 3 exemplarily shows a moving image viewing screen 60 displayed on the display device 12. As shown in FIG. 3, the video viewing screen 60 includes a distribution video display area 62 for displaying a distribution video and an avatar display area 64 for displaying an avatar. On the moving image viewing screen 60, an avatar display region 64 arranged below the distributed moving image display region 62 is arranged so that each of the distributed moving image display region 62 and the avatar display region 64 reminds of the stage and the audience seats.

In the present embodiment, when the distribution moving image is displayed in the distribution moving image display area 62 like a stage, the avatar of the user who views the distribution moving image (hereinafter referred to as user avatar) is displayed in the avatar display area 64 like a spectator seat. To do. At this time, by displaying the avatars of other users who have watched the video displayed in the distribution video display area 62 in the past (hereinafter referred to as “other avatars”) in the avatar display area 64 together with the user avatars, the distribution video is displayed. The viewing user creates a sense of unity and presence as if he or she were watching the same stage with other users.

Next, the functional configuration of each device that constitutes the information presentation system 100 of the present embodiment will be described based on the functional block diagram shown in FIG.

As shown in FIG. 4, the PC 10 of this embodiment includes a distribution moving image acquisition unit 102, an avatar information acquisition unit 103, a distribution moving image display unit 104, an avatar display unit 105, a physical motion information acquisition unit 106, an avatar generation unit 107, and a body. A configuration including a characteristic information acquisition unit 108, a physical activity calculation unit 109, a voice data acquisition unit 110, a voice output unit 112, a comment acquisition unit 114, an overlap determination unit 116, an avatar display position change unit 117, and an avatar information generation unit 118. To be done. The above-described functional units are realized by installing a dedicated application on the PC 10.

The distribution moving image acquisition unit 102 is a functional unit for acquiring a moving image from the moving image distribution server 30, and downloads the moving image selected by the user from the moving image distribution server 30.

The avatar information acquisition unit 103 is a functional means for acquiring the avatar information of another user from the avatar information distribution server 20, and when the user views the video distributed from the video distribution server 30, the avatar information is pasted. The avatar information of another user who has viewed the above is acquired from the avatar information distribution server 20.

The distribution moving image display unit 104 is a functional unit for displaying a moving image acquired from the moving image distribution server 30 in the distribution moving image display area 62 (see FIG. 3). For example, a moving image distributed from the moving image distribution server 30 is a distributed moving image. Streaming reproduction is performed in the display area 62.

The avatar display unit 105 displays the user avatar of the user who views the video displayed on the display device 12 and the avatar of another user who has viewed the video in the past together in the avatar display area 64 (see FIG. 3 ). It is a functional means for. Here, in the present embodiment, the “divided display mode” and the “collective display mode” are prepared as the display modes for displaying the avatars in the avatar display area 64, and via the condition setting screen 70 shown in FIG. The user can select either mode.

The physical motion information acquisition unit 106 is a functional unit that acquires physical motion information related to the physical motion of the user who views the moving image displayed in the avatar display area 64. The physical motion information here is data representing the physical motion of the user tracked by the motion capture device 14, and in the present embodiment, the time series of the position coordinates (hereinafter, referred to as skeleton coordinates) of the joints of the user's body. Data is used as physical movement information.

The avatar generation unit 107 generates a user avatar expressing the physical motion of the user based on the physical motion information acquired by the physical motion information acquisition unit 106, and based on the avatar information acquired from the avatar information distribution server 20, Is a functional means for generating another person's avatar that reproduces the body movement of the user. The avatar generation unit 107 also assigns the drawing color of the avatar.

The physical characteristic information acquisition unit 108 is a functional unit for acquiring physical characteristic information relating to the physical characteristic of the user as attribute information of the user who views the moving image displayed in the distribution moving image display area 62.

The physical activity calculation unit 109 is a functional unit that calculates the physical activity based on the physical motion information of the user as attribute information of the user who views the video displayed in the distribution video display area 62.

The voice data acquisition unit 110 is a functional unit for acquiring voice data of voice uttered by the user as attribute information of the user who views the video displayed in the distribution video display area 62.

The voice output unit 112 is a functional unit for outputting the voice of another user in synchronization with the moving image displayed in the distribution moving image display area 62 based on the avatar information received from the avatar information distribution server 20.

The comment acquisition unit 114 is a functional unit for acquiring, as attribute information of the user who views the video displayed in the distribution video display area 62, a comment in which the utterance extracted from the voice data of the voice uttered by the user is converted into a text. Is.

The overlap determination unit 116 determines whether the display position of the user avatar and the display position of the other avatar in the avatar display area 64 overlap when the user selects the “collective display mode” as the avatar display mode. Is a functional means of.

The avatar display position changing unit 117 is a functional means for changing the display position of the avatar so that the display position of the user avatar and the display position of the other avatar do not overlap.

The avatar information generation unit 118 is a functional means for generating avatar information and transmitting it to the avatar information distribution server 20. Here, the avatar information to be transmitted includes the physical movement information of the user who views the video displayed in the distribution video display area 62, the attribute information of the user, and the information acquisition date and time when these pieces of information were acquired.

On the other hand, as shown in FIG. 4, the avatar information distribution server 20 of the present embodiment includes an avatar information distribution unit 22, an avatar information management unit 24, and a database 26.

The database 26 is a functional means for storing the avatar information including the physical movement information when the user views the moving image in association with the moving image.

In response to a request from the PC 10 that has distributed the moving image, the avatar information distribution unit 22 reads from the database 26 avatar information relating to another avatar of the user who has viewed the moving image in the past, and transmits the avatar information to the PC 10. Is.

The avatar information management unit 24 is a functional means for storing the avatar information received from the PC 10 in the database 26.

The functional configurations of the respective devices configuring the information presentation system 100 according to the present embodiment have been described above. Next, the content of the processing executed by each of the functional means described above will be described based on the flowchart shown in FIG. 6.

When the user activates the dedicated application installed on the PC 10, a moving image search screen is displayed on the display screen of the display device 12, as shown in FIG. In response to this, when the user inputs a keyword into the text box displayed on the video search screen and executes the search, the search result screen shown in FIG. 7B is displayed on the display screen. In response to this, when the user selects an arbitrary moving image from the list with moving image thumbnail images displayed on the search result screen, the following process is started.

First, in step 101, the distribution video acquisition unit 102 reads the video selected by the user from the video distribution server 30. In the following step 102, an avatar information acquisition request specifying the video ID of the video read by the avatar information acquisition unit 103 is transmitted to the avatar information distribution server 20. In response to this, the avatar information distribution unit 22 of the avatar information distribution server 20 reads from the database 26 one or more pieces of avatar information associated with the moving image ID specified by the avatar information acquisition request received from the PC 10, and transmits the avatar information to the PC 10. ..

FIG. 8A schematically shows the data structure of the avatar information stored in the database 26. The avatar information is information about an avatar of a user who has watched a moving image in the past, and as shown in FIG. 8A, is collectively described in a file of a predetermined format (for example, a CSV file) associated with the moving image ID. It Here, each avatar information includes a user ID, time-series data of skeleton coordinates of the user to which a frame number of the avatar animation is added, acquisition date and time of avatar information, physical activity, height of the user, size of the user, voice data. The path to the comment information and the path to the comment information are described.

FIG. 8B schematically shows the data structure of the comment information stored in the database 26. The comment information is information regarding a comment added to the moving image by a user who viewed the distributed moving image in the past, and as shown in FIG. 8B, a file in a predetermined format associated with the moving image ID (for example, CSV file). Here, in each comment information, the user ID, the comment giving time, and the content of the comment are described. It should be noted that the comment addition time here means a relative time from the start of the reproduction of the moving image.

The audio data stored in the database 26 is audio data recorded while a user who has watched a distributed moving image in the past is watching the moving image.

The avatar information distribution server 20 reads the comment information and the voice data based on the path described in the avatar information associated with the moving image ID specified by the avatar information acquisition request, collectively transmits them to the PC 10 as avatar information, and The information acquisition unit 103 acquires this.

Returning to FIG. 6 again, the explanation will be continued.

In the following step 103, tracking of body movements by the motion capture device 14 and voice recording by the microphone 16 are started, and at the same time, in the following step 104, the moving image read by the distribution moving image display unit 104 in step 101 is displayed in the distribution moving image display area 62 (Fig. 3) and start playback.

Then, steps 105 to 110 described below are repeatedly executed in synchronization with the frame interval (for example, 0.1 [sec]) of the avatar animation.

First, in step 105, the body motion information acquisition unit 106 analyzes the tracking data of the motion capture device 14 and acquires the coordinates of the joints of the user's body as the skeleton coordinates, and the acquired skeleton coordinates (three-dimensional coordinates) are two-dimensional. Convert to coordinates. Then, the body motion information acquisition unit 106 adds the frame number [1] to the converted skeleton coordinates (two-dimensional coordinates) and stores it in the temporary storage.

In the present embodiment, in step 105, as shown in FIG. 9A, the coordinates of 14 joints present in the head, spine, and limbs are acquired as skeletal coordinates. The coordinates of any joint among the joints of the body can be acquired as the skeleton coordinates. When using Kinect of Microsoft Corp. equipped with a function of outputting skeleton coordinates (three-dimensional coordinates) as the motion capture device 14, the body motion information acquisition unit 106 causes the skeleton coordinates (three-dimensional coordinates) output by Kinect. And convert it to two-dimensional coordinates.

In the following step 106, the avatar generation unit 107 reads the skeleton coordinates (two-dimensional coordinates) to which the frame number [1] is added from the avatar information of the other avatars acquired in the previous step 102.

In the following step 107, the avatar generation unit 107 generates a user avatar based on the skeleton coordinates (two-dimensional coordinates) acquired in the previous step 105, and based on the skeleton coordinates (two-dimensional coordinates) acquired in the previous step 106. Generate one or more other party avatars. FIG. 8C schematically shows the data structure of the avatar class generated in step 107. As shown in FIG. 8C, the avatar class describes a frame counter, skeleton coordinates (two-dimensional coordinates) of the avatar, and offset values (described later). At this stage, the offset value is set to the initial value 0. In this embodiment, as shown in FIG. 9B, a circle centering on the coordinates of the head is drawn, an ellipse centering on the coordinates of the spine is drawn, and a line segment connecting the coordinates of the extremities is drawn. While this produces a humanoid avatar, other embodiments may produce avatars according to other suitable rules.

After generating the avatar in step 107, in the following step 108, the avatar display position changing unit 117 executes the skeleton coordinate conversion process, and in the following step 109, the avatar generating unit 107 executes the color allocation process. At this time, in step 108 and step 109, different processing is executed depending on the display mode selected by the user.

Here, first, the skeleton coordinate conversion processing executed by the avatar display position changing unit 117 when the user selects the “divided display mode” will be described based on the flowchart shown in FIG.

First, in step 201, the avatar display area 64 is divided by the total number of avatars (user avatar and other avatars) generated in step 107. In the following step 202, the most central area among the plurality of divided areas is set as the display area of the user avatar, and the remaining area is set as the display area of the other avatar in step 203.

In the following step 204, the skeleton coordinates of each avatar are converted so that each avatar fits within the display area set in step 203, and the process ends. Specifically, the size of each avatar is reduced so that it fits within the set display area, and the skeleton coordinates of each avatar after reduction are shifted so that the avatar moves within the set display area.

Next, the color allocation process executed by the avatar generation unit 107 when the user selects the “divided display mode” will be described based on the flowchart shown in FIG.

First, in step 301, an arbitrary color is assigned to the drawing color of the user avatar. Next, in step 302, a color that can be distinguished from the drawing color of the user avatar is assigned to the drawing color of the other avatar, and the process ends. In step 302, a set of distinguishable colors may be prepared in advance, and each color may be assigned to the drawing color of each avatar. Alternatively, the drawing color selected by the user may be assigned to the user avatar. A color that can be distinguished from the drawing color may be automatically assigned to another avatar.

11A to 11C exemplify the avatar display area 64 in the "divided display mode". In the example shown in FIG. 11, the user avatar is drawn as a white avatar, and the other avatar is drawn as a black avatar. In this way, in the “divided display mode”, the user avatar is arranged in the center in a color different from that of the other avatars, so that the user does not lose sight of his or her own avatar.

Next, the skeleton coordinate conversion processing executed by the avatar display position changing unit 117 when the user selects the “collective display mode” will be described based on the flowchart shown in FIG.

First, in step 401, as shown in FIG. 9B, two rectangular regions centering on the skeletal coordinates (two-dimensional coordinates) of the user's wrists acquired in the previous step 105 and the skeleton of the spine. One rectangular area centered on the coordinates is set as the judgment area C1, while only one rectangular area centered on the skeletal coordinates of the spine of the other avatar is set as the judgment area C2.

In the following step 402, it is determined whether or not the determination area C1 of the user avatar and the determination area C2 of the other avatar overlap. Here, with respect to the determination area C1 and the determination area C2 defined based on the value obtained by adding the offset value to the skeleton coordinates of the avatar class (see FIG. 8C), the overlapping of the both is determined. As a result, when it is determined that the determination area C1 and the determination area C2 do not overlap (No in S402), the processing is ended as it is, and the offset value of the avatar class of each avatar is not changed. For example, as shown in FIG. 13A, when the determination area C1 of the user avatar U and the determination area C2 of the other avatar A do not overlap, the offset of each avatar class of the user avatar U and the other avatar A. The values are not changed, and the display positions of the user avatar U and the other person's avatar A are maintained as they are.

On the other hand, if it is determined that the determination area C1 and the determination area C2 overlap (Yes in S402), the process proceeds to step 403. In step 403, the offset value of the avatar class related to the other avatar related to the overlapping determination area C2 is increased or decreased by the amount of overlap between the determination areas C1 and C2. In the following step 404, the skeleton coordinates of the other avatar related to the overlapping determination area C2 are shifted based on the latest offset value. Specifically, the latest offset value is added to the skeleton coordinates described in the avatar class of the other avatar related to the overlapping determination area C2 to shift the skeleton coordinates of the other avatar.

For example, when the determination area C1 of the user avatar U and the determination area C2 of the other person avatar A are overlapped as shown in FIG. 13B, in step 403, the other person avatar A in the X-axis direction is detected. The offset value is decremented by the overlap amount α, and in step 404, the skeleton coordinate is shifted in the negative direction of the X-axis direction by adding the offset value decremented by the overlap amount α to the skeleton coordinate of the other avatar A. In this case, the other person's avatar A moves by α in the leftward direction of the paper surface. On the other hand, when the determination area C1 of the user avatar U and the determination area C2 of the other person avatar A overlap in the form as shown in FIG. The offset value is incremented by the overlap amount β, and in step 304, the skeleton coordinate is shifted in the positive direction of the X-axis direction by adding the offset value incremented by the overlap amount β to the skeleton coordinate of the other avatar A. In this case, the other person's avatar A moves by β in the right direction on the paper surface.

Next, the color allocation process executed by the avatar generation unit 107 when the user selects the “collective display mode” will be described based on the flowchart shown in FIG.

First, in step 501, a color having a predetermined hue angle in the HSV color space is assigned to the drawing color of the user avatar. In step 501, a preset color may be assigned to the drawing color of the user avatar, or the user may be allowed to select the drawing color.

In the following step 502, the hue angle of the color to be assigned to the next stranger avatar is calculated. Specifically, the value obtained by dividing 360 by the total number of avatars is set as the angle increment, and the angle increment is added to the hue angle of the color assigned to the drawing color of the user avatar in the previous step 501. In the following step 503, the color of the calculated hue angle is assigned to the drawing color of the other avatar that has not been assigned yet.

In the following step 504, it is determined whether or not the colors of all the other avatars have been assigned. If not (S504, No), the process returns to step 502. In step 502, the angle increment is added to the hue angle of the color assigned to the drawing color of the user avatar, and in step 503, the drawing color of the other avatar to which the newly calculated hue angle has not been assigned yet. Assign to.

After that, the processes in steps 502 to 504 are repeated until the colors of all the other avatars have been assigned. When the colors of all the other avatars have been assigned (S504, Yes), the process proceeds to step 505. In the following step 505, a predetermined transparency (for example, 70%) is set for the color set for the other avatar, and the process is ended.

Returning to FIG. 6 again, the explanation will be continued.

In the following step 110, the avatar display unit 105 displays the user avatar and the other-person avatar generated based on the skeleton coordinates shifted in the skeleton coordinate conversion process (step 108) and the drawing color assigned in the color assignment process (step 109). The image (first frame image) is drawn in the avatar display area 64 (see FIG. 3). After that, while the moving image is being reproduced in the distribution moving image display area 62, steps 105 to 110 described above are repeatedly executed.

In the meantime, in step 105, the frame number given to the newly acquired skeleton coordinate of the user is incremented by 1, and in step 106, the frame number of the skeleton coordinate of another avatar read from the avatar information is incremented by 1. The color allocation process of step 109 can be skipped in the second and subsequent processes by maintaining the drawing color allocated the previous time.

FIG. 14 exemplarily shows an animation of an avatar displayed in the avatar display area 64 in the “collective display split mode”. In the example shown in FIG. 14A, in the avatar display area 64, other than the user avatar U of the user who is viewing the video displayed in the distribution video display area 62 and the four users who have viewed the video in the past. Person Avatar A is also displayed. At this time, when the user who views the moving image makes a gesture to push away the left and right spaces, in response to this, the display position of the alien avatar A is changed as shown in FIG. 14B.

In this way, in the "collective display split mode", the user avatar U is arranged in the center in a color different from that of the other avatar A, and therefore the user does not lose sight of his or her own avatar. In addition, the appearance that the other person's avatar A leaves in real time in response to the approaching user avatar U reminds us of the feeling that the self and the other person keep a distance in the real world, and the user uses the avatar expression to express the other user's avatar A. You can feel a sense of unity with. In the "collective display split mode", different colors are assigned to the different avatars A, and the transparent processing is performed on the respective colors, so that even when the display positions of a plurality of different avatars A overlap. , It becomes possible to distinguish each other's avatar A.

In addition, in the above-described embodiment, a mode in which a rectangular area centering on the skeletal coordinates of both wrists or a rectangular area centering on the skeletal coordinates of the spine is set as the determination area is shown, but this is merely an example. It goes without saying that various design changes can be made to the position, number and shape of the determination areas. For example, the determination areas related to the skeleton coordinates of both ankles may be set, or the determination areas related to all the skeleton coordinates may be set. When the determination regions are set to all the skeleton coordinates, other avatars around the user will behave naturally so as to take a distance from the user avatar, even if the user does not actively take action such as a gesture of pushing away.

When the reproduction of the moving image ends, in step 111, the tracking by the motion capture device 14 and the sound recording by the microphone 16 are simultaneously ended, and in the following step 112, the sound data file of the sound recorded by the sound data acquisition unit 110 is generated. get. After that, in the following step 113, the avatar information generation unit 118 executes the avatar information generation process, and the process ends. Hereinafter, the avatar information generation process will be described based on the flowchart shown in FIG.

First, in step 601, the skeleton coordinates (two-dimensional coordinates) of the user for all the frames acquired in the previous step 105 are read.

Subsequently, in step 602, the physical characteristic information acquisition unit 108 calculates “height of user” and “size of user”. Here, the “user height” and the “user size” may be appropriate indexes representing the height of the user and the size of the body of the user, respectively. For example, the body feature information acquisition unit 108 determines the distance between the coordinates of the head and the coordinates of the joint of the ankle included in the skeleton coordinates (three-dimensional coordinates) of the user acquired by the body motion information acquisition unit 106 as “the height of the user. , And the projected area of the user calculated based on the number of pixels and the depth of the region in which the user is captured in the captured image of the motion capture device 14 is acquired as “the size of the user”.

In the following step 603, the comment acquisition unit 114 executes the comment information generation process. The comment information generation process executed in step 603 will be described below with reference to the flowchart shown in FIG.

First, in step 701, the recorded voice data is read, and then in step 702, the speech of the user is extracted from the voice data by voice recognition processing, and the content thereof is converted into text. Finally, in step 703, comment information is generated with the utterance content (text) as the comment content and the utterance recording time as the comment addition time. The utterance recording time here is a relative time from the start of the reproduction of the moving image.

When the comment information is generated by the procedure described above, the process proceeds to step 604, and the physical activity calculating section 109 executes the physical activity calculating process. The physical activity calculation process executed in step 604 will be described below with reference to the flowchart shown in FIG.

First, in step 801, the skeleton coordinates of both hands are read from the skeleton coordinates (two-dimensional coordinates) of the user for all the frames read in the previous step 601. In the following step 802, the variances of the left hand skeleton coordinates and the variances of the left hand skeleton coordinates are obtained. Finally, in step 803, the total value of the variances of the skeleton coordinates of both hands obtained in step 802 is set as the physical activity of the user.

In the following step 605, the avatar information generation unit 118 acquires the “user's skeleton coordinates”, “user's height”, “user's size”, “physical activity”, and these information acquired in the previous steps 601-604. The avatar information file that describes the acquisition date and time, the moving image ID of the reproduced moving image, and the user ID is generated, the generated avatar information file, the “comment information” generated in step 603, and the “voice information acquired by the voice data acquisition unit 110. The data is also transmitted to the avatar information distribution server 20, and the process ends.

On the other hand, the avatar information management unit 24 of the avatar information distribution server 20 stores the “comment information” and the “voice data” received from the PC 10 in the database 26 while associating the “comment information” and the “voice data” with the moving image ID and the storage destination path and the PC 10. Avatar information (see FIG. 8A) that describes information included in the received avatar information file is generated and stored in the database 26.

The content of the process executed by the information presentation system 100 according to the present embodiment has been described above. Next, the information presentation method using the above-described “comment information” and “voice data” will be described.

In this embodiment, a mode (hereinafter, referred to as a voice reproduction mode) for reproducing the voice of another user who has watched the video displayed in the distribution video display area 62 in the past is prepared, and the condition setting screen 70 (FIG. 5) is provided. The user can select the mode via the (see). Here, when the user sets the audio reproduction mode, the audio output unit 112 outputs the audio data included in the avatar information received from the avatar information distribution server 20 in synchronization with the video displayed in the distribution video display area 62. To do.

FIG. 17A shows the moving image viewing screen 60 in the “sound reproduction mode”. In the example shown in FIG. 17A, a moving image of a rock concert is displayed in a distribution moving image display area 62 that is like a stage, and a user avatar U is displayed in the center of an avatar display area 64 that is like a spectator seat. Around it, many other avatars A are displayed. In the "voice reproduction mode", the sound of the other user who has watched the video of the rock concert displayed in the distribution video display area 62 at the same time is output at the same time, thereby producing the buzz of the spectator seats and the cheers resounding in the spectator seats. be able to. Also, by outputting the voice data related to the other avatar adjacent to the user avatar at a volume higher than that of the voice data related to the other avatars other than that, it seems that the voice of the audience next to the user is leaking. You can also create a realistic feeling.

Further, in the present embodiment, a mode (hereinafter referred to as a comment display mode) for displaying a comment of another user who has viewed the video displayed in the distribution video display area 62 in the past is prepared, and the condition setting screen 70 ( The user can set the mode via (see FIG. 5). Here, when the user sets the comment display mode, the avatar display unit 105 displays the comment content (text) of the comment information included in the avatar information received from the avatar information distribution server 20 according to the comment giving time.

FIG. 17B shows the moving image viewing screen 60 when both the “voice reproduction mode” and the “comment display mode” are set. In the example shown in FIG. 17B, in addition to the sounds of other users who have previously viewed the video of the rock concert displayed in the distribution video display area 62 being output at the same time, in addition to the case of being adjacent to the user avatar U, Only the comment corresponding to the person avatar A is displayed. In this case, the content of the utterance of another avatar A (another user) located next to the other can be understood in the buzz of the spectator seats and the cheers resounding in the spectator seats.

Finally, a method of selecting another avatar to be displayed in the avatar display area 64 will be described. In the present embodiment, the upper limit number of other avatars displayed in the avatar display area 64 and the search conditions thereof can be set through the condition setting screen 70 shown in FIG. In the example shown in FIG. 5, the sorting condition can be set as a search condition for other avatars. From the pull-down menu, “new order”, “moving order”, “tall order”, “body order” "Large order of" can be selected as an item of the sorting condition.

For example, when the user sets the upper limit number “N” and the item “newest first” via the condition setting screen 70, the avatar information acquisition unit 103 causes the avatar information acquisition unit 103 to set the search condition set in step 102 (see FIG. 6). Is transmitted to the avatar information distribution server 20. In response to this, the avatar information distribution unit 22 reads the top N avatar information with the new “acquisition date/time” value from the plurality of avatar information stored in the database 26 and associated with the moving image ID. After that, the avatar information distribution unit 22 reads the comment information and the voice data based on the path described in the read avatar information, and collectively sends them to the PC 10 as avatar information.

When the user selects the "order of frequent movements", the avatar information distribution unit 22 reads the top N avatar information with the highest "physical activity" value from the database 26, and the user is "tall". If "order" is selected, the top N avatar information with a large "user height" value is read from the database 26, and if the user selects "large order", the database 26 displays "user". The top N avatar information having a large value of “size” are read.

As described above, according to the present embodiment, the user avatar is arranged in the center of the avatar display area in a color different from that of the other avatars, so that the user does not lose sight of his own avatar. In addition, it is possible to create a sense of unity with other users who watch the distribution video at different times by interactively expressing the user avatar and the avatar of another person.

The first embodiment of the present invention has been described above, and then the second embodiment of the present invention will be described. It should be noted that in the following, description of the parts common to the contents of the first embodiment will be omitted, and only the differences from the first embodiment will be explained.

(Second embodiment)
The second embodiment mainly differs from the first embodiment in the following points (1) to (4).

(1) Point where "viewing mode" of moving image can be selected In the second embodiment, two "viewing modes", "dancing mode while watching" and "only watching mode" are prepared. Here, the "watching and dancing mode" is an operation mode similar to that described in the first embodiment, and in this mode, the user avatar that is linked to the movement of the user is displayed on the moving image viewing screen 60. On the other hand, the "simply watch mode" is a mode in which the user enjoys watching and enjoying the movements of other avatars linked to the distribution video. In this mode, the user's shooting and audio recording are not performed, but the movements of the user are linked. The user avatar that does this is not displayed on the video viewing screen 60.

(2) A variation of the avatar expression is prepared In the second embodiment, as shown in FIGS. 18A and 18B, the avatar expression that can identify the sex of the user and the viewing mode selected by the user. Is prepared. In this regard, FIG. 18A exemplarily shows avatar expressions of the male user and the female user when the “dancing while watching mode” is selected. In addition, FIG. 18B exemplarily shows the avatar expressions of the male user and the female user when the “only watch mode” is selected. In addition, in the present embodiment, as shown in FIG. 18C, an avatar expression that can identify the physical characteristics of the user is prepared.

(3) Points that can vote for other users In the second embodiment, in the “seen-only mode”, other users (hereinafter, other users) where the other person avatar displayed on the video viewing screen 60 expresses their physical movement. Who are called avatar users).

(4) Point where message can be sent to other avatar user In the second embodiment, a message can be sent to the other avatar user displayed on the moving image viewing screen 60 in the “seen-only mode”. it can.

Below, the differences mentioned in the above (1) to (4) will be explained exclusively.

Here, first, based on the functional block diagram shown in FIG. 19, the functional configuration of each device constituting the information presentation system 100 of the second embodiment will be described.

As shown in FIG. 19, the PC 10 of the second embodiment includes a voting acceptance unit 120, a message acceptance unit 122, and an interactive information generation unit 123 in addition to the configuration of the PC 10 of the first embodiment shown in FIG.

The vote accepting unit 120 is a functional unit for accepting a vote for the user of the other avatar via the other avatar displayed in the avatar display area 64.

The message accepting unit 122 is a functional means for accepting a message addressed to the user of the other avatar via the other avatar displayed in the avatar display area 64.

The interactive information generation unit 123 generates interactive information including any of the identification information of the user of the other avatar selected by the user, the vote for the user of the other avatar, and the message addressed to the user of the other avatar, and the avatar. It is a functional means for transmitting to the information distribution server 20.

On the other hand, as shown in FIG. 19, in addition to the configuration of the avatar information distribution server 20 of the first embodiment shown in FIG. 4, the avatar information distribution server 20 of the second embodiment has a user authentication unit 21, a selection history management. The unit 23, the voting history management unit 27, the message history management unit 28, and the notification screen generation unit 29 are provided.

The user authentication unit 21 is a functional unit for executing user authentication of the user of the avatar information distribution server 20.

The selection history management unit 23 is a functional unit for managing the history of the selection of another avatar by the user.

The voting history management unit 27 is a functional unit for managing the history of voting for the user of another avatar.

The message history management unit 28 is a functional unit for managing the history of messages addressed to users of other avatars.

The notification screen generation unit 29 generates a notification screen that displays one of the other user who has selected the user, the other user who has voted for the user, and the other user who has sent the message to the user, and sends the notification screen to the PC 10. It is a functional means for.

In addition, in the second embodiment, the database 26 of the avatar information distribution server 20 has four tables (user information management table 500, avatar selection history management table 600, voting history management table) shown in FIGS. 700, message history management table 800). The contents of each table will be described below.

As shown in FIG. 20A, the user information management table 500 includes a field 501 for storing the user's “user ID”, a field 502 for storing the user's “password”, and a user's “user”. It includes a field 503 for storing “first name” and a field 504 for storing “gender” of the user, and the above information (user ID, password, user name, gender,...) Input at the time of user registration. And manage it. The “user name” is not limited to the real name of the user, and may be a nickname.

As shown in FIG. 20B, the avatar selection history management table 600 includes a field 601 for storing a user ID of a user represented by the selected avatar (hereinafter, referred to as “selected user”), A field 602 for storing the user IDs of other users who have selected an avatar (hereinafter referred to as “selection source user”), and the total number of times the selection source user has selected the selection destination user (hereinafter, “cumulative selection count”). A field 603 for storing a "viewing mode" selected by the selection source user when the selection destination user is selected, and a selection source user selects the selection destination user. A field 605 for storing the latest date and time (hereinafter referred to as “last selected date and time”).

As shown in FIG. 20C, the voting history management table 700 includes a field 701 for storing a user ID of a voted user (hereinafter, referred to as a “voteee user”) and other votes for the voteee user. 702 for storing the user IDs of users (hereinafter referred to as "voting source users") and fields for storing the date and time when the voting source user voted for the voting destination user (hereinafter referred to as "voting date and time"). 703 and.

As shown in FIG. 20D, the message history management table 800 includes a field 801 for storing a user ID of a user who has sent a message (hereinafter, referred to as “destination user”) and a message to the destination user. A field 802 for storing the user ID of another user who has transmitted the message (hereinafter referred to as "source user"), and the date and time when the source user transmitted a message to the destination user (hereinafter referred to as "transmission date and time"). A field 803 for storing a message, a field 804 for storing a message (text data) transmitted by the transmission source user, and a video (hereinafter, referred to as a video capturing screen 60 of the video viewing screen 60 when the transmission source user transmits the message). Field 805 for storing a "captured video").

Here, FIG. 21 shows a data structure of avatar information stored in the database 26 of the avatar information distribution server 20 of the second embodiment. As shown in FIG. 21, in the second embodiment, the attributes described in the avatar information of the first embodiment (user ID, acquisition date and time of avatar information, physical activity, height of user, size of user, voice) Data path, comment information path, time-series data of the user's skeleton coordinates), as well as the user's "user name", the user's "sex", and the cumulative number of votes cast by the user. Four attributes "total number of votes" and "conversation mode" are added to the avatar information. The details of the "conversation mode" will be described later.

Although the functional configurations of the respective devices configuring the information presentation system 100 of the second embodiment have been described above, the contents of the processing executed by the information presentation system 100 of the second embodiment will be sequentially described.

In the present embodiment, when using the service, the user logs in via a predetermined login screen provided by the avatar information distribution server 20. FIG. 22A exemplarily shows a login screen provided by the avatar information distribution server 20. When the user inputs his/her user ID and password on the login screen and clicks the login button, the PC 10 sends an authentication request including the input user ID and password to the avatar information distribution server 20. In response to this, the user authentication unit 21 of the avatar information distribution server 20 collates the user ID and password included in the authentication request received from the PC 10 with the values of the fields 501 and 502 of the user information management table 500, respectively. If the two match, the login of the user is permitted.

In response to the login permission of the user, the notification screen generation unit 29 informs the login user of the existence of another user who has made three types of approaches (selection, voting, and message transmission) to the login user. Generates a notification screen 80 for sending to the PC 10, and the PC 10 displays it.

FIG. 22B exemplifies a notification screen 80 displayed on the PC 10. As shown in FIG. 22B, the notification screen 80 has an area 82 for displaying the selection source user who has selected the login user, an area 84 for displaying the voting source user who has voted for the login user, and the login. The notification screen generating unit 29 includes the area 86 for displaying the transmission source user who has transmitted the message to the user, and generates the notification screen 80 by the following procedure.

First, the field 601 of the avatar selection history management table 600 (see FIG. 20B) is searched with the user ID of the logged-in user as a key, and the record matching the key is searched for with the value of the field 604 (date and time of last selection). Sort in ascending order. Next, the “user ID” and “viewing mode” of the selection source user are acquired from the fields 602 and 604 of the top N records of the sorted records, respectively, and the “user ID” associated with the acquired “user ID” is acquired. The “user name” and “sex” are acquired from the user information management table 500. Next, an icon representing an avatar in a form corresponding to the acquired "sex" and "viewing mode ("dancing mode while watching" or "only watching mode")" is generated, and the acquired " A user name” is attached and they are arranged in the area 82.

Next, using the user ID of the logged-in user as a key, the field 701 of the voting history management table 700 (see FIG. 20C) is searched, and the record that matches the key is updated with the new value (voting date and time) of the field 703. Sort in order. Next, the “user ID” of the voting source user is acquired from the field 702 of the top N records of the sorted records, and the “user name” and “sex” associated with the acquired user ID are acquired in the user information management table 500. To get from. Next, create an icon that imitates an avatar according to the acquired "gender" and "viewing mode (in this case, "only watch mode")", and add the acquired user name on the icon. Are arranged in the area 84.

Next, using the user ID of the logged-in user as a key, the field 801 of the message history management table 800 (see FIG. 20D) is searched, and the record matching the key is updated with the new value (transmission date and time) of the field 803. Sort in order. Next, the “user ID” of the destination user is acquired from the field 802 of the top N records of the sorted records, and the “user name” and “sex” associated with the acquired user ID are stored in the user information management table 500. To get from. Next, create an icon that imitates an avatar according to the acquired "gender" and "viewing mode (in this case, "only watch mode")", and add the acquired user name on the icon. Are arranged in the area 86.

After that, in response to the notification screen 80 being displayed, when the user clicks the “video search screen” button in the upper right of the screen, the avatar information distribution server 20 provides the video search screen 90 to the PC 10 in response to this. , PC 10 displays this.

FIG. 23A exemplifies a moving image search screen 90 displayed on the PC 10. As shown in FIG. 23A, in the moving image search screen 90 of this embodiment, in addition to a text box for inputting a search word, pull-down menus (1) to (4) below are displayed in a region 92. To be done.

(1) Pull-down menu for selecting rearrangement condition of avatar priority In this embodiment, the pull-down menu displayed on the condition setting screen 70 (see FIG. 5) in the first embodiment is the moving image search screen 90. Displayed in. In the pull-down menu of the present embodiment, "new order", "moving order", "order with the most votes", and "order with the most selected times" can be selected as the avatar rearrangement condition items. It is like this.

(2) Pull-down menu for selecting gender of avatar In the present embodiment, it is possible to select the gender condition of avatar (user). From the pull-down menu, "sex independent", "male", and "female" are selected. You can choose either.

(3) Pull-down menu for selecting viewing mode In the present embodiment, as described above, the viewing mode can be selected. You can choose one of the modes.

(4) Pull-down menu for selecting conversation mode In the present embodiment, the conversation mode can be selected, and either "all users" or "selected user" can be selected from the pull-down menu. It has become. Here, the "conversation mode" is a mode in which when the logged-in user selects the "dancing mode while watching", the permission destination of the reproduction of the recorded sound is selected. As the conversation mode, "all users" is selected. When selected, all users who use this service are permitted to play the recorded audio of the logged-in user. When "selected user" is selected as the conversation mode, the logged-in user is Only the user of the other avatar selected when viewing the corresponding video is permitted to play the voice of the logged-in user.

In response to the display of the moving image search screen 90 described above, when the user selects each pull-down menu item displayed in the area 92 and then inputs a keyword in the text box to execute the search, the screen is The screen changes to the search result screen shown in FIG. In response to this, when the user selects an arbitrary moving image from the list with moving image thumbnail images displayed on the search result screen, the process starts.

Here, first, the content of the process executed by each functional unit shown in FIG. 19 when the user selects the “dancing mode while watching” as the viewing mode will be described based on the flowchart shown in FIG.

First, in step 901, the distribution video acquisition unit 102 reads the video selected by the login user from the video distribution server 30. In the following step 902, the avatar information acquisition unit 103 specifies the moving image ID of the read moving image, the “sorting condition” selected by the login user in the pull-down menu of the moving image search screen 90, and the “sex condition” of the login user. The avatar information acquisition request is transmitted to the avatar information distribution server 20. In response to this, the avatar information distribution unit 22 of the avatar information distribution server 20 first specifies in the request from one or more pieces of avatar information associated with the video ID specified in the avatar information acquisition request received from the PC 10. The avatar information that matches the “gender condition” is selected.

Specifically, when "male" is specified as the "gender condition", avatar information in which "male" is described as the gender attribute is selected, and when "female" is specified The avatar information in which “female” is described as the gender attribute is selected, and when “sex unquestioned” is specified, all the avatar information associated with the specified moving image ID is selected.

Next, the avatar information distribution unit 22 selects avatar information that matches the “sorting condition” specified in the received avatar information acquisition request from the avatar information that matches the “sex condition”. Here, when “order with the largest number of votes” is specified as the “sorting condition”, the avatar information distribution unit 22 describes the “total number of votes” described in the avatar information selected earlier. The top N avatar information with a large value of are selected.

On the other hand, when "order of most selected times" is specified as the "sorting condition", the avatar information distribution unit 22 first uses the user ID of the logged-in user as a key, and the avatar selection history management table 600 (Fig. 20(b)) field 601 is searched, and the value of the field 603 (cumulative selection count) is set as the first priority for the record matching the key, and the value of 605 (last selection date/time) is set as the second priority. Sort in ascending order of priority. Next, the value of the field 603 (user ID of the selection source user) and the user ID described in the previously selected avatar information are collated in order from the top of the sorted records, and the matching user ID is described. By repeating the procedure N times such as sequentially selecting avatar information, the top N avatar information is selected.

The avatar information distribution unit 22 reads the N pieces of avatar information selected by the above-described procedure from the database 26 and transmits them to the PC 10.

In the following step 903, the tracking of the body motion by the motion capture device 14 and the voice recording by the microphone 16 are started, and at the same time, in the following step 904, the distribution moving image display unit 104 starts the reproduction of the moving image read in step 901. Then, steps 905 to 910 are repeatedly executed in synchronization with the frame interval (for example, 0.1 [sec]) of the avatar animation. Here, since the contents of the processing executed in steps 905 to 910 are basically the same as those in steps 105 to 110 shown in FIG. 6 of the first embodiment, here, among them, the first embodiment will be described. The processing of steps 906 and 907, which are partially different in content from, will be described.

In step 906 of this embodiment, the avatar generation unit 107 reads out the skeleton coordinates (two-dimensional coordinates) to which the frame number [1] is added from the avatar information of the other avatar acquired in step 902, and the “user name , "Sex", "height of user", and "size of user".

In step 907 of the present embodiment, the avatar generation unit 107 acquires the “sex” of the login user from the user information management table 500, and the skeleton coordinates (two-dimensional coordinates) acquired in step 905 and the “sex of the login user. A user avatar is generated based on. Specifically, an avatar (see FIG. 18) in a mode according to the “sex” of the logged-in user is generated so that the sex of the user can be identified. In addition, the avatar generation unit 107 adds the “user name” of the login user to the head of the generated avatar of the login user.

At the same time, the avatar generation unit 107 stores the five types of information of the skeleton coordinates (two-dimensional coordinates) acquired in step 906, “user name”, “sex”, “user height”, and “user size”. Generate another avatar based on this. Specifically, after selecting a mode according to the acquired "sex", the height of the other avatar is increased or decreased according to the value of "user height", and according to the value of "user size". By increasing or decreasing the width of another user's avatar, it is possible to identify the sex of the other user as represented by the other user's avatar and its physical characteristics. In addition, the avatar generation unit 107 adds the "user name" of the user of the other avatar to the head of the generated other avatar. FIG. 25A exemplifies a video viewing screen 60 in the case where the user has selected “new order” or “order selected the most times” as the avatar rearrangement condition in the “watching and dancing mode”. To indicate.

On the other hand, when the logged-in user has selected "order of movement" as the avatar rearrangement condition, the avatar generation unit 107 causes the "crown" icon to appear above the other avatar whose rank is first. Is added. Furthermore, if the logged-in user has selected “in descending order of the number of votes”, the “crown” icon is added to the head of the other avatar who ranked first and the other avatars "Number of votes" is displayed next to the user name. FIG. 25B exemplarily shows the moving image viewing screen 60 when the user has selected “order with the largest number of votes” as the avatar rearrangement condition in the “dancing while watching” mode.

In the present embodiment, steps 905 to 910 described above are repeatedly executed during the reproduction of the moving image. After that, when the reproduction of the moving image ends, in step 911, the tracking by the motion capture device 14 and the sound recording by the microphone 16 are simultaneously ended, and in the following step 912, the sound data file of the sound recorded by the sound data acquisition unit 110 is created. Generate and get. After that, in the following step 913, the avatar information generation unit 118 executes the avatar information generation processing.

Note that in step 913, the avatar information generation unit 118 executes the avatar information generation processing (steps 601 to 605) shown in FIG. 15 as in the first embodiment. The contents differ in the following points. That is, in step 605 of this embodiment, the avatar information generation unit 118 acquires the “user name” and “sex” of the login user from the user information management table 500, and also acquires the “conversation mode” selected by the login user. If the "conversation mode" is "selected user", the user ID of the user of the other avatar selected by the logged-in user who is the reproduction permission destination of the recorded voice (hereinafter, reproduction permission destination user ID). To get Then, the avatar information generation unit 118 acquires the “user name”, the “sex”, the “conversation mode”, the “reproduction permission destination user ID”, and the “user skeletal coordinates” acquired in steps 601 to 604. , “User height”, “User size”, “Physical activity”, “Acquisition date” of these information, “Movie ID” of the played video, and “User ID” of the logged-in user. An information file is generated, and the generated avatar information file, the “comment information” generated in step 603, and the “voice data” acquired by the voice data acquisition unit 110 are transmitted to the avatar information distribution server 20.

In response to this, the avatar information management unit 24 of the avatar information distribution server 20 stores the “comment information” and the “voice data” received from the PC 10 in the database 26 by associating them with the moving image ID and the path of the storage destination. And information included in the avatar information file received from the PC 10 (“user ID”, “user name”, “sex”, “acquisition date”, “physical activity”, “height of user”, “size of user”) , “User skeleton coordinates”, “conversation mode”, “reproduction permission destination user ID”) are generated and stored in the database 26.

Finally, in step 914, the interactive information generation unit 123 executes "interactive information generation processing". The "interactive information generation process" will be described below with reference to the flowchart shown in FIG.

First, in step 1001, the following information (1) to (5) is stored in the interactive information.
(1) User ID of the other avatar user selected by the login user
(2) User ID of login user
(3) Viewing mode selected by the logged-in user (in this case, "dancing while watching")
(4) Selected date and time of the avatar of another person (for example, the date and time when the reproduction of the distributed video is finished)
(5) Video ID of the distribution video viewed by the login user

In the following step 1002, the process branches depending on the viewing mode selected by the logged-in user. In this case, the viewing mode selected by the logged-in user is the “dancing while watching” mode, and the process proceeds to step 1005. In the following step 1005, the interactive information in which the above information (1) to (5) is stored is transmitted to the avatar information distribution server 20, and the process is ended.

On the other hand, in response to the avatar information distribution server 20 receiving the interactive information from the PC 10, the selection history management unit 23 makes the avatar selection history based on the information (1) to (4) included in the interactive information. The management table 600 (see FIG. 20B) is updated. Specifically, using the user ID of the user of another avatar and the user ID of the logged-in user included in the interactive information as keys, the fields 601 and 602 of the avatar selection history management table 600 are searched, and records matching the two keys are searched. , The value of the field 603 (cumulative number of selections) is incremented by 1, and the value of the field 604 (viewing mode) of the record is updated to “dancing while watching” included in the interactive information, and the record The value of the field 605 (last selection date/time) is updated to “selection date/time” included in the interactive information.

If there are no records that match the two keys, new records are added by the number of the combinations, and the fields 601, 602, 603, 604, and 605 of the added records are added with interactive information. The user ID of the user of the other avatar included, the user ID of the login user included in the interactive information, the cumulative number of selections “1”, the viewing mode “dancing while watching” included in the interactive information, and the “selection date and time” included in the interactive information. Is stored.

The processing executed when the user selects the “dancing mode while watching” as the viewing mode has been described above. Next, the processing executed when the user selects the “only watching mode” as the viewing mode. Will be described based on the flowchart shown in FIG.

First, in step 1101, the distribution moving image acquisition unit 102 reads the moving image selected by the user from the moving image distribution server 30. In the following step 1102, the avatar information acquisition unit 103 requests the avatar information acquisition in which the moving image ID relating to the read moving image and the “sorting condition” and “gender condition” selected by the user in the pull-down menu of the moving image search screen 90 are specified. Is transmitted to the avatar information distribution server 20. In response to this, the avatar information distribution unit 22 of the avatar information distribution server 20 first specifies in the request from one or more pieces of avatar information associated with the video ID specified in the avatar information acquisition request received from the PC 10. The N pieces of avatar information that match the “gender condition” and the “sorting condition” are selected, the selected N pieces of avatar information are read from the database 26, and are transmitted to the PC 10. The processing up to this point is the same as that of the "dancing while watching" mode.

In the following step 1103, video capture of the moving image viewing screen 60 is started, and in the following step 1104, the distribution moving image display unit 104 starts playing the moving image read in step 1101. Then, steps 1105 to 1111 are repeatedly executed in synchronization with the frame interval (for example, 0.1 [sec]) of the avatar animation. Here, in each of Steps 1105 to 1109, the same processing as Steps 906 to 910 shown in FIG. 24 is executed, but in the “seen mode”, the contents of Steps 1107 and 1109 are as follows. different.

Specifically, in the “only watch mode”, as shown in FIG. 28( a ), a moving image viewing screen 60 is provided for selecting three approach items “moving avatar”, “voting”, and “message”. When the pull-down menu 65 is displayed and the user selects “move avatar” from the pull-down menu 65, the moving image viewing screen 60 transitions to a state in which drag and drop of another avatar is accepted. In this case, the user can move the desired other avatar to an arbitrary place by dragging and dropping it, as shown in FIG. In the “skeleton coordinate conversion process” of step 1107, the avatar display position changing unit 117 causes an offset with respect to the skeleton coordinates described in the avatar class of the moving other avatar as the user moves the other avatar. The values are added to shift the skeleton coordinates of the other avatar.

In addition, in the "only watch mode", unlike the "dancing while watching mode", the user avatar linked to the logged-in user is not generated. As shown in FIG. 5, an icon representing the avatar of the logged-in user sitting on the chair is drawn in the center of the avatar display area.

Further, in the “only watch mode”, the voting accepting unit 120 executes the “voting accepting process” (step 1110) and the message accepting unit 122 executes the “message accepting process” in parallel with the above-described Steps 1105-1109. (Step 1111).

In the “only watch mode”, when the user selects “vote” from the pull-down menu 65, the video viewing screen 60 transitions to a state in which a vote is accepted via another person's avatar, and the vote accepting unit 120 causes the vote accepting unit 120 in FIG. ) The process shown in the flowchart is executed.

In step 1201, the click of another avatar is waited for (step 1201, No), and when it is detected that the user has clicked any other avatar as shown in FIG. 28B (step 1201, Yes). In the following step 1202, the user ID of the user of the clicked other avatar, the number of votes “1”, the vote date and time (the date and time when the other avatar was clicked), and the video ID of the distribution video viewed by the logged-in user are displayed. The linked information (hereinafter referred to as voting-related information) is recorded in temporary storage.

On the other hand, when the user selects “Message” from the pull-down menu 65, the video viewing screen 60 transitions to a state in which the transmission of a message is accepted via another person's avatar, and the message accepting unit 122 displays the flowchart of FIG. Perform the indicated process.

In step 1301, the process waits for a click on another avatar (step 1301, No), and when it is detected that the user has clicked on any other avatar (step 1301, Yes), then in step 1302, as shown in FIG. ), a message input form 66 is overlay-displayed on the moving image viewing screen 60 to accept message input. In response to this, when the user inputs a message in the input form 66 and clicks the send button, in the next step 1303, the user ID of the clicked user of the other avatar and the message (text) input in the input form 66. Information (hereinafter referred to as message-related information) in which the data) and the transmission date and time (the date and time when the transmission button is clicked) are associated is recorded in the temporary storage.

In the present embodiment, steps 1105 to 1111 described above are repeatedly executed during the reproduction of the moving image. After that, when the reproduction of the moving image ends, in step 1112, the video capture of the moving image viewing screen 60 ends, and in the following step 1113, a captured moving image file including the video capture of the moving image viewing screen 60 is acquired. After that, in the following step 1114, the interactive information generation unit 123 executes the "interactive information generation process" and ends the process. Here, the “interactive information generation process” executed by the interactive information generation unit 123 in the “seen mode” will be described with reference to the flowchart shown in FIG.

First, in step 1001, the following information (1) to (4) is stored in the interactive information.
(1) User ID of the other avatar user selected by the login user
(2) User ID of login user
(3) Viewing mode selected by the logged-in user (in this case, "just watch")
(4) Selected date and time of the avatar of another person (for example, the date and time when the reproduction of the distributed video is finished)
(5) Video ID of the distribution video viewed by the login user

In the following step 1002, the process branches depending on the viewing mode selected by the logged-in user. In this case, the viewing mode selected by the logged-in user is the “only view mode”, and therefore the process proceeds to step 1003. In the following step 1003, if the above-mentioned voting-related information is stored in the temporary storage, the voting-related information is stored in the interactive information, and the process proceeds to step 1004. In the following step 1004, when the above-mentioned message-related information is stored in the temporary storage, the message-related information and the captured moving image file acquired in the previous step 1113 (see FIG. 27) are stored in the interactive information, The process proceeds to step 1005.

In the following step 1005, the interactive information in which the information of (1) to (5), the voting related information, the message related information, and the captured moving image file are stored is transmitted to the avatar information distribution server 20, and the process ends.

On the other hand, in response to the avatar information distribution server 20 receiving the interactive information from the PC 10, the selection history management unit 23 reads the above information (1) to (4) included in the interactive information and “dancing while watching. The avatar selection history management table 600 (see FIG. 20B) is updated by the same procedure as that in the "mode". However, in the “only watch mode”, the value (viewing mode) of the field 604 of the avatar selection history management table 600 is updated to “only watch”.

In addition, in the “only watch mode”, the voting history management unit 27 updates the voting history management table 700 (see FIG. 20C) based on the voting-related information included in the received interactive information. Specifically, new records are added by the number of user IDs of the users of other avatars included in the voting-related information, and the fields are added to the fields 701, 702, and 703 of the added record as voting-related information. The user ID of the user of the other avatar, the user ID of the login user, and the voting date and time are stored. In addition, the voting history management unit 27 searches the avatar information stored in the database 26 using the moving image ID included in the voting-related information and the user ID of the user of the other avatar as a key, and the avatar information matching the two keys. The value of "cumulative number of votes" described in 1 is incremented by 1.

In addition, in the "only watch mode", the message history management unit 28 saves the captured moving image file included in the received interactive information in the database 26, and based on the message-related information included in the received interactive information. The message history management table 800 (see FIG. 20D) is updated. Specifically, new records are added by the number of user IDs of the users of other avatars included in the message-related information, and messages are added to fields 801, 802, 803, 804, and 805 of the added records, respectively. The user ID of the other avatar user, the user ID of the logged-in user, the transmission date and time, the message (text data), and the path to the saved captured moving image file included in the related information are stored.

The processing executed when the user selects the “only watch mode” as the viewing mode has been described above. Subsequently, the “message notification screen generation processing” executed by the notification screen generation unit 29 of the avatar information distribution server 20. Will be described with reference to the flowchart shown in FIG.

The notification screen generation unit 29 of the avatar information distribution server 20 waits for the click of another avatar (the icon thereof) displayed in the area 86 after providing the notification screen 80 (see FIG. 22B) to the user's PC 10. (Step 1201, No). Then, when it is detected that the user has clicked (an icon of) another avatar displayed in the area 86 (step 1401, Yes), in the subsequent step 1402, the user ID of the clicked user of the other avatar. The key is used as a key to search the message history management table 800, the message (text data) is read from the field 804 of the record matching the key, and at the same time, the path stored in the field 804 is accessed to read the captured moving image file.

Then, the notification screen generation unit 29 generates a message notification screen based on the message (text data) read in the previous step 1402 and the captured moving image file (step 1403), and transmits the generated message notification screen to the user's PC 10. Then (step 1404), the process ends. FIG. 30B exemplarily shows the message notification screen 67 transmitted to the PC 10. In the example shown in FIG. 30B, as the message notification screen 67, a dialog screen formed by overlaying a message on the captured moving image is overlay-displayed on the notification screen 80.

The second embodiment of the present invention has been described above, and then the third embodiment of the present invention will be described. Note that, in the following, description of the parts common to the contents of the second embodiment will be omitted, and only the differences from the second embodiment will be explained.

(Third Embodiment)
In the third embodiment, as a display mode of the avatar, in addition to the “divided display mode” and the “collective display mode” mentioned above, a “return mode” is prepared, and the condition setting screen 70 shown in FIG. 23 is displayed. This is different from the second embodiment in that the user can select one of the modes via the above. Hereinafter, the "convey mode" will be described.

When the logged-in user selects the “dancing mode while watching” as the viewing mode and selects the “entrainment mode”, the avatar display position changing unit 117 causes the avatar (refer to FIG. 24) generated in step 907 described above. The same number of rectangular areas as the total number of user avatars and other avatars) is defined, and the defined rectangular areas are arranged in the moving image viewing screen 60 so as to surround the distribution moving image display region. Then, the rectangular area located in the center of the moving image viewing screen 60 is allocated as the display area of the user avatar, and the remaining rectangular area is allocated as the display area of the other avatar. Here, with respect to the display areas of the other avatars set to the left and right of the display area of the user avatar, the priorities are associated in the order close to the display area of the user avatar. In accordance with the ascending order when is selected, the display areas having higher priorities (that is, the display areas closer to the user avatar) are allocated in order from the other person's avatar having a higher order. The avatar display position changing unit 117 transforms the skeleton coordinates of each avatar so that the avatar fits within the display area allocated by each avatar.

FIG. 33(a) exemplarily shows the moving image viewing screen 60 when the "return mode" is selected. In FIG. 33A, a rectangular area indicated by a broken line is an avatar display area, and each display area is given priority by circled numbers (however, the broken line indicating the display area and the priority order are on the screen). Not displayed).

Here, in the third embodiment, as shown in FIG. 33B, the login user 40 sets the positions of circled numbers 0 to 8 (hereinafter, referred to as positions 0 to 8) in front of the motion capture device 14. The user avatar of the logged-in user is configured to move in the display area (hereinafter, referred to as positions 0 to 8) indicated by circled numbers 0 to 8 shown in FIG. 33A in synchronization with the movement to the left and right. ing. Specifically, the avatar display position changing unit 117 analyzes the input information of the motion capture device 14 to determine which of the positions 0 to 8 shown in FIG. 33B the login user stands in, The user avatar is moved to the display area (0 to 8) corresponding to the position (0 to 8) where the logged-in user stands.

As a result, when the logged-in user moves to the position 8 in FIG. 33B, for example, in response to this, the user avatar A displays the display area 8 on the moving image viewing screen 60 as shown in FIG. Move to. At this time, the other person's avatar E is already arranged in the display area 8, and the user avatar A and the other person's avatar E are simultaneously arranged in the display area 8. In the present embodiment, the avatar display position changing unit 117 monitors the success or failure of a predetermined start condition when the user avatar and the other avatar are simultaneously placed in the same display area, and the start condition is satisfied. In this case, the other avatar makes a transition to a state in which the other avatar moves following the user avatar (hereinafter referred to as a "carrying state"). After the transition to the entrainment state, when the logged-in user moves the position (0 to 8) shown in FIG. 33B to the left or right, the user avatar and the avatar of another person synchronize with this and hold the hand as if they were holding hands. The display area (0 to 8) is moved even if it is accompanied by a person.

Here, in the present embodiment, the content of the above-described start condition is not limited, and for example, based on the input information of the motion capture device 14, a predetermined gesture of the login user (for example, a gesture of grasping an object or a beckoning gesture). ) Is detected as the start condition, or the elapsed time after the user avatar and the other avatar are placed in the same display area at the same time may be set as the start condition. ..

On the other hand, the avatar display position changing unit 117 monitors whether or not a predetermined ending condition is satisfied after the user avatar and the other-person avatar have transited to the accompanying state, and cancels the accompanying state when the ending condition is satisfied. Here, in the present embodiment, the content of the above-described termination condition is not limited, and for example, from the input information of the motion capture device 14, a predetermined gesture of the login user (for example, a gesture of letting go of an object or a gesture of expelling a person). May be used as the end condition, and the end condition may be that the elapsed time after the user avatar and the other avatar have arrived at a certain display area reaches a predetermined threshold value.

In the present embodiment, the logged-in user can take (move) the desired other-person avatar to an arbitrary display position by utilizing the above-described transition to the entrainment state and the mechanism of its cancellation. Hereinafter, this point will be described based on a specific example.

FIG. 34B shows a state in which the user avatar A is carrying the other person's avatar E and is moving to the display area 2. In the case of this example, an exclamation mark “!” indicating that the other person's avatar E has transitioned to the entrainment state is displayed. After that, the user avatar A and the other person's avatar E enter the display area 2 where the other person's avatar D is already arranged, and the user avatar A and the other person's avatar E remain in the display area 2 for a predetermined period of time. When the ending condition of is satisfied, the entrainment state of the user avatar A and the other person's avatar E is canceled, and at that time, the display area 2 is set to the display area of the other person's avatar E.

In response to this, another avatar of another person moves as shown in FIG. That is, the other avatar D originally arranged in the display area 2 moves from the display area 2 to the display area 4, and the other avatar H originally arranged in the display area 4 moves from the display area 4 to the display area 6. The other person's avatar K originally arranged in the display area 6 moves from the display area 6 to the display area 2 in which the other person's avatar E was originally arranged. Finally, by moving the user avatar, which has been removed from the entrainment state with the other person's avatar E, to the original display area 0, the arrangement mode of the avatars of the moving image viewing screen 60 becomes the mode shown in FIG. 35(b).

As described above, according to the present embodiment, not only is it possible to perform virtual interaction with another person's avatar on the video viewing screen 60, but it is possible for another user to communicate with the other person via the other person's avatar. By voting or sending a message to other users, a real interaction trigger with a real other person is provided.

Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments, and within the scope of the embodiments conceivable to those skilled in the art, the functions and effects of the present invention can be obtained. As long as it is effective, it is included in the scope of the present invention.

Each function of the above-described embodiment can be realized by a device-executable program described in C, C++, C#, Java (registered trademark), and the like, and the program of the present embodiment is a hard disk device, a CD-ROM, an MO. , DVD, flexible disk, EEPROM, EPROM, etc., and can be stored in a device-readable recording medium for distribution, and can be transmitted via a network in a format that can be used by other devices.

10... Information presenting device 12... Display device 14... Motion capture device 16... Microphone 18... Speaker 20... Avata information distribution server 21... User authentication part 22... Avata information distribution part 23... Selection history management part 24... Avata information management part 26 ... database 27 ... voting history management unit 28 ... message history management unit 29 ... notification screen generation unit 30 ... video distribution server 40 ... user 50 ... network 60 ... video viewing screen 62 ... distribution video display area 64 ... avatar display area 65 ... pulldown Menu 66... Input form 67... Message notification screen 70... Condition setting screen 80... Notification screens 82, 84, 86... Area 90... Video search screen 92... Area 100... Information presentation system 102... Delivery video acquisition unit 103... Avatar information acquisition Unit 104... Delivery moving image display unit 105... Avatar display unit 106... Physical movement information acquisition unit 107... Avatar generation unit 108... Physical characteristic information acquisition unit 109... Physical activity calculation unit 110... Voice data acquisition unit 112... Voice output unit 114 ... comment acquisition unit 116... overlap determination unit 117... avatar display position changing unit 118... avatar information generation unit 120... voting reception unit 122... message reception unit 123... interactive information generation unit 500... user information management table 600... avatar selection history management Table 700... Voting history management table 800... Message history management table

Claims (25)

An information presentation system in which a plurality of information presentation devices and an information distribution server are connected via a network,
The information presentation device,
A distribution video display unit for displaying a distribution video distributed via the network in the first display area;
A physical motion information acquisition unit that acquires physical motion information related to the physical motion of the user who views the distribution video displayed in the first display area;
An avatar generation unit that generates a first avatar expressing the physical movement of the user based on the physical movement information;
An avatar display section for displaying the first avatar in a second display area,
An avatar information generation unit that generates avatar information including the physical motion information of the user who views the distribution video displayed in the first display area and transmits the avatar information to the information distribution server;
A physical characteristic information acquisition unit that acquires physical characteristic information relating to the physical characteristics of the user as attribute information of the user who views the distribution video displayed in the first display area;
Including
The information distribution server,
A database that stores avatar information including physical motion information related to physical motion when a user views a distribution video in the past,
An avatar information distribution unit that transmits the avatar information of another user who has watched the distribution video in the past, in response to a request from the information presentation device that displays the distribution video,
Including, save the avatar information received from the information presentation device in the database,
The avatar generation unit,
Generating a second avatar that reproduces the physical motion of the other user based on the physical motion information included in the avatar information received from the information distribution server,
The avatar display section,
Displaying the first avatar and the second avatar together in the second display area ,
Transmitting the avatar information including the physical movement information, the attribute information, and the information acquisition date and time to the information distribution server,
The information presentation device,
The upper limit number of the second avatars to be displayed and a search condition are specified to request the avatar information from the information distribution server, and the information distribution server is within the upper limit number that matches the specified search condition. An information presentation system for transmitting the avatar information related to a second avatar to the information presentation device . The body characteristic information includes height and body size of the user,
The information presentation system according to claim 1 . The information presentation device,
A physical activity calculation unit that calculates the activity of the body based on the physical movement information of the user as the attribute information of the user who views the distribution video displayed in the first display area.
The information presentation system according to claim 1 . The information presentation device,
A voice data acquisition unit that acquires voice data of voice uttered by the user as attribute information of the user who views the distribution video displayed in the first display area.
Information presentation system according to any one of claims 1 to 3. The information presentation device,
As attribute information of the user who views the distribution video displayed in the first display area, a comment acquisition unit that acquires a comment in which the utterance extracted from the voice data of the voice uttered by the user is converted into text is included.
The information presentation system according to any one of claims 1 to 4 . The information presentation device,
A voice output unit that outputs the voice data included in the avatar information received from the information delivery server in synchronization with a delivery video displayed in the first display area.
The information presentation system according to claim 1 . The voice output unit,
Outputting the voice data of the second avatar adjacent to the first avatar at a higher volume than the voice data of the other second avatars.
The information presentation system according to claim 6 . The avatar display section,
The comment included in the avatar information received from the information distribution server in synchronization with the distribution moving image displayed in the first display area is displayed in association with the corresponding second avatar of the other user,
The information presentation system according to any one of claims 1 to 7 . The avatar display section,
Display only the comment corresponding to the second avatar adjacent to the first avatar,
The information presentation system according to claim 8. The search condition is a condition for rearranging predetermined items, and the predetermined item includes any of the information acquisition date/time, the height, the size of the body, and the activity level of the body,
The information presentation system according to claim 1 . The information presentation device,
An overlap determination unit that determines whether or not the first display position of the first avatar and the second display position of the second avatar in the second display area overlap with each other,
An avatar display position changing unit that changes the second display position so that the first display position and the second display position do not overlap each other when it is determined that the first display position and the second display position overlap.
The information presentation system according to any one of claims 1 to 10 . The avatar generation unit,
Different drawing colors are assigned to the first avatar and the second avatar, and a predetermined transparency is set for the drawing color of the second avatar.
The information presentation system according to claim 11 . The avatar display section,
The second display area is divided by the total number of the first avatar and the second avatar, and the area located at the center of the plurality of divided areas is set as the display area of the first avatar. And set the remaining area as the display area of the second avatar,
Information presentation system according to any one of claims 1 to 12. The avatar generation unit,
Assigning different drawing colors to each of the first avatar and the second avatar,
The information presentation system according to claim 13 . The information presentation device,
A vote accepting unit that accepts a vote for the other user where the second avatar expresses its physical action via the second avatar displayed in the second display area,
The information presentation system according to any one of claims 1 to 14. The information presentation device,
A message accepting unit that accepts a message addressed to the other user where the second avatar expresses its physical action via the second avatar displayed in the second display area,
The information presentation system according to claim 15 . The information presentation device,
Generating interactive information including identification information of the other user where the second avatar selected by the user represents the physical movement, voting for the other user, and a message addressed to the other user, An interactive information generation unit for transmitting to the information distribution server,
The information presentation system according to claim 16 . The information distribution server,
A selection history management unit that manages a history of selection of the second avatar by the user,
The information presentation system according to claim 17 . The information distribution server,
A voting history manager that manages a history of votes for the other user where the second avatar expresses its physical activity;
The information presentation system according to claim 18 . The information distribution server,
The second avatar includes a message history management unit that manages a history of messages addressed to the other user who expresses the physical action.
The information presentation system according to claim 19 . The information distribution server,
A notification screen generation unit that generates a notification screen that displays the other user who has selected a user, the other user who has voted for the user, and the other user who has sent a message to the user, and which transmits the notification screen to the information presentation device ,
The information presentation system according to claim 20 . The notification screen generation unit,
A message notification screen for displaying a message addressed to the user by the other user is generated and transmitted to the information presentation device,
The information presentation system according to claim 21 . The message notification screen is
A dialog screen in which the message is overlay-displayed on a moving image capturing a display screen when the other user sends a message,
The information presentation system according to claim 22 . The avatar display position changing unit,
The same number of rectangular areas as the total number of the first avatars and the second avatars are defined, the rectangular area located in the center of the display screen is assigned as the display area of the first avatar, and the remaining left and right areas are defined. A rectangular area is assigned as the display area of the second avatar, and the first avatar is moved to the display area of the second avatar in synchronization with the user moving left and right with respect to the display screen.
The information presentation system according to any one of claims 11 to 23 . The avatar display position changing unit,
Moving said when a first avatar is moved to the display region of the second avatar, in response to a predetermined start condition is met, avatar of said second to follow the said first avatar To the driving state, and in response to the satisfaction of a predetermined termination condition, canceling the driving state.
The information presentation system according to any one of claims 11 to 24 .