JP7102457B2 - Content distribution systems, content distribution methods, and computer programs - Google Patents

Description

The present disclosure describes a content distribution system, a content distribution method, a computer program, a content distribution server, and a transmission path for distributing live contents such as public programs in which a photographed person who is an actual performer and a virtual character (avatar) participate. Regarding.

Conventionally, in terrestrial digital broadcasting, BS digital broadcasting, live Internet broadcasting, etc. (hereinafter, these are collectively referred to as "broadcasting"), programs in which a plurality of performers appear in the scene are often broadcast.

For example, in FIG. 7A schematically showing a scene of a broadcast program according to the prior art, the interviewee 101 to be interviewed faces the interviewer 102 to be interviewed at a position close to the interviewee 101. And the interviewer 102 are photographed by a camera (not shown) so that the interviewer 102 and the interviewer 102 fit into one scene, and an interview scene 100 is generated, which is broadcast and viewed by the viewer. If the camera gets closer to them or zooms in on the camera lens to a longer focal point, a bust shot 103 of the same interview is obtained, as seen in FIG. 7 (B).

In such an interview in broadcasting, the interviewee 101 and the interviewer 102 may not be in the same place, and in that case, FIG. 8 (A) schematically showing another scene of the broadcasting program according to the prior art. ), The scene where the interviewee 107 at the shooting site is photographed by the camera alone and only the interviewee 107 appears is shot, and the interviewer is in the studio of the broadcasting station and talks with the interviewee 107, for example. This interviewer does not appear in the broadcast scene 106, only the interviewer's spoken voice is broadcast.

Further, as shown in FIG. 8B, the interviewer 111 is also in a place different from the interviewee 110, for example, in the studio of a broadcasting station, but the interviewer 111 is photographed by a different camera, and the interviewee 110 is photographed. In some cases, the interview scene 108 is created by a picture-in-picture method in which the sub screen 109 is fitted inside the photographed main screen.

In this way, the scene where multiple performers appear in the broadcast is not limited to the interview, but for example, when a reporter or an announcer reports from the scene where the incident occurred, a scene where he talks with another announcer in the broadcasting station studio. , There can be many entertainment programs such as song programs and manga, dramas, and so on.

By the way, a game in which a virtual character (hereinafter also referred to as "avatar") created using CG (computer graphic) technology is drawn in a virtual three-dimensional space also created by CG to move or move. , Broadcasting, and content are being actively produced.

For example, in Patent Document 1 below, a participant who intends to move an avatar in a virtual space acquires the position information of the participant in the real space by using GPS (global positioning system) technology. Is disclosed to be converted into position information in the virtual space and used for the position operation in the virtual space of the avatar.

Japanese Patent No. 4637889

The virtual character (avatar) that is becoming popular in this way is not just drawn in the virtual space, but drawn in the real space to be photographed as a real performer. It is surprising and interesting that there is an artificial character that can not be found in a real scene by drawing on the same screen as the person (actor, performer, or passerby who is interviewed) and the avatar. Alternatively, a proposal has recently been made to make it possible to enjoy the fun of adding a sense of reality to an avatar that should be virtual and artificial.

However, in broadcasting or content in which an image in real space and an artificial avatar are combined and drawn in one scene in this way, the following essential problems have existed and are unsolved. rice field.

As described above, in each of the scenes shown in FIGS. 7 (A) and 7 (B), in which a plurality of actual performers are photographed as one scene, for example, the two performers 101, 102, 104, appearing in each scene. Since 105 visually recognizes each other and has a direct conversation, the conversation in the scene is naturally natural and comfortable, and is natural from the viewer's point of view.

However, if a real performer in the real space assumes a program that co-stars with an artificial avatar and fits in one scene, the avatar does not actually exist in front of the real performer, so it is real. The performer is not in front of the person with whom he is talking, and he is forced to act as if he were talking alone, and the direction in which the actual performer throws his eyes cannot be determined, making the entire scene unnatural. turn into.

Therefore, as shown in FIG. 8B, if the camera shoots only the actual performer and the avatar appears on the inset screen (picture in picture), it actually exists in the same place. Since the real performer and the avatar are not forcibly drawn on one screen, the real performer acts as if he were talking as if he had no one to talk to in front of him. Since it is not necessary, the unnaturalness disappears, but the scene is different from the one that draws an artificial avatar in the real space.

Therefore, as mentioned above, it is surprising, interesting, or virtual that there is an artificial character that cannot be found in a real scene, which occurs when a real performer and an avatar are drawn on the same screen. There was a risk that viewers would not be able to fully enjoy the fun of adding a sense of reality to avatars that should be artificial.

Therefore, in this disclosure, a content distribution system, a content distribution method, a computer program, and content for eliminating the unnaturalness in a scene in which a real performer and an artificial (virtual) character, an avatar, appear together. The purpose is to provide a distribution server and a transmission line.

In order to solve the above problems, the present invention provides the content distribution system, content distribution method, computer program, content distribution server, and transmission path described in 1) to 11) below.
1)
It is a content distribution system that displays and distributes the photographed person in one scene together with a virtual character.
A camera unit for scene shooting that shoots so that the subject and the dummy object fit in one scene,
An image replacement unit that replaces the image of a dummy object in the shot scene with an image of a virtual character and delivers it.
A content distribution system characterized by being equipped with.
2)
The content distribution system according to 1), wherein the dummy object includes a display for displaying an image of a virtual character.
3)
The image of the virtual character that has been replaced in the image delivered by the image replacement unit is characterized in that the image of the virtual character that has been replaced is different from the image of the virtual character displayed on the display included in the dummy object 2). Described content distribution system.
4)
The content distribution system according to 3), wherein the different aspects are based on different viewpoints.
5)
The dummy object is characterized by having a camera that shoots the photographed person from a position different from the scene shooting camera unit and / or a microphone that picks up the sound of the photographed person, according to any one of 1) to 4). Content distribution system described in.
6)
It is characterized by further having an operation unit for changing the appearance display mode of the virtual character and an operation unit for operating the appearance display mode of the virtual character that displays information collected by the camera and / or the microphone. ) Described in the content distribution system.
7)
The operation unit is configured to change the appearance display mode of the virtual character displayed on the display and / or the appearance display mode of the image of the virtual character in the image delivered by the image replacement unit. The content distribution system according to 6), which is a feature.
8)
The broadcast transmission unit that broadcasts the content including the distribution image of the image replacement unit via the communication path, and the reception unit that receives the user-posted information to be displayed on the screen of the content being broadcast, and the user-posted information is The content distribution system according to any one of 2) to 7), wherein the display is also configured to display an image of a virtual character.
9)
The broadcast transmission unit that broadcasts the content including the distribution image of the image replacement unit via the communication path, and the reception unit that receives the user-posted information to be displayed on the screen of the content being broadcast, and the user-posted information The content distribution system according to any one of 6) to 8), characterized in that the display unit for operating the appearance display mode of the virtual character is also displayed.
10)
It is a content distribution method that displays and distributes the photographed person in one scene together with a virtual character.
A step in which the camera unit for scene shooting shoots so that the subject and the dummy object fit in one scene.
The step in which the image replacement unit replaces the image of the dummy object in the shot scene with the image of the virtual character and delivers it.
A content distribution method characterized by being equipped with.
11)
A computer program comprising a code group in which a computer executes each step in the content distribution method according to claim 10.
12)
A content distribution server that displays and distributes the photographed person in one scene together with a virtual character.
Content characterized by being configured to replace the image of a dummy object in a scene shot so that the photographed person and the dummy object fit into one scene with an image of a virtual character and deliver it. Delivery server.
13)
A transmission line for transmitting the signal delivered by the content distribution server according to 12).

According to the present disclosure, it is possible to eliminate the unnaturalness in a scene in which the photographed person and the avatar, which is an artificial (virtual) character, appear together.

It is a figure which showed typically the structure of the content distribution reception system which concerns on this invention. It is a block diagram of the content distribution server which the content distribution reception system of FIG. 1 has. It is a block diagram of the avatar change unit which the content distribution reception system of FIG. 1 has. FIG. 5 is a flowchart showing a process in which the content distribution / reception system of FIG. 1 transmits and displays an image in which a photographed person and an avatar are drawn. FIG. 5 is a flowchart showing a process in which the content distribution / reception system of FIG. 1 draws an avatar on a display for viewing by a photographed person. (A) is a schematic view of an image taken by a scene shooting camera included in the content distribution / reception system of FIG. 1. (B) It is a schematic diagram which shows the image of the avatar generated by the avatar change unit which the content distribution reception system of FIG. 1 has. (C) It is a schematic diagram of the scene in which the photographed person and the avatar are stored on one screen generated by the content distribution server included in the content distribution reception system of FIG. 1. It is a schematic diagram of the scene in the broadcasting which concerns on the prior art. It is a schematic diagram of the scene in the broadcasting which concerns on the prior art.

Hereinafter, embodiments according to the present disclosure will be described. It should be noted that the embodiments described below are merely examples, and various modifications can be made without departing from the spirit of the present invention, which are also included in the present invention.

[Configuration of content distribution / reception system 1]
FIG. 1 is a diagram schematically showing a configuration of a content distribution / reception system 1 according to the present invention. The content distribution / reception system 1 is an avatar operation equipped with a motion capture system or the like for the avatar operator to operate the studio where the actual performer is photographed, the performer filming site 2 which is a broadcast relay site, or the movement of the avatar. The following configurations are arranged in the studio 3, the distribution environment 4 of the content distributor that broadcasts the program, and the viewing environment 5 of the content viewer who is the end user who watches the broadcast program.

The above-mentioned "real performer" refers to an actual photographed person who is not a virtual character (avatar), and includes not only the performer who performs the acting but also a passing interview in a street interview, for example. ..

It is not essential for the practice of the present invention that the configurations are arranged in different places in this way, and any or all of the configurations may be arranged in the same place.

A performer 20 is present at the performer shooting site 2 to perform an act for a program, and a performer display stand unit 21 is arranged in the vicinity of the performer. The speaker display stand unit 21 includes a speaker shooting camera 21a for photographing the performer 20 and making it visible to the avatar operator 30, and picking up the utterances of the performer 20 and using them for listening to the avatar operator 30. Alternatively, the performer uses the performer's utterance microphone 21b for use as the voice of the program, the speaker for listening to the performer 21c for uttering the utterance voice emitted by the avatar 30, or the command voice from the broadcast director (not shown), and the image 21e of the avatar. The performer visual display 21d, which provides an environment in which the performer 20 visually recognizes and as if the performer 20 faces an artificial avatar, and the stand 21f that supports each configuration of these performer display stand units 21 are provided. I have.

Further, a scene shooting camera 22 is arranged at the performer shooting site 2, and is used for shooting so that the performer 20 and the performer display stand unit 21 fit on one screen (scene shooting camera shooting zone 23).

Each signal system of the performer display stand unit 21 and the scene shooting camera 22 is data-connected to the content distribution server 40 via the Internet communication networks 60, 61, 62, 66.

The description and illustration of the connection nodes between the Internet lines 60, 61, 62, and 66 will be omitted. In carrying out the present invention, the transmission of signals and data described in each place can use various lines and communication networks, and is not limited to the Internet lines 60, 61, 62, 66. This point is the same in the following description.

As described above, the avatar operation studio 3 is a place for the avatar operator 30 to operate the movement of the avatar using the motion capture technology. As an example in FIG. 1, the optical marker 31 is the body of the avatar operator 30. The motion capture camera 32 captures each of these optical markers 31, and the avatar change unit 34, which will be described later, detects the position coordinates of each optical marker 31 in three-dimensional space using the principle of triangular distance measurement. is doing.

As a motion capture technology, in addition to the combination of the above optical markers and the corresponding camera, a method of detecting the movement of the avatar operator 30 using an acceleration sensor and a high-circumferential electromagnetic wave are applied to the avatar operator. It is possible to use various methods such as a method of detecting the movement of the body of the avatar operator 30 by the Doppler phenomenon from the analysis result of the electromagnetic wave reflected and returned by irradiating the 30.

A display stand unit 33 for the avatar operator is arranged in the avatar operation studio 3, and the unit 33 blinks as necessary to notify the avatar operator 30 that the avatar is currently on the air. Tally lamp 33a, avatar operator shooting microphone 33b for picking up the avatar operator's voice, avatar operator listening speaker 33c for uttering the voice of the performer 20 or the program director (not shown), performer Each configuration of the avatar operator visual display 33d and the avatar operator display stand unit 33 that displays the image 33e of the above and provides an environment as if talking with the performer 20 in the distance in front of the eyes. Each of the stands 33f is provided to support the above. A camera (not shown) that captures the facial expression of the avatar operator 30 is provided near or at another position of the tally lamp 33a, and the facial expression of the avatar operator 30 is used when rendering the avatar. It may be configured as follows.

Each signal system of the motion capture camera 32 and the avatar operator display stand unit 33 has a data connection with the avatar change unit 34 via the Internet communication networks 63 and 64. The configuration of the avatar change unit 34 will be described separately. The avatar change unit 34 has a data connection with the content distribution server 40 via the Internet communication networks 60, 65, 66.

The content distribution server 40 may include a part or all of the above-mentioned avatar change unit 34. In that case, the part described as the configuration, function, and process of the avatar change unit 34 is the configuration, function, and process of the content distribution server 40.

Further, a content distribution server 40 is arranged in the distribution environment 4 of the content distribution company, and its configuration will be described separately.

A general-purpose personal computer 50 is provided in the viewing environment 5 of the content viewer, and the content viewer (not shown) can enjoy the program drawn on the personal computer display 50a. As will be described later, the screen of the program is configured so that the image 50b of the performer and the image 50c of the avatar are drawn together, and as a result, the content viewer has the same screen as the actual performer and the avatar. By drawing on the avatar, it is surprising and interesting that there is an artificial character that can not be found in the real scene, or it is interesting to add a real feeling to the avatar that should be virtual and artificial. You can fully enjoy it.

[Structure of avatar change unit 34]
As shown in the block diagram of FIG. 3, the avatar change unit 34 is a control unit 340, which is a CPU that controls and controls the entire unit 34, a memory 341, and a motion tracker that digitizes the movement of the avatar operator 30 by motion capture technology. 343, Avatar data in memory 341 Data such as the appearance of the avatar (image data) stored in the memory 341b and the movement data of the avatar operator 30 generated by the motion tracker 343 are used to obtain the current avatar posture. It is provided with a rendering unit 344 that generates a corresponding CG image, a communication interface 345 that performs communication transmission / reception inside and outside the unit 34, and the like, and each of these configurations is connected to each other via a data bus line 342.

Further, the program memory 341a included in the memory 341 is executed by the avatar change unit 34, which includes an execution code group for the control unit 340 and others to execute each operation including the operation disclosed in the embodiment described in the present specification. The computer program to be stored is readable, and the temporary memory 341c is also used as a so-called cache memory.

[Configuration of content distribution server 40]
As shown in the block diagram of FIG. 2, the content distribution server 40 is for a performer photographed by a control unit 41, a memory 42, and a scene shooting camera 22 which are CPUs that control and control the entire server 40. In order to identify which area the image of the display stand unit 21 is, the image of the display stand unit 21 for performers stored in the template image memory 42b in the memory 42 is used as a template (pattern) for the scene shooting camera. The above-mentioned identification of the image of the current avatar sent from the template image tracker 44 and the avatar change unit 34, which 22 takes a picture and identifies the area using a technique such as a matching method (statistical correlation matching) for the image. A content image generation unit 45 that generates a broadcast content image in which the performer 20 and the avatar are drawn together by superimposing or replacing the area, and a communication interface 46 that transmits / receives data between the inside and outside of the server 40. Each of these configurations is connected to each other via a data bus line 43.

Further, the program memory 42a included in the memory 42 is a computer to be executed by the server 40, which includes an execution code group for the control unit 41 and others to execute each operation including the operations disclosed in the embodiments described in the present specification. The program is readable and stored, and the temporary memory 42c is also used as a so-called cache memory.

Further, as will be described later, the content distribution server 40 is a server that displays and distributes the photographed person together with the virtual character (avatar) in one scene, and the photographed person and the dummy object are combined into one scene. The image of the dummy object in the scene shot so as to fit is replaced with the image of the virtual character and distributed. Further, the signal distributed by the content distribution server 40 is carried by the Internet communication networks 60, 66, 67 and the like, which are transmission lines, and is transmitted via each of these transmission lines.

[Generation and display processing of content image 73]
Next, an image 73 in which the performer 20 and the avatar are drawn together, which should be used for the content (broadcast) (hereinafter, also referred to as a “content image”, is a schematic diagram of FIG. 6 (C), or a personal computer in FIG. 1). An example thereof is also shown in the computer display 50a), and the process until the viewing user (not shown) can view the content is explained with reference to the flowchart of FIG. 4 and the schematic diagram of FIG. conduct.

First, the control unit 340 of the avatar change unit 34 reads out the avatar data (image data representing the appearance of the virtual character, etc.) to be drawn in the content image stored in the avatar data memory 341b in the memory 341 (step S101). ). Next, the control unit 340 controls and controls the motion tracker 343, obtains the movement of the avatar operator 30 by the motion capture technique described above, and converts it into data (step S102).

Next, the control unit 340 causes the rendering unit 344 to perform rendering to generate a 3D drawing image of the avatar using the obtained motion capture data and the avatar data (step S103). Since it is important from which position the image of the avatar is generated when rendering, the rendering unit 344 receives information on the position of this viewpoint from the content distribution server 40 in advance, specifically, the performer shooting site in FIG. Information on the position of the scene shooting camera 22 in 2, and as will be described later, information on the relative position of the performer display stand unit 21, which is an object to be replaced with the image of the avatar, is stored. ing. From this point of view, there are no restrictions on the direction in which the avatar is rendered, but if the scene is a scene in which the performer 20 and the avatar are facing each other and performing an interview, the avatar is viewed from the side. Rendering the scene is a good choice.

In addition, it is possible to make another aspect having a configuration partially different from the aspect described above, and in one alternative aspect, the avatar change unit 34 performs only the operation related to the motion capture, and the result is obtained. The data may be transmitted to the content distribution server 40, and the content distribution server 40 may render the avatar image described above. In this case, the configuration related to rendering is included in the content distribution server 40 instead of being included in the avatar change unit 34 in the above description.

Further, a part or all of the configurations arranged in the avatar operation studio 3 described above may be arranged in the distribution environment 4 of the content distributor or another place to have each function.

Since the differences in the configuration and the differences in the processing in such another aspect can be easily understood from the various parts of the present specification, the description thereof will be omitted.

The image data of the avatar obtained by rendering is transmitted to the content distribution server 40 via the communication interface 345 and the Internet communication networks 60, 65, 66 (step S104).

The content distribution server 40 receives the transmitted avatar image data (step S204) and executes the following operations in advance.

First, an image of an object to be replaced with an avatar image later, and an image taken by the scene shooting camera 22 together with the performer 20 (the performer display stand unit 21 in FIG. 6A) is a template (pattern). ) The image is stored in the template image memory 42b of the content distribution server 40 in advance, and the control unit 41 orders the template image tracker 44 to read the template image (step S201).

Further, in response to the control control of the control unit 41, the image 70 (hereinafter, also referred to as “scene shooting image”) obtained by the scene shooting camera 22 and the performer 20 and the performer display stand unit 21 is captured by the Internet communication networks 60 and 62. , 66, reception is performed via the communication interface 46 (step S202).

Next, according to the control control of the control unit 41, the content image generation unit 45 specifies the position where the template image, that is, the performer display stand unit 21 is drawn in the received scene captured image 70 (step S203). .. In specifying, there is a method such as selecting the position having the highest correlation by a matching method that performs correlation matching with the template image while shifting the position little by little with respect to the scene captured image 70, and other methods may be used.

For example, the position on the screen on which the image 72 of the avatar should be drawn may be determined according to the operation of an operator (not shown).

Next, the content image generation unit 45 transfers the template image, that is, the image of the performer display stand unit 21 at the position of the specified template image, that is, the performer display stand unit 21, with the avatar image 72 transmitted in step S204. Replace (step S205). By the replacement, a content image 73 in which the performer 20 and the avatar image 72 are drawn together on the screen is generated, so that the content image 73 is a personal computer via the communication interface 46 and the Internet communication networks 60, 66, 67. Transmission is made to 50 (step S206). That is, the Internet communication networks 60, 66, and 67 are in a scene in which the performer 20 who is the photographed person and the dummy object, which are signals or information generated by the content distribution server 40, are photographed so as to fit in one scene. The personal computer 50, which is a receiving terminal that replaces the image of the dummy object in the above with the image of the virtual character (avatar), transmits or carries the distributed signal or information, and will be described later. Will be supplied to.

As in the above aspect, the image of the dummy object (in the example, the display stand unit 21 for the performer) to be replaced with the avatar image is stored as a template in advance, and the drawing position of the dummy object is set by the server 40 by statistical processing. In addition to the mode in which the avatar is judged, the method in which the operator may manually specify the drawing position of the avatar on the screen has been described above.

On the other hand, the personal computer 50 used by the browsing user (not shown) displays the content image 73 transmitted by the content distribution server 40 on the personal computer display 50a (step S302).

As shown in the flowchart of FIG. 4, each process described above is repeated unless the end operation is detected in the determination steps S105, S207, and S303 waiting for the end operation, so that the content image 73 is repeatedly updated. As a result, the viewing user can view the moving image drawn on the screen together with the performer 20 and the image 73 of the avatar. At this time, the performer 20 looks at the display stand unit 21 for the performer, and the scene in which the performer is talking or looking at the person as if he / she is a real interlocutor is the scene shot image 70 before the replacement. The browsing user who visually recognizes the later content image 73 can get the feeling that the performer 20 is really talking with the avatar, and as a result, according to the present embodiment, with the actual performer. It is possible to eliminate the unnaturalness in the scene where an artificial (virtual) character, an avatar, appears together.

Alternatively, even in drawing in a mode in which the performer 20 does not directly face the avatar, the performer 20 can perform a conversation or act as if the avatar exists at the position of the performer visual display 21d, and such a mode. Even in, it is possible to eliminate the unnaturalness in the scene where the real performer and the artificial (virtual) character avatar appear together.

Therefore, the real performer and the avatar should be drawn on the same screen, and it should be surprising and interesting that there is an artificial character that is impossible in the real scene, or it should be virtual and artificial. Browsing users who are viewers can fully enjoy the fun of adding a sense of reality to avatars.

[Processing of avatar image 21e display on performer visual display 21d]
The above description has focused on the process of drawing the avatar image 50c on the display screen of the personal computer display 50a, which is browsed by the browsing user (not shown).

Next, the process of displaying the image 21e of the avatar on the display 21d for viewing the performer, which the performer 20 browses, will be described without the main purpose of browsing by the browsing user.

It has been described above that in the generated content image 73, drawing as if the performer 20 faces, faces, and looks at the avatar is preferable in order to achieve the effect of the present invention. ..

Therefore, if a schematic front image in which the avatar faces the performer 20 is drawn on the performer visual display 21d provided in the performer display stand unit 21 facing the performer 20, the performer 20 is really relative to the avatar. As a result, the browsing user can also have a realistic feeling when viewing the content image 73.

Therefore, as shown in the flowchart of FIG. 5, in the avatar change unit 34, first, the control unit 340 of the avatar change unit 34 is the avatar that is stored in the avatar data memory 341b in the memory 341 and wants to be drawn on the content image. Data (image data representing the appearance of the virtual character, etc.) is read out (step S401). Next, the control unit 340 controls and controls the motion tracker 343, obtains the movement of the avatar operator 30 by the motion capture technique described above, and converts it into data (step S402).

Next, the control unit 340 causes the rendering unit 344 to perform rendering to generate a 3D drawing image of the avatar using the obtained motion capture data and the avatar data (step S403). Since it is important from which position the image of the avatar is generated when rendering, the rendering unit 34 receives information on the position of this viewpoint from the content distribution server 40 in advance, specifically, the performer shooting site in FIG. Information on the position of the scene shooting camera 22 in 2, and as will be described later, information on the relative position of the performer display stand unit 21 which is an object to be replaced with the image of the avatar is used. .. From this point of view, there are no restrictions on which direction the avatar is facing, but if the scene is a scene in which the performer 20 and the avatar are facing each other and performing an interview, the avatar is viewed from the front. Rendering the scene is a good choice.

The image data of the avatar obtained by rendering is transmitted to the content distribution server 40 via the communication interface 345 and the Internet communication networks 60, 65, 66 (steps S404, S501).

The content distribution server 40 processes the transmitted avatar image as it is or adds an appropriate background, and sends it to the performer visual display 21d via the Internet communication networks 60, 61, 66 for drawing (step). S502).

As shown in the flowchart of FIG. 5, each process described above is repeated unless the end operation is detected in the determination steps S405 and S503 waiting for the end operation. Therefore, the image of the avatar drawn on the performer visual display 21d. Will be updated repeatedly, and as a result, the performer 20 will be able to view the moving image of the avatar image.

At this time, the performer 20 looks at the display stand unit 21 for the performer, and the scene in which the performer talks or looks at the scene as if he / she is a real interlocutor is the scene shot image before the replacement explained above. Since it is 70, the browsing user who visually recognizes the content image 73 after the replacement can get a feeling as if the performer 20 is really talking with the avatar, and as a result, according to the present embodiment. For example, it is possible to eliminate the unnaturalness in a scene in which a real performer and an artificial (virtual) character, an avatar, appear together.

Alternatively, even in drawing in a mode in which the performer 20 does not directly face the avatar, the performer 20 can perform a conversation or act as if the avatar exists at the position of the performer visual display 21d, and such a mode. Even in, it is possible to eliminate the unnaturalness in the scene where the real performer and the artificial (virtual) character avatar appear together.

Therefore, the real performer and the avatar should be drawn on the same screen, and it should be surprising and interesting that there is an artificial character that is impossible in the real scene, or it should be virtual and artificial. Browsing users who are viewers can fully enjoy the fun of adding a sense of reality to avatars.

In addition to the avatar image 21e as in the above embodiment, the performer visual display 21d may include instructional comments (so-called “canning paper”, “campe”) created for the photographed person to read, and a program. Images (not shown) and voices of staff such as the director of the company may be displayed or uttered by the attached speaker.

[Display user posted information]
In addition to the above-described embodiment, or in addition to this configuration, the following configuration may be used.

As a way for viewers (not shown) to enjoy content that is broadcast live using the Internet communication network, etc., the viewer's personal impressions and tweets of the viewer (hereinafter, also referred to as "comments") are personalized. If the input operation is performed by the computer 50 and transmitted, and these input comments are displayed in real time on a terminal used by another viewer (not shown), the viewer who wrote the comment can send his / her own remarks to other viewers. Viewers who read comments written by others can not only browse and enjoy the relevant content, but also know the thoughts and impressions of other viewers, so it is as if they were at one venue. You can get a sense of pseudo (virtual) simultaneous participation and a sense of solidarity, like enjoying a concert or event.

At that time, in this system 1, the comment posted by any of the viewers is displayed on at least one of the performer visual display 21d and the avatar operator visual display 33d via the content distribution server 40. Can be configured to be. As a result, the performer 20 or the avatar operator 30 who has read the comment can obtain the above-mentioned sense of simultaneous participation and a sense of solidarity, and can perform more realistic performances, responses, and operate the avatar.

[Handling of audio in the content distribution / reception system 1]
As described above, the content distribution / reception system 1 according to the present embodiment can generate a content image 73 as if the performer 20 and the avatar exist together in the real space. .. In order to increase the sense of realism, it is a natural idea to add the utterance voice of the performer 20 and the voice of the avatar, for example, the voice of the avatar operator 30 to the broadcast, together with the content image 73. Yes, and it's an important point. Therefore, the system 1 has a speaker utterance microphone 21b for picking up the utterance of the performer, an avatar operator shooting microphone 33b for picking up the utterance of the avatar operator 30, and the utterance of the performer 20. The avatar operator 30 is provided with an avatar operator listening speaker 33c for listening to the utterance, and a performer listening speaker 21c for the performer 20 to listen to the utterance of the avatar operator 30. Since the signal transmission is clear from the embodiments related to the transmission of each moving image described above, repeated explanations will be avoided.

Since this embodiment relates to broadcast content, in the case where this broadcast is a live broadcast, the voice of the performer 20 and the voice of the avatar replaced with the voice of the avatar operator 30, for example, are the voices of the performer 20. Since it can be produced by a general-purpose personal computer for viewing live broadcasting, a speaker attached to or connected to the general-purpose personal computer may be used instead of the speakers 21c and 33c described above.

Similarly, if the image 33e of the performer 20 taken by the performer shooting camera 21a is drawn on the avatar operator viewing display 33d, the avatar operator 30 also faces the performer 20 in front of him. Since the avatar can be operated as if it were, the effect of the invention can be further improved.

[Modification example]
The content distribution / reception system 1 according to the present embodiment described above is not limited to the configuration described above, and can be modified in various ways, all of which are included in the present invention.

For example, the performer visual display 21d may be omitted, and the performer 20 may listen to only the voice of the avatar operator 30 and have a dialogue while not visually recognizing the image of the avatar operator 30. The system has a simpler configuration, which reduces costs and improves reliability.

Alternatively, instead of using the display stand unit 21 for the performer, a mere figurine or a stuffed animal, that is, a dummy object is arranged, the performer 20 performs a dialogue operation, and the dummy object is subjected to the same processing as the replacement described above. The display portion may be replaced with an avatar image to generate a content image 73 for viewing. In this case, if it is a live broadcast, the performer 20 can recognize the movement and speech of the avatar by the image and sound of the live broadcast, so that the avatar and the performer 20 can view the broadcast drawn on the same screen in the same natural way. It can be enjoyed by the user, and such a system also has a simpler configuration, which reduces costs and improves reliability. In this way, the performer 20 plays while watching a personal computer (not shown) that transmits live video and audio, and the personal computer is placed near a dummy object (such as a figurine or a stuffed animal). , It may be placed in a remote place, and this personal computer may or may not be included in the scene shooting camera shooting zone 23. Further, the personal computer itself may be used as a dummy object whose drawing is replaced by the image of the avatar.

(Explanation of the effect of the invention)
In content such as broadcasting, the actual performer and avatar are drawn on the same screen, and it is surprising and interesting that there is an artificial character that is impossible in the real scene, or it is virtual and artificial. Viewers can fully enjoy the fun of adding a sense of realism to the supposed avatar, and at the same time, it is unnatural in a scene where a real performer and an artificial (virtual) character avatar appear together. The present invention provides a content distribution system, a content distribution method, a computer program, a content distribution server, and a transmission path that can solve the problem.
The present invention can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof shall be included in the scope of the invention described in the claims and the equivalent scope thereof, as well as in the scope and gist of the invention.

1 Content distribution reception system 2 Performer shooting site 3 Avatar operation studio 4 Content distributor 5 Content viewer's house 20 Performer 21 Performer display Stand unit 21a Performer shooting camera 21b Performer speaking microphone 21c Performer listening speaker 21d Performer viewing display 21e Avatar image 21f Stand 22 Scene shooting camera 23 Scene shooting camera Shooting zone 30 Avatar operator 31 Motion capture optical marker 32 Motion capture camera 33 Avatar operator display Stand unit 33a Tally lamp 33b Avatar operation Person shooting microphone 33c Avatar operator listening speaker 33d Avatar operator visual display 33e Performer image 33f Stand 34 Avatar change unit 340 Control unit 341 Memory 341a Program memory 341b Avatar data memory 341c Temporary memory 342 Data bus line 343 Motion tracker 344 Rendering unit 345 Communication interface 40 Content distribution server 41 Control unit 42 Memory 42a Program memory 42b Template image memory 42c Temporary memory 43 Data bus line 44 Template image tracker 45 Content image generator 46 Communication interface 50 Personal computer 50a Personal computer display 50b Image of performer in scene 50c Image of avatar in scene 60, 61, 62, 63, 64, 65, 66, 67 Internet communication network

Claims (11)

It is a content distribution system that displays and distributes the photographed person in one scene together with a virtual character.
A scene shooting camera unit that shoots the photographed person and a dummy object having a display for displaying an image of the virtual character so as to fit in one scene.
An image generation unit that generates an image in which an image of the virtual character is superimposed on an image of the dummy object in the shot scene.
With
The superimposed image of the virtual character in the image generated by the image generation unit is different from the image of the virtual character displayed on the display included in the dummy object, and is different from the image of the virtual character displayed on the display of the dummy object. A content distribution system that distributes images generated by . The content distribution system according to claim 1 , wherein the different aspects are based on different viewpoints . A broadcast transmission unit that broadcasts content including an image generated by the image generation unit via a communication path, and
It is further provided with a receiving unit that receives user-posted information to be displayed on the screen of the content being broadcast.
The content distribution system according to claim 1 or 2, wherein the user-submitted information is also displayed on a display displaying an image of the virtual character . It is a content distribution system that displays and distributes the photographed person in one scene together with a virtual character.
A scene shooting camera unit that shoots the photographed person and a dummy object having a display for displaying an image of the virtual character so as to fit in one scene.
An image generation unit that generates an image in which an image of the virtual character is superimposed on an image of the dummy object in the shot scene.
With
The dummy object has a camera that shoots the photographed person from a position different from that of the scene shooting camera unit and / or a microphone that picks up the sound of the photographed person, and distributes an image generated by the image generating unit. Content distribution system. The claim further comprises an operation unit for changing the appearance display mode of the virtual character, and a display unit for operating the appearance display mode of the virtual character that utters information collected by the camera and / or the microphone. The content distribution system according to 4 . The operation unit is configured to change the appearance display mode of the virtual character displayed on the display and / or the appearance display mode of the image of the virtual character in the image generated by the image generation unit. The content distribution system according to claim 5. A broadcast transmission unit that broadcasts content including an image generated by the image generation unit via a communication path, and
It is further provided with a receiving unit that receives user-posted information to be displayed on the screen of the content being broadcast.
The content distribution system according to any one of claims 4 to 6, wherein the user-posted information is also displayed on the display that displays an image of the virtual character . A broadcast transmission unit that broadcasts content including an image generated by the image generation unit via a communication path, and
It is further provided with a receiving unit that receives user-posted information to be displayed on the screen of the content being broadcast .
The content distribution system according to claim 5 or 6, wherein the user-posted information is also displayed on a display unit for operating the appearance display mode of the virtual character. It is a content distribution method that displays and distributes the photographed person in one scene together with a virtual character.
A step in which the camera unit for scene shooting takes a picture so that the person to be photographed and a dummy object having a display for displaying an image of the virtual character are contained in one scene.
A step in which the image generation unit generates an image in which the image of the virtual character is superimposed on the image of the dummy object in the scene in which the shooting was performed.
With
The superimposed image of the virtual character in the image generated by the image generation unit is different from the image of the virtual character displayed on the display included in the dummy object, and is different from the image of the virtual character displayed on the display of the dummy object. A content distribution method that distributes the images generated by . The content distribution method according to claim 9 , wherein the different aspects are based on different viewpoints . A computer program comprising a set of codes in which a computer executes each step in the content distribution method according to claim 9 or 10.

Images