JP7029189B2 - Viewpoint confirmation system - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Law 1. Act 1 Release date September 12, 1st Reiwa Release location Tokyo Game Show 2019 Makuhari Messe 2. Act 2 Distribution date September 12-15, 1st year Reiwa Distribution place Tokyo Game Show 2019 Makuhari Messe 3. Act 3 Website publication date September 12, 1st Reiwa Website URL https: // www. konami. com / ginza / https: // www. konami. com / ginza / school / 4. Act 4 Release date (a) Reiwa September 28, 29, October 26, 27, November 12, 14, 19, 21, 21, 26, 28, December 10 Sunday, 12th (b) December 23rd, 25th, 26th, Reiwa 1st January 11th, 12th, 26th, February 8th, 11th, 12th, 16th , 22nd, 23rd, 24th, 29th, March 1st Public place (a) Tokyo Midtown (b) Konami Creative Center Ginza 5. Act 5 Distribution date December 13, 1st Reiwa Distribution location Each customer store in Tokyo, Kanagawa, Chiba, and Saitama prefectures 6. Act 6 Release date January 11th and 12th, 2nd year of Reiwa Publication place TOKYO eSports Festa Tokyo Big Sight 7. Act 7 Distribution date January 11th and 12th, 2nd year of Reiwa Distribution place TOKYO eSports Festa Tokyo Big Sight 8. Act 8 Release date January 12, 2nd year Reiwa Place Release location Konami Creative Center Ginza 9. Act 9 Website publication date (a) Reiwa January 14, 2 (b) Reiwa January 18, 2 (c) Reiwa January 21, 2 Website URL (a) https: / / Www. 4gamer. net / games / 465 / G046503 / 202001140886 / (b) https: // hochi. news / articles / 20200117-OHT1T50149. html https: // hochi. news / articles / 202001117-OHT1T50145. html (c) html: // gamebiz. jp /? p = 257993 10. Act 10 Publication date January 17, 2nd year Reiwa Publication Newspaper (Sports Hochi) Dated January 17th, 2nd year, 12th page 11. Act 11 Publication date (a) February 12, 2nd year of Reiwa (b) February 12th, 2nd year of Reiwa (c) February 14th, 2nd year of Reiwa (d) February 15th, 2nd year of Reiwa (e) ) February 16, 2nd year of Reiwa

Application of Article 30, Paragraph 2 of the Patent Act (f) February 28, 2nd year of Reiwa (g) February 29, 2nd year of Reiwa Publication place Konami Creative Center Ginza 12. Act 12 Date of publication on the website (a) February 16, 2nd year of Reiwa (b) 27th February, 2nd year of Reiwa (c) 28th February, 2nd year of Reiwa (2) 28th February, 2nd year of Reiwa Sun (e) February 29, 2nd year of Reiwa (f) March 9th, 2nd year of Reiwa (g) March 9th, 2nd year of Reiwa URL to the website (a) https: // e-sports- today. com / news / 2519 / (b) https: // www. youtube. com / watch? v = a379Ej4866k (c) https: // www. Famitsu. com / news / 20102/28193400. html (2) html: // weekly. ASCII. jp / elem / 000/000/440/440301 / (e) https: // www. youtube. com / watch? v = PncTPwIw2V0 (f) https: // www. youtube. com / watch? v = wrMZhF20J_w (G) https: // www. youtube. com / watch? v = vinPeX_l8io 13. Act 13 Publication date March 5, 2020 Publication (a) Weekly Famitsu March 19, 2020 P143 (b) Weekly Famitsu March 19, 2020 P168

The present invention relates to a viewpoint confirmation system.

There is a viewpoint detection (line-of-sight detection) technology (so-called eye tracking) that detects the position of the viewpoint of a person who is looking at the screen. For example, by detecting the viewpoint of the player while playing the game and displaying the detection result on the game image in real time, it is possible to confirm where the player is watching while playing (for example, Non-Patent Document 1). ).

[Special project] "Large dissection of the eyes of the strong men! Search for the secret of strength with" DFF "eye tracking", [online], [Search on December 13, 2019], Internet <URL: https: // www .. Famitsu. com / matome / dff / diary54. html>

However, if the viewpoint detection result of the viewpoint detection target person (player, etc.) is displayed in real time on the game image viewed by the target person during game play, the viewpoint being played by the target person can be confirmed. On the other hand, since it becomes difficult for the target person to check the game play status, the target person's activity (game play, etc.) may be hindered.

It is one of the objects of the present invention to provide a viewpoint confirmation system capable of confirming the viewpoint of the subject during the activity without interfering with the activity of the subject.

Another object of the present invention is to provide a viewpoint confirmation system capable of exerting the effects described in the embodiments described later.

In order to solve the above-mentioned problems, one aspect of the present invention is a first display device on which a game image output from the game device is displayed, and a viewpoint position of a target person facing the display screen of the first display device. A viewpoint confirmation system including a viewpoint detection device for detecting, wherein the first video for inputting the output game video to at least the first display device and a second video whose input destination is different from the first video A distribution device that distributes to the video, a generator that generates composite video information by adding viewpoint information indicating the viewpoint position detected by the viewpoint detection device to the video information based on the second video, and the composite video information. A video process having a video output unit that outputs a synthetic game video based on the above, an information including video information based on the second video and the viewpoint information, or a storage control unit that stores the synthetic video information in a storage unit. It is a viewpoint confirmation system equipped with a device.

Further, one aspect of the present invention is a first display device for displaying a game image output from the game device, and a viewpoint detection device for detecting the viewpoint position of the target person facing the display screen of the first display device. A viewpoint confirmation system comprising: a distribution device that distributes the output game video to at least a first video for inputting to the first display device and a second video having an input destination different from that of the first video. And, a generation unit that generates synthetic video information by adding viewpoint information indicating the viewpoint position detected by the viewpoint detection device to the video information based on the second video, and outputs a synthetic game video based on the synthetic video information. A video processing device including a video output unit, and a third display device different from the first display device, wherein the first display device displays the first video and the third display device. Is a viewpoint confirmation system that displays the synthetic game image.

The schematic diagram which shows the outline of the viewpoint confirmation system which concerns on 1st Embodiment. The figure which shows the use example of the viewpoint confirmation system which concerns on 1st Embodiment. The block diagram which shows an example of the structure of the viewpoint confirmation system which concerns on 1st Embodiment. The block diagram which shows an example of the structure of the image processing apparatus which concerns on 1st Embodiment. The schematic diagram which shows the outline of the viewpoint confirmation system which concerns on 2nd Embodiment. The figure which shows the use example of the viewpoint confirmation system which concerns on 2nd Embodiment. The block diagram which shows an example of the structure of the viewpoint confirmation system which concerns on 2nd Embodiment. The schematic diagram which shows the outline of the viewpoint confirmation system which concerns on 3rd Embodiment. The figure which shows the use example of the viewpoint confirmation system which concerns on 3rd Embodiment. The block diagram which shows an example of the structure of the viewpoint confirmation system which concerns on 3rd Embodiment. The block diagram which shows an example of the structure of the image processing apparatus which concerns on 3rd Embodiment. The schematic diagram which shows the outline of the viewpoint confirmation system which concerns on 4th Embodiment. The figure which shows the use example of the viewpoint confirmation system which concerns on 4th Embodiment. The schematic diagram which shows the outline of the viewpoint confirmation system which concerns on 5th Embodiment. The figure which shows the use example of the viewpoint confirmation system which concerns on 5th Embodiment. The block diagram which shows an example of the structure of the viewpoint confirmation system which concerns on 5th Embodiment. The block diagram which shows an example of the structure of the image processing apparatus which concerns on 5th Embodiment. The schematic diagram which shows the outline of the viewpoint confirmation system which concerns on modification 1. The figure which shows the use example of the viewpoint confirmation system which concerns on modification 1. The block diagram which shows an example of the structure of the viewpoint confirmation system which concerns on modification 1. The block diagram which shows an example of the structure of the image processing apparatus which concerns on modification 1. The block diagram which shows an example of the structure of the image processing apparatus which concerns on modification 2. The schematic diagram which shows the outline of the viewpoint confirmation system which concerns on modification 3. The figure which shows the use example of the viewpoint confirmation system which concerns on modification 3. The block diagram which shows an example of the structure of the viewpoint confirmation system which concerns on modification 3.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
First, the first embodiment of the present invention will be described.
In recent years, so-called e-sports (Electronic Sports), in which a game match is regarded as a competition, have become popular. Along with this, in order to develop human resources who are active in the e-sports world, for example, it is necessary to have a training place where game players can train practical skills and learn necessary skills and abilities (skills). ing. For example, it is assumed that an instructor trains students (players, etc.) in an e-sports school or the like. Players are divided into beginners, intermediate players, advanced players, etc., depending on the level of skill and ability. Smooth play operation is one of the factors that divide the stages, but it is also an important factor to accurately grasp the game situation for each scene. For example, by improving the ability to grasp the game situation, it is possible to improve the skills and abilities of beginners to intermediate players and intermediate players to advanced players. In order to grasp the game situation, it is important to check which part of the game image displayed on the game screen at what timing according to the game situation. The game video contains a lot of information. Advanced players are efficiently playing the game by efficiently confirming and grasping necessary information according to the game situation and performing play operations while anticipating the next development. By acquiring the viewpoint position of the player with respect to the game screen, it is expected that the skill and ability of the player will be improved by evaluating and improving which information is confirmed at which timing regarding the game situation. The viewpoint confirmation system according to the present embodiment is a system that can confirm the viewpoint position of the player with respect to the game screen. According to this viewpoint confirmation system, it is possible to evaluate and improve the viewpoint position of the player with respect to the game screen, and it is possible to improve the skill and ability of the player. Hereinafter, the viewpoint confirmation system according to this embodiment will be described in detail.

FIG. 1 is a schematic diagram showing an outline of the viewpoint confirmation system 1 according to the present embodiment. In the viewpoint confirmation system 1, for example, a player who plays a game on the game device GM sees and plays a game image displayed on the display screen (game screen) of the display device D1. The game image is an image showing a game situation that progresses according to the player playing. The game video output from the game device GM is distributed to two systems by the distribution device DT. The distribution device DT outputs one of the distributed game images to the display device D1 and outputs the other game image to the image processing device 100. The viewpoint detection device EY detects the viewpoint position of the player facing the display screen of the display device D1, and outputs the viewpoint information indicating the detected viewpoint position to the video processing device 100. For example, the viewpoint information output by the viewpoint detection device EY is information including the position information of the detected viewpoint position. The image processing device 100 generates a synthetic game image in which viewpoint information is added to the game image and stores it in a storage unit. Further, the video processing device 100 can output the stored synthetic game video to the display device D1. The synthetic game video in which the viewpoint information is added to the game video is, for example, a video in which a mark based on the viewpoint information is synthesized with the game video. This game image and viewpoint information are synchronized on the time axis. That is, the synthetic game video is a video in which the game video being played by the player and the viewpoint position of the player being played are displayed at the same time. The display device D1 switches between the game video input from the game device GM via the distribution device DT and the synthetic game video synthesized by the video processing device 100 by switching the input of the video to be displayed on the display screen. Can be displayed. Further, although the details will be described later, the target for detecting the viewpoint position by the viewpoint detection device EY is the actual situation by looking at the game image displayed on the display screen (game screen) of the display device D1 instead of the player playing the game. It may be a commentator who does. That is, it can also be used to confirm and evaluate the position and movement of the viewpoint of the commentator who is commenting on the play of the game.

FIG. 2 is a diagram showing a usage example of the viewpoint confirmation system 1 shown in FIG. The user of the display device D1 (the person facing the display screen of the display device D1) is, for example, a player. A game image and a synthetic game image are input to the display device D1. For example, the viewpoint confirmation system 1 displays a game image from the game device GM on the display device D1 during game play, and a mark based on the player's viewpoint information is displayed after play (after the end of play or after interruption of play). The synthesized synthetic game image is displayed on the display device D1. As a result, the player plays while watching the game image from the game device GM in real time during play, and the position of the viewpoint of the player during play is displayed in the synthetic game image in which the mark based on the viewpoint information is synthesized after the play. You can check and evaluate the movement. The viewpoint confirmation and evaluation may be performed by the player himself / herself, or may include an instructor or another player in place of or in addition to the player.

Next, the configuration of the viewpoint confirmation system 1 will be described in detail with reference to FIGS. 3 and 4.
FIG. 3 is a block diagram showing an example of the configuration of the viewpoint confirmation system 1 according to the present embodiment. The viewpoint confirmation system 1 includes a game device GM, a distribution device DT, a display device D1, a viewpoint detection device EY, and a video processing device 100.

The game device GM is configured to include a CPU (Central Processing Unit), and processes a game by executing a game program. For example, the game device GM performs game processing according to a player's operation on a controller (input device) (not shown), and outputs a game video PV0 according to the progress of the game to the distribution device DT.

The distribution device DT is a device that distributes the game video output from the game device GM to a plurality of game videos. Each of the distributed game images is the same image, but the resolution and the effective display area of the display screen may be different according to the distribution destination device. Specifically, when the game video PV0 output from the game device GM is input, the distribution device DT distributes the input game video PV0 into two systems, the first video PV1 and the second video PV2. Output. The first video PV1 is input to the display device D1. The second video PV2 is input to the video processing device 100. For example, the game video PV0, the first video PV1, and the second video PV2 are transmitted as video signals corresponding to HDMI (registered trademark). The game video PV0, the first video PV1, and the second video PV2 are not limited to being transmitted by a video signal corresponding to HDMI, and include DVI (Digital Visual Interface), composite video, and composite video. It may be transmitted as a video signal corresponding to any standard, such as an S video separated into two systems of a brightness signal and a color signal.

The viewpoint detection device EY detects (eye tracking) the viewpoint position of the user (for example, the player) of the display device D1. For example, the viewpoint detection device EY includes a camera that captures an image of the direction facing the display screen of the display device D1, and detects the viewpoint position on the display screen based on the captured image. Any method can be used as the method for detecting the viewpoint position. For example, the viewpoint position may be detected based on the position of the iris with respect to the position of the inner corner of the eye of the person captured by the visible light camera, or the position of the corneal reflex formed by irradiating infrared light may be used as the reference position with respect to the reference position. The viewpoint position may be detected based on the position of the pupil. For example, the viewpoint detection device EY continuously detects and tracks the viewpoint position of the player during play at a predetermined sampling cycle. The viewpoint detection device EY outputs the viewpoint information ED1 indicating the detected viewpoint position to the video processing device 100. For example, the viewpoint information is transmitted using USB (Universal Serial Bus).

The video processing device 100 stores the game video and the viewpoint information, synthesizes the game video and the viewpoint information, and the like. For example, the video processing device 100 is a computer device including a CPU (Central Processing Unit) and the like.
FIG. 4 is a block diagram showing an example of the configuration of the video processing apparatus 100 according to the present embodiment. The video processing apparatus 100 includes a generation unit 110, a storage control unit 120, and a video output unit 130.

The second video PV2 output from the distribution device DT and the viewpoint information ED1 output from the viewpoint detection device EY are input to the generation unit 110. The generation unit 110 converts the input second video PV2 into video information PV2'which can be processed into video, and the viewpoint position detected by the viewpoint detection device EY in the video information PV2' based on the second video PV2. Synthetic video information CV1'with the viewpoint information ED1 indicating the above is generated. More specifically, the synthetic video information CV1'is generated by adding the viewpoint information ED1'to the video information PV2', which is data-converted so that the viewpoint information ED1 can be added to the video information PV2'. The composite video information in which the viewpoint information is added to the video information is, for example, information obtained by synthesizing a mark based on the viewpoint information into the video information. For example, the composite video information CV1'is video information in which a mark based on the viewpoint information ED1'is combined with the video information PV2'based on the second video PV2, and the video information PV2'and the viewpoint information ED1' are on the time axis. Synchronized with. The generation unit 110 outputs the generated synthetic video information CV1'to the storage control unit 120.

The storage control unit 120 stores the synthetic video information CV1'generated by the generation unit 110 in a storage unit (not shown). The storage unit includes, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), an SSD (Solid State Drive), a flash ROM (Read-Only Memory), and the like. The storage unit may be built in the video processing device 100, or may be an external storage device connected by USB or the like.

The video output unit 130 outputs a synthetic game video CV1 based on the synthetic video information CV1'stored in the storage unit by the storage control unit 120. For example, the video output unit 130 converts the synthetic video information CV1'to HDMI-compatible synthetic game video CV1 and outputs it. As a result, the synthetic game video CV1 is transmitted from the video processing device 100 to the display device D1 as a video signal corresponding to HDMI. The synthetic game video CV1 is not limited to being transmitted as a video signal corresponding to HDMI, and DVI (Digital Visual Interface), composite video, and composite video are separated into two systems, a brightness signal and a color signal. It may be transmitted by a video signal corresponding to any standard such as the S video.

The display device D1 is a display for displaying information such as an image or text, and includes, for example, a liquid crystal display panel, an organic EL (Electroluminescence) display panel, and the like. The display device D1 is provided with two or more video input terminals, and can switch and display two or more input videos. For example, a first video PV1 output from the distribution device DT and a synthetic game video CV1 output from the video processing device 100 are input to the video input terminal of the display device D1. The display device D1 can switch between the first video PV1 distributed by the distribution device DT and the synthetic game video CV1 output from the video processing device 100. The first video PV1 is a video corresponding to the game video P0 output from the game device GM.

As described above, the viewpoint confirmation system 1 according to the present embodiment is an example of the display device D1 (first display device) on which the game video PV0 (more specifically, the first video PV1) output from the game device GM is displayed. ), And a viewpoint detection device EY that detects the viewpoint position of a player (an example of a target person) facing the display screen of the display device D1. Further, the viewpoint confirmation system 1 distributes the game video PV0 output from the game device GM to at least the first video PV1 for inputting to the display device D1 and the second video PV2 having an input destination different from that of the first video PV1. The distribution device DT and the image processing device 100 are provided. The video processing device 100 adds the viewpoint information ED1 (more specifically, the viewpoint information ED1') indicating the viewpoint position detected by the viewpoint detection device EY to the video information PV2'based on the second video PV2, and adds the composite video information CV1'. A generation unit 110 that generates the And have.

As a result, the viewpoint confirmation system 1 displays the player's viewpoint information in the image processing device 100 other than the display device D1 while playing the game while watching the display device D1 that displays the game image output from the game device GM. By recording as a composite game video combined with the game video, it is possible to prevent the display device D1 from displaying the mark based on the viewpoint information during play. Therefore, it is possible to confirm the viewpoint of the player during play without interfering with the player's play of the game.

For example, the display device D1 (an example of the first display device) switches between the first video PV1 distributed from the game video PV0 output from the game device GM and the synthetic game video CV1 output from the video output unit 130. Can be displayed. For example, the display device D1 includes a plurality of video inputs (for example, 2ch (channel) video inputs). The video input may be switched manually (for example, by operating the input switching button) or automatically (for example, by detecting and switching the video signal).

As a result, in the viewpoint confirmation system 1, the player of the game can play the game while looking at the display device D1 and can check the synthesized game image while looking at the same display device D1 after the play, so that the minimum of one display device is The system allows you to check the viewpoint of the player who is playing.

Further, the storage control unit 120 stores the synthetic video information CV1'generated by the generation unit 110 in the storage unit. The video output unit 130 outputs a synthetic game video CV1 based on the synthetic video information CV1'stored in the storage unit.

As a result, the viewpoint confirmation system 1 can store the composite video information obtained by synthesizing the mark based on the player's viewpoint information and the game video and output it later, so that the subject himself / herself is active after playing. You can check the viewpoint of the target person.

In addition, game commentators also play an important role in e-sports, and their training is also an issue. Like game players, game commentators have stages in their abilities. It depends on each individual's speaking ability, but in many cases it is not possible to play live well because the points for checking the situation according to the game situation of each game are not known. Therefore, it can be expected that the live performance ability will be improved by confirming and imitating the viewpoint position of the live performance person who is said to have high ability in the actual live performance. Therefore, an example of applying the viewpoint confirmation system 1 according to the present embodiment to the confirmation of the viewpoint of a game commentator will be described.
In this example, the user of the display device D1 shown in FIGS. 1 to 3 described above is a live person instead of the player. The viewpoint detection device EY detects the viewpoint position of the live person facing the display screen of the display device D1, and outputs the viewpoint information indicating the detected viewpoint position to the video processing device 100. In this case, the game video is a video showing a game situation in which the game device GM automatically plays (automatically plays) without a player's play operation. In other words, the commentator in this example is a commentator at the time of automatic play, for example, a student practicing the commentary while watching the game video by automatic play, or a lecturer demonstrating the commentary while watching the game video by automatic play. .. The display device D1 switches between the game video input from the game device GM via the distribution device DT and the synthetic game video synthesized by the video processing device 100 by switching the input of the video to be displayed on the display screen. Can be displayed.
As a result, the viewpoint confirmation system 1 displays a mark based on the viewpoint information of the commentator during the actual situation of the game played while watching the display device D1 displaying the game image output from the game device GM, and the image other than the display device D1. By recording the composite game video combined with the game video on the processing device 100, it is possible to prevent the display device D1 from displaying the mark based on the viewpoint information during the actual situation. Therefore, it is possible to confirm the viewpoint of the commentator during the commentary without interfering with the commentary of the game by the commentator.

[Second Embodiment]
Next, a second embodiment of the present invention will be described.
In the present embodiment, a configuration example in which a display device used by the player is separately provided will be described with respect to the first embodiment in which the position and movement of the viewpoint of the live person are confirmed and evaluated.
FIG. 5 is a schematic diagram showing an outline of the viewpoint confirmation system 1B according to the present embodiment. In the case of confirming and evaluating the position and movement of the viewpoint of the commentator in the first embodiment, the commentary was performed while watching the game image by automatic play. In the present embodiment, it is possible to play live while watching the game image actually played by the player instead of the automatic play. Therefore, the viewpoint confirmation system 1B has a configuration in which the display device D2 used by the player is added to the viewpoint confirmation system 1 according to the first embodiment. That is, the display device D1 is used by the commentator, and the display device D2 is used by the player.
Further, since the distribution device DT needs to distribute the game video to the added display device D2, the game video output from the game device GM is distributed to the three systems. That is, the distributed game video is output to the display device D1, the display device D2, and the video processing device 100, respectively.

The player plays the game by watching the game image displayed on the display device D2. The commentator sees the game image displayed on the display device D1 (the same game image as the game image displayed on the display device D2) and plays the game. The viewpoint detection device EY detects the viewpoint position of the user of the display device D1 and outputs the viewpoint information indicating the detected viewpoint position to the video processing device 100. The video processing device 100 can detect the viewpoint position of the live person together with the game video, generate a synthetic game video in which the viewpoint information is added to the game video, and output it to the display device D1.

FIG. 6 is a diagram showing a usage example of the viewpoint confirmation system 1B shown in FIG. The user who uses the display device D1 is a commentator. A game image and a synthetic game image are input to the display device D1. On the other hand, the user who uses the display device D2 is a player. A game image is input to the display device D2. For example, the viewpoint confirmation system 1B displays the game image from the game device GM on the display device D1 and the display device D2 during the game play (during the game live), and after the play (after the end of the play or after the play is interrupted). Displays a synthetic game image on which a mark based on the player's viewpoint information is synthesized on the display device D1. As a result, the commentator performs the commentary while watching the game image from the game device GM in real time while the player is playing (during the game commentary), and after the play, the comment is a synthetic game image in which the mark based on the viewpoint information is synthesized. It is possible to confirm and evaluate the position and movement of the viewpoint of the commentator during the commentary. The confirmation and evaluation of the viewpoint may be performed by the commentator himself / herself, or may include a lecturer or another player in place of or in addition to the commentator. In addition, the player can play while watching the game image from the game device GM in real time during play.

Next, the configuration of the viewpoint confirmation system 1B will be described in detail with reference to FIG. 7.
FIG. 7 is a block diagram showing an example of the configuration of the viewpoint confirmation system 1B according to the present embodiment. In FIG. 7, the same configurations as those in FIG. 3 are designated by the same reference numerals, and the description thereof will be omitted. The configuration of the viewpoint confirmation system 1B shown in FIG. 7 is different from the configuration of the viewpoint confirmation system 1 shown in FIG. 3 in that the display device D2 is added. The distribution device DT distributes and outputs the game video PV0 input from the game device GM to three systems of the first video PV1, the second video PV2, and the third video PV3. The third video PV3 is input to the display device D2. As in the configuration shown in FIG. 3, the first video PV1 is input to the display device D1, and the second video PV2 is input to the video processing device 100.

As described above, in the viewpoint confirmation system 1B, the display device D1 is a display device for a live person, and the display device D2 is a display device for a player. The display device D1 for the commentator can switch between the first video PV1 distributed by the distribution device DT and the synthetic game video CV1 output from the video processing device 100. On the other hand, the display device D2 for the player displays the third video PV3 (video corresponding to the game video P0 output from the game device GM) distributed by the distribution device DT.

Further, in the viewpoint confirmation system 1B, the distribution device DT adds the game video PV0 output from the game device GM to the first video PV1 and the second video PV2, and further distributes the game video PV3 to the third video PV3. Then, the display device D2 (an example of the second display device) displays the third video PV3.

As a result, the viewpoint confirmation system 1B displays the viewpoint of the commentator who plays the game while watching the game image on the display device D1 in the usage situation where the player is playing the game while watching the game image on the display device D2. It can be confirmed on the same first display device after the game is live.

[Third Embodiment]
Next, a third embodiment of the present invention will be described.
In this embodiment, a configuration example in which a display device for confirming a viewpoint is separately provided for the first embodiment will be described.
FIG. 8 is a schematic diagram showing an outline of the viewpoint confirmation system 1C according to the present embodiment. The viewpoint confirmation system 1C includes a display device D3 in addition to the display device D1 for the player, and the point that the synthesized game image is input to the display device D3 instead of the display device D1 is the viewpoint confirmation system shown in FIG. Different from 1. This display device D3 is used, for example, as a display device for a lecturer or a student. As described in the first embodiment, the display device D1 can be used not only for the player but also for the live player during automatic play.

FIG. 9 is a diagram showing a usage example of the viewpoint confirmation system 1C shown in FIG. The user of the display device D1 is a player or a commentator. A game image is input to the display device D1. On the other hand, the user of the display device D3 is a lecturer or a student. A synthetic game image is input to the display device D3. Here, the synthetic game video input to the display device D3 is a video in which viewpoint information is added in real time to the game video, and is the same as the game video input to the display device D1 in real time during play. It is an image to be displayed (specifically, the synthetic game image input to the display device D3 is delayed by the time required for the process of adding the viewpoint information to the game image input to the display device D1). For example, the viewpoint confirmation system 1C displays a game image from the game device GM on the display device D1 during game play, and the display device D3 is composed of a mark based on the viewpoint information of the player or the commentator. Display the game image. As a result, the player or the commentator plays or plays the game while watching the game image from the game device GM in real time, and a third party such as a lecturer or a student can see the viewpoint of the player or the commentator during the play or the commentary. You can proceed with the lecture while checking and evaluating the position and movement in real time. When the user of the display device D1 is a live person, the game image is an image showing a game situation that progresses according to the automatic play by the game device GM without the player's play operation.

Next, the configuration of the viewpoint confirmation system 1C will be described in detail with reference to FIGS. 10 and 11. FIG. 10 is a block diagram showing an example of the configuration of the viewpoint confirmation system 1C according to the present embodiment. In FIG. 10, the same configurations as those in FIG. 3 are designated by the same reference numerals, and the description thereof will be omitted. The configuration of the viewpoint confirmation system 1C shown in FIG. 10 is that the display device D3 is added to the configuration of the viewpoint confirmation system 1 shown in FIG. 3, and the input destination of the synthetic game video CV1 is replaced with the display device D1. The difference is that it is a display device D3. Further, the video processing apparatus 100C is different from the video processing apparatus 100 described in the above embodiment in that the viewpoint information ED1 is added to the second video PV2 in real time.

FIG. 11 is a block diagram showing an example of the configuration of the video processing apparatus 100C according to the present embodiment. The video processing apparatus 100C includes a generation unit 110 and a video output unit 130. The generation unit 110 converts the input second video PV2 into video information PV2'which can be processed into video, and the viewpoint position detected by the viewpoint detection device EY in the video information PV2' based on the second video PV2. The synthetic video information CV1'added with the viewpoint information ED1 (more specifically, the viewpoint information ED1') is generated in real time in response to the input of the second video PV2 and the viewpoint information ED1 and output to the video output unit 130. The video output unit 130 outputs a synthetic game video CV1 based on the synthetic video information CV1'generated by the generation unit 110. For example, the video output unit 130 converts the synthetic video information CV1'to HDMI-compatible synthetic game video CV1 and outputs it. As a result, the display device D3 can display the synthetic game video CV1 output from the video processing device 100C during the player's play.

As described above, the viewpoint confirmation system 1C according to the present embodiment is an example of the display device D1 (first display device) on which the game video PV0 (more specifically, the first video PV1) output from the game device GM is displayed. ), And a viewpoint detection device EY for detecting the viewpoint position of a target person, for example, a player facing the display screen of the display device D1 or a live person. Further, the viewpoint confirmation system 1C distributes the game video PV0 output from the game device GM to at least the first video PV1 for inputting to the display device D1 and the second video PV2 whose input destination is different from that of the first video PV1. A distribution device DT, a video processing device 100C, and a display device D3 (an example of a third display device) different from the display device D1 are provided. The video processing device 100C is a composite video information CV1'in which the viewpoint information ED1 (more specifically, the viewpoint information ED1') indicating the viewpoint position detected by the viewpoint detection device EY is added to the video information PV2'based on the second video PV2. It has a generation unit 110 for generating the above, and a video output unit 130 for outputting the synthetic game video CV1 based on the synthetic video information CV1'generated by the generation unit 110. Then, the display device D1 displays the first video PV1. On the other hand, the display device D3 displays the synthetic game video CV1.

As a result, the viewpoint confirmation system 1C displays a mark based on the viewpoint information of the target person other than the display device D1 in the play or the actual situation of the game performed while watching the display device D1 that displays the game image output from the game device GM. In the video processing device 100C of the above, the composite game video CV1 synthesized with the game video is prevented from being displayed on the display device D1, while the display device D3 is used for a third party (for example, a teacher or a student) other than the target person. So, since the viewpoint can be confirmed during the play or live of the target person, the viewpoint of the target person during play or live can be confirmed without interfering with the play or live of the game by the target person. be able to.

In this embodiment, an example in which the display device D3 displays the synthetic game video CV1 in real time during the player's play has been described, but the synthetic video information CV1'is displayed in the same manner as in the first and second embodiments. You may store and display the synthetic game video CV1 based on the stored synthetic video information CV1'after playing. In this case, the video processing device 100C may have the same configuration as the video processing device 100 shown in FIG. 4, or may be combined with the configuration of the video processing device 100 shown in FIG.

Further, in the present embodiment, the input destination of the synthetic game video CV1 is only the display device D3, but by setting both the display device D1 and the display device D3, the synthetic game video can be used by either the display device D1 or the display device D3. The configuration may be such that the CV1 can be displayed.

[Fourth Embodiment]
Next, a fourth embodiment of the present invention will be described.
FIG. 12 is a schematic diagram showing an outline of the viewpoint confirmation system 1D according to the present embodiment. The viewpoint confirmation system 1D has the same basic system configuration as the viewpoint confirmation system 1C according to the third embodiment, but is different from the player A who plays using the display device D1 (for example, an opponent). ) Is the user of the display device D3.

FIG. 13 is a diagram showing a usage example of the viewpoint confirmation system 1D shown in FIG. The user of the display device D1 is the player A. On the other hand, the user of the display device D3 is the player B. Player A and player B are, for example, players who play against each other in a battle game. Each of player A and player B is a player who simultaneously plays the same game on the game device GM, and plays using each of two input devices (controllers and the like) connected to the game device GM. Further, this game is played by the player A and the player B in the same game space, and the game image seen by the player A during the play and the game image seen by the player B during the play are generated by the game device GM. It is assumed that the same game video can be used.

A game image from the game device GM is input to the display device D1. On the other hand, the display device D3 is input with a synthetic game video in which the viewpoint information indicating the viewpoint position of the player A is added to the game video. For example, the viewpoint confirmation system 1D displays a game image from the game device GM on the display device D1 during game play, and a synthetic game image in which a mark based on the viewpoint information of the player A is synthesized on the display device D3. Is displayed.

As a result, the player A plays while looking at the display device D1 that displays the game image output from the game device GM during play, while the player B who plays while looking at the display device D3 is the player A during play. You can play while checking the viewpoint. For example, when the player B is a lecturer or an advanced player, it is possible to confirm the position and movement of the viewpoint of the player A while playing the game together with the player A and give an improvement instruction or the like. Further, when the player A is a lecturer or an advanced player and the player B is a student, the player B can predict the next action of the player A from the position and movement of the viewpoint of the player A and play the game. can. Player B can be expected to improve by playing while being aware of the points to be concerned about in each game scene.

[Fifth Embodiment]
Next, a fifth embodiment of the present invention will be described.
In this embodiment, an example of detecting the viewpoint positions of a plurality of players and synthesizing them with a game image will be described.
FIG. 14 is a schematic diagram showing an outline of the viewpoint confirmation system 1E according to the present embodiment. In the viewpoint confirmation system 1E, the player A who plays the game on the game device GMa sees and plays the game image displayed on the display device D1a, and the player B who plays the game on the game device GMb is displayed on the display device D1b. Watch the game video and play. The game video output from the game device GMa is distributed to two systems by the distribution device DTa, one game video is input to the display device D1a, and the other game video is input to the video processing device 100E. Similarly, the game video output from the game device GMb is distributed to two systems by the distribution device DTb, one game video is input to the display device D1b, and the other game video is input to the video processing device 100E. Further, the viewpoint detection device EYa detects the viewpoint position of the user of the display device D1a, and outputs the viewpoint information indicating the detected viewpoint position to the video processing device 100E. Similarly, the viewpoint detection device EYb detects the viewpoint position of the user of the display device D1b, and outputs the viewpoint information indicating the detected viewpoint position to the video processing device 100E.

The video processing device 100E causes the display device D3 to display a composite composite game video in which the viewpoint information is added to the play video of each of the plurality of players. As shown in the display screen G10, the composite synthetic game video is divided into two left and right with the center of the display screen as the boundary, and the composite game video with the player A's viewpoint information added to the player A's game video on the left side and the player A's viewpoint information on the right side. It may be combined so as to be a synthetic game video in which the viewpoint information of the player B is added to the game video of the player B. Further, as shown on the display screen G20, a composite composite game video in which both the viewpoint information of the player A and the viewpoint information of the player B are added to the game video of either the game video of the player A or the game video of the player B. May be. In the example shown on the display screen G10, the display of the viewpoint position of each player is easy to see, and in the example shown on the display screen G20, the relationship between the viewpoints according to the game situation of each player is clarified.

Instead of dividing the display screen into two parts on the left and right with the center of the display screen as a boundary and displaying them on the left side and the right side, the display screen may be divided into two parts on the upper and lower sides and displayed on the upper side and the lower side. Further, the composite composite game video in which the viewpoint information is added to the play video of each of the plurality of players may be displayed side by side on one screen, or at least a part thereof may be overlapped and displayed. For example, if the game images of a plurality of players are the same, they may be displayed in an overlapping manner, and if they are different, they may be displayed side by side.

FIG. 15 is a diagram showing a usage example of the viewpoint confirmation system 1E shown in FIG. The user of the display device D1a is the player A, and the user of the display device D1b is the player B. The user of the display device D3 is a lecturer, a student, or the like. A play video is input to the display device D1a and the display device D1b, and the composite composite game video is displayed on the display device D3. For example, the viewpoint confirmation system 1E displays a game image on the display device D1a and the display device D1b during game play, and the display device D3 is a composite in which marks based on the viewpoint information of the player A and the player B are synthesized. A synthetic game image (see the display screens G10 and G20 in FIG. 14) is displayed. Further, the viewpoint confirmation system 1E may display the composite composite game image on the display device D3 after playing the game.

In this way, it is also meaningful to detect and compare the viewpoint positions of a plurality of players at the same time. However, when the game video of each player is different, it is the same as the configuration and effect of the first to fourth embodiments described above. This embodiment is more effective when comparing viewpoint information when playing the same game video with a plurality of players. For example, in a battle game in which multiple players fight while watching the same game video (for example, a fighting game), each player plays while watching which position, such as when attacking, defending, or taking time. It is possible to confirm and compare whether or not it is done in the composite synthetic game video, and it is possible to clarify the difference in the viewpoint position in offense and defense.

Next, the configuration of the viewpoint confirmation system 1E will be described in detail with reference to FIGS. 16 and 17. FIG. 16 is a block diagram showing an example of the configuration of the viewpoint confirmation system 1E according to the present embodiment. The viewpoint confirmation system 1E includes a game device GMa, a distribution device DTa, a display device D1a, a viewpoint detection device EYa, a game device GMb, a distribution device DTb, a display device D1b, a viewpoint detection device EYb, and video processing. The device 100E and the display device D3 are provided.

The game device GMa outputs the game video PV0a according to the progress of the game to the distribution device DTa. The distribution device DTa distributes and outputs the game video PV0a input from the game device GMa to two systems, the first video PV1a and the second video PV2a. The first video PV1a is input to the display device D1a, and the second video PV2a is input to the video processing device 100E. The viewpoint detection device EYa detects the viewpoint position of the user (for example, player A) of the display device D1a, and outputs the viewpoint information ED1a indicating the detected viewpoint position to the video processing device 100E.

Further, the game device GMb outputs the game video PV0b according to the progress of the game to the distribution device DTb. The distribution device DTb distributes and outputs the game video PV0b input from the game device GMb to two systems, the first video PV1b and the second video PV2b. The first video PV1b is input to the display device D1b, and the second video PV2b is input to the video processing device 100E. Further, the viewpoint detection device EYb detects the viewpoint position of the user (for example, player B) of the display device D1b, and outputs the viewpoint information ED1b indicating the detected viewpoint position to the video processing device 100E.

The video processing apparatus 100E generates composite composite video information based on the input second video PV2a, viewpoint information ED1a, second video PV2b, and viewpoint information ED1b, and outputs the composite composite game video.
FIG. 17 is a block diagram showing an example of the configuration of the video processing apparatus 100E according to the present embodiment. The video processing apparatus 100E includes a generation unit 110E, a storage control unit 120, and a video output unit 130.

The generation unit 110E has a second video PV2a output from the distribution device DTa, a viewpoint information ED1a output from the viewpoint detection device EYa, a second video PV2b output from the distribution device DTb, and a viewpoint detection device EYb. The output viewpoint information ED1b is input. The generation unit 110E converts the input second video PV2a into video information PV2a'that can be processed into video, and the viewpoint position detected by the viewpoint detection device EYa on the video information PV2a' based on the second video PV2. The first synthetic video information to which the viewpoint information ED1a indicating (the viewpoint position of the player A) (more specifically, the viewpoint information ED1a' obtained by data conversion so that the processing of adding the viewpoint information ED1a to the video information PV2a'can be performed) is added. To generate. Further, the generation unit 110E converts the input second video PV2b into video information PV2b'that can be processed into video, and the video information PV2b'based on the second video PV2b is detected by the viewpoint detection device EYb. Second synthesis with added viewpoint information ED1b indicating the viewpoint position (viewpoint position of player B) (more specifically, viewpoint information ED1b'data converted so that the viewpoint information ED1b can be added to the video information PV2b'). Generate video information. Then, the generation unit 110E generates the composite composite video information CV10'based on the first composite video information and the second composite video information. For example, the generation unit 110E is divided into two left and right sides with the center of the display screen as a boundary, and the left side is a video based on the first synthetic video information (game video to which the viewpoint information of the player A is added), and the right side is the second synthetic video information. The composite composite video information CV10'is generated so that the video based on (the game video to which the viewpoint information of the player B is added) is displayed (see the display screen G10 in FIG. 14). The generation unit 110E outputs the generated composite composite video information CV10'to the storage control unit 120 and the video output unit 130.

In addition, the generation unit 110E has a viewpoint position (player) detected by the viewpoint detection device EYa in the video information of either the video information PV2a'based on the second video PV2 or the video information PV2b' based on the second video PV2b. It was detected by the viewpoint information ED1a indicating (the viewpoint position of A) (more specifically, the viewpoint information ED1a' obtained by data conversion so that the viewpoint information ED1a can be added to the video information PV2a') and the viewpoint detection device EYb. Composite synthesis with addition of viewpoint information ED1b indicating the viewpoint position (viewpoint position of player B) (more specifically, viewpoint information ED1b' data-converted so that the viewpoint information ED1b can be added to the video information PV2b'). The video information CV10'may be generated (see the display screen G20 in FIG. 14).

The video output unit 130 can output the composite composite game video CV10 based on the composite composite video information CV10'generated by the generation unit 110E during play. Further, the video output unit 130 can output the composite composite video CV10 based on the composite composite video information CV10'stored in the storage unit by the storage control unit 120 after the play.

With this configuration, the display device D1a used by the player A and the display device D1b used by the player B display the game image output from the game device GM in real time during play. On the other hand, the display device D3 may display the composite composite game video CV10 output from the video processing device 100E during play, or display the composite composite game video CV10 output from the video processing device 100E after play. Is also possible.

As described above, the viewpoint confirmation system 1E according to the present embodiment uses a plurality of viewpoint detection devices EY (EYa, EYb) for detecting the viewpoint positions of a plurality of players (an example of a target person). The distribution device DT is configured to include distribution devices DTa and DTb (an example of individual distribution devices) that distribute game images corresponding to each of a plurality of players. The generation unit 110E includes a plurality of viewpoint information ED1 (ED1a, ED1b) corresponding to each of the plurality of players, and game video (video information PV2a'based on the second video PV2 and video information PV2b'based on the second video PV2b. Generates composite composite video information CV10'based on either one or both). The video output unit 130 outputs the composite composite game video CV10 based on the composite composite video information CV10'generated by the generation unit 110E. For example, the video output unit 130 outputs the composite composite game video CV10 based on the composite composite video information CV10'generated by the generation unit 110E in real time during play. Further, after the play, the video output unit 130 outputs the composite composite video CV10 based on the composite composite video information CV10'stored in the storage unit by the storage control unit 120.

As a result, the viewpoint confirmation system 1E allows the plurality of players (for example, player A and player B) to view each of the plurality of display devices D1a and D1b for displaying the game video output from the respective game devices GMa and GMb. During the play of the game to be performed, a composite synthetic game video CV10 in which a mark based on the viewpoint information of each of a plurality of players (for example, player A and player B) is synthesized with the game video is displayed on a display device other than the display devices D1a and D1b. In D3, since it is displayed to a third party (for example, an instructor or a student), the viewpoints of a plurality of players playing can be viewed by a third party (for example, an instructor) without interfering with the player's play of the game. And students) can be confirmed. Further, the viewpoint confirmation system 1E can also enable a third party (for example, a lecturer or a student) to confirm the viewpoints of a plurality of players during play after the play.

[Modification 1]
A configuration example in which the display device D2 is added to the viewpoint confirmation system 1C (see FIG. 8) according to the third embodiment described above will be described.
FIG. 18 is a schematic diagram showing an outline of the viewpoint confirmation system 1F according to the modified example 1. In the viewpoint confirmation system 1F illustrated, the user of the display device D1 is limited to the live person, and the game image is displayed not only on the display device D1 during play but also on the display device D2 used for the player to play the game. The point of display and the point of displaying the synthesized game image on one or both of the display device D1 and the display device D3 after the play are different from the viewpoint confirmation system 1C shown in FIG. For example, the display device D1 is used as a display device for a commentator, the display device D2 is used as a display device for a player, and the display device D3 is used as a display device for a lecturer or a student. In the viewpoint confirmation system 1F, the video processing device 100F stores a composite game video in which a play video and a mark based on the viewpoint information of the commentator are combined, and then displays the video on one or both of the display device D1 and the display device D3. You may.

FIG. 19 is a diagram showing a usage example of the viewpoint confirmation system 1F shown in FIG. In the usage example of the display device D1 and the display device D3, the user of the display device D1 is a live person, as opposed to the usage example of the viewpoint confirmation system 1C according to the third embodiment shown in FIG. The difference is that the display device D3 can display the synthetic game image after playing. In the display device D2, the user is a player, and the game image is displayed during play. The usage example of the display device D1 used by the commentator and the display device D2 used by the player is the same as the usage example of the viewpoint confirmation system 1B shown in FIG. For example, a synthetic game video is a video in which the viewpoint information of a commentator is added in real time to the game video. For example, the display device D3 displays a synthetic game image in which marks based on the viewpoint information of the commentator are synthesized during the play of the game. As a result, a third party such as a lecturer or a student can see the viewpoint of the commentator during the commentary without interfering with the game image that the commentator is watching during the commentary of the game played by the player using the display device D2. You can proceed with the lecture while checking and evaluating the position and movement in real time. Further, one or both of the display device D1 and the display device D3 can display the synthetic game image after the play. As a result, the position and movement of the viewpoint of the commentator can be confirmed by the commentator himself after the play (after the commentary), or the lecturer or the commentator himself can give a lecture to the students.

Next, the configuration of the viewpoint confirmation system 1F will be described in detail with reference to FIGS. 20 and 21. FIG. 20 is a block diagram showing an example of the configuration of the viewpoint confirmation system 1F according to the modified example 1. In FIG. 20, the same configurations as those in FIGS. 7 and 10 are designated by the same reference numerals, and the description thereof will be omitted. The configuration of the viewpoint confirmation system 1F shown in FIG. 20 is such that the third video PV3 in which the game video PV0 is distributed by the distribution device DT is input to the display device D2 with respect to the configuration of the viewpoint confirmation system 1C shown in FIG. The difference is that the video processing device 100F can display the synthesized game video on the display device D1 and the video processing device 100F includes the storage control unit 120.

FIG. 21 is a block diagram showing an example of the configuration of the video processing apparatus 100F according to the first modification. The video processing device 100F includes a storage control unit 120 in addition to the generation unit 110 and the video output unit 130 included in the video processing device 100C shown in FIG. 11, and the composite video information generated by the generation unit 110. CV1'is output not only to the video output unit 130 but also to the storage control unit 120. The video output unit 130 can also output the synthetic game video CV1 based on the synthetic video information CV1'stored in the storage unit by the storage control unit 120, and is also generated by the generation unit 110 in real time during play (during live performance). It is also possible to output a synthetic game video CV1 based on the synthetic video information CV1'.

The video output unit 130 is based on the synthetic game video CV1 based on the synthetic video information CV1'generated in real time and the synthetic video information CV1'read after temporarily storing the generated synthetic video information CV1'. It may be configured to output only one of the synthetic game video CV1. For example, when the video output unit 130 outputs only the synthetic game video CV1 based on the synthetic video information CV1'that has been stored once, the configuration of the video processing device 100F may be the configuration of the video processing device 100 shown in FIG. Further, when the video output unit 130 outputs only the synthetic game video CV1 based on the synthetic video information CV1'generated in real time, the configuration of the video processing device 100F may be the configuration of the video processing device 100C shown in FIG. good. In this case, there may be no connection path for making the synthetic game video CV1 the display device D1.

[Modification 2]
A modified example of the image processing apparatus 100 will be described. For example, in the configuration example of the video processing device 100 shown in FIG. 4, the game video and the mark based on the viewpoint information are combined and then stored, but the game video and the viewpoint information are individually stored before being combined. , When outputting the composite game video, the game video and the mark based on the viewpoint information may be composited.

FIG. 22 is a block diagram showing an example of the configuration of the video processing apparatus 100B according to the modified example 2. The video processing device 100B includes a generation unit 110, a storage control unit 120, and a video output unit 130, similarly to the video processing device 100 shown in FIG. 4, but the order of processing is different.

The second video PV2 output from the distribution device DT and the viewpoint information ED1 output from the viewpoint detection device EY are input to the storage control unit 120. The storage control unit 120 converts the input second video PV2 into video information PV2 ′ capable of video processing and stores it in the storage unit. As described above, the storage unit may be built in the video processing device 100, or may be an external storage device connected by USB or the like. Further, the storage control unit 120 stores the input viewpoint information ED1 (more specifically, the viewpoint information ED1') in the storage unit. That is, the storage control unit 120 individually stores the video information PV2'based on the input second video PV2 and the viewpoint information ED1'in the storage unit. The video information PV2'and the viewpoint information ED1'are stored in association with each other so as to be synchronized on the time axis.

The generation unit 110 reads out the video information PV2'and the viewpoint information ED1'stored in the storage unit by the storage control unit 120, and generates the composite video information CV1'in which the viewpoint information ED1' is added to the video information PV2'. .. For example, the composite video information CV1'is video information in which a mark based on the viewpoint information ED1'is combined with the video information PV2'based on the second video PV2, and the video information PV2'and the viewpoint information ED1' are on the time axis. Synchronized with. The generation unit 110 outputs the generated composite video information CV1'to the video output unit 130.

The video output unit 130 outputs a synthetic game video CV1 based on the synthetic video information CV1'generated by the generation unit 110. For example, the video output unit 130 converts the synthetic video information CV1'to HDMI-compatible synthetic game video CV1 and outputs it.

As described above, in the video processing apparatus 100B according to the modified example 2, the storage control unit 120 individually stores the video information PV2'based on the second video PV2 and the viewpoint information ED1 (ED1') in the storage unit. , The generation unit 110 generates synthetic video information in which the viewpoint information ED1'stored in the storage unit is added to the video information PV2'stored in the storage unit, and the video output unit 130 generates the composite video information in the generation unit 110. The synthetic game video CV1 based on the combined video information CV1'is output.

The video processing device 100B according to the second modification is a synthetic game in which the viewpoint information of the player or the commentator and the game video are individually stored, and the mark based on the viewpoint information and the game video are combined after the play or the commentary. Since the video can be output, the player or the commentator himself can confirm the viewpoint during the play or the commentary after the play or the commentary. That is, the video processing device 100B stores the game video and the viewpoint information individually, so that the synthetic game video in which the viewpoint information is added to the game video is played or the actual situation as in the video processing device 100 shown in FIG. It can be output later.

As for the video processing apparatus 100E shown in FIG. 17, the second video PV2 (PV2a, PV2b) before the composite composite video information CV10'is generated, instead of being stored after the composite composite video information CV10'is generated. The individual information including the video information PV2'(PV2a', PV2b') and the viewpoint information ED1'(ED1a, ED1b) based on the above is stored, and is stored when the composite composite game video CV10 is output. The composite composite video information CV10'may be generated based on the existing game video and the viewpoint information. Similarly, for the video processing device 100F shown in FIG. 21, the video information PV2 based on the second video PV2 before the composite video information CV1'is generated, instead of being stored after the composite video information CV1'is generated. Individual information including'and viewpoint information ED1'is stored individually, and when the synthetic game video CV1 is output, the composite video information CV1'is generated based on the stored video information and the viewpoint information. You may.

[Modification 3]
In the fourth embodiment, the game image seen by the player A during play and the game image seen by the player B during play are the same game images generated by the game device GM, and are viewed by the player B during play. An example of the viewpoint confirmation system 1D that adds the viewpoint information of the player A to the game video has been described. Here, a configuration will be described in which the game device GM is used individually for each player, and similarly, the viewpoint information of the player A is added to the game video viewed by the player B during play.

FIG. 23 is a schematic diagram showing an outline of the viewpoint confirmation system 1G according to the modified example 3. Player A plays a game using the game device GMa and the display device D1. The display device D1 displays a game image output from the game device GMa. The viewpoint detection device EY detects the viewpoint position of the user of the display device D1 and outputs the viewpoint information indicating the detected viewpoint position to the video processing device 100C. On the other hand, the player B plays a game using the game device GMb and the display device D3. The game video output from the game device GMb is input to the video processing device 100C. The video processing device 100C generates a synthetic game video in which viewpoint information is added to the game video, and outputs the composite game video to the display device D3. The display device D3 used by the player B displays a synthetic game video in which the viewpoint information of the player A is added to the game video.

Player A and player B are, for example, players who play against each other in a battle game. For example, the game device GMa used by the player A and the game device GMb used by the player B are connected by a communication network, and a network battle is performed. For example, the game video output from the game device GMa and the game video output from the game device GMb are the same game video based on the same game space.

FIG. 24 is a diagram showing a usage example of the viewpoint confirmation system 1G shown in FIG. 23. The user of the display device D1 is the player A. On the other hand, the user of the display device D3 is the player B. A game image from the game device GMa is input to the display device D1. On the other hand, the display device D3 is input with a synthetic game image in which the viewpoint information of the player A is added to the game image from the game device GMb. For example, the viewpoint confirmation system 1G displays a game image on the display device D1 and displays a synthetic game image on which a mark based on the viewpoint information of the player A is synthesized on the display device D3 during game play. As described above, this viewpoint confirmation system 1G can be used in the same manner as the usage example (see FIG. 13) of the viewpoint confirmation system 1D according to the fourth embodiment.

FIG. 25 is a block diagram showing an example of the configuration of the viewpoint confirmation system 1G according to the modified example 3. The viewpoint confirmation system 1G includes a game device GMa, a display device D1, a viewpoint detection device EY, a game device GMb, a display device D3, and a video processing device 100C. In FIG. 25, the same reference numerals are given to the configurations corresponding to the respective parts of FIG.

The game device GMa performs game processing according to the operation of the player A on a controller (input device) (not shown), and outputs the game video PV0a according to the progress of the game to the display device D1. The display device D1 displays the game video PV0a input from the game device GMa. That is, the game video PV0a corresponds to the first video PV1 in FIG. On the other hand, the game device GMb performs game processing according to the operation of the player B on a controller (input device) (not shown), and outputs the game video PV0b according to the progress of the game to the video processing device 100C. That is, the game video PV0b corresponds to the second video PV2 in FIG.

The viewpoint detection device EY detects the viewpoint position of the user of the display device D1 and outputs the viewpoint information ED1 indicating the detected viewpoint position to the video processing device 100C. The video processing device 100C outputs a synthetic game video CV1 based on the game video PV0b (corresponding to the second video PV2) input from the game device GMb and the viewpoint information ED1 input from the viewpoint detection device EY to the display device D3. do. As a result, the display device D3 used by the player B can display the game video (synthetic game video CV1) to which the viewpoint information of the player A is added during play.

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to the above-described embodiments, and includes designs and the like within a range that does not deviate from the gist of the present invention. For example, the configurations described in the above-described first to fifth embodiments and the first to third modifications can be arbitrarily combined.

The game device GM may be, for example, a game device installed in an e-sports school, or a game device (game machine, etc.) owned by an instructor or student attending the school. It may be. For example, the game device GM may be a stationary game machine or a portable game machine. Further, the game device GM is not limited to the device dedicated to the game, and may be a smartphone or a personal computer capable of executing the application program of the game.

Further, a program for realizing the functions of the above-mentioned video processing devices 100, 100B, 100C, 100E, and 100F is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into the computer system. , You may perform the processing as the image processing apparatus 100, 100B, 100C, 100E, 100F by executing. Here, "loading and executing a program recorded on a recording medium into a computer system" includes installing the program in the computer system. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. Further, the "computer system" may include a plurality of computer devices connected via a network including a communication line such as the Internet, WAN, LAN, and a dedicated line. Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, and a storage device such as a hard disk built in a computer system. As described above, the recording medium in which the program is stored may be a non-transient recording medium such as a CD-ROM. The recording medium also includes an internal or external recording medium accessible from the distribution server for distributing the program. The code of the program stored in the recording medium of the distribution server may be different from the code of the program in a format that can be executed by the terminal device. That is, the format stored in the distribution server does not matter as long as it can be downloaded from the distribution server and installed in a form that can be executed by the terminal device. It should be noted that the configuration in which the program is divided into a plurality of parts and downloaded at different timings and then combined by the terminal device, or the distribution server for distributing each of the divided programs may be different. Furthermore, a "computer-readable recording medium" is a volatile memory (RAM) inside a computer system that serves as a server or client when a program is transmitted via a network, and holds the program for a certain period of time. It shall include things. Further, the above program may be for realizing a part of the above-mentioned functions. Further, a so-called difference file (difference program) may be used, which can realize the above-mentioned function in combination with a program already recorded in the computer system.

Further, some or all the functions of the above-mentioned video processing devices 100, 100B, 100C, 100E, and 100F may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each of the above-mentioned functions may be made into a processor individually, or a part or all of them may be integrated into a processor. Further, the method of making an integrated circuit is not limited to the LSI, and may be realized by a dedicated circuit or a general-purpose processor. Further, when an integrated circuit technology that replaces an LSI appears due to advances in semiconductor technology, an integrated circuit based on this technology may be used.

The above-mentioned synthetic game video is not limited to the mark based on the viewpoint information, and may be a video in which a heat map, a gaze plot, a focus map, a beesworm, or the like based on the viewpoint information is added to the game video. Further, the above-mentioned synthetic video information is not limited to the mark based on the viewpoint information, and may be information obtained by adding a heat map, a gaze plot, a focus map, a beesworm, or the like based on the viewpoint information to the video information.

[Additional Notes]
From the above description, the present invention can be grasped as follows, for example. Reference numerals in the accompanying drawings are added in parentheses for convenience in order to facilitate understanding of the present invention, but the present invention is not limited to the illustrated embodiment.

(Appendix 1) The viewpoint confirmation system (1, 1B, 1E, 1F) according to one aspect of the present invention includes a first display device (D1) for displaying a game image output from the game device (GM) and the above-mentioned. A viewpoint confirmation system including a viewpoint detection device (EY) for detecting the viewpoint position of a target person (for example, a player, a commentator, etc.) facing the display screen of the first display device, and the output game image ( A distribution device (DT) that distributes PV0) into at least a first image (PV1) for inputting to the first display device and a second image (PV2) whose input destination is different from that of the first image, and the first image. 2 Generation unit (110) for generating synthetic video information (CV1') by adding viewpoint information (ED1 (ED1')) indicating a viewpoint position detected by the viewpoint detection device to video information (PV2') based on video. ), A video output unit (130) that outputs a composite game video (CV1) based on the composite video information, and information including video information based on the second video and the viewpoint information, or the composite video information is stored. It includes a storage control unit (120) for storing in the unit, and a video processing device (100, 100B, 100E, 100F) having the storage control unit (120).

According to the configuration of Appendix 1, a target person (for example, a target person) who performs an activity during an activity (for example, game play or a game commentary) performed while watching a first display device for displaying a game image output from the game device GM. By recording a mark or the like based on the viewpoint information of the player or the commentator together with the synthetic game video or the game video synthesized with the game video by a video processing device other than the first display device, the first display device is used during the activity. It is possible to prevent the display of marks and the like based on viewpoint information. Therefore, it is possible to confirm the viewpoint of the target person during the activity without interfering with the activity of the target person (for example, game play or game play).

(Appendix 2) Further, one aspect of the present invention is the viewpoint confirmation system (1, 1B, 1F) described in Appendix 1, wherein the first display device is from the game image output from the game device. The distributed first video and the synthetic game video output from the video output unit can be switched and displayed.

According to the configuration of Appendix 2, the target person (for example, a player or a commentator) performs an activity (for example, a game play or a game commentary) while looking at the first display device, and sees the same first display device after the activity. However, since the synthetic game image can be confirmed, the viewpoint of the subject during the activity can be confirmed with the minimum system of one display device.

(Appendix 3) Further, one aspect of the present invention is the viewpoint confirmation system (1B, 1F) according to the appendix 1 or 2, wherein the distribution device further obtains the output game image and a third image. A second display device (D2) that distributes to (PV3) and displays the third image is provided.

According to the configuration of Appendix 3, a target person who performs an activity (game live) while watching the game video on the first display device in a usage situation in which the player is playing the game while watching the game video on the second display device. The viewpoint of the (live player) can be confirmed on the same first display device after the activity (after the game live).

(Appendix 4) Further, one aspect of the present invention is the viewpoint confirmation system (1, 1B, 1E, 1F) according to any one of the appendices 1 to 3, wherein the memory control unit is the generation unit. The synthetic video information generated in 1 is stored in the storage unit, and the video output unit outputs the synthetic game video based on the synthetic video information stored in the storage unit.

According to the configuration of Appendix 4, it is possible to store the composite video information obtained by synthesizing the mark or the like based on the viewpoint information of the target person (for example, the player or the commentator) performing the activity and the game video, and output it later. Therefore, after the activity, the target person can confirm the viewpoint of the target person during the activity.

(Appendix 5) Further, one aspect of the present invention is the viewpoint confirmation system (1, 1B, 1E, 1F) according to any one of the appendices 1 to 3, wherein the memory control unit is the second. The video information based on the video and the viewpoint information are individually stored in the storage unit, and the generation unit adds the viewpoint information stored in the storage unit to the video information stored in the storage unit. Information is generated, and the video output unit outputs the synthetic game video based on the synthetic video information generated by the generation unit.

According to the configuration of Appendix 5, the viewpoint information of the target person (for example, the player or the commentator) performing the activity and the game video are individually stored, and the viewpoint information is stored after the activity (for example, after the play or the commentary). Since it is possible to output a synthetic game video in which a mark or the like based on the above and the game video are combined, the target person himself / herself can confirm the viewpoint of the target person during the activity after the activity.

(Appendix 6) Further, the viewpoint confirmation system (1C, 1D, 1E, 1F) according to one aspect of the present invention includes a first display device (D1) for displaying a game image output from the game device (GM). A viewpoint confirmation system including a viewpoint detection device (EY) for detecting the viewpoint position of a target person (for example, a player, a live person, etc.) facing the display screen of the first display device, and the output game. A distribution device (DT) that distributes an image (PV0) into at least a first image (PV1) for inputting to the first display device and a second image (PV2) whose input destination is different from that of the first image. A generator that generates synthetic video information (CV1') by adding viewpoint information (ED1 (ED1')) indicating a viewpoint position detected by the viewpoint detection device to video information (PV2') based on the second video. The video processing device (100C, 100E, 100F) having the (110) and the video output unit (130) for outputting the synthetic game video (CV1) based on the synthetic video information is different from the first display device. A third display device (D3) is provided, the first display device displays the first image, and the third display device displays the synthetic game image.

According to the configuration of Appendix 6, a target person (for example, a player or a live commentary) who performs an activity during an activity (game play or a game live commentary) performed while watching a first display device displaying a game image output from the game device. While preventing the first display device from displaying the synthetic game video that combines the mark etc. based on the viewpoint information of the person) with the game video, for third parties other than the target person (for example, instructors and students) Since the viewpoint information can be confirmed during the activity of the target person on the third display device, the viewpoint of the target person during the activity can be confirmed without interfering with the activity of the target person (for example, game play or live game). Can be done.

(Appendix 7) Further, one aspect of the present invention is the viewpoint confirmation system (1E, 1F) described in the appendix 6, in which the video processing apparatus obtains video information based on the second video and the viewpoint information. A storage control unit (120) for storing individual information individually included or the synthetic video information in a storage unit is provided, and the video output unit is a synthetic game video based on the individual information stored in the storage unit, or the above-mentioned. Outputs a synthetic game video based on synthetic video information.

According to the configuration of Appendix 7, the viewpoint information of the target person (for example, the player or the commentator) performing the activity and the game video are individually stored, or the composite video information obtained by synthesizing the mark or the like based on the viewpoint information and the game video. Can be stored and the synthetic game video can be output after the activity (for example, after playing or after the live performance), so that the target person himself can confirm the viewpoint of the target person during the activity after the activity.

(Appendix 8) Further, one aspect of the present invention is the viewpoint confirmation system (1B, 1F) according to any one of the appendices 1 to 7, wherein the viewpoint confirmation system is a plurality of subjects (for example, a plurality of subjects (for example). A plurality of viewpoint detection devices (EYa, EYb) for detecting the viewpoint positions of the players A and B) are used, and the distribution device is a game image (PV0a, PV0b) corresponding to each of the plurality of target persons. ) Is included in the individual distribution device (DTa, DTb), and the generation unit is based on the plurality of viewpoint information (ED1a, ED1b) corresponding to each of the plurality of subjects and the game image. The composite composite video information (CV10') is generated, and the video output unit outputs the composite composite game video (CV10) based on the composite composite video information generated by the generation unit.

According to the configuration of Appendix 8, a game played while a plurality of target persons (for example, player A and player B) are watching each of a plurality of first display devices for displaying a game image output from each game device. During play, a composite synthetic game video obtained by synthesizing a mark or the like based on the viewpoint information of each of a plurality of target persons (for example, player A and player B) with the game video is displayed on a third display device other than the first display device. Since it is displayed to a third party (for example, an instructor or a student), the viewpoints of a plurality of target persons during the activity can be viewed by the third party (for example, game play) without interfering with the activity of the target person (for example, game play). For example, it can be made visible to teachers and students).

1,1B, 1C, 1D, 1E, 1F, 1G viewpoint confirmation system, GM game device, DT distribution device, D1, D2, D3 display device, EY viewpoint detection device, 100, 100B, 100C, 100E, 100F video processing device , 110 generator, 120 storage control, 130 video output

Claims (8)

A viewpoint confirmation system including a first display device for displaying a game image output from a game device and a viewpoint detection device for detecting the viewpoint position of a target person facing the display screen of the first display device.
A distribution device that distributes a game image output from the game device to at least a first image to be input to the first display device and a second image whose input destination is different from that of the first image.
A generator that generates synthetic video information by adding viewpoint information indicating the viewpoint position detected by the viewpoint detection device to the video information based on the second video, and a video that outputs a synthetic game video based on the synthetic video information. An image processing device having an output unit, information including image information based on the second image and the viewpoint information, or a storage control unit for storing the synthesized image information in the storage unit.
A viewpoint confirmation system equipped with. The first display device can switch between the first video distributed from the game video output from the game device and the synthetic game video output from the video output unit.
The viewpoint confirmation system according to claim 1. The distribution device further distributes the game video output from the game device to the third video.
A second display device for displaying the third image is provided.
The viewpoint confirmation system according to claim 1 or 2. The storage control unit stores the synthetic video information generated by the generation unit in the storage unit.
The video output unit outputs the synthetic game video based on the synthetic video information stored in the storage unit.
The viewpoint confirmation system according to any one of claims 1 to 3. The storage control unit individually stores the video information based on the second video and the viewpoint information in the storage unit.
The generation unit generates synthetic video information in which the viewpoint information stored in the storage unit is added to the video information stored in the storage unit.
The video output unit outputs the synthetic game video based on the synthetic video information generated by the generation unit.
The viewpoint confirmation system according to any one of claims 1 to 3. A viewpoint confirmation system including a first display device for displaying a game image output from a game device and a viewpoint detection device for detecting the viewpoint position of a target person facing the display screen of the first display device.
A distribution device that distributes a game image output from the game device to at least a first image to be input to the first display device and a second image whose input destination is different from that of the first image.
A generator that generates synthetic video information by adding viewpoint information indicating the viewpoint position detected by the viewpoint detection device to the video information based on the second video, and a video that outputs a synthetic game video based on the synthetic video information. An output unit, a video processing device having, and
A third display device different from the first display device is provided.
The first display device displays the first image and displays the first image.
The third display device displays the synthetic game image.
Viewpoint confirmation system. The video processing apparatus includes a storage control unit that stores individual information including video information based on the second video and viewpoint information individually, or synthetic video information in a storage unit.
The video output unit outputs a synthetic game video based on the individual information stored in the storage unit or a synthetic game video based on the synthetic video information.
The viewpoint confirmation system according to claim 6. The viewpoint confirmation system uses a plurality of viewpoint detection devices that detect the viewpoint positions of a plurality of subjects.
The distribution device is configured to include an individual distribution device that distributes game images corresponding to each of the plurality of target persons.
The generation unit generates composite composite video information based on the plurality of viewpoint information corresponding to each of the plurality of target persons and the game video.
The video output unit outputs a composite composite game video based on the composite composite video information generated by the generator.
The viewpoint confirmation system according to any one of claims 1 to 7.

Images