JP2021137425A - View point confirmation system - Google Patents

Abstract

To make it possible to confirm a view point of a target person in activity without disturbing the target person's activity.SOLUTION: A view point confirmation system comprising a first display device on which game video output from a game device is displayed and a view point detection device for detecting a view point position of a target person facing a display screen of the first display device comprises: a distribution device for distributing the output game video into first video to be input to at least the first display device and second video to be input to a destination differing from that of the first video; and a video processing device. The video processing device includes: a generation unit for generating synthesized video information obtained by adding view point information showing the view point position detected by the view point detection device to video information based on the second video; a video output unit for outputting synthesized game video based on the synthesized video information; and a storage control unit for storing the synthesized video information or information including the video information based on the second video and the view point information in a storage part.SELECTED DRAWING: Figure 1

Description

The present invention relates to a viewpoint confirmation system.

There is a viewpoint detection (line-of-sight detection) technology (so-called eye tracking) that detects the position of the viewpoint of a person who is looking at the screen. For example, by detecting the viewpoint of the player during game play and displaying the detection result on the game image in real time, it is possible to confirm where the player is watching while playing (for example, Non-Patent Document 1). ).

[Special project] "Dissecting the eyes of the strong men! Searching for the secret of strength with" DFF "eye tracking", [online], [Search on December 13, 2019], Internet <URL: https: // www .. Famitsu. com / matome / dff / diary 54. html>

However, if the viewpoint detection result of the viewpoint detection target person (player, etc.) is displayed in real time on the game image viewed by the target person during game play, the viewpoint being played by the target person can be confirmed. On the other hand, since it becomes difficult for the target person to check the game play status, the target person's activity (game play, etc.) may be hindered.

One of the objects of the present invention is to provide a viewpoint confirmation system capable of confirming the viewpoint of the subject during the activity without interfering with the activity of the subject.

Another object of the present invention is to provide a viewpoint confirmation system capable of exerting the effects described in the embodiments described later.

In order to solve the above-mentioned problems, one aspect of the present invention is a first display device on which a game image output from the game device is displayed, and a viewpoint position of a target person facing the display screen of the first display device. This is a viewpoint confirmation system including a viewpoint detection device for detecting information, and a second image in which the output game image is input to at least the first display device and a second image in which the input destination is different from the first image. A distribution device that distributes to the video, a generator that generates composite video information by adding viewpoint information indicating the viewpoint position detected by the viewpoint detection device to the video information based on the second video, and the composite video information. Video processing including a video output unit that outputs a synthetic game video based on the above, information including video information based on the second video and the viewpoint information, or a storage control unit that stores the synthetic video information in a storage unit. It is a viewpoint confirmation system equipped with a device.

Further, one aspect of the present invention is a first display device for displaying a game image output from the game device, and a viewpoint detection device for detecting the viewpoint position of a target person facing the display screen of the first display device. A viewpoint confirmation system including, which distributes the output game video to at least a first video for inputting to the first display device and a second video whose input destination is different from that of the first video. And a generator that generates synthetic video information by adding viewpoint information indicating the viewpoint position detected by the viewpoint detection device to the video information based on the second video, and outputs a synthetic game video based on the synthetic video information. A video processing device having a video output unit, a third display device different from the first display device, and the first display device display the first video and the third display device. Is a viewpoint confirmation system that displays the composite game video.

The schematic diagram which shows the outline of the viewpoint confirmation system which concerns on 1st Embodiment. The figure which shows the use example of the viewpoint confirmation system which concerns on 1st Embodiment. The block diagram which shows an example of the structure of the viewpoint confirmation system which concerns on 1st Embodiment. The block diagram which shows an example of the structure of the image processing apparatus which concerns on 1st Embodiment. The schematic diagram which shows the outline of the viewpoint confirmation system which concerns on 2nd Embodiment. The figure which shows the use example of the viewpoint confirmation system which concerns on 2nd Embodiment. The block diagram which shows an example of the structure of the viewpoint confirmation system which concerns on 2nd Embodiment. The schematic diagram which shows the outline of the viewpoint confirmation system which concerns on 3rd Embodiment. The figure which shows the use example of the viewpoint confirmation system which concerns on 3rd Embodiment. The block diagram which shows an example of the structure of the viewpoint confirmation system which concerns on 3rd Embodiment. The block diagram which shows an example of the structure of the image processing apparatus which concerns on 3rd Embodiment. The schematic diagram which shows the outline of the viewpoint confirmation system which concerns on 4th Embodiment. The figure which shows the use example of the viewpoint confirmation system which concerns on 4th Embodiment. The schematic diagram which shows the outline of the viewpoint confirmation system which concerns on 5th Embodiment. The figure which shows the use example of the viewpoint confirmation system which concerns on 5th Embodiment. The block diagram which shows an example of the structure of the viewpoint confirmation system which concerns on 5th Embodiment. The block diagram which shows an example of the structure of the image processing apparatus which concerns on 5th Embodiment. The schematic diagram which shows the outline of the viewpoint confirmation system which concerns on modification 1. The figure which shows the use example of the viewpoint confirmation system which concerns on modification 1. The block diagram which shows an example of the structure of the viewpoint confirmation system which concerns on modification 1. The block diagram which shows an example of the structure of the image processing apparatus which concerns on modification 1. The block diagram which shows an example of the structure of the image processing apparatus which concerns on modification 2. The schematic diagram which shows the outline of the viewpoint confirmation system which concerns on modification 3. The figure which shows the use example of the viewpoint confirmation system which concerns on modification 3. The block diagram which shows an example of the structure of the viewpoint confirmation system which concerns on modification 3.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
First, the first embodiment of the present invention will be described.
In recent years, so-called e-sports (Electronic Sports), in which game competitions are regarded as competitions, have become popular. Along with this, in order to develop human resources who are active in the e-sports world, for example, it is necessary to have a training place where game players can train practical skills and learn necessary skills and abilities (skills). ing. For example, it is assumed that an instructor trains students (players, etc.) in an e-sports school or the like. Players are divided into beginners, intermediate players, advanced players, etc., depending on the level of skill and ability. Smooth play operation is one of the factors that divide the stages, but it is also an important factor to accurately grasp the game situation for each scene. For example, by improving the ability to grasp the game situation, it is possible to improve the skills and abilities of beginners to intermediate players and intermediate players to advanced players. In order to grasp the game situation, it is important to check which part of the game image displayed on the game screen at what timing according to the game situation. The game video contains a lot of information. Advanced players are efficiently playing the game by efficiently confirming and grasping necessary information according to the game situation and performing play operations while anticipating the next development. By acquiring the viewpoint position of the player with respect to the game screen, it is expected that the skill and ability of the player will be improved by evaluating and improving which information is confirmed at what timing regarding the game situation. The viewpoint confirmation system according to the present embodiment is a system that enables confirmation of the viewpoint position of the player with respect to the game screen. According to this viewpoint confirmation system, it is possible to evaluate and improve the viewpoint position of the player with respect to the game screen, and it is possible to improve the skill and ability of the player. Hereinafter, the viewpoint confirmation system according to the present embodiment will be described in detail.

FIG. 1 is a schematic diagram showing an outline of the viewpoint confirmation system 1 according to the present embodiment. In the viewpoint confirmation system 1, for example, a player who plays a game on the game device GM sees and plays a game image displayed on the display screen (game screen) of the display device D1. The game image is an image showing a game situation that progresses according to the player playing. The game video output from the game device GM is distributed to two systems by the distribution device DT. The distribution device DT outputs one of the distributed game images to the display device D1 and outputs the other game image to the image processing device 100. The viewpoint detection device EY detects the viewpoint position of the player facing the display screen of the display device D1 and outputs the viewpoint information indicating the detected viewpoint position to the video processing device 100. For example, the viewpoint information output by the viewpoint detection device EY is information including the position information of the detected viewpoint position. The image processing device 100 generates a synthetic game image in which viewpoint information is added to the game image and stores it in a storage unit. Further, the video processing device 100 can output the stored synthetic game video to the display device D1. The composite game video in which the viewpoint information is added to the game video is, for example, a video in which a mark based on the viewpoint information is synthesized with the game video. This game image and viewpoint information are synchronized on the time axis. That is, the synthetic game video is a video in which the game video being played by the player and the viewpoint position of the player being played are displayed at the same time. The display device D1 switches between the game video input from the game device GM via the distribution device DT and the composite game video synthesized by the video processing device 100 by switching the input of the video to be displayed on the display screen. Can be displayed. Further, although the details will be described later, the target for which the viewpoint detection device EY detects the viewpoint position is the actual situation by looking at the game image displayed on the display screen (game screen) of the display device D1 instead of the player playing the game. It may be a commentator who does. That is, it can also be used to confirm and evaluate the position and movement of the viewpoint of the commentator who is commenting on the game play.

FIG. 2 is a diagram showing a usage example of the viewpoint confirmation system 1 shown in FIG. The user of the display device D1 (the person facing the display screen of the display device D1) is, for example, a player. A game image and a composite game image are input to the display device D1. For example, the viewpoint confirmation system 1 displays a game image from the game device GM on the display device D1 during game play, and a mark based on the player's viewpoint information is displayed after play (after the end of play or after interruption of play). The synthesized synthetic game image is displayed on the display device D1. As a result, the player plays while watching the game image from the game device GM in real time during play, and the position of the viewpoint of the player during play is displayed in the synthetic game image in which the mark based on the viewpoint information is synthesized after the play. The movement can be confirmed and evaluated. The viewpoint confirmation and evaluation may be performed by the player himself / herself, or may include a lecturer or another player in place of or in addition to the player.

Next, the configuration of the viewpoint confirmation system 1 will be described in detail with reference to FIGS. 3 and 4.
FIG. 3 is a block diagram showing an example of the configuration of the viewpoint confirmation system 1 according to the present embodiment. The viewpoint confirmation system 1 includes a game device GM, a distribution device DT, a display device D1, a viewpoint detection device EY, and a video processing device 100.

The game device GM is configured to include a CPU (Central Processing Unit), and processes a game by executing a game program. For example, the game device GM performs game processing according to a player's operation on a controller (input device) (not shown), and outputs a game video PV0 according to the progress of the game to the distribution device DT.

The distribution device DT is a device that distributes the game video output from the game device GM to a plurality of game videos. Each of the distributed game images is the same image, but the resolution and the effective display area of the display screen may be different according to the distribution destination device. Specifically, when the game video PV0 output from the game device GM is input, the distribution device DT distributes the input game video PV0 into two systems, the first video PV1 and the second video PV2. Output. The first video PV1 is input to the display device D1. The second video PV2 is input to the video processing device 100. For example, the game video PV0, the first video PV1, and the second video PV2 are transmitted as video signals corresponding to HDMI (registered trademark). The game video PV0, the first video PV1, and the second video PV2 are not limited to being transmitted by a video signal corresponding to HDMI, and include DVI (Digital Visual Interface), composite video, and composite video. A video signal corresponding to an arbitrary standard, such as an S video separated into two systems of a brightness signal and a color signal, may be transmitted.

The viewpoint detection device EY detects (eye tracking) the viewpoint position of the user (for example, the player) of the display device D1. For example, the viewpoint detection device EY includes a camera that captures a direction facing the display screen of the display device D1, and detects a viewpoint position on the display screen based on the captured image. Any method can be used as the method for detecting the viewpoint position. For example, the viewpoint position may be detected based on the position of the iris with respect to the position of the inner corner of the eye of the person captured by the visible light camera, or the position of the corneal reflex formed by irradiating infrared light may be used as the reference position with respect to the reference position. The viewpoint position may be detected based on the position of the pupil. For example, the viewpoint detection device EY continuously detects and tracks the viewpoint position of the player during play at a predetermined sampling cycle. The viewpoint detection device EY outputs the viewpoint information ED1 indicating the detected viewpoint position to the image processing device 100. For example, the viewpoint information is transmitted using USB (Universal Serial Bus).

The video processing device 100 stores the game video and the viewpoint information, synthesizes the game video and the viewpoint information, and the like. For example, the video processing device 100 is a computer device including a CPU (Central Processing Unit) and the like.
FIG. 4 is a block diagram showing an example of the configuration of the video processing device 100 according to the present embodiment. The video processing device 100 includes a generation unit 110, a storage control unit 120, and a video output unit 130.

The second video PV2 output from the distribution device DT and the viewpoint information ED1 output from the viewpoint detection device EY are input to the generation unit 110. The generation unit 110 converts the input second video PV2 into video information PV2'which can be processed into video, and the viewpoint position detected by the viewpoint detection device EY in the video information PV2' based on the second video PV2. Synthetic video information CV1'with the viewpoint information ED1 indicating the above is generated. More specifically, the composite video information CV1'is generated by adding the viewpoint information ED1', which is data-converted so that the viewpoint information ED1 can be added to the video information PV2', to the video information PV2'. The composite video information in which the viewpoint information is added to the video information is, for example, information obtained by synthesizing a mark based on the viewpoint information with the video information. For example, the composite video information CV1'is video information in which a mark based on the viewpoint information ED1'is combined with the video information PV2' based on the second video PV2, and the video information PV2'and the viewpoint information ED1' are on the time axis. Is synchronized with. The generation unit 110 outputs the generated composite video information CV1'to the storage control unit 120.

The storage control unit 120 stores the synthetic video information CV1'generated by the generation unit 110 in a storage unit (not shown). The storage unit includes, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), an SSD (Solid State Drive), a flash ROM (Read-Only Memory), and the like. The storage unit may be built in the video processing device 100, or may be an external storage device connected by USB or the like.

The video output unit 130 outputs a synthetic game video CV1 based on the synthetic video information CV1'stored in the storage unit by the storage control unit 120. For example, the video output unit 130 converts the composite video information CV1'to HDMI-compatible synthetic game video CV1 and outputs it. As a result, the synthetic game video CV1 is transmitted from the video processing device 100 to the display device D1 as a video signal corresponding to HDMI. The composite game video CV1 is not limited to being transmitted as a video signal corresponding to HDMI, and DVI (Digital Visual Interface), composite video, and composite video are separated into two systems, a brightness signal and a color signal. It may be transmitted by a video signal corresponding to an arbitrary standard such as the S video.

The display device D1 is a display for displaying information such as an image or text, and includes, for example, a liquid crystal display panel, an organic EL (Electroluminescence) display panel, and the like. The display device D1 is provided with two or more video input terminals, and can switch and display two or more input videos. For example, the first video PV1 output from the distribution device DT and the synthetic game video CV1 output from the video processing device 100 are input to the video input terminal of the display device D1. The display device D1 can switch between the first video PV1 distributed by the distribution device DT and the synthetic game video CV1 output from the video processing device 100 for display. The first video PV1 is a video corresponding to the game video P0 output from the game device GM.

As described above, the viewpoint confirmation system 1 according to the present embodiment is an example of the display device D1 (first display device) on which the game video PV0 (more specifically, the first video PV1) output from the game device GM is displayed. ), And a viewpoint detection device EY that detects the viewpoint position of a player (an example of a target person) facing the display screen of the display device D1. Further, the viewpoint confirmation system 1 distributes the game video PV0 output from the game device GM into at least the first video PV1 for inputting to the display device D1 and the second video PV2 whose input destination is different from that of the first video PV1. The distribution device DT and the image processing device 100 are provided. The video processing device 100 adds the viewpoint information ED1 (more specifically, the viewpoint information ED1') indicating the viewpoint position detected by the viewpoint detection device EY to the video information PV2'based on the second video PV2, and adds the composite video information CV1'. The generation unit 110 that generates the And have.

As a result, the viewpoint confirmation system 1 displays the player's viewpoint information in the image processing device 100 other than the display device D1 during the play of the game while watching the display device D1 that displays the game image output from the game device GM. By recording as a composite game video combined with the game video, it is possible to prevent the display device D1 from displaying the mark based on the viewpoint information during play. Therefore, it is possible to confirm the viewpoint of the player during play without interfering with the player's play of the game.

For example, the display device D1 (an example of the first display device) switches between the first video PV1 distributed from the game video PV0 output from the game device GM and the synthetic game video CV1 output from the video output unit 130. Can be displayed. For example, the display device D1 includes a plurality of video inputs (for example, 2ch (channel) video inputs). The video input may be switched manually (for example, by operating the input switching button) or automatically (for example, by detecting a video signal and switching).

As a result, in the viewpoint confirmation system 1, the player of the game can play the game while looking at the display device D1 and can check the composite game image while looking at the same display device D1 after the play. The system allows you to check the viewpoint of the player who is playing.

Further, the storage control unit 120 stores the composite video information CV1'generated by the generation unit 110 in the storage unit. The video output unit 130 outputs a synthetic game video CV1 based on the synthetic video information CV1'stored in the storage unit.

As a result, the viewpoint confirmation system 1 can store the composite video information obtained by synthesizing the mark based on the player's viewpoint information and the game video and output it later, so that the target person himself / herself is active after playing. You can check the viewpoint of the target person.

In addition, game commentators also play an important role in e-sports, and their training is also an issue. Like game players, game commentators have stages in their abilities. Although it depends on the speaking ability of each individual, it is often difficult to perform the actual situation well because the point of checking the situation according to the game situation of each game is not known. Therefore, it can be expected that the live performance ability will be improved by confirming and imitating the viewpoint position of the live broadcaster, who is said to have high ability, in the actual live performance. Therefore, an example of applying the viewpoint confirmation system 1 according to the present embodiment to the confirmation of the viewpoint of a game commentator will be described.
In this example, the user of the display device D1 shown in FIGS. 1 to 3 described above is a live person instead of the player. The viewpoint detection device EY detects the viewpoint position of the actual person facing the display screen of the display device D1 and outputs the viewpoint information indicating the detected viewpoint position to the video processing device 100. In this case, the game video is a video showing a game situation in which the game device GM automatically plays (automatically plays) without a player's play operation. In other words, the commentator in this example is a commentator at the time of automatic play, for example, a student practicing the commentary while watching the game video by automatic play, or a lecturer demonstrating the commentary while watching the game video by automatic play. .. The display device D1 switches between the game video input from the game device GM via the distribution device DT and the composite game video synthesized by the video processing device 100 by switching the input of the video to be displayed on the display screen. Can be displayed.
As a result, the viewpoint confirmation system 1 puts a mark based on the viewpoint information of the commentator on the image other than the display device D1 during the actual situation of the game played while watching the display device D1 that displays the game image output from the game device GM. By recording as a composite game video combined with the game video on the processing device 100, it is possible to prevent the display device D1 from displaying the mark based on the viewpoint information during the actual situation. Therefore, it is possible to confirm the viewpoint of the commentator during the commentary without interfering with the commentary of the game by the commentator.

[Second Embodiment]
Next, a second embodiment of the present invention will be described.
In the present embodiment, a configuration example in which a display device used by the player is separately provided will be described with respect to the first embodiment in which the position and movement of the viewpoint of the actual person are confirmed and evaluated.
FIG. 5 is a schematic view showing an outline of the viewpoint confirmation system 1B according to the present embodiment. When confirming and evaluating the position and movement of the viewpoint of the commentator in the first embodiment, the commentary was performed while watching the game image by automatic play. In the present embodiment, the game can be played while watching the game image actually played by the player instead of the automatic play. Therefore, the viewpoint confirmation system 1B has a configuration in which the display device D2 used by the player is added to the viewpoint confirmation system 1 according to the first embodiment. That is, the display device D1 is used by the commentator, and the display device D2 is used by the player.
Further, since the distribution device DT needs to distribute the game video to the added display device D2, the game video output from the game device GM is distributed to three systems. That is, the distributed game video is output to the display device D1, the display device D2, and the video processing device 100, respectively.

The player plays the game by watching the game image displayed on the display device D2. The commentator sees the game image displayed on the display device D1 (the same game image as the game image displayed on the display device D2) and plays the game. The viewpoint detection device EY detects the viewpoint position of the user of the display device D1 and outputs the viewpoint information indicating the detected viewpoint position to the image processing device 100. The video processing device 100 can detect the viewpoint position of the commentator together with the game video, generate a synthetic game video in which the viewpoint information is added to the game video, and output it to the display device D1.

FIG. 6 is a diagram showing a usage example of the viewpoint confirmation system 1B shown in FIG. The user who uses the display device D1 is a commentator. A game image and a composite game image are input to the display device D1. On the other hand, the user who uses the display device D2 is a player. A game image is input to the display device D2. For example, the viewpoint confirmation system 1B displays the game image from the game device GM on the display device D1 and the display device D2 during the game play (during the live game), and after the play (after the end of the play or after the interruption of the play). Displays a synthetic game image in which marks based on the player's viewpoint information are synthesized on the display device D1. As a result, the commentator performs the commentary while watching the game image from the game device GM in real time during the player's play (during the game commentary), and after the play, the commentary is a composite game video in which the mark based on the viewpoint information is synthesized. It is possible to confirm and evaluate the position and movement of the viewpoint of the commentator during the commentary. The viewpoint confirmation and evaluation may be performed by the commentator himself / herself, or may include a lecturer or another player in place of or in addition to the commentator. In addition, the player can play while watching the game image from the game device GM in real time during play.

Next, the configuration of the viewpoint confirmation system 1B will be described in detail with reference to FIG. 7.
FIG. 7 is a block diagram showing an example of the configuration of the viewpoint confirmation system 1B according to the present embodiment. In FIG. 7, the same configurations as those in FIG. 3 are designated by the same reference numerals, and the description thereof will be omitted. The configuration of the viewpoint confirmation system 1B shown in FIG. 7 is different from the configuration of the viewpoint confirmation system 1 shown in FIG. 3 in that the display device D2 is added. The distribution device DT distributes and outputs the game video PV0 input from the game device GM into three systems of the first video PV1, the second video PV2, and the third video PV3. The third video PV3 is input to the display device D2. Similar to the configuration shown in FIG. 3, the first video PV1 is input to the display device D1, and the second video PV2 is input to the video processing device 100.

As described above, in the viewpoint confirmation system 1B, the display device D1 is a display device for a live broadcaster, and the display device D2 is a display device for a player. The display device D1 for the live broadcaster can switch between the first video PV1 distributed by the distribution device DT and the synthetic game video CV1 output from the video processing device 100. On the other hand, the display device D2 for the player displays the third video PV3 (video corresponding to the game video P0 output from the game device GM) distributed by the distribution device DT.

Further, in the viewpoint confirmation system 1B, the distribution device DT adds the game video PV0 output from the game device GM to the first video PV1 and the second video PV2, and further distributes the game video PV3 to the third video PV3. Then, the display device D2 (an example of the second display device) displays the third video PV3.

As a result, the viewpoint confirmation system 1B displays the viewpoint of the commentator who plays the game while watching the game image on the display device D1 in the usage situation where the player is playing the game while watching the game image on the display device D2. It can be confirmed on the same first display device after the game is live.

[Third Embodiment]
Next, a third embodiment of the present invention will be described.
In the present embodiment, a configuration example in which a display device for confirming the viewpoint is separately provided with respect to the first embodiment will be described.
FIG. 8 is a schematic view showing an outline of the viewpoint confirmation system 1C according to the present embodiment. The viewpoint confirmation system 1C includes a display device D3 in addition to the display device D1 for the player, and the point that the synthesized game image is input to the display device D3 instead of the display device D1 is the viewpoint confirmation system shown in FIG. Different from 1. This display device D3 is used, for example, as a display device for teachers and students. As described in the first embodiment, the display device D1 can be used not only for the player but also for the live broadcaster at the time of automatic play.

FIG. 9 is a diagram showing a usage example of the viewpoint confirmation system 1C shown in FIG. The user of the display device D1 is a player or a commentator. A game image is input to the display device D1. On the other hand, the user of the display device D3 is a lecturer or a student. A synthetic game image is input to the display device D3. Here, the synthetic game video input to the display device D3 is a video in which viewpoint information is added in real time to the game video, and is real-time during play like the game video input to the display device D1. It is a displayed image (specifically, the synthetic game image input to the display device D3 is delayed by the time required for the process of adding the viewpoint information to the game image input to the display device D1). For example, the viewpoint confirmation system 1C displays a game image from the game device GM on the display device D1 during game play, and synthesizes a mark based on the viewpoint information of the player or the commentator on the display device D3. Display the game image. As a result, the player or the commentator plays or plays the game while watching the game image from the game device GM in real time, and the third party such as the lecturer or the student can see the viewpoint of the player or the commentator during the play or the commentary. Lectures can be advanced while confirming and evaluating the position and movement in real time. When the user of the display device D1 is a live person, the game image is an image showing a game situation that progresses according to the automatic play by the game device GM without the player's play operation.

Next, the configuration of the viewpoint confirmation system 1C will be described in detail with reference to FIGS. 10 and 11. FIG. 10 is a block diagram showing an example of the configuration of the viewpoint confirmation system 1C according to the present embodiment. In FIG. 10, the same configurations as those in FIG. 3 are designated by the same reference numerals, and the description thereof will be omitted. The configuration of the viewpoint confirmation system 1C shown in FIG. 10 is that the display device D3 is added to the configuration of the viewpoint confirmation system 1 shown in FIG. 3, and the input destination of the synthetic game video CV1 is replaced with the display device D1. The difference is that the display device D3 is used. Further, the video processing device 100C is different from the video processing device 100 described in the above embodiment in that the viewpoint information ED1 is added to the second video PV2 in real time.

FIG. 11 is a block diagram showing an example of the configuration of the video processing device 100C according to the present embodiment. The video processing device 100C includes a generation unit 110 and a video output unit 130. The generation unit 110 converts the input second video PV2 into video information PV2'which can be processed into video, and the viewpoint position detected by the viewpoint detection device EY in the video information PV2' based on the second video PV2. The composite video information CV1'added with the viewpoint information ED1 (more specifically, the viewpoint information ED1') is generated in real time in response to the input of the second video PV2 and the viewpoint information ED1 and output to the video output unit 130. The video output unit 130 outputs the composite game video CV1 based on the composite video information CV1'generated by the generation unit 110. For example, the video output unit 130 converts the composite video information CV1'to HDMI-compatible synthetic game video CV1 and outputs it. As a result, the display device D3 can display the synthetic game video CV1 output from the video processing device 100C during the player's play.

As described above, the viewpoint confirmation system 1C according to the present embodiment is an example of the display device D1 (first display device) on which the game video PV0 (more specifically, the first video PV1) output from the game device GM is displayed. ), And a viewpoint detection device EY that detects the viewpoint position of a target person, for example, a player or a live person facing the display screen of the display device D1. Further, the viewpoint confirmation system 1C distributes the game video PV0 output from the game device GM into at least the first video PV1 for inputting to the display device D1 and the second video PV2 whose input destination is different from that of the first video PV1. A distribution device DT, a video processing device 100C, and a display device D3 (an example of a third display device) different from the display device D1 are provided. The video processing device 100C is a composite video information CV1'in which the viewpoint information ED1 (more specifically, the viewpoint information ED1') indicating the viewpoint position detected by the viewpoint detection device EY is added to the video information PV2'based on the second video PV2. It has a generation unit 110 for generating the above, and a video output unit 130 for outputting the synthetic game video CV1 based on the synthetic video information CV1'generated by the generation unit 110. Then, the display device D1 displays the first video PV1. On the other hand, the display device D3 displays the synthetic game video CV1.

As a result, the viewpoint confirmation system 1C displays a mark based on the viewpoint information of the target person other than the display device D1 in the game play or the actual situation while watching the display device D1 that displays the game image output from the game device GM. The video processing device 100C of the above prevents the composite game video CV1 synthesized with the game video from being displayed on the display device D1, while the display device D3 is used for a third party (for example, a teacher or a student) other than the target person. So, since the viewpoint can be confirmed during the play or live of the target person, the viewpoint of the target person during play or live can be confirmed without interfering with the play or live of the game by the target person. be able to.

In the present embodiment, an example in which the display device D3 displays the synthetic game video CV1 in real time during the player's play has been described, but the synthetic video information CV1'is displayed in the same manner as in the first and second embodiments. You may store and display the synthetic game video CV1 based on the stored synthetic video information CV1'after playing. In this case, the video processing device 100C may have the same configuration as the video processing device 100 shown in FIG. 4, or may be combined with the configuration of the video processing device 100 shown in FIG.

Further, in the present embodiment, the input destination of the composite game video CV1 is only the display device D3, but by setting both the display device D1 and the display device D3, both the display device D1 and the display device D3 can input the composite game video. The configuration may be such that the CV1 can be displayed.

[Fourth Embodiment]
Next, a fourth embodiment of the present invention will be described.
FIG. 12 is a schematic view showing an outline of the viewpoint confirmation system 1D according to the present embodiment. The viewpoint confirmation system 1D has the same basic system configuration as the viewpoint confirmation system 1C according to the third embodiment, but is different from the player A who plays using the display device D1 (for example, an opponent). ) Is the user of the display device D3.

FIG. 13 is a diagram showing a usage example of the viewpoint confirmation system 1D shown in FIG. The user of the display device D1 is player A. On the other hand, the user of the display device D3 is the player B. Player A and player B are, for example, players who play against each other in a battle game. Each of player A and player B is a player who simultaneously plays the same game on the game device GM, and plays using each of two input devices (controllers and the like) connected to the game device GM. Further, this game is played by the player A and the player B in the same game space, and the game image seen by the player A during the play and the game image seen by the player B during the play are generated by the game device GM. It is assumed that the same game video can be used.

The game image from the game device GM is input to the display device D1. On the other hand, the display device D3 is input with a composite game video in which the viewpoint information indicating the viewpoint position of the player A is added to the game video. For example, the viewpoint confirmation system 1D displays a game image from the game device GM on the display device D1 during game play, and a composite game image in which a mark based on the viewpoint information of the player A is synthesized on the display device D3. Is displayed.

As a result, the player A plays while looking at the display device D1 that displays the game image output from the game device GM during play, while the player B who plays while looking at the display device D3 is playing while watching the player A. You can play while checking the viewpoint. For example, when the player B is a lecturer or an advanced player, it is possible to confirm the position and movement of the viewpoint of the player A while playing the game together with the player A and give an improvement instruction or the like. Further, when the player A is a lecturer or an advanced player and the player B is a student, the player B can predict the next action of the player A from the position and movement of the viewpoint of the player A and play the game. can. Player B can be expected to improve by playing while being aware of the points to be concerned about in each game scene.

[Fifth Embodiment]
Next, a fifth embodiment of the present invention will be described.
In this embodiment, an example of detecting the viewpoint positions of a plurality of players and synthesizing them with a game image will be described.
FIG. 14 is a schematic view showing an outline of the viewpoint confirmation system 1E according to the present embodiment. In the viewpoint confirmation system 1E, the player A who plays the game on the game device GMa sees and plays the game image displayed on the display device D1a, and the player B who plays the game on the game device GMb is displayed on the display device D1b. Watch the game video and play. The game video output from the game device GMa is distributed to two systems by the distribution device DTa, one game video is input to the display device D1a, and the other game video is input to the video processing device 100E. Similarly, the game video output from the game device GMb is distributed to two systems by the distribution device DTb, one game video is input to the display device D1b, and the other game video is input to the video processing device 100E. Further, the viewpoint detection device EYa detects the viewpoint position of the user of the display device D1a, and outputs the viewpoint information indicating the detected viewpoint position to the video processing device 100E. Similarly, the viewpoint detection device EYb detects the viewpoint position of the user of the display device D1b, and outputs the viewpoint information indicating the detected viewpoint position to the video processing device 100E.

The image processing device 100E causes the display device D3 to display a composite composite game image in which each viewpoint information is added to each play image of a plurality of players. As shown in the display screen G10, the composite composite game video is divided into two left and right with the center of the display screen as a boundary, and the composite game video in which the player A's viewpoint information is added to the player A's game video on the left side and the player A's viewpoint information on the right side. The game image of the player B may be combined so as to be a composite game image in which the viewpoint information of the player B is added. Further, as shown on the display screen G20, a composite composite game video in which both the viewpoint information of the player A and the viewpoint information of the player B are added to the game video of either the game video of the player A or the game video of the player B. May be. In the example shown on the display screen G10, the display of the viewpoint position of each player is easy to see, and in the example shown on the display screen G20, the relationship between the viewpoints according to the game situation of each player is clarified.

Instead of dividing the display screen into two parts on the left and right sides and displaying them on the left side and the right side, the display screen may be divided into two parts on the upper and lower sides and displayed on the upper side and the lower side. Further, the composite composite game video in which the respective viewpoint information is added to the play video of each of the plurality of players may be displayed side by side on one screen, or at least a part thereof may be overlapped and displayed. For example, if the game images of a plurality of players are the same, they may be displayed in an overlapping manner, and if they are different, they may be displayed side by side.

FIG. 15 is a diagram showing a usage example of the viewpoint confirmation system 1E shown in FIG. The user of the display device D1a is the player A, and the user of the display device D1b is the player B. The user of the display device D3 is a lecturer, a student, or the like. The play video is input to the display device D1a and the display device D1b, and the composite composite game video is displayed on the display device D3. For example, the viewpoint confirmation system 1E displays a game image on the display devices D1a and the display device D1b during game play, and the display device D3 is a composite in which marks based on the viewpoint information of the players A and B are synthesized. A composite game image (see display screens G10 and G20 in FIG. 14) is displayed. Further, the viewpoint confirmation system 1E may display the composite composite game image on the display device D3 after playing the game.

In this way, it is also meaningful to detect and compare the viewpoint positions of a plurality of players at the same time. However, when the game video of each player is different, it is the same as the configuration and effect of the first to fourth embodiments described above. This embodiment is more effective when comparing viewpoint information when playing the same game video with a plurality of players. For example, in a fighting game in which multiple players fight while watching the same game image (for example, a fighting game), each player plays while watching which position, such as when attacking, defending, or taking time. You can check and compare with the composite composite game video to see if you are doing it, and you can clarify the difference in the viewpoint position in offense and defense.

Next, the configuration of the viewpoint confirmation system 1E will be described in detail with reference to FIGS. 16 and 17. FIG. 16 is a block diagram showing an example of the configuration of the viewpoint confirmation system 1E according to the present embodiment. The viewpoint confirmation system 1E includes a game device GMa, a distribution device DTa, a display device D1a, a viewpoint detection device EYa, a game device GMb, a distribution device DTb, a display device D1b, a viewpoint detection device EYb, and video processing. The device 100E and the display device D3 are provided.

The game device GMa outputs the game video PV0a according to the progress of the game to the distribution device DTa. The distribution device DTa distributes and outputs the game video PV0a input from the game device GMa to two systems, the first video PV1a and the second video PV2a. The first video PV1a is input to the display device D1a, and the second video PV2a is input to the video processing device 100E. The viewpoint detection device EYa detects the viewpoint position of the user (for example, player A) of the display device D1a, and outputs the viewpoint information ED1a indicating the detected viewpoint position to the video processing device 100E.

Further, the game device GMb outputs the game video PV0b according to the progress of the game to the distribution device DTb. The distribution device DTb distributes and outputs the game video PV0b input from the game device GMb to two systems, the first video PV1b and the second video PV2b. The first video PV1b is input to the display device D1b, and the second video PV2b is input to the video processing device 100E. Further, the viewpoint detection device EYb detects the viewpoint position of the user (for example, player B) of the display device D1b, and outputs the viewpoint information ED1b indicating the detected viewpoint position to the video processing device 100E.

The video processing device 100E generates composite composite video information based on the input second video PV2a, viewpoint information ED1a, second video PV2b, and viewpoint information ED1b, and outputs the composite composite game video.
FIG. 17 is a block diagram showing an example of the configuration of the video processing device 100E according to the present embodiment. The video processing device 100E includes a generation unit 110E, a storage control unit 120, and a video output unit 130.

The generation unit 110E receives the second video PV2a output from the distribution device DTa, the viewpoint information ED1a output from the viewpoint detection device EYa, the second video PV2b output from the distribution device DTb, and the viewpoint detection device EYb. The output viewpoint information ED1b is input. The generation unit 110E converts the input second video PV2a into video information PV2a'that can be processed into video, and the viewpoint position detected by the viewpoint detection device EYa on the video information PV2a' based on the second video PV2. The first composite video information to which the viewpoint information ED1a indicating (the viewpoint position of the player A) (more specifically, the viewpoint information ED1a' obtained by data conversion so that the viewpoint information ED1a can be added to the video information PV2a') is added. To generate. Further, the generation unit 110E converts the input second video PV2b into video information PV2b'which can be processed into video, and the viewpoint detection device EYb detects the video information PV2b' based on the second video PV2b. A second composition to which the viewpoint information ED1b indicating the viewpoint position (the viewpoint position of the player B) (more specifically, the viewpoint information ED1b' data converted so that the viewpoint information ED1b can be added to the video information PV2b') is added. Generate video information. Then, the generation unit 110E generates the composite composite video information CV10'based on the first composite video information and the second composite video information. For example, the generation unit 110E divides the display screen into two left and right sides with the center of the display screen as a boundary, and the left side is a video based on the first composite video information (game video to which the viewpoint information of the player A is added), and the right side is the second composite video information. The composite composite video information CV10'is generated so that the video based on the above (game video to which the viewpoint information of the player B is added) is displayed (see the display screen G10 in FIG. 14). The generation unit 110E outputs the generated composite composite video information CV10'to the storage control unit 120 and the video output unit 130.

In addition, the generation unit 110E has a viewpoint position (player) detected by the viewpoint detection device EYa in the video information of either the video information PV2a'based on the second video PV2 or the video information PV2b' based on the second video PV2b. It was detected by the viewpoint information ED1a indicating (the viewpoint position of A) (more specifically, the viewpoint information ED1a' data converted so that the viewpoint information ED1a can be added to the video information PV2a') and the viewpoint detection device EYb. Composite synthesis with addition of viewpoint information ED1b indicating the viewpoint position (viewpoint position of player B) (more specifically, viewpoint information ED1b'data converted so that the viewpoint information ED1b can be added to the video information PV2b'). The video information CV10'may be generated (see the display screen G20 in FIG. 14).

The video output unit 130 can output the composite composite game video CV10 based on the composite composite video information CV10'generated by the generation unit 110E during play. Further, the video output unit 130 can output the composite composite game video CV10 based on the composite composite video information CV10'stored in the storage unit by the storage control unit 120 after the play.

With this configuration, the display device D1a used by the player A and the display device D1b used by the player B display the game image output from the game device GM in real time during play. On the other hand, the display device D3 may display the composite composite game video CV10 output from the video processing device 100E during play, or display the composite composite game video CV10 output from the video processing device 100E after play. Is also possible.

As described above, the viewpoint confirmation system 1E according to the present embodiment uses a plurality of viewpoint detection devices EY (EYa, EYb) for detecting the viewpoint positions of a plurality of players (an example of a target person). The distribution device DT includes distribution devices DTa and DTb (an example of individual distribution devices) that distribute game images corresponding to each of a plurality of players. The generation unit 110E includes a plurality of viewpoint information ED1 (ED1a, ED1b) corresponding to each of the plurality of players, and game video (video information PV2a'based on the second video PV2 and video information PV2b' based on the second video PV2b. Generate composite composite video information CV10'based on either one or both). The video output unit 130 outputs the composite composite game video CV10 based on the composite composite video information CV10'generated by the generation unit 110E. For example, the video output unit 130 outputs the composite composite game video CV10 based on the composite composite video information CV10'generated by the generation unit 110E in real time during play. Further, after the play, the video output unit 130 outputs the composite composite game video CV10 based on the composite composite video information CV10'stored in the storage unit by the storage control unit 120.

As a result, the viewpoint confirmation system 1E allows the plurality of players (for example, players A and B) to view each of the plurality of display devices D1a and D1b for displaying the game images output from the respective game devices GMa and GMb. During the play of the game to be performed, a composite composite game video CV10 in which marks based on the viewpoint information of each of a plurality of players (for example, player A and player B) are combined with the game video is displayed on a display device other than the display devices D1a and D1b. In D3, since it is displayed to a third party (for example, an instructor or a student), the viewpoints of a plurality of players during play can be displayed to a third party (for example, an instructor) without interfering with the player's play of the game. And students) can be confirmed. Further, the viewpoint confirmation system 1E can also enable a third party (for example, a teacher or a student) to confirm the viewpoints of a plurality of players during play after the play.

[Modification 1]
A configuration example in which the display device D2 is added to the viewpoint confirmation system 1C (see FIG. 8) according to the third embodiment described above will be described.
FIG. 18 is a schematic view showing an outline of the viewpoint confirmation system 1F according to the first modification. In the viewpoint confirmation system 1F shown in the figure, the user of the display device D1 is limited to the live person, and the game image is displayed not only on the display device D1 during play but also on the display device D2 used for the player to play the game. The point of display and the point of displaying the composite game image on one or both of the display device D1 and the display device D3 after the play are different from the viewpoint confirmation system 1C shown in FIG. For example, the display device D1 is used as a display device for a commentator, the display device D2 is used as a player, and the display device D3 is used as a display device for a teacher or a student. In the viewpoint confirmation system 1F, the video processing device 100F stores a composite game video in which a play video and a mark based on the viewpoint information of the commentator are combined, and then displays the video on one or both of the display devices D1 and the display device D3. You may.

FIG. 19 is a diagram showing a usage example of the viewpoint confirmation system 1F shown in FIG. The usage example of the display device D1 and the display device D3 is different from the usage example of the viewpoint confirmation system 1C according to the third embodiment shown in FIG. 9, in that the user of the display device D1 is a live person, and the display device D1 The difference is that the display device D3 can display the composite game image after playing. In the display device D2, the user is a player, and the game image is displayed during play. The usage example of the display device D1 used by the commentator and the display device D2 used by the player is the same as the usage example of the viewpoint confirmation system 1B shown in FIG. For example, the synthetic game video is a video in which the viewpoint information of the commentator is added in real time to the game video. For example, the display device D3 displays a synthetic game image in which marks based on the viewpoint information of the commentator are synthesized during the play of the game. As a result, a third party such as a lecturer or a student can view the viewpoint of the commentator during the commentary without interfering with the game image that the commentator is watching during the commentary of the game played by the player using the display device D2. Lectures can be advanced while confirming and evaluating the position and movement in real time. Further, one or both of the display devices D1 and the display device D3 can display the composite game image after the play. As a result, the position and movement of the viewpoint of the commentator can be confirmed by the commentator himself after the play (after the commentary), or the lecturer or the commentator himself can give a lecture to the students.

Next, the configuration of the viewpoint confirmation system 1F will be described in detail with reference to FIGS. 20 and 21. FIG. 20 is a block diagram showing an example of the configuration of the viewpoint confirmation system 1F according to the modified example 1. In FIG. 20, the same configurations as those in FIGS. 7 and 10 are designated by the same reference numerals, and the description thereof will be omitted. The configuration of the viewpoint confirmation system 1F shown in FIG. 20 is that the third video PV3 in which the game video PV0 is distributed by the distribution device DT is input to the display device D2 with respect to the configuration of the viewpoint confirmation system 1C shown in FIG. The difference is that the video processing device 100F can display the synthesized game video on the display device D1 and the video processing device 100F includes the storage control unit 120.

FIG. 21 is a block diagram showing an example of the configuration of the video processing device 100F according to the first modification. The video processing device 100F includes a storage control unit 120 in addition to the generation unit 110 and the video output unit 130 included in the video processing device 100C shown in FIG. 11, and the composite video information generated by the generation unit 110. CV1'is output not only to the video output unit 130 but also to the storage control unit 120. The video output unit 130 can also output the synthetic game video CV1 based on the synthetic video information CV1'stored in the storage unit by the storage control unit 120, and is also generated by the generation unit 110 in real time during play (live). It is also possible to output a synthetic game video CV1 based on the synthetic video information CV1'.

The video output unit 130 is based on the synthetic game video CV1 based on the synthetic video information CV1'generated in real time and the synthetic video information CV1'read after temporarily storing the generated synthetic video information CV1'. It may be configured to output only one of the synthetic game video CV1. For example, when the video output unit 130 outputs only the composite game video CV1 based on the composite video information CV1'that has been stored once, the configuration of the video processing device 100F may be the configuration of the video processing device 100 shown in FIG. Further, when the video output unit 130 outputs only the composite game video CV1 based on the composite video information CV1'generated in real time, the configuration of the video processing device 100F may be the configuration of the video processing device 100C shown in FIG. good. In this case, there may be no connection path for converting the synthetic game video CV1 into the display device D1.

[Modification 2]
A modified example of the image processing apparatus 100 will be described. For example, in the configuration example of the video processing device 100 shown in FIG. 4, the game image and the mark based on the viewpoint information are combined and then stored, but the game image and the viewpoint information are individually stored before being combined. , When outputting the composite game video, the game video and the mark based on the viewpoint information may be composited.

FIG. 22 is a block diagram showing an example of the configuration of the video processing device 100B according to the second modification. The video processing device 100B includes a generation unit 110, a storage control unit 120, and a video output unit 130, similarly to the video processing device 100 shown in FIG. 4, but the order of processing is different.

The second video PV2 output from the distribution device DT and the viewpoint information ED1 output from the viewpoint detection device EY are input to the storage control unit 120. The storage control unit 120 converts the input second video PV2 into video information PV2 ′ capable of video processing and stores it in the storage unit. As described above, the storage unit may be built in the video processing device 100, or may be an external storage device connected by USB or the like. Further, the storage control unit 120 stores the input viewpoint information ED1 (more specifically, the viewpoint information ED1') in the storage unit. That is, the storage control unit 120 individually stores the video information PV2'based on the input second video PV2 and the viewpoint information ED1'in the storage unit. The video information PV2'and the viewpoint information ED1' are stored in association with each other so that they can be synchronized on the time axis.

The generation unit 110 reads out the video information PV2'and the viewpoint information ED1'stored in the storage unit by the storage control unit 120, and generates the composite video information CV1' in which the viewpoint information ED1'is added to the video information PV2'. .. For example, the composite video information CV1'is video information in which a mark based on the viewpoint information ED1'is combined with the video information PV2' based on the second video PV2, and the video information PV2'and the viewpoint information ED1' are on the time axis. Is synchronized with. The generation unit 110 outputs the generated composite video information CV1'to the video output unit 130.

The video output unit 130 outputs the composite game video CV1 based on the composite video information CV1'generated by the generation unit 110. For example, the video output unit 130 converts the composite video information CV1'to HDMI-compatible synthetic game video CV1 and outputs it.

As described above, in the video processing device 100B according to the second modification, the storage control unit 120 individually stores the video information PV2'based on the second video PV2 and the viewpoint information ED1 (ED1') in the storage unit. , The generation unit 110 generates synthetic video information in which the viewpoint information ED1'stored in the storage unit is added to the video information PV2'stored in the storage unit, and the video output unit 130 generates the composite video information in the generation unit 110. Outputs the composite game video CV1 based on the composite video information CV1'.

The video processing device 100B according to the second modification is a composite game in which the viewpoint information of the player or the commentator and the game image are individually stored, and the mark based on the viewpoint information and the game image are combined after the play or the actual condition. Since the video can be output, the player or the commentator himself can confirm the viewpoint during the play or the commentary after the play or the commentary. That is, the video processing device 100B stores the game video and the viewpoint information individually, so that the synthetic game video in which the viewpoint information is added to the game video is played or the actual situation, as in the video processing device 100 shown in FIG. It can be output later.

The video processing device 100E shown in FIG. 17 is also the second video PV2 (PV2a, PV2b) before the composite composite video information CV10'is generated, instead of being stored after the composite composite video information CV10'is generated. Individual information including the video information PV2'(PV2a', PV2b') and the viewpoint information ED1'(ED1a, ED1b) based on the above is stored, and is stored when the composite composite game video CV10 is output. The composite composite video information CV10'may be generated based on the existing game video and the viewpoint information. Similarly, for the video processing device 100F shown in FIG. 21, the video information PV2 based on the second video PV2 before the composite video information CV1'is generated, instead of being stored after the composite video information CV1'is generated. Individual information including'and viewpoint information ED1'is stored individually, and when the composite game video CV1 is output, the composite video information CV1'is generated based on the stored video information and the viewpoint information. You may.

[Modification 3]
In the fourth embodiment, the game image viewed by the player A during play and the game image viewed by the player B during play are the same game images generated by the game device GM, and are viewed by the player B during play. An example of the viewpoint confirmation system 1D that adds the viewpoint information of the player A to the game image has been described. Here, a configuration will be described in which the game device GM is used individually for each player, and similarly, the viewpoint information of the player A is added to the game video viewed by the player B during play.

FIG. 23 is a schematic view showing an outline of the viewpoint confirmation system 1G according to the modified example 3. Player A plays a game using the game device GMa and the display device D1. The display device D1 displays a game image output from the game device GMa. The viewpoint detection device EY detects the viewpoint position of the user of the display device D1 and outputs the viewpoint information indicating the detected viewpoint position to the image processing device 100C. On the other hand, the player B plays a game using the game device GMb and the display device D3. The game video output from the game device GMb is input to the video processing device 100C. The video processing device 100C generates a synthetic game video in which viewpoint information is added to the game video, and outputs the composite game video to the display device D3. The display device D3 used by the player B displays a synthetic game video in which the viewpoint information of the player A is added to the game video.

Player A and player B are, for example, players who play against each other in a battle game. For example, the game device GMa used by the player A and the game device GMb used by the player B are connected by a communication network, and a network battle is performed. For example, the game video output from the game device GMa and the game video output from the game device GMb are the same game video based on the same game space.

FIG. 24 is a diagram showing a usage example of the viewpoint confirmation system 1G shown in FIG. 23. The user of the display device D1 is player A. On the other hand, the user of the display device D3 is the player B. The game image from the game device GMa is input to the display device D1. On the other hand, the display device D3 is input with a synthetic game image in which the viewpoint information of the player A is added to the game image from the game device GMb. For example, the viewpoint confirmation system 1G displays a game image on the display device D1 and displays a composite game image on which a mark based on the viewpoint information of the player A is synthesized on the display device D3 during game play. As described above, the viewpoint confirmation system 1G can be used in the same manner as the usage example (see FIG. 13) of the viewpoint confirmation system 1D according to the fourth embodiment.

FIG. 25 is a block diagram showing an example of the configuration of the viewpoint confirmation system 1G according to the modified example 3. The viewpoint confirmation system 1G includes a game device GMa, a display device D1, a viewpoint detection device EY, a game device GMb, a display device D3, and a video processing device 100C. In FIG. 25, the same reference numerals are given to the configurations corresponding to the respective parts of FIG.

The game device GMa performs game processing according to the operation of the player A on a controller (input device) (not shown), and outputs the game video PV0a according to the progress of the game to the display device D1. The display device D1 displays the game video PV0a input from the game device GMa. That is, the game video PV0a corresponds to the first video PV1 in FIG. On the other hand, the game device GMb performs game processing according to the operation of the player B on a controller (input device) (not shown), and outputs the game video PV0b according to the progress of the game to the video processing device 100C. That is, the game video PV0b corresponds to the second video PV2 of FIG.

The viewpoint detection device EY detects the viewpoint position of the user of the display device D1 and outputs the viewpoint information ED1 indicating the detected viewpoint position to the video processing device 100C. The video processing device 100C outputs the composite game video CV1 based on the game video PV0b (corresponding to the second video PV2) input from the game device GMb and the viewpoint information ED1 input from the viewpoint detection device EY to the display device D3. do. As a result, the display device D3 used by the player B can display the game video (synthetic game video CV1) to which the viewpoint information of the player A is added during play.

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to the above-described embodiments, and includes designs and the like within a range that does not deviate from the gist of the present invention. For example, the configurations described in the above-described first to fifth embodiments and modifications 1 to 3 can be arbitrarily combined.

The game device GM may be, for example, a game device installed in an e-sports school, or a game device (game machine, etc.) owned by an instructor or student attending the school. It may be. For example, the game device GM may be a stationary game machine or a portable game machine. Further, the game device GM is not limited to a device dedicated to the game, and may be a smartphone or a personal computer capable of executing a game application program.

Further, a program for realizing the functions of the video processing devices 100, 100B, 100C, 100E, and 100F described above is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into the computer system. , You may perform the processing as the image processing apparatus 100, 100B, 100C, 100E, 100F by executing. Here, "loading and executing a program recorded on a recording medium into a computer system" includes installing the program in the computer system. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. Further, the "computer system" may include a plurality of computer devices connected via a network including a communication line such as the Internet, WAN, LAN, and a dedicated line. Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, or a storage device such as a hard disk built in a computer system. As described above, the recording medium in which the program is stored may be a non-transient recording medium such as a CD-ROM. The recording medium also includes an internal or external recording medium that can be accessed from the distribution server to distribute the program. The code of the program stored in the recording medium of the distribution server may be different from the code of the program in a format that can be executed by the terminal device. That is, the format stored in the distribution server does not matter as long as it can be downloaded from the distribution server and installed in a form that can be executed by the terminal device. The program may be divided into a plurality of parts, downloaded at different timings, and then combined by the terminal device, or the distribution server for distributing each of the divided programs may be different. Furthermore, a "computer-readable recording medium" is a volatile memory (RAM) inside a computer system that serves as a server or client when a program is transmitted via a network, and holds the program for a certain period of time. It shall also include things. Further, the above program may be for realizing a part of the above-mentioned functions. Further, it may be a so-called difference file (difference program) that can realize the above-mentioned function in combination with a program already recorded in the computer system.

Further, some or all the functions of the above-mentioned video processing devices 100, 100B, 100C, 100E, and 100F may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each of the above-mentioned functions may be made into a processor individually, or a part or all of them may be integrated into a processor. Further, the method of making an integrated circuit is not limited to the LSI, and may be realized by a dedicated circuit or a general-purpose processor. Further, when an integrated circuit technology that replaces an LSI appears due to advances in semiconductor technology, an integrated circuit based on this technology may be used.

The above-mentioned synthetic game video is not limited to the mark based on the viewpoint information, and may be a video in which a heat map, a gaze plot, a focus map, a beesworm, or the like based on the viewpoint information is added to the game video. Further, the above-mentioned synthetic video information is not limited to the mark based on the viewpoint information, and may be information obtained by adding a heat map, a gaze plot, a focus map, a beesworm, or the like based on the viewpoint information to the video information.

[Additional Notes]
From the above description, the present invention can be grasped as follows, for example. Reference numerals in the accompanying drawings are added in parentheses for convenience in order to facilitate understanding of the present invention, but the present invention is not limited to the illustrated mode.

(Appendix 1) The viewpoint confirmation system (1, 1B, 1E, 1F) according to one aspect of the present invention includes a first display device (D1) on which a game image output from the game device (GM) is displayed, and the above. A viewpoint confirmation system including a viewpoint detection device (EY) for detecting the viewpoint position of a target person (for example, a player, a commentator, etc.) facing the display screen of the first display device, and the output game image ( A distribution device (DT) that distributes PV0) into at least a first image (PV1) for inputting to the first display device and a second image (PV2) whose input destination is different from that of the first image, and the first image. 2 Generation unit (110) that generates composite video information (CV1') by adding viewpoint information (ED1 (ED1')) indicating the viewpoint position detected by the viewpoint detection device to video information (PV2') based on the video. ), A video output unit (130) that outputs a composite game video (CV1) based on the composite video information, information including video information based on the second video and the viewpoint information, or the composite video information is stored. A storage control unit (120) for storing in the unit and a video processing device (100, 100B, 100E, 100F) having the storage control unit (120) are provided.

According to the configuration of Appendix 1, a target person (for example, a target person) who performs an activity during an activity (for example, game play or a game commentary) performed while watching a first display device for displaying a game image output from the game device GM. By recording a mark or the like based on the viewpoint information of the player or the commentator together with the synthetic game video or the game video synthesized with the game video by a video processing device other than the first display device, the first display device is displayed during the activity. It is possible to prevent the mark or the like based on the viewpoint information from being displayed. Therefore, it is possible to confirm the viewpoint of the target person during the activity without interfering with the activity of the target person (for example, game play or game play).

(Appendix 2) Further, one aspect of the present invention is the viewpoint confirmation system (1, 1B, 1F) described in Appendix 1, wherein the first display device is from the game image output from the game device. The distributed first video and the synthetic game video output from the video output unit can be switched and displayed.

According to the configuration of Appendix 2, the target person (for example, a player or a commentator) performs an activity (for example, a game play or a game commentary) while looking at the first display device, and sees the same first display device after the activity. However, since the composite game image can be confirmed, the viewpoint of the target person during the activity can be confirmed with the minimum system of one display device.

(Appendix 3) Further, one aspect of the present invention is the viewpoint confirmation system (1B, 1F) described in Appendix 1 or 2, wherein the distribution device further adds the output game image to a third image. A second display device (D2) that distributes to (PV3) and displays the third image is provided.

According to the configuration of Appendix 3, a target person who performs an activity (game live) while watching the game video on the first display device in a usage situation in which the player is playing the game while watching the game video on the second display device. The viewpoint of the (live player) can be confirmed on the same first display device after the activity (after the game live).

(Appendix 4) Further, one aspect of the present invention is the viewpoint confirmation system (1, 1B, 1E, 1F) according to any one of the appendices 1 to 3, wherein the memory control unit is the generation unit. The synthetic video information generated in the above is stored in the storage unit, and the video output unit outputs the synthetic game video based on the synthetic video information stored in the storage unit.

According to the configuration of Appendix 4, it is possible to store the composite video information obtained by synthesizing the mark or the like based on the viewpoint information of the target person (for example, the player or the commentator) performing the activity and the game video, and output it later. Therefore, after the activity, the target person can confirm the viewpoint of the target person during the activity.

(Appendix 5) Further, one aspect of the present invention is the viewpoint confirmation system (1, 1B, 1E, 1F) according to any one of the appendices 1 to 3, wherein the memory control unit is the second. The video information based on the video and the viewpoint information are individually stored in the storage unit, and the generation unit adds the viewpoint information stored in the storage unit to the video information stored in the storage unit. The information is generated, and the video output unit outputs the synthetic game video based on the synthetic video information generated by the generation unit.

According to the configuration of Appendix 5, the viewpoint information of the target person (for example, the player or the commentator) performing the activity and the game image are individually stored, and the viewpoint information is stored after the activity (for example, after the play or the commentary). Since it is possible to output a composite game video in which a mark or the like based on the above and the game video are combined, the target person himself / herself can confirm the viewpoint of the target person during the activity after the activity.

(Appendix 6) Further, the viewpoint confirmation system (1C, 1D, 1E, 1F) according to one aspect of the present invention includes a first display device (D1) for displaying a game image output from the game device (GM). A viewpoint confirmation system including a viewpoint detection device (EY) for detecting the viewpoint position of a target person (for example, a player, a commentator, etc.) facing the display screen of the first display device, and the output game. A distribution device (DT) that distributes the video (PV0) into at least a first video (PV1) for inputting to the first display device and a second video (PV2) whose input destination is different from that of the first video. A generator that generates composite video information (CV1') by adding viewpoint information (ED1 (ED1')) indicating the viewpoint position detected by the viewpoint detection device to the video information (PV2') based on the second video. The video processing device (100C, 100E, 100F) having the (110) and the video output unit (130) for outputting the composite game video (CV1) based on the composite video information is different from the first display device. A third display device (D3) is provided, the first display device displays the first image, and the third display device displays the synthetic game image.

According to the configuration of Appendix 6, the target person (for example, the player or the live commentary) who performs the activity during the activity (game play or the game live commentary) performed while watching the first display device displaying the game image output from the game device. While preventing the first display device from displaying the composite game video that combines the mark, etc. based on the viewpoint information of the person) with the game video, for third parties (for example, instructors and students) other than the target person, Since the viewpoint information can be confirmed during the activity of the target person on the third display device, the viewpoint of the target person during the activity can be confirmed without interfering with the activity of the target person (for example, game play or live game). Can be done like this.

(Appendix 7) Further, one aspect of the present invention is the viewpoint confirmation system (1E, 1F) described in Appendix 6, wherein the image processing apparatus provides image information based on the second image and the viewpoint information. A storage control unit (120) for storing individual information individually included or the synthetic video information in a storage unit is provided, and the video output unit is a synthetic game video based on the individual information stored in the storage unit, or the synthetic game video or the above. Outputs a synthetic game video based on synthetic video information.

According to the configuration of Appendix 7, the viewpoint information of the target person (for example, the player or the commentator) performing the activity and the game video are individually stored, or the composite video information obtained by synthesizing the mark or the like based on the viewpoint information and the game video. Can be stored and the composite game video can be output after the activity (for example, after playing or after the live performance), so that the target person can confirm the viewpoint of the target person during the activity after the activity.

(Appendix 8) Further, one aspect of the present invention is the viewpoint confirmation system (1B, 1F) according to any one of Addendums 1 to 7, wherein the viewpoint confirmation system is a plurality of target persons (for example, A plurality of viewpoint detection devices (EYa, EYb) for detecting the viewpoint positions of the players A and B) are used, and the distribution device is a game image (PV0a, PV0b) corresponding to each of the plurality of target persons. ) Is included, and the generation unit is based on the plurality of viewpoint information (ED1a, ED1b) corresponding to each of the plurality of target persons and the game image. The composite composite video information (CV10') is generated, and the video output unit outputs a composite composite game video (CV10) based on the composite composite video information generated by the generation unit.

According to the configuration of Appendix 8, a game played while a plurality of target persons (for example, player A and player B) are watching each of a plurality of first display devices for displaying game images output from the respective game devices. During play, a composite composite game video obtained by synthesizing a mark or the like based on the viewpoint information of each of a plurality of target persons (for example, player A and player B) with the game video is displayed on a third display device other than the first display device. Since it is displayed to a third party (for example, a teacher or a student), the viewpoints of a plurality of target persons during the activity can be viewed by the third party (for example, game play) without interfering with the activity of the target person (for example, game play). For example, it can be made visible to teachers and students.

1,1B, 1C, 1D, 1E, 1F, 1G viewpoint confirmation system, GM game device, DT distribution device, D1, D2, D3 display device, EY viewpoint detection device, 100, 100B, 100C, 100E, 100F video processing device , 110 generator, 120 memory control, 130 video output

Claims (8)

A viewpoint confirmation system including a first display device for displaying a game image output from a game device and a viewpoint detection device for detecting the viewpoint position of a target person facing the display screen of the first display device.
A distribution device that distributes the output game video to at least a first video that is input to the first display device and a second video whose input destination is different from that of the first video.
A generator that generates composite video information by adding viewpoint information indicating the viewpoint position detected by the viewpoint detection device to the video information based on the second video, and a video that outputs a composite game video based on the composite video information. An image processing device having an output unit, information including image information based on the second image and the viewpoint information, or a storage control unit that stores the synthetic image information in the storage unit.
A viewpoint confirmation system equipped with. The first display device can switch and display the first video distributed from the game video output from the game device and the synthetic game video output from the video output unit.
The viewpoint confirmation system according to claim 1. The distribution device further distributes the output game video to the third video.
A second display device for displaying the third image is provided.
The viewpoint confirmation system according to claim 1 or 2. The storage control unit stores the synthetic video information generated by the generation unit in the storage unit.
The video output unit outputs the synthetic game video based on the synthetic video information stored in the storage unit.
The viewpoint confirmation system according to any one of claims 1 to 3. The storage control unit individually stores the video information based on the second video and the viewpoint information in the storage unit.
The generation unit generates synthetic video information in which the viewpoint information stored in the storage unit is added to the video information stored in the storage unit.
The video output unit outputs the composite game video based on the composite video information generated by the generation unit.
The viewpoint confirmation system according to any one of claims 1 to 3. A viewpoint confirmation system including a first display device for displaying a game image output from a game device and a viewpoint detection device for detecting the viewpoint position of a target person facing the display screen of the first display device.
A distribution device that distributes the output game video to at least a first video that is input to the first display device and a second video whose input destination is different from that of the first video.
A generator that generates composite video information by adding viewpoint information indicating the viewpoint position detected by the viewpoint detection device to the video information based on the second video, and a video that outputs a composite game video based on the composite video information. A video processing device having an output unit,
A third display device different from the first display device is provided.
The first display device displays the first image and displays the first image.
The third display device displays the synthetic game image.
Viewpoint confirmation system. The video processing device includes a storage control unit that stores individual information that individually includes video information based on the second video and the viewpoint information, or the composite video information in a storage unit.
The video output unit outputs a synthetic game video based on the individual information stored in the storage unit or a synthetic game video based on the synthetic video information.
The viewpoint confirmation system according to claim 6. The viewpoint confirmation system uses a plurality of viewpoint detection devices that detect the viewpoint positions of a plurality of subjects.
The distribution device is configured to include an individual distribution device that distributes game images corresponding to each of the plurality of target persons.
The generation unit generates composite composite video information based on the plurality of viewpoint information and the game video corresponding to each of the plurality of target persons.
The video output unit outputs a composite composite game video based on the composite composite video information generated by the generation unit.
The viewpoint confirmation system according to any one of claims 1 to 7.

Images