JP7314605B2 - Display control device, display control method and display control program - Google Patents

Description

The present invention relates to a display control device, a display control method, and a display control program.

Images are used for watching various competitive sports, for example, competitive sports in general including e-sports. For example, the highlight scenes of the competitive sports are presented as highlight scenes on a screen installed at the spectator venue or on the display of the spectators who are watching the game from a distance.

Here, as an example of a technique for presenting highlight scenes, there is a technique for extracting highlight scenes from recorded video content. This technology proposes a display control device that detects a highlight scene in a content based on an evaluation value calculated based on feature data extracted from image or audio data of the content.

JP-A-2007-123981

However, with the above technology, as will be described below, it may be difficult for spectators to grasp the highlight scenes of each of the multiple teams in the match.

In other words, in the display control device described above, since the image for a predetermined time up to the highlight scene is displayed on the display screen as a replay image, it is difficult to grasp which team the highlight scene being displayed on the display screen is about. For example, even if a spectator of a competitive sport is cheering for one of a plurality of teams, it is difficult to grasp whether the highlight scene is the team that the spectator is cheering for or the team that the spectator is cheering for and the team that is currently competing. Even if highlight scenes are presented to the spectators in this way, it is difficult for the spectators to understand the fun of the competitive sports because the spectators cannot understand the progress of the team they are supporting.

An object of the present invention is to provide a display control device, a display control method, and a display control program that enable spectators to easily grasp the highlight scenes of each of a plurality of teams during a match.

In one aspect, the display control device includes an extraction unit that extracts a highlight scene based on an analysis result of a video of a match by a plurality of teams, a determination unit that determines which of the plurality of teams the highlight scene corresponds to, and a display control unit that displays the highlight scene in a display area corresponding to the team determined by the determination unit, among display areas assigned to each of the plurality of teams and which do not overlap with each other, corresponding to the team determined by the determination unit.

You can easily grasp the highlight scenes of each of the multiple teams during the match.

1 is a block diagram illustrating an example of a functional configuration of a display control device according to a first embodiment; FIG. FIG. 2 is a diagram showing an example of a screen layout. FIG. 3 is a diagram showing an example of tags attached to highlight scenes. FIG. 4 is a diagram showing an example of highlight scene information. FIG. 5A is a diagram illustrating an example of a method for determining the clip display size. FIG. 5B is a diagram showing an example of a clip format. FIG. 6 is a diagram showing an example of a method of shifting arranged clips. FIG. 7 is a diagram showing an example of a method of narrowing down clips relating to a match that has already been played. FIG. 8 is a diagram illustrating an example of a highlight scene distribution method. FIG. 9 is a flowchart illustrating the procedure of clip placement processing according to the first embodiment. FIG. 10 is a flowchart illustrating the procedure of clip update processing according to the first embodiment. FIG. 11 is a flowchart illustrating the procedure of clip distribution processing according to the first embodiment. FIG. 12 is a diagram showing another example of the screen layout. FIG. 13 is a diagram showing another example of the screen layout. FIG. 14A is a diagram showing an application example of highlight classification. FIG. 14B is a diagram showing an application example of highlight classification. FIG. 14C is a diagram showing an example application of highlight classification. FIG. 15 is a diagram showing another example of the screen layout. FIG. 16 is a diagram showing an example of camera arrangement in motion tracking. FIG. 17 is a diagram illustrating a hardware configuration example of a computer.

A display control device, a display control method, and a display control program according to the present application will be described below with reference to the accompanying drawings. Note that this embodiment does not limit the disclosed technology. Further, each embodiment can be appropriately combined within a range that does not contradict the processing contents.

[System configuration]
FIG. 1 is a block diagram showing an example of the functional configuration of a display control device 10 according to the first embodiment. The display control device 10 shown in FIG. 1 has a display control function related to the match video of a competitive sports, for example, a display control function for highlight scenes.

The display control device 10 is an example of a computer that provides the display control function described above.

As one embodiment, the display control device 10 can be implemented by installing a display control program that realizes the display control function described above as package software or online software in any computer. For example, the display control device 10 can be implemented as a server that provides the above display control function on-premises. In this case, the display control device 10 does not necessarily have to be provided outside the competition venue, and may be incorporated into a part of an information system installed at the competition venue, such as the video system 20. The display control device 10 is not limited to this, and may be implemented as a SaaS (Software as a Service) type application to provide the above display control function as a cloud service.

Here, in the present embodiment, e-sports (electronic sports) is taken as an example of competitive sports, and a use case in which watching sports is performed through video content such as a game video displayed on a screen at an e-sports venue and its highlight scenes is taken as an example.

As shown in FIG. 1 , a video system 20 and a user terminal 50 can be connected to the display control device 10 . The display control device 10, the video system 20, and the user terminal 50 are communicably connected via an arbitrary network NW. The network NW may be any type of communication network such as the Internet or a LAN (Local Area Network), whether wired or wireless. Note that FIG. 1 shows an example in which one user terminal 50 is connected to one display control device 10 for convenience of explanation, but a plurality of user terminals 50 are connected to one display control device 10. It does not prevent.

The video system 20 is a system for displaying e-sports game video and the like. For example, video system 20 may include game device 21, switcher 23, and display device 25, as shown in FIG.

The game device 21 is a computer on which computer game software is executed. Also, the game device 21 is a device operated by a player. The "game device" referred to here is a label given from the aspect that computer game software is executed, and the hardware configuration and the type of software installed are not limited, and it can be any type of computer. Also, "computer game" may be any title or genre. Examples include first- and third-person shooters, real-time strategy, multiplayer online battle arenas, fighting games, sports games, racing games, and puzzle games. A game screen from the player's viewpoint of the game device 21 is input to the switcher 23 via a capture board or the like installed in the game device 21 .

The switcher 23 is a device that switches images displayed on a predetermined screen by the display device 25 . For example, the switcher 23 outputs to the display device 25 an image selected via a control panel or the like (not shown) from among the game screen output from the game device 21 from the player's point of view and the game screen from a bird's-eye view in which fields and maps in the game are viewed from above. A game video displayed on a screen in time series by such a selection operation may be referred to as a "game video".

The display device 25 is a device that displays an image on a predetermined screen.

By way of example only, the display device 25 may employ a liquid crystal display, an organic EL (electroluminescence) display, or the like, which realizes display by light emission. As another example, the display device 25 may employ a projector that realizes display by projection. When a projector is used in this way, a dedicated screen can be used, or a flat surface such as a wall or floor can be used as the screen.

For example, the display device 25 displays a live game video input from the switcher 23 on a predetermined screen. In addition, the display device 25 can also display a highlight scene of an e-sports game video on a predetermined screen.

The user terminal 50 is an example of a computer used by spectators of e-sports. The "user terminal" referred to here is a label given from the viewpoint of the user of the computer, and the hardware configuration and the type of installed software are not limited, and it can be any type of computer. For example, the user terminal 50 corresponds to mobile terminal devices in general, such as smart phones, mobile phones, PHS (Personal Handyphone Systems), and tablet terminals. These are merely examples, and the user terminal 10 may be any computer such as a desktop computer such as a personal computer, a laptop computer, or a wearable terminal.

[One aspect of the challenge]
The technology mentioned in the Background Art column has a problem that it is difficult to grasp the highlight scenes of each of the multiple teams during the match.

That is, in the display control device mentioned in the background art section above, video for a predetermined time up to the highlight scene is simply displayed on the display screen as a replay image. For this reason, it is difficult to grasp which team the highlight scene being displayed on the display screen relates to.

Taking e-sports as just one example, e-sports tournaments and the like have an aspect in which many core players and core fans participate. Core players and core fans have a high level of knowledge about esports games. On the other hand, non-players and light fans do not have the same level of knowledge as core players and core fans. However, in the current situation where games for consumers are becoming popular, non-players and light fans can become potential spectators of e-sports. For such non-players and light fans, the gap in knowledge level with core players and core fans is one of the factors that increase the hurdles of watching games.

For example, non-players and light fans do not necessarily have knowledge of the avatars and the like used by the players of each team. Therefore, even if the highlight scene described in the background art section above is displayed, it is difficult to grasp whether the highlight scene is the team supported by the spectators or the team currently playing against the team supported by the spectators. In this way, in a situation in which the spectators do not know whether the highlights of the team they are cheering for or the highlights of the team currently competing are displayed, it is difficult for the spectators to understand the fun of watching sports. For this reason, it is not possible to enjoy watching sports together with core players and core fans, which is one of the factors that raise the hurdles of watching sports.

[One aspect of problem-solving approach]
Therefore, the display control device 10 according to the present embodiment realizes display control to display the highlight scene in the display area of the team corresponding to the highlight scene extracted from the e-sports match video among the display areas corresponding to the respective e-sports competing teams.

FIG. 2 is a diagram showing an example of a screen layout. FIG. 2 shows the layout of the screen when two teams, A team and B team, each consisting of four players, compete in an e-sports competition. As shown in FIG. 2, a video system 20 installed at an e-sports venue includes game devices 21 PA1 to 21 PA4 used by four players _PA1 to PA4 of team A, and game devices 21 _PB1 to 21 _PB4 used by four players PB1 to _PB4 of team B. Further, in the video system 20, a team screen 251A and a team screen 251B are arranged on the left and right sides of the main screen 250 as an example of display areas in which video is displayed by the display device 25. FIG. Hereinafter, when the labels of the team screen 251A and the team screen 251B are not distinguished, they may be referred to as "team screen 251".

The main screen 250, the team screen 251A, and the team screen 251B can be implemented by allocating three displays or by allocating three display areas divided into one display. Also, multiple displays can be combined to form display areas such as main screen 250, team screen 251A, or team screen 251B. In addition, when projection display is performed, a part of the e-sports venue, for example, a wall surface, can be assigned as the main screen 250, the team screen 251A, or the team screen 251B.

The main screen 250 displays a live game video of e-sports. For example, the game video displayed on the main screen 250 is switched by the switcher 23 . In the example shown in FIG. 2, game screens from the viewpoints of the players PA1 to _PA4 and _PB1 to PB4 input from the game devices 21 _PA1 to 21 _PA4 and the game devices 21 PB1 to 21 PB4 are input to the switcher 23 . In this case, in the switcher 23, one viewpoint is selected from the viewpoints of the players PA1 to PA4, the viewpoints of the players PB1 to PB4, and the bird's-eye view from which the game field is viewed from above via a control panel (not shown). Such a selection operation is performed by a staff member called a switcher, or a commentator or commentator, just as an example.

Also, the team screen 251 corresponds to an example of a display area that is assigned to each of a plurality of teams and that does not overlap each other. For example, the team screen 251A displays the highlight scene of the A team in the e-sports game video. In the example shown in FIG. 2, the team screen 251A is provided with small areas CA1 to CA7 in which the moving images of the highlight scenes of the A team are displayed. Furthermore, the team screen 251B displays the highlight scene of the B team in the e-sports game video. In the example shown in FIG. 2, the team screen 251B is provided with small areas CB1 to CB2 in which the video of the highlight scene of the B team is displayed. Hereafter, the video of the highlight scene clipped on the team screen 251A or the team screen 251B may be referred to as a "clip".

In this manner, the display control device 10 according to the present embodiment implements display control for displaying the highlight scenes of each team in order on the screen assigned to each team. Through such display control, it is possible for spectators to identify whether the highlight of the team they are cheering for is being displayed or the highlight of the team currently competing. For example, in the example shown in FIG. 2, when a spectator cheers for Team A, he/she can watch the highlight of Team A by watching clips CA1 to CA7 displayed on the team screen 251A assigned to Team A.

Therefore, according to the display control device 10 of the present embodiment, it becomes possible for spectators to easily grasp the highlight scenes of each of the multiple teams during the match.

Furthermore, in the display control device 10 according to the present embodiment, a plurality of clips are arranged in chronological order on the team screen 251 . For example, on the team screen 251A and the team screen 250B, the earlier the clip is extracted, the farther the clip is displayed from the main screen 250 . On the team screen 251A and the team screen 250B, on the other hand, as the clip extraction timing approaches the current time, for example, the time when the latest frame of the live game video is being reproduced, the clips are displayed side by side on the main screen 250 side. Furthermore, on the team screen 251A and the team screen 250B, in addition to the current match, clips extracted from the previous match are arranged in chronological order.

For example, assume that the match between Team A and Team B shown in FIG. 2 is the final. In this case, on the team screen 251A, in order from the side closest to the main screen 250, that is, from the right end of the team screen 251A when viewed from the front of the main screen 250, the final clips CA1 to CA4 and the semifinal clips CA5 to CA7, which were played before the final, are displayed. Furthermore, among the clips of the final round, the latest clip CA1, the second newest clip CA2, the third newest clip CA3, and the fourth newest clip CA3 are displayed in order from the right. Furthermore, in the semi-final clips, the latest clip CA5, the second newest clip CA6, and the third newest clip CA7 are displayed in order from the right.

Also, on the team screen 251B, in order from the side closer to the main screen 250, that is, in order from the left end of the team screen 251B when viewed from the front of the main screen 250, the clips CB1 to CB2 of the final and the clips CC1 to CB5 of the 3rd place match played as the game before the final are displayed. Furthermore, among the final clips, the latest clip CB1 and the second newest clip CB2 are displayed in order from the left. Furthermore, in the clips of the third-place match, the latest clip CC1, the second newest clip CC2, the third newest clip CC3, the fourth newest clip CC4, and the fifth newest clip CC5 are displayed in order from the left.

Here, when the clip of the game before the current game is displayed, as shown in the team screen 251B, not only the clip of Team B, but also the video of the highlight scene of Team C, who played an active part in the third-place match in the past, for example, can be displayed as clips CA5 to CA7.

In this way, since a plurality of clips for each team are arranged in chronological order on the team screen 251, it is possible to easily comprehend the progress of the e-sports match in chronological order. Furthermore, clips of previous games are also arranged in chronological order on the team screen 251, so that a log of the games through the e-sports tournament can be displayed. As a result, spectators can look back on past games, so that it becomes easier to grasp the flow of the entire game, and even if the timing of watching the game is delayed, the time series of the tournament can be easily grasped.

In addition, the display control described above allows spectators to intuitively grasp the superiority or inferiority of the e-sports match. That is, since scenes that affect the game situation in e-sports are extracted as highlight scenes, there is a relationship between the number of clips displayed on each team's screen and the game situation. Therefore, by relatively comparing the number of clips displayed on each team's screen, it is possible to grasp the superiority or inferiority of the game situation in e-sports. For example, in the example shown in FIG. 2, four clips CA1 to CA4 are displayed on the team screen 251A in the final, while only two clips CB1 to CB2 are displayed on the team screen 251B. From this, the spectators can intuitively understand that the A team, which has a larger number of highlight scene videos clipped on the team screen 251A, is leading the game ahead of the B team.

[Configuration of display control device 10]
Next, an example of the functional configuration of the display control device 10 according to this embodiment will be described. As shown in FIG. 1 , the display control device 10 has a communication interface 11 , a storage section 13 and a control section 15 . Although FIG. 1 shows a solid line representing the relationship of data transfer, for convenience of explanation, only the minimum portion is shown. That is, input/output of data relating to each processing unit is not limited to the illustrated example, and input/output of data other than those illustrated, for example, input/output of data between a processing unit and a processing unit, between a processing unit and data, and between a processing unit and an external device may be performed.

The communication interface 11 is an interface that controls communication with other devices such as the video system 20 and the user terminal 50 .

As an example only, the communication interface 11 can employ a network interface card such as a LAN card. For example, the communication interface 11 receives various videos from the video system 20, such as live game videos and game videos from each player's viewpoint, and outputs to the video system 20 display instructions for clips to be placed on each team screen 251. The communication interface 11 also receives requests from the user terminal 50 to download moving images of highlight scenes, and transmits clips corresponding to the download requests to the user terminal 50 .

The storage unit 13 is a functional unit that stores data used for various programs such as the display control program described above, including an OS (Operating System) executed by the control unit 15 .

As an example only, the storage unit 13 corresponds to an auxiliary storage device in the display control device 10 . For example, HDDs (Hard Disk Drives), optical disks, SSDs (Solid State Drives), etc. correspond to auxiliary storage devices. In addition, flash memory such as EPROM (Erasable Programmable Read Only Memory) also corresponds to the auxiliary storage device.

The storage unit 13 stores moving image information 13 a and highlight scene information 13 b as an example of data used in the program executed by the control unit 15 . In addition to these data, the storage unit 13 may store data referred to or generated by the display control program, such as setting information such as the duration of a moving image per highlight. The moving image information 13a and the highlight scene information 13b will be explained together with the explanation of the control unit 15 that refers to or generates each data.

The control unit 15 is a functional unit that performs overall control of the display control device 10 .

As one embodiment, the control unit 15 can be implemented by a hardware processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). Here, a CPU and an MPU are illustrated as examples of processors, but any processor can be used regardless of whether it is a general-purpose type or a specialized type. Alternatively, the control unit 15 may be realized by hardwired logic such as ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

By executing the above display control program, the control unit 15 virtually realizes the processing unit shown in FIG. 1 on a RAM work area such as a DRAM (Dynamic Random Access Memory) implemented as a main storage device (not shown). Here, an example is given in which package software in which a series of functions such as a highlight scene extraction function, a display control function, and a distribution function are packaged is executed, but the program that operates on the display control device 10 is not limited to this. For example, any function of the highlight scene extraction function, the display control function, or the distribution function among the functions of the above package software may be extracted and executed as a program module or referred to as a library.

For example, as shown in FIG. 1, the control unit 15 includes a collection unit 15A, an extraction unit 15B, a determination unit 15C, a display control unit 15D, a determination unit 15E, and a distribution unit 15F.

The collection unit 15A is a processing unit that collects various data.

As one aspect, the collection unit 15A collects game videos, game videos from each player's viewpoint, and the like from the switcher 23 in real time in synchronization with the progress of the game in the computer game. For example, the collection unit 15A receives stream data of moving images transmitted from the switcher 23 from the start to the end of a match in a computer game for each image such as a match image or a game image from each player's viewpoint. The game video and the game video from each player's point of view collected in real time in this manner are stored in the storage unit 13 as the video information 13a.

Here, as an example, an example in which each video is collected in real time is given, but each video may not necessarily be collected in real time, and video for a predetermined period may be collected by batch processing. Also, here, an example in which the moving image information 13a stored in the storage unit 13 is read out by the extraction unit 15B is taken, but the game video stored in the buffer area set on the memory may be referred to by the extraction unit 15B.

The extraction unit 15B is a processing unit that extracts highlight scenes based on the analysis results of the game video collected by the collection unit 15A.

As an example only, the extracting unit 15B extracts a video of a highlight scene based on the degree of excitement obtained for each scene of the game video using a prediction model learned to predict the volume of the audience from the game video of the match game.

More specifically, when the latest frame of the game video is added to the moving image information 13a stored in the storage unit 13, the extracting unit 15B determines whether the difference between the latest frame and the previous frame, such as SAD (Sum of Absolute Difference), is equal to or greater than a predetermined threshold. At this time, if the difference between frames is equal to or greater than the threshold, it can be determined that there is a high possibility that a scene change has occurred in which the viewpoint is switched between the bird's-eye viewpoint and the player's viewpoint, or between each player's viewpoint. In this case, the extraction unit 15B extracts, as the latest scene, the game video of the section from the start of the game or the detection of the previous scene change to the current time.

After the latest scene is extracted, or in parallel with the latest scene extraction, the extraction unit 15B divides the game video into still images (hereinafter, game images) at predetermined time intervals (for example, 1 second). Specifically, the extraction unit 15B extracts frames at intervals of, for example, one second from the match video of a predetermined frame rate. Next, the extraction unit 15B inputs the divided game image to a prediction model for predicting the degree of excitement of the spectators who viewed the game image for each divided game image, thereby obtaining the degree of excitement when the spectators watch the game as the output of the prediction model. As such a prediction model, for example, a learning model created by machine learning such as deep learning can be used based on teacher data in which the degree of excitement of spectators for the game image is given as a correct answer.

After that, the extraction unit 15B cuts out a section of the time length set as the number of seconds per highlight in setting information (not shown) from the latest scene. For example, when the time length for one highlight is designated as 10 seconds, the extraction unit 15B cuts out, as a highlight scene candidate, a section of 5 seconds before and after the time when the degree of excitement reaches a peak in the game image included in the latest scene. Subsequently, the extracting unit 15B calculates (predicts) the excitement level of the highlight scene candidate. For example, the extracting unit 15B obtains statistical values of excitement levels of game images included in highlight scene candidates, such as average values, median values, modes, and maximum values. Finally, when the statistic value of the excitement level of the highlight candidate is equal to or greater than a predetermined threshold, the extraction unit 15B extracts the highlight scene candidate as a highlight scene.

By repeatedly extracting such highlight scenes, even if a sensor such as a microphone is installed in the e-sports venue and the voice of the spectator is not sensed, the extraction of the highlight scene that excites spectators of the computer game can be realized.

Here, as an example, extraction of a highlight scene using the above prediction model has been exemplified, but it goes without saying that other known methods for extracting a highlight scene, such as those disclosed in Patent Document 1, can also be used.

The determination unit 15C is a processing unit that determines for which of a plurality of teams the highlight scene is the highlight scene.

As an embodiment, when a new highlight scene is extracted by the extraction unit 15B, the determination unit 15C determines which team the highlight scene belongs to by specifying the team to which the player who takes action in the highlight scene belongs.

For example, when the game video in the section corresponding to the highlight scene is the game video from the player's viewpoint, the determination unit 15C determines that the highlight scene extracted by the extraction unit 15B is the highlight for the team to which the player from the player's viewpoint belongs.

On the other hand, when the game video in the section corresponding to the highlight scene is a bird's eye view game video, the determination unit 15C reads the character string included in each frame of the bird's eye view game video in the section corresponding to the highlight scene by image processing such as character recognition. Then, the determination unit 15C determines whether or not the character string read from each frame includes a specific phrase including the player name. An example of such a specific phrase is a phrase related to the esports game situation, eg, kills or scores, eg, "○○ defeated." At this time, if the specific phrase is included, the determination unit 15C determines that the highlight scene extracted by the extraction unit 15B is the highlight for the team to which the player included in the specific phrase belongs. It should be noted that it is not always necessary to use a bird's-eye view game video to identify the player. For example, the determination unit 15C reads a character string included in each game video from each player's viewpoint in the section corresponding to the highlight scene extracted by the extraction unit 15B by image processing such as character recognition. Then, the determination unit 15C identifies the player's point of view from which the character string corresponding to a specific action in the computer game, such as a kill (Kill) or consecutive kills, is read. Then, the determination unit 15C determines that the highlight scene extracted by the extraction unit 15B is a highlight for the team to which the player from the player's viewpoint whose character string corresponding to the specific action is read belongs.

In this way, the determination unit 15C determines the team name and player name to which the highlight scene corresponds by image processing such as character recognition, and further determines various items such as the action to which the highlight scene corresponds and the number of consecutive actions by image processing such as character recognition.

Based on the determination results of these items, the determination unit 15C generates tags to be attached to the highlight scene. FIG. 3 is a diagram showing an example of tags attached to highlight scenes. As shown in FIG. 3, the extracting unit 15B cuts out, as highlight scene candidates, five seconds before and after the time when the degree of excitement peaks in the game image included in the latest scene. When the statistical value of the degree of excitement of the game image in this highlight scene candidate is equal to or greater than a predetermined threshold value, the highlight scene candidate is extracted as a highlight scene by the extraction unit 15B. When a highlight scene is extracted in this way, the determination unit 15C generates tags including determination results such as a serial number, time, team name, player screen, player name, action, number of consecutive times, and degree of excitement.

The “serial number” referred to here refers to a number assigned to the highlight scenes in the order in which the highlight scenes are extracted from the start of the game. Also, the “time” refers to at least one of the central time when the degree of excitement in a highlight scene peaks, the start time of the highlight scene, or the end time of the highlight scene. Also, the "team name" refers to the team corresponding to the highlight scene, in other words, the team corresponding to the highlight. Also, the "player name" refers to the player corresponding to the highlight scene, in other words, the player corresponding to the highlight. Also, the “player screen” refers to the game video from the player's viewpoint in the section corresponding to the highlight scene. Also, "action" refers to an action detected in a highlight scene. Also, the “continuous number” refers to the number of times the action continues in the highlight scene. Also, the "level of excitement" refers to the level of excitement corresponding to a highlight scene, for example, a statistic value of the level of excitement of a game image included in a highlight scene.

After that, the determination unit 15C additionally registers, in the highlight scene information 13b of the storage unit 13, a new record in which various kinds of information including the above tag are associated with the moving image data of the highlight scene.

FIG. 4 is a diagram showing an example of the highlight scene information 13b. As shown in FIG. 4, in the highlight scene information 13b, in addition to highlight scene video data, for example, a binary format video file and the above tags, a new record associated with information such as a video ID, silhouette shape, number of downloads, and comments is additionally registered.

The “moving image ID (IDentification)” referred to here refers to an example of identification information given to a highlight scene from the aspect of identifying the highlight scene stored in the storage unit 13 . Also, the “silhouette shape” refers to the outline shape when the video of the highlight scene is clipped to the team screen 251 . For example, for clips extracted from the same game, a unique silhouette shape is set for each highlight scene from the aspect of identifying the clip corresponding to each highlight scene. The silhouette shape set for the clip in this way may be a figure having an arbitrary outline, but the more the silhouette shape with a lower degree of similarity between the highlight scenes is set, the more accurately the individual clip can be identified.

Also, the “number of downloads” refers to the number of times the highlight scene has been downloaded. A "comment" refers to a comment posted on a video of a highlight scene. These "number of downloads" and "comment" are set to initial values, NULL values, or blanks when a new record corresponding to a highlight scene is additionally registered in the highlight scene information 13b. After that, when a highlight scene is downloaded, the number of downloads of the highlight scene is incremented. Also, when a comment is posted to a highlight scene, the posted comment is added.

The display control unit 15D is a processing unit that performs various display controls.

As one embodiment, the display control unit 15D performs the following processing when the highlight scene is extracted by the extraction unit 15B. That is, the display control unit 15D sets the team screen 251 corresponding to the “team name” among the tags given to the highlight scenes extracted by the extraction unit 15B as the display destination of the clip. Furthermore, the display control unit 15D sets the display size of the clip based on the “level of excitement” among the tags given to the highlight scenes extracted by the extraction unit 15B. Furthermore, the display control unit 15D sets the shape corresponding to the “silhouette shape” among the tags given to the highlight scenes extracted by the extraction unit 15B as the display shape of the clip.

FIG. 5A is a diagram illustrating an example of a method for determining the clip display size. As shown in FIG. 5, the display control unit 15D sets a smaller display size as the display size of the clip as the tag "excitement degree" assigned to the highlight scene becomes lower. On the other hand, the display control unit 15D sets a larger display size as the display size of the clip as the tag "excitement" added to the highlight scene increases. Note that FIG. 5 shows an example in which the display size of a clip is set based on the tag "excitement" attached to the highlight scene, but the present invention is not limited to this. For example, the display control unit 15D may set the display size of the clip based on the number of downloads instead of the degree of excitement, or may set the display size of the clip based on both the degree of excitement and the number of downloads.

After that, the display control unit 15D instructs the display device 25 to arrange the clip in the above display size and display shape at the initial position on the team screen 251 where the clip is to be displayed, for example, at a position closer to the main screen 250 . The clip arranged on the team screen 251 of the display destination in this way can perform other displays in addition to the display of looping the playback of the video of the highlight scene.

FIG. 5B is a diagram showing an example of a clip format. FIG. 5B shows an example of displaying a clip corresponding to the record shown in FIG. 4 in the highlight scene information 13b. As shown in FIG. 5B, a clip can include a display area for reproducing a moving image of a highlight scene, that is, a hatched portion in the figure, and a display area related to actions. For example, in the example shown in FIG. 5B, the number "2" of consecutive actions "kill" that occurred in the highlight scene is displayed, and the player name "XXXXXX" that caused the action is also displayed. Such an action-related display allows the spectators to grasp the extent to which the match progresses in the highlight scene. In addition, the clip also includes an area in which the number of times "57" the video of the highlight scene has been downloaded is displayed. Such a display of the number of downloads allows spectators to grasp the popularity of the highlight scene.

In parallel with the process of arranging the clip in this way, the display control unit 15D performs the process of updating the position and display content of the clip already arranged on each team screen 251 .

As one aspect, the display control unit 15D executes the following processing each time a predetermined cycle, for example, a cycle corresponding to the frame rate of the game video, or a constant cycle elapses. That is, the display control unit 15D synchronizes the number of downloads displayed for the clip already arranged on each team screen 251 with the number of downloads corresponding to the video ID of the clip in the highlight scene information 13b. This updates the display of the number of downloads to the latest one. Furthermore, the display control unit 15D shifts the position of each clip that has been arranged on each team screen 251 to the side opposite to the main screen 250 . As a result, a clip whose highlight scene is extracted earlier is displayed on the opposite side of the main screen 250, and a clip whose highlight scene is extracted later is displayed closer to the main screen 250. - 特許庁

FIG. 6 is a diagram showing an example of a method of shifting arranged clips. For convenience of explanation, FIG. 6 shows an excerpt of the display of the team screen 251B of the team screen 251A and the team screen 251B. Furthermore, FIG. 6 shows an example in which the right side of the game title "final" and the team name "B team" arranged at the left end position closest to the main screen 250 of the team screen 251B is the initial position of the clip. Note that t0, t1, t2 and t3 are assumed to be t0<t1<t2<t3.

As shown in FIG. 6, no clip is extracted at time t0. Subsequently, at time t1, the clip CB11 is placed at the initial position on the team screen 251B. After that, the process of shifting the clip CB11 to the right side of the team screen 251B by a constant shift amount is continued until the clip CB12 is arranged at the initial position of the team screen 251B at time t2. Then, the process of shifting the clip CB11 and the clip CB12 to the right side of the team screen 251B by a constant shift amount is continued until the clip CB13 is arranged at the initial position of the team screen 251B at time t3.

For convenience of explanation, FIG. 6 shows only the team screen 251B of the team screens 251A and 251B. However, the team screen 251A is the same except that the clip initial position and shift direction are horizontally reversed.

As a further aspect, the display control unit 15D narrows down the display to clips whose number of downloads satisfies a predetermined condition in games subsequent to the current game among the clips related to the game in which the competition has ended. As an example only, the display control unit 15D executes the following process every time a predetermined period, for example, a period corresponding to the frame rate of the game video, or a certain period elapses in subsequent games. That is, the display control unit 15D removes a clip whose number of downloads is less than the statistical value of the number of downloads of all clips, such as the average value, the median value, or the mode, among clips related to matches that have already been placed on the team screen 251 and the match has ended. After that, the display control unit 15D shifts the position of the clip related to the match that has already been arranged on the team screen 251 and for which the match has ended to the side opposite to the main screen 250 . The display position of the clip that has moved to the end of the team screen 251 opposite to the main screen 250 is fixed.

FIG. 7 is a diagram showing an example of a method of narrowing down clips relating to a match that has already been played. For convenience of explanation, FIG. 7 shows an extracted display of the team screen 251B of the team screen 251A and the team screen 251B. Furthermore, FIG. 7 shows an example in which the right side of the game title "first round" and the team name "B team" arranged at the leftmost position of the team screen 251B closest to the main screen 250 is set as the initial position of the clip. Note that t1, t2 and t3 are assumed to be t1<t2<t3.

As shown in FIG. 7, seven clips CB21 to CB27 are arranged on the team screen 251B at the stage of time t1 when the game of the B team ends. After that, the process of shifting the seven clips CB21 to CB27 to the right side of the team screen 251B by a constant amount of shift is continued until time t2 when the next game starts. Then, at time t2 when the next game starts, the number of downloads of the two clips, clip CB24 and clip CB27, is less than the statistical value of all clips being displayed on each team screen 251, and they are removed. As a result, the display of Team B on the team screen 251B is narrowed down to five clips: clip CB21, clip CB22, clip CB23, clip CB25, and clip CB26. Thereafter, the process of shifting the five clips of clip CB21, clip CB22, clip CB23, clip CB25, and clip CB26 to the right side of team screen 251B by a constant shift amount is continued until time t3 when the next game starts. At the time t3 when the next game starts, the number of downloads of the three clips of the clip CB21, the clip CB22 and the clip CB25 becomes less than the statistical value of all the clips displayed on each team screen 251, and the clips are further removed. As a result, the display of team B on the team screen 251B is narrowed down to two clips, clip CB23 and clip CB26.

For convenience of explanation, FIG. 7 shows only the team screen 251B of the team screens 251A and 251B, but the team screen 251A is the same except that the clip initial position and shift direction are horizontally reversed.

Although FIGS. 6 and 7 show examples in which all clips are shifted by a constant shift amount, different shift amounts may be used depending on the clip. For example, the shift amount of a clip can be changed depending on the position where the clip is arranged or the display size of the clip. For example, the clip can be shifted by a smaller shift amount as the position where the clip is arranged is closer to the main screen 250, or the clip can be shifted by a smaller shift amount as the display size of the clip is larger.

The decision unit 15E is a processing unit that decides which of the clips being displayed on the team screen 251 is to download the moving image of the highlight scene corresponding to the clip.

As one embodiment, the determination unit 15E receives a request to download a video of a highlight scene. Hereinafter, a request for downloading a video of a highlight scene may be referred to as a "DL (Download) request". As an example only, the DL request can be accepted via the user terminal 50 in which an application program for watching e-sports is installed. Hereinafter, an application program for watching e-sports may be referred to as an "e-sports watching application". For example, when a GUI operation for executing a DL request is performed while an e-sports watching application is activated on the user terminal 50 , the e-sports watching application activates a camera mounted on the user terminal 50 . After that, when the shooting operation of the camera is performed, the e-sports watching application makes a DL request of the highlight scene together with the clip image captured by the camera through the shooting operation to the display control device 10 .

When receiving a DL request for a highlight scene in this way, the determination unit 15E extracts the silhouette shape of the clip from the clip image received together with the DL request. After that, the determination unit 15E compares the silhouette shape included in the highlight scene information 13b and the silhouette shape extracted from the clip image by image processing such as template matching. As a result, when a silhouette shape that matches the silhouette shape extracted from the clip image is hit in the highlight scene information 13b, the determination unit 15E determines the highlight scene having the silhouette shape hit in the matching among the highlight scenes included in the highlight scene information 13b to be downloaded by the user terminal 50.

The distribution unit 15F is a processing unit that distributes the video of the highlight scene.

As one embodiment, the distribution unit 15</b>F distributes, to the user terminal 50 , the video of the highlight scene determined by the determination unit 15</b>E to be downloaded by the user terminal 50 . As a result, the video of the highlight scene is downloaded by the user terminal 50 . This makes it possible to provide spectators with a function of collecting their favorite highlight scenes by viewing the team screen 251 . By providing such a highlight scene collection function, it is possible to increase the incentive to visit esports venues.

FIG. 8 is a diagram illustrating an example of a highlight scene distribution method. FIG. 8 shows an example of downloading a highlight scene played back in clip CB31 of team screen 251B. As shown in FIG. 8, when a clip CB31 displayed on the team screen 251B is captured using the camera of the user terminal 50, a DL request for a highlight scene is sent from the user terminal 50 to the display control device 10 together with the clip image 51 captured by the camera (step S1). When a DL request for a highlight scene is received in this manner, a silhouette shape that matches the silhouette shape extracted from the clip image 51 is retrieved from the highlight scene information 13b. In the example shown in FIG. 8, the silhouette shape extracted from the clip image 51 hits the silhouette shape of the moving image ID "YYYYYYY". In this case, the highlight scene with the moving image ID "YYYYYYY" is determined as a download target by the user terminal 50 (step S2). After that, the video of the highlight scene determined to be downloaded by the user terminal 50 is delivered to the user terminal 50 (step S3). As a result, the video of the highlight scene is downloaded by the user terminal 50 . When the movie of the highlight scene is downloaded in this way, the number of downloads of the movie ID "YYYYYYY" in the record of the highlight scene information 13b is incremented from "76DL" to "77DL".

[Clip arrangement processing]
FIG. 9 is a flowchart illustrating the procedure of clip placement processing according to the first embodiment. As an example, step S101 of this process is executed before the start of the computer game match. As shown in FIG. 9, after the team screen 251 is assigned to each competing team of the computer game (step S101), the start of the computer game match is awaited (step S102).

After the game of the computer game is started, the collection unit 15A collects the game video and the game video from each player's point of view from the switcher 23 (step S103). Subsequently, the extracting unit 15B extracts a video of a highlight scene based on the degree of excitement obtained for each scene of the match video using a prediction model that has learned to predict the volume of the audience from the match video (step S104).

Then, when a new highlight scene is extracted in step S104 (Yes in step S105), the determination unit 15C identifies the team to which the player who takes action in the highlight scene belongs, thereby determining which team the highlight scene extracted in step S104 belongs to (step S106).

Subsequently, the determination unit 15C generates tags including various items such as the team name and player name to which the highlight scene determined in step S106 corresponds, the action to which the highlight scene corresponds, and the number of consecutive actions (step S107).

After that, the display control unit 15D sets the team screen 251 corresponding to the "team name" among the tags given to the highlight scenes extracted in step S104 as the display destination of the clip (step S108).

Further, the display control unit 15D sets the display size of the clip based on the "level of excitement" among the tags attached to the highlight scenes extracted in step S104 (step S109).

Further, the display control unit 15D sets the shape corresponding to the "silhouette shape" among the tags given to the highlight scenes extracted in step S104 as the display shape of the clip (step S110).

Then, the display control unit 15D instructs the display device 25 to arrange the clip in the display size set in step S109 and the display shape set in step S110 at the initial position on the team screen 251 where the clip is to be displayed set at step S108, for example, at a position closer to the main screen 250 (step S111).

Thereafter, the processes from step S103 to step S111 are repeatedly executed until the game of the computer game ends (step S112 No). Then, when the game of the computer game ends (step S112 Yes), the process ends.

[Clip update process]
FIG. 10 is a flowchart illustrating the procedure of clip update processing according to the first embodiment. This process can be executed in parallel with the process shown in FIG. As shown in FIG. 10, when a predetermined cycle, for example, a cycle corresponding to the frame rate of the game video, or a constant cycle has passed (step S301 Yes), the following processing is executed.

That is, the display control unit 15D synchronizes the number of downloads displayed for the clip already arranged on each team screen 251 with the number of downloads corresponding to the video ID of the clip in the highlight scene information 13b (step S302). This updates the display of the number of downloads to the latest one.

Further, the display control unit 15D shifts the position of each clip already arranged on each team screen 251 to the side opposite to the main screen 250 (step S303). As a result, a clip whose highlight scene is extracted earlier is displayed closer to the opposite side of the main screen 250 , and a clip whose highlight scene is extracted later is displayed closer to the main screen 250 .

Thereafter, the processes from step S301 to step S303 are repeatedly executed until the game of the computer game ends (step S304 No). Then, when the game of the computer game ends (step S304 Yes), the process ends.

[Clip delivery process]
FIG. 11 is a flowchart illustrating the procedure of clip distribution processing according to the first embodiment. This processing can be executed in parallel with the processing shown in FIGS. 9 and 10, for example. As shown in FIG. 11, when a DL request for a highlight scene is received (step S501 Yes), the determination unit 15E extracts the silhouette shape of the clip from the clip image received together with the DL request (step S502).

After that, the determination unit 15E searches for a silhouette shape that matches the silhouette shape extracted from the clip image from the highlight scene information 13b by image processing such as template matching (step S503).

As a result, when a silhouette shape matching the silhouette shape extracted from the clip image is hit in the highlight scene information 13b (step S504 Yes), the determination unit 15E performs the following processing. That is, the determining unit 15E determines highlight scenes having a silhouette shape hit by matching among the highlight scenes included in the highlight scene information 13b to be downloaded by the user terminal 50 (step S505).

Then, the distribution unit 15F distributes, to the user terminal 50, the video of the highlight scene determined by the determination unit 15E to be downloaded by the user terminal 50 (step S506). As a result, the video of the highlight scene is downloaded by the user terminal 50 . After that, among the records of the highlight scene information 13b, the number of downloads of the highlight scenes that have been downloaded is incremented (step S507), and the process ends.

On the other hand, if no silhouette shape that matches the silhouette shape extracted from the clip image is hit in the highlight scene information 13b (step S504 No), the determination unit 15E performs the following processing. That is, the determination unit 15E responds to the user terminal 50 that the download of the video of the highlight scene has failed (step S508), and ends the process.

[One aspect of the effect]
As described above, the display control device 10 according to the present embodiment implements display control for displaying the highlight scene in the display area of the team whose highlight scene is extracted from the e-sports game video, among the display areas corresponding to the e-sports competing teams. Through such display control, it is possible for spectators to identify whether the highlight of the team they are cheering for is being displayed or the highlight of the team currently competing. Therefore, according to the display control device 10 according to the present embodiment, it is possible for spectators to easily grasp the highlight scenes of each of the multiple teams during the match.

Although embodiments of the disclosed apparatus have been described so far, the present invention may be embodied in various forms other than the embodiments described above. Therefore, other embodiments included in the present invention will be described below.

[Screen layout application example 1]
In the above-described first embodiment, the layout of the screen when there are two competing teams is illustrated, but the scope of application of the above display control function is not limited to when there are two competing teams, and the above display control function can also be applied when there are three or more competing teams.

FIG. 12 is a diagram showing another example of the screen layout. FIG. 12 shows the layout of the screen when four teams, A team, B team, C team, and D team, each consisting of four teams, compete in an e-sports competition. As shown in FIG. 12, when viewing the main screen 250 from the front, the team screen 251A, the team screen 251B, the main screen 250, the team screen 251C, and the team screen 251D are arranged in this order from the left.

Among them, the team screen 251A is assigned to the A team, and the team screen 251B is assigned to the B team. Furthermore, the team screen 251C is assigned to the C team, and the team screen 251D is assigned to the D team.

A plurality of clips are arranged in chronological order on the team screen 251A, team screen 251B, team screen 251C, and team screen 251D. In the screen layout shown in FIG. 12, similarly to the screen layout shown in FIG. 2, the earlier the clip is extracted, the farther the clip is displayed from the main screen 250, and the closer the clip extraction timing is to the current time, for example, the time when the latest frame of the live game video is being reproduced, the more the clips are displayed side by side on the main screen 250.

[Screen layout application example 2]
In the first embodiment, the team screen 251A and the team screen 251B are arranged side by side on the left and right of the main screen 250. However, the team screen 251 does not necessarily have to be arranged on the left and right of the main screen 250, and the shape of the team screen 251 is not necessarily limited to a specific figure such as a rectangle.

FIG. 13 is a diagram showing another example of the screen layout. FIG. 13 also shows the layout of the screen when four teams, A team, B team, C team, and D team, each consisting of four teams, compete in an e-sports competition. As shown in FIG. 13, a team screen 251A, a team screen 251B, a team screen 251C, and a team screen 251D can be arranged radially around the main screen 250 in a shape other than a rectangle. For example, when the main screen 250 is viewed from the front, the team screens 251A, 251B, 251C, and 251D are arranged in clockwise order from 9 o'clock.

In these team screens 251A, 251B, 251C and 251D as well, multiple clips are arranged in chronological order. In the screen layout shown in FIG. 13, similarly to the screen layout shown in FIG. 2, the earlier the clip is extracted, the farther the clip is displayed from the main screen 250, and the closer the clip extraction timing is to the current time, for example, the time when the latest frame of the live game video is being reproduced, the more the clips are displayed side by side on the main screen 250.

[Application example of highlight classification]
In the above-described first embodiment, an example is given in which highlight scenes are classified by game and by team and clips are arranged, but the classification of highlights is not necessarily limited to this, and clips can be arranged by other classifications.

FIGS. 14A-14C are diagrams illustrating application examples of highlight classification. For convenience of explanation, FIGS. 14A to 14C selectively show the display of the team screen 251B of the team screens 251A and 251B.

For example, clips can be arranged by classifying highlight scenes according to past games and teams. That is, as shown in FIG. 14A, the clip of the B team's game today and the clip of the B team's yesterday's game can be arranged in order of proximity from the main screen 250 . You can also arrange clips by classifying highlight scenes by action. That is, as shown in FIG. 14B , a clip related to the action “dunk” and a clip related to the action “crossover” can be arranged in order of proximity from the main screen 250 . Furthermore, clips can be arranged by classifying highlight scenes by player. That is, as shown in FIG. 14C, a clip related to player C and a clip related to player D can be arranged in descending order from the main screen 250 .

[Application to real sports]
In the above-described first embodiment, an example in which the display control function is applied to watching e-sports is given, but the scope of application of the display control function is not limited to watching e-sports, and the display control function can also be applied to watching real sports other than e-sports.

FIG. 15 is a diagram showing another example of the screen layout. FIG. 15 shows the layout of the screen when the A team and the B team are competing in basketball. For example, when watching a basketball game, as shown in FIG. 15, clips can be arranged by classifying highlight scenes according to the 1st quarter, 2nd quarter, 3rd quarter, and 4th quarter.

When determining for which team a highlight scene of a real sport such as basketball is a highlight scene, the determination unit 15C can use motion tracking technology.

FIG. 16 is a diagram showing an example of camera arrangement in motion tracking. As shown in FIG. 16, the real space on the basketball field F is generated as a free-viewpoint video of watching a basketball game. In this case, a plurality of cameras 41A-41N are arranged from the periphery of the field F toward the interior of the field F. FIG. At this time, each camera 20 is installed in such an arrangement that the entire area of the field F fits within the photographing range of the plurality of cameras 41A to 41N by combining the photographing ranges of the plurality of cameras 20. FIG. Furthermore, in order to analyze the three-dimensional structure of the subject existing on the field F from the pixels on the images common to the two or more cameras 41, each of the cameras 41A to 41N is arranged in such a manner that part of the photographing range overlaps with that of the other camera 41. Under such an arrangement, the plurality of cameras 41A to 41N are synchronously photographed for each frame, so that a plurality of images photographed at the same timing for different viewpoints, that is, multi-viewpoint images are obtained for each frame.

A tracking device 40 that acquires multi-viewpoint images from these cameras 41A to 41N analyzes three-dimensional information of subjects such as players and balls from the multi-viewpoint images captured by the plurality of cameras 41A to 41N. This makes it possible to generate a free-viewpoint video of watching a basketball game. In addition, the tracking device 40 can provide the display control device 10 with tracking information in which movements of the ball, players, and referees are tracked from the above three-dimensional information. Using such tracking information, when a highlight scene is extracted, the determination unit 15C can determine that the highlight scene is the highlight scene for the team to which these players belong by specifying the player who made the shot that scored the goal or the player who passed the shot to the player who took the shot. In this way, when specifying a player related to a score, score, points, etc., taking ball games as an example, the team can be determined from the player who was in contact with the ball or the like immediately before or immediately before the score or score changed. Alternatively, the team whose score or score has changed may be identified from the scoreboard or the like by character recognition.

Distributed and integrated
Also, each component of each illustrated device may not necessarily be physically configured as illustrated. In other words, the specific form of distribution and integration of each device is not limited to the illustrated one, and all or part of them can be functionally or physically distributed and integrated in arbitrary units according to various loads and usage conditions. For example, the collection unit 15A, the extraction unit 15B, the determination unit 15C, the display control unit 15D, the determination unit 15E, or the distribution unit 15F may be connected to the display control device 10 via a network as external devices. Alternatively, the collecting unit 15A, the extracting unit 15B, the determining unit 15C, the display control unit 15D, the determining unit 15E, or the distributing unit 15F may be provided in separate devices, which may be connected to a network and cooperate to realize the functions of the display control device 10 described above.

[Display control program]
Moreover, various processes described in the above embodiments can be realized by executing a prepared program on a computer such as a personal computer or a work station. Therefore, an example of a computer that executes a display control program having functions similar to those of the above embodiments will be described below with reference to FIG.

FIG. 17 is a diagram illustrating a hardware configuration example of a computer. As shown in FIG. 17, the computer 100 has an operation section 110a, a speaker 110b, a camera 110c, a display 120, and a communication section . Furthermore, this computer 100 has a CPU 150 , a ROM 160 , an HDD 170 and a RAM 180 . Each part of these 110 to 180 is connected via a bus 140 .

As shown in FIG. 17, the HDD 170 stores a display control program 170a that exhibits the same functions as the collection unit 15A, the extraction unit 15B, the determination unit 15C, the display control unit 15D, the determination unit 15E, and the distribution unit 15F shown in the first embodiment. The display control program 170a may be integrated or separated like the components of the collection unit 15A, the extraction unit 15B, the determination unit 15C, the display control unit 15D, the determination unit 15E, and the distribution unit 15F shown in FIG. That is, the HDD 170 does not necessarily store all the data shown in the first embodiment, and the HDD 170 only needs to store data used for processing.

Under such an environment, the CPU 150 reads the display control program 170 a from the HDD 170 and develops it in the RAM 180 . As a result, the display control program 170a functions as a display control process 180a as shown in FIG. The display control process 180a expands various data read from the HDD 170 into an area assigned to the display control process 180a among the storage areas of the RAM 180, and executes various processes using the expanded various data. For example, examples of processing executed by the display control process 180a include the processing shown in FIGS. 9 to 11. FIG. Note that the CPU 150 does not necessarily have to operate all the processing units described in the first embodiment, as long as the processing units corresponding to the processes to be executed are virtually realized.

Note that the display control program 170a described above does not necessarily have to be stored in the HDD 170 or the ROM 160 from the beginning. For example, the display control program 170a is stored in a “portable physical medium” such as a flexible disk inserted into the computer 100, so-called FD, CD-ROM, DVD disk, magneto-optical disk, IC card, or the like. Then, the computer 100 may acquire and execute the display control program 170a from these portable physical media. Alternatively, the display control program 170a may be stored in another computer or server device connected to the computer 100 via a public line, the Internet, LAN, WAN, etc., and the computer 100 may acquire the display control program 170a therefrom and execute it.

The following notes are further disclosed with respect to the embodiments including the above examples.

(Appendix 1) an extraction unit that extracts highlight scenes based on analysis results of videos of matches between multiple teams;
a determination unit that determines for which of the plurality of teams the highlight scene is a highlight scene;
a display control unit that displays the highlight scene in a display area corresponding to the team determined by the determination unit, among display areas that are assigned to each of the plurality of teams and do not overlap with each other;
A display control device comprising:

(Appendix 2) The display control device according to appendix 1, wherein a plurality of highlight scenes of the team determined by the determination unit, including the highlight scene, are displayed in chronological order in a display area corresponding to the team determined by the determination unit.

(Supplementary note 3) The display control device according to Supplementary note 1, wherein the determination unit determines which of the plurality of teams the highlight scene is for, based on character information of the player read from the video.

(Supplementary Note 4) The display control device according to Supplementary Note 1, wherein the determination unit determines which of the plurality of teams the highlight scene is for, based on the tracking information of the video.

(Additional remark 5) The display control unit displays a plurality of highlight scenes of the team determined by the determination unit, including the highlight scene, in a plurality of small areas having different shapes within the team display area,
a determination unit that receives a request for video information from a terminal operated by a user, identifies the shape of a small area included in the request, and determines one of the plurality of highlight scenes corresponding to the identified shape;
The display control device according to Supplementary Note 1, further comprising: a distribution unit that distributes the moving image information of the determined highlight scene to the terminal of the user.

(Appendix 6) Based on the analysis results of the video of the match by multiple teams, extract the highlight scene,
determining which of the plurality of teams the highlight scene is for,
displaying the highlight scene in a display area corresponding to the team determined by the determining process, among the display areas assigned to each of the plurality of teams and not overlapping each other;
A display control method characterized in that a computer executes processing.

(Appendix 7) The display control method according to appendix 6, wherein a plurality of highlight scenes of the team determined by the determination process, including the highlight scene, are displayed in chronological order in a display area corresponding to the team determined by the determination process.

(Supplementary Note 8) The display control method according to Supplementary note 6, wherein the determining process determines which of the plurality of teams the highlight scene is for, based on character information of the player read from the video.

(Supplementary note 9) The display control method according to Supplementary note 6, wherein the determining process determines which of the plurality of teams the highlight scene belongs to, based on the tracking information of the video.

(Supplementary Note 10) The display process displays a plurality of highlight scenes of the team determined by the determination process, including the highlight scene, in a plurality of small areas having different shapes within the team display area,
A process of accepting a request for video information from a terminal operated by a user, identifying the shape of a small area included in the request, and determining any one of the plurality of highlight scenes corresponding to the identified shape;
7. The display control method according to claim 6, wherein the computer further executes a process of distributing the moving image information of the determined highlight scene to the terminal of the user.

(Appendix 11) extracting highlight scenes based on analysis results of videos of matches by multiple teams,
determining which of the plurality of teams the highlight scene is for,
displaying the highlight scene in a display area corresponding to the team determined by the determining process, among the display areas assigned to each of the plurality of teams and not overlapping each other;
A display control program that causes a computer to execute processing.

(Supplementary note 12) The display control program according to Supplementary note 11, wherein a plurality of highlight scenes of the team determined by the determination process, including the highlight scene, are displayed in chronological order in a display area corresponding to the team determined by the determination process.

(Supplementary Note 13) The display control program according to Supplementary Note 11, wherein the determining process determines which of the plurality of teams the highlight scene is for, based on character information of the player read from the video.

(Supplementary note 14) The display control program according to Supplementary note 11, wherein the determining process determines which of the plurality of teams the highlight scene belongs to, based on the tracking information of the video.

(Supplementary Note 15) The displaying process displays a plurality of highlight scenes of the team determined by the determining process, including the highlight scene, in a plurality of small areas having different shapes within the team display area,
A process of accepting a request for video information from a terminal operated by a user, identifying the shape of a small area included in the request, and determining any one of the plurality of highlight scenes corresponding to the identified shape;
12. The display control program according to appendix 11, further causing a computer to execute a process of distributing the moving image information of the determined highlight scene to the terminal of the user.

1 system 10 display control device 11 communication interface 13 storage unit 13a video information 13b highlight scene information 15 control unit 15A collection unit 15B extraction unit 15C determination unit 15D display control unit 15E determination unit 15F distribution unit

Claims (7)

an extraction unit that extracts a highlight scene including a frame with a peak degree of excitement by inputting images of frames included in a video of a match between a plurality of teams to a machine learning model that uses the images as input and outputs the degree of excitement of spectators;
a determination unit that determines for which of the plurality of teams the highlight scene is a highlight scene;
a display control unit that sets, based on the excitement level of the highlight scene, a display size when arranging a clip reproducing the video of the highlight scene in the display area corresponding to the team determined by the determination unit among the display areas that are assigned to each of the plurality of teams and do not overlap with each other;
A display control device comprising: 2. The display control device according to claim 1, wherein a plurality of highlight scenes of the team determined by the determination unit, including the highlight scene, are displayed in chronological order in a display area corresponding to the team determined by the determination unit. 2. The display control device according to claim 1, wherein the determination unit determines which of the plurality of teams the highlight scene belongs to, based on character information of the player read from the video. 2. The display control device according to claim 1, wherein the determination unit determines which of the plurality of teams the highlight scene belongs to, based on the tracking information of the video. The display control unit displays a plurality of highlight scenes of the team determined by the determination unit, including the highlight scene, in a plurality of small areas having different shapes within the display area of the team, and
a determination unit that receives a request for video information from a terminal operated by a user, identifies the shape of a small area included in the request, and determines one of the plurality of highlight scenes corresponding to the identified shape;
2. The display control apparatus according to claim 1, further comprising a distribution unit that distributes the moving image information of the determined highlight scene to the terminal of the user. By inputting the image of the frame included in the video of the match between multiple teams into a machine learning model that uses the image as input and outputs the degree of excitement of the spectators, extracting the highlight scene including the frame with the peak degree of excitement,
determining which of the plurality of teams the highlight scene is for,
setting a display size for arranging a clip for reproducing the video of the highlight scene in the display area corresponding to the team determined by the determination process, among the display areas assigned to each of the plurality of teams and not overlapping each other, based on the degree of excitement of the highlight scene;
A display control method characterized in that a computer executes processing. By inputting the image of the frame included in the video of the match between multiple teams into a machine learning model that uses the image as input and outputs the degree of excitement of the spectators, extracting the highlight scene including the frame with the peak degree of excitement,
determining which of the plurality of teams the highlight scene is for,
setting a display size for arranging a clip for reproducing the video of the highlight scene in the display area corresponding to the team determined by the determination process, among the display areas assigned to each of the plurality of teams and not overlapping each other, based on the degree of excitement of the highlight scene;
A display control program that causes a computer to execute processing.

Images