JP7302256B2 - HIGHLIGHT MOVIE GENERATION PROGRAM, HIGHLIGHT MOVIE GENERATION METHOD, AND INFORMATION PROCESSING APPARATUS - Google Patents

Description

TECHNICAL FIELD Embodiments of the present invention relate to a program for generating a highlight moving image, a method for generating a highlight moving image, and an information processing apparatus.

2. Description of the Related Art In recent years, watching computer games such as e-Sprots has become popular, and when watching a game, exciting highlight scenes of the game are presented as a digest. As a conventional technique for presenting such a highlight scene, there is a technique for detecting a highlight scene in content based on an evaluation value calculated based on feature data extracted from image or audio data of the content. Are known.

JP-A-2007-123981

However, in the above-described prior art, since the highlight scene is detected from the image or audio data of the content, there is a problem that the highlight scene may be presented as a highlight scene unrelated to the excitement of the audience of the computer game. be.

An object of one aspect of the present invention is to provide a highlight movie generation program, a highlight movie generation method, and an information processing apparatus that can support the presentation of highlight scenes that excite the audience of a computer game.

In the first proposal, the highlight moving image generation program causes a computer to execute division processing, calculation processing, generation processing, and output processing. The dividing process divides the moving image of the computer game into scenes. The calculation process is based on the game image included in the scene for each divided scene, using a prediction model that predicts the degree of excitement of the audience who viewed the game image from the game image of the computer game. Calculate the degree of excitement in the case. The generating process selects a scene to be used for the highlight moving image from among the divided scenes based on the calculated excitement level, and generates the highlight moving image. The output process outputs the generated highlight video.

According to one embodiment of the present invention, it is possible to support the presentation of highlight scenes that excite the audience of a computer game.

FIG. 1 is an explanatory diagram for explaining the outline of the embodiment. FIG. 2 is an explanatory diagram for explaining a screen example of a game moving image. FIG. 3 is a block diagram of a functional configuration example of the information processing apparatus according to the embodiment; FIG. 4 is a flowchart illustrating an example of processing for generating a prediction model. FIG. 5 is an explanatory diagram for explaining an example of calculation of the excitement degree by the prediction model. 6 is a flowchart illustrating an operation example of the information processing apparatus according to the embodiment; FIG. FIG. 7 is an explanatory diagram showing an example of division for each scene. FIG. 8 is an explanatory diagram for explaining the calculation of the excitement degree of each scene. FIG. 9 is an explanatory diagram for explaining the calculation of the excitement degree of each scene. FIG. 10 is an explanatory diagram illustrating generation of a highlight video. FIG. 11 is an explanatory diagram illustrating an example of providing a highlight video. FIG. 12 is a block diagram showing an example of a computer that executes a highlight moving image generation program.

A highlight video generation program, a highlight video generation method, and an information processing apparatus according to embodiments will be described below with reference to the drawings. Configurations having the same functions in the embodiments are denoted by the same reference numerals, and overlapping descriptions are omitted. Note that the highlight moving image generation program, highlight moving image generation method, and information processing apparatus described in the following embodiments are merely examples, and do not limit the embodiments. Moreover, each of the following embodiments may be appropriately combined within a non-contradictory range.

[Overview of Embodiment]
FIG. 1 is an explanatory diagram for explaining the outline of the embodiment. In this embodiment, for example, based on a video image of a computer game such as e-Sprots (hereinafter referred to as a game video), a highlight video that presents a digest of exciting highlight scenes is created. For example, a PC (personal computer) or the like can be applied as an information control device for creating this highlight moving image.

Note that computer games for creating highlight videos include shooting games, strategy games, fighting games, sports games, racing games, puzzle games, etc., and are not particularly limited. In the present embodiment, as an example, a real-time strategy game, which is a battle-type game in which a plurality of players compete against each other, is illustrated with a game screen in which each player in the game is viewed from above.

As shown in FIG. 1, the information processing apparatus first receives data input regarding highlight creation (S1). Specifically, the information control device receives data input of a game video for which highlights are to be created and setting information such as the number of highlight scenes to be created and the number of seconds per scene.

The game moving image is a moving image related to the game screen from the start to the end of the game, and is moving image data of a predetermined frame rate (for example, 30 fps).

FIG. 2 is an explanatory diagram for explaining a screen example of a game moving image. As shown in FIG. 2, the game animation G includes, for example, a game screen G1 during play and an information screen G2 that displays the state of the player and various statuses related to the game. The game screen G1 has a configuration in which the players P1 and P2 in the game are viewed from above as images of a virtual camera. The screen configuration of the game screen G1 is not limited to a bird's-eye view of the players P1 and P2, and may be a view of the game field as the players P1 and P2 in the game.

Also, on the game screen G1, the viewpoints of the players P1 and P2 are switched by camera switching. For example, the game screen G1 changes from a continuous scene (upper part in the drawing) of the viewpoint of the player P1 centered on the player P1 to a continuous scene of the viewpoint of the player P2 centered on the player P2 (Fig. middle row). Similarly, the game screen G1 returns from the continuous scene of the viewpoint of the player P2 (middle part of the figure) to the continuous scene of the viewpoint of the player P1 (lower part of the figure) by switching the camera.

Next, the information processing device creates a highlight moving image based on data input regarding highlight creation (S2). Specifically, the information processing device divides the game video into scenes at the timing of camera switching (S2a). Here, the information processing device tags which of the first half, middle stage, and second half of the game each divided scene corresponds to.

Further, the information processing device divides the game moving image into still images (hereinafter referred to as game images) at intervals of a predetermined time (for example, 1 second). Specifically, the information processing device extracts frames at intervals of, for example, one second from a game video having a predetermined frame rate.

Next, the information processing device uses a prediction model for predicting the degree of excitement of the spectators who viewed the game image for each of the divided game images, and calculates the degree of excitement when the spectators view the game image (S2b). The predictive model used to calculate the degree of excitement is a learning model created by machine learning such as deep learning, based on training data in which the degree of excitement of the audience for the game image is given as the correct answer (details will be described later). .

Next, the information processing device cuts each scene divided in S2a into the number of seconds designated by the setting information. Next, the information processing device calculates (predicts) the degree of excitement when the audience views each cut scene based on the degree of excitement of each game image calculated in S2b (S2c). As an example, the information processing device obtains the average excitement degree of the game images included in each scene.

Next, the information processing device selects and joins a specified number of scenes from among the divided scenes based on the excitement level of each scene to generate a highlight video (S2d). For example, the information processing apparatus arranges the scenes in order of excitement level and selects the scenes in descending order of excitement. Next, the information processing device arranges the selected scenes in chronological order and connects them to generate a highlight video.

Here, the information processing device refers to the tag indicating which part of the first half, middle stage, or latter half the scene belongs to, and selects at least one scene for each part. As a result, it is possible to generate a highlight video suitable for a series of digests from the start to the end of the game, including exciting scenes in each of the first half, the middle stage, and the second half.

Next, the information processing apparatus outputs the highlight moving image to the user by displaying the generated highlight moving image on the display, outputting the file, or the like (S3). As a result, the user can easily obtain a highlight video of a highlight scene that excites the audience from a game video of a computer game.

[About the information processing device]
FIG. 3 is a block diagram of a functional configuration example of the information processing apparatus according to the embodiment; As shown in FIG. 3 , the information processing device 1 has a communication section 10 , a storage section 20 and a control section 30 .

The communication unit 10 controls communication with a connected external device regardless of whether it is wired or wireless. The communication unit 10 is, for example, a communication interface such as a NIC (Network Interface Card).

The storage unit 20 stores various data such as programs executed by the control unit 30, game moving image information 21, setting information 22, prediction model information 23, and scene-specific information DB 24, for example. The storage unit 20 corresponds to a semiconductor memory device such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, or a storage device such as HDD (Hard Disk Drive).

The game video information 21 is information on various game videos such as a game video for which highlights are to be created and a game video for creating teacher data. Specifically, the game moving image information 21 is a moving image file in a predetermined file format such as AVI format or MPEG-4 format. It is assumed that the game moving image for creating the teacher data contains the sound volume of the game moving image at the spectator seats, that is, the sound volume uttered by the spectators as sound volume data.

The setting information 22 is, for example, various setting data received in data input related to highlight creation. Specifically, the setting information 22 includes the number of highlight scenes to be created, the number of seconds per scene, and the like.

The prediction model information 23 is information relating to a prediction model for predicting the excitement level of the spectators who viewed the game image.

Here, the prediction model is, for example, a learning model created by machine learning such as deep learning so as to predict the excitement level of the audience for the game image. As a learning model, for example, a neural network in which units simulating brain neurons are hierarchically connected from an input layer to an output layer via an intermediate layer can be applied. Therefore, the prediction model information 23 is information (for example, parameters in each node of the neural network) related to a learning model (prediction model) that has been trained by machine learning such as deep learning.

The scene-by-scene information DB 24 is a database that stores various kinds of information about each scene divided from the game moving image of the game moving image information 21 (details will be described later).

The control unit 30 is a processing unit that controls overall processing of the information processing device 1 . The control unit 30 is implemented by, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like executing a program stored in an internal storage device using a RAM as a work area. Also, the control unit 30 may be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

The control unit 30 has an input unit 31 , a division unit 32 , a teacher data creation unit 33 , a prediction model creation unit 34 , a degree of excitement calculation unit 35 , a highlight video creation unit 36 and an output unit 37 . Note that the input unit 31, the dividing unit 32, the teacher data creating unit 33, the prediction model creating unit 34, the degree of excitement calculating unit 35, the highlight moving image creating unit 36, and the output unit 37 are examples of electronic circuits possessed by the processor and It is an example of a process to execute.

The input unit 31 is a processing unit that performs input processing of various data input via the communication unit 10 or the like. For example, the input unit 31 receives inputs such as game videos for which highlights are to be created and game videos for creating teacher data, and stores them in the storage unit 20 as game video information 21 . The input unit 31 also receives input such as the number of highlight scenes to be created and the number of seconds of the scene, and stores the input as the setting information 22 in the storage unit 20 . The input unit 31 also stores information about the prediction model that is input when the prediction model is created by an external device in the storage unit 20 as the prediction model information 23 .

The dividing unit 32 is a processing unit that performs processing related to dividing the game moving image. The dividing unit 32 reads out the game moving image information 21 stored in the storage unit 20 and divides the game moving image at scene boundaries that satisfy a predetermined condition. For example, the dividing unit 32 compares the RGB values of the frames before and after each frame, and divides the game moving image when there is a difference equal to or greater than a predetermined value. As a result, the dividing unit 32 divides the game video into scenes at the timing of camera switching.

Further, the dividing unit 32 divides the game moving image into game images at predetermined time intervals (for example, one second) by extracting frames at predetermined intervals. Next, the division unit 32 outputs the extracted game image to the teacher data creation unit 33, excitement degree calculation unit 35, and the like.

Further, the dividing unit 32 stores information about the divided scenes in the scene-by-scene information DB 24 . Specifically, the dividing unit 32 stores information such as a game image extracted for each scene in the scene-by-scene information DB 24 for each scene name (for example, scene A, scene B, . . . ) that identifies the divided scenes.

The teacher data creation unit 33 is a processing unit that creates a data set (teacher data) by combining game images and the degree of excitement of the audience with respect to the game images from game animations for creating teacher data.

The prediction model creation unit 34 is a processing unit that creates a prediction model by machine learning such as deep learning based on the teacher data created by the teacher data creation unit 33 .

Here, the creation of the prediction model will be described in detail. FIG. 4 is a flowchart illustrating an example of processing for creating a prediction model.

As shown in FIG. 4, when the process is started, the dividing unit 32 divides the game video for creating teacher data into a game image at predetermined time intervals (for example, one second) and volume (spectator seats of the game video). volume) and (S10).

Next, the teacher data creation unit 33 normalizes the volume divided for each predetermined time (S11). Specifically, the teacher data creation unit 33 sets the maximum volume to 1 and the minimum volume to 0 among all the volumes divided from the game video, and normalizes the values (0 to 1) according to the volume. become Due to this normalization, 1 is assigned to the loudest and most excited sounds of the audience, and 0 is assigned to the least enthusiastic sounds.

Next, the teacher data creation unit 33 classifies the normalized volume data into a plurality of categories according to volume (S12). For example, when classifying into categories from 0 to 3 according to the volume, the teacher data creation unit 33 assigns 3 (extremely exciting) when the normalized volume is 0.8 or more. Also, the teacher data creation unit 33 assigns 2 (excited) when the normalized volume is 0.6 or more and less than 0.8. Also, the teacher data creation unit 33 sets 1 (slightly raised) when the normalized volume is 0.4 or more and less than 0.6. Also, the teacher data creation unit 33 sets the normalized volume to 0 (not upbeat) when it is less than 0.4. In this way, the teacher data creation unit 33 classifies the images into categories according to the excitement of the audience.

Next, the training data creation unit 33 determines which category the game image divided at predetermined time intervals belongs to (S13). Specifically, the teacher data creation unit 33 determines the category based on which category the volume of the divided game image is classified.

Next, the training data creation unit 33 performs the above-described processing of S11 to S13 on each of the divided game images, thereby creating a data set ( teacher data) is created (S14).

Next, the prediction model creation unit 34 creates a prediction model 2 that predicts the degree of excitement (category value indicating the excitement of the audience) by machine learning such as deep learning based on the created data set (S15). Next, the predictive model creating unit 34 stores the created predictive model 2 information as the predictive model information 23 in the storage unit 20 .

For example, the predictive model creation unit 34 inputs a game image to the input layer of the predictive model 2 and causes the output layer to output an output value indicating the calculation result. Next, the prediction model creation unit 34 sets the parameters at each node of the neural network of the prediction model 2 based on the comparison between the correct label (category value indicating the excitement of the audience) given to the game video and the output value. learn. Specifically, the prediction model creation unit 34 learns the parameters of the neural network by error back propagation (BP) method using the comparison result between the output value and the correct label. Next, the predictive model creating unit 34 stores information such as parameter values of each node in the created predictive model 2 in the storage unit 20 as the predictive model information 23 .

By using the prediction model 2 created in this way, the information processing device 1 calculates (predicts) the excitement level of the audience from the game image. FIG. 5 is an explanatory diagram for explaining an example of calculation of the excitement degree by the prediction model 2. As shown in FIG.

As shown in FIG. 5, the prediction model 2 is subjected to machine learning such as deep learning using a data set 2a that is a combination of, for example, 200,000 or more game images and the degree of excitement for the game images. Therefore, by using the prediction model 2, the information processing device 1 can calculate the degree of excitement in each game image G10 extracted from the game moving image G every second, for example. Specifically, the information processing apparatus 1 inputs each game image G10 to the prediction model 2, and obtains an output of the degree of excitement from the prediction model 2. FIG.

Returning to FIG. 3, for each scene divided by the dividing unit 32, the excitement level calculation unit 35 calculates the excitement level when the audience views the scene using the prediction model 2 based on the game image G10 included in the scene. It is a processing unit that predicts.

The highlight moving image creating unit 36 is a processing unit that selects a scene to be used for the highlight moving image from among the divided scenes based on the excitement degree predicted by the excitement calculating unit 35, and generates the highlight moving image. The output unit 37 is a processing unit that outputs the highlight moving image generated by the highlight moving image generating unit 36 .

Here, the details of the processing relating to the excitement level calculation unit 35, the highlight moving image creation unit 36, and the output unit 37 will be described. 6 is a flowchart illustrating an operation example of the information processing apparatus according to the embodiment; FIG.

As shown in FIG. 6, when the process is started, the input unit 31 receives an input of a game animation G for which highlights are to be created (S20). The received game animation G is stored in the storage unit 20 as the game animation information 21 .

Next, the dividing unit 32 reads out the game moving image information 21 and divides the game moving image G into scenes (S21). Further, the dividing unit 32 divides (divides) the game animation G into a plurality of parts from the start to the end of the game (S22).

FIG. 7 is an explanatory diagram showing an example of division for each scene. As shown in FIG. 7, the dividing unit 32 compares the RGB values before and after the game images G10a, G10b, . Next, the dividing unit 32 determines that the camera has switched when the change in RGB values before and after is greater than a predetermined value (large change), and performs scene division. In the illustrated example, the dividing unit 32 divides the game image G10b and the game image G10c into scene A and scene B because there is a large change. The dividing unit 32 assigns a scene name and the like to each of the divided scenes (scene A, scene B, . . . ) and registers them in the scene-specific information DB 24 .

Further, the dividing unit 32 divides (divides) the game into a plurality of parts from the start to the end of the game, and gives scene tags indicating which part contains each scene. For example, the dividing unit 32 may divide the game from the start to the end into three equal parts, and give each scene a scene tag indicating which of the first half, the middle part, and the second half it belongs to. Note that the number of divided parts is not limited to three, and may be any number, such as four parts for starting, turning, and ending. Also, the division method is not limited to equal division, and division may be performed in intervals of a predetermined time (for example, 10 minutes). The scene tag information for each scene is stored in the scene-specific information DB 24 .

Returning to FIG. 6, after S20, the dividing unit 32 divides the game moving image G into game images G10 at predetermined time intervals (for example, one second) (S23). Information about the divided game image G10 (for example, file name (image name)) is stored in the scene-specific information DB 24 as the game image G10 of the corresponding scene among the divided scenes in S21.

Next, the excitement level calculation unit 35 predicts the excitement level of each divided game image G10 by inputting each game image G10 into the prediction model 2 (S24). The processing of S23 and S24 may be performed in parallel with S21 and S22, or may be performed in series. The value of the degree of excitement predicted for each game image G10 is stored in the scene-specific information DB 24 as the degree of excitement of each game image G10.

Next, the excitement degree calculation unit 35 calculates the excitement degree for each scene based on the predicted excitement degree for each of the game images G10 included in each scene (S25).

8 and 9 are explanatory diagrams for explaining calculation of the excitement level of each scene. As shown in FIG. 8, a scene tag is assigned to each divided scene by the processing of S21 and S22. In addition, by the processing of S23 and S24, the degree of excitement is predicted for the game image G10 divided at predetermined time intervals. Therefore, the scene-specific information DB 24 stores data obtained by combining the above processing results. Specifically, the scene-specific information DB 24 stores, for each scene, the scene tag, the game image G10 included in the scene, and the excitement level.

By referring to the scene-by-scene information DB 24, the excitement degree calculation unit 35 calculates the excitement degree for each scene. For example, when calculating the excitement level of scene A, the excitement level calculation unit 35 reads the excitement level of the game image G10 (001.img, 002.img, 003.img) corresponding to scene A, and reads the excitement level of scene A. degree (for example, total value).

Next, the highlight moving image creation unit 36 divides each scene in the scene-specific information DB 24 into the time specified as the number of seconds per scene in the setting information 22 (S26). For example, as shown on the right side of FIG. 8, when one scene is set to 10 seconds, the highlight movie creating unit 36 sets the time (t1) at which the degree of excitement peaks for each scene in the scene-specific information DB 24. 5 seconds before and after the center is taken as one scene.

Next, the highlight moving image creating unit 36 determines whether or not scenes overlap due to division by time designation (S27). If the scenes overlap, the highlight moving image creating unit 36 excludes the scene with a lower degree of excitement among the overlapping scenes from the selection target of the highlight moving image (hereinafter referred to as the selection target) (S28).

For example, as shown on the left side of FIG. 9, for scene A, one scene is five seconds before and after the time (t1) at which the degree of excitement peaks. As for the scene B, one scene is five seconds before and after the time (t2) at which the degree of excitement peaks. If the peak times (t1, t2) of the degree of excitement are close to each other, scenes overlap.

When such duplication occurs, the highlight moving image creating unit 36 compares the peak value of the degree of excitement in each scene, and excludes scenes with a low degree of excitement from the selection targets. For example, in the illustrated example, scene B is excluded from selection targets.

Next, the highlight moving image creating unit 36 calculates the average excitement level for each scene (S29). For example, as shown on the right side of FIG. 9, for scene A, an average value of (2+3)/2=2.5 is obtained based on the excitement level of each game image G10 included in the scene. Here, the highlight moving image creating unit 36 selects the number of scenes designated as the number of highlight scenes in the setting information 22 in descending order of the calculated average.

Next, the highlight moving image creating unit 36 refers to the scene tag and determines whether or not each part has a scene to be selected (S30).

If there is a part that does not have an exciting scene (S30: No), the highlight moving image creating unit 36 selects a scene with an exciting degree exceeding a predetermined threshold in that part (S31). If each part has a prime scene (S30: Yes), the process proceeds directly to S32.

FIG. 10 is an explanatory diagram illustrating generation of a highlight video. As shown in FIG. 10, the highlight moving image creation unit 36 selects four scenes from scenes A to I in descending order of excitement (average) (S31a). Here, there is no scene to be selected for the middle part. In this way, for the middle part without scenes to be selected, the highlight moving image creating unit 36 adds the scene E whose degree of excitement exceeds a predetermined threshold value (for example, 1.5 or more) to the selection targets (S31b).

Note that, when adding a new selection target, the highlight moving image creation unit 36 may exclude a scene with a low degree of excitement from the selection targets among the parts with a plurality of selection targets. For example, when adding a middle part scene E, the highlight moving image creation unit 36 excludes a less exciting scene (for example, scene B) from the selection targets in the first half part, which has a plurality of selection targets. In this way, the highlight moving image creating unit 36 may maintain the number of scenes to be selected at a predetermined number so that the specified number of scenes is achieved.

Next, the highlight moving image creating unit 36 generates a highlight moving image by arranging the scenes to be selected in chronological order and connecting them. Next, the output unit 37 performs display output, file output, and the like of the highlight moving image generated by the highlight moving image creation unit 36, and outputs the highlight moving image (S32).

FIG. 11 is an explanatory diagram illustrating an example of providing a highlight video. As shown in FIG. 11, a user (for example, a tournament organizer) inputs a game video (game video G) to the information processing device 1 after the e-Sprots tournament ends (S40). The information processing apparatus 1 uses a prediction model 2 based on deep learning to create a highlight video with scenes corresponding to the excitement of the audience (S41). Next, the information processing apparatus 1 provides the audience with the highlight video by outputting the generated highlight video on a large display installed facing the audience seats, for example (S42). This allows the user to easily provide the spectators with highlight scenes in the game video.

[About effect]
As described above, the information processing apparatus 1 includes the dividing section 32 , the degree of excitement calculating section 35 , the highlight moving image creating section 36 and the output section 37 . The dividing unit 32 divides the moving image of the computer game into scenes. Based on the game image G10 included in each divided scene, the excitement level calculation unit 35 uses a prediction model 2 that predicts the excitement level of the audience who viewed the game image from the game image of the computer game, To calculate the degree of excitement when a scene is viewed by an audience. The highlight moving image creating unit 36 selects a scene to be used for the highlight moving image from among the divided scenes based on the calculated excitement degree, and generates the highlight moving image. The output unit 37 outputs the highlight moving image generated by the highlight moving image generating unit 36 . Therefore, the information processing apparatus 1 can easily present a highlight video of a scene corresponding to the excitement of the audience from each scene of the computer game.

Prediction model 2 is created by machine learning based on teacher data (data set 2a) in which a game image G10 of a computer game is given a degree of excitement based on the volume of the audience who viewed the game image G10 as a correct answer. be. Therefore, the information processing apparatus 1 can calculate the excitement level of the audience for each divided scene by applying the game image G10 included in the scene to the prediction model 2 learned by machine learning.

In addition, the highlight movie creation unit 36 selects at least one scene for each part obtained by dividing the computer game from start to finish by a predetermined number. As a result, the information processing apparatus 1 can create a highlight video including exciting scenes in each of the first half, the middle, and the second half, for example, when the video is divided into three parts such as the first half, the middle, and the second half.

In addition, the highlight moving image creation unit 36 arranges the selected scenes in chronological order to generate a highlight moving image. Therefore, the information processing apparatus 1 can create a highlight moving image that matches the progress of the computer game.

Also, the computer game may be a competitive game in which a plurality of players compete against each other. The information processing apparatus 1 may regard computer games as sports, for example, and create highlight videos from video games such as e-Sprots in which a plurality of players compete.

Further, the dividing unit 32 divides the scene based on switching from the viewpoint of the first player among the plurality of players to the viewpoint of the second player different from the first player. As a result, the information processing apparatus 1 can create a highlight moving image from an exciting scene among scenes divided according to the viewpoint of each player in a battle-type game in which a plurality of players compete.

The information processing apparatus 1 may further include an input unit 31 that receives settings for the number of scenes used in the highlight video and the time of the scenes. The highlight moving image creation unit 36 selects scenes according to the number of scenes received by the input unit 31 and the time of the scenes. In this way, in the information processing apparatus 1, the user may set the number of scenes used in the highlight video and the duration of the scenes.

[others]
In the above-described embodiment, error backpropagation was exemplified as a learning method of the neural network in the prediction model 2, but various known methods other than error backpropagation can be adopted. Further, the neural network has a multi-stage structure composed of, for example, an input layer, an intermediate layer (hidden layer), and an output layer, and each layer has a structure in which a plurality of nodes are connected by edges. Each layer has a function called "activation function", edges have a "weight", the value of each node is derived from the value of the node in the previous layer, the weight value of the connecting edge, the activation function that the layer has Calculated. In addition, about the calculation method, well-known various methods can be employ|adopted. Moreover, as machine learning, various methods such as SVM (support vector machine) may be used in addition to the neural network.

Also, each constituent element of each part illustrated does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution and integration of each part is not limited to the one shown in the figure, and all or part of it can be functionally or physically distributed and integrated in arbitrary units according to various loads and usage conditions. can be configured as

For example, the input unit 31 and the dividing unit 32, the teacher data creating unit 33 and the predictive model creating unit 34, or the excitement level calculating unit 35 and the highlight moving image creating unit 36 may be integrated. Further, the information processing apparatus 1 may not have a configuration including the teacher data creation unit 33 and the prediction model creation unit 34 , and may have a configuration in which the prediction model information 23 created outside is stored in the storage unit 20 . Moreover, the illustrated processes are not limited to the order described above, and may be performed simultaneously or in a different order as long as the contents of the processes are not inconsistent.

Furthermore, the various processing functions performed by each device may be executed in whole or in part on a CPU (or a microcomputer such as an MPU or MCU (Micro Controller Unit)). Also, various processing functions may be executed in whole or in part on a program analyzed and executed by a CPU (or a microcomputer such as an MPU or MCU) or on hardware based on wired logic. It goes without saying that it is good. Also, various processing functions may be executed in cooperation with a plurality of computers by cloud computing.

[About hardware configuration example]
By the way, the various processes described in each of the above embodiments can be realized by executing a prepared program on a computer. Therefore, an example of a computer that executes a learning program having functions similar to those of the above-described embodiments will be described below. FIG. 12 is a block diagram showing an example of a computer that executes a highlight movie creation program.

As shown in FIG. 12, the computer 100 has a CPU 101 that executes various arithmetic processes, an input device 102 that receives data input, and a monitor 103 . The computer 100 also includes a medium reading device 104 for reading a program or the like from a storage medium, an interface device 105 for connecting with various devices, and a communication device 106 for connecting with other information processing devices or the like by wire or wirelessly. have The computer 100 also has a RAM 107 that temporarily stores various information, and a hard disk device 108 . Each device 101 to 108 is also connected to a bus 109 .

The hard disk device 108 includes the input unit 31, the division unit 32, the teacher data creation unit 33, the prediction model creation unit 34, the degree of excitement calculation unit 35, the highlight video creation unit 36, and the output unit 37 shown in FIG. A highlight movie creation program 108A having the same function as the part is stored. The hard disk device 108 also stores various data related to the game moving image information 21, the setting information 22, the prediction model information 23, and the scene-specific information DB 24. FIG. The input device 102 receives input of various information such as operation information from the user of the computer 100, for example. The monitor 103 displays various screens such as a display screen to the user of the computer 100, for example. The interface device 105 is connected with, for example, a printing device. The communication device 106 is connected to a network (not shown) and exchanges various information with other information processing devices.

The CPU 101 reads the highlight moving image creation program 108A stored in the hard disk device 108, develops it in the RAM 107, and executes it to perform various processes. The highlight moving image creating program 108 A also includes the computer 100 as an input unit 31 , a dividing unit 32 , a teacher data creating unit 33 , a prediction model creating unit 34 , a excitement level calculating unit 35 , a highlight moving image creating unit 36 and an output unit 37 . can function as

Note that the highlight moving image creation program 108</b>A described above does not have to be stored in the hard disk device 108 . For example, the computer 100 may read and execute the highlight movie creation program 108A stored in a storage medium readable by the computer 100 . Examples of storage media readable by the computer 100 include portable recording media such as CD-ROMs, DVDs (Digital Versatile Discs), and USB (Universal Serial Bus) memories, semiconductor memories such as flash memories, and hard disk drives. . Alternatively, the highlight moving image creation program 108A may be stored in a device connected to a public line, the Internet, a LAN, or the like, and the computer 100 may read and execute the highlight moving image creation program 108A from these devices.

Further, the following additional remarks are disclosed with respect to the above embodiment.

(Appendix 1) dividing a moving image of a computer game into scenes,
For each divided scene, the audience viewed the scene using a prediction model for predicting the degree of excitement of the audience who viewed the game image from the game image of the computer game based on the game image included in the scene. Calculate the degree of excitement in the case,
generating a highlight video by selecting a scene to be used for a highlight video from among the divided scenes based on the calculated degree of excitement;
outputting the generated highlight video;
A highlight moving image generating program characterized by causing a computer to execute processing.

(Appendix 2) The prediction model is created by machine learning based on teacher data in which the degree of excitement based on the volume of the audience who viewed the game image is given as a correct answer to the game image of the computer game.
A highlight moving image generation program according to Supplementary Note 1, characterized by:

(Appendix 3) The generating process selects at least one scene for each part divided by a predetermined number from the start to the end of the computer game.
A highlight moving image generation program according to appendix 1 or 2, characterized by:

(Appendix 4) The generating process arranges the selected scenes in chronological order to generate the highlight video.
The highlight moving image generation program according to any one of appendices 1 to 3, characterized by:

(Appendix 5) The computer game is a competitive game in which a plurality of players compete against each other.
5. The highlight moving image generation program according to any one of appendices 1 to 4, characterized by:

(Additional remark 6) The dividing process is based on switching from the viewpoint of the first player among the plurality of players to the viewpoint of the second player different from the first player. perform the division of
A highlight video generation program according to appendix 5, characterized by:

(Supplementary Note 7) causing the computer to further execute a process of accepting settings for the number of scenes used in the highlight video and the time of the scene;
The generating process selects a scene according to the number of received scenes and the time of the scene.
7. The highlight moving image generation program according to any one of appendices 1 to 6, characterized by:

(Appendix 8) dividing a moving image of a computer game into scenes,
For each divided scene, the audience viewed the scene using a prediction model for predicting the degree of excitement of the audience who viewed the game image from the game image of the computer game based on the game image included in the scene. Calculate the degree of excitement in the case,
generating a highlight video by selecting a scene to be used for a highlight video from among the divided scenes based on the calculated degree of excitement;
outputting the generated highlight video;
A highlight moving image generating method characterized in that processing is executed by a computer.

(Appendix 9) The prediction model is created by machine learning based on teacher data in which the degree of excitement based on the volume of the audience who viewed the game image is given as a correct answer to the game image of the computer game.
The highlight moving image generation method according to appendix 8, characterized by:

(Appendix 10) The generating process selects at least one scene for each part divided by a predetermined number from the start to the end of the computer game.
10. The highlight moving image generation method according to appendix 8 or 9, characterized by:

(Appendix 11) The generating process arranges the selected scenes in chronological order to generate the highlight video.
11. The highlight moving image generation method according to any one of appendices 8 to 10, characterized by:

(Appendix 12) The computer game is a competitive game in which a plurality of players compete against each other.
12. The highlight moving image generation method according to any one of appendices 8 to 11, characterized by:

(Supplementary Note 13) The dividing process is based on switching from a viewpoint of a first player among the plurality of players to a viewpoint of a second player different from the first player. perform the division of
13. The highlight video generation method according to appendix 12, characterized by:

(Supplementary Note 14) causing the computer to further execute processing for receiving settings for the number of scenes used in the highlight video and the time of the scenes;
The generating process selects a scene according to the number of received scenes and the time of the scene.
14. The highlight moving image generation method according to any one of appendices 8 to 13, characterized by:

(Appendix 15) a dividing unit that divides a moving image of a computer game into scenes;
For each divided scene, the audience viewed the scene using a prediction model for predicting the degree of excitement of the audience who viewed the game image from the game image of the computer game based on the game image included in the scene. a prediction unit that calculates the degree of excitement in the case;
a generation unit that selects a scene to be used for a highlight video from among the divided scenes based on the calculated excitement level and generates a highlight video;
an output unit that outputs the generated highlight video;
An information processing device comprising:

(Appendix 16) The prediction model is created by machine learning based on teacher data in which the degree of excitement based on the volume of the audience who viewed the game image is given as a correct answer to the game image of the computer game.
16. The information processing apparatus according to appendix 15, characterized by:

(Appendix 17) The generation unit selects at least one scene for each part obtained by dividing the computer game from start to finish by a predetermined number.
17. The information processing apparatus according to appendix 15 or 16, characterized by:

(Appendix 18) The generation unit arranges the selected scenes in chronological order to generate the highlight video.
18. The information processing apparatus according to any one of appendices 15 to 17, characterized by:

(Appendix 19) The computer game is a competitive game in which a plurality of players compete against each other.
19. The information processing apparatus according to any one of appendices 15 to 18, characterized by:

(Supplementary Note 20) The dividing unit divides the scene based on the switching from the viewpoint of the first player among the plurality of players to the viewpoint of the second player different from the first player. make a split,
19. The information processing apparatus according to appendix 19, characterized by:

(Appendix 21) further comprising an input unit for receiving settings of the number of scenes used in the highlight video and the time of the scenes;
The generation unit selects a scene according to the number of received scenes and the time of the scene.
21. The information processing apparatus according to any one of appendices 15 to 20, characterized by:

1... Information processing device 2... Prediction model 2a... Data set 10... Communication unit 20... Storage unit 21... Game video information 22... Setting information 23... Prediction model information 24... Scene-specific information DB
REFERENCE SIGNS LIST 30: control unit 31: input unit 32: division unit 33: teacher data creation unit 34: predictive model creation unit 35: excitement level calculation unit 36: highlight video creation unit 37: output unit 100: computer 101: CPU
102... Input device 103... Monitor 104... Medium reading device 105... Interface device 106... Communication device 107... RAM
108 Hard disk device 108A Highlight video creation program 109 Bus G Game video G1 Game screen G2 Information screens G10, G10a to G10d Game images P1, P2 Players t1, t2 Time

Claims (8)

A video image of a computer game is divided into scenes,
For each divided scene, the audience viewed the scene using a prediction model for predicting the degree of excitement of the audience who viewed the game image from the game image of the computer game based on the game image included in the scene. Calculate the degree of excitement in the case,
Based on the calculated degree of excitement, a scene to be used for a highlight video is selected from among the divided scenes, and for overlapping scenes among the scenes to be selected, a scene with a lower degree of excitement is selected. generate a highlight video by excluding from
outputting the generated highlight video;
let the computer do the work,
The prediction model is created by machine learning based on teacher data in which a game image of the computer game is given a degree of excitement based on the volume of the audience who viewed the game image as a correct answer,
A highlight video generation program characterized by: If the selected scene is not included in each part divided by a predetermined number from the start to the end of the computer game, the generating process is performed such that if the selected scene is not included in the part, the degree of excitement is equal to or greater than a predetermined value among the scenes in the part. generating the highlight video by adding the scene to the selection target ;
The highlight moving image generation program according to claim 1, characterized by: The generating process arranges the selected scenes in chronological order to generate the highlight video.
3. The highlight moving image generating program according to claim 1 or 2, characterized by: The computer game is a competitive game in which a plurality of players compete against each other.
4. The highlight moving image generation program according to any one of claims 1 to 3, characterized by: In the dividing process, the scene is divided based on switching from a viewpoint of a first player among the plurality of players to a viewpoint of a second player different from the first player. ,
5. The highlight moving image generating program according to claim 4, characterized by: causing the computer to further execute processing for receiving settings for the number of scenes used in the highlight video and the time of the scenes;
The generating process selects a scene according to the number of received scenes and the time of the scene.
6. The highlight moving image generation program according to any one of claims 1 to 5, characterized by: A video image of a computer game is divided into scenes,
For each divided scene, the audience viewed the scene using a prediction model for predicting the degree of excitement of the audience who viewed the game image from the game image of the computer game based on the game image included in the scene. Calculate the degree of excitement in the case,
Based on the calculated degree of excitement, a scene to be used for a highlight video is selected from among the divided scenes, and for overlapping scenes among the scenes to be selected, a scene with a lower degree of excitement is selected. generate a highlight video by excluding from
outputting the generated highlight video;
A computer performs the processing,
The prediction model is created by machine learning based on teacher data in which a game image of the computer game is given a degree of excitement based on the volume of the audience who viewed the game image as a correct answer,
A highlight video generation method characterized by: a dividing unit that divides a moving image of a computer game into scenes;
For each divided scene, the audience viewed the scene using a prediction model for predicting the degree of excitement of the audience who viewed the game image from the game image of the computer game based on the game image included in the scene. a prediction unit that calculates the degree of excitement in the case;
Based on the calculated degree of excitement, a scene to be used for a highlight video is selected from among the divided scenes, and for overlapping scenes among the scenes to be selected, a scene with a lower degree of excitement is selected. a generation unit that generates a highlight video by excluding from
an output unit that outputs the generated highlight video;
has
The prediction model is created by machine learning based on teacher data in which the game image of the computer game is given a degree of excitement based on the volume of the audience who viewed the game image as a correct answer.
An information processing device characterized by:

Images