JP2022123345A - Program, method, information processing device and system - Google Patents

Abstract

To insert an advertisement into a dynamic object area in which a shape and size are changed in an image.SOLUTION: There is provided a program to be executed by a computer including a processor and a memory. The program causes the processor to execute steps of: analyzing an image for every frame and acquiring information relating to a dynamic object contained in the image as a subject; generating an advertisement image based on a shape, size and direction of an area in the dynamic object identified from the acquired information; and displaying the generated advertisement image in association with the area.SELECTED DRAWING: Figure 4

Description

The present disclosure relates to programs, methods, information processing apparatuses, and systems.

In recent years, advertisements are displayed in all kinds of media. For example, in the field of sports videos, there is a known method of posting advertising information on the background of the video, such as the wall of the stadium or the surrounding billboards.

By the way, in sports such as soccer and baseball, for example, advertisements are physically attached to uniforms worn by players. Advertisements attached to uniforms and the like are fixed and cannot be easily switched like images. As a result, sponsors who provide ads on uniforms cannot change their ads lightly. Also, even if there are other companies that want to display advertisements, they cannot display advertisements if the uniform space is already filled.

The technology disclosed in Japanese Patent Laid-Open No. 2002-200012 combines information related to a subject with a background area near the subject based on positional information of the subject in a free-viewpoint video area generated based on a three-dimensional model of the subject. , a technique for generating a composite video is described.

WO2019/021375

Japanese Patent Application Laid-Open No. 2002-200002 describes synthesizing information related to a subject with a background area near the subject, but does not effectively utilize the area on the subject.

It is therefore an object of the present disclosure to effectively insert advertisements into dynamically moving regions within a video.

According to one embodiment, a program to be executed by a computer comprising a processor and a memory, the program instructs the processor to analyze an image frame by frame and obtain information about a dynamic object included in the image as an object. generating an advertisement image based on the shape, size, and orientation of the area in the dynamic object ascertained from the obtained information; and displaying the generated advertisement image in association with the area. A program is provided for performing the steps of:

According to the present disclosure, advertisements can be effectively inserted into dynamically moving regions within a video.

1 is a diagram showing the overall configuration of a system 1; FIG. 3 is a diagram showing a functional configuration of a server 20; FIG. It is a figure which shows the data structure of the advertisement information database 2021 which the server 20 memorize|stores. 1 is a diagram showing an overview of devices and the like that constitute system 1. FIG. 4 is a flow chart showing a series of processes in which the server 20 generates an advertisement image and synthesizes the generated advertisement image with a video. 4 is a diagram showing a display example of video managed by the server 20. FIG. 4 is a diagram showing a display example of video distributed from the server 20 and displayed on the terminal device 10. FIG.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are given the same reference numerals. Their names and functions are also the same. Therefore, a detailed description thereof will not be repeated.

<Overview>
In the following embodiments, a system 1 for effectively inserting an advertisement into a dynamic object area whose shape, size, and inclination change in a video imaged using an imaging device or the like will be described. Specifically, for example, the system 1 displays dynamic objects such as players playing the game, tools used in the game, etc. for a viewer watching a video of a game such as soccer. An advertisement image is superimposed and displayed on a predetermined area in a manner corresponding to the area. Note that in the present embodiment, the dynamic object represents a moving object on which an advertisement image is superimposed. Also, the advertisement image represents an image of an advertisement.

A system 1 acquires an image including a dynamic object from an imaging device. The system 1 performs image analysis on the acquired video for each frame and acquires information on the dynamic object included in the image. The system 1 determines the aspect of the advertisement image to be displayed based on the acquired information. The system 1 superimposes an advertisement image in a determined manner on a predetermined area of a dynamic object, such as a person's chest, abdomen, back, shoulder, roof, side, front wing, rear wing, etc. of a moving object. do. System 1 determines advertisements based on video and information about the viewer. As a result, the system 1 can display the advertisement image desired by the viewer on the dynamic object included in the video. Therefore, it is possible to display an arbitrary advertisement in an area in which a fixed advertisement was conventionally displayed, so that the area on the dynamic object can be effectively used.

<1 Configuration diagram of the entire system>
FIG. 1 is a diagram showing the overall configuration of a system 1. As shown in FIG.

As shown in FIG. 1, the system 1 includes a terminal device 10, a server 20, an edge server 30, and a photographing device . The terminal device 10 , the server 20 and the edge server 30 are connected for communication via the network 80 . The edge server 30 is connected with the imaging device 40 . For example, the imaging device 40 is a transmission/reception device based on a communication standard used in a short-range communication system between information devices. Specifically, the photographing device 40 receives a beacon signal from another information device equipped with a Bluetooth (registered trademark) module, for example, using a 2.4 GHz band such as a Bluetooth (registered trademark) module. The edge server 30 acquires information transmitted from the imaging device 40 based on the beacon signal using the near field communication. In this way, the photographing device 40 transmits the acquired video to the edge server 30 by short-range communication without going through the network 80 . Note that the edge server 30 may be communicatively connected to the photographing device 40 via the network 80 .

In the system 1, the edge server 30 and the imaging device 40 do not necessarily have to be provided. The edge server 30 and the imaging device 40 do not have to be included in the system 1 when pre-captured video is stored in the server 20 .

The terminal device 10 is, for example, a device operated by a user viewing video distributed from the server 20 . The terminal device 10 may be, for example, a mobile terminal such as a smart phone or a tablet, a stationary PC (Personal Computer), or a laptop PC.

The server 20 manages video distribution. The server 20, for example, superimposes an advertisement image according to the viewer on the dynamic object in the video. The server 20 manages the display mode of the advertisement image, such as the size and orientation, for displaying the advertisement image superimposed on the dynamic object. In addition, the server 20 manages various types of information about viewers, such as age, address, preferences, and lifestyle habits. The server 20 shown in FIG. 1 has a communication IF 22 , an input/output IF 23 , a memory 25 , a storage 26 and a processor 29 .

The communication IF 22 is an interface for inputting and outputting signals for the server 20 to communicate with an external device. The input/output IF 23 functions as an interface with an input device for receiving input operations from the user and an output device for presenting information to the user. The memory 25 temporarily stores programs and data processed by the programs, and is a volatile memory such as a DRAM (Dynamic Random Access Memory). The storage 26 is a storage device for storing data, such as a flash memory or HDD (Hard Disc Drive). The processor 29 is hardware for executing an instruction set described in a program, and is composed of arithmetic units, registers, peripheral circuits, and the like.

In the present embodiment, an aggregate of devices consisting of a plurality of servers 20 can also be grasped as one information processing device. That is, the system 1 may be formed as an aggregation of a plurality of servers 20. FIG. For example, a different server 20 may take charge of each function exhibited by the server 20, and the data obtained by each server 20 may be aggregated and analyzed by another server 20, or the like.

The edge server 30 receives information transmitted from the imaging device 40 and transmits the received information to the server 20 . Also, the edge server 30 transmits information acquired from the server 20 to the imaging device 40 . The information acquired from the server 20 includes, for example, information for updating the settings of the imaging device 40 and the like.

The imaging device 40 is a device for receiving light with a light receiving element and outputting it to the edge server 30 as image data. The imaging device 40 is assumed to be, for example, one of the following devices.
・Visible light camera ・Infrared camera ・Ultraviolet camera ・Ultrasonic sensor ・RGB-D camera ・LiDAR (Light Detection and Ranging)

<1.1 Configuration of Server 20>
FIG. 2 is a diagram showing a functional configuration of the server 20. As shown in FIG. As shown in FIG. 2 , the server 20 functions as a communication section 201 , a storage section 202 and a control section 203 .

The communication unit 201 performs processing for the server 20 to communicate with an external device such as the edge server 30 .

The storage unit 202 stores data and programs used by the server 20 . The storage unit 202 stores an advertisement information database 2021, a viewer information database 2022, a video database 2023, and the like.

The advertisement information database 2021 stores various information related to advertisements to be superimposed and displayed on dynamic objects by the system 1 . Details will be described later.

The viewer information database 2022 stores various information about viewers of the video. Details will be described later.

The image database 2023 stores images captured by the image capturing device 40 or images captured in advance by another image capturing device. In one aspect, the video database 2023 may store information about videos to be live distributed, for example, information about distribution content, distribution time, and the like.

The control unit 203 exhibits functions shown as various modules when the processor of the server 20 performs processing according to a program.

The reception control module 2031 controls processing for the server 20 to receive a signal from an external device according to a communication protocol.

The transmission control module 2032 controls the processing by which the server 20 transmits signals to external devices according to a communication protocol.

Advertisement determination module 2033 determines an advertisement to be displayed based on the information of the viewer who is viewing the video. Specifically, for example, the advertisement determination module 2033 extracts information about the viewer from the viewer information database 2022 based on the log-in information of the viewer viewing the video received by the reception control module 2031 . The advertisement determination module 2033 determines, from the advertisement information database 2021, an advertisement to be displayed in the video that the viewer is viewing, based on the extracted information. The determined advertisement may be plural or singular. When the advertisement display target is set in the advertisement information database 2021, the advertisement determination module 2033 may refer to the set target to determine the advertisement.

Advertisement determination module 2033 may determine an advertisement to be displayed in the video based on the extracted information and information associated with the video. For example, the information associated with the video is as follows.
・Information about sponsor companies related to the video ・Information related to the content of the video ・Information about the products that appear in the video

Algorithms for determining advertisements may use existing algorithms. Also, the advertisement determination module 2033 may determine an advertisement at predetermined timing (a predetermined cycle, a predetermined time, when a predetermined event occurs), for example. At this time, the advertisement determination module 2033 may change the sponsor of the advertisement before and after switching. Also, the advertisement determination module 2033 may change the attributes of advertisements, products, etc. before and after switching.

The image analysis module 2034 obtains information for creating advertising images in the image generation module 2035 based on the dynamic objects included in the video.

Specifically, for example, the image analysis module 2034 obtains information about dynamic objects included in the video according to the manner in which the advertising image is generated by the image generation module 2035 . Information about dynamic objects is, for example:
・Region of the dynamic object included in the image ・Predetermined region in the dynamic object included in the image (chest, back, shoulder, front wing, body, rear wing, etc.)
・Movement direction of the dynamic object included in the image ・Shape of the dynamic object included in the image

More specifically, the image analysis module 2034 acquires, for example, a region of a dynamic object included in the video and a predetermined region in the dynamic object using a trained model. A trained model is created by causing a machine learning model to perform machine learning according to a model learning program based on learning data. The learning data, for example, receives a plurality of images including dynamic objects as input, and includes predetermined tags, identifiers, etc. assigned to the input images as correct output. Predetermined tags are, for example, dynamic objects such as players, balls, cars, etc.; Including areas such as the rear wing. The image analysis module 2034 inputs, for example, a video to be delivered to the viewer to the trained model and causes it to output regions about the dynamic object and predetermined regions in the dynamic object.

Also, the image analysis module 2034 acquires, for example, the moving direction of the dynamic object included in the video by taking the difference between the frames forming the video. At this time, the frames from which the difference is to be calculated may be continuous or separated by a predetermined number of frames.

The image analysis module 2034 also obtains, for example, the shape of a dynamic object included in the video based on pixel information in the frame.

It should be noted that any existing method may be used as a method of acquiring information about the dynamic object. For example, the image analysis module 2034 may obtain the shape of a dynamic object included in the video using a trained model. At this time, the correct answer output includes, for example, the shape of the dynamic object, such as a square, a trapezoid, and the like. Also, the correct answer output may be a number designating a preset shape such as shape 1, shape 2, and the like.

The image generation module 2035 generates advertisement images based on the information obtained by the image analysis module 2034 and the advertisements determined by the advertisement determination module 2033 .

Specifically, the image generation module 2035, for example, obtains from the image analysis module 2034, the area of the dynamic object included in the video, the predetermined area in the dynamic object, the movement direction of the dynamic object 4. Based on the shape of the dynamic object, obtain the size, shape, surface inclination of the dynamic object, etc. in which the advertising image is synthesized in the dynamic object. The image generation module 2035 determines the size and shape of the image, the inclination of the image in the depth direction, etc., based on the size and shape of the area where the advertisement image is synthesized, the inclination of the surface of the dynamic object, and the like. The image generation module 2035 generates an advertisement image taking into account the size and shape of the image, the inclination of the image in the depth direction, and the like. The image generation module 2035 generates an advertisement image by transforming the original image of the advertisement using, for example, affine transformation.

In addition, the image generation module 2035 may generate an advertisement image in a form corresponding to the shape of the dynamic object acquired by the image analysis module 2034, for example. Specifically, for example, conversion rules for the original image of the advertisement are set in advance for each shape of the dynamic object. Image generation module 2035 applies a transformation equation set based on the shape of the dynamic object to the original image for the advertisement. The image generation module 2035 generates the advertisement image by determining the scale of the image to which the transform is applied based on the size of the dynamic object.

Also, the image generation module 2035 may, for example, create a 3D model from the image of the dynamic object acquired by the image analysis module 2034, and generate an advertisement image using the created 3D model. Specifically, for example, the image generation module 2035 acquires two-dimensional images of the dynamic object from multiple directions. Image generation module 2035 creates a three-dimensional model of the dynamic object based on two-dimensional images acquired from multiple directions. The image generation module 2035 pastes the original image of the advertisement determined by the advertisement determination module 2033 onto the surface of the three-dimensional model. The image generation module 2035 generates an advertisement image by executing rendering processing on the original image attached to the 3D model.

Also, the advertisements to be superimposed may be different for each type of area in the dynamic object. For example, if the dynamic object is a person, different advertisements may be overlaid on the chest, back, and shoulders. Also, for example, when the dynamic object is an automobile, different advertisements may be superimposed on the front wing, body, and rear wing. The image generation module 2035 generates an advertisement image suitable for the area by transforming the original image of the advertisement corresponding to the detected area according to the aspect of the area.

The synthesizing module 2036 synthesizes the advertisement image generated by the image generating module 2035 with the video. Specifically, for example, the synthesizing module 2036 superimposes the advertisement image generated by the image generating module 2035 on the corresponding area in the video.

The presentation module 2037 presents the video combined with the advertising content to the viewer.

As a result, the server 20 can accurately superimpose and present the advertising content to the viewer according to the dynamic objects of various modes included in the video.

<2 Data structure>
FIG. 3 is a diagram showing the data structures of the advertisement information database 2021 and the viewer information database 2022 stored by the server 20. As shown in FIG.

As shown in FIG. 3, the advertisement information database 2021 includes an item "advertisement ID", an item "advertiser", an item "product", an item "attribute", an item "target", and an item "advertisement size". etc.

The item “advertisement ID” is information for identifying each advertisement.

The item "advertiser" is information for identifying an advertiser who provides each advertisement. For example, it indicates that the advertiser with the advertisement ID "A001" is "A company".

The item "merchandise" is information indicating the details of the advertisement. For example, it indicates that the product with the advertisement ID “A001” is “tea”.

The item "target" indicates the attributes of viewers to whom the advertisement is to be displayed. For example, it indicates that the target of the advertisement ID "A001" is "general".

The item "attribute" indicates the attribute given to the advertisement. Specifically, it shows information about what classification each advertisement content belongs to. For example, the attribute of the advertisement ID "A001" indicates "luxury goods". In one aspect, for example, the advertisement determination module 2033 randomly determines one advertisement among the advertisements whose attribute is "luxury goods."

The item "advertisement size" is information indicating the size of the advertisement. Here, the size represents, for example, the size displayed in a predetermined area of the dynamic object. Multiple sizes may be stored for the same advertisement. For example, it indicates that the advertisement size for the product "tea" with the advertisement ID "A001" is "** cm x ** cm", "++ cm x ++ cm", and "〇〇 cm x 〇〇 cm". Here, the advertisement size may have a resolution of dpi (dots per inch). In one aspect, the advertisement determination module 2033 may determine the size of the advertisement based on the shape of the dynamic object included in the video, the area within the dynamic object, etc. analyzed by the image analysis module 2034. good.

As shown in FIG. 3, the viewer information database 2022 includes an item “viewer ID”, an item “registered name”, an item “age”, an item “gender”, an item “hobbies”, an item “browsing history” and items such as “purchase history”.

The item “viewer ID” is information for identifying each viewer who views the video.

The item "registered name" is information for identifying the name of the viewer. For example, the registered name of the viewer ID "U001" is "A". The registered name may be the name of the viewer, or may be a handle name in a service for viewing video.

The item “age” is information indicating the age of the viewer.

The item "gender" is information indicating the gender of the viewer.

The item "hobby" is information about the viewer's hobby. For example, it indicates that the hobbies of the viewer ID "U001" are "sports", "movies", "games", and the like. The server 20 may store the information by receiving an input operation from a user or a viewer, or may estimate the listener's hobby information from information such as browsing history and purchase history.

The item “browsing history” indicates information related to the history of viewers browsing websites, videos of video distribution services, and the like. Specifically, the server 20 stores the viewing history by input operation from the viewer, or by receiving the information from a website, video distribution service, etc. affiliated with the service provided by the system 1 to the viewer. Get information. For example, the viewing history of the viewer ID "U001" indicates "A broadcast", "https://**", "B broadcast", and the like.

The item “purchase history” indicates information about the viewer's purchase history on the EC site. Specifically, the server 20 acquires purchase history information by an input operation from the viewer or by receiving the information from an EC site or the like affiliated with the service provided by the system 1 to the viewer. . For example, the purchase history of the viewer ID "U001" indicates "2020/10/01 Dumbbell", "2020/10/3 Protein", "2020/10/5 Blu-ray", and the like.

As a result, the server 20, based on various information of the viewer, for example, information such as age, gender, hobbies, websites and video distribution services viewed by the viewer, history of products purchased by the viewer, etc. It becomes possible to determine the optimum advertisement for each. This allows the server 20 to further enhance the user's viewing experience.

<3 Summary>
FIG. 4 is a diagram showing an overview of the system 1. As shown in FIG. In the example shown in FIG. 4, for example, display of an advertisement in a video of a game such as soccer captured by the imaging device 40 will be described.

The image capturing device 40 captures an image including a dynamic object. The imaging device 40 transmits the image including the captured dynamic object to the edge server 30 .

The edge server 30 transmits the video including the received dynamic object to the server 20 .

The server 20 analyzes the video obtained from the edge server 30 frame by frame to obtain information about the dynamic objects in the video. Server 20 generates an advertisement image based on the acquired information. The server 20 synthesizes the generated advertisement image with the video and displays it to the viewer.

Thereby, the server 20 generates an advertisement image based on the size, shape, orientation, etc. of the dynamic object included in each frame from the video including the dynamic object, and moves the generated advertisement image. superimposed on the target object. Therefore, the server 20 can naturally superimpose the advertisement on the dynamic object without making the user feel uncomfortable.

A viewer who watches a video, for example, in a video about a game such as soccer, sees the video because the advertisement is displayed in a natural way on the player, which is a dynamic object. You can watch the video without feeling uncomfortable, and there is no fear that the feeling of immersion will be reduced. A natural form means that when a dynamic object such as a person changes direction, the advertising content does not protrude from the person, and the advertising content is always displayed according to the movement of the person. indicates that

This allows the viewer to view the advertisement combined with the dynamic object with the same feeling as the advertisement printed on the dynamic object.

Also, even if the advertisement is printed on the actual dynamic object, the server 20 can composite the advertisement image onto the dynamic object. This allows an advertiser company to display an advertisement that is different from the advertisement physically printed on the dynamic object when distributed through the network, even if the advertisement is physically printed. be able to.

<4 Operation>
A series of processes in which the server 20 generates an advertisement image and synthesizes the generated advertisement image with a video will be described below. In the following description, for example, as shown in FIG. 4, the imaging device 40 will be used to capture a game, such as soccer, in which a plurality of dynamic objects are present.

The photographing device 40 photographs an image in a preset photographing direction. The imaging direction of the imaging device 40 may include multiple dynamic objects, such as a player wearing a uniform, a soccer ball, and the like. The imaging device 40 transmits a video signal of the captured video to the server 20 via the edge server 30 .

FIG. 5 is a flowchart showing a series of processes in which the controller 203 of the server 20 generates an advertisement image and synthesizes the generated advertisement image with a video.

First, the viewer operates the terminal device 10 to select a desired video. The video to be selected may be a video stored in the video database 2023 or a video distributed in real time.

In step S501, upon receiving a video selection from the terminal device 10, the control unit 203 of the server 20 determines an advertisement to be displayed based on the selected video and viewer information. Specifically, for example, the control unit 203 uses the advertisement determination module 2033 to identify the viewer based on the viewer's log-in information, and extracts information about the identified viewer from the viewer information database 2022 . Advertisement determination module 2033 determines an appropriate advertisement to display to the viewer, eg, based on information associated with the video selected by the viewer and the extracted viewer information.

In step S502, the control unit 203 acquires the video selected by the viewer. Specifically, the control unit 203 reads the video selected by the viewer from the video database 2023 . Alternatively, the control unit 203 acquires the video selected by the viewer from a predetermined video distribution source.

In step S503, the control unit 203 causes the image analysis module 2034 to perform image analysis, for example, for each video frame. The image analysis module 2034 obtains information about dynamic objects included in the video through image analysis.

In step S<b>504 , the control unit 203 causes the image generation module 2035 to generate an advertisement image based on the information acquired by the image analysis module 2034 and the advertisement determined by the advertisement determination module 2033 . Specifically, the control unit 203, for example, based on the information acquired by the image analysis module 2034, the mode of displaying the advertisement for each frame, for example, the size and shape of the advertisement, the inclination of the image in the depth direction, etc. decide. The control unit 203 converts the advertisement determined by the advertisement determination module 2033 based on the determined size, shape, inclination, and the like to generate an advertisement image.

In step S505, the control unit 203 causes the synthesizing module 2036 to superimpose the advertisement image generated by the image generating module 2035 on the predetermined area in the dynamic object recognized by the image analysis. The control unit 203 executes the processing of steps S503 to S505, for example, for each frame.

As a result, the server 20 can accurately superimpose the advertisement image corresponding to the viewer on the dynamic object in consideration of the motion of the dynamic object. Therefore, the server 20 can present the advertisement to the viewer in a natural manner without making the viewer feel uncomfortable.

In addition, in FIG. 5, the case where the control unit 203 determines an advertisement at the start of video viewing has been described as an example. However, ad decisions are not limited to when a video starts to be viewed. For example, the control unit 203 may determine an advertisement at predetermined timing (predetermined period, predetermined time, occurrence of a predetermined event, etc.). At this time, the control unit 203 may change the advertisement sponsor before and after switching.

<5 Screen example>
Next, screen examples of images displayed on the terminal device 10 will be described. In the following description, a scene in which the viewer is watching soccer will be described as an example.

FIG. 6 is a diagram showing a display example of video managed by the server 20. As shown in FIG. FIG. 7 is a diagram showing a display example of video distributed from the server 20 and displayed on the terminal device 10. As shown in FIG.

The image shown in FIG. 6 represents an image of a predetermined frame in video. The image shown in FIG. 6 includes dynamic objects 602A, 602B, 602C, 602D, 603A, 603B. Dynamic objects 602A, 602B, 602C, 603A, 603B represent players and dynamic object 602D represents a soccer ball. In the image shown in FIG. 6, predetermined regions of the dynamic objects 602A, 602B, 602C, 602D, 603A, and 603B, such as chest regions 604A, 604B, and 604C, back region 605A, shoulder regions 605B, and ball center region 604D, include: Advertising images are not composited.

The control unit 203 acquires information on the dynamic objects 602A, 602B, 602C, 602D, 603A, and 603B included in the video using the image analysis module 2034, for example. The control unit 203 uses the image generation module 2035 to generate an advertisement image based on the information acquired by the image analysis module 2034 and the advertisement determined by the advertisement determination module 2033 . Specifically, the control unit 203 converts the advertisement based on, for example, the size, shape, inclination in the depth direction, etc. of the chests 604A, 604B, 604C, the back 605A, the shoulders 605B, and the center of the ball 604D. Advertising images 606A, 606B, 606C, 606D, 607A, 607B are generated.

The control unit 203 causes the synthesizing module 2036 to combine the advertisement images 606A, 606B, 606C, 606D, 607A, and 607B generated by the image generating module 2035 into a predetermined region (chest region 604A) in the dynamic object recognized by image analysis. , 604B, 604C, ball center portion 604D, back portion 605A, shoulder portion 605B).

The image shown in FIG. 7 is an image when the image shown in FIG. 6 is displayed to a predetermined viewer. In the image shown in FIG. 7, advertisement images 606A, 606B, 606C, 606D, 607A, 607B are superimposed on each other.

As a result, the server 20 can naturally superimpose an advertisement that suits each viewer's taste on the video being viewed on the dynamic object without giving the user a sense of discomfort. Therefore, the advertiser who provides the advertisement can easily switch the advertisement to be displayed on the dynamic object according to the viewer. Also, even when there are multiple advertisers, it is possible to display advertisements on dynamic objects. In addition to the advertisement actually printed on the dynamic object, it is possible to combine the advertisement with the image to be distributed, so that it is possible to recruit more advertisers than before.

In a certain aspect, the control unit 203 may superimpose an advertisement image of a different type, for example, a different sponsor for each team in a competitive competition or the like held by teams. For example, the control unit 203 may change the advertiser of the advertisement between the advertisement images 606A, 606B, 606C and the advertisement images 607A, 607B. As a result, the server 20 can display advertisements in such a way as to clearly indicate the sponsors of each team, display advertisements in more effective areas, and so on, so as to improve the effectiveness of advertisements for viewers.

<6 Modifications>
In the above embodiment, a case has been described in which the server 20 analyzes an image to acquire information about a dynamic object and determine an area for displaying an advertisement. However, these processes may be performed by the imaging device 40 or the edge server 30 . That is, for example, the imaging device 40 acquires information about the dynamic object, determines the display area and mode of the advertisement, and transmits the information to the server 20 . The server 20 generates an advertisement image based on the information received from the imaging device 40 and synthesizes the advertisement so as to be superimposed on the dynamic object. Also, for example, the edge server 30 acquires information about the dynamic object, determines the display area and mode of the advertisement, and transmits the information to the server 20 . The server 20 generates an advertisement image based on the information received from the edge server 30 and synthesizes the advertisement to be superimposed on the dynamic object.

In the above embodiment, a case has been described in which a three-dimensional model is created based on images acquired from a plurality of photographing devices 40, and an advertisement image is displayed as a texture on the surface of the three-dimensional model. However, sensors (not shown) may be attached to the dynamic object. The sensor is a motion sensor that is placed on some region of the body of the dynamic object and detects information about the motion of the body of the placed dynamic object. For example, the sensors detect information such as body tilt, motion speed, motion direction, etc. of the dynamic object. The sensor is implemented by, for example, a gyro sensor, an acceleration sensor, or the like. The sensor transmits various sensing information detected to the edge server 30 . The edge server 30 transmits sensing information to the server 20 . The server 20 may create a three-dimensional model based on the received sensing information using the image generation module 2035 .

In the above embodiment, the information about the dynamic object acquired by the image analysis module 2034 includes:
・Region of the dynamic object included in the image ・Predetermined region in the dynamic object included in the image (chest, back, shoulder, front wing, body, rear wing, etc.)
・Movement direction of the dynamic object included in the image ・The shape of the dynamic object included in the image is illustrated. However, the information acquired by the image analysis module 2034 is not limited to these. The information obtained by the image analysis module 2034 may be:
・A region containing a predetermined color (for example, uniform color, etc.)
・A region containing predetermined characters or marks (for example, a logo mark, etc.)
・An area that has a predetermined positional relationship with a characteristic part (for example, the chest is detected by relative coordinates from the face area)

These embodiments can be implemented in various other forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. These embodiments and their modifications are intended to be included in the invention described in the claims and their equivalents as well as included in the scope and gist of the invention.

<Appendix>
The items described in the above embodiments will be added below.

(Appendix 1)
A program to be executed by a computer 20 comprising a processor 29 and a memory 25, the program causing the processor 29 to analyze an image for each frame and acquire information about a dynamic object included in the image as an object. generating an advertisement image based on the shape, size, and orientation of the area in the dynamic object grasped from the acquired information (S503); A program for executing a step (S505) of displaying in association with .

(Appendix 2)
2. The program according to appendix 1, wherein in the step of generating the advertisement image (S504), the advertisement image is generated by converting the original image of the advertisement in correspondence with the shape, size and orientation of the area in the dynamic object. (Paragraph 0042 to paragraph 0045)

(Appendix 3)
2. The program according to supplementary note 1, wherein in the step of generating the advertisement image (S504), the advertisement image is generated by converting the original image of the advertisement based on a rule preset according to the shape of the dynamic object. (Paragraph 0042 to paragraph 0045)

(Appendix 4)
The program according to appendix 1, wherein in the step of generating the advertisement image (S504), the advertisement image is generated by creating a three-dimensional model of the dynamic object and rendering the original image of the advertisement pasted on the three-dimensional model. (Paragraph 0042 to paragraph 0045)

(Appendix 5)
5. The method according to any one of appendices 1 to 4, wherein in the step of generating the advertisement image (S504), a plurality of types of regions are set on the dynamic object, and an advertisement to be displayed is set for each type of region. program. (Paragraph 0042 to paragraph 0045)

(Appendix 6)
Referring to viewer information, causing the processor 29 to execute a step of determining an advertisement suitable for the viewer (S501), and generating an advertisement image using the determined advertisement in a step of generating an advertisement image (S504). A program according to any one of claims 1 to 5. (Paragraph 0032)

(Appendix 7)
7. The program according to appendix 6, wherein in the step of determining the advertisement (S501), the advertisement is determined based on information associated with the video. (Paragraph 0032 to Paragraph 0034)

(Appendix 8)
8. The program according to appendix 6 or 7, causing the processor 29 to execute the step of determining the advertisement (S501) at a predetermined timing. (Paragraph 0034)

(Appendix 9)
The step of determining the advertisement (S501) is executed by the processor 29 at a predetermined timing, and in the step of determining the advertisement (S501), the advertisement is determined so that the advertiser is different based on the information associated with the video. 8. The program according to 8. (Paragraph 0034)

(Appendix 10)
A method for execution by a computer 20 comprising a processor 29 and a memory 25, wherein the processor 29 analyzes a video frame by frame to obtain information about a dynamic object included as a subject in the video. generating an advertisement image based on the shape, size, and orientation of the area in the dynamic object grasped from the acquired information (S503); displaying in association with (S505).

(Appendix 11)
An information processing apparatus 20 comprising a control unit 203, in which the control unit 203 analyzes an image for each frame and acquires information on a dynamic object included as an object in the image (S503); A step of generating an advertisement image based on the shape, size, and orientation of the area in the dynamic object grasped from the information (S504), and a step of displaying the generated advertisement image in association with the area (S505). and the information processing device 20 that executes.

(Appendix 12)
A means (S503) for analyzing an image for each frame and acquiring information about a dynamic object included as a subject in the image; A system comprising: means for generating an advertisement image based on orientation (S504); and means for displaying the generated advertisement image in association with an area (S505).

20 server, 22 communication IF, 23 input/output IF, 25 memory, 26 storage, 29 processor, 30 edge server, 40 imaging device, 80 network, 201 communication unit, 202 control unit, 203 communication unit, 2021 advertisement information database, 2022 Viewer information database, 2023 Video database.

Claims (12)

A program to be executed by a computer comprising a processor and a memory, the program comprising:
analyzing a video frame by frame to obtain information about a dynamic object included as a subject in the video;
generating an advertisement image based on the shape, size, and orientation of a region in the dynamic object ascertained from the acquired information;
and displaying the generated advertisement image in association with the area. 2. The program according to claim 1, wherein in the step of generating the advertisement image, the advertisement image is generated by converting an original image of the advertisement in correspondence with the shape, size and orientation of the area in the dynamic object. 2. The program according to claim 1, wherein in the step of generating the advertisement image, the advertisement image is generated by transforming an original image of the advertisement based on rules preset according to the shape of the dynamic object. 2. The program according to claim 1, wherein in the step of generating the advertisement image, the advertisement image is generated by creating a three-dimensional model of the dynamic object and rendering an original image of the advertisement pasted on the three-dimensional model. . 5. The method according to any one of claims 1 to 4, wherein in the step of generating the advertisement image, a plurality of types of regions are set on the dynamic object, and an advertisement to be displayed is set for each type of region. program. causing the processor to refer to viewer information and determine advertisements that are appropriate for the viewer;
6. The program according to any one of claims 1 to 5, wherein in the step of generating the advertisement image, the determined advertisement is used to generate the advertisement image. 7. The program of claim 6, wherein determining the advertisement determines the advertisement based on information associated with the video. 8. The program according to claim 6 or 7, causing the processor to execute the step of determining the advertisement at a predetermined timing. causing the processor to execute the step of determining the advertisement at a predetermined timing;
8. The program according to claim 7, wherein in determining the advertisement, the advertisement is determined for different advertisers based on information associated with the video. A method for executing a computer comprising a processor and a memory, the method comprising:
analyzing a video frame by frame to obtain information about a dynamic object included as a subject in the video;
generating an advertisement image based on the shape, size, and orientation of a region in the dynamic object ascertained from the acquired information;
and displaying the generated advertisement image in association with the region. An information processing device comprising a control unit, wherein the control unit
analyzing a video frame by frame to obtain information about a dynamic object included as a subject in the video;
generating an advertisement image based on the shape, size, and orientation of a region in the dynamic object ascertained from the acquired information;
and displaying the generated advertisement image in association with the area. means for analyzing a video frame by frame and acquiring information about a dynamic object included as a subject in the video;
means for generating an advertisement image based on the shape, size, and orientation of a region in the dynamic object grasped from the acquired information;
and means for displaying the generated advertisement image in association with the area.

Images