JP7366606B2 - Image providing system, image providing device, control method and program - Google Patents

Description

The present invention relates to an image providing system, an image providing device, a control method, and a program.

There is a demand for photographing users having fun at sports matches and tourist facilities. Patent Document 1 discloses that when a user in a spectator seat inputs a photographing request such as a photographing area and a photographing time, a camera photographs an area where the audience is present based on the input photographing request.

Japanese Patent Application Publication No. 2007-142896

In the method of Patent Document 1, there is a possibility that the image cannot be provided to the target audience if the audience cannot be identified, such as an image projected on a screen in the venue.

The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to make it possible to provide an image displayed on a screen to an audience.

The image providing system of the present invention includes a display control means for displaying an image taken of a spectator in one of a plurality of seats in a facility on a screen installed in the facility , and an image of the spectator on the screen. an acquisition means for acquiring facial information of the audience, individual information for each audience member, and seat information for each audience member prepared before displaying the screen ; Control means to match the image of the audience displayed on the screen with at least one of the individual information and the seat information when the face information is found to match by the matching means and the matching means. It is characterized by having a control means for.

According to the present invention, an image displayed on a screen can be provided to the audience.

1 is a layout diagram of a facility that uses the image providing system in this embodiment. FIG. 1 is an overall system configuration diagram of an image providing system according to the present embodiment. FIG. 1 is a block diagram of an image processing device in this embodiment. It is a figure which shows the example of the setting of the trimming position for each seat in this embodiment. It is a figure which shows the example of the setting of the trimming position for each seat in this embodiment. It is a figure showing an example of a data structure in this embodiment. It is a flowchart showing ticket purchase processing in this embodiment. It is a flowchart which shows the setting process before a game start in this embodiment. It is a flowchart which shows the setting process after a match starts in this embodiment. It is a flowchart which shows trimming processing in this embodiment. FIG. 3 is a diagram illustrating an example of seat arrangement according to the number of people in a group in this embodiment. FIG. 3 is a diagram showing the relationship between seats in the seat arrangement according to the present embodiment. FIG. 3 is a diagram of an example data structure storing a method of calculating a trimming position in this embodiment. FIG. 3 is a diagram of an example data structure storing a method of calculating a trimming position in this embodiment. It is a figure which shows the calculation example of the trimming position in this embodiment. FIG. 2 is an overall configuration diagram of a system in the case of providing images after the end of a match in this embodiment. FIG. 2 is an overall configuration diagram of a system in the case of providing images after the end of a match in this embodiment. It is a flowchart regarding screen pre-registration processing in this embodiment. It is a flowchart regarding screen registrant list creation processing in this embodiment. It is a flowchart regarding screen image provision processing in this embodiment.

Hereinafter, exemplary embodiments for carrying out the present invention will be described in detail with reference to the drawings. However, the components described in this embodiment are merely examples, and the scope of the present invention is not intended to be limited thereto.
FIG. 1 is a facility layout diagram for performing remote control photography using the image providing system in this embodiment. For convenience of explanation, FIG. 1 uses a baseball stadium as an example, but it can also be applied to facilities where a large number of spectators can view from a fixed position, such as a stadium for sports competitions or an arena venue for concerts.

In the example of the baseball field shown in FIGS. 1A, 1B, and 1C, the photography instruction device 101 simultaneously issues photography instructions to the cameras 102 to 105 for photographing the infield seats on the first base side. In this example, the number of cameras will be described as four in order to simplify photographing, but the present invention is not limited to the number of cameras as long as it is within the range where photographing instructions can be physically given at the same time.
When photographing from the upper rows of spectator seats on the opposite side, such as at a baseball stadium, the distance to the spectators (subjects) is long. Therefore, even if a telephoto lens with a large focal length is attached, a certain amount of the audience will fit within one frame, making it necessary to perform cropping processing for each seat and group. In addition, telephoto lenses with large focal lengths are expensive, so if you try to reduce investment, you will not be able to increase the focal length, and the number of people who can fit in one frame will increase, increasing the need for cropping processing. become.
Regarding photography instructions, two methods are provided: interval photography (for example, issuing a release instruction once every 10 seconds), and manual photography according to the operator's intention.

Furthermore, depending on the facility, a location where it is easier to see a competition or event may be set at a higher price, and a location where it is harder to see may be set at a lower price, and it is common for there to be multiple ticket types. In FIGS. 1(a), (b), and (c), there are area 1 closest to the ground, area 2 with the largest number of seats, and area 3, each of which is a separate space.
Area 1 is an area where spectators want to watch the game closer, and because the number of seats is limited, they often watch the game on special occasions with a small number of people, such as families, rather than dozens of people. It is. Therefore, providing memorable images with only the members who came together will lead to improved audience satisfaction.
Area 2 is an area where many fans come to watch games with high frequency, and the fans with high enthusiasm are seated together. Furthermore, Area 2 includes spectators from various demographics, including those who came to watch the match as an experience, and those who are not very enthusiastic about the match itself. Therefore, this is an area where some spectators want images of themselves watching the game, while others think it is unnecessary, and providing images according to the wishes of the spectators will improve the satisfaction of the spectators.
Area 3 improves customer satisfaction by providing images of spectators enjoying the game in the space and images of specific spectators making good expressions.

As mentioned above, audience needs differ depending on the area. In order to meet the needs of all audiences, it is necessary to prepare captured images according to various patterns, but the larger the amount of data, the higher the cost and the audience cannot easily purchase images. There is also an issue. Further, by providing images while spectators are watching the game, the satisfaction of the spectators with the game can be further improved, but the amount of image data is large. Therefore, if you try to process the image data of audience members who are unlikely to purchase the product, it will take a long time to process it, and it will take time to provide the images.

It can be determined that the audience in areas 1 and 3 are more likely to purchase the photographed images. Therefore, in FIG. 1(a), area 1 is covered by two cameras 102 and 103, but in FIG. 1(b), area 2, which is wider than area 1, is covered by two cameras 104 and 105. Covered. As a result, the number of people that can fit in the image in area 1 is smaller, and the audience is shown larger, so the resolution of the cropped image is higher, making it possible to obtain an image with higher product quality, which leads to more purchases. I can do it. In addition, in area 3, by photographing an even narrower range than area 1 with two cameras, the resolution of the cropped image is further increased. In area 3, higher-resolution trimmed images can be acquired, so image analysis of the captured images is also possible. In Area 3, there are often multiple spaces including the viewing space due to the structure, so it is necessary to determine if there are more than a certain number of spectators in the viewing space, and to select a better image by analyzing facial expressions. It is possible to obtain an image with even higher marketability than in Area 1. Alternatively, in Area 1, since the audience is not always in a position where the camera can take pictures, it is possible to observe multiple divided spaces with one camera and use a pan head to take pictures of the audience. You may take photos of specific spaces depending on the situation.

FIG. 1C shows a configuration diagram for issuing a shooting instruction from the shooting instruction device 101 to each camera group. The photographing instruction device 101 is capable of issuing photographing instructions to a plurality of camera groups A to D, as shown in FIG. 1(c). Camera group C covers a wide outfield seating area 4 with four cameras, and camera group D covers a wide outfield seating area 5 with four cameras.

The cameras 102 to 105 are camera units each including an image sensor (imaging sensor) formed of a CCD or CMOS element that converts an optical image into an electrical signal. A camera includes a lens group (taking lens) including a zoom lens and a focus lens, a shutter with an aperture function, an image sensor, an A/D converter that converts an analog signal output from the image sensor into a digital signal, and an imaging system. Contains a barrier to cover and prevent dirt and damage. The image processing unit within the camera performs predetermined pixel interpolation, resizing processing such as reduction, and color conversion processing on data acquired by imaging with the camera. Based on the calculation results obtained by the image processing section, the CPU in the camera performs exposure control, distance measurement control, and AWB (auto white balance) processing. Image data for display captured by the camera and image-processed by the image processing section is displayed on a display 04 of the PC, which will be described later. The digital signal captured by the camera, once A/D converted by the A/D converter, and stored in the RAM is converted to analog by the D/A converter, and is sequentially transferred to the display 04 for display, allowing live view display. (LV display). Live view can be displayed in a still image shooting standby state, a moving image shooting standby state, and when a moving image is being recorded, and the captured subject image is displayed almost in real time. The CPU controls the camera and image processing so as to start operations such as AF (autofocus) processing, AE (automatic exposure) processing, and AWB processing in response to shooting preparation instructions based on user operations performed on the input device 06. control the department. In response to the shooting instruction, the CPU on the camera side performs main exposure, reads out signals from the imaging element, processes the captured image in the image processing unit, generates an image file, and records it in external storage. Control is performed to start the series of photographic processing (actual photographing) up to this point. The photographing instruction can be given by a user operation on the input device 06 of the PC 201a, or by pressing the shutter button on the camera side. The camera can take still images and videos.

Note that depending on the facility, the shooting distance and illuminance differ depending on the structure of the camera and the seat area, so depending on these conditions, there may be differences in image quality when shooting in this embodiment, which covers a large area in one shot. Sometimes it comes out. Therefore, the price may be changed or the number of images provided may be increased depending on the image quality. In addition, instead of simply setting a high price for images of seats that are close to the ground and close to the camera, there are cases where seats that have better image quality depending on conditions such as brightness and the number of cameras are set at a higher price. There is also.

FIG. 2 is an overall system configuration diagram of the image providing system in this embodiment. The image processing device 201 includes a photography instruction section 101, an image transfer reception section 201b, a trimming section 202, a selection/upload section 203, and storages 204 and 205.
In FIG. 2, each part 101, 201a, 201b, 202, 203, 204, and 205 is shown in one image processing device 201, but they may be physically separate processing devices. This configuration does not limit the present invention.

The photographing instruction unit 101 corresponds to the photographing instruction device shown in FIG. 1, and simultaneously issues photographing instructions to the cameras 102 to 105. Regarding shooting instructions, two methods are provided: interval shooting (for example, issuing a release instruction once every 10 seconds), and manual shooting according to the operator's intention.
The image transfer receiving unit 201b records in the storage 204 image data of images taken by the cameras 102 to 105 transferred from the camera side. As a protocol for transmitting and receiving image data, FTP (File Transfer Protocol) or the like is used.

The trimming unit 202 is triggered by the transfer of image data from the cameras 102 to 105 to the storage 204, performs trimming processing on each image data, and records the trimmed image and seat number in the storage 205.
Although the storages 204 and 205 are expressed as separate storages here, they may be physically the same.
In addition, when performing the trimming process, the group number, seat number, and seat arrangement purchased in advance are obtained from the ticket purchasing unit 206, which will be described later, and the trimming process is performed based on that information. will be described later in FIGS. 10 to 15.

The selection/upload unit 203 uploads the trimmed image data, seat number, and event date and time information stored in the storage 205 to the distribution unit 207 (distribution server). When the shooting instruction unit 101 performs interval shooting during a competition or event, a large number of trimmed images are created, so the operator extracts and selects the scenes where the audience is excited and sends the images to the distribution unit. 207. Although the selection method is not shown, the target image may be selected visually from a list of images, or data at the corresponding time may be extracted by inputting a specific time. It does not limit the invention. On the other hand, if the photographing instruction unit 101 only performs manual photographing by the operator, the photographing instruction will normally be given at a scene where the audience is excited, so all images may be uploaded without any particular selection.

The ticket purchase unit 206 is a mechanism for a user to purchase a ticket, and the process at the time of purchase will be described later using FIG. 8. The user enters the date and number of seats he or she wishes to purchase and pays to obtain the desired number of seats.

The distribution unit 207 uses the event date and seat number input by the user as keys, and presents the user with a group of image data of the corresponding seat number on the corresponding date. Regarding user input, the user may look at the ticket and enter it manually, or the user may print a QR code (registered trademark) on the ticket and read and recognize it with a smartphone. The input method does not limit the present invention. Furthermore, the presentation to the user may be in the form of a smartphone application or a web application accessed through a browser, and the present invention is not limited to the presentation method.

FIG. 3 shows an example of the configuration of a PC 201a (personal computer) in the image processing device 201 as an example of a device to which the present invention can be applied. The image processing device 201 may be composed of a plurality of electronic devices or may be composed of a single PC.
In FIG. 3, a CPU 01, a RAM 03, a ROM 02, a display 04, an input device 06, and an I/F 08 are connected to an internal bus 09. Each unit connected to the internal bus 09 is configured to be able to exchange data with each other via the internal bus 09.
The RAM 03 is, for example, a RAM (volatile memory using a semiconductor element, etc.). The CPU01 controls each part of the PC 201a according to a program stored in the ROM02, for example, using the RAM03 as a work memory. The ROM02 stores image data, audio data, other data, various programs for the CPU01 to operate, and the like. The ROM02 is composed of, for example, a hard disk (HD) or a ROM.

The display 04 displays images, a GUI screen forming a GUI (Graphical User Interface), etc. under the control of the CPU 01. The CPU01 generates a display control signal according to a program, and controls each part of the PC201a to generate a video signal to be displayed on the display 04 and output it to the display 04. The display 04 displays video based on the output video signal. Note that the PC 201a itself includes an interface for outputting a video signal to be displayed on the display 04, and the display 04 may be an external monitor (such as a television).
The input device 06 is an input device for receiving user operations, including a character information input device such as a keyboard, a pointing device such as a mouse or a touch panel, a button, a dial, a joystick, a touch sensor, a touch pad, and the like. Note that the touch panel is an input device that is configured in a planar manner so as to be superimposed on the display 04, and outputs coordinate information according to a touched position.

4 and 5 are diagrams showing examples of setting trimming positions for each seat in this embodiment.
FIG. 4 shows an example of setting the trimming position for an image taken from a camera substantially in front of the audience. As shown in Figure 4(a), define a rectangle for the trimming position for each seat, define a coordinate system with the horizontal direction as the X axis and the vertical direction as the Y axis, and set the top left apex position 401 and the bottom right apex position. The coordinates of 402 are stored as a data structure shown in FIG. 6, which will be described later.
By specifying the above operation for each seat included in the photographed image, the trimming positions for all seats are specified as shown in FIG. 4(b). Although a rectangular rectangle is illustrated here, the present invention is not limited to the shape of the rectangle, as long as it can be expressed in a coordinate system, such as a rectangle defined by a polygon or an ellipse.

FIG. 5 shows an example of setting the trimming position for an image taken from a camera slightly diagonally above the audience. Similar to the example of FIG. 4, a rectangle for the trimming position is defined for each seat, and a coordinate system of the X and Y axes is defined. As a result, even if the seat position is slanted, the coordinates of the upper left apex position 501 and the lower right apex position 502 can be stored as a data structure shown in FIG. 6, which will be described later.

FIG. 6 is a diagram showing an example of a data structure for storing the trimming position of each seat in the embodiment of the present invention. The data structure 600 includes a column number 601, a seat number 602, a camera number 603, an X coordinate 604 of the top left vertex coordinate, a Y coordinate 605 of the top left vertex coordinate, and an X coordinate 606 of the bottom right vertex coordinate. , and the Y coordinate of the lower right vertex coordinate 607.
In the data structure 600, the X coordinate and Y coordinate are managed as separate columns, but they may also be managed in one column as list information such as (X, Y). Furthermore, although the row number 601 and seat number 602 are numerical data in FIG. 6, they may be character string data such as alphabets. In coordinate information 604 to 607, numerical data representing pixel positions is recorded.
In this embodiment, a description will be given of providing images by taking photographs of spectators watching a baseball game as users, and trimming the images of the grouped spectators. However, the example of a baseball game explained in this embodiment is just an example, and the present invention is also applicable to other sports games, tourist spots, theaters, and other places.

The ticket purchase process in this embodiment will be explained using FIG. 7.
FIG. 7 is a flowchart showing the process when a user purchases a ticket in an application or Internet site where the user purchases a ticket. This process is realized by loading a program recorded in ROM02 (non-volatile memory) into RAM03 (system memory) and executing it by CPU01. Note that this process starts when the camera is powered on and can accept user operations on the operation unit 105.

In S701, the CPU 01 receives an input of the number of purchased tickets from the user.
In S702, the CPU 01 receives an input from the user as to whether or not he/she intends to purchase the image. The input in S702 may be such that the user can directly select whether or not to purchase an image on the purchase screen, or the ticket and image purchase may be set as a set, and the user may select whether to purchase the image or not. You can do it all at once. In addition, some seats may be available for image purchase even if the user does not make a selection, and the user may have an intention to purchase an image at the time of selecting the target seat.

In S703, the CPU 01 determines whether or not there is an intention to purchase the image based on the input in S702. If it is determined that the user has an intention to purchase the image, the process advances to S704; otherwise, the process advances to S706.
In S704, the CPU01 selects the seat specified by the user. At this time, the user may select one representative seat, and the user may easily select a seat by presenting a plurality of seat position patterns corresponding to the number of purchased tickets input in S701. Furthermore, in the image trimming process described later with reference to FIG. 10, the trimming range is changed according to the user's designation of the seat position.

In S705, the CPU01 generates group information based on the information received in S704. The group information is information that associates the representative seat with the seat position of the tickets purchased at the same time input in S701. Even if there is only one user, information is generated in step S705 in association with the seat as group information. Note that the group information may be such that other seats can be added to the same group by inputting information written on tickets that were not purchased at the same time through an app or an Internet site. However, it is preferable that the seats in a group are adjacent to each other either vertically or horizontally. This is because when trimming, seats that are far apart may be photographed by different cameras, so it may not be possible to fit the images into one trimming range. Furthermore, if the cropped image includes many other seats that are not included in the group, the size of each person will become smaller.
In S706, the CPU01 selects the seat specified by the user.

Next, using FIG. 8, a description will be given of the setting process before the start of the match in this embodiment.
FIG. 8 is a flowchart showing the setting process before the start of the match. This processing is performed by the image processing device 201 before the start of the match. This process is performed when the image processing device 201 is powered on before a match and an instruction is given by the system operator to start preparing for the target match, or before a predetermined time such as 3 hours or the day before the match. will be started. This process is realized by loading a program recorded in ROM02 (non-volatile memory) into RAM03 (system memory) and executing it by CPU01.

In S801, the CPU 01 connects with the distribution unit 207 and ticket purchase unit 206.
In S802, the CPU 01 acquires ticket information for the target match from the ticket purchasing unit 206. The ticket information includes information about the number of tickets purchased, seat position information, and seat arrangement for groups purchased at the same time, which are necessary during the trimming process. The following information is also included in order to select images with higher marketability. For example, it is information regarding the type of seat (information such as whether it is a non-reserved seat or a reserved seat). Additionally, there is information regarding the area, such as whether the seat is in the infield, outfield, or VIP area, and whether the seat is in the area to be photographed. The information also includes information on whether the user frequently watches games, information on whether the user is a member of a fan club, and the like.

In S803, the CPU 01 calculates the trimming range based on the information acquired in S802. Using the number of people and arrangement pattern of the group acquired in S802, a method for calculating the trimming position for each number of people and pattern defined in FIGS. 13 and 14, which will be described later, is acquired, and the trimming range is calculated. The trimming range here refers to the rectangle surrounded by the X and Y coordinates of the upper left vertex position and the X and Y coordinates of the lower right vertex position of the rectangle to be trimmed. . Although a rectangular rectangle defined by two vertices is used here, any rectangle that can be expressed in a coordinate system may be used, such as a rectangle defined by a polygon or an ellipse, and the shape of the rectangle does not limit the present invention. It's not something you do.

FIGS. 13 and 14 show examples of data structures that store the method of calculating the trimming position in this embodiment. FIG. 13 is an example of the definition for an image taken almost from the front of the audience, as shown in FIG. .
As shown in FIGS. 13 and 14, the number of people in the group 1301, the seating arrangement pattern 1302, the X coordinate calculation method 1303 of the upper left vertex position, the Y coordinate calculation method 1304 of the upper left vertex position, and the lower right vertex position It manages an X coordinate calculation method 1305, a Y coordinate calculation method 1306 of the lower right vertex position, and information 1307 about seats outside the group.
The number of people in the group 1301 and the seating arrangement pattern 1302 match the purchased group number and seating arrangement obtained in advance from the ticket purchasing unit 206. The coordinate calculation methods 1303 to 1306 use the trimming position information of one of the seat positions in the group in the examples of FIGS. 13 and 14, but it is also possible to define a specific number of pixels or a calculation formula. do not have. Information 1307 on seats outside the group is defined as information on seat positions that fit within the rectangle in the seat arrangement pattern but have not been purchased as seats within the group.

Returning to FIG. 8, in S804, the CPU 01 prepares face data, which is face information. Based on the information acquired in S802, face data is prepared for users who frequently watch matches, that is, for spectators whose faces have been detected frequently in the past, and for spectators who have joined a fan club. . As a result, if there is an audience member in the photographed image whose face is determined to match the face in the face data, the images of the specific audience member can be linked and collected as data and provided to the specific audience member at once. I can do it. In addition, spectators who are members of a fan club can automatically collect images without having to indicate their intention to purchase an image each time they purchase a ticket, allowing them to look back on the game at the end of the season. . Depending on the audience, the frequency with which they watch games varies greatly, so some spectators may want multiple images of a single game, while others may only need one image per game, but may want images of all the games they watched. There is sex. Therefore, by preparing the face data as described above, it is possible to provide images to spectators who frequently attend the game without having to go through the process of indicating their intention to purchase images each time. In addition, even if the user forgets to indicate his or her intention to purchase an image each time, the user can still provide the image. Furthermore, images of specific spectators can be collected even if they are seated in unreserved seats whose seating positions are unknown. Note that instead of the information obtained at the time of ticket purchase obtained in S802, a plan is set up for spectators to purchase images separately from the purchase of spectator tickets, and if the plan is selected, facial data is registered and the spectators It may also be possible to provide an image of the game being watched each time. In the above cases, by preparing face data, images can be smoothly provided to the spectators during the match.

In S805, the CPU01 sets the clock.
In S806, the CPU 01 inputs shooting factors. The shooting factor may be based on an interval shooting period such as 10 seconds or 20 seconds, or based on detection such as when the cheers of the audience exceed a predetermined value. In addition, we have prepared a control unit that analyzes the video of the match and the match data, and can shorten the interval depending on the score of the match, the progress of the match, and the position of the players. , it is also possible to switch to video shooting. Furthermore, interval shooting may be stopped during breaks, and may be started during breaks when events such as cheering contests and quizzes occur. It goes without saying that a manual photographing instruction may also be combined.

In S807, the CPU 01 actually starts up the camera and performs processing for photographing and trimming as a system test. Based on the captured image, the shooting conditions are set according to the brightness and amount of light that day, and when the captured image is cropped based on the coordinates associated with the seat, the target seat is cropped. You can check if it is possible. Note that it is also possible to place an object that serves as an index point on the seat side and check whether or not the object in the image taken by the camera is located at predetermined coordinates.
In S808, the CPU 01 sets imaging conditions based on the system test in S807.

Next, with reference to FIG. 9, processing after the start of the match in this embodiment will be described.
FIG. 9 is a flowchart showing the setting process after the start of the match, in which (a) shows the process of the image processing device 201, (b) shows the process related to shooting, and (c) shows the process of trimming and distributing the image. Each process is started before the match starts or before the start of shooting before the start of the match. This process is realized by loading a program recorded in ROM02 (non-volatile memory) into RAM03 (system memory) and executing it by CPU01.

The processing of the image processing device 201 in FIG. 9(a) will be explained.
In S901, the CPU01 acquires midway ticket information. The mid-game ticket information is information regarding tickets purchased after the start of the match or after the process of S802 in FIG. 9. This information may be acquired from the ticket purchase unit 206, or may be notified to the image processing apparatus 201 from the ticket purchase unit 206 periodically or every time there is an update.
In S902, the CPU01 acquires information regarding the camera corresponding to camera c=1. For example, information regarding the photographing range of camera c=1 and the target seat is acquired.

In S903, the CPU 01 determines whether the camera c that is currently being processed is larger than the total number K of cameras, and if so, it is determined that the processing for all cameras has been completed and the process proceeds to S907. If not, the process advances to S904.
In S904, the CPU 01 acquires trimming information in the range of the seat to be photographed from the position of the target camera c.

In S905, the CPU 01 acquires the photographing factors in the range from the position of the target camera c to the seat where the photograph is to be taken. For example, when Team A and Team B are playing against each other, it may be better to change the shooting interval depending on the timing of offense and defense. Also, depending on the type of seat, it may be better to take pictures more frequently. Therefore, by acquiring the shooting factors according to the seats included in the shooting range of the camera, it is possible to avoid missing a shooting opportunity without shooting many images that are likely not to be provided to the audience. Note that the shooting factor in S905 may be the same for all cameras.
When a new ticket is purchased or the audience indicates a new intention to purchase an image, that is, a new trimming group is generated, the processes of S803 and S804 in FIG. 8 are performed in S904 and S905. .

In S906, the CPU 01 sets camera c=c+1 as in S902, and acquires information regarding the next camera.
In S907, the CPU01 determines whether a predetermined time has elapsed. The predetermined time is measured as the time from S901 if the determination in S907 is made for the first time after the start of the process in FIG. S907 is for acquiring en route ticket information at predetermined intervals. If the determination is Yes in S907, the process advances to S901; otherwise, the process advances to S908.

In S908, the CPU 01 determines whether or not there was a shooting factor that caused the shooting to take place in S905. If the determination is Yes in S908, the process advances to S909; otherwise, the process advances to S907.
In S909, the CPU 01 notifies the camera side of an instruction to take a photograph.
In S910, the CPU 01 notifies the trimming section 202 of trimming information. Since trimming information and information regarding shooting factors are updated at predetermined intervals, even spectators who purchase tickets midway through the match will be able to obtain cropped images from midway through the match.
In S911, the CPU 01 determines whether to end the process, that is, whether to end the match and end the shooting. If the process ends, the process advances to S912; otherwise, the process advances to S901.
In S912, the CPU 01 notifies the camera that the power will be turned off.

As explained above, since the trimming information is sent to the trimming unit 202 almost simultaneously with the shooting instruction, the registered trimming process can be executed quickly after shooting. When the image is sent from the camera side to the storage 204, the trimming unit 202 knows the group to be trimmed and can calculate the trimming range, so it can quickly perform the trimming process and allow the audience to check the image.

Next, the processing related to the photographing shown in FIG. 9(b) will be explained. The process shown in FIG. 9(b) is a process performed in each camera.
In S930, the CPU 01 causes the camera side to acquire the shooting conditions.
In S931, the camera side determines whether or not a photographing instruction has been given. If a photographing instruction has been given, the process advances to S932; otherwise, the process waits for a photographing instruction to be given.
In S932, the camera side performs photography under the set photography conditions.
In S933, the CPU 01 controls the camera to transfer the image to the image transfer receiving unit 201b, and the transferred image data is stored in the storage 204. Alternatively, images taken by the camera are sequentially transferred to the storage 204.
In S934, the CPU 01 determines whether to end the process of FIG. 9(b). The determination in S934 becomes Yes if an instruction to turn off the camera power is received in S912 of FIG. 9(a). If it is determined in S934 that the process in FIG. 9(b) is to be ended, the process in FIG. 9(b) is ended; otherwise, the process advances to S931.

Next, the process of trimming and distributing the image shown in FIG. 9(c) will be explained.
In S951, the CPU 01 determines whether an instruction to acquire an image has been given. In the case of baseball, the instruction to acquire an image refers to an event time such as the time of a home run or the time of a fine play, based on the clock time set in S805 of FIG. This is an instruction to obtain a photographed image. The image acquisition instruction is given in response to the occurrence of an event time. Note that even if it is not the event time, for example, the CPU 01 may perform an analysis of the match and issue an instruction to shoot an image and an instruction to obtain an image at the same time in response to the occurrence of an event.
In S952, the CPU 01 acquires the target image from the storage 204.
In S953, the trimming unit 202 acquires trimming information.
In the processing after S954, trimming processing is started.

In S954, the trimming unit 202 acquires information about the seat n=1. Then, let m be the number of seats in the photographing range of the camera that photographed the image that is currently being trimmed. In this embodiment, for the sake of simplicity, n = 1 to m for the seat numbers, but this is not the only option. If covered, it may be linked to the seats actually used at the stadium. In order to cover the audience seats with multiple cameras, the shooting range may overlap with that of other cameras. Even if the audience is photographed by two cameras, an image trimmed from the image photographed by one of the cameras is provided. Therefore, the photographing range of the camera and the trimming range may be different. In S954, the number of seats in all the trimming ranges is acquired as m. The trimming range may be variable during the game, or may be determined in advance by the stadium.

In S955, the CPU01 determines whether the seat is in the area to be trimmed. At baseball stadiums, there may be areas where cameras are blind due to nets, poles, roofs, etc., or where photography is prohibited. Furthermore, in areas where the positions of spectators are unknown, such as non-reserved seats, trimming corresponding to the position of the seats is not performed in the first place. Therefore, the determination in S955 is No. Note that even if the seats are non-reserved seats, trimming by face detection or trimming of a wider range can be performed as described later. If the determination is Yes in S955, the process advances to S956; otherwise, the process advances to S963. Note that it may be possible to indicate to the audience that this is an area where image purchases are not possible when purchasing tickets or near the seats.
In S956, the CPU 01 determines whether the seat n currently being determined as a trimming target is smaller than the number m of seats within the trimming range. That is, it is checked whether the determination as to whether or not to perform trimming has been completed for all seats in the trimming range. If the determination in S956 is Yes, the process advances to S957; otherwise, the process advances to S965.

In S957, the CPU01 determines whether seat n is a seat to be trimmed. Whether the seat is to be trimmed or not is determined depending on whether or not the audience has indicated an intention to purchase the image in FIG. That is, the determination in S955 is Yes if the seat is included in the trimming group, and No otherwise.
In S958, the CPU 01 selects a trimmed image, which is the original image to be trimmed, from the image acquired in S952. The cropped image may be selected depending on the appearance of the audience in the cropping group for seat n, which is currently being targeted, by selecting the one with the best expression or the one with the most movement compared to the previous and subsequent images. good. Alternatively, images may be selected manually without changing the images selected for each trimming group, or the movement of the audience within the trimming range may be quantified and the image that is thought to have the largest movement may be selected.

In S959, the CPU01 executes trimming processing. The trimming process will be described later using FIG. 10.
In S960, the CPU01 sets seat n=n+1. Note that if seat n is included in a trimming group that has already undergone trimming processing, then n+1 is further set and a search is made for a seat that has not yet undergone the determination in S957.
In S961 and S962, processing when the group trimming of the audience at seat n is not included will be explained. In this embodiment, the area is trimmed in association with the seat number so that the audience can obtain the image after the match. That is, instead of determining the trimming range so that the audience who purchased at the same time enters and other audience members do not enter, as in group trimming, trimming is performed for a predetermined number of people, such as 30 or 50 people. This reduces the time and amount of data required for cropping, making it possible to quickly provide images to audience members targeted for group trimming, while also allowing images to be provided later to audiences who are not eligible for group trimming. I have to.

In S961, the CPU 01 selects a trimmed image similarly to S958. In the case of area trimming, since trimmed images are not saved in accordance with the audience at each seat, a trimmed image is not selected depending on the audience at seat n. In the case of area trimming, images may be selected manually, or the movement of the audience within the trimming range may be digitized and the image with the largest movement may be selected. Alternatively, an image for which face data has been prepared and shows a good facial expression or a large movement of the audience may be selected. In this way, even if the audience is not subject to group cropping, by selecting images with good facial expressions or large movements of the audience who are likely to purchase the image, it is possible to increase the likelihood of purchasing the image in the future. It can satisfy more audience satisfaction. If images with good facial expressions of each audience member are saved, the amount of data increases, leading to increased costs, and retrieving the data also takes time. By prioritizing images for spectators who are likely to purchase and frequently watch games, images can be provided more quickly and at a lower cost. In addition, for spectators whose faces have been detected, images of the target spectators may be collected and notified after a predetermined number of games, such as 10 games or 5 games, or when watching the next game. You may notify the audience of the image and encourage them to purchase it. Furthermore, by presenting the images taken at the previous game to the spectators when purchasing tickets for the next game, the spectators can understand what kind of images will be taken and decide whether or not to purchase the image. It may be easier to make a decision.

In S962, the CPU01 performs area trimming. As described above, area trimming involves trimming an image divided by a predetermined number of people. If the seat n is included in an image that has already been area trimmed in the process for another seat n, area trimming is not performed again and the data amount of the image is reduced.
In S963 and S964, processing similar to S961 and S962 is executed. If it is an unreserved seat that is not a seat to be trimmed, processing based on face detection may be performed, and if the seat is not a seat to be trimmed and the audience is not within the group trimming range, area trimming may not be performed. Furthermore, the range of area trimming may be widened in seats where some enthusiastic fans are seated or where a group is sitting. For example, users wearing clothes of the same color may be set as the trimming range. This allows groups wearing the uniforms of the team they support to be captured in one photo.

In S965, the CPU 01 determines whether or not there is face data in the trimmed image. If there is face data, the process advances to S966; otherwise, the process advances to S967.
In S966, the CPU 01 associates each spectator's ID with the image, and turns on the image flag for the match that the spectator is watching. If the image flag is ON, it is possible to present to the audience which game the image was taken for.
In S967, the CPU 01 uploads the cropped image to the distribution unit 207, and makes the image viewable by the audience targeted for group trimming. Note that the trimmed image may be presented even if the audience is not the target of group trimming. At this time, characters such as samples are added with a watermark to the audience who are not subject to group trimming. Note that by deleting the image data that has not been uploaded from the storage 204 or recording it with a reduced image quality, the capacity of the image can be reduced, leading to cost reduction. However, after the match, the images selected in S958, 961, and 963 may be saved for a predetermined period of time, or each spectator may be notified before deletion.

FIG. 10 is a flowchart showing trimming processing in this embodiment. This process is realized by loading a program recorded in ROM02 (non-volatile memory) into RAM03 (system memory) and executing it by CPU01.
In S1001, the CPU 01 reads out the purchased group number, seat number, and seat arrangement obtained in advance from the ticket purchasing unit 206.

FIG. 11 is a diagram showing an example of seat arrangement according to the number of people in the group. FIG. 11A shows an example of a seating arrangement for a group of four people. In pattern (1), four people are lined up side by side, and in pattern (2), two people are lined up one behind the other. The pattern is just an example, and the combination of pattern shape and number of participants does not limit the present invention. Similarly, FIG. 11(b) is an example of a seating arrangement for a group of five people, and FIG. 11(c) is an example of a seating arrangement for a group of six people.

FIG. 12 is a diagram showing the relationship between seats in the seat arrangement. Based on the lower left seat position in the group, the horizontal seat position is expressed as "ban" and the vertical seat position is expressed as "column", and the "ban" number is incremented towards the right side in the horizontal direction, and the vertical seat position is expressed as "row". The "column" number is incremented in the upward direction.
FIG. 12 shows a specific example of the seat arrangement in pattern (2) of FIG. 11(b). Although there are no users in the group in column X+2 and Y, since this is necessary as data when trimming the rectangle, it is defined as a seat outside the group in the data structure of FIGS. 13 and 14, which will be described later.

Returning to the explanation of FIG. 10, in S1002, the CPU 01 acquires the trimming range and out-of-group seat information. Note that the trimming range may be calculated in S1001 without calculating the trimming range in advance.
In S1003, the CPU 01 determines whether information on seats outside the group exists in the group information acquired in S1002. If it is determined that information on seats outside the group exists, the process advances to S1004; otherwise, the process skips S1004 and advances to S1005.
In S1004, the CPU 01 fills in the trimming range of the seat position defined as the seat outside the group with a single gray color. If the entire range of trimming positions of seats outside the group is filled in, some of the trimming positions may overlap with the trimming positions of seats in the adjacent group above and below, so it is desirable to fill in slightly inside the trimming position. This is a process to prevent spectators outside the group from being photographed, but the blackout method is not limited to this name. Furthermore, if there is no particular problem with portrait rights and the user's intention is that there is no problem even if a person outside the group is included in the photo, the process of S1004 may not be necessary.
In S1005, the CPU 01 determines the trimming position according to the coordinate calculation methods 1303 to 1306 (trimming position calculation method) acquired in S1002.

FIG. 15 is a diagram showing an example of calculating the trimming position in this embodiment. Figure 15(a) is an example of the calculation method for an image taken almost from the front of the audience as shown in Figure 4, and Figure 15(b) is an example of the calculation method for an image taken with the audience slightly diagonally as shown in Figure 5. This is an example of how to calculate. In FIG. 15A, the upper left vertex of the upper left seat and the lower right vertex of the lower right seat are simply used as the vertices of the entire trimming position. In FIG. 15(b), since the upper and lower columns are slightly shifted, the upper left vertex and the lower right vertex are defined as follows, and the coordinate positions of the upper left vertex and the lower right vertex are determined using this calculation method.
Upper left vertex X coordinate = X coordinate of the upper left vertex of the lower left seat Y coordinate = Y coordinate of the upper left vertex of the upper left seat Lower right vertex X coordinate = X coordinate of the lower right vertex of the upper right seat Y coordinate

Returning to the explanation of FIG. 10, in S1006, the CPU 01 generates a trimmed image cut out from the entire image taken by the camera at a trimming position according to the number of people and the seating arrangement, according to the trimming position determined in S1005. The CPU 01 stores the trimmed image in the storage 205 together with the photographing date and seat number information.
Note that in the embodiment described above, the system may be operated as follows to allow the audience to acquire images more easily.
・The viewing request to the distribution server is made by reading the QR code of the ticket.
- Group information is acquired in advance, but it may be acquired each time the process is performed. In this case, the ticket purchasing system also needs to constantly recreate the group information, which is acquired from the trimming unit 202 each time, which increases the amount of processing for the system as a whole. However, for example, in the case of baseball, a large percentage of same-day ticket sales are made, and this is effective when a large amount of group information is created after the game starts.

Next, with reference to FIGS. 16 and 17, a case will be described in which a spectator does not indicate an intention to purchase an image in advance, but attempts to purchase an image after the game ends.
FIG. 16 is an overall system configuration diagram for providing images after the match in this embodiment. The image processing device 201 includes a shooting instruction section 101, cameras 102 to 105, an image transfer reception section 201b, a selection/upload section 203, and a storage 204.
In FIG. 16, each unit 101, 201b, 203, and 204 is shown in one image processing device 201, but they may be physically separate processing devices, and the configuration of the processing unit may be different from the present invention. It is not limited. Similarly, the distribution unit 207 and the trimming unit 202 may also be physically separate processing devices.

The photographing instruction section 101, cameras 102 to 105, image transfer/reception section 201b, and ticket purchase section 206 are the same as those described in FIG. 2, so a description thereof will be omitted.
The selection/upload unit 203 is triggered by the storage of the captured image data in the storage 204, and uploads the captured image data to the distribution unit 207 along with event date and time information and trimming position information for each seat.
The captured image data here is high-resolution data that has not been cropped, so when the number of shots increases due to interval shooting, etc., or when the number of cameras increases, a large amount of data will be sent to the distribution unit. It will be uploaded to 207. Therefore, when attempting group trimming later, the system must hold a larger amount of data.
Note that although the trimming position information for each seat is uploaded from the image processing device 201 to the distribution unit 207 here, the distribution unit 207 and the trimming unit 202 may have the information. However, the trimming position information is only the position information for the images taken by the cameras 102 to 105, so it must be updated each time the mounting position or angle of view of the cameras 102 to 105 on the image processing device 201 side is changed. It becomes necessary. Therefore, the systems including the image processing device 201, the distribution unit 207, and the trimming unit 202 need to cooperate more closely.

The distribution unit 207 sends the thumbnail position information and seat number obtained from the selection/upload unit to the trimming unit 202 using the event date and seat number input by the user as keys, and instructs the trimming process to display the obtained trimming. Present the completed images to the user.
As for the user's input, the user may look at the ticket and manually enter the information as in FIG. 2. Furthermore, the input method is not limited to the above-mentioned method, such as printing a QR code on the ticket and reading it with a smartphone for recognition, or facial detection if the ticket information is unknown.
Furthermore, the presentation to the user may be in the form of a smartphone application or a web application accessed through a browser, and the present invention is not limited to the presentation method.

Although the processing itself is the same as that shown in FIG. 2, the trimming unit 202 generates a trimmed image each time the user wishes to purchase. Therefore, the trimming time is included in the time it takes for the user to obtain the trimmed image, making it impossible to quickly provide the image. In addition, in order to perform image processing quickly, an environment that can perform image processing at high speed is required.

Next, a case where image data is stored in the storage 204 will be described using FIG. 17. FIG. 17 is a diagram for explaining the overall system configuration and processing when providing images after the match ends. In the process shown in FIG. 17, unlike the case in FIG. 16, a large amount of image data is not transmitted to the distribution unit 207. However, since the image processing device 201 is requested to upload the image taken by the target camera after receiving a trigger to purchase an image from the user, it takes longer than the case in FIG. 16 to acquire the image. Put it away. Therefore, it takes more time for the user to obtain the cropped image data.

In the example shown in FIG. 17, on the premise that the user has purchased a ticket, the process starts when the user instructs the distribution unit 207 to purchase an image (step 1).
The distribution unit 207 requests the image processing device 201 for an image of the target camera (step 2), and requests the image from the storage 204 within the image processing device 201 (step 3). Thereafter, the selection/upload unit 203 transmits the image data to the distribution unit 207 (step 4). At this time, since the image data includes date information, trimming position information for each seat, and image data before trimming, the amount of data becomes large. When the distribution unit 207 issues a trimming instruction to the trimming unit 202 (step 5), the trimming unit 202 performs trimming, and the trimmed image data is distributed to the distribution unit 207 (step 6). Thereafter, the trimmed image is distributed from the distribution unit to the audience who purchased the image (step 7).

In this way, in the process shown in FIG. 17, even if the audience does not indicate an intention to purchase the image in advance, it is possible to provide the audience with an image that has been trimmed for the trimming group. Note that there is a lot of interaction within the system and within the device, and it may take longer to provide the images than if the audience had the intention of purchasing the images in advance.
When providing an image to the audience, seat information is also linked and provided to the audience, so that when the audience uploads the image to SNS, the person viewing the image can take a good image based on which seat position. be able to recognize what is going on.
Furthermore, by performing image processing related to brightness according to the seat position and providing an appropriate image for dark and bright areas, it is possible to capture a wide range of audience members within the same angle of view. This makes it possible to provide even better images. If the brightness varies by more than a predetermined value within the same angle of view, a RAW image may be acquired to facilitate image processing later.

Next, we will explain how to provide images of the audience reflected on the monitors at the venue. Sports venues and entertainment facilities such as aquariums often have large screens. There are also events where the audience is reflected on the screen. At this time, in order to provide the image reflected on the screen to the audience, it is necessary to identify the audience. However, when face detection is used, it takes a long time for sports games with tens of thousands of spectators.
In this embodiment, a method for providing an image reflected on a screen to an audience will be explained using FIGS. 18 to 20.

FIG. 18 is a flowchart regarding screen pre-registration processing. This process can be initiated in response to ticket sales starting prior to the start of the game. This process is realized by loading a program recorded in ROM02 (non-volatile memory) into RAM03 (system memory) and executing it by CPU01.
In S1801, the CPU 01 determines whether the audience has the intention of purchasing the trimmed image. If it is determined that the user intends to purchase the trimmed image, the process advances to S1802; otherwise, the process skips S1802 and advances to S1803.
In S1802, the CPU 01 registers the seat number in the image purchaser list.
In S1803, the CPU 01 determines whether advance registration for screen image acquisition has been made. To enable a user to select whether or not he or she wants an image when it appears on a screen when purchasing a ticket. If pre-registration is selected, the process advances to step S1804; otherwise, the pre-registration process ends.

In S1804, the CPU 01 registers the seat number in the screen list. Note that the image purchaser list and the screen list are created for each match.
In S1805, the CPU 01 determines whether face registration has been performed. If pre-registration is selected in S1803, a guide is displayed informing the audience that if they take a picture of their face, they will have a higher probability of obtaining the image that appears on the screen. If the audience selects to register their face, the face of the registrant is photographed in S1806. If it is selected not to register the face, the process advances to S1808. Regarding photography, if the application terminal is a smartphone, it may be taken on the spot with a built-in camera, or it may be selected from images that have been taken in advance.

In S1807, the CPU 01 links the seat number and face data.
In S1808, the CPU 01 issues a temporary ID to the audience, and registers the linked information in the screen list in S1807. The temporary ID is individual information distributed to spectators who have pre-registered for each game, and is used to obtain images.
According to the embodiments described above, by registering their faces in advance, the audience can easily obtain an image of themselves reflected on the screen.

Next, the spectator list created on the day of the match will be explained using FIG. 19.
FIG. 19 is a flowchart regarding screen registrant list creation processing. This process is performed within a limited time such as 10 or 20 minutes after the start of the match. This process is realized by loading a program recorded in ROM02 (non-volatile memory) into RAM03 (system memory) and executing it by CPU01.
In S1901, the CPU 01 acquires an image purchase list and a screen list that have been created in advance.
In S1902, the CPU 01 determines whether a spectator who has a fixed ID has entered the venue. A fixed ID is to be distinguished from a temporary ID, and is individual information distributed to spectators who have joined a fan club or registered with a team app. It is assumed that the spectators who hold the fixed ID are those who attend the games frequently, and because they have come many times, they already have facial data.

In S1903, the CPU 01 registers the spectators with fixed IDs who have entered the screen in the screen list, and prepares face data.
In S1904, the CPU 01 accumulates face data of audience members who have an intention to purchase the trimmed image. Spectators who are willing to purchase the cropped image accumulate face data from the cropped image.
In S1905, the CPU 01 accumulates face data of spectators for whom there is no face data among the spectators registered for screen image acquisition. Trimming is performed to fit the audience at the position registered in S1804, and face data is accumulated.

In S1906, the CPU 01 creates an event subject list from the image purchase list and the screen list. This is a list used when the cheering spectators are filmed with a video camera and shown on a screen between games. Even if multiple video cameras are used to film, not all seats will be shown, so seats that are not captured by the video cameras during the event should not be included in the event subject list. For example, seats on a floor that cannot be seen from the position of the event video camera or seats in front of the event video camera are not included in the event subject list.

In S1907, the CPU 01 creates a list of reflections during the match from the image purchase list and the screen list. The spectators may be displayed on the screen along with the players during the match. For example, there is a high possibility that spectators seated in area 1 will be seen together with the players when they are shown. Therefore, an image purchase list and a screen list are created for seats that can be photographed by a video camera for showing players during a match, and a list for reflections during the match is created.

According to the embodiment described above, since a list is created by narrowing down the list to the audience who may want the image displayed on the screen, it is possible to reduce the processing load during face detection, and Images can be quickly provided to the audience. Furthermore, by creating different lists depending on the factors displayed on the screen, the processing load can be further reduced.

Next, the image providing process will be explained using FIG. 20.
FIG. 20 is a flowchart regarding screen image provision processing. This process is started in response to the audience starting to be displayed on the screen. This process is realized by loading a program recorded in ROM02 (non-volatile memory) into RAM03 (system memory) and executing it by CPU01.
In S2001, the CPU01 determines whether an event is in progress. If it is determined that an event is in progress, the process advances to S2002; otherwise, the process advances to S2003.
In S2002, the CPU 01 prepares an event subject list.
In S2003, the CPU 01 prepares a list of objects reflected during the match.

In S2004, the CPU 01 acquires a screen image of the audience. This image may be obtained from a video camera that photographs the audience, or a camera may be provided for viewing the screen.
In S2005, the CPU 01 detects the face of the main subject from the image acquired in S2004.
In S2006, the CPU 01 compares the face detected in S2005 with the face data of the ID holders in the list prepared in S2002 or S2003, and determines whether they match. If it is determined that the ID holder and the face data match, the process advances to S2007; otherwise, the process advances to S2008.

In S2007, the CPU 01 stores in the storage 204 the image displayed on the screen along with the seat number and date of the audience whose face data was determined to match in S2006, and immediately distributes the image to the audience. Note that what is distributed here may not be a still image, but a video.
In S2008, the CPU 01 determines whether or not the face detected in S2005 matches the face data of the audience members in the list prepared in S2002 or S2003 who had an intention to purchase the trimmed image. If it is determined that the detected face matches the face data of the audience member who intended to purchase the trimmed image, the process advances to S2009; otherwise, the process advances to S2010.

In S2009, the CPU 01 saves the image displayed on the screen together with the trimmed images before and after the photographing time of the audience whose face data was determined to match in S2008 in the storage 204, and immediately distributes the image to the audience.
In S2010, the CPU 01 determines whether the face detected in S2005 matches the face data of a person who desires to obtain a screen image in the list prepared in S2002 or S2003. If it is determined that the person who wishes to obtain the screen image and the face data match, the process advances to S2011, and if not, the process of FIG. 20 ends. Note that if it is determined that an event in which the audience is reflected on the screen has occurred, the process proceeds to S2001 again and processing is performed.

In S2011, the CPU 01 stores the seat number and date in the storage 204, and distributes it to the audience. Note that the distribution method may be a different notification method from the method of distributing the trimmed image to the person who wishes to purchase the trimmed image, or may be the same.
Furthermore, the image displayed on the screen that is distributed to the audience may be an image from a video camera, or may be an image taken from the screen itself.

According to the embodiment described above, when an image is provided to the audience after detecting the face of the audience displayed on the screen, the image can be provided quickly and reliably.
It creates a list of spectators who are likely to want images, and then narrows down the list to a range of spectators who are likely to be imaged for each event to determine matches with detected faces, resulting in faster processing. It can be carried out.

<Other embodiments>
The present invention provides a program that implements one or more functions of each embodiment described above to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device reads the program. This can also be achieved by executing a process. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.
The embodiments described above are merely examples of implementation of the present invention, and the technical scope of the present invention should not be construed as limited by these embodiments. That is, the present invention can be implemented in various forms without departing from its technical idea or main features.
Note that the various controls described above as being performed by the image processing device 201 may be performed by a single piece of hardware, or by sharing the processing with multiple pieces of hardware (for example, multiple processors or circuits), the device It is also possible to perform overall control.

Further, although the present invention has been described in detail based on its preferred embodiments, the present invention is not limited to these specific embodiments, and the present invention may be applied to various forms without departing from the gist of the present invention. included. Furthermore, each of the embodiments described above is merely one embodiment of the present invention, and it is also possible to combine the embodiments as appropriate.
Although the above-described embodiment has been described as an image providing system, it may also be configured as an image providing device.
Further, in the above-described embodiment, the case where the present invention is applied to the image processing device 201 has been described as an example, but this is not limited to this example, and the present invention is not limited to this example, and the present invention is not limited to this example, and the present invention is not limited to this example. It is applicable to any system consisting of That is, the present invention is applicable to personal computers, PDAs, mobile phone terminals, portable image viewers, printer devices with displays, digital photo frames, music players, game machines, electronic book readers, and the like. It goes without saying that it is also applicable to perform control in conjunction with AI (artificial intelligence).

The present invention is also realized by performing the following processing. That is, the software (program) that realizes the functions of the embodiments described above is supplied to a system or device via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or device reads the program code. This is the process to be executed. In this case, the program and the storage medium storing the program constitute the present invention.

Claims (4)

a display control means for displaying an image taken of a spectator in one of a plurality of seats in the facility on a screen installed in the facility ;
acquisition means for acquiring face information of the audience, individual information for each audience, and seat information for each audience, prepared before displaying images of the audience on the screen ;
collation means for detecting the faces of the audience displayed on the screen and collating them with the facial information;
control means for controlling the image of the audience displayed on the screen to be associated with at least one of the individual information and the seat information when the face information is found to match by the matching means;
An image providing system comprising: a display control means for displaying an image taken of a spectator in one of a plurality of seats in the facility on a screen installed in the facility ;
acquisition means for acquiring face information of the audience, individual information for each audience, and seat information for each audience, prepared before displaying images of the audience on the screen ;
collation means for detecting the faces of the audience displayed on the screen and collating them with the facial information;
control means for controlling the image of the audience displayed on the screen to be associated with at least one of the individual information and the seat information when the face information is found to match by the matching means;
An image providing device comprising: displaying an image taken of a spectator in one of a plurality of seats in the facility on a screen installed in the facility;
obtaining face information of the audience, individual information for each audience, and seat information for each audience, which are prepared before displaying images of the audience on the screen;
detecting the faces of the audience displayed on the screen and comparing them with the facial information;
a step of associating the image of the audience displayed on the screen with at least one of the individual information and the seat information if the face information is determined to match through the comparison;
A method for controlling an image providing system, comprising: Displaying an image taken of a spectator sitting in one of a plurality of seats in the facility on a screen installed in the facility;
A process for acquiring face information of the audience, individual information for each audience, and seat information for each audience, which is prepared before displaying images of the audience on the screen;
a process of detecting faces of the audience displayed on the screen and comparing them with the face information;
a process of linking the image of the audience displayed on the screen with at least one of the individual information and the seat information if the facial information is determined to match through comparison;
A program that causes a computer to execute

Images