JP7246146B2 - Information processing device and video projection system - Google Patents

Description

The present invention relates to technology for projecting images.

As a technique for projecting an image, Patent Document 1 discloses a technique for adjusting unevenness in a superimposed area where projected images are superimposed when projecting a tiling image by combining projected images from a plurality of projectors.

Japanese Patent Application Laid-Open No. 2018-5018

For example, in a sports broadcast, multiple images of the athletes captured from various angles using multiple cameras (in some cases, images representing the field of view of the athletes) are broadcast while switching between them. are increasingly being offered. On the other hand, when displaying an image, it is often the case that the image is displayed on the display surface provided by the display device itself, such as a television. Because it becomes the main, it is hard to say that those multiple images are fully utilized.

A projection device such as a projector used in the technology disclosed in Patent Document 1 can display a larger image area than a television or the like relative to the size of the device itself. It is difficult to be subject to size restrictions when displaying images.
SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to effectively utilize a plurality of images captured by a plurality of imaging devices using a projection device.

In order to achieve the above object, the present invention provides an object acquisition unit for acquiring object information indicating a plurality of objects to be photographed respectively by a plurality of photographing devices; Two or more images to be photographed, wherein the plurality of images to be photographed include an objective image photographed from the surroundings of the subject of the event and a subjective image photographed by a photographing device attached to the subject. is assigned to each projection unit of a projection device having a plurality of projection units, an instruction unit for instructing to project the image according to the allocation of the allocation unit , and the photographing device a excitement degree determination unit that determines the excitement degree of the event from the video, and the allocation unit projects the objective video when the determined excitement degree is equal to or higher than a predetermined standard. Provided is an information processing device that allocates the subjective image to a projection unit that has been used .

Further, the present invention provides an object acquisition unit that acquires object information indicating a plurality of objects to be photographed by a plurality of imaging devices, and two or more objects to be photographed among the plurality of objects to be photographed indicated by the acquired object information. A plurality of videos including a wide-range video including a specific shooting target and having a range wider than a predetermined range, and a local video including the shooting target and having a range within the predetermined range. an allocation unit that allocates to each projection unit of a projection device having a projection unit, an instruction unit that instructs to project an image according to the allocation of the allocation unit, and a speed identification unit that identifies the speed of the specific shooting target wherein the allocating unit allocates the wide-range image to a projection unit that projects the local image when a speed equal to or greater than a threshold value is specified for the specific shooting target appearing in the local image, or Provided is an information processing device that allocates the local image to a projection unit that projects the wide-range image when a speed of less than a threshold is identified for the specific shooting object appearing in the wide-range image.

Further, when a reference image serving as a reference is designated, the allocation unit projects the reference image according to the positional relationship in the real space between the object to be photographed in the reference image and the object to be photographed in another image. The images of the two or more shooting targets may be assigned so as to have a positional relationship between the projection area and the projection area onto which the other image is projected.

Further, a excitement degree determination unit is provided for determining the degree of excitement of the event from the images captured by the plurality of shooting devices with the event as a shooting target, and the assignment unit determines the priority of the plurality of projection units. In this case, the image of the shooting location with the higher determined degree of excitement may be assigned to the projection unit with the higher priority.

Further, a congestion degree determination unit is provided for determining a degree of crowding of people in a place where the photographing device shoots, and the allocating unit determines the degree of congestion of the video when the plurality of projection units are prioritized. The image may be assigned to a projection unit having a priority according to the degree of congestion.

Further, the apparatus includes a preference acquisition unit that acquires preference information indicating user preferences, and the allocating unit, when priority is determined for the plurality of projection units, determines whether or not the user's preferences indicated by the acquired preference information are obtained. Therefore, a place that the user is likely to visit may be assigned to a projection unit having a higher priority for the image to be shot.

Further, when the projection device projects an image designated by a user for each of the projection units, the storage unit stores a history of designation by the user, and the allocation unit performs designation by the user to the projection units. If not, an image that is highly likely to be designated to the projection unit from the history stored in the storage unit may be assigned to the projection unit.

Further, the present invention provides any one of the information processing devices described above, a receiving unit that receives a video distributed from the video distribution device according to a projection instruction from the information processing device, and a receiving unit that receives the video from the video distribution device. Provided is a video projection system including a projection device including a plurality of projection units for projecting images and a projection control unit for causing the plurality of projection units to project the images received by the reception unit.

According to the present invention, it is possible to effectively utilize a plurality of images shot by a plurality of shooting devices using a projection device.

1 is a diagram showing an example of the overall configuration of a video projection system according to an embodiment; FIG. FIG. 3 is a diagram showing an example of hardware configuration of a video distribution device and a video allocation control device; A diagram showing an example of the hardware configuration of a projector. Diagram representing the appearance of the projector A diagram showing an example of a projection area where a projector projects an image. Diagram showing the functional configuration realized by each device Diagram showing an example of projection unit information A diagram showing an example of the arrangement of multiple cameras Diagram showing an example of shooting information A diagram showing an example of video allocated using the first allocation method. A diagram showing an example of video allocated using the second allocation method. A diagram showing an example of positional relationship information Diagram showing an example of allocation information A diagram showing an example of the operation procedure of each device in allocation processing and projection processing. A diagram showing a functional configuration realized in a modified example A diagram showing an example of the arrangement of a plurality of cameras in a modified example. A diagram showing a functional configuration realized in a modified example A diagram showing a functional configuration realized in a modified example A diagram showing a functional configuration realized in a modified example A diagram showing a functional configuration realized in a modified example

[1] Examples

FIG. 1 shows an example of the overall configuration of a video projection system 1 according to an embodiment. The image projection system 1 is a system for simultaneously projecting a plurality of images captured by a plurality of cameras and allowing users to enjoy many scenes at the same time. The video projection system 1 includes a network 2, a projector 10, a video distribution device 30, a plurality of cameras 3 (cameras 3-1, 3-2, . . . ), and a video allocation control device .

A network 2 is a communication system including a mobile communication network, the Internet, etc., and relays data exchange between devices accessing the system. The network 2 is accessed by the projector 10, the video distribution device 30, and the video allocation control device 40 through wired communication (or wireless communication).

The projector 10 includes a plurality of projection units that each project an image, and each projection unit projects an image onto a projection area such as a screen. The projector 10 is an example of the "projection device" of the present invention. A projection area is an area where a projected image is displayed. When the projector 10 is used in a room at home, the plurality of projection areas may all be prepared as screens, but since this is usually difficult, walls and ceilings may be used as projection areas.

The video distribution device 30 is a device that provides a program whose content is video, and distributes a plurality of videos captured by a plurality of cameras 3 . The video distribution device 30 may distribute one of a plurality of videos, but can also distribute two or more videos at the same time. Note that the video distribution device 30 is an example of a device that distributes video, and there are other such devices.

The plurality of cameras 3 are installed at different locations, respectively, to photograph the same object at different angles, or to photograph different objects separately. There are various things to be photographed. For example, in the case of a sports program such as soccer and baseball, which is played in a stadium, the players (one or more), the stadium (partially or entirely), the bench, the director, the spectators, etc. are photographed. In the case of a golf program, the tee ground of each hole, the point of the second shot, the green, the practice range, the interview room, the player of interest, and the like are photographed.

Also, in the case of sports programs such as marathons and bicycle road races that take place along the road, athletes (leading group, chasing group, etc.), specific points (start, finish, etc.), spectators, etc. are photographed. In addition to sports, for example, in the case of a music program, singers, performers, stages and spectators are photographed. In this embodiment, a case will be described in which a plurality of cameras 3 are installed at various locations in a soccer stadium, and a video distribution device 30 provides a soccer match program.

The image allocation control device 40 performs allocation processing for allocating a plurality of images captured by a plurality of cameras 3 to each projection unit provided in the projector 10 . The video allocation control device 40 is an example of the "information processing device" of the present invention. Projector 10 projects a plurality of assigned images at the same time, so that the user can view a plurality of scenes shown by the plurality of images captured in the program (for example, in the case of a soccer program, the player playing, the central player, the entire court, and Bench) can be viewed at the same time.

FIG. 2 shows an example of the hardware configuration of the video distribution device 30 and the video allocation control device 40. As shown in FIG. The video distribution device 30 and the video allocation control device 40 are physically computers including a processor 31, a memory 32, a storage 33, a communication device 34, an input device 35, an output device 36, a bus 37, and the like. It may be configured as a device. Note that in the following description, the term "apparatus" can be read as a circuit, device, unit, or the like.

Also, one or more of each device may be included, or some devices may not be included. The processor 31, for example, operates an operating system to control the entire computer. The processor 31 may be configured by a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic device, registers, and the like.

For example, the baseband signal processing section and the like may be implemented by the processor 31 . The processor 31 also reads programs (program codes), software modules, data, etc. from at least one of the storage 33 and the communication device 34 to the memory 32, and executes various processes according to these. As the program, a program that causes a computer to execute at least part of the operations described in the above embodiments is used.

Although it has been explained that the various processes described above are executed by one processor 31, they may be executed by two or more processors 31 simultaneously or sequentially. Processor 31 may be implemented by one or more chips. Note that the program may be transmitted from a network via an electric communication line. Memory 32 is a computer-readable recording medium.

The memory 32 may be composed of at least one of, for example, ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), and the like. The memory 32 may also be called a register, cache, main memory (main storage device), or the like. The memory 32 can store executable programs (program codes), software modules, etc. for implementing the wireless communication method according to an embodiment of the present disclosure.

The storage 33 is a computer-readable recording medium, for example, an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disc, a magneto-optical disc (e.g., a compact disc, a digital versatile disc, a Blu-ray disk), smart card, flash memory (eg, card, stick, key drive), floppy disk, magnetic strip, and/or the like.

The storage 33 may also be called an auxiliary storage device. The storage medium described above may be, for example, a database, server, or other suitable medium including at least one of memory 32 and storage 33 . The communication device 34 is hardware (transmitting/receiving device) for communicating between computers via at least one of a wired network and a wireless network, and is also called a network device, a network controller, a network card, a communication module, or the like.

The communication device 34 includes a high-frequency switch, a duplexer, a filter, a frequency synthesizer, etc., for realizing at least one of frequency division duplex (FDD) and time division duplex (TDD). may consist of For example, the above-described transmitting/receiving antenna, amplifier section, transmitting/receiving section, transmission path interface, etc. may be implemented by the communication device 34 . The transceiver may be physically or logically separate implementations for the transmitter and receiver.

The input device 35 is an input device (for example, keyboard, mouse, microphone, switch, button, sensor, etc.) that receives input from the outside. The output device 36 is an output device (for example, display, speaker, LED lamp, etc.) that outputs to the outside. Note that the input device 35 and the output device 36 may be integrated (for example, a touch panel). Each device such as the processor 31 and the memory 32 is connected by a bus 37 for communicating information. The bus 37 may be configured using a single bus, or may be configured using different buses between devices.

In addition, the video distribution device 30 and the video allocation control device 40 include a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). It may be configured including hardware such as Also, the video distribution device 30 and the video allocation control device 40 may be realized by the hardware in part or all of the respective functional blocks. For example, processor 31 may be implemented using at least one of these pieces of hardware.

FIG. 3 shows an example of the hardware configuration of the projector 10. As shown in FIG. The projector 10 is a computer device that physically includes a processor 11, a memory 12, a storage 13, a communication device 14, an input device 15, an output device 16, a bus 17, an image projection device 20, and the like. may be configured. Of these, the hardware with the same name shown in FIG. 2 is hardware of the same type, although there are differences in performance, specifications, and the like.

It should be noted that the input device 15 and the output device 16 may be provided not only by being built into the main body, but also by a remote controller. The image projection device 20 is a device that includes components such as a light source and an optical system (lens, mirror, etc.) and projects an image indicated by image data supplied from an external device. The image projection device 20 may project an image by any method such as a liquid crystal method, a reflective liquid crystal method, or a DLP (registered trademark) (Digital Light Processing) method.

The image projection device 20 includes a front projection section 21 , an upper projection section 22 , a right projection section 23 and a left projection section 24 . These plurality of projection units project images to the front, upper, right, and left sides of the image projection device 20, respectively.
FIG. 4 shows the appearance of the projector 10. As shown in FIG. 4A shows a plan view of the projector 10, and FIG. 4B shows a right side view of the projector 10. As shown in FIG.

4(c) shows a front view of the projector 10, and FIG. 4(d) shows a left side view of the projector 10. As shown in FIG. FIG. 4 shows a rectangular parallelepiped housing 25, a front projection unit 21 with a lens facing the front side of the housing 25, an upward projection unit 22 with a lens facing the upper side of the housing 25, A right projection unit 23 with a lens facing the right side of the housing 25 and a left projection unit 24 with a lens facing the left side of the housing 25 are shown.

FIG. 5 shows an example of a projection area onto which the projector 10 projects an image. In the example of FIG. 5, the projector 10 is installed in a rectangular parallelepiped room surrounded by walls D1, D3, D4 and a ceiling D2. In the projector 10, the front projection section 21 projects an image on a portion of the wall D1 as a projection area C1, and the upward projection section 22 projects an image on a portion of the ceiling D2 as a projection area C2. In the projector 10, the right side projection section 23 projects an image using a portion of the wall D3 as a projection area C2, and the left side projection section 24 projects an image using a portion of the wall D4 as a projection area C4.

Each function of each device of the projector 10, the video distribution device 30, and the video allocation control device 40 is performed by the processor by loading predetermined software (program) on hardware such as each processor and memory. It is realized by controlling communication by each communication device and controlling at least one of reading and writing of data in memory and storage.

FIG. 6 shows the functional configuration realized by each device. The projector 10 includes a multiple projection unit 101 , a projection control unit 102 , an image reception unit 103 and an image allocation request unit 104 . The video distribution device 30 includes a requested video transmission unit 301 , a multiple video acquisition unit 302 , and a target information storage unit 303 . The video allocation control device 40 includes a target information acquisition unit 401 , a video allocation unit 402 , a video reception unit 403 and an allocation instruction processing unit 404 .

The multiple projection units 101 of the projector 10 are multiple projection units that each project an image, and are realized by the image projection device 20 shown in FIG. The multiple projection unit 101 has a plurality of projection units including a front projection unit 21, an upper projection unit 22, a right projection unit 23, and a left projection unit 24. As described with reference to FIG. and the projection area on the left side. A projection control unit 102 controls the projected images of the multiple projection unit 101 .

The projection control unit 102 projects the image assigned to each projection unit by the image assignment control device 40 as described above (the details of the assignment method will be described later). The projection control unit 102 transmits distribution request data requesting distribution of the assigned video to the video distribution device 30 . The video to be requested is specified by, for example, the camera ID of the camera that shoots the video. The video distribution device 30 supplies the received distribution request data to the requested video transmission unit 301 .

When the distribution request data is supplied, the requested video transmission unit 301 requests the multiple video acquisition unit 302 for the video indicated by the supplied distribution request data, that is, the video requested to be distributed. A multiple image acquisition unit 302 acquires images captured by a plurality of cameras 3 in real time. The multiple video acquisition unit 302 supplies the requested video among the acquired multiple videos to the requested video transmission unit 301 .

The requested video transmission unit 301 transmits the supplied video to the requesting device (the projector 10 in this case). The image receiving unit 103 of the projector 10 receives the transmitted image, that is, the image distributed by the image distribution device 30 . The video receiving unit 103 is an example of the "receiving unit" of the present invention. The video reception unit 103 supplies the received video to the projection control unit 102 .

The projection control unit 102 causes the multiple projection unit 101 to project the supplied image, that is, the image received by the image receiving unit 103 . The projection control unit 102 is an example of the "projection control unit" of the present invention. The above is the description of the basic functions related to projection of a plurality of images by the multiple projection unit 101 . Next, functions related to video allocation will be described.

The image allocation request unit 104 of the projector 10 requests the image allocation control device 40 to allocate images to the plurality of projection units provided in the projector 10 . The image allocation requesting unit 104 displays, for example, a program list on the display of the output device 16 or projects it on the image projection device 20, and requests allocation of images of programs selected by the user from the list. The video allocation requesting unit 104 transmits allocation request data indicating the selected program and projection unit information regarding a plurality of projection units to the video allocation control device 40 .

FIG. 7 shows an example of projection unit information. In FIG. 7, projection unit IDs "F21", "F22", "F23", and "F24" are associated with projection directions "front", "upper", "right", and "left", respectively. part information is shown. For example, "F21" is the identification information of the front projection unit 21 shown in FIG. 4, and indicates that the "front" is the projection direction. The video allocation control device 40 supplies the received allocation request data to the target information acquisition unit 401 .

The target information acquisition unit 401 acquires target information indicating a plurality of shooting targets shot by a plurality of shooting devices (a plurality of cameras 3 in this embodiment) in the program indicated by the supplied allocation request data. The target information acquisition unit 401 is an example of the "target acquisition unit" of the present invention. The target information acquisition unit 401 transmits information request data indicating a request for the selected program and target information to the video distribution apparatus 30 .

The video distribution device 30 supplies the received information request data to the target information storage unit 303 . The target information storage unit 303 stores target information of a shooting target to be shot in a program distributed by its own device. In this embodiment, the target information storage unit 303 stores target information of shooting targets to be shot in a soccer program. An object to be photographed in this case will be described based on the arrangement of the plurality of cameras 3. FIG.

FIG. 8 shows an example of the arrangement of a plurality of cameras 3. As shown in FIG. FIG. 8 shows the arrangement of 13 cameras 3 from cameras 3-1 to 3-13. In FIG. 8, the soccer court is divided into A court and B court along the center line, and divided into 12 areas from A1 to A6 for A court and from B1 to B6 for B court. In addition, in FIG. 8, in order to show the angles of the camera, the direction from the main stand to the back stand is "0 degree", and clockwise 90 degrees, 180 degrees, and 270 degrees.

For example, the camera 3-1 is arranged near the corner flag and shoots a player in the A1 area in the direction of 120±30 degrees. The camera 3-2 is placed behind the goal of court A, and shoots players (mainly shooting scenes in front of the goal) in the A2 area in the direction of 90±30 degrees. Also, the camera 3-3 is fixed to the goal post of Court A, and photographs the goal area as an object to be photographed.

In addition, for example, a camera 3-5 installed in the middle of the stands on the A court side shoots a specific player (popular player, etc.) as an object to be photographed. In addition, cameras 3-6, 3-7, and 3-8 are installed on the upper level of the main stand. B1 to B6), and the camera 3-7 targets the center of the court (A4 to A6, B4 to B6). The target information storage unit 303 stores shooting information indicating the shooting target determined as described above.

FIG. 9 shows an example of shooting information. In FIG. 9, the target information storage unit 303 stores shooting information in which a camera ID (the symbols shown in FIG. 8 are used as they are), a shooting target, and a shooting direction are associated with each other. For example, the camera ID “3-1” is associated with the shooting target “A1” and the shooting direction “120±30 degrees”, and the camera ID “3-2” is associated with the shooting target “A2” and the shooting direction “90±30 degrees”. degree” is associated with it.

Further, the camera ID "3-5" is associated with the shooting target "athlete α" and the shooting direction "0±90 degrees". This means that the cameras 3-5 follow and photograph the player α wherever he is. When the information request data is supplied, the target information storage unit 303 reads the stored target information and transmits it to the video allocation control device 40 which is the source of the request. The video allocation control device 40 supplies the received target information to the target information acquisition unit 401 .

After acquiring the target information in this manner, the target information acquiring unit 401 supplies the acquired target information to the video allocation unit 402 together with the projection unit information. The image allocation unit 402 allocates images of two or more shooting targets among the plurality of shooting targets indicated by the target information acquired by the target information acquisition unit 401 to a projection device (projector 10 in this embodiment) having a plurality of projection units. assigned to each projection of the The video allocation unit 402 is an example of the "allocation unit" of the present invention.

Since the projector 10 has four shooting units as indicated by the projection unit information, the image assigning unit 402 selects four from the 13 shooting targets indicated by the target information and assigns them to each shooting unit. In this embodiment, the video allocation unit 402 performs allocation using either the first allocation method or the second allocation method depending on the situation. First, the first allocation method will be described below.

The image allocation unit 402 uses the first allocation method when a plurality of projection units, like the multiple projection unit 101, project images onto at least the front and left and right projection areas of the projection device (projector 10 in this embodiment). Specifically, the image allocation unit 402 allocates the images so that the positional relationship of the projection areas corresponds to the positional relationship in the real space of the plurality of shooting targets.

FIG. 10 shows an example of images allocated using the first allocation method. In FIG. 10, the image allocation unit 402 allocates the image of the camera 3-7 to the front projection area C1, and allocates the image of the camera 3-5 to the upper projection area C2. The image allocation unit 402 also allocates the image of the camera 3-12 to the projection area C3 on the right side, and the image of the camera 3-1 to the projection area C4 on the left side.

In the example of FIG. 10, with respect to the center of the court (A4 to A6, B4 to B6) reflected in the front projection area C1, the area B2 projected in the projection area C3 on the right side is located on the right side even in the real space, The area A1 reflected in the projection area C4 is also located on the left side in the actual space. In other words, the positional relationship of the projection areas onto which the images of the shooting targets are projected corresponds to the positional relationship in the actual space where the shooting targets exist.

It should be noted that a specific player to be photographed by the camera 3-5 can move anywhere on the court, so that the positional relationship with other photographing subjects changes. Therefore, in this embodiment, the image allocation unit 402 allocates the image of the camera 3-5 to the upper projection area C2, which is not affected by the positional relationship with respect to the front and left and right directions.

For example, the user himself/herself selects all four images shown in FIG. This selection may be made at the same time when the program is selected, or may be made by opening the menu screen while viewing the video. For example, assume that the user has selected the images of cameras 3-2, 3-6 and 3-7. The objects to be photographed by these cameras are area A2, the entire court A, and the center of the court. For example, area A2 and court A overlap.

Since the center position of the area A2 is on the left side of the entire A court, the image allocation unit 402 allocates the image of the camera 3-2 to the projection area on the left side of the image of the camera 3-6. In addition, since the position of the center of the entire A court is to the left of the center of the court, the image allocation unit 402 allocates the image of the camera 3-6 to the projection area on the left side of the image of the camera 3-7. In summary, the image allocation unit 402 allocates the image of the camera 3-2 to the left projection area C4, the image of the camera 3-6 to the front projection area C1, and the image of the camera 3-7 to the right projection area C3. assign.

For example, when a video of golf is distributed, allocation is performed as follows. The image allocation unit 402 allocates the image of the player hitting the shot from behind to the front projection area C1, and allocates the image of the flying ball to the upper projection area C2. The image allocation unit 402 also allocates the sliced image of the landing point of the ball to the right projection area C3, and allocates the image of the landing point of the hooked ball to the left projection area C4.

Next, the second allocation method will be explained. The image allocation unit 402 uses the second allocation method, for example, when the user designates a reference image (hereinafter referred to as a “reference image”). Specifically, the image allocation unit 402 determines that the positional relationship between the object to be photographed in the reference image and the object to be photographed in the other image is the positional relationship between the projection area for projecting the reference image and the projection area for projecting the other image. Two or more images to be photographed are assigned so that

This second allocation method is typically used when the image of camera 3-5 is designated as the reference image. This reference video is a video of a specific player whose positional relationship with other shooting targets changes as he moves on the court.
FIG. 11 shows an example of images allocated using the second allocation method. In FIG. 11(a), it is assumed that a specific player whose reference image is to be photographed is in area B5.

The video allocation unit 402 determines the area where a specific player is located, for example, from the video. The image allocation unit 402 requests the image of the entire court from the image reception unit 403 . The video receiving unit 403 transmits video request data requesting the requested video of the entire court to the video distribution device 30 . The video distribution device 30 supplies the received video request data to the requested video transmission unit 301 . The requested image transmission unit 301 transmits to the projector 10 an image of the entire court indicated by the supplied image request data (for example, images captured by the cameras 3-6, 3-7, and 3-8).

The video reception unit 403 receives the transmitted video and supplies it to the video allocation unit 402 . The video allocation unit 402 determines the area where the specific player is located from the supplied video. The video allocation unit 402 recognizes the position of each player in the video of the entire court using, for example, a known pedestrian recognition technology. Then, the image allocation unit 402 recognizes the face of a specific player using a known face recognition technology or recognizes the jersey number of a specific player using a known character recognition technology, thereby determining the position of each player. Determine the position of a particular player from within.

Also, the image allocation unit 402 recognizes the positions in the image of the center line, side lines, center circle, penalty area, goal area, etc. from the shape of the white line appearing in the image of the entire court. The image allocation unit 402 determines the area in which the specific player is located from the relationship between the recognized position of the center line or the like in the image and the position of the specific player in the image. For example, if a specific player is positioned near the center circle on the left side of the center line, the image allocation unit 402 determines that the player is in area A5.

Further, the image allocation unit 402 determines that the player is in area B2 when the specific player is positioned in the penalty area on the right side of the center line. The video allocation unit 402 determines that a specific player is in area B5 in the example of FIG. 11(a) by the method described above. The image allocation unit 402 uses the positional relationship information representing the positional relationship in the real space of the shooting target of each camera in order to determine the allocation of the image of the specific player, that is, the reference image.

FIG. 12 shows an example of positional relationship information. In FIG. 12, camera IDs "3-1", "3-2", and "3-4" are arranged vertically on the left end, and "3-3" and "3-6" are arranged to the left of "3-2". , “3-7”, “3-8”, “3-12”, and “3-11” are arranged in order, with “3-13” and “3-11” above and below “3-11”. 10” is displayed as positional relationship information in which the camera IDs are arranged.

In the case of the example of FIG. 11(a), the area B5 where a specific player is located is the shooting target of the camera 3-7 (areas A4-A6, B4-B6) and the shooting target of the camera 3-8 (areas B1-B6). included. In the positional relationship information, cameras are also adjacent to the left and right of these cameras. Therefore, the image allocation unit 402 first allocates the reference image to the front projection area C1. Then, the image allocation unit 402 allocates the image of the camera 3-12 adjacent to the right of the camera 3-8 on the right to the projection area C3 on the right, and the image of the camera 3-12 adjacent to the left of the camera 3-7 on the left is allocated to the projection area C3 on the right. 6 is assigned to the left projection area C4.

In FIG. 11(b), it is assumed that a specific player has moved to area A2. The area A2 is included in the shooting targets (A1 to A6) of the camera 3-6 and the shooting target (A2) of the camera 3-2. In the positional relationship information, cameras are also adjacent to the left and right of these cameras. Therefore, the image allocation unit 402 allocates the reference image to the front projection area C1, as in the example of FIG. 11(a).

Then, the image allocation unit 402 allocates the image of the camera 3-7 adjacent to the right of the camera 3-6 on the right to the projection area C3 on the right, and the image of the camera 3-7 adjacent to the left of the camera 3-2 on the left is allocated to the projection area C3 on the right. 3 (shooting target is the goal area) is allocated to the left projection area C4. Note that when the position of a specific player is close to the sideline on the backstand side, the image allocation unit 402 assigns the image of the camera 3-1, which captures the backstand side of the camera 3-3, to the projection area C4 on the left side. assign.

Further, if the position of the specific player is close to the sideline on the main stand side, the image allocation unit 402 allocates the image of the camera 3-4 to the projection area C4. Also, when a specific player has moved to the goal area, there is no camera on the left side of the camera 3-3 that shoots the goal area. Therefore, the image allocation unit 402 allocates a reference image in which a specific player is shown to the projection area C4 on the left side.

Then, the image allocation unit 402 allocates the image of the camera 3-2 adjacent to the right of the camera 3-3 to the front projection area C1, and the image of the camera 3-6 adjacent to the right of the camera 3-2 to the right projection area C1. Allocate to projection area C3. Note that the reference image is not limited to an image of a shooting target whose positional relationship with other shooting targets changes, such as a specific athlete. For example, if the image of the camera 3-7 is specified as the reference image, the image allocation unit 402 sequentially allocates the images of the cameras 3-6, 3-7, and 3-8 from the left projection area (the user selects the reference image). This assignment will not change unless you change it.)

After performing the allocation as described above, the video allocation unit 402 supplies allocation information indicating the allocation result to the allocation instruction processing unit 404 . The image allocation unit 402, for example, assigns information in which the projection unit ID for identifying each projection unit shown in FIG. to the allocation instruction processing unit 404.

FIG. 13 shows an example of allocation information. FIG. 13 shows the allocation result described in FIG. That is, the camera 3-7 is associated with the front projection unit 21 (projection unit ID: F21), and the camera 3-5 is associated with the upper projection unit 22 (projection unit ID: F22). The camera 3-12 is associated with the right projection unit 23 (projection unit ID: F23), and the camera 3-1 is associated with the left projection unit 24 (projection unit ID: F24).

The allocation instruction processing unit 404 gives instructions for projecting images according to allocation by the image allocation unit 402 . Allocation instruction processing unit 404 is an example of the "instruction unit" of the present invention. The allocation instruction processing unit 404 requests the video distribution device 30 for two or more images indicated by the supplied allocation information, and provides instruction data indicating an instruction to perform control to project the images on the projection unit indicated by the allocation information. to the projector 10 .

The projector 10 supplies the transmitted instruction data to the projection control unit 102 . The projection control unit 102 transmits distribution request data requesting distribution of two or more images indicated by the supplied instruction data to the video distribution device 30 . The image receiving unit 103 receives images distributed in response to this request, that is, two or more images distributed according to the projection instruction from the image allocation control device 40 . The projection control unit 102 causes each projection unit to project the received two or more images according to the allocation indicated by the instruction data.

As a result of the above instruction, the images assigned as shown in FIG. 10 are displayed in each projection area. It should be noted that depending on how the projector 10 is installed, the projected image may be distorted and the shape of the originally rectangular image may become a trapezoid. In that case, each projection unit included in the multiple projection unit 101 may correct image distortion using, for example, a known trapezoidal correction technique, or may similarly use a known lens shift function technique to correct image distortion. can be corrected.

Each device included in the video projection system 1 performs the allocation process described above and the projection process of projecting the allocated video based on the above configuration.
FIG. 14 shows an example of the operation procedure of each device in allocation processing and projection processing. This operation procedure is started, for example, when the user installs the projector 10 in a room and performs an operation to select a program.

First, the projector 10 (video allocation requesting unit 104) accepts a program selection operation by the user (step S11), and transmits allocation request data indicating the selected program and projection unit information to the video allocation control device 40 (step S11). S12). The video allocation control device 40 (target information acquisition unit 401) transmits information request data indicating a request for the selected program and target information to the video distribution device 30 (step S13).

The video distribution device 30 (target information storage unit 303) reads the requested target information and transmits it to the video allocation control device 40 (step S14). The video allocation control device 40 (target information acquisition unit 401) acquires the transmitted target information (step S15). Then, the image allocation control device 40 (image allocation unit 402) allocates images of two or more shooting targets among the plurality of shooting targets indicated by the acquired target information to each projection unit of the projector 10 (step S16). The above is the operation procedure in the allocation process.

Subsequently, the video allocation control device 40 (allocation instruction processing unit 404) issues an instruction to project the video according to the allocation in the allocation process. Specifically, the video allocation control device 40 (allocation instruction processing unit 404) requests the video distribution device 30 for two or more allocated videos, and instructs the projection unit to project the videos. to the projector 10 (step S21).

The projector 10 (projection control unit 102) transmits distribution request data requesting distribution of two or more images indicated by the transmitted instruction data to the video distribution device 30 (step S22). The video distribution device 30 (requested video transmission unit 301) transmits two or more videos indicated by the transmitted distribution request data to the projector 10 (step S23). The projector 10 (multiple projection unit 101, projection control unit 102, image reception unit 103) projects the transmitted two or more images using the multiple projection units to which they are assigned (step S24). The above is the operation procedure in the projection process.

In this embodiment, the image allocation control device 40 allocates two or more images among the plurality of images captured by the plurality of cameras 3 as described above to the plurality of projection units. Then, the projector 10 having the plurality of projection units projects the assigned two or more images. By assigning and projecting two or more images, a plurality of images captured by a plurality of cameras 3 can be effectively utilized using the projector 10 .

Also, in this embodiment, as described with reference to FIG. 10, the positional relationship of each projection area onto which two or more images are projected is the positional relationship corresponding to the positional relationship in the real space of the shooting target of these images. Allocation is made so that According to the present embodiment, the user viewing the projected image can more easily grasp the actual positional relationship of the photographed objects appearing in the image, compared to the case where this allocation is not performed.

Also, in this embodiment, as described with reference to FIG. 11, the positional relationship in the real space between the object to be photographed in the reference image and the object to be photographed in another image is different from the projection area onto which the reference image is projected. are assigned according to the positional relationship of the projection areas for projecting the images. According to the present embodiment, compared to the case where this allocation is not performed, even if the position in the real space of the shooting target shown in the reference image changes, the user who viewed the projected image can be seen in the image. It is possible to easily grasp the actual positional relationship of the object to be photographed.

[2] Modifications The embodiment described above is merely an example of implementation of the present invention, and may be modified as follows.

[2-1] Third Allocation Method: Degree of Excitement The allocation method used by the video allocation unit 402 is not limited to the first and second allocation methods. In this modified example, the third allocation method will be described. The video allocation unit 402 uses the third allocation method when a plurality of cameras 3 shoot an event as a shooting target. An event is an event that attracts people's attention, and includes sports matches such as soccer, music concerts, lectures, and the like.

In the following, a case will be described in which an event called a bicycle road race held along a road is being photographed.
FIG. 15 shows the functional configuration realized in this modified example. FIG. 15 shows a video allocation control device 40a that includes an excitement degree determination unit 405 in addition to the units shown in FIG. The degree of excitement determination unit 405 determines the degree of excitement of the event from the images captured by the plurality of cameras 3 of the event (a soccer match in this embodiment). The excitement level determination unit 405 is an example of the "climax level determination unit" of the present invention.

The degree of excitement means the degree to which spectators watching the event are excited. The spectators here include not only the spectators directly watching the event on site but also the spectators watching the event through video. Therefore, for example, in a bicycle road race, there are places where there are no spectators, such as a long downhill slope, but the bike cameras are chasing the spectators and the images are being taken, so even such places can become lively scenes.

The excitement degree determination unit 405 determines the excitement degree from, for example, the video of the event. The excitement level determination unit 405 acquires the video of the event via the video reception unit 403 using the same method as the video allocation unit 402 of the second allocation method.
FIG. 16 shows an example of arrangement of a plurality of cameras 3 in this modified example. In FIG. 16, an intermediate sprint point and a summit point are set between the start point and the goal point of a bicycle road race, and a state in which an escape group, a chase group, a main group, and a rear group are formed is shown. .

In the example of FIG. 16, seven cameras 3-21 to 3-27 photograph the rear group, the main group, the chase group, the escape group, the intermediate sprint point, the summit point, and the goal point, respectively. The excitement degree determination unit 405 acquires the images captured by these cameras 3, and calculates, for example, the average value of the volume of the voices (cheers) included in the images. determined to be high.

In addition, the excitement level determination unit 405 recognizes, for example, the shapes of the person and the bicycle appearing in the acquired video and their positions in the image using a known pattern recognition technique. Then, when the recognized person or bicycle makes a specific movement (passing, running out, gesturing, falling, etc.), the excitement degree determination unit 405 adds points according to each movement, and the greater the point, the more exciting the movement. It may be determined that the degree is large. In addition, the excitement level determination unit 405 may also add points based on the loudness of the voice to these points.

The excitement degree determination unit 405 supplies the determined excitement degree to the video allocation unit 402 . When priority is set for a plurality of projection units provided in the projector 10, the image allocation unit 402 assigns a higher priority to the image of the shooting location determined by the excitement level determination unit 405 to have a higher priority. Allocate to department. This priority may be predetermined in the projector 10, for example, or may be set by the user.

In this modification, the front projection part 21 has the highest priority, the right projection part 23 is second, the left projection part 24 is third, and the upward projection part 22 is fourth. For example, when the magnitude of excitement determined at the beginning of the race is in the order of escape group>main group>chasing group, the video allocation unit 402 selects the camera 3 that captures the escape group with the highest excitement. -24 images are assigned to the front projection area C1 with the highest priority.

In addition, the image allocation unit 402 allocates the image of the camera 3-22, which projects the main group with the second highest degree of excitement, to the projection area C3 on the right side, and the camera 3, which projects the chasing group with the third highest degree of excitement. -23 image is assigned to the left projection area C4. Then, as the race progresses and the main group approaches the summit point, it is assumed that the magnitude of the determined degree of excitement is in the order of the main group>the vicinity of the summit>the escape group.

In this case, the image allocation unit 402 divides the image of the camera 3-22 that projects the main group, the image of the camera 3-26 that projects the vicinity of the mountaintop, and the image of the camera 3-24 that projects the escape group into the front projection area C1 and the right side projection area C1, respectively. are assigned to the projection area C3 on the left side and the projection area C4 on the left side. In this modified example, as described above, images are allocated according to the degree of excitement, so that when the exciting scene changes in the event, the user does not miss the exciting scene. can be made

[2-2] Subjective video and objective video Depending on the event, the subject of the event (e.g. athletes, singers, musicians, lecturers, bicycles and cars driven by athletes, etc.) may be captured from the surroundings. objective video) and subjective video (subjective video) captured by a camera attached to the subject. For example, if the event is a bicycle road race, the subjective video is a video captured by a camera attached to the bicycle of the athlete.

If the event is a motor sport, the video is captured by a camera attached to a machine driven by a player. In this modification, the excitement degree determination unit 405 shown in FIG. 15 determines the excitement degree of the event in which the subjective video and the objective video are captured. Then, when the excitement degree determined by the excitement degree determination unit 405 is greater than or equal to a predetermined reference, the image allocation unit 402 allocates the subjective image to the projection unit that projected the objective image.

A case of motor sports will be described as an example. In this example, the video allocation unit 402 determines the average value of the excitement levels of the individual videos as the excitement level of the entire event, and determines whether or not the excitement level of the entire event is equal to or higher than a predetermined standard. do. The image allocation unit 402 allocates, for example, the image specified by the user as it is when the excitement level of the entire event is less than a predetermined standard.

Then, when the video allocation unit 402 judges that the excitement level of the entire event has exceeded a predetermined standard, and if there is a projection region to which the objective video has been allocated until then, the video allocation unit 402 allocates the subjective video to that projection region. The image allocation unit 402 allocates the image (subjective image) of the on-vehicle camera to the projection area where the image (objective image) of the entire circuit was projected, for example. A subjective image is an image that is more likely to enhance the sense of presence than an objective image.

In this modified example, when the event as a whole becomes lively and the user's mood is uplifted as described above, the objective image is switched to the subjective image to enhance the sense of realism and further uplift the user's mood. Although the image allocation unit 402 determines the excitement of the event as a whole in the above example, it may determine the excitement of each image. In this case, the image assigning unit 402 preferably assigns the subjective image captured by the camera attached to the subject appearing in the objective image to the projection unit that was projecting the objective image.

When performing this assignment, the location information of the on-board camera that shoots the main video (the position of the car equipped with the on-board camera) can be obtained from the on-board camera or the GPS (Global Positioning System) attached to the car. . For example, in a state in which an objective image projecting the final corner is assigned to the front projection area C1, if the degree of excitement determined from the objective image exceeds a predetermined standard, the image assigning unit 402 assigns Instead, the subjective image captured by the vehicle-mounted camera of the car running at the final corner is assigned to the projection area C1. This allocation can also make the user feel better.

[2-3] Wide-range image and local image The image includes a wide-range image that includes a specific shooting target and is wider than a predetermined range, and a wide-range image that includes a specific shooting target and is within a predetermined range. may include localized images. In the case of a car race, for example, a video containing a predetermined number or more of cars is defined as a wide range video, and a video containing less than a predetermined number of cars is defined as a local video.

In the case of a bicycle road race, for example, images taken from an airship are defined as wide-range images, and images taken from a motorcycle are defined as local images. In the case of a soccer match, for example, the image from the upper stage of the stands is defined as the wide-range image, and the image from the lower stage of the stands is defined as the local image. In the case of a concert, for example, the image from behind the audience seats is defined as the wide-range image, and the image from the front of the audience seats is defined as the local image.

However, it may be determined that a local image is obtained even from the upper stage of the stand and the back of the audience seats when the image is zoomed and enlarged, for example, when the size of a person becomes equal to or larger than a predetermined size. These wide-range images and local images may be switched according to the situation.
FIG. 17 shows the functional configuration realized in this modified example. FIG. 17 shows a video allocation control device 40b including a speed specifying unit 406 in addition to the units shown in FIG.

A speed identification unit 406 identifies the speed of a specific object to be photographed. The speed identification unit 406 is an example of the "speed identification unit" of the present invention. If the specific shooting target is a car or a bicycle, for example, the running speed may be measured by the organizer of the competition. Determine the speed of Alternatively, the speed identification unit 406 may identify the speed of a specific shooting target from the video.

In that case, the speed identification unit 406 acquires the video via the video reception unit 403 using the same method as the video allocation unit 402 of the second allocation method. In a soccer match, if the subject to be photographed is a ball, the speed identifying unit 406 recognizes the ball and the white line appearing in the image using, for example, a well-known pattern recognition technique. The speed identification unit 406 identifies the position of the ball from the recognized positional relationship with the white line, and identifies a faster speed as the position changes faster.

The speed specifying unit 406 notifies the video allocation unit 402 of the specified speed of the specific shooting target. The image allocation unit 402 first identifies wide-range images and local images in which a specific shooting target is captured, among the plurality of images. When the wide-range image and the local image are determined by the camera that shoots the wide-range image or the local image, the image allocation unit 402 selects the wide-range image or the local image based on the camera ID of the camera that shot the image in which the specific shooting target is recognized using the pattern recognition technology. Identify any of the local images.

In addition, in the case of a camera image whose captured range changes, the image allocation unit 402 provides information indicating the recognized range using pattern recognition technology (the number of cars for car races, the number of people for soccer games and concerts, etc.). size) to identify whether it is a wide-range image or a local image. Then, when a speed equal to or greater than a threshold value is identified for a specific shooting target appearing in the local image, the image allocation unit 402 allocates the wide-range image to the projection unit that projected the local image.

In addition, when a speed less than a threshold value is specified for a specific shooting target in the wide-range image, the image allocation unit 402 allocates the local image to the projection unit that projected the wide-range image. In other words, the image allocation unit 402 switches the allocation between the wide-range image and the local image in which the specific object to be imaged is displayed depending on whether the speed specified for the specific object to be imaged is equal to or greater than the threshold or less than the threshold.

For example, in the case of a car race, the image allocation unit 402 switches the allocation to the local image when the image of the place where the speed decreases, such as a compound corner, is a wide-range image. Also, when the video of a place where the speed increases like the final straight meets the local video, the video allocation unit 402 switches the allocation to the wide range video. For example, in the case of a bicycle road race, the image allocation unit 402 switches images of places where speed decreases like hill climbs to local images, and switches images of places where speed increases like downhills to wide range images.

For example, in the case of a soccer match, the image allocation unit 402 switches the image to the wide-range image when the speed increases, such as a quick attack from a counter, and switches to the local image when the speed decreases due to the stop of the quick attack. . In this modification, by switching between a wide range image and a local image according to the speed of movement of a specific target, when the movement is fast, it is difficult to lose sight of the specific shooting target, and when the movement is slow, the specific shooting target is selected. It allows you to observe the fine movements of the target.

[2-4] Imaging target The imaging target is not limited to those described above. For example, sports (rugby, skiing, air races, darts, etc.) different from the above-mentioned events may be photographed. In addition, events other than the types described above (fireworks displays, flea markets, festivals, parades, etc.) may be photographed.

In addition, even if it is not a so-called event, for example, a plurality of objects to be photographed by a plurality of cameras arranged in each area (attraction, etc.) of a facility where people gather, such as an amusement park, a department store, and a beach, may be a plurality of photographing objects. . Also, for example, multiple ramen shops, multiple restaurants, multiple stations, and other dispersed places visited by people may be multiple imaging targets. Moreover, not only a specific place but also a specific person (a specific player, etc.) or a specific object (a specific car, etc.) may be photographed as described in the embodiment. In the present invention, any object may be imaged as long as the object information can be acquired.

[2-5] Congestion Degree The video allocation unit 402 may determine allocation according to the congestion degree of the shooting location.
FIG. 18 shows the functional configuration realized in this modified example. FIG. 18 shows a video allocation control device 40c including a congestion degree determination unit 407 in addition to the units shown in FIG.

A congestion degree determination unit 407 determines the degree of congestion of people in a place where the image capturing device captures images. The congestion degree determination unit 407 is an example of the "congestion degree determination unit" of the present invention. The congestion degree determination unit 407 determines the congestion degree from the video, for example. In that case, the congestion degree determination unit 407 acquires the video via the video reception unit 403 using the same method as the video allocation unit 402 of the second allocation method. The congestion degree determination unit 407 recognizes the face of the person appearing in the video using, for example, a well-known pattern recognition technique.

The congestion degree determination unit 407 determines the congestion degree based on the size and number of recognized faces. When the degree of congestion is the same, the larger the size of the face, the narrower the area being shot, and the fewer faces are included in the video. Therefore, the congestion level determination unit 407 calculates a value obtained by multiplying the number of faces by a coefficient that increases as the size of the face increases, and calculates the congestion level represented by the magnitude of the calculated value (the larger the value, the more (indicating congestion) is determined. Note that the method of determining the degree of congestion is not limited to this, and other well-known methods may be used.

The congestion degree determination unit 407 supplies the determined congestion degree to the video allocation unit 402 . When priority is assigned to a plurality of projection units, the image allocation unit 402 allocates the image to the projection unit having the priority according to the degree of congestion determined by the congestion degree determination unit 407 for the image. This priority may be predetermined in the projector 10, for example, or may be set by the user.

In this modification, the front projection part 21 has the highest priority, the right projection part 23 is second, the left projection part 24 is third, and the upward projection part 22 is fourth. Suppose that video of each hole of a golf tournament is distributed, and the degree of congestion is in the order of final hole > 1st hole > 17th hole > 9th hole > . For example, the video allocation unit 402 allocates a video with a higher determined congestion degree to a projection unit with a higher priority.

In this case, the image allocation unit 402 allocates the image of the last hole with the highest degree of congestion to the front projection area C1 with the highest priority, and the image of the 1st hole with the second degree of congestion to the right projection area C3. assign. Further, the image allocation unit 402 allocates the image of the 17th hole, which has the third highest congestion degree, to the projection area C4 on the left side, and allocates the image of the 9th hole, which has the fourth congestion degree, to the upper projection area C2.

Once this assignment is made, it will be possible to know which pair are in the spotlight of the local crowd. Note that the video allocation unit 402 may allocate a video with a lower determined congestion degree to a projection unit with a higher priority. In this case, the image allocation unit 402 allocates the projection areas C1, A3, A4, and A2 in order from the least crowded hall. Note that if this assignment is made when, for example, each area in the theme park is the subject of photography, it is possible to know in advance which areas are vacant when actually visiting the site.

[2-6] User's Preference The video allocation unit 402 may determine allocation according to the user's preference.
FIG. 19 shows the functional configuration realized in this modified example. FIG. 19 shows a video allocation control device 40d that includes a preference information acquisition unit 408 in addition to the units shown in FIG.

The preference information acquisition unit 408 acquires preference information indicating user preferences. The preference information acquisition unit 408 is an example of the "preference acquisition unit" of the present invention. For example, if the user has registered his or her preference information in the projector 10 or the image allocation control device 40 in advance, the preference information acquisition unit 408 acquires the registered preference information. In addition, the preference information acquisition unit 408 periodically acquires a usage history from a terminal such as a smartphone used by the user, and acquires the acquired usage history as preference information.

For example, if the usage history includes a history of positions measured by the terminal, the position of the terminal, that is, the places visited by the user represent the user's preferences (like riding on a circuit or a mountain pass, like playing in the water in a pool or the sea, and Mountains and observatories like scenery, etc.). Also, if the usage history includes a schedule, the schedule including the past and the future represents the user's preferences (likes to ride in driving and touring, likes to play in the water in the case of swimming in the sea, and likes movies in the case of watching movies).

The preference information acquisition unit 408 supplies the acquired preference information to the video allocation unit 402 . When priority is assigned to a plurality of projection units, the video allocation unit 402 selects, as shooting targets, locations that are highly likely to be visited by the user based on the user's preference indicated by the preference information acquired by the preference information acquisition unit 408. A video with a higher priority is assigned to a projection unit. This priority is determined, for example, in the same manner as in the above examples.

When an image of each area of an amusement park is distributed, the preference information acquisition unit 408 displays an image of a go-kart stand that is most likely to be visited by a user whose preference information indicating, for example, a preference for rides is acquired, in the front projection area. C1 is assigned, and the image of the roller coaster that is likely to be visited next is assigned to the right projection area C3.

Further, the preference information acquisition unit 408 assigns the image of the waterslide, which is most likely to be visited, to the front projection area C1 for the user whose preference information indicating that he or she likes playing in water is acquired, and rafting, which is likely to be visited next. is assigned to the projection area C3 on the right side. When this assignment is made, it is possible to preferentially view shooting targets that are likely to be of interest to oneself.

[2-7] User Designation History The video allocation unit 402 may determine allocation according to the trend of allocations designated by the user in the past. The projector 10 of this modified example can select a designation mode in which the user designates the images to be allocated to the projection unit, and an automatic allocation mode in which the apparatus side allocates the images without the user's designation.

FIG. 20 shows the functional configuration realized in this modified example. FIG. 20 shows a video allocation control device 40e including a designation history storage unit 409 in addition to the units shown in FIG. The designation history storage unit 409 stores a history of user designation when the projector 10 operates in the designation mode, that is, when the projector 10 projects images designated by the user for each projection unit. The designation history storage unit 409 is an example of the "storage unit" of the present invention.

In this modification, when the video allocation unit 402 supplies the allocation information indicating the allocation specified by the user to the allocation instruction processing unit 404, the allocation information is associated with the user's identification information (for example, user ID). It is also supplied to the designation history storage unit 409. The designation history storage unit 409 stores the supplied allocation information and user ID as a history of user designation. The designation history storage unit 409 stores the designation history of the user when the projector 10 operates in the designated mode.

Then, when the projector 10 operates in the automatic allocation mode, that is, when the user does not designate the projection unit, the image allocation unit 402 selects the projection unit from the designation history stored in the designation history storage unit 409 . Allocate an image that is likely to be designated to the projection unit. When a video of a soccer match is distributed, for example, if the video most frequently specified by the user in the past in the front projection area C1 is a video of the entire court as a shooting target, the video allocation unit 402 is assigned to the front projection area C1.

Also, if the image most often specified by the user in the upper projection area C2 in the past is the image of the player α, the image allocation unit 402 similarly allocates the image of the player α to the upper projection area C2. In this case, when the player α does not participate or the video for the player α is not captured, the video allocation unit 402 selects the video that has been specified by the user the second most in the past in the upper projection area C2. Allocate to the upper projection area C2.

As described above, the video allocation unit 402 determines that the greater the number of past allocations, the higher the possibility of being designated. In addition to this, the video allocation unit 402 may determine that the greater the number of designations in the past fixed period, the higher the possibility of designation. The image allocation unit 402 may also consider the number of times other projection areas have been specified when determining the image to be allocated to the front projection area C1.

Specifically, when judging the possibility that a certain image is designated as the front projection area C1, the image allocation unit 402 determines the number of times that the image is designated as the front projection area C1, and the number of times that the image is designated as the front projection area C1. Sum half the number of times that image was specified in the region. The image allocation unit 402 determines that the greater the total value, the higher the possibility of being designated as the front projection area C1. When the allocation of this modified example is performed, the user can view the video that he or she likely wants to see without specifying the video by himself/herself.

[2-8] Plural Projection Units Although the projector 10 of the embodiment projects images in the four directions of the front, the top, and the left and right, a projection device having a plurality of projection units is not limited to this. The projection device may project images in two directions, three directions, or five or more directions. Also, the projection device may project an image not only in a direction forming an angle of 90 degrees such as front, upward, left and right, but also in an oblique direction. In short, the projection device of the present invention may project images in any direction as long as it has a plurality of projection units.

[2-9] Apparatus for realizing each function The apparatus for realizing each function shown in FIG. 6 and the like is not limited to the above-described apparatus. For example, the video allocation request unit 104 may be realized by providing a screen for selecting a video to be distributed by the video allocation control device 40 on a website or the like. Also, the projector 10 may implement all the functions implemented by the video allocation control device 40 . In that case, the processor 11 of the projector 10 is an example of the "information processing apparatus" of the present invention. In short, it is sufficient that each function shown in FIG.

[2-10] Projection instruction Allocation instruction processing unit 404 transmits instruction data to projector 10 in the embodiment, but is not limited to this and may transmit instruction data to video distribution apparatus 30 . In this case, the image indicated by the instruction data received by the requested image transmission unit 301 of the image distribution device 30 is transmitted to the projector 10, and the transmitted image is projected by each projection unit. Thus, even if the transmission destination of the instruction data is the video distribution device 30 , the instruction for projecting the video is made according to the allocation of the video allocation unit 402 .

[2-11] Category of Invention The present invention can be regarded not only as a projection device such as the projector 10, but also as an information processing method for realizing processing performed by the projection device, and as a computer that controls the projection device. It can also be regarded as a program for functioning. This program may be provided in the form of a recording medium such as an optical disc storing it, or may be provided in the form of being downloaded to a computer via a network such as the Internet and installed and made available. may be

[2-12] Functional Blocks The block diagrams used in the description of the above embodiments show blocks in functional units. These functional blocks (components) are realized by any combination of at least one of hardware and software. Also, the method of implementing each functional block is not particularly limited.

That is, each functional block may be implemented using one device that is physically or logically coupled, or directly or indirectly using two or more devices that are physically or logically separated (e.g. , wired, wireless, etc.) and may be implemented using these multiple devices. A functional block may be implemented by combining software in the one device or the plurality of devices.

Functions include judging, determining, determining, calculating, calculating, processing, deriving, investigating, searching, checking, receiving, transmitting, outputting, accessing, resolving, selecting, choosing, establishing, comparing, assuming, expecting, assuming, Broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping, assigning, etc. can't For example, a functional block (component) that makes transmission work is called a transmitting unit or a transmitter. In either case, as described above, the implementation method is not particularly limited.

[2-13] Handling of input/output information, etc. Input/output information, etc. may be stored in a specific location (for example, memory), or may be managed using a management table. Input/output information and the like can be overwritten, updated, or appended. The output information and the like may be deleted. The entered information and the like may be transmitted to another device.

[2-14] Judgment method Judgment may be made by a value represented by one bit (0 or 1), by a boolean value (Boolean: true or false), or by a numerical value. may be performed by a comparison of (eg, a comparison with a predetermined value).

[2-15] Processing Procedures, etc. The processing procedures, sequences, flowcharts, etc. of each aspect/embodiment described in the present disclosure may be rearranged in order as long as there is no contradiction. For example, the methods described in this disclosure present elements of the various steps using a sample order, and are not limited to the specific order presented.

[2-16] Handling of input/output information, etc. Input/output information, etc. may be stored in a specific location (for example, memory), or may be managed in a management table. Input/output information and the like can be overwritten, updated, or appended. The output information and the like may be deleted. The entered information and the like may be transmitted to another device.

[2-17] Software Software, whether referred to as software, firmware, middleware, microcode, hardware description language or otherwise, means instructions, instruction sets, code, code segments, program code, programs , subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, and the like.

Software, instructions, information, etc. may also be sent and received over a transmission medium. For example, the software uses wired technology (coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), etc.) and/or wireless technology (infrared, microwave, etc.) to create websites, Wired and/or wireless technologies are included within the definition of transmission medium when sent from a server or other remote source.

[2-18] Information, Signals Information, signals, etc. described in this disclosure may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may refer to voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these. may be represented by a combination of

[2-19] System, Network The terms "system" and "network" used in this disclosure are used interchangeably.

[2-20] Parameters, Channel Names In addition, the information, parameters, etc. described in this disclosure may be expressed using absolute values, or may be expressed using relative values from a predetermined value. and may be represented using other corresponding information. For example, radio resources may be indexed.

The names used for the parameters described above are not limiting names in any way. Further, the formulas, etc., using these parameters may differ from those expressly disclosed in this disclosure. Since the various channels (e.g., PUCCH, PDCCH, etc.) and information elements can be identified by any suitable name, the various names assigned to these various channels and information elements are in no way restrictive names. isn't it.

[2-21] "judgment", "decision"
As used in this disclosure, the terms "determining" and "determining" may encompass a wide variety of actions. "Judgement", "determining" are, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiring (eg, lookup in a table, database, or other data structure), ascertaining as "judged" or "determined", and the like.

Also, "judgment" and "determination" are used for receiving (e.g., receiving information), transmitting (e.g., transmitting information), input, output, access (accessing) (for example, accessing data in memory) may include deeming that a "judgment" or "decision" has been made. In addition, "judgment" and "decision" are considered to be "judgment" and "decision" by resolving, selecting, choosing, establishing, comparing, etc. can contain. In other words, "judgment" and "decision" may include considering that some action is "judgment" and "decision". Also, "judgment (decision)" may be read as "assuming", "expecting", "considering", or the like.

[2-22] Meaning of "Based on" As used in this disclosure, the phrase "based on" does not mean "based only on," unless explicitly stated otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

[2-23] "Different"
In the present disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean that "A and B are different from C". Terms such as "separate,""coupled," etc. may also be interpreted in the same manner as "different."

[2-24] "and", "or"
In the present disclosure, regarding configurations that can be implemented with either “A and B” or “A or B,” the configuration described in one expression may be used as the configuration described in the other expression. For example, when "A and B" are described, they may be used as "A or B" as long as they are not inconsistent with other descriptions and practicable.

[2-25] Variations of Aspects, etc. Each aspect/embodiment described in the present disclosure may be used alone, may be used in combination, or may be used by switching according to execution. In addition, the notification of predetermined information (for example, notification of “being X”) is not limited to being performed explicitly, but may be performed implicitly (for example, not notifying the predetermined information). good too.

Although the present disclosure has been described in detail above, it should be apparent to those skilled in the art that the present disclosure is not limited to the embodiments described in this disclosure. The present disclosure can be practiced with modifications and variations without departing from the spirit and scope of the present disclosure as defined by the claims. Accordingly, the description of the present disclosure is for illustrative purposes and is not meant to be limiting in any way.

REFERENCE SIGNS LIST 1 video projection system 2 network 3 camera 10 projector 20 video projection device 21 front projection unit 22 upper projection unit 23 right projection unit 24 left projection unit 25 Housing 30 Video distribution device 40 Video allocation control device 101 Plural projection unit 102 Projection control unit 103 Video reception unit 104 Video allocation request unit 301 Request video transmission unit 302 Multiple video acquisition unit 303 Target information storage unit 401 Target information acquisition unit 402 Video allocation unit 403 Video reception unit 404 Allocation instruction processing unit 405 Height determination unit 406 Speed identification Section 407 ... Congestion degree determination section 408 ... Preference information acquisition section 409 ... Designation history storage section.

Claims (9)

a target acquiring unit for acquiring target information indicating a plurality of shooting targets respectively shot by a plurality of shooting devices;
Images of two or more shooting targets among the plurality of shooting targets indicated by the acquired target information, wherein the images of the plurality of shooting targets include an objective video of the subject of the event shot from the surroundings, and an allocation unit that allocates an image including a subjective image captured by an imaging device attached to a subject to each projection unit of a projection device having a plurality of projection units;
an instruction unit that instructs to project an image according to the allocation of the allocation unit ;
A excitement level determination unit that determines the excitement level of the event from the video captured by the imaging device,
The allocation unit is an information processing device that allocates the subjective image to a projection unit that has projected the objective image when the determined excitement degree is equal to or greater than a predetermined standard. a target acquiring unit for acquiring target information indicating a plurality of shooting targets respectively shot by a plurality of shooting devices;
Wide-range video images of two or more shooting targets out of a plurality of shooting targets indicated by the acquired target information, the wide-range video including a specific shooting target and having a range wider than a predetermined range, and the shooting target. an allocation unit that allocates an image including a local image in which a range within the predetermined range is captured to each projection unit of a projection device having a plurality of projection units;
an instruction unit that instructs to project an image according to the allocation of the allocation unit;
a speed identification unit that identifies the speed of the specific imaging target;
The allocating unit allocates the wide-range image to a projection unit that projects the local image when a speed equal to or greater than a threshold is specified for the specific shooting target reflected in the local image, or the wide-range image an information processing apparatus that allocates the local image to a projection unit that projects the wide-range image when a velocity of less than a threshold is specified for the specific object being imaged in the image. The allocating unit, when the plurality of projection units project images onto at least front and left and right projection areas of the projection device, sets the positional relationship of the projection areas to match the positional relationship of the plurality of shooting targets in the real space. 3. The information processing apparatus according to claim 1, wherein the images are allocated so as to have a positional relationship according to . When a reference image serving as a reference is designated, the allocation unit determines that the positional relationship in the real space between the object to be photographed in the reference image and the object to be photographed in another image is a projection for projecting the reference image. 4. The information processing apparatus according to claim 3 , wherein the two or more images to be captured are allocated so as to have a positional relationship between the area and the projection area onto which the other image is projected. 2. The information according to claim 1 , wherein, when priorities are set for the plurality of projection units, the allocating unit allocates a video of a shooting location with a higher determined excitement degree to a projection unit with a higher priority. processing equipment. A congestion degree determination unit that determines the degree of congestion of people in the place where the imaging device shoots,
6. The allocating unit according to any one of claims 1 to 5 , wherein, when the plurality of projecting units are prioritized, the allocating unit allocates the image to the projecting unit having the priority according to the degree of congestion determined for the image. 1. The information processing apparatus according to 1. A preference acquisition unit that acquires preference information indicating user preferences,
When the plurality of projection units are prioritized, the allocating unit selects, from the user's preference indicated by the acquired preference information, the location where the user is highly likely to visit as the shooting target. The information processing apparatus according to any one of claims 1 to 6 , wherein a projection unit having a higher priority is assigned. a storage unit for storing a history of designation by the user when the projection device projects an image designated by the user for each of the projection units;
The allocation unit allocates, to the projection unit, an image that is highly likely to be designated to the projection unit based on the history stored in the storage unit when the user does not designate the projection unit. The information processing apparatus according to any one of claims 1 to 7 . an information processing apparatus according to any one of claims 1 to 8 ;
a receiving unit for receiving, from the video distribution device, a video distributed from the video distribution device in accordance with a projection instruction from the information processing device ; a plurality of projection units each projecting a video; a projection device comprising a projection control unit that causes the plurality of projection units to project;
A video projection system.

Images