WO2022024594A1 - 情報処理装置、情報処理方法、及びプログラム - Google Patents

情報処理装置、情報処理方法、及びプログラム Download PDF

Info

Publication number
WO2022024594A1
WO2022024594A1 PCT/JP2021/023652 JP2021023652W WO2022024594A1 WO 2022024594 A1 WO2022024594 A1 WO 2022024594A1 JP 2021023652 W JP2021023652 W JP 2021023652W WO 2022024594 A1 WO2022024594 A1 WO 2022024594A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
image
ornamental
user
viewer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/023652
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
貴嗣 青木
史憲 入江
一紀 田村
真彦 宮田
泰規 村上
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujifilm Corp
Original Assignee
Fujifilm Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujifilm Corp filed Critical Fujifilm Corp
Priority to JP2022540066A priority Critical patent/JP7546676B2/ja
Publication of WO2022024594A1 publication Critical patent/WO2022024594A1/ja
Priority to US18/156,361 priority patent/US20230156244A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G06T15/10—Geometric effects
    • G06T15/20—Perspective computation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/37—Details of the operation on graphic patterns
    • G09G5/377—Details of the operation on graphic patterns for mixing or overlaying two or more graphic patterns
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/2668—Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/475—End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data
    • H04N21/4755—End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data for defining user preferences, e.g. favourite actors or genre
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/858—Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules

Definitions

  • the technology of this disclosure relates to an information processing device, an information processing method, and a program.
  • Japanese Unexamined Patent Publication No. 2014-215828 discloses an image data reproduction device that reproduces an image from an arbitrary viewpoint with respect to the input content data.
  • the image data reproduction device described in JP-A-2014-215828 includes a separation unit, a viewpoint determination unit, a viewpoint image generation unit, and an individual viewpoint information generation unit.
  • the separation unit uses the image data specified for at least one or more image data and at least one of the at least one image data from the content data input from the outside, and the image at which viewpoint. Is output separately from the viewpoint information including at least one individual viewpoint information indicating whether to generate.
  • the viewpoint determination unit generates any one of the individual viewpoint information included in the viewpoint information as the viewpoint selection information.
  • the viewpoint image generation unit generates and outputs a viewpoint image indicated by the viewpoint selection information as a viewpoint image using the image data designated by the viewpoint selection information among at least one or more image data.
  • the individual viewpoint information generation unit adds user attribute information, which is information representing the user's attribute, to the viewpoint selection information, and generates user attribute-added individual viewpoint information.
  • Japanese Unexamined Patent Publication No. 2020-06301 discloses a terminal used by a user at a shooting location.
  • the terminal described in Japanese Patent Application Laid-Open No. 2020-06301 has an output unit that outputs viewpoint information to an information processing device that manages a plurality of images shot from a plurality of viewpoints at a shooting location, and a plurality of terminals according to the viewpoint information.
  • the first video selected from the video includes an input unit input from the information processing apparatus and a display unit for displaying the first video.
  • Japanese Unexamined Patent Publication No. 2019-197340 discloses an information processing apparatus having an acquisition means, a determination means, and a presentation means.
  • the acquisition means acquires viewpoint information regarding a designated virtual viewpoint corresponding to a virtual viewpoint image generated based on a plurality of captured images acquired by a plurality of image pickup devices.
  • the determination means determines an object included in at least one of a plurality of captured images and included in a range within the field of view of the virtual viewpoint specified by the viewpoint information acquired by the acquisition means.
  • the presenting means presents information according to the determination result by the determination means for a plurality of virtual viewpoints specified by the viewpoint information acquired by the acquisition means.
  • One embodiment according to the technique of the present disclosure provides an information processing device, an information processing method, and a program that can easily generate sympathy among a plurality of viewers who are viewing an ornamental image.
  • a first aspect of the technique of the present disclosure comprises a processor, memory built into or connected to the processor, and viewed by a viewer based on an image obtained by imaging by an image pickup device. It is an information processing device that generates an ornamental image, and the processor acquires the request information for requesting the generation of the ornamental image, executes the generation process for generating the ornamental image according to the acquired request information, and executes the generation process.
  • the request information includes setting information indicating the setting of the ornamental image, and the generation process creates an ornamental image in which the setting information reflects the viewer information about the viewer within the default range in the request information of a plurality of viewers. It is an information processing device that is a process to generate.
  • the second aspect according to the technique of the present disclosure is the information processing apparatus according to the first aspect, wherein the ornamental image includes a virtual viewpoint image created based on the image.
  • a third aspect according to the technique of the present disclosure relates to the second aspect in which the setting information includes the gaze position specifying information capable of specifying the gaze position used for generating the virtual viewpoint image in the area shown by the image. It is an information processing device.
  • the fourth aspect according to the technique of the present disclosure is the information processing apparatus according to the third aspect in which the gaze position is the position of a specific object included in the area.
  • the fifth aspect according to the technique of the present disclosure is the information processing apparatus according to the third aspect or the fourth aspect, in which the gaze position specifying information includes the gaze position path information indicating the path of the gaze position.
  • a sixth aspect according to the technique of the present disclosure is from the second aspect in which the processor generates an ornamental image by superimposing the viewer information on the viewer whose setting information is within the predetermined range on the virtual viewpoint image.
  • the information processing device according to any one of the fifth aspects.
  • a seventh aspect according to the technique of the present disclosure is that the ornamental image is of audible data related to the viewer whose setting information is within the default range and visible data related to the viewer whose setting information is within the default range.
  • the information processing apparatus according to any one of the first to sixth aspects including at least one of the above.
  • the eighth aspect according to the technique of the present disclosure is a timing in which the ornamental image is a moving image and the processor sets at least one of the audible data and the visible data at the time of reproducing the ornamental image by the viewer.
  • This is an information processing device according to a seventh aspect, which generates an ornamental image in which viewer information is reflected by adding the ornamental image to the ornamental image.
  • a ninth aspect according to the technique of the present disclosure is any of the first to eighth aspects, wherein the ornamental image includes a viewer-specific image in which the setting information can visually identify the viewer within a predetermined range. It is an information processing apparatus according to one aspect.
  • a tenth aspect according to the technique of the present disclosure is from the first aspect to the ninth aspect in which the processor stores the viewer information in a memory and generates an ornamental image reflecting the viewer information stored in the memory. It is an information processing apparatus which concerns on any one aspect of.
  • the eleventh aspect according to the technique of the present disclosure is an information processing device according to any one of the first to tenth aspects in which the viewer information includes an attribute relating to the taste of the viewer.
  • the twelfth aspect according to the technique of the present disclosure is an information processing device according to any one of the first to eleventh aspects in which the requested information includes the viewer information.
  • a thirteenth aspect according to the technique of the present disclosure is a first aspect in which the setting information includes information on which of a plurality of moving images obtained by performing imaging by a plurality of imaging devices is to be viewed. It is an information processing apparatus according to.
  • a thirteenth aspect according to the technique of the present disclosure is a thirteenth aspect in which a processor generates an ornamental moving image by superimposing viewer information on a viewer whose setting information is within a predetermined range on the moving image to be viewed. It is an information processing device related to.
  • any one of a plurality of edited moving images created based on a plurality of moving images obtained by performing imaging by a plurality of imaging devices for setting information is viewed.
  • the information processing apparatus according to the first aspect which includes information on a camera.
  • a sixteenth aspect according to the technique of the present disclosure is a fifteenth aspect in which a processor superimposes viewer information on a viewer whose setting information is within a predetermined range on the edited video to be viewed to generate an ornamental video. It is an information processing device according to the embodiment.
  • a seventeenth aspect according to the technique of the present disclosure is an information processing method for generating an ornamental image to be viewed by a viewer based on an image obtained by imaging by an imaging device, and is for ornamental use.
  • the request information includes acquiring the request information requesting the generation of the image and executing the generation process of generating the ornamental image according to the acquired request information, and the request information includes the setting information indicating the setting of the ornamental image.
  • the generation process is an information processing method that generates an ornamental image in which the setting information reflects the viewer information about the viewer within the default range in the request information of a plurality of viewers.
  • An eighteenth aspect according to the technique of the present disclosure is for causing a computer to perform information processing to generate an ornamental image to be viewed by a viewer based on an image obtained by performing imaging by an imaging device.
  • the information processing includes acquiring the request information requesting the generation of the ornamental image and executing the generation process of generating the ornamental moving image according to the acquired request information, and the request information is the ornamental information.
  • the generation process includes setting information indicating the setting of the image for viewing, and is a process of generating an ornamental image in which the setting information reflects the viewer information about the viewer within the default range in the request information of a plurality of viewers. It is a program.
  • CPU is an abbreviation for "Central Processing Unit”.
  • RAM is an abbreviation for "RandomAccessMemory”.
  • SSD is an abbreviation for “Solid State Drive”.
  • HDD is an abbreviation for "Hard Disk Drive”.
  • EEPROM refers to the abbreviation of "Electrically Erasable and Programmable Read Only Memory”.
  • I / F refers to the abbreviation of "Interface”.
  • IC refers to the abbreviation of "Integrated Circuit”.
  • ASIC refers to the abbreviation of "ApplicationSpecific Integrated Circuit”.
  • PLD is an abbreviation for "Programmable Logic Device”.
  • FPGA refers to the abbreviation of "Field-Programmable Gate Array”.
  • SoC is an abbreviation for "System-on-a-chip".
  • CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor”.
  • CCD refers to the abbreviation of "Charge Coupled Device”.
  • EL refers to the abbreviation of "Electro-Luminescence”.
  • GPU refers to the abbreviation of "Graphics Processing Unit”.
  • LAN is an abbreviation for "Local Area Network”.
  • 3D refers to the abbreviation of "3 (three) Dimensional”.
  • USB is an abbreviation for "Universal Serial Bus”.
  • ID refers to the abbreviation of "Identification”.
  • a CPU is illustrated as an example of the "processor” according to the technique of the present disclosure, but the "processor” according to the technique of the present disclosure includes a plurality of processing devices such as a CPU and a GPU. It may be a combination of.
  • the GPU operates under the control of the CPU and is responsible for executing image processing.
  • match is, in addition to perfect match, an error that is generally acceptable in the technical field to which the technology of the present disclosure belongs and is to the extent that it does not contradict the gist of the technology of the present disclosure. Refers to a match in the sense of including.
  • short time is an error generally allowed in the technical field to which the technology of the present disclosure belongs, and is an error to the extent that it does not contradict the purpose of the technology of the present disclosure. Refers to the same time in the sense of including.
  • the information processing system 10 includes an information processing device 12, a plurality of image pickup devices 14 connected to the information processing device 12, and a plurality of user devices 16.
  • the image pickup device 14 is an image pickup device having a CMOS image sensor, and is equipped with an optical zoom function and / or a digital zoom function. Instead of the CMOS image sensor, another type of image sensor such as a CCD image sensor may be adopted.
  • the image pickup device 14 is an example of the "imaging device" according to the technique of the present disclosure.
  • the plurality of image pickup devices 14 are installed in the soccer field 18. Each of the plurality of image pickup devices 14 is arranged so as to surround the soccer field 20, and images an area in the soccer field 18 as an image pickup area.
  • a plurality of image pickup devices 14 are arranged so as to surround the soccer field 20 is given, but the technique of the present disclosure is not limited to this, and the arrangement of the plurality of image pickup devices 14 is the user A. , User B, User C, and the like, and are determined according to the virtual viewpoint image requested to be generated.
  • the plurality of image pickup devices 14 may be arranged so as to surround the entire soccer field 20, or may be arranged so as to surround a specific part thereof.
  • the image pickup by the image pickup device 14 refers to, for example, an image pickup at an angle of view including an image pickup region.
  • imaging area includes not only the concept of the area showing the whole in the soccer field 18 but also the concept of the area showing a part of the soccer field 18.
  • the imaging region is changed according to the imaging position, imaging direction, and angle of view of the imaging device 14.
  • the information processing device 12 is installed in the control room 21.
  • the plurality of image pickup devices 14 and the information processing device 12 are connected via a cable 30 (for example, a LAN cable).
  • the information processing device 12 controls a plurality of image pickup devices 14 and acquires an image captured image 60 (see FIG. 4) obtained by being imaged by each of the plurality of image pickup devices 14.
  • a cable 30 for example, a LAN cable
  • the information processing device 12 controls a plurality of image pickup devices 14 and acquires an image captured image 60 (see FIG. 4) obtained by being imaged by each of the plurality of image pickup devices 14.
  • the connection using the wired communication method by the cable 30 is illustrated here, the connection is not limited to this, and the connection using the wireless communication method may be used.
  • the captured image 60 acquired by each imaging device 14 is an example of an "image" according to the technique of the present disclosure.
  • the plurality of user devices 16 are personal computers.
  • the user device 16 is connected to the information processing device 12 via a communication network 17 (for example, the Internet or the like).
  • a personal computer is applied as an example of the user device 16, but the personal computer is only an example.
  • the user device 16 may be, for example, a smartphone, a tablet terminal, a portable multifunctional terminal such as a head-mounted display, or a large display used in a public viewing venue or the like.
  • the information processing device 12 is a device corresponding to a server, and the user device 16 is a device corresponding to a client terminal for the information processing device 12.
  • the user device 16 requests the information processing device 12 to provide the ornamental image 68.
  • the information processing device 12 generates an ornamental image 68 based on the captured image 60 (see FIG. 4) obtained by performing imaging by the imaging device 14 in response to a request from the user device 16, and then generates an ornamental image 68.
  • the generated ornamental image 68 is transmitted to the user device 16.
  • the information processing device 12 is an example of the "information processing device” according to the technique of the present disclosure.
  • the ornamental image 68 is an example of the “ornamental image” according to the technique of the present disclosure.
  • the plurality of user devices 16 are used by users A, B, and C existing outside the soccer field 18, respectively.
  • Users A, B, and C each view the ornamental image 68 provided by the information processing apparatus 12 using the user device 16.
  • Users A, B, and C are examples of "viewers" according to the technology of the present disclosure.
  • FIG. 1 illustrates three users A, B, and C as users viewing the ornamental image 68, but the number of users is not limited to this, and may be more or less than three. You may.
  • a plurality of users may view the ornamental image 68 via one user device 16.
  • users when it is not necessary to distinguish between users A, B, and C, they are collectively referred to as "users" without reference numerals.
  • the information processing apparatus 12 includes a computer 24, a reception device 26, a display 28, a communication I / F 32 for an image pickup device, and a communication I / F 34 for a user device.
  • the computer 24 includes a CPU 24A, a storage 24B, and a memory 24C, and the CPU 24A, the storage 24B, and the memory 24C are connected via the bus 36.
  • one bus is shown as the bus 36 for convenience of illustration, but a plurality of buses may be used.
  • the bus 36 may include a serial bus or a parallel bus composed of a data bus, an address bus, a control bus, and the like.
  • the CPU 24A controls the entire information processing device 12.
  • the storage 24B stores various parameters and various programs.
  • the storage 24B is a non-volatile storage device.
  • EEPROM, SSD, and HDD are adopted as an example of the storage 24B, but the present invention is not limited to this, and HDD, SSD, EEPROM, or the like may be used, or a combination of a plurality of non-volatile storage devices may be used.
  • the memory 24C is a storage device. Various information is temporarily stored in the memory 24C.
  • the memory 24C is used as a work memory by the CPU 24A.
  • RAM is adopted as an example of the memory 24C, but the present invention is not limited to this, and other types of storage devices may be used.
  • the memory 24C may be a memory built in the CPU 24A.
  • the CPU 24A is an example of a "processor” according to the technique of the present disclosure.
  • the memory 24C is an example of the "memory” according to the technique of the present disclosure.
  • the reception device 26 receives instructions from the administrator of the information processing device 12 and the like. Examples of the reception device 26 include a keyboard, a touch panel, a mouse, and the like. The reception device 26 is connected to a bus 36 or the like, and the instruction received by the reception device 26 is acquired by the CPU 24A.
  • the display 28 is connected to the bus 36 and displays various information under the control of the CPU 24A.
  • An example of the display 28 is a liquid crystal display. Not limited to the liquid crystal display, another type of display such as an EL display (for example, an organic EL display or an inorganic EL display) may be adopted as the display 28.
  • an EL display for example, an organic EL display or an inorganic EL display
  • the image pickup device communication I / F 32 is connected to the cable 30.
  • the communication I / F 32 for an image pickup device is realized by, for example, a device having an FPGA.
  • the image pickup device communication I / F 32 is connected to the bus 36 and controls the exchange of various information between the CPU 24A and the plurality of image pickup devices 14.
  • the communication I / F 32 for an image pickup device controls a plurality of image pickup devices 14 according to the request of the CPU 24A.
  • the communication I / F 32 for an image pickup device stores an image captured image 60 obtained by being imaged by each of the plurality of image pickup devices 14 in the storage 24B (see FIG. 4).
  • the communication I / F 32 for an image pickup device is exemplified here as a wired communication I / F, it may be a wireless communication I / F such as a high-speed wireless LAN.
  • the user device communication I / F 34 is communicably connected to the user device 16 via the communication network 17.
  • the communication I / F 34 for the user device is realized by, for example, a device having an FPGA.
  • the communication I / F 34 for the user device is connected to the bus 36.
  • the user device communication I / F 34 controls the exchange of various information between the CPU 24A and the user device 16 in a wireless communication method via the communication network 17.
  • at least one of the communication I / F 32 for the image pickup device and the communication I / F 34 for the user device can be configured by a fixed circuit instead of the FPGA.
  • at least one of the communication I / F32 for the image pickup device and the communication I / F34 for the user device may be a circuit composed of an ASIC, an FPGA, and / or a PLD or the like.
  • the user device 16 includes a computer 38, a reception device 40, a display 42, a microphone 44, a speaker 46, a camera 48, and a communication I / F 50.
  • the computer 38 includes a CPU 38A, a storage 38B, and a memory 38C, and the CPU 38A, the storage 38B, and the memory 38C are connected via the bus 52.
  • one bus is shown as the bus 52 for convenience of illustration, but the bus 52 may be a plurality of buses.
  • the bus 52 may be a serial bus, or may be a parallel bus including a data bus, an address bus, a control bus, and the like.
  • the CPU 38A controls the entire user device 16.
  • the storage 38B stores various parameters and various programs.
  • the storage 38B is a non-volatile storage device.
  • a flash memory is adopted as an example of the storage 38B.
  • the flash memory is merely an example, and examples of the storage 38B include various non-volatile memories such as a magnetoresistive memory and / or a ferroelectric memory in place of the flash memory or in combination with the flash memory. ..
  • the non-volatile storage device may be EEPROM, HDD, and / or SSD or the like.
  • the memory 38C temporarily stores various information and is used as a work memory by the CPU 38A.
  • An example of the memory 38C is RAM, but the memory 38C is not limited to this, and other types of storage devices may be used.
  • the reception device 40 receives instructions from users and the like.
  • the reception device 40 includes a mouse 40A and a keyboard (see FIG. 1). Further, the reception device 40 may include a touch panel.
  • the reception device 40 is connected to the bus 52, and the instruction received by the reception device 40 is acquired by the CPU 38A.
  • the display 42 is connected to the bus 52 and displays various information under the control of the CPU 38A.
  • An example of the display 42 is a liquid crystal display. Not limited to the liquid crystal display, another type of display such as an EL display (for example, an organic EL display or an inorganic EL display) may be adopted as the display 42.
  • an EL display for example, an organic EL display or an inorganic EL display
  • the microphone 44 converts the collected sound into an electric signal.
  • the microphone 44 is connected to the bus 52.
  • the electric signal obtained by converting the sound collected by the microphone 44 is acquired by the CPU 38A via the bus 52.
  • the speaker 46 converts an electric signal into sound.
  • the speaker 46 is connected to the bus 52.
  • the speaker 46 receives the electric signal output from the CPU 38A via the bus 52, converts the received electric signal into sound, and outputs the sound obtained by converting the electric signal to the outside of the user device 16.
  • the speaker 46 is integrated with the user device 16, but a sound output from headphones connected to the user device 16 by wire or wirelessly may be adopted.
  • the headphones also include earphones.
  • the camera 48 acquires an image showing the subject by taking an image of the subject.
  • the camera 48 is connected to the bus 52.
  • the image obtained by capturing the subject by the camera 48 is acquired by the CPU 38A via the bus 52.
  • the communication I / F 50 is communicably connected to the information processing device 12 via the communication network 17.
  • the communication I / F50 is realized, for example, by a device composed of a circuit (for example, ASIC, FPGA, and / or PLD, etc.).
  • the communication I / F 50 is connected to the bus 52.
  • the communication I / F 50 controls the exchange of various information between the CPU 38A and the information processing device 12 in a wireless communication system via the communication network 17.
  • the storage 24B stores an ornamental moving image generation program 54.
  • the CPU 24A reads the ornamental moving image generation program 54 from the storage 24B, and executes the read ornamental moving image generation program 54 on the memory 24C.
  • the CPU 24A operates as an information acquisition unit 56, a virtual viewpoint image generation unit 57, and an ornamental image generation unit 58 according to the ornamental video generation program 54 executed on the memory 24C, thereby performing the ornamental video generation processing described later.
  • the ornamental moving image generation program 54 is a program for causing the computer 24 to execute a process, and is an example of a "program" according to the technique of the present disclosure.
  • the computer 24 is an example of a "computer” according to the technique of the present disclosure.
  • the CPU 24A acquires the request information 64 requesting the generation of the ornamental image 68 from each user device 16 via the communication I / F 34 for the user device.
  • the request information 64 includes instruction information 64-1 instructing the display of the information acquisition screen 66 (see FIG. 5), setting information 64-2 indicating the setting of the ornamental image 68, and user information 64-2 indicating information about the user. 3 and is included.
  • the request information 64 is an example of "request information" related to the technique of the present disclosure.
  • the CPU 24A executes an ornamental moving image generation process for generating an ornamental image 68 according to the acquired request information 64.
  • the ornamental video generation process will be described in detail later, but in the request information 64 from a plurality of users, the ornamental image 68 in which the setting information 64-2 reflects the user information 64-3 regarding the users within the default range is generated. It is a process to do.
  • the ornamental moving image generation process is an example of the "generation process" according to the technique of the present disclosure.
  • the setting information 64-2 is an example of "setting information” related to the technology of the present disclosure
  • the user information 64-3 is an example of "viewer information” related to the technology of the present disclosure.
  • the information acquisition unit 56 receives the setting information 64-2 and the user information 64-3 of the user A via the communication I / F 34 for the user device, and stores the received setting information 64-2 and the user information 64-3 in the memory 24C.
  • the team name to be supported is stored as an attribute 77A related to the user's preference.
  • the user ID 71A, the attribute 77A, and the face image 76A are stored in the memory 24C as user information 64-3.
  • the information about the player of interest will be described in detail later, but it is information used as a gaze object 78 when the virtual viewpoint image generation unit 57 generates a virtual viewpoint image, and is stored in the memory 24C as setting information 64-2.
  • Attribute 77A is an example of an "attribute" according to the technique of the present disclosure.
  • the setting information 64-2 and the user information 64-3 of each user acquired from each user device 16 using the information acquisition screen 66 are stored in association with each user.
  • reference numeral 76B indicates a face image of user B
  • reference numeral 76C indicates a face image of user C.
  • the face images 76A, 76B, and 76C are collectively referred to as “face image 76”.
  • the face image 76 is an example of a “viewer-specific image” according to the technique of the present disclosure.
  • the virtual viewpoint image generation unit 57 generates a virtual viewpoint image 62 based on the captured image 60 stored in the storage 24B and the setting information 64-2 received from each user.
  • the virtual viewpoint image 62 is an image generated by image processing from the captured image 60, and is an image corresponding to the case where the imaged region is viewed from an arbitrary viewpoint (virtual viewpoint).
  • the virtual viewpoint image 62 is an example of a “virtual viewpoint image” according to the technique of the present disclosure.
  • the setting information 64-2 includes gaze position specifying information capable of specifying the gaze position 80 used for generating the virtual viewpoint image 62 in the area indicated by the captured image 60.
  • the gaze position 80 is the position of a specific object included in the region indicated by the captured image 60, for example, the position of a player designated as a player of interest.
  • the ornamental moving image generation process is executed by the CPU 24A when the instruction information 64-1 of the request information 64 is received from at least one of the plurality of user devices 16. As shown in FIG. 5, as an example, when the instruction information 64-1 is received from the user device 16, the information acquisition unit 56 first generates the information acquisition screen 66 according to a predetermined format. The information acquisition unit 56 transmits the generated information acquisition screen 66 to the user device 16 which is the output source of the instruction information 64-1.
  • the user device 16 receives the information acquisition screen 66, and displays the received information acquisition screen 66 on the display 42.
  • FIG. 6 shows an information acquisition screen 66 displayed on the display 42 of the user device 16 of the user A.
  • the title "Japan vs. England" of the ornamental image 68 that the user A wants to see, and the message "Please enter your information” prompting the user A to enter the information are displayed. It is displayed.
  • an input field 70 for inputting the user ID 71A of the user A, a selection button 72 for selecting a team supported by the user A, a display frame 73 for displaying the input face image, and the user A.
  • a selection button 74 and a send button 75 for selecting a player of interest are displayed.
  • User A inputs the user ID 71A from the reception device 40 to the input field 70. Further, the user A selects a team to be supported by clicking one of the selection buttons 72 with the mouse 40A. In the example shown in FIG. 6, "Japan" is selected as the supporting team.
  • the user A uses the mouse 40A to display, for example, an icon showing a face image obtained by using the camera 48 of the user device 16 to image his / her face and capturing the image by the camera 48. Drag it onto the frame 73. As a result, the face image 76A of the user A is displayed on the display frame 73.
  • the user A selects a player of interest by clicking one of the selection buttons 74 with the mouse 40A.
  • the player is represented by the team name to which the player belongs and the player's uniform number.
  • "Japan-9” represents a player with a uniform number "9" on the "Japan” team.
  • "Japan-9" is selected as the player of interest.
  • User A inputs information on the information acquisition screen 66, and then clicks the send button 75 with the mouse 40A.
  • the information input to the information acquisition screen 66 is transmitted from the user device 16 to the information processing device 12 as the setting information 64-2 and the user information 64-3 of the user A.
  • the virtual viewpoint image generation unit 57 reads out the gaze object 78 stored in association with the user who is the output source of the request information 64 from the memory 24C.
  • the gaze object 78 is a player (Japan-9) with a uniform number 9 of the Japan team.
  • the virtual viewpoint image generation unit 57 acquires the coordinates of the gaze object 78 in the soccer field 18, and defines a region having a radius of several meters (for example, 1 m) around these coordinates as the gaze position 80.
  • the gaze object 78 is an example of a "specific object” according to the technique of the present disclosure.
  • the gaze position 80 is an example of the "gaze position” according to the technique of the present disclosure.
  • the coordinates and radius of the gaze object 78 are an example of "gaze position specifying information" according to the technique of the present disclosure.
  • the virtual viewpoint image generation unit 57 acquires the first captured image 60-1 and the second captured image 60-2 from the storage 24B.
  • the first captured image 60-1 and the second captured image 60-2 are captured images acquired at the same time by two different image pickup devices 14 among the plurality of image pickup devices 14.
  • the virtual viewpoint image generation unit 57 generates a virtual viewpoint image 62 by generating 3D polygons based on the first captured image 60-1 and the second captured image 60-2 with reference to the gaze position 80.
  • the virtual viewpoint image generation unit 57 stores the generated virtual viewpoint image 62 in the storage 24B.
  • the number of captured images used to generate 3D polygons does not have to be two.
  • the virtual viewpoint image generation unit 57 when the gaze object 78 is a person, the virtual viewpoint image generation unit 57 has the viewpoint position 82 and the line-of-sight direction 84 of the virtual viewpoint in the position and direction facing the person. To determine. Further, the virtual viewpoint image generation unit 57 determines the field of view 88 of the virtual viewpoint image based on the predetermined angle of view 86. The virtual viewpoint image generation unit 57 generates a virtual viewpoint image 62 based on the determined field of view 88. That is, the virtual viewpoint image 62 is a virtual image when the imaging region is observed from the viewpoint position 82 in the line-of-sight direction 84 at an angle of view 86.
  • the virtual viewpoint image generation unit 57 generates a virtual viewpoint image 62 for each gaze object 78. For example, when user A and user C specify "Japan-9" as the gaze object 78, and user B specifies "England-9” as the gaze object 78 (see FIG. 4), the virtual viewpoint image generation unit. 57 generates a virtual viewpoint image 62 in which the position of “Japan-9” is the gaze position 80 and a virtual viewpoint image 62 in which the position of “England-9” is the gaze position 80, and stores the virtual viewpoint image 62 in the storage 24B. .. It is not necessary to generate the virtual viewpoint image 62 for all the gaze objects 78, and the virtual viewpoint image 62 may be generated only for the gaze object 78 specified by the user.
  • the ornamental image generation unit 58 superimposes the user information 64-3 regarding the user having the same setting information 64-2 on the virtual viewpoint image 62 corresponding to the user's setting information 64-2, thereby displaying the ornamental image 68.
  • the ornamental image 68 is an image including the virtual viewpoint image 62.
  • the fact that the setting information 64-2 is the same is an example of "the setting information is within the specified range" according to the technique of the present disclosure.
  • the ornamental image generation unit 58 receives the virtual viewpoint image 62 corresponding to the setting information 64-2 of the user A, that is, “Japan-9”.
  • the virtual viewpoint image 62 with the position of the gaze position 80 is acquired from the storage 24B.
  • the ornamental image generation unit 58 acquires the user information 64-3 regarding the user (user C and others) who has set the same setting information 64-2 as the user A from the memory 24C.
  • the ornamental image generation unit 58 generates the ornamental image 68 by superimposing the user information 64-3 acquired from the memory 24C on the virtual viewpoint image 62 acquired from the storage 24B.
  • the user who sets the same setting information 64-2 as the user A may be the user who currently sets the same setting information 64-2, or may be the user who has set the same setting information 64-2 in the past. It may be the user who set the above, or both.
  • the ornamental image generation unit 58 adds the face image 76 that can visually identify the user having the same setting information 64-2 to the virtual viewpoint image 62, so that the ornamental image 64-3 is reflected in the ornamental image generation unit 58.
  • Generate image 68 That is, in the example shown in FIG. 9, the ornamental image generation unit 58 attaches the face image 76A of the user A and the face image 76C of the user C to the virtual viewpoint image 62 corresponding to the setting information 64-2 of the user A. By superimposing, the ornamental image 68 is generated. That is, in this example, the setting information 64-2 of the user A and the user C is the same.
  • the user A when the user A views the ornamental image 68, it is not necessary to superimpose the face image 76A of the user A. If the user A appreciates the ornamental image 68 on which the face image 76C of the user C is superimposed, the user A can obtain the sensation of viewing the image together with the user C.
  • the face images 76A and 76C are superimposed on the spectator seats of the soccer field 18, but the positions and sizes of the face images 76A and 76C are not limited to this.
  • the user A and the user C are associated with each other based on the setting information 64-2, and an ornamental image 68 reflecting the user information 64-3 of the user A and the user C is generated.
  • the number of users associated based on the setting information 64-2 is not limited to two.
  • An ornamental image 68 may be generated that reflects the user information 64-3 of a large number of users having the same setting information 64-2.
  • the ornamental image 68 is generated by the ornamental image generation unit 58 at a predetermined frame rate (for example, 60 fps).
  • a series of ornamental images 68 continuously generated by the ornamental image generation unit 58 at a predetermined frame rate is transmitted to the user device 16 as an ornamental moving image by the ornamental image generation unit 58.
  • the user device 16 receives the ornamental video and displays the received ornamental video on the display 42.
  • the ornamental moving image is an example of the "moving image" according to the technique of the present disclosure.
  • the viewing image 68 may be displayed on the display 42 as a still image instead of the viewing moving image.
  • a bird's-eye view image 97 showing the position of the gaze position 80 used to generate the time 94, the comment entry field 96, and the virtual viewpoint image 62 is superimposed on the lower side of the ornamental image 68.
  • Time 94 indicates the playback time of the ornamental moving image.
  • the comment entry field 96 is an entry field for the user to enter a comment 92 while watching the ornamental video.
  • the image showing the position of the gaze position 80 is not limited to the bird's-eye view image 97 looking down on the imaging region from directly above, but may be an image looking down on the imaging region from diagonally above. Alternatively, the image showing the position of the gaze position 80 may be two images, a bird's-eye view image 97 and an image of the imaging region viewed from the side.
  • the ornamental image generation unit 58 adds at least one of the voice 90 from the same user with the setting information 64-2 and the comment 92 from the user with the same setting information 64-2 to the virtual viewpoint image 62. Generates an ornamental image 68 reflecting the user information 64-3.
  • the voice 90 is a user's voice, music, or the like collected by the microphone 44 of each user device 16.
  • the comment 92 is a character string input by the user in the comment entry field 96 at an arbitrary timing using the reception device 40 while watching the ornamental video displayed on the display 42 of the user device 16.
  • the voice 90 is an example of "audible data" according to the technique of the present disclosure
  • the comment 92 is an example of "visible data” according to the technique of the present disclosure.
  • the voice 90 and the comment 92 are transmitted from the user device 16 of the user C to the ornamental image generation unit 58 via the communication I / F50 and the communication I / F34 for the user device, for example.
  • the ornamental image generation unit 58 receives the audio 90 and / or the comment 92, and when the ornamental image 68 is reproduced, the received audio 90 and / or the comment 92 is transmitted to the virtual viewpoint image 62 at a timing set by the user. Add.
  • the voice 90 and / or the comment 92 is displayed on the user device 16 of the user A and / or is output from the user device 16 at the timing set by the user C.
  • the "timing set by the user” is the time when the audio 90 and / or the comment 92 is received by the ornamental image generation unit 58 at the reproduction time of the ornamental moving image. For example, when the user C and the user A are watching the viewing video at the same time, the audio 90 and / or the comment 92 of the user C is displayed on the user device 16 of the user A in real time, and / or the user device 16. Is output from.
  • the “timing set by the user” is not limited to this, and may be a time specified by the user from the reception device 40 or the like.
  • the ornamental image generation unit 58 associates the voice 90 and / or the comment 92 input from the user device 16 with the time when the voice 90 and / or the comment 92 is received as the user information 64-3, and the user. Each time, it is stored in the memory 24C (see FIG. 4).
  • the ornamental image generation unit 58 acquires the voice 90 and / or the comment 92 in addition to the face image 76 from the memory 24C, and generates the ornamental image 68 reflecting the voice 90 and / or the comment 92. That is, the ornamental image generation unit 58 generates the ornamental image 68 by adding the sound 90 and / or the comment 92 acquired from the memory 24C to the virtual viewpoint image 62 at the time associated with the respective data. do.
  • the voice 90 and / or the comment 92 of the user C is stored in the memory 24C.
  • the ornamental image generation unit 58 By generating the ornamental image 68 as described above by the ornamental image generation unit 58, the user A who has viewed the ornamental video at a timing different from that of the user C is a user together with the ornamental video at a timing set by the user C.
  • the voice 90 and / or the comment 92 of C can be appreciated.
  • a user for example, a user who has set the comment 92 of the user A and the same setting information 64-2 as the user A at the playback time “00:05:30” of the ornamental moving image.
  • the comment 92 or the like of B or user C or the like) is added to the virtual viewpoint image 62 and displayed on the display 42.
  • the voice 90 is added to the ornamental image 68 at a time associated with the voice data. That is, the audio 90 is reproduced by the speaker 46 of the user device 16 at the reproduction time “00:05:30” of the ornamental moving image.
  • the "timing set by the user” is the playback time "00:05:30” of the ornamental moving image.
  • the comment 92 may be continuously displayed for, for example, several seconds after the "timing set by the user".
  • the ornamental moving image generation process shown in FIG. 10 is realized by the CPU 24A executing the ornamental moving image generation program 54. Further, the ornamental moving image generation process shown in FIG. 10 is started when the CPU 24A receives instruction information 64-1 from at least one of the plurality of user devices 16.
  • step ST101 the information acquisition unit 56 generates the information acquisition screen 66, and the generated information acquisition screen 66 is displayed by the user who is the output source of the instruction information 64-1. Send to device 16.
  • the user device 16 receives the information acquisition screen 66, and displays the received information acquisition screen 66 on the display 42.
  • step ST102 the ornamental moving image generation process shifts to step ST102.
  • step ST102 the information acquisition unit 56 determines whether or not the user information 64-3 and the setting information 64-2 requested on the information acquisition screen 66 have been input.
  • the determination is affirmed and the ornamental moving image generation process proceeds to step ST103. If the user information 64-3 and the setting information 64-2 are not input in step ST102, the determination is denied and the ornamental moving image generation process proceeds to step ST102.
  • step ST103 the virtual viewpoint image generation unit 57 determines whether or not the timing for generating the virtual viewpoint image (hereinafter, also referred to as “virtual viewpoint image generation timing”) has arrived.
  • the virtual viewpoint image generation timing is, for example, a timing determined based on a default frame rate constituting the ornamental moving image.
  • the determination is affirmed, and the ornamental moving image generation process shifts to step ST104. If the virtual viewpoint image generation timing has not arrived in step ST103, the determination is denied, and the ornamental moving image generation process shifts to step ST111.
  • step ST104 the virtual viewpoint image generation unit 57 determines the gaze position 80 based on the gaze object 78 set as the setting information 64-2. After that, the ornamental moving image generation process shifts to step ST105.
  • step ST105 the virtual viewpoint image generation unit 57 generates a virtual viewpoint image 62 based on the gaze position 80 determined in step ST104. After that, the ornamental moving image generation process proceeds to step ST106.
  • step ST106 the virtual viewpoint image generation unit 57 stores the virtual viewpoint image 62 generated in step ST105 in the storage 24B. After that, the ornamental moving image generation process shifts to step ST107.
  • step ST107 the ornamental image generation unit 58 has the user information 64-3 of the user who is the output source of the instruction information 64-1 and the same gaze object 78 as the user who is the output source of the instruction information 64-1.
  • the user information 64-3 of the user is added to the virtual viewpoint image 62 and output as an ornamental image 68 to the user device 16 which is the output source of the instruction information 64-1.
  • the ornamental moving image generation process shifts to step ST108.
  • step ST108 the ornamental image generation unit 58 determines whether or not the voice 90 or the comment 92 of the user who is the output source of the instruction information 64-1 or the user who has the same gaze object 78 is input.
  • the determination is affirmed, and the ornamental moving image generation process shifts to step ST109. If the audio 90 or the comment 92 is not input in step ST108, the determination is denied and the ornamental moving image generation process proceeds to step ST111.
  • the determination in step ST108 is not limited to the determination of whether or not the voice 90 or the comment 92 is input in real time, but includes the determination of whether or not the voice 90 or the comment 92 is input in advance for the same ornamental moving image. But it may be.
  • step ST109 the ornamental image generation unit 58 adds the input voice 90 or the comment 92 to the virtual viewpoint image 62 in addition to the user information 64-3 of the user having the same gaze object 78.
  • the ornamental image generation unit 58 transmits the virtual viewpoint image 62 to which the voice 90 or the comment 92 is added as the ornamental image 68 to the user device 16 of the user who is the output source of the instruction information 64-1. After that, the ornamental moving image generation process shifts to step ST110.
  • step ST110 the ornamental image generation unit 58 stores the input voice 90 or the comment 92 in the memory 24C in association with the reproduction time of the ornamental moving image including the series of ornamental images 68. After that, the ornamental moving image generation process proceeds to step ST111.
  • step ST111 the ornamental image generation unit 58 determines whether or not the end condition is satisfied.
  • the end condition include the end of the captured image 60, the operation of the stop button, and the like.
  • the stop button is displayed, for example, as a softkey on the display 42 of the user device 16. Specifically, it is displayed in the playback screen including the ornamental video. If the end condition is satisfied in step ST111, the determination is affirmed, and the ornamental moving image generation process ends. If the end condition is not satisfied in step ST111, the determination is denied and the ornamental moving image generation process proceeds to step ST103.
  • the information processing apparatus 12 includes a CPU 24A and a memory 24C connected to the CPU 24A.
  • the information processing device 12 executes an ornamental moving image generation process for generating an ornamental image 68 to be viewed by the user based on the captured image 60 obtained by performing imaging by the image pickup device 14.
  • the information acquisition unit 56 of the CPU 24A acquires the request information 64 requesting the generation of the ornamental image 68.
  • the request information 64 includes the setting of the ornamental image 68, that is, the setting information 64-2 indicating the gaze position 80 of the virtual viewpoint image 62 included in the ornamental image 68.
  • the virtual viewpoint image generation unit 57 of the CPU 24A generates a virtual viewpoint image 62 based on the acquired gaze position 80.
  • the ornamental image generation unit 58 of the CPU 24A uses the generated virtual viewpoint image 62 and the user information 64-3 regarding the same user as the setting information 64-2 in the request information 64 of a plurality of users, and the user information 64. Generate an ornamental image 68 reflecting -3. Therefore, according to this configuration, it is possible to easily generate sympathy among users who are viewing the ornamental image 68, as compared with the case where the user is allowed to view the unprocessed virtual viewpoint image 62 as it is.
  • the ornamental image 68 includes a virtual viewpoint image 62 created based on the captured image 60. Therefore, according to this configuration, the user can view the ornamental image 68 including the virtual viewpoint image 62 observed from a free viewpoint as compared with the case where the ornamental image 68 does not include the virtual viewpoint image 62.
  • the setting information 64-2 includes gaze position specifying information capable of specifying the gaze position 80 used for generating the virtual viewpoint image 62 in the region indicated by the captured image 60. Therefore, according to this configuration, it is possible to easily generate sympathy among users who are viewing the ornamental image 68 including the same virtual viewpoint image 62.
  • the gaze position 80 is the position of the gaze object 78 included in the region indicated by the captured image 60. Therefore, according to this configuration, it is easy to generate sympathy among a plurality of users who are viewing the ornamental image 68 including the virtual viewpoint image 62 generated based on the gaze position specifying information indicating the same gaze object 78. Can be done.
  • the ornamental image generation unit 58 generates the ornamental image 68 by superimposing the user information 64-3 regarding the user having the same setting information 64-2 on the virtual viewpoint image 62. do. Therefore, according to this configuration, the presence of the ornamental image 68 can be enhanced as compared with the case where the user information 64-3 relating to the same user as the setting information 64-2 is not superimposed on the virtual viewpoint image 62.
  • the ornamental image generation unit 58 includes the voice 90 in which the setting information 64-2 is related to the same user and the comment 92 in which the setting information 64-2 is related to the same user.
  • an ornamental image 68 reflecting the user information 64-3 is generated. Therefore, according to this configuration, the same virtual viewpoint is compared with the case where the voice 90 related to the same user with the setting information 64-2 or the comment 92 related to the same user with the setting information 64-2 is not added. It is possible to easily generate sympathy among users who are viewing the ornamental image 68 including the image 62.
  • the ornamental image generation unit 58 reflects the user information 64-3 by adding the face image 76 whose setting information 64-2 can visually identify the same user.
  • the ornamental image 68 is generated. Therefore, according to the present configuration, the user viewing the ornamental image 68 including the same virtual viewpoint image 62 as compared with the case where the ornamental image 68 does not include the face image 76 that can visually identify the user. It is possible to easily create empathy between them.
  • the ornamental image 68 is a moving image
  • the ornamental image generation unit 58 sets at least one of the audio 90 and the comment 92 by the user at the time of reproducing the ornamental image 68. It is added to the ornamental image 68 at the timing. Therefore, according to this configuration, at least one of the audio 90 and the comment 92 is for ornamental use according to the scene of the ornamental image 68, as compared with the case where at least one of the audio 90 and the comment 92 is not added to the ornamental image 68 at the timing set by the user. It is possible to easily generate empathy among users who are viewing the image 68.
  • the ornamental image generation unit 58 stores the user information 64-3 in the memory 24C, and the ornamental image 68 reflecting the user information 64-3 stored in the memory 24C. Generate. Therefore, according to this configuration, it is not necessary for the user to input the user information 64-3 every time the ornamental image 68 is viewed, as compared with the case where the user information 64-3 is not stored in the memory 24C.
  • the user information 64-3 includes an attribute related to the user's preference. Therefore, according to this configuration, it is possible to generate the ornamental image 68 according to the user's preference as compared with the case where the ornamental image 68 is not generated by using the attribute related to the user's preference.
  • the request information 64 includes the user information 64-3. Therefore, according to this configuration, the setting information 64-2 included in the request information 64 and the user information 64-3 can be associated and stored in the memory 24C.
  • the gaze position 80 used to generate the virtual viewpoint image 62 is the position of the gaze object 78, and the gaze position specifying information is the coordinates and radius of the gaze object 78. Not limited to this.
  • the gaze position 80 may be coordinates indicating an area in the soccer field 18 arbitrarily designated by the user.
  • the gaze position specifying information may be the coordinates of the gaze position 80.
  • the gaze position specifying information may be the viewpoint position 82 of the virtual viewpoint, the line-of-sight direction 84, and the angle of view 86.
  • the gaze position specifying information capable of specifying the gaze position 80 may include the gaze position path 98 indicating the path of the gaze position 80.
  • the gaze position path 98 can be said to be a set in which a plurality of gaze positions 80 are linearly connected.
  • the gaze position path 98 corresponds to the locus of the player's movement.
  • the virtual viewpoint path 99 becomes a path as shown in FIG. Further, as shown in FIG.
  • the gaze position path 98 may be displayed on the bird's-eye view image 97 and superimposed on the ornamental image 68.
  • the gaze position path 98 is an example of "gaze position path information" according to the technique of the present disclosure.
  • the gaze object 78 is a specific player selected by the user as a player of interest on the information acquisition screen 66, but the technique of the present disclosure is not limited to this.
  • the gaze object 78 may be an object such as a ball, a goal, a line, or a pole, or may be an object arbitrarily designated by the user from an area within the soccer field 18.
  • the gaze position 80A of the user A is set to a position including a specific player
  • the gaze position 80C of the user C is set to a position including a soccer goal.
  • the ornamental image generation unit 58 is used to gaze position 80.
  • an ornamental image 68 may be generated that reflects user information 64-3 about a user whose gaze position 80 or gaze position path 98 is within a predetermined range.
  • the ornamental image generation unit 58 displays the user information 64-3 of the users A and C as a virtual viewpoint image. By adding to 62, an ornamental image 68 is generated.
  • the ornamental image generation unit 58 is at least one of the viewpoint position 82, the line-of-sight direction 84, and the angle of view 86. Either one may generate an ornamental image 68 that reflects user information 64-3 regarding users within the default range. Further, the ornamental image generation unit 58 may generate an ornamental image 68 reflecting user information 64-3 regarding a user whose viewpoint position 82, line-of-sight direction 84, and angle of view 86 are all within a predetermined range. ..
  • the default range is a value derived as a distance between the gaze positions 80 where a similar virtual viewpoint image 62 is generated by, for example, a test using an actual machine and / or a computer simulation.
  • the similar virtual viewpoint image 62 is, for example, an image in which the same player can be seen.
  • the predetermined range may be a range determined without computer simulation, or may be within a range of numerical values roughly determined, for example, within 2 meters in real space.
  • the gaze position path 98 may be within the predetermined range, for example, the distance between the paths may be within a roughly determined numerical range such as within 2 meters on average.
  • the defined range is, for example, within 1 meter in the real space with respect to the viewpoint position 82 determined by the user. It is within 3 degrees with respect to the line-of-sight direction 84 determined by the user, and within 10 degrees with respect to the angle of view 86 determined by the user.
  • the specified range is not limited to these examples. Further, the specified range may be changed by the user. By allowing the user to change the default range, for example, if the default range is narrow in the initial setting and another user within the default range cannot be found in the setting information 64-2, the default range can be changed. It will be possible to find another user.
  • the default range is an example of a "default range" according to the technique of the present disclosure.
  • step ST107 is replaced with step ST120 in the ornamental moving image generation process performed by the CPU 24A. That is, in step ST120, the ornamental image generation unit 58 adds the user information 64-3 of the user whose gaze position path 98 or gaze position 80 is within the predetermined range to the virtual viewpoint image 62, and the user is used as the ornamental image 68. Send to device 16. Since the other steps are the same as those shown in FIG. 10, the description thereof will be omitted.
  • the ornamental image 68 including the virtual viewpoint image 62 generated based on the similar gaze position path 98 is viewed. It is possible to easily generate empathy among users. Further, according to the configuration in which the position of the object arbitrarily designated by the user is determined as the gaze position 80, the ornamental image 68 including the virtual viewpoint image 62 generated based on the gaze position 80 within the predetermined range is viewed. It is possible to easily generate empathy among existing users.
  • the setting information 64-2 includes the gaze position specifying information that can specify the gaze position 80 used for generating the virtual viewpoint image, but in the second embodiment, the setting information 64-2 2 contains information on which of the plurality of moving images obtained by performing imaging by the plurality of imaging devices 14 is to be viewed.
  • the CPU 24A generates the ornamental moving image 168 by superimposing the user information 64-3 regarding the user whose setting information 64-2 is within the predetermined range on the moving image to be watched.
  • the information processing system 100 includes a first image pickup device 14-1, a second image pickup device 14-2, a third image pickup device 14-3, and a fourth image pickup device. It is equipped with 14-4.
  • the first to fourth image pickup devices 14-1 to 14-4 are arranged one by one on each of the four wall surfaces surrounding the soccer field 18 having a substantially rectangular shape.
  • the first to fourth image pickup devices 14-1 to 14-4 take an image of the area in the soccer field 18 as an image pickup area.
  • the first image pickup apparatus 14-1 transmits the moving image acquired by performing the imaging to the information processing apparatus 12 as the first moving image 60-1.
  • the second image pickup device 14-2 transmits the moving image acquired by performing the image pickup to the information processing device 12 as the second moving image 60-2.
  • the third image pickup apparatus 14-3 transmits the moving image acquired by performing the imaging to the information processing apparatus 12 as the third moving image 60-3.
  • the fourth image pickup apparatus 14-4 transmits the moving image acquired by performing the imaging to the information processing apparatus 12 as the fourth moving image 60-4.
  • the first to fourth moving images 60-1 to 60-4 are stored in the storage 24B via the image pickup device communication I / F32.
  • the first to fourth moving images 60-1 to 60-4 are examples of "plurality of moving images" according to the technique of the present disclosure.
  • the CPU 24A of the information processing apparatus 12 operates as an information acquisition unit 156 and an ornamental moving image generation unit 158 according to the ornamental moving image generation program 154 to execute an ornamental moving image generation process.
  • the information acquisition unit 156 when the information acquisition unit 156 receives the instruction information 64-1 transmitted from at least one of the plurality of user devices 16, the information acquisition unit 156 is shown in FIG. 17 as an example.
  • the information acquisition screen 166 is generated.
  • the information acquisition unit 56 transmits the generated information acquisition screen 166 to the user device 16 which is the output source of the instruction information 64-1.
  • the user device 16 receives the information acquisition screen 166 and displays the received information acquisition screen 166 on the display 42.
  • FIG. 17 shows an information acquisition screen 166 displayed on the display 42 of the user device 16 of the user A.
  • the information acquisition screen 166 differs from the information acquisition screen 66 according to the first embodiment in that it does not have a selection button 74 for selecting a player of interest.
  • the information input to the information acquisition screen 166 is transmitted from the user device 16 to the information processing apparatus 12 as the user information 64-3.
  • the information acquisition unit 56 receives the user information 64-3 transmitted from the user device 16 and stores it in the memory 24C.
  • the information acquisition unit 156 generates the moving image selection screen 167 shown in FIG. 18 as an example.
  • the information acquisition unit 156 transmits the generated moving image selection screen 167 to the user device 16 which is the output source of the user information 64-3.
  • the user device 16 receives the video selection screen 167 and displays the received video selection screen 167 on the display 42.
  • the moving image selection screen 167 On the moving image selection screen 167, the first to fourth moving images 60-1 to 60-4 acquired by being imaged by the first to fourth imaging devices 14-1 to 14-4 are displayed.
  • the user selects any one of the first to fourth moving images 60-1 to 60-4 on the moving image selection screen 167 using the pointer 40B of the mouse 40A as the moving image to be viewed.
  • the first moving image 60-1 is selected.
  • the moving image selection information indicating the first moving image 60-1 is transmitted from the user device 16 to the information processing apparatus 12.
  • the information acquisition unit 56 receives the moving image selection information transmitted from the user device 16 and stores it in the memory 24C as the setting information 64-2 of the user A.
  • the moving image selection information is an example of "information regarding which of a plurality of moving images is to be viewed" according to the technique of the present disclosure.
  • the ornamental video generation unit 158 generates the ornamental video 168 by superimposing the user information 64-3 regarding the same user with the setting information 64-2 on the video selected as the video to be viewed.
  • the ornamental moving image generation unit 158 acquires the first moving image 60-1 corresponding to the setting information 64-2 of the user A from the storage 24B. do. Further, the ornamental moving image generation unit 158 acquires the user information 64-3 regarding the user (user C and others) who has set the same setting information 64-2 as the user A from the memory 24C. The ornamental moving image generation unit 158 generates the ornamental moving image 168 by superimposing the user information 64-3 acquired from the memory 24C on the first moving image 60-1 acquired from the storage 24B. The ornamental moving image generation unit 158 transmits the generated ornamental moving image 168 to the user device 16 of the user A.
  • the ornamental moving image generation process shown in FIG. 20 is realized by the CPU 24A executing the ornamental moving image generation program 154. Further, the ornamental moving image generation process shown in FIG. 20 is started when the CPU 24A receives instruction information 64-1 from at least one of the plurality of user devices 16.
  • step ST201 the information acquisition unit 156 generates an information acquisition screen 166, and the generated information acquisition screen 166 is a user device that is an output source of the instruction information 64-1. Send to 16.
  • the user device 16 receives the information acquisition screen 166 and displays the received information acquisition screen 166 on the display 42. After that, the ornamental moving image generation process proceeds to step ST202.
  • step ST202 the information acquisition unit 156 determines whether or not the user information 64-3 requested on the information acquisition screen 166 has been input.
  • the determination is affirmed and the ornamental moving image generation process shifts to step ST203. If the user information 64-3 is not input in step ST202, the determination is denied and the ornamental moving image generation process shifts to step ST202.
  • step ST203 the information acquisition unit 156 generates a video selection screen 167, and transmits the generated video selection screen 167 to the user device 16 which is the output source of the instruction information 64-1.
  • the user device 16 receives the moving image selection screen 167 and displays the received moving image selection screen 167 on the display 42. After that, the ornamental moving image generation process shifts to step ST204.
  • step ST204 the information acquisition unit 156 determines whether or not the video to be viewed is selected on the video selection screen 167. When the moving image to be viewed is selected in step ST204, the determination is affirmed and the viewing moving image generation process proceeds to step ST205. If the moving image to be viewed is not selected in step ST204, the determination is denied and the viewing moving image generation process shifts to step ST204.
  • step ST205 the ornamental moving image generation unit 158 has the same setting information 64-2 as the user information 64-3 of the user who is the output source of the instruction information 64-1 and the user who is the output source of the instruction information 64-1.
  • the user information 64-3 of the user having the above is added to the selected moving image and transmitted as an ornamental moving image 168 to the user device 16 of the user who is the output source of the instruction information 64-1.
  • the ornamental moving image generation process proceeds to step ST206.
  • steps ST206 to ST209 are the same as ST108 to ST111 of the ornamental moving image generation processing shown in FIG. 10, the description thereof will be omitted.
  • the setting information 64-2 is obtained by performing imaging by the first to fourth imaging devices 14-1 to 14-4, and the first to fourth moving images are obtained. Contains information on which of 60-1 to 60-4 will be viewed. Therefore, according to this configuration, it is possible to easily generate sympathy among users who are watching the ornamental moving image 168, as compared with the case where the user is allowed to watch the unprocessed moving image as it is.
  • the ornamental moving image generation unit 158 generates the ornamental moving image 168 by superimposing the user information 64-3 regarding the user having the same setting information 64-2 on the moving image to be viewed. do. Therefore, according to this configuration, it is possible to enhance the presence of the ornamental moving image 168 as compared with the case where the user information 64-3 relating to the same user as the setting information 64-2 is not superimposed on the moving image to be viewed.
  • the number of image pickup devices provided in the soccer field 18 is four, but the technique of the present disclosure is not limited to this, and even if the number of image pickup devices is more than four, the number of image pickup devices is not limited to this. It may be less.
  • the ornamental moving image generation unit 158 sets the user information 64-3 of the user who designates the moving image obtained by the imaging device whose distance between the imaging devices is within the predetermined range as the setting information 64-2 into the moving image to be viewed. By superimposing it, an ornamental moving image 168 may be generated.
  • the ornamental moving image generation unit 158 of the user who designates the moving image obtained by imaging a similar area in the soccer field 18 as the setting information 64-2 regardless of the distance between the imaging devices.
  • the ornamental moving image 168 may be generated by using the user information 64-3.
  • the default range is a value derived as a distance between image pickup devices from which similar moving images are acquired by, for example, a test using an actual machine and / or a computer simulation.
  • the default range is an example of a "default range" according to the technique of the present disclosure.
  • the setting information 64-2 is among the first edited video 160-1 and the second edited video 160-2 created based on the first to fourth videos 60-1 to 60-4. Contains information about which one is to be viewed.
  • the differences from the second embodiment will be described.
  • the same configurations as those of the second embodiment are represented by the same reference numerals as those of the second embodiment, and the description thereof will be omitted for the same configurations and operations as those of the second embodiment.
  • the editor 112 creates the first and second edited videos 160-1 and 160-2 based on the first to fourth videos 60-1 to 60-4 by using the computer 24 included in the information processing device 12. do.
  • the first edited video 160-1 is, for example, a video for fans of the Japan team, which includes contents specialized for the Japan team. Support and / or commentary, etc. specialized for the Japanese team may be added to the first edited video 160-1 as audible data or visible data.
  • the second edited video 160-2 is an image for fans of the England team, which includes contents specialized for the England team. Support and / or commentary, etc.
  • the editor 112 stores the created first and second edited moving images 160-1 and 160-2 in the storage 24B.
  • the first and second edited videos 160-1 and 160-2 are examples of "plurality of edited videos" according to the technique of the present disclosure.
  • the first edited video 160-1 and the second edited video 160-2 are displayed on the moving image selection screen 167.
  • the user selects any one of the first edited moving image 160-1 and the second edited moving image 160-2 by using the pointer 40B of the mouse 40A as the moving image to be viewed.
  • FIG. 22 shows a moving image selection screen 167 displayed on the user device 16 of the user A, and the first edited moving image 160-1 is selected.
  • the ornamental video generation unit 158 generates the ornamental video 168 by superimposing the user information 64-3 regarding the same user with the setting information 64-2 on the edited video selected as the video to be watched.
  • the ornamental moving image generation unit 158 transmits the first edited video 160-1 corresponding to the setting information 64-2 of the user A from the storage 24B. get. Further, the ornamental moving image generation unit 158 acquires the user information 64-3 regarding the user (user C and others) who has set the same setting information 64-2 as the user A from the memory 24C. The ornamental moving image generation unit 158 generates the ornamental moving image 168 by superimposing the user information 64-3 acquired from the memory 24C on the first edited moving image 160-1 acquired from the storage 24B. The ornamental moving image generation unit 158 transmits the generated ornamental moving image 168 to the user device 16 of the user A.
  • the setting information 64-2 is obtained by performing imaging by the first to fourth imaging devices 14-1 to 14-4, and the first to fourth moving images are obtained. It contains information on which of the first edited video 160-1 and the second edited video 160-2 created based on 60-1 to 60-4 is to be viewed. Therefore, according to this configuration, it is possible to easily generate sympathy among users who are watching the edited video, as compared with the case where the user is allowed to watch the unprocessed edited video as it is.
  • the ornamental moving image generation unit 158 superimposes the user information 64-3 regarding the user having the same setting information 64-2 on the edited moving image to be viewed, thereby displaying the ornamental moving image 168. Generate. Therefore, according to this configuration, it is possible to enhance the presence of the ornamental moving image 168 as compared with the case where the user information 64-3 relating to the same user as the setting information 64-2 is not superimposed on the edited moving image to be viewed.
  • the number of edited videos is two, but the technique of the present disclosure is not limited to this, and the number of edited videos may be three or more.
  • the ornamental video generation unit 158 uses the user information 64-3 of the user who designates the similar edited video as the setting information 64-2.
  • the ornamental moving image 168 may be generated by superimposing it on the edited moving image to be viewed.
  • the ornamental video generation unit 158 generates the ornamental video 168 by superimposing the user information 64-3 regarding the user whose setting information 64-2 is within the default range on the edited video to be viewed. May be good.
  • the default range is a range defined as the similarity between the edited moving images is equal to or higher than the threshold value.
  • the default range is an example of a "default range" according to the technique of the present disclosure.
  • the face image 76 acquired by the camera 48 is exemplified as an image in which the setting information 64-2 can visually identify the user within the predetermined range, but the technique of the present disclosure is limited to this. Not done.
  • the image in which the setting information 64-2 can visually identify the user within the default range may be an image acquired by an image pickup device other than the camera 48, and if the image can identify the user, the user's image may be used. It may be an avatar image, an illustration image, or an image other than a face.
  • the user information 64-3 is acquired via the information acquisition screen 66 or 166, but the technique of the present disclosure is not limited to this, and the user information 64-3 is the instruction information 64-1. May be registered in the information processing apparatus 12 by the user in advance before the output of. Further, the user information 64-3 does not necessarily have to be acquired, and only the setting information 64-2 may be acquired. In this case, instead of superimposing the user information 64-3, for example, the number of users whose setting information 64-2 is the same or within the specified range is displayed as the viewing image 68, the viewing video 168, and the first edited video 160. It may be displayed together with -1 or the second edited moving image 160-2.
  • the number of users whose setting information 64-2 is the same or within the specified range is an example of "viewer information" according to the technique of the present disclosure. Further, depending on the number of users whose setting information 64-2 is the same or within the specified range, for example, an object such as an audience may be added and displayed in an superimposed manner.
  • the soccer field 18 is illustrated, but this is only an example, and is a baseball field, a rugby field, a curling field, an athletic field, a swimming pool, a concert hall, an outdoor music field, and a theater venue. As long as a plurality of physical cameras can be installed, the place may be any place.
  • the computer 24 is exemplified, but the technique of the present disclosure is not limited to this.
  • a device including an ASIC, FPGA, and / or PLD may be applied.
  • a combination of a hardware configuration and a software configuration may be used.
  • the ornamental moving image generation program 54 or the ornamental moving image generation program 154 is stored in the storage 24B, but the technique of the present disclosure is not limited to this, and as shown in FIG. 24 as an example.
  • the ornamental moving image generation program 54 or 154 may be stored in any portable storage medium 200.
  • the storage medium 200 is a non-temporary storage medium. Examples of the storage medium 200 include SSD and USB memory.
  • the ornamental moving image generation program 54 or 154 stored in the storage medium 200 is installed in the computer 24, and the CPU 24A executes the ornamental moving image generation process according to the ornamental moving image generation program 54 or 154.
  • the ornamental moving image generation program 54 or 154 is stored in the program memory of another computer or server device connected to the computer 24 via the communication network (not shown), and the information processing device 12 requests.
  • the ornamental moving image generation program 54 or 154 may be downloaded to the information processing apparatus 12. In this case, information processing based on the downloaded ornamental moving image generation program 54 or 154 is executed by the CPU 24A of the computer 24.
  • processors can be used as hardware resources for executing information processing.
  • Examples of the processor include, as described above, software, that is, a CPU, which is a general-purpose processor that functions as a hardware resource that executes information processing according to a program.
  • a dedicated electric circuit which is a processor having a circuit configuration specially designed for executing a specific process such as FPGA, PLD, or ASIC can be mentioned.
  • a memory is built in or connected to any processor, and each processor executes information processing by using the memory.
  • the hardware resource that executes information processing may be composed of one of these various processors, or a combination of two or more processors of the same type or different types (for example, a combination of a plurality of FPGAs or a CPU). And FPGA). Further, the hardware resource for executing information processing may be one processor.
  • one processor is configured by a combination of one or more CPUs and software, and this processor performs information processing.
  • this processor performs information processing.
  • SoC there is a form of using a processor that realizes the functions of the entire system including a plurality of hardware resources for executing information processing with one IC chip.
  • information processing is realized by using one or more of the above-mentioned various processors as hardware resources.
  • a and / or B is synonymous with "at least one of A and B". That is, “A and / or B” means that it may be only A, it may be only B, or it may be a combination of A and B. Further, in the present specification, when three or more matters are connected and expressed by "and / or", the same concept as “A and / or B" is applied.
  • An information processing device that generates an ornamental image to be viewed by a plurality of viewers based on an image obtained by performing imaging by the image pickup device.
  • the above processor Acquires the request information for requesting the information processing device to generate the ornamental image, and obtains the request information.
  • the generation process to generate the ornamental image is executed according to the acquired request information, and the generation process is executed.
  • the request information includes setting information indicating the setting of the ornamental image and viewer information regarding the viewer viewing the ornamental image, and is information corresponding to each of the plurality of viewers.
  • the generation process is a process of generating the ornamental image in which the setting information of the viewer information is reflected in the viewer information within the default range in the request information of the plurality of viewers. Device.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Computer Hardware Design (AREA)
  • Human Computer Interaction (AREA)
  • Computer Graphics (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Geometry (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Controls And Circuits For Display Device (AREA)
  • User Interface Of Digital Computer (AREA)
  • Image Generation (AREA)
  • Studio Devices (AREA)
  • Processing Or Creating Images (AREA)
PCT/JP2021/023652 2020-07-31 2021-06-22 情報処理装置、情報処理方法、及びプログラム Ceased WO2022024594A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2022540066A JP7546676B2 (ja) 2020-07-31 2021-06-22 情報処理装置、情報処理方法、及びプログラム
US18/156,361 US20230156244A1 (en) 2020-07-31 2023-01-18 Information processing apparatus, information processing method, and program

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020-131167 2020-07-31
JP2020131167 2020-07-31

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/156,361 Continuation US20230156244A1 (en) 2020-07-31 2023-01-18 Information processing apparatus, information processing method, and program

Publications (1)

Publication Number Publication Date
WO2022024594A1 true WO2022024594A1 (ja) 2022-02-03

Family

ID=80035443

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/023652 Ceased WO2022024594A1 (ja) 2020-07-31 2021-06-22 情報処理装置、情報処理方法、及びプログラム

Country Status (3)

Country Link
US (1) US20230156244A1 (https=)
JP (1) JP7546676B2 (https=)
WO (1) WO2022024594A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2024017779A (ja) * 2022-07-28 2024-02-08 キヤノン株式会社 画像処理装置、画像処理方法及びコンピュータプログラム

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7581019B2 (ja) * 2020-11-11 2024-11-12 キヤノン株式会社 画像処理装置、画像処理方法、およびプログラム
US12003806B2 (en) * 2021-03-11 2024-06-04 Quintar, Inc. Augmented reality system for viewing an event with multiple coordinate systems and automatically generated model

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010232860A (ja) * 2009-03-26 2010-10-14 Sony Corp 情報処理装置、コンテンツ処理方法及びプログラム
JP2014215828A (ja) * 2013-04-25 2014-11-17 シャープ株式会社 画像データ再生装置、および視点情報生成装置
JP2018046448A (ja) * 2016-09-15 2018-03-22 キヤノン株式会社 画像処理装置および画像処理方法

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11289524A (ja) * 1998-04-01 1999-10-19 Nippon Telegr & Teleph Corp <Ntt> 仮想空間会議方法およびこの方法を記録した記録媒体
JP2000165831A (ja) 1998-11-30 2000-06-16 Nec Corp 多地点テレビ会議システム
JP2003333572A (ja) 2002-05-08 2003-11-21 Nippon Hoso Kyokai <Nhk> 仮想観客形成装置およびその方法、仮想観客形成受信装置およびその方法ならびに仮想観客形成プログラム
JP2007328389A (ja) 2006-06-06 2007-12-20 Reishi Tonuma 仮想空間表示方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010232860A (ja) * 2009-03-26 2010-10-14 Sony Corp 情報処理装置、コンテンツ処理方法及びプログラム
JP2014215828A (ja) * 2013-04-25 2014-11-17 シャープ株式会社 画像データ再生装置、および視点情報生成装置
JP2018046448A (ja) * 2016-09-15 2018-03-22 キヤノン株式会社 画像処理装置および画像処理方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2024017779A (ja) * 2022-07-28 2024-02-08 キヤノン株式会社 画像処理装置、画像処理方法及びコンピュータプログラム

Also Published As

Publication number Publication date
US20230156244A1 (en) 2023-05-18
JP7546676B2 (ja) 2024-09-06
JPWO2022024594A1 (https=) 2022-02-03

Similar Documents

Publication Publication Date Title
JP7715305B2 (ja) プログラム、表示を提供する方法、通信端末および画像通信システム
JP6756269B2 (ja) 通信端末、画像通信システム、通信方法、及びプログラム
JP6907861B2 (ja) 通信端末、画像通信システム、表示方法、及びプログラム
US10944619B2 (en) Communication terminal, image communication system, and display control method
US10764513B2 (en) Communication terminal, image data communication system, and communication method
JP6309749B2 (ja) 画像データ再生装置および画像データ生成装置
US20230156244A1 (en) Information processing apparatus, information processing method, and program
US12389120B2 (en) Communication terminal, image communication system, and method of displaying image
JP2025098206A (ja) 情報処理装置、情報処理装置の作動方法、及びプログラム
US12347055B2 (en) Visual effects and content enhancements for VR
WO2021149336A1 (ja) 情報処理装置、情報処理方法、及びプログラム
US12548273B2 (en) Visual effects and content enhancements for VR
JP7017045B2 (ja) 通信端末、表示方法、及びプログラム
US11863902B2 (en) Techniques for enabling high fidelity magnification of video
JP7389140B2 (ja) 情報処理装置、情報処理方法、及びプログラム
JP7829041B2 (ja) 高忠実度倍率を可能にするインタラクティブビデオ再生技法
CN109804620A (zh) 用于生成捕获图像的显示设备和方法
JP2019036785A (ja) 情報処理装置、動画配信方法、及び動画配信プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21848875

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022540066

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21848875

Country of ref document: EP

Kind code of ref document: A1