WO2022091215A1 - 映像配信装置、映像配信システム、映像配信方法、及びプログラム - Google Patents

映像配信装置、映像配信システム、映像配信方法、及びプログラム Download PDF

Info

Publication number
WO2022091215A1
WO2022091215A1 PCT/JP2020/040259 JP2020040259W WO2022091215A1 WO 2022091215 A1 WO2022091215 A1 WO 2022091215A1 JP 2020040259 W JP2020040259 W JP 2020040259W WO 2022091215 A1 WO2022091215 A1 WO 2022091215A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
public place
video
teaching
terminal device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/040259
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
出 瀬之口
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Amatelus
Amatelus Inc
Original Assignee
Amatelus
Amatelus Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Amatelus, Amatelus Inc filed Critical Amatelus
Priority to JP2022517484A priority Critical patent/JP7208695B2/ja
Priority to PCT/JP2020/040259 priority patent/WO2022091215A1/ja
Priority to EP20959733.5A priority patent/EP4240019A4/en
Publication of WO2022091215A1 publication Critical patent/WO2022091215A1/ja
Priority to JP2022207147A priority patent/JP7677639B2/ja
Priority to US18/139,397 priority patent/US20230269411A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/34—Indicating arrangements 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/235—Processing of additional data, e.g. scrambling of additional data or processing content descriptors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
    • H04N21/25866—Management of end-user data
    • H04N21/25875—Management of end-user data involving end-user authentication
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/437—Interfacing the upstream path of the transmission network, e.g. for transmitting client requests to a VOD server
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47217—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/27—Server based end-user applications
    • H04N21/274—Storing end-user multimedia data in response to end-user request, e.g. network recorder
    • H04N21/2743—Video hosting of uploaded data from client
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462—Content or additional data management e.g. creating a master electronic programme guide from data received from the Internet and a Head-end or controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4627—Rights management associated to the content

Definitions

  • the present invention relates to a technique for using free-viewpoint video data, etc., and particularly for setting and opening a public place, which is a place for editing free-viewpoint video data, generating teaching files for autopilot, and publishing them. Regarding the related technology.
  • a live view image acquisition unit that is wirelessly or wiredly connected to a plurality of cameras capable of capturing moving images and acquires one or a plurality of live view images from the plurality of cameras, and a live view image acquisition unit that acquires the images.
  • a display unit that displays one or more live view images, an operation unit that manually switches the live view image to be displayed on the display unit, and an operation history record that records operation history information that indicates the operation history of the operation unit.
  • one moving image is automatically generated based on the plurality of moving images captured by the plurality of cameras and the operation history information recorded in the operation history recording unit. It is equipped with a video editing unit that edits the video to be created.
  • a usage authority management device for a content sharing system that manages the usage authority of the content when sharing the content in the sharing unit of the sharing group (see, for example, Patent Document 3).
  • Patent Document 1 discloses a technique for combining moving images of each designated camera taken by a plurality of cameras each time, and edits the moving images to generate a teaching file for autopilot. The point to be done is not disclosed.
  • Patent Document 2 merely discloses that video editing for automatically creating one video based on a plurality of videos captured by a plurality of cameras and operation history information is performed. It is not disclosed that annotations such as camera and voice are added to the video and that the editing result is distributed as a teaching file for autopilot.
  • Patent Document 3 there is no disclosure or suggestion that the content can be distributed by opening a public place having a hierarchical nested structure in which the authority such as login is individually defined by the user's setting. not.
  • the present invention has been made in view of such a problem, and an object of the present invention is to provide a technique for setting and opening a hierarchical content disclosure place in which authority such as login is defined and distributing the content. To provide.
  • the video distribution system is a video distribution system including a video distribution device and a user's terminal device, and the video distribution device is from the terminal device.
  • the public place setting unit that sets the public place that publishes content data including at least free-viewpoint video data and teaching files so that it can be distributed, and the user about the content data of the public place.
  • the terminal device has a determination unit for executing the process, and the terminal device has a request unit for making a request for distribution of the content data of the public place of the video distribution device, and the video distribution device. It has an acquisition unit for acquiring the distributed content data and a display unit for displaying an image based on the content data.
  • the video distribution method is a video distribution method using a video distribution device and a user's terminal device, based on the public place setting information transmitted from the terminal device by the video distribution device.
  • the public place for distributing content data including free viewpoint video data and teaching files is set and the user's terminal device requests distribution of the content data in the public place.
  • the processing is executed and the terminal device performs the processing.
  • the content data of the public place of the video distribution device is requested to be distributed, the content data distributed from the video distribution device is acquired, and the video based on the content data is displayed.
  • the video distribution device is a video distribution device capable of communicating with a user's terminal device, and is at least free viewpoint video data and teaching based on public place setting information transmitted from the terminal device.
  • the public place setting unit that sets the public place that publishes the content data including the file so that it can be distributed. It has a determination unit that determines whether or not the processing related to the request is possible based on the authority of the user and the authority set in the public place, and if the conditions are satisfied, executes the processing.
  • the program according to the fourth aspect of the present invention uses a computer capable of communicating with a user's terminal device to provide content data including at least free viewpoint video data and a teaching file based on public place setting information transmitted from the terminal device.
  • the public place setting unit that sets the public place to be made publicly available for distribution, and the authority of the user who made the request at least when the terminal device of the user requests the content data of the public place. And, based on the authority set in the public place, it is determined whether or not the process related to the request is possible, and if the condition is satisfied, it functions as a determination unit for executing the process.
  • FIG. 15A is a diagram for defining and explaining a type of content
  • FIG. 15B is a diagram for defining and explaining a type of authority.
  • 16 (a) is a diagram showing an example of a user table
  • FIG. 16 (b) is a diagram showing an example of a public place table
  • FIG. 16 (c) is a diagram showing an example of a content table.
  • FIG. 1 shows and describes a configuration of a video distribution system according to an embodiment of the present invention.
  • the video distribution device 1, the editor's terminal device 2, and the viewer's terminal device 3 are connected wirelessly or by wire to a communication network 4 such as the Internet.
  • the video distribution device 1 may be realized by one or more server devices, a computer, or the like.
  • the terminal device 2 of the editor there are various types of terminals such as smartphones, tablet terminals, notebook personal computers, desktop personal computers, head-mounted displays, etc., which can accept operation input and display information. Things can be adopted.
  • the viewer's terminal device 3 is a terminal capable of accepting operation input and displaying information, such as a smartphone, a tablet terminal, a notebook personal computer, a desktop personal computer, and a head mount display. If there is, various things can be adopted.
  • the video distribution device 1 when the video distribution device 1 receives a request from the editor's terminal device 2, it transmits free-viewpoint video data or the like obtained by shooting a subject with a plurality of cameras to the editor's terminal device 2. ..
  • a predetermined editing screen which will be described in detail later, is displayed, and while viewing the free viewpoint video data, video switching (viewpoint switching), enlargement, reduction, and various annotations (characters, figures, symbols) are displayed. , And audio, etc.), and the teaching data, which is the editing result, is transmitted to the video distribution device 1.
  • the teaching data is transmitted from the terminal device 2 of each editor to the video distribution device 1.
  • the video distribution device 1 Upon receiving the teaching data, the video distribution device 1 generates a teaching file for the autopilot based on the teaching data. Further, the video distribution device 1 presents the teaching file for the autopilot to the terminal device 3 of the viewer so that it can be distributed. This presentation may be performed on a dedicated website or the like, or may be performed on a screen displayed by executing an application program on the terminal device 2.
  • autopilot means switching the viewpoint and moving the playback time position by automatic operation based on the composition of the teaching file without any operation by the viewer regarding the display of free viewpoint video data. It means to display by performing such as.
  • Live autopilot refers to the sequential generation and distribution of teaching files for autopilot after a predetermined time has elapsed, or as soon as possible, and live distribution of free-viewpoint video data (live distribution). ), It can be performed regardless of the distribution form such as on-demand distribution.
  • the video distribution device 1 When the instruction file for the autopilot is selected on the website provided by the video distribution device 1 on the terminal device 3 of the viewer and the distribution request is made, the video distribution device 1 is used for the selected autopilot.
  • the teaching file is distributed to the viewer's terminal device 3. At this time, depending on the authority of the viewer, it may be decided whether or not the teaching file can be distributed and whether or not the teaching file can be displayed. Therefore, for example, the presence / absence of the teaching file can be displayed to all, but the distribution of the teaching file can be made possible only by the person who has the purchased authority, or the teaching file itself can be hidden if there is no purchased authority.
  • the viewer's terminal device 3 When the viewer's terminal device 3 receives the instruction file for the autopilot, it reproduces the free viewpoint video based on the instruction file. Conventionally, the terminal device 3 of the viewer reproduces the free viewpoint image while switching to the desired viewpoint or the like, but according to the present embodiment, it is useful while automatically switching the viewpoint or the like. Reproduction is realized. As for the acquisition of the content accompanying the reproduction of the free-viewpoint video, for example, online streaming, download, or combined may be performed.
  • the viewer's terminal device 3 can freely play the free viewpoint video even if it is not in a communication environment, and edits them.
  • the teaching file can be regenerated.
  • the video distribution device 1 opens a public place, for example, on a website or the like, which is a place for publishing various contents such as free-viewpoint video data and teaching files in response to a request from a user's terminal devices 2 and 3.
  • the public place setting information is received from the terminal device 2 of the editor, the terminal device 3 of the viewer, etc., and the public place having a hierarchical nested structure is opened based on the setting information. ..
  • a user is defined as an editor, a viewer, and a general user. Since various permissions such as login to the site, streaming of content, and download can be set for the user, the public place, and the content, the video distribution device 1 receives various requests for the public place. Executes processing based on authority (for example, download, etc.). That is, even a general user can open a public place or the like as long as he / she has acquired the relevant authority. Details will be described later.
  • FIG. 2 shows and describes a detailed configuration of a video distribution device in a video distribution system.
  • the video distribution device 1 including a server device or the like has a control unit 10 that controls the entire control, a RAM (RandomAccessMemory) 11 and a ROM (ReadOnlyMemory) 12 as memories, and an MPEG. It has a decoding module 13, a storage unit 14 including an HDD (Hard Disc Drive), an SSD (Solid State Drive), a flash memory, and an I / O port 15, and these are connected to a bus line.
  • a router 17 is connected to the I / O port 15 via the HUB 16.
  • the control unit 10 may be composed of, for example, a CPU (Central Processing Unit), a microprocessor (Microprocessor), a multiprocessor (Multiprocessor), an ASIC, an FPGA, or the like.
  • the storage unit 14 has a content storage unit 14a, an operation data storage unit 14b, a teaching file storage unit 14c, a public place information storage unit 14k, and a user information storage unit 14l.
  • the content storage unit 14a stores free-viewpoint video data, still image data obtained by dividing the free viewpoint video data, and the like.
  • the operation data storage unit 14b stores operation data and the like sent from the editor's terminal device 2.
  • the teaching file storage unit 14c stores the generated teaching file for the autopilot.
  • the public place information storage unit 14k stores public place setting information, which will be described in detail later.
  • the user information storage unit 14l stores information such as user attribute information, authority, and browsing history.
  • the storage unit 14 further stores the OS 14d, the data acquisition program 14e, the data generation program 14f, the teaching file generation program 14g, the selection program 14h, the distribution program 14i, and the content generation program 14j. Therefore, by executing the distribution program 14i, the control unit 10 functions as a distribution unit 10a, a public place setting unit 10h, a determination unit 10i, a notification unit 10j, and a relevance suggestion unit 10k, and executes the data acquisition program 14e. By doing so, it functions as an acquisition unit 10b, by executing the data generation program 14f, it functions as a data generation unit 10c, and by executing the selection program 14h, it functions as a specified value reception unit 10d and a selection unit 10e, and is a teaching file. By executing the generation program 14g, it functions as a teaching file generation unit 10f. By executing the content generation program 14j, it also functions as a content generation unit 10g. In addition, it functions as a settlement unit 10l under OS14d.
  • the acquisition unit 10a acquires a plurality of video data as free viewpoint video data via the I / O port 15.
  • the acquisition unit 10a acquires a plurality of moving image data obtained by capturing images of the subject from different directions.
  • the acquired free viewpoint video data is stored in the content storage unit 14a.
  • the data generation unit 10c generates still image data by extracting frames at predetermined time intervals as still images for each of the free viewpoint video data acquired by the acquisition unit 10b, that is, a plurality of video data. More specifically, the data generation unit 10c decompresses the moving image data stored in the content storage unit 14a by the MPEG decoding module 13, prepares it as a set of still image data, and stores it in the content storage unit 14a. At this time, each still image data is saved in association with time data indicating the timing at which each image was taken.
  • the designated value receiving unit 10d receives the direction designated value (operation data) for designating the position data in the still image data to be browsed from the terminal device 3 of the viewer.
  • the selection unit 10e selects still image data along with time data based on the direction designation value received by the designated value reception unit 10d, and transmits the still image data to the viewer's terminal device 3 via the communication network 4.
  • the viewer's terminal device 3 receives the still image data and generates an image.
  • the teaching file generation unit 10f generates a teaching file for the autopilot based on the teaching data from the terminal device 2 of the editor, and stores it in the teaching file storage unit 14c.
  • the file structure of the teaching file will be described in detail later.
  • the distribution unit 10a reads the designated autopilot instruction file from the instruction file storage unit 14c in accordance with the distribution request from the viewer's terminal device 3, and transmits the instruction file to the viewer's terminal device 3 via the communication network 4. ..
  • the corresponding content data (including the divided still image data) may be transmitted at the same time, or the corresponding content data may be transmitted each time in the browsing process.
  • the content generation unit 10g generates the content of the free viewpoint video data or the content such as the streaming data of the moving image based on the free viewpoint video data and the teaching file. This content is also transmitted to the viewer's terminal device 3 by the distribution unit 10a.
  • the public place setting unit 10h opens a public place having a hierarchical nested structure based on the public place setting information from the terminal devices 2 and 3, and enables distribution of contents.
  • the determination unit 10i downloads, etc., based on the user's authority, the public place authority, and the content authority. Determine whether processing is possible.
  • the notification unit 10j notifies the public, etc. of the free viewpoint video data or the like published in the public place when the free viewpoint video data of another viewpoint is uploaded.
  • the relevance suggestion unit 10k suggests relevance when a related moving image or the like exists when the content is being played. Then, the payment unit 10l makes electronic payments related to sales and the like for the contents uploaded to the public place.
  • FIG. 3 shows and describes the configuration of the editor's terminal device 2 in the video distribution system.
  • the viewer's terminal device 3 has the same basic configuration.
  • the editor's terminal device 2 (the same applies to the viewer's terminal device 3) includes a control unit 21, a RAM 22, a ROM 23, a JPEG decoding module 24, an I / O port 25, and the like. It has a wireless communication unit 26, a drawing unit 27, a display monitor 28, an operation recognition unit 29, an operation unit 30, a storage unit 31, an image pickup unit 32, and a GPS unit 33. Each part is connected via a bus line.
  • the control unit 10 may be composed of, for example, a CPU, a microprocessor, a multiprocessor, an ASIC, an FPGA, or the like.
  • the storage unit 31 is an HDD or a flash memory.
  • the storage unit 31 has a content storage unit 31a, an operation data storage unit 31b, and a teaching file storage unit 31c.
  • the content storage unit 31a stores free-viewpoint video data, still image data, and the like sent from the video distribution device 1.
  • the operation data storage unit 31b stores the operation data when the viewpoint or the like of the reproduced video is changed based on the screen operation.
  • the teaching file storage unit 31c stores the teaching file sent from the video distribution device 1 and the teaching data generated at the time of editing.
  • the storage unit 31 stores the OS 31d, the browser program 31e, the editing program 31f, and the teaching file generation program 31g. Therefore, the control unit 21 functions as a request unit 21a, an acquisition unit 21d, a transmission unit 21f, a code analysis unit 21g, and a position information acquisition unit 21h based on the OS 31d, and executes the browser program 31e to generate a video generation unit. It functions as 21b, and functions as an editorial unit 21c by executing the editing program 31f.
  • the storage unit 31 stores the touch panel control firmware.
  • the control unit 21 functions as the teaching file generation unit 21e by executing the teaching file generation program 31g.
  • each part performs the following processing. That is, the requesting unit 21a requests the video distribution device 1 for free viewpoint video data (including divided still image data). The transmission of this request is performed by the wireless communication unit 26 connected via the I / O port 25. A wired communication unit may be provided instead of the wireless communication unit 26.
  • the acquisition unit 21d acquires free viewpoint video data (including divided still image data) sent from the video distribution device 1.
  • the video generation unit 21b generates content that can be displayed on the terminal device 3 from the free viewpoint video data.
  • the drawing unit 27 controls the display on the display monitor 28.
  • the acquired still image data is decoded by the JPEG decoding module 24.
  • the editorial unit 21c changes the viewpoint of free viewpoint video data (including divided still image data), screen allocation, enlargement / reduction, and change of playback speed based on the operation by the editor on the screen described in detail later.
  • Editing processing including addition of annotations (characters, figures, symbols, voices, etc.) is executed, teaching data is generated, and stored in the teaching file storage unit 31c.
  • the operation recognition unit 29 recognizes the operation of the operation unit 30 and stores it in the operation data storage unit 31b as operation data including the direction designation value.
  • the teaching file generation unit 21e generates a teaching file for autopilot based on the teaching data and stores it in the teaching file storage unit 31c.
  • the transmission unit 21f transmits the content data (for example, video streaming data, etc.) of the content storage unit 31a, the teaching data, the teaching file, and the like to the video distribution device 1 via the wireless communication unit 26.
  • each part when playing back based on the teaching file for the autopilot, each part performs the following processing. That is, the requesting unit 21a requests the video distribution device 1 for the teaching file for the autopilot.
  • the acquisition unit 21d acquires the teaching file for the autopilot sent from the video distribution device 1.
  • content data such as three-dimensional point cloud data, three-dimensional CG, moving image data, and still image data necessary for reproduction may be acquired.
  • the image generation unit 21b generates an image based on the teaching file, and the drawing unit 27 reproduces the moving image on the display monitor 28. If the teaching file contains annotation data or the like, the audio, text, figures, etc. are played back at the timing defined in the teaching file as well as the moving image.
  • the operation recognition unit 29 recognizes the operation and relates to the direction specified value. Generate operation data.
  • the request unit 21a transmits the operation data related to the direction designation value to the video distribution device 1 and requests the change of the viewpoint or the like.
  • the acquisition unit 21d acquires free viewpoint video data (including divided still image data) whose viewpoint has been changed from the video distribution device 1, playback based on the teaching file is temporarily stopped and playback with a different viewpoint is performed. Is executed.
  • the code analysis unit 21g analyzes a two-dimensional code such as a QR code (registered trademark) imaged by the image pickup unit 32 and acquires code information.
  • This code information may include, for example, a seat number, position information associated with the seat, and the like.
  • the position information acquisition unit 21h acquires position information based on the communication environment of the wireless communication unit 26 or the data acquired from the GPS unit 33. This location information may be uploaded to the public place in association with the generated content.
  • FIG. 4 shows and describes an example of an editing screen displayed on the editor's terminal device 2.
  • a file of free viewpoint video data that can be selected as an edit target is presented in the area 100a of the edit screen 100, and the editor can edit the free viewpoint video data (in this example, the division). (Still image data) can be selected.
  • a chat can be written in the area 100b, and when the editing work is divided by a plurality of editors, the work can be carried out while communicating with each other.
  • edit logs and the like can be displayed in the area 100b, and unnecessary edits can be invalidated according to the authority, and the invalidated edits can be undone.
  • a display area or function for voice call, video chat, etc. may be provided separately.
  • playback display is performed based on the selected free viewpoint video data.
  • the free viewpoint video data selected as the editing target in the area 100a is divided into predetermined units, and each division unit is shown together with a thumbnail or the like. In this example, the selected split unit is indicated by a dashed line.
  • various annotations can be added to each division unit by operating the operation unit 30.
  • Reference numeral 100e indicates that annotations such as text and figures have been added, and reference numeral 100f indicates that voice annotations have been added.
  • Reference numeral 100g indicates the current position of the live when editing is performed so as to follow the live distribution. In addition to the above, it may be possible to display how much the live is delayed from the current live, the remaining time until the arbitrary time when the live can be played after an arbitrary time, and the like.
  • the moving image data D1 is composed of a plurality of frames F1, F2, F3 ....
  • the data generation unit 10c of the video distribution device 1 may divide the moving image data into a plurality of frames and store the moving image data in the content storage unit 14a in units of the divided data. For example, as shown in FIG. 5B, when the data is divided into three frames, the divided data D2 divided into frames F1 to F3 and the divided data D2 divided into frames F4 to F6. , Multiple frames of moving image data are sequentially divided.
  • the data generation unit 10c may divide the moving image data into a plurality of frames and one frame and store the moving image data in the content storage unit 14a in units of the divided data.
  • the divided data includes a plurality of divided data (D2M) composed of a plurality of frames and a single divided data (D2S) composed of one frame. ).
  • the data generation unit 10c divides the moving image data so that one plurality of divided data and the plurality of single divided data are alternately arranged in time series, and stores the moving image data in the content storage unit 14a. You may.
  • the multi-sheet division data (D2M) divided into frames F1 to F3, the single-sheet division data D2S divided into frames F4, and the single-sheet division into frame F5.
  • the single-sheet divided data and the plurality of single-sheet divided data are divided so as to alternate in time series.
  • the video data B is divided into the divided data D2-A1, D2-A2, D2-A3, D2-A4, etc. obtained by dividing the moving image data A.
  • the divided data D2-B1, D2-B2, D2-B3, D2-B4 may be taken at different imaging times.
  • the distribution unit 10a when the distribution unit 10a sequentially transmits the divided data D2-A1 and D2-A2 based on the moving image data A and then receives the switching request from the viewer's terminal device 3, the distribution unit 10a receives the switching request.
  • the distribution unit 10a sequentially transmits the divided data D2-A1 and D2-A2 based on the moving image data A, and then receives the switching request from the viewer's terminal device 3. Then, the divided data D2-B2, which has the same timing as the divided data D2-A2 in terms of time, is read from the content storage unit 14a, and then the divided data D2-D3 ... It may be read from the storage unit 14a and transmitted sequentially.
  • the distribution unit 10a Since information on the imaging time is added to each moving image data, the distribution unit 10a reads out the divided data and other divided data in a timely continuous or substantially continuous manner based on the information on the imaging time. Delivery is possible.
  • the teaching file includes screen teaching data, content teaching data, and annotation teaching data.
  • FIG. 7 shows and explains the structure of the screen teaching data included in the teaching file.
  • the screen teaching data includes an object type, an object ID / URL, a teaching data object ID, time adjustment data, and screen allocation data.
  • the object type is screen.
  • the object ID / URL is an object ID in the teaching data.
  • the time adjustment data is data for operating with the time code including the adjustment time when the screen allocation data includes the time code.
  • the screen allocation data is basically the same as the screen allocation data of the content teaching data described later.
  • FIG. 8 shows and explains the structure of the content teaching data included in the teaching file.
  • the content teaching data includes pilot time code, object type, object ID / URL, teaching data object ID, pilot time code arrival action, content end action, specified time code arrival action, and start. It has a time code, an end time code, viewpoint-related data, playback speed data, zoom-related data, and screen allocation data.
  • the pilot time code defines the start time on the autopilot.
  • the object type is content.
  • the object ID / URL is an ID / URL that uniquely indicates the content on the system.
  • the teaching data object ID is an object ID in the teaching data.
  • the action at the time of reaching the pilot time code can specify the start position of the time code of the content or the action when the time in the pilot time code reaches the set start time code. For example, play, stop, video effect, etc. are specified. In the action when the specified time code is reached, it is determined whether the pilot time code or the time code of the content is used as the base point for each specified action, and the action to be executed when the time code as the base point is reached or passed is specified.
  • the term “passing” means, for example, when the pilot time code jumps from the 8th second to 15 seconds at once with a seek bar or the like for the sound that should have been played after 10 seconds on the pilot time code.
  • An example of the behavior is that the sound is reproduced from an appropriate audio position as long as it is within the audio reproduction range.
  • the behavior at the time of passing also behaves in an appropriate state with respect to the action at the time of reaching the pilot time code, the action at the end of the content, the action at the end of the pilot time code described later, etc., which are related to the time code. be.
  • the start time code is the playback start time on the content
  • the end time code is the playback end time. If the start time code and end time code are specified retroactively, reverse playback will occur.
  • the viewpoint-related information is information that can be specified according to the distribution method of the free viewpoint video. For example, in the case of the still image transmission method and in the case of the moving image transmission method, it becomes the camera ID, and in the case of the multi-camera method. Is a multi-camera ID, and in the case of 3D point group data or 3D CG, it is, for example, a 4 ⁇ 4 view conversion matrix.
  • the format does not matter as long as it is an expression method that can specify the position of the camera, the direction in which the camera is facing (the gazing point), and the posture of the camera.
  • the reproduction speed can be defined as 0.125, 0.25, 0.5, 0, 1, 1.25, 1.5, 2, 4, etc. from stop to shift reproduction.
  • the screen allocation data is the allocation data when displaying a plurality of contents on one screen.
  • the screen allocation data specify the basic position such as screen top left, screen top right, screen bottom left, screen bottom right, top, bottom, etc., specify in pixel units, set the ratio of the display area to the entire screen, etc. can do.
  • the display area is not limited to a rectangle, and a figure such as a perfect circle, a Pezier curve, a spline curve, a plurality of straight lines, or a polyline can be specified.
  • a wipe it is possible to superimpose another content on one content and display it. It is also possible to specify one or more time codes and the corresponding display area form at that time.
  • the time code the time when the display time of the screen object is reached can be specified as 0 seconds or the like, and it is of course possible to specify the time code based on the autopilot time code.
  • the above is the basic configuration of the content teaching data, but it can also be configured with only the pilot time code, teaching data object ID, and viewpoint-related data, and can be the minimum configuration when expressing the content.
  • it is composed only of pilot time code, teaching data object ID, start time code, end time code, and viewpoint-related data, and one or more time codes and the corresponding viewpoint-related information at that time are included in the viewpoint-related data. It can also be the minimum configuration.
  • FIG. 9 shows and explains the structure of annotation teaching data (voice) included in the teaching file.
  • the annotation teaching data (voice) includes pilot time code, object type, object ID / URL, teaching data object ID, pilot aim code arrival action, content end action, and specified time code arrival time. It has an action, a start time code, an end time code, a playback speed, and data.
  • the pilot time code is the start time on the autopilot.
  • the object type is content.
  • the object ID / URL is an ID / URL that uniquely indicates the position of the data on the system.
  • the teaching data object ID is an object ID on the teaching data. Playback, stop, video effects, etc. can be specified as actions when the pilot time code is reached.
  • an action to be performed when the time code at which the reproduction of the content ends is reached can be specified.
  • the start time code is the playback start time on the voice
  • the end time code is the playback end time on the voice.
  • the reproduction speed can be defined as 0.125, 0.25, 0.5, 0, 1, 1.25, 1.5, 2, 4, etc. from the reproduction stop to the shift reproduction.
  • the playback speed specified in the teaching data can be specified without affecting each other.
  • the playback speed of audio is specified without affecting the playback speed specified for the content. It is possible. For example, content is at 2x speed and audio is at 1x speed.
  • FIG. 10 shows and explains the structure of annotation teaching data (character strings, figures, images, etc.) included in the teaching file.
  • the annotation teaching data (character string, figure, image, etc.) is the pilot time code, end pilot time code, object type, object ID / URL, teaching data object ID, and action when the pilot time code is reached. , Pilot time code end action, specified time code arrival action, annotation action, time adjustment data, data, and screen allocation data.
  • the pilot time code is the start time on the autopilot.
  • the end pilot time code is the end time on the autopilot.
  • the object type is content.
  • the object ID / URL is an ID / URL that uniquely indicates the position of the data on the system.
  • the teaching data object ID is an object ID in the teaching data. Playback, stop, video effects, etc. can be specified as actions when the pilot time code is reached. Video effects, audio playback, video wipe display playback, etc. can be specified as actions at the end of the pilot time code. In the action when the specified time code is reached, it is determined whether the pilot time code or the time code of the content is used as the base point for each specified action, and the action to be executed when the time code as the base point is reached or passed is specified.
  • an action to be taken when a predetermined voice is input from a click, a tap, a microphone, etc. in the display area can be specified.
  • This action includes, for example, arbitrary audio output, returning the time of the pilot time code, stopping the playback of the content for a predetermined time and outputting the audio during that time, video effects, video playback, and the like.
  • the above-mentioned actions can be appropriately specified for actions at the time of reaching the pilot time code, actions at the end of the pilot time code, actions at the end of the content, actions at the time of reaching the specified time code, etc. Of course.
  • Time adjustment data is data for operating with a time code that takes into account the adjustment time.
  • the data it is possible to specify a character string, a figure, an image, etc. to be displayed, a display position, a display style, and the like. If the screen allocation data is not set, it is displayed on the entire display screen with a layer above the content.
  • the teaching file including the screen teaching data, the content teaching data, and the annotation teaching data is delivered as a video when the teaching data generated based on the operation on the editing screen of FIG. 4 described above is sent from the editor's terminal device 2.
  • the device 1 receives it, the teaching file generation unit 10f generates it based on the teaching data, and stores it in the teaching file storage unit 14c.
  • the generated instruction file for the autopilot is published on the web site or the like operated by the video distribution device 1 and is provided as appropriate. Therefore, the viewer's terminal device 3 wants to view the instruction file from the instruction file. To receive.
  • the viewer's terminal device 3 stores the received teaching file in the teaching file storage unit 31c, and based on the teaching file, the video generation unit 21b generates content that can be displayed on the terminal device 3 and displays the display monitor 28. Playback display is performed above.
  • the teaching file for the autopilot defines the viewpoint, playback speed, presence / absence of zoom, screen allocation, etc. of the content (for example, divided still image data), playback is executed according to these specified contents. ..
  • the instruction file for the autopilot also contains annotation instruction data such as voice and characters, the reproduction of characters and voice is performed at a specified timing in synchronization with the reproduction according to the annotation instruction data. Can be done. Therefore, if the viewer acquires a teaching file for the autopilot that matches his / her taste and purpose, he / she can automatically obtain an opportunity to browse the content suitable for himself / herself without changing his / her viewpoint. Will be.
  • the playback of the autopilot is temporarily interrupted, and the screen based on the user operation is performed. Switching and playback are executed.
  • this point will be described in detail with reference to FIG.
  • the autopilot playback it is possible to shift playback, rewind, move to an arbitrary autopilot time code, etc. in the autopilot time code, and further, except for switching the viewpoint, the autopilot does not have to be interrupted.
  • the above-mentioned user operation includes an operation of pressing the UI such as a button of suspending the autopilot.
  • the vertical axis is the identification number as the direction data
  • the horizontal axis is the time data
  • the file names of the still image data corresponding to the vertical axis and the horizontal axis are shown.
  • the displayed still image data changes according to the user operation as shown in the lower part of FIG. 11. That is, it means that the still image data corresponding to the cell through which the arrow indicated by the solid line passes in the figure is displayed on the terminal device 3 of the viewer.
  • the moving images are sequentially reproduced in chronological order.
  • the designated value receiving unit 10d receives the direction specified value by the swipe operation of the viewer during the execution of the automatic reproduction
  • the automatic reproduction based on the teaching file for the autopilot is temporarily suspended, and the selection unit 10e specifies the direction.
  • the still image data (K005) whose direction specified value is specified by the swipe operation is displayed, the state is temporarily stopped. Then, when the user presses the playback start button again, the still image data corresponding to the direction designation value at that time is continuously played. It should be noted that the still image data corresponding to the direction specified value at that time may be continuously reproduced without being stopped once.
  • the selection unit 10e selects the still image data corresponding to the same time data frame by frame as the direction specified value changes, as described above. Select in the order of identification numbers (K100 to F100). Then, after the still image data (F100) designated by the swipe operation is displayed, if the stop state is not once stopped, the still image data corresponding to the same direction specified value is continuously reproduced.
  • the embodiment in which the selection unit 10e selects the same time data as the still image data based on the time data (t 5p) of the corresponding still image data when the direction designation value is received.
  • the time data next to the current time data may be sequentially selected. In this case, the moving image can be maintained in the played state without being stopped even during the swipe.
  • the selection unit 10e is stationary so that the direction data are continuously connected when the designated value receiving unit 10d does not accept the direction specified value and the amount of change in the direction specified value per unit time is less than the threshold value.
  • Select image data On the other hand, the selection unit 10e selects the still image data so that the direction data is intermittently connected when the change amount of the direction designation value per unit time is equal to or more than the threshold value.
  • intermittently means to acquire only a part of the directional data that are continuously arranged.
  • the operation recognition unit 29 determines that the operation amount by the swipe operation is large due to the user 40 moving the finger large or fast, the still image data corresponding to the adjacent direction data is not acquired. Still image data corresponding to distant direction data can be acquired from the original direction data. Then, the terminal device 3 displays the still image data of the subject at the direction designation value that changes based on the direction designation operation during the direction designation operation by the still image data received from the selection unit 10e. On the other hand, when the direction designation operation is not performed, the terminal device 3 corresponds to the completion position by sequentially receiving and displaying the still image data of the direction designation value corresponding to the completion position of the direction designation operation in chronological order. Display a pseudo video from the direction you want to.
  • control unit 21 executes the editing program 31f in the terminal device 2 of the viewer, and the editing screen 100 as shown in FIG. 4 is displayed on the display monitor 28.
  • the display mode is not limited to FIG.
  • the requesting unit 21a requests the video distribution device 1 to distribute the free-viewpoint video data (S1).
  • the acquisition unit 10b receives the distribution request (S2), and the distribution unit 10a reads the free viewpoint video data related to the distribution request from the content storage unit 14a and directs the viewer to the terminal device 2.
  • Deliver (S3) the free viewpoint video data related to the distribution request from the content storage unit 14a and directs the viewer to the terminal device 2.
  • the viewer's terminal device 2 receives this free viewpoint video data (S4), the video generation unit 21b generates content that can be displayed on the terminal device 2, and the area of the editing screen 100 displayed on the display monitor 28.
  • the selected video is displayed in the 100c and the area 100d (S5).
  • the divided still image data is displayed together with thumbnails and the like in a manner in which the viewer can grasp the division unit.
  • the image selected as the editing target is reproduced.
  • the editorial unit 21c executes the editing process (S6).
  • the details of the editing process will be described in detail later, but for example, selection of divided still image data (viewpoint information), reproduction speed, addition of various annotations, and the like will be performed.
  • the editing unit 21c stores, for example, teaching data conceptually including screen teaching data, content teaching data, and annotation teaching data in the teaching file storage unit 31c and transmits the teaching data to the video distribution device 1. (S7).
  • the acquisition unit 10b receives the teaching data and stores it in the teaching file storage unit 14c (S8), and receives the teaching data from the terminal devices 2 of all the editors (S9 is Yes).
  • the teaching file generation unit 10f generates a teaching file based on the stored teaching data (S10) and stores it in the file storage unit 14c (S11). In this way, a series of processes related to the generation of the teaching file for the autopilot is completed.
  • the teaching file for the autopilot stored in the teaching file storage unit 14c is published to the viewer in a selectable manner on a predetermined website or the like.
  • the editorial unit 21c determines whether or not the content has been selected (S6-1), and when the content is selected (branches S6-1 to Yes), teaches the content teaching data. It is stored in the file storage unit 31c (S6-2). If no content is selected (S6-1 is branched to No), the process proceeds to step S6-3.
  • the content teaching data stored in the teaching file storage unit 31c in step S6-2 includes a pilot time code, an object type, an object ID / URL, a teaching data object ID, an action when the pilot time code is reached, an action when the content ends, and a designated time. It has a code arrival action, a start time code, an end time code, viewpoint-related information, playback speed, zoom-related information, and screen allocation information. These details are as described above.
  • the editorial unit 21c determines whether or not the annotation (character) has been added (S6-3), and when the annotation (character) is added (branches S6-3 to Yes), the annotation is taught. Data (characters) are stored in the teaching file storage unit 31c (S6-4). If the annotation (character) is not added (branch S6-3 to No), the process proceeds to step S6-5.
  • the annotation teaching data (characters) stored in the teaching file storage unit 31c in step S6-4 are the pilot time code, the end pilot time code, the object type, the object ID / URL, the teaching data object ID, and the action when the pilot time code is reached. , Action at the end of the pilot time code, action at the time of reaching the specified time code, annotation action, time adjustment, data, and screen allocation information. These details are as described above.
  • the editorial unit 21c determines whether or not the annotation (figure, symbol, etc.) has been added (S6-5), and when the annotation (figure, symbol, etc.) is added (S6-5 Yes. (Branch to), and the annotation teaching data (graphics, symbols, etc.) are stored in the teaching file storage unit 31c (S6-6). If no annotation (figure, symbol, etc.) is added (S6-5 is branched to No), the process proceeds to step S6-7.
  • the annotation teaching data (figures, symbols, etc.) stored in the teaching file storage unit 31c in step S6-6 are a pilot time code, an end pilot time code, an object type, an object ID / URL, a teaching data object ID, and a pilot time code. It has arrival action, pilot time code end action, specified time code arrival action, annotation action, time adjustment, data, and screen allocation information. These details are as described above.
  • the editorial unit 21c determines whether or not the annotation (voice) has been added (S6-7), and when the annotation (voice) is added (branches S6-7 to Yes), the annotation is taught.
  • the data (voice) is stored in the teaching file storage unit 31c (S6-8). If the annotation (voice) is not added (branch S6-7 to No), the process proceeds to step S6-9.
  • the annotation teaching data (voice) stored in the teaching file storage unit 31c in step S6-8 includes a pilot time code, an object type, an object ID / URL, a teaching data object ID, an action when the pilot aim code is reached, and an action when the content ends. , Action when the specified time code is reached, start time code, end time code, playback speed, and data.
  • the editorial unit 21c determines whether or not all the edits have been completed (S6-9), and if all the edits have not been completed (branches S6-9 to No), step S6-1.
  • the edit process is completed and the process returns to the process after step S8 in FIG.
  • the video distribution device 1 presents a plurality of teaching files for autopilot in a selectable manner on the website.
  • the acquisition unit 21d acquires the teaching file and executes reproduction based on the teaching file (S21).
  • the requesting unit 21a requests the video distribution device 1 to distribute free viewpoint video data (including divided still image data and the like) taught by the content teaching data. (S22).
  • the video distribution device 1 receives a distribution request (S23), and the distribution unit 10a reads out the corresponding free viewpoint video data from the content storage unit 14a and distributes it (S24).
  • the viewer's terminal device 3 receives the free viewpoint video data (S25), the video generation unit 21b generates content that can be displayed on the terminal device 3 based on the free viewpoint video data, and displays the content on the display monitor 28 for playback.
  • Do (S26).
  • it is determined whether or not a user operation for example, a screen swipe operation
  • S27 is if there is no operation (S27 is branched to No), until the end of reproduction (S34).
  • the control unit 21 transmits the operation data (including the direction specified value) to the video distribution device (S28).
  • the video distribution device 1 receives the operation data and stores it in the operation data storage unit 14b (S29).
  • the selection unit 10e selects the free viewpoint video data (still image data) whose direction designation value is designated by the user operation based on the time when the direction designation value is received (S30), and the distribution unit 10a is selected.
  • the free viewpoint video data (still image data) is distributed to the viewer's terminal device 3 (S31).
  • the acquisition unit 21d receives the selected free viewpoint video data (still image data) (S32), and the video generation unit 21b generates content that can be displayed on the terminal device 3. ,
  • the display of the display monitor 28 is switched (S33).
  • the control unit 21 determines whether or not to end the reproduction (S34), returns to the step S22 if the reproduction is not terminated, repeats the above process, and if the reproduction is terminated, a series of steps. Ends the processing of.
  • the end of this reproduction is the end of the automatic reproduction based on the teaching file for the autopilot, and the end of the reproduction when the autopilot is temporarily interrupted based on the user operation and the reproduction is performed based on the user operation. including.
  • the data that the video distribution device 1 can publish in the public place arbitrarily and hierarchically set by the user are classified as shown in FIG. 15A, for example. That is, first is free viewpoint video data, second is a teaching file, third is free viewpoint video data and teaching file, and fourth is a moving image file generated by free viewpoint video data and teaching file.
  • first is free viewpoint video data
  • second is a teaching file
  • third is free viewpoint video data and teaching file
  • fourth is a moving image file generated by free viewpoint video data and teaching file.
  • the teaching file and the moving image file generated by the teaching file or the like will be referred to as an autopilot file.
  • the data published in the public place in this way can be streamed, downloaded, etc. by the terminal device 2 of another user based on his / her own authority.
  • the distribution device 1 determines whether or not to download or the like based on the authority given to the data that can be disclosed and the authority of another user such as viewing.
  • the data that can be disclosed and the authority given to other users such as viewing are classified as shown in FIG. 15 (b), for example.
  • the determination unit 10i of the distribution device 1 determines whether or not the request is possible based on these permissions (A1 to A13), and if the permissions are satisfied, proceeds with processing according to the request such as download and streaming. Become.
  • a user table is stored in the user information storage unit 14l of the storage unit 14 of the video distribution device 1.
  • An example of this user table is as shown in FIG. 16 (a), for example, attribute information such as a name, an address, and a maid address, and the authority (A1 to A1 to) given to the user in association with the user ID.
  • A13 history information such as downloads and purchases is stored.
  • the public place table is stored in the public place information storage unit 14k of the storage unit 14 of the video distribution device 1.
  • An example of a public place table is, for example, as shown in FIG. 16 (b).
  • the public place table the hierarchy of the public place, the relationship with the upper layer, the authority given to the public place or the content, the disclosure range, the publication deadline, the publication target, the published user, and the public place table are linked to the place ID.
  • the content ID is stored.
  • the content stored in the content storage unit 14a of the storage unit 14 of the video distribution device 1 also stores the content associated with the public place in the content table.
  • An example of this content table is as shown in FIG. 16 (c).
  • the type of content (C1 to C4), content data, relevance information, and user ID of the user who is the creator are stored in association with the content ID.
  • the relevance information is information for suggesting that there is a related free-viewpoint video data or an autopilot file for an arbitrary time code when playing back the free-viewpoint video data or the like.
  • the relevance suggestion unit 10k refers to the relevance information at the time of reproducing the free viewpoint video data and the like, and executes the suggestion at the associated timing.
  • the public place setting unit 10h sets the public place table based on the public place setting information sent from the terminal device 2, etc., and opens the public place.
  • the public place includes the hierarchy and the relationship with the upper layer, the public place explained can have a multi-layer nested structure.
  • Public place (organizer; company name, etc.) -2. Public Place (Event Category) -3. Public place (event) -4. Public place (for upload) That is, in the uppermost layer, the public place is published by the name of the organizer, in the middle layer, the public place is published by the category of the event handled by the organizer, and in the lower layer, the public place is published by the event belonging to each category.
  • a public place for uploading content by browsing users and the like can be published.
  • the authority (A1 to A13) can be given to each public place.
  • the public place can have the following nested structure, for example. 1.
  • the authority can be set as follows for each public place. That is, a user login (ID, password) is required for the music company public place. Content in the public place for each artist can be viewed by anyone who can log in to the public place of the music company. The music video / live video purchaser public place can be logged in by entering a PIN code, etc., and the target free-viewpoint video data can be viewed.
  • ID a user login
  • password a password
  • Content in the public place for each artist can be viewed by anyone who can log in to the public place of the music company.
  • the music video / live video purchaser public place can be logged in by entering a PIN code, etc., and the target free-viewpoint video data can be viewed.
  • this is an example, and the present invention is not limited to this.
  • a user with creation authority can create an autopilot file and upload it to the public place.
  • the user having the sales authority may present the price and receive the settlement by the settlement unit 10l, or may receive the grant of alternative points or the like.
  • the official autopilot file may be sold by a music company that is a content holder.
  • the range of people who can purchase the official teaching and video files can be expanded to the music company public place, or it can be narrowed down to only the music video / live video public place, and the range of people who can purchase after a predetermined number of days has passed. It is also possible to change the price and change the price. These changes are made by updating the public place table by the public place setting unit 10h.
  • the nested structure makes it possible to publish only the autopilot file group without disclosing the free viewpoint video data as shown below.
  • 1. Music company public place -2. Public place for each artist -3. Music video / live video purchaser public place -4. Autopilot file group public place In this case, it is possible to disable the transition to the free viewpoint video data for the autopilot file as an authority.
  • the disclosure of publicly available data uploaded to the public place can be regulated by authority, disclosure range, release deadline, disclosure target, etc. If the scope of disclosure is limited to contributors or groups of contributors, the scope of disclosure can be restricted to those persons by restricting that fact within the scope of disclosure. In this case, when a plurality of users shoot the original data of the free viewpoint video data and upload it to the public place, the plurality of users are in a unit called a group. In addition, if access is restricted according to the authority, it may be restricted by the authority. In addition, it is also possible to set that the SNS is allowed to be the public destination within the public range.
  • the terminal device 2 accepts the input of the public place setting information by the operation of the operation unit 30, for example, by accessing the website or the like provided by the distribution device 1, and the transmission unit 21f receives the public place setting information. It is transmitted to the distribution device 1 (S51).
  • the public place setting information includes information such as the hierarchy of the public place, the upper layer if there is an upper layer, the authority, the disclosure range, the publication deadline, and the disclosure target.
  • the acquisition unit 10b acquires the public place setting information (S52)
  • the distribution device 1 sets the public place setting information in the public place information storage unit 14k (S53). At this time, a place ID is assigned and associated with the user ID of the public user.
  • the notification unit 10j transmits a setting completion notification to the terminal device 2 (S54).
  • the terminal device 2 when the acquisition unit 21d receives the setting completion notification, it displays it on the display monitor 28 (S55). Then, the transmission unit 21f transmits the contents (C1 to C4) to be uploaded to the set public place (S56).
  • the acquisition unit 10b receives the sent content (S57), it is associated with the place ID of the public place, registered in the content storage unit 14a, and registered in the content storage unit 14a, and the public place of the public place information storage unit 14k. Update the contents of the table (S58). In this way, the public place setting unit 10h starts publishing the public place (S59).
  • the request unit 21a sends a place ID and a user ID of the selected public place to the distribution device 1 to make a request for browsing or the like (S61).
  • the determination unit 10i determines whether or not the request is possible (S63). Specifically, the determination unit 10i refers to the user table of the user information storage unit 14l, identifies the user from the user ID, confirms the authority given to the user, and publishes the public place information storage unit 14k. The public place is specified from the place ID by referring to the place table, the authority set for the public place is confirmed, and the approval or disapproval of the request is determined by comparing these authority.
  • the determination unit 10i determines that the user has the authority to request processing such as browsing the public place, the determination unit 10i performs the processing according to the authority (S64). Specifically, if the request is download or streaming, those processes are performed.
  • the terminal device 2 receives the content (S65), the terminal device 2 starts playback or the like (S66).
  • the distribution device 1 establishes a public place that accepts uploads of videos taken by users, 3D CG data, 3D point cloud data, etc. (hereinafter referred to as "original data"), and is based on these original data. You may be prompted to generate free-viewpoint video data or an auto-pilot file. For example, by uploading data taken by a plurality of users of a soccer game at a stadium or the like, it is urged to generate free viewpoint video data or an autopilot file based on the original data.
  • Such a public place may be held as a public place related to a shooting event by the event management side, or may be a public place related to an event arbitrarily created by the user side.
  • the determination unit 10i determines whether or not the public place of the original data can be updated based on the authority.
  • the shooting of the original data in addition to being able to shoot only according to the navigation of the official app distributed by the distribution device 1, it is also possible to shoot with a normal camera, ToF, etc.
  • the disclosure target as part of the public place information, it is possible to define acceptable original data.
  • location information can be associated with the original data. For example, when the original data is obtained by shooting the above-mentioned official application, the position information can be obtained from the ticket or the seat.
  • the shooting location may be specified by reading a two-dimensional code such as a QR code (registered trademark) written on the ticket, and whether or not shooting is possible may be known only after going to the seat.
  • shooting can be restricted based on location information, such as the connection status to the designated WiFi, the reception status of Bluetooth installed at the venue, and the ability to participate if the GPS is within the expected range.
  • position identification communication between terminals (BlueTooth, UWB, etc.), position information such as WiFi, wide area position information such as GPS, matching of feature points between photographed objects (ToF, photogrammetry, etc.) Considerable, direction information, pitch angle, roll angle, yaw angle, sea level information, etc. may be acquired and transmitted if they can be acquired. In this case, these information are managed in association with the original data.
  • the time during shooting the time when the file itself was generated, the time stamp included in the original data, the time information acquired from GPS, etc., included in the shot may be used.
  • the time during shooting is calculated from the match with the position identification information (especially the communication information between terminals at that time, the match of feature points, etc.). Of course, it is also good.
  • the distribution device 1 has the qualification authority (purchased, teaching file only, video only, both, etc.) capable of creating an autopilot file for the data when the free viewpoint video data is published to the public place. You may decide (such as granting authority), and for autopilot file generation, give settings for the autopilot file generation upper limit time (60 seconds, etc.) and / or the lower limit time (10 seconds or more, etc.). Is also possible.
  • the qualification authority purchased, teaching file only, video only, both, etc.
  • the user may be able to set the autopilot file generation upper limit time individually.
  • the information on the upper limit time is associated with the data. For example, the total generation upper limit time (600 seconds, etc.) of the autopilot file that can be generated in one month, the generation upper limit time (30 seconds, etc.) when one autopilot file is generated, and the like. It may be possible to extend the upper limit time by paying some consideration for any generation upper limit time, or by obtaining a certain evaluation for the autopilot generated from another user and raising the rank. For example, it is an image that the total generation upper limit time in the period is extended to 1200 seconds, or the generation upper limit time is extended to 120 seconds.
  • the distribution device 1 may enable the free viewpoint video data of each viewpoint to be selectively acquired when a plurality of viewpoints exist in the free viewpoint video data.
  • a service such as cheaper than buying the 5-viewpoint individually for each viewpoint.
  • you may give the privilege viewpoint or privilege autopilot file for a fee or free of charge.
  • you have the right to create an autopilot file for the free viewpoint video data for a fee or free of charge. You may give it at. As a result, it is possible to purchase only the necessary viewpoints and make it cheaper as a result, or to give motivation to complete all viewpoints.
  • the notification unit 10j acquires the free viewpoint video data or the autopilot file, and the new viewpoint or the autopilot file is added to the data later, the user obtains the free viewpoint video data or the autopilot file. Notify the side (whether notification is possible or not, notification reception conditions may be decided by the user side).
  • the notification method may be Web push, application notification, attention at the time of viewing the data, notification to the user's email, or the like.
  • the notification unit 10j even if the user has not acquired the free viewpoint video data and the autopilot file, the notification unit 10j notifies the user side when the user has made settings such as anxiety. (The notification availability and notification reception conditions may be set by the user).
  • the relevance suggestion unit 10k suggests that there is a related free viewpoint video data or an autopilot file for an arbitrary time code when viewing normal free viewpoint video data.
  • notifications such as autopilot files such as fine play
  • cosmetologists for education when it was difficult to understand the explanation of the cut in a certain time code, a normal video was associated and uploaded. It is like notifying (suggesting) to that effect.
  • the relevance suggestion unit 10k is a player corresponding to the autopilot file viewing, it suggests that there is a related free viewpoint video data or an autopilot file for an arbitrary time code. For example, in the case of sports, it is like an "autopilot file" such as fine play. In this case as well, it is possible to notify (suggest) at any time when viewing the data regardless of the time code.
  • a privilege such as access to free-viewpoint video data or an autopilot file may be given to a physical device (DVD, etc.) or a music video acquired (purchased, etc.) digitally.
  • the user may create an autopilot file so that it can be shared on the public place according to the authority. Whether or not a paid setting can be made can be decided by setting qualifications on the content holder side. You may be able to give points when you get a high evaluation and use it like money in the public place.
  • the content holder side can also publish the autopilot file (the upper limit of the number of publications may be set by the contract plan with our company). You may set the amount by watching for one month as a subscription.
  • there is a wonderful autopilot file created by the user or a highly evaluated one that surpasses the one generated by the content holder it is possible to link it to a mechanism that allows you to request work or recruit. ..
  • the distribution device 1 can also be set as a swipe time based on the instruction of the publisher. When you enter this swipe time, it will automatically stop at any time code (you can stop the sound or select the sound during the swipe time separately), and it will automatically rotate there (not only the movement of the viewpoint but also the time code). It is also possible to proceed at any speed), and when it finishes turning arbitrarily, playback will start automatically, or you will instruct yourself to play. Regarding this swipe time, the viewer log may be taken and skipped from the second time may be arbitrarily selected, or the arbitrary selection may be selected by the setting on the public side.
  • the video distribution system of the present invention can also be used for auditions and contests.
  • an idol audition will be held on the official app for C, and participation will be possible if the participation qualifications, shooting conditions, etc. are met.
  • This audition or contest can be held in tie-up with a company, and if the user has the authority, the audition or contest can be held by setting the disclosure range, granting viewing qualification, etc.
  • a normal user's usage method for example, it can be considered to be used at a school festival or the like.
  • 5 smartphones with the official app for C installed will shoot according to the guide of the official app for C, for example, the 5 smartphones are within the expected range in terms of GPS location information, and each terminal will It is possible to specify a state in which communication is bidirectional by BlueTooth, UWB, etc. and it is guaranteed to exist within the expected range with high accuracy.
  • the size of the subject with respect to the shooting range is attended (please lower it a little so that the entire subject can be included, etc.) and the conditions are not met, shooting cannot be started.
  • one terminal is used as a parent to start shooting, and its children (here, the other four) behave synchronously in response to a shooting start command, a pause command, and a complete stop command.
  • a cout down may be spoken and displayed before the start and stop of shooting. It is possible to have a terminal that participates in shooting as a parent or a terminal that does not participate in shooting as a parent. Furthermore, the parent can be the parent who started shooting first, or can be the parent only for a fixed terminal. In addition, it is possible for a child to pause or completely stop shooting. If the participation qualification is 15 years old or older, 18 years old or younger, etc., it is possible to set a flow for whether or not the participation qualification is made. For example, it is like sending a photo of a face such as a student ID card and a photo of one's own face together. Regarding the participation qualification, there are cases where the participation qualification is judged immediately, and there are cases where only the items necessary for the participation qualification are sent and the participation qualification is judged later after shooting.
  • the organizer can make various settings for the distribution device 1. For example, the free-viewpoint video data and videos of the dance that you want to dance first will be released from a week ago, and you will be asked to dance the dance. It's like having them speak with a gesture. I know what the audition is (such as the role of the movie %), but without prior information on the audition content, the audition content will be announced from the smartphone after the participation starts, and singing and dancing to the music. , It is possible to use various methods such as ad-lib play for flowing lines from smartphones, and it is also possible to set restrictions such as the script etc. being displayed after the start of participation and the shooting for posting must be completed within 1 hour. be.
  • the organizer may set the disclosure range and establish a mechanism to win by the evaluation from the browsing user, or the user takes the initiative until the third selection.
  • the final selection can be decided by the organizer, etc., a tournament ceremony divided into groups, a repechage, etc. are also possible.
  • the video distribution system of the present invention it is possible to utilize free-viewpoint video data by tie-up planning. For example, if you take a picture of the QR code (registered trademark) given to the product when you purchase the soft drink according to the guide of the official app, you can upload the original data, and the tie-up company side selected the original data. It is a usage method such as creating free viewpoint video data from the above and diverting it to campaign videos, commercials, etc.
  • the imaging timing of the original data is a pattern in which shooting is performed by the application at a predetermined date and time (for example, 30 seconds from 23:59:45 on December 31 to 0: 0:15 at the beginning of the year, etc., in each country.
  • the result data created from the original data may be other than the free viewpoint video data. For example, videos of thousands of people may be merged and arranged in tiles.
  • messages and fixed songs are accepted during the time of the earthquake (for example, 1 minute), and the place is like around a street piano in Kobe City.
  • the shooting spots of the free viewpoint video data scattered in various places are displayed on the map, the shot spots can be recognized, and if the publishing authority is turned on, other people participating However, if you know that you shot at that place, or if you have the public authority for data and the public authority for autopilot files turned on, you can have other people see your own free-viewpoint video data, etc. Is also possible.
  • the teaching file for the autopilot can be generated. Therefore, if the terminal device of the viewer acquires the teaching file for the autopilot, the teaching file is the teaching file. Automatic playback based on is possible. At this time, since the teaching file contains various annotation teaching data such as voice, characters, images, and figures, additional effects are automatically reproduced along with the reproduction. Therefore, the viewer can enjoy the reproduction accompanied by the switching of the desired viewpoint or the like only by acquiring and executing the teaching file that meets the needs without switching the viewpoint or the like by himself / herself.
  • Such an autopilot responds to needs such as complicated browsing while changing the viewpoint by oneself, not knowing when it is useful, and wanting to browse in a short time at once.
  • a live video or the like may be first published as free viewpoint video data (including divided still image data), and a teaching file generated based on the teaching data edited by the editor may be published later. It is possible. For example, when handling free-viewpoint video data related to a live performance, it is possible to generate a teaching file or the like that enables reproduction following only a specific artist.
  • the editor's terminal device 2 basically generates various teaching data as described above, and the video distribution device 1 side generates a teaching file for the autopilot. It is also possible to record user operations (eg, swipe operations) on devices 2 and 3 and use them as part of the teaching data.
  • user operations eg, swipe operations
  • the automatic playback can be temporarily interrupted and the viewpoint can be switched based on the user operation.
  • a degree of freedom for example, a swipe operation
  • teaching file for autopilot can be re-edited by forking (branching and copying), merging (combining), cloning (copying), etc., so the published teaching file can be shared by multiple people. By doing so, it can be expected that it will develop into various things.
  • the embodiment of the present invention also includes the following contents.
  • the teaching file for the autopilot may be generated based on the free viewpoint video data in the live distribution (live distribution).
  • the teaching file for the autopilot automatically generated by machine learning may be live-streamed (live-streamed), created by humans (co-editing work is also possible), and live at any time (for example, 5). After a minute delay, etc.), the viewer may be able to play the content from the start of the live with a delay of any time.
  • the editor may view and edit the teaching file for autopilot created by machine learning in a complex manner and perform live distribution (live distribution).
  • the normal free-viewpoint video data may be distributed for a certain period of time, and the teaching file for the autopilot may be live-distributed (live-distributed) again at the stage of creation.
  • teaching file for the autopilot For co-editing of teaching files for autopilot, you can quickly autopilot by performing editing merge using already established co-editing technology, exclusive control on your own timeline, OT (Operational Transformation) method, etc. You may create it.
  • the teaching file for autopilot is automatically generated by machine learning, and the teaching file for autopilot generated by machine learning can be viewed and edited by the editor, auto.
  • a "video file" up to the point where the work is completed while creating the teaching file for the autopilot may be generated, or it may be distributed as a normal video file.
  • information about free-viewpoint video content, etc. included in the components of each time code for the video file can be embedded in the video as metadata (for example, XMP format, etc.), or can be obtained by associating it as a separate file (for example).
  • the location of the file may be described in XMP, etc., for example, in the case of HLS format, the location and contents of the file may be described in the m3u8 file, etc., or an inquiry may be made to a predetermined server.
  • the metadata By referring to the metadata, the content of the file and the location of the file may be obtained), and if the video player supports it, the free-viewpoint video currently being viewed from the video file during playback, etc.
  • the content of the free viewpoint video of the generation source is arbitrarily transitioned to.
  • the free-viewpoint video can be viewed, and the autopilot and live autopilot (both free-viewpoint video and video file) can be restarted by returning to the transition point.
  • the content owner creates it personally (collaboration work is also possible, and it is possible to grant the collaborative editing function to general users based on authority), and a teaching file for live autopilot after a predetermined time has elapsed. May be delivered.
  • the teaching file for the live autopilot may be generated and distributed from the most commonly viewed viewpoint. good.
  • moving image streaming data such as, for example, HLS format may be sequentially generated from the teaching file for the live autopilot and live-distributed (live-distributed).
  • information about free-viewpoint video content, etc. included in the components of each time code for video streaming data can be embedded in the video as metadata (for example, XMP format, etc.) or associated with it as a separate file.
  • the location of the file may be described in XMP, etc., or for example, in the case of HLS format, the location and contents of the file may be described in the m3u8 file, etc., and an inquiry is made to a predetermined server. (You may acquire the contents of the file and the location of the file, etc.), and by referring to the metadata, if it is a compatible video player, the free viewpoint that was currently being viewed from the video file during playback, etc. Of course, it may be possible to transition to the video content and change to an arbitrary viewpoint, and it may be possible to return to the position at the transition point of the moving image file and resume playback.
  • a teaching file for autopilot that realizes content playback with a viewpoint, enlargement, etc. suitable for the user is generated based on the attribute information.
  • it may be provided to the user. For example, in the case of sports, if "ball" or the like is selected, a teaching file for an autopilot that recognizes the ball as an object and always follows it may be generated and distributed.
  • Public place information storage 14l ... User Information storage unit, 15 ... I / O port, 16 ... hub, 17 ... router, 21 ... control unit, 21a ... request unit, 21b ... video generation unit, 21c ... editorial unit, 21d ... acquisition unit, 22 ... RAM, 23 ... ROM, 24 ... PEG code module, 25 ... I / O port, 26 ... wireless communication unit, 27 ... drawing unit, 28 ... display monitor, 29 ... operation recognition unit, 30 ... operation unit, 31 ... storage unit, 31a ... Content storage unit, 31b ... Operation data storage unit, 31c ... Teaching file storage unit, 31d ... OS, 31e ... Browser program, 31f ... Editing program.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Graphics (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Information Transfer Between Computers (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
PCT/JP2020/040259 2020-10-27 2020-10-27 映像配信装置、映像配信システム、映像配信方法、及びプログラム Ceased WO2022091215A1 (ja)

Priority Applications (5)

Application Number Priority Date Filing Date Title
JP2022517484A JP7208695B2 (ja) 2020-10-27 2020-10-27 映像配信装置、映像配信システム、映像配信方法、及びプログラム
PCT/JP2020/040259 WO2022091215A1 (ja) 2020-10-27 2020-10-27 映像配信装置、映像配信システム、映像配信方法、及びプログラム
EP20959733.5A EP4240019A4 (en) 2020-10-27 2020-10-27 VIDEO DISTRIBUTION DEVICE, VIDEO DISTRIBUTION SYSTEM, VIDEO DISTRIBUTION METHOD AND PROGRAM
JP2022207147A JP7677639B2 (ja) 2020-10-27 2022-12-23 映像配信装置、映像配信システム、映像配信方法、及びプログラム
US18/139,397 US20230269411A1 (en) 2020-10-27 2023-04-26 Video distribution device, video distribution system, video distribution method, and program

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/040259 WO2022091215A1 (ja) 2020-10-27 2020-10-27 映像配信装置、映像配信システム、映像配信方法、及びプログラム

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/139,397 Continuation US20230269411A1 (en) 2020-10-27 2023-04-26 Video distribution device, video distribution system, video distribution method, and program

Publications (1)

Publication Number Publication Date
WO2022091215A1 true WO2022091215A1 (ja) 2022-05-05

Family

ID=81382029

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/040259 Ceased WO2022091215A1 (ja) 2020-10-27 2020-10-27 映像配信装置、映像配信システム、映像配信方法、及びプログラム

Country Status (4)

Country Link
US (1) US20230269411A1 (https=)
EP (1) EP4240019A4 (https=)
JP (2) JP7208695B2 (https=)
WO (1) WO2022091215A1 (https=)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6571859B1 (ja) * 2018-12-26 2019-09-04 Ａｍａｔｅｌｕｓ株式会社 映像配信装置、映像配信システム、映像配信方法及び映像配信プログラム
IL295170B2 (en) * 2020-01-30 2026-04-01 Amatelus Inc Device, system, method and software for video distribution
US11968185B2 (en) * 2022-02-25 2024-04-23 Microsoft Technology Licensing, Llc On-device experimentation
US12126699B2 (en) 2022-10-24 2024-10-22 Microsoft Technology Licensing, Llc Superposition-based content serving
US12327150B2 (en) 2022-11-01 2025-06-10 Microsoft Technology Licensing, Llc On-device privatization of multi-party attribution data

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4697468B2 (ja) 2007-01-31 2011-06-08 日本電気株式会社 利用権限管理装置、コンテンツ共有システム、コンテンツ共有方法、及びコンテンツ共有用プログラム
JP2011228845A (ja) * 2010-04-16 2011-11-10 Canon Inc 画像処理装置および方法
JP2015177394A (ja) 2014-03-17 2015-10-05 カシオ計算機株式会社 画像処理装置、画像処理方法及びプログラム
JP6302564B2 (ja) 2014-09-05 2018-03-28 富士フイルム株式会社 動画編集装置、動画編集方法及び動画編集プログラム
WO2018147089A1 (ja) * 2017-02-10 2018-08-16 ソニー株式会社 情報処理装置および方法
JP2019133214A (ja) * 2018-01-29 2019-08-08 電駆ビジョン株式会社 画像表示装置、該装置を含む映像表示システム、画像表示方法および画像表示用プログラム
JP2019139443A (ja) * 2018-02-08 2019-08-22 キヤノン株式会社 通信装置、通信方法、及びプログラム

Family Cites Families (55)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004207948A (ja) * 2002-12-25 2004-07-22 Fuji Xerox Co Ltd ビデオ閲覧システム
EP1883241B1 (en) 2005-05-18 2016-01-13 NEC Corporation Content display system and content display method
US20070103558A1 (en) * 2005-11-04 2007-05-10 Microsoft Corporation Multi-view video delivery
US20070157281A1 (en) * 2005-12-23 2007-07-05 United Video Properties, Inc. Interactive media guidance system having multiple devices
US20100153885A1 (en) * 2005-12-29 2010-06-17 Rovi Technologies Corporation Systems and methods for interacting with advanced displays provided by an interactive media guidance application
US7822280B2 (en) * 2007-01-16 2010-10-26 Microsoft Corporation Epipolar geometry-based motion estimation for multi-view image and video coding
US20090144237A1 (en) * 2007-11-30 2009-06-04 Michael Branam Methods, systems, and computer program products for providing personalized media services
US8566353B2 (en) * 2008-06-03 2013-10-22 Google Inc. Web-based system for collaborative generation of interactive videos
JP2010124179A (ja) 2008-11-19 2010-06-03 Axis Web Japan Inc コンテンツ提供システム
JP2010218423A (ja) * 2009-03-18 2010-09-30 Toshiba Corp 音楽概念データ処理方法及び映像表示装置及び音楽概念データ処理サーバ
US8769589B2 (en) * 2009-03-31 2014-07-01 At&T Intellectual Property I, L.P. System and method to create a media content summary based on viewer annotations
US20110078717A1 (en) * 2009-09-29 2011-03-31 Rovi Technologies Corporation System for notifying a community of interested users about programs or segments
US8930849B2 (en) * 2010-03-31 2015-01-06 Verizon Patent And Licensing Inc. Enhanced media content tagging systems and methods
US9565476B2 (en) * 2011-12-02 2017-02-07 Netzyn, Inc. Video providing textual content system and method
US20150208103A1 (en) * 2012-08-08 2015-07-23 National University Of Singapore System and Method for Enabling User Control of Live Video Stream(s)
US9699485B2 (en) * 2012-08-31 2017-07-04 Facebook, Inc. Sharing television and video programming through social networking
US9497424B2 (en) * 2012-12-05 2016-11-15 At&T Mobility Ii Llc System and method for processing streaming media of an event captured by nearby mobile phones
US20160006981A1 (en) * 2013-02-19 2016-01-07 Wizeo Methods and systems for hosting interactive live stream video events for payment or donation
US20160037217A1 (en) * 2014-02-18 2016-02-04 Vidangel, Inc. Curating Filters for Audiovisual Content
WO2016002445A1 (ja) 2014-07-03 2016-01-07 ソニー株式会社 情報処理装置、情報処理方法、及びプログラム
US20160366464A1 (en) * 2015-06-11 2016-12-15 Flune Interactive, Inc. Method, device, and system for interactive television
US9967618B2 (en) * 2015-06-12 2018-05-08 Verizon Patent And Licensing Inc. Capturing a user reaction to media content based on a trigger signal and using the user reaction to determine an interest level associated with a segment of the media content
US20180176650A1 (en) * 2015-06-12 2018-06-21 Sony Corporation Information processing apparatus and information processing method
US9883249B2 (en) * 2015-06-26 2018-01-30 Amazon Technologies, Inc. Broadcaster tools for interactive shopping interfaces
US10390064B2 (en) * 2015-06-30 2019-08-20 Amazon Technologies, Inc. Participant rewards in a spectating system
JP6753405B2 (ja) * 2015-09-14 2020-09-09 ソニー株式会社 情報処理装置及び情報処理方法
US10187687B2 (en) * 2015-11-06 2019-01-22 Rovi Guides, Inc. Systems and methods for creating rated and curated spectator feeds
US10114689B1 (en) * 2015-12-28 2018-10-30 Amazon Technologies, Inc. Dynamic playlist generation
US10775882B2 (en) * 2016-01-21 2020-09-15 Microsoft Technology Licensing, Llc Implicitly adaptive eye-tracking user interface
US11012719B2 (en) * 2016-03-08 2021-05-18 DISH Technologies L.L.C. Apparatus, systems and methods for control of sporting event presentation based on viewer engagement
JPWO2017159063A1 (ja) * 2016-03-14 2019-01-17 ソニー株式会社 表示装置並びに情報処理端末装置
JP2018026104A (ja) * 2016-08-04 2018-02-15 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカＰａｎａｓｏｎｉｃ Ｉｎｔｅｌｌｅｃｔｕａｌ Ｐｒｏｐｅｒｔｙ Ｃｏｒｐｏｒａｔｉｏｎ ｏｆ Ａｍｅｒｉｃａ アノテーション付与方法、アノテーション付与システム及びプログラム
US20190253743A1 (en) * 2016-10-26 2019-08-15 Sony Corporation Information processing device, information processing system, and information processing method, and computer program
KR102739585B1 (ko) * 2016-12-23 2024-12-06 삼성전자주식회사 가상현실 영상에 대한 정보를 제공하는 장치 및 방법
US10148910B2 (en) * 2016-12-30 2018-12-04 Facebook, Inc. Group video session
WO2018152455A1 (en) * 2017-02-20 2018-08-23 Vspatial, Inc. System and method for creating a collaborative virtual session
JP2018163461A (ja) * 2017-03-24 2018-10-18 ソニー株式会社 情報処理装置、および情報処理方法、並びにプログラム
US10187690B1 (en) * 2017-04-24 2019-01-22 Gopro, Inc. Systems and methods to detect and correlate user responses to media content
JP7079242B2 (ja) 2017-04-25 2022-06-01 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 画像表示方法及び画像表示装置
JP3211786U (ja) * 2017-05-24 2017-08-03 ボーダレス・ビジョン株式会社 ライブ映像利用の対話装置
US10419790B2 (en) * 2018-01-19 2019-09-17 Infinite Designs, LLC System and method for video curation
US11064102B1 (en) * 2018-01-25 2021-07-13 Ikorongo Technology, LLC Venue operated camera system for automated capture of images
GB2571106A (en) * 2018-02-16 2019-08-21 Sony Corp Image processing apparatuses and methods
US11206462B2 (en) * 2018-03-30 2021-12-21 Scener Inc. Socially annotated audiovisual content
US11477516B2 (en) * 2018-04-13 2022-10-18 Koji Yoden Services over wireless communication with high flexibility and efficiency
US10757109B2 (en) * 2018-05-10 2020-08-25 Rovi Guides, Inc. Systems and methods for connecting a public device to a private device with pre-installed content management applications
CN113228690B (zh) * 2018-12-25 2023-09-08 索尼集团公司 视频再现装置、再现方法和程序
US11141656B1 (en) * 2019-03-29 2021-10-12 Amazon Technologies, Inc. Interface with video playback
JP7544034B2 (ja) * 2019-03-29 2024-09-03 ソニーグループ株式会社 画像処理装置、画像処理方法およびプログラム
JP6776400B1 (ja) * 2019-04-26 2020-10-28 株式会社コロプラ プログラム、方法、および情報端末装置
JP6971282B2 (ja) * 2019-08-30 2021-11-24 株式会社コロプラ プログラム、方法、および情報端末装置
US20210097338A1 (en) * 2019-09-26 2021-04-01 International Business Machines Corporation Using Domain Constraints And Verification Points To Monitor Task Performance
IL295170B2 (en) * 2020-01-30 2026-04-01 Amatelus Inc Device, system, method and software for video distribution
WO2021251127A1 (ja) * 2020-06-10 2021-12-16 ソニーグループ株式会社 情報処理装置、情報処理方法、撮像装置、画像転送システム
JP7083380B2 (ja) * 2020-09-28 2022-06-10 グリー株式会社 コンピュータプログラム、方法及びサーバ装置

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4697468B2 (ja) 2007-01-31 2011-06-08 日本電気株式会社 利用権限管理装置、コンテンツ共有システム、コンテンツ共有方法、及びコンテンツ共有用プログラム
JP2011228845A (ja) * 2010-04-16 2011-11-10 Canon Inc 画像処理装置および方法
JP2015177394A (ja) 2014-03-17 2015-10-05 カシオ計算機株式会社 画像処理装置、画像処理方法及びプログラム
JP6302564B2 (ja) 2014-09-05 2018-03-28 富士フイルム株式会社 動画編集装置、動画編集方法及び動画編集プログラム
WO2018147089A1 (ja) * 2017-02-10 2018-08-16 ソニー株式会社 情報処理装置および方法
JP2019133214A (ja) * 2018-01-29 2019-08-08 電駆ビジョン株式会社 画像表示装置、該装置を含む映像表示システム、画像表示方法および画像表示用プログラム
JP2019139443A (ja) * 2018-02-08 2019-08-22 キヤノン株式会社 通信装置、通信方法、及びプログラム

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4240019A4

Also Published As

Publication number Publication date
EP4240019A4 (en) 2024-06-05
US20230269411A1 (en) 2023-08-24
JP7208695B2 (ja) 2023-01-19
EP4240019A1 (en) 2023-09-06
JP7677639B2 (ja) 2025-05-15
JP2023027378A (ja) 2023-03-01
JPWO2022091215A1 (https=) 2022-05-05

Similar Documents

Publication Publication Date Title
JP7677639B2 (ja) 映像配信装置、映像配信システム、映像配信方法、及びプログラム
US11743544B2 (en) Interactive content feedback system
JP5767108B2 (ja) 媒体生成システム及び方法
Allocca Videocracy: how YouTube is changing the world... with double rainbows, singing foxes, and other trends we can’t stop watching
US12436665B2 (en) Digital story generation
US20140108932A1 (en) Online search, storage, manipulation, and delivery of video content
CN114761943A (zh) 创建、维护和发展虚拟音乐主题世界
US12100425B2 (en) Machine learned video template usage
US20110194839A1 (en) Mass Participation Movies
US20140149867A1 (en) Web-based interactive experience utilizing video components
US20220329644A1 (en) Real-time system and method for silent party hosting and streaming
US12063423B1 (en) Enhanced interactive web features for displaying and editing digital content
US20040098754A1 (en) Electronic messaging synchronized to media presentation
US20090063496A1 (en) Automated most popular media asset creation
JP7773112B2 (ja) サーバ装置、サーバ装置用プログラム及び端末装置用プログラム
CN114846808B (zh) 内容发布系统、内容发布方法以及存储介质
JP7365076B2 (ja) 映像配信装置、映像配信システム、映像配信方法、及びプログラム
Kyncl et al. Streampunks: How YouTube and the new creators are transforming our lives
Lastufka et al. Youtube: An insider's guide to climbing the charts
Vollans Cross media promotion: entertainment industries and the trailer
Herman For who page? TikTok creators’ algorithmic dependencies
JP7445272B1 (ja) 動画処理方法、動画処理システム、及び動画処理プログラム
CN118368464A (zh) 一种视频互动方法、装置、电子设备和存储介质
US20260025533A1 (en) Server, terminal, and method
JP7604698B1 (ja) 動画配信システム、動画配信装置、動画配信方法、及び動画配信プログラム

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2022517484

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20959733

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202327028261

Country of ref document: IN

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2020959733

Country of ref document: EP

Effective date: 20230530