WO2018147089A1 - Dispositif et procédé de traitement d'informations - Google Patents

Dispositif et procédé de traitement d'informations Download PDF

Info

Publication number
WO2018147089A1
WO2018147089A1 PCT/JP2018/002379 JP2018002379W WO2018147089A1 WO 2018147089 A1 WO2018147089 A1 WO 2018147089A1 JP 2018002379 W JP2018002379 W JP 2018002379W WO 2018147089 A1 WO2018147089 A1 WO 2018147089A1
Authority
WO
WIPO (PCT)
Prior art keywords
captured image
imaging
image data
information
imaging target
Prior art date
Application number
PCT/JP2018/002379
Other languages
English (en)
Japanese (ja)
Inventor
高林　和彦
Original Assignee
ソニー株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ソニー株式会社 filed Critical ソニー株式会社
Publication of WO2018147089A1 publication Critical patent/WO2018147089A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • G11B27/034—Electronic editing of digitised analogue information signals, e.g. audio or video signals on discs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • the present disclosure relates to an information processing apparatus and method, and more particularly, to an information processing apparatus and method that allow a user to select desired content more easily.
  • MPEG-DASH Moving Picture Experts Group Phase-Dynamic Dynamic Adaptive Streaming over HTTP
  • MPD Media Presentation Description
  • this MPD corresponds to a “play list” that describes a plurality of different video streams in an integrated manner.
  • SRD spatial Relation ⁇ ⁇ ⁇ Description
  • SRD spatial Relation ⁇ ⁇ ⁇ Description
  • ISO / IEC 23009-1 Information technology-Dynamic adaptive streaming over HTTP (DASH)-Part1: Media presentation description and segment formats, http: //standards.iso.org/ittf/PubliclyAvailableStandards/c065274_ISO_IEC_23009-1_2014.zip and ISO / IEC 23009-1: (same) m Amendment 2: Spatial Relationship Description, Generalized URL parameters and other extensions
  • This disclosure has been made in view of such a situation, and enables a user to select desired content more easily.
  • One aspect of the present technology includes a generation unit that generates a multi-view viewing playlist including a list of information that can be used to reproduce a plurality of captured image data generated by imaging the same imaging target from different positions.
  • Information processing apparatus includes a generation unit that generates a multi-view viewing playlist including a list of information that can be used to reproduce a plurality of captured image data generated by imaging the same imaging target from different positions.
  • the information that can be used to reproduce the captured image data can be information that can be used to select the captured image data to be reproduced.
  • the information that can be used to reproduce the captured image data can include information indicating the position of the imaging target.
  • the information that can be used for reproduction of the captured image data can include information indicating an imaging direction viewed from the imaging target.
  • the information that can be used for reproduction of the captured image data can include information indicating the acquisition destination of the captured image data.
  • the information usable for reproduction of the captured image data can include information indicating a period not included in the captured image data.
  • the multi-view viewing playlist may include a list for each imaging target of information that can be used to reproduce the captured image data for a plurality of imaging targets.
  • the multi-view viewing playlist may be generated for a predetermined period and include information indicating a start time and a length of the period.
  • the multi-view viewing playlist may include a list for each imaging target of information that can be used for reproducing the captured image data for each predetermined period for a plurality of periods.
  • the generation unit may generate the multi-view viewing playlist as an MPEG-DASH (Moving Picture Experts Group phase-Dynamic Dynamic Adaptive Streaming over HTTP) MPD (Media Presentation Description).
  • MPEG-DASH Motion Picture Experts Group phase-Dynamic Dynamic Adaptive Streaming over HTTP
  • MPD Media Presentation Description
  • the multi-view appreciation playlist can include information indicating the imaging direction viewed from the imaging target, listed for each area to be imaged.
  • the multi-view viewing playlist can include information indicating a center position and a radius of the area to be imaged.
  • the multi-view appreciation playlist can manage information on each captured image data with AdaptationSet.
  • the generation unit can group the captured image data based on the imaging target and generate a list of information that can be used for reproduction of the captured image data for each group.
  • the generation unit collects all the captured image data into one group, groups the captured image data using a preset area, or selects the captured image according to the position of each imaging target. Data can be grouped.
  • the image processing apparatus may further include an analysis unit that analyzes an imaging state, and the generation unit may be configured to generate the multi-view viewing playlist based on a result of analysis by the analysis unit.
  • the analysis unit can obtain information related to imaging, information related to the imaging target, and information related to the imaging direction viewed from the imaging target.
  • the analysis unit can analyze the imaging state based on the metadata of the captured image data and information on the venue.
  • a providing unit that provides the multi-view viewing playlist generated by the generating unit can be further provided.
  • One aspect of the present technology is also an information processing that generates a multi-view viewing playlist including a list of information that can be used to reproduce a plurality of captured image data generated by imaging the same imaging target from different positions. Is the method.
  • a multi-view viewing playlist including a list of information that can be used to reproduce a plurality of captured image data generated by imaging the same imaging target from different positions is generated.
  • information can be processed.
  • the user can select desired content more easily.
  • Image sharing and selection> it is possible to share information, such as moving images and still images captured at the same event, etc., and audio, etc., among friends using SNS and data sharing services.
  • the shared data is classified and managed in units such as the date and time of the event, the location, etc., or every period of a small event in the event.
  • metadata such as an imaging location (position), a focus distance, and a zoom level may be added to the captured image data.
  • position an imaging location
  • focus distance a focus distance
  • zoom level a zoom level
  • this metadata is added to each captured image data and is not managed collectively. Therefore, when the user specifies a desired captured image using the metadata, the user needs to check the metadata of each captured data one by one, and a complicated operation is required. Furthermore, as the number of shared information increases, the amount of work further increases.
  • such metadata is composed of information related to captured image data to which the metadata itself is added, and information related to other captured image data is not included. Therefore, it is difficult to specify the relationship between captured images, for example, the subject is the same, from this metadata. Therefore, for example, it has been difficult to perform a more complicated search such as specifying other captured image data that is the same imaging target.
  • package media such as DVD-ROM (Digital Versatile Disc-Read-Only Memory) and BD-ROM (Blu-ray (registered trademark) Disc-read-Only Memory)
  • DVD-ROM Digital Versatile Disc-Read-Only Memory
  • BD-ROM Blu-ray (registered trademark) Disc-read-Only Memory
  • MPEG-DASH Moving / Picture / Experts / Group / phase / -Dynamic / Adaptive / Streaming / over / HTTP
  • MPD Media / Presentation
  • An extension called Spatial Relation Description (SRD) ⁇ ⁇ ⁇ ⁇ ⁇ ⁇ has been defined as a mechanism for associating a partial video stream in one video with the original (whole) video.
  • SRD Spatial Relation Description
  • ⁇ Generation of multi-view viewing playlist> Therefore, a multi-view viewing playlist including a list of information that can be used to reproduce a plurality of captured image data generated by imaging the same imaging target from different positions is generated. By doing in this way, the user who views the captured image can perform a more advanced search of the captured image data based on the multi-view viewing playlist. That is, the user can more easily select and acquire desired content.
  • FIG. 1 is a block diagram illustrating a main configuration example of an image providing system which is an embodiment of an image processing system to which the present technology is applied.
  • An image providing system 100 shown in FIG. 1 is a system that provides an image sharing service.
  • Image sharing services include images taken at any event, such as concerts, plays, sports games and competitions, local and school events (eg festivals, ceremonies, flea markets, school performances, sports events, etc.) This is a service that enables viewers to share and browse image data (captured image data).
  • the event may be anything as long as the time and place can be specified, such as a concert, a stage, a professional sports game, etc. It may be public such as festivals, ceremonies, amateur sports competitions, flea markets, school events such as athletic meet and athletic meet, or private such as birthday parties.
  • the provision (distribution) of the captured image data may be performed immediately (so-called real time (ignoring time lag such as transmission delay and processing delay)) with respect to imaging, or the accumulated image data may be stored. It may be in a format that is distributed upon request (so-called on-demand).
  • the provider of this image sharing service may be the same as or different from the event operator.
  • the photographer may be an event operator (including staff on the operation side), an image sharing service provider, or a content broadcast / distributor other than those. It may be an event participant (actor, player, spectator, etc.). This photographer may or may not be registered in advance in the image sharing service. That is, only a specific person registered as a member in advance may be allowed to share captured image data, or an unspecified number of persons may be allowed to share captured image data.
  • the viewer who shares the captured image data may be an event participant (performer, player, spectator, etc.), or may be a person who has not participated in the event. Of course, it may be an event operator. It is also possible for the photographer to be a viewer. The viewer may or may not be registered in advance in the image sharing service.
  • the image providing system 100 includes an imaging device 101, an integrated server 102, and a terminal device 103.
  • the imaging device 101 and the terminal device 103 are communicably connected to the integrated server 102 via an arbitrary communication medium such as the Internet.
  • an arbitrary communication medium such as the Internet.
  • one imaging device 101, one integrated server 102, and one terminal device 103 are shown, but these numbers are arbitrary and may be plural.
  • the image providing system 100 is a system in which captured image data (captured image data) and the like captured by the image capturing apparatus 101 are accumulated by the accumulation server 102, and the terminal apparatus 103 acquires and reproduces it. That is, the integrated server 102 provides a service for sharing the captured image data captured by the imaging device 101 (the user who is the user) to the terminal device 103 (the viewer who is the user).
  • the imaging device 101 performs processing related to imaging and communication. For example, the imaging device 101 captures a subject (also referred to as an imaging target) and generates captured image data that is data of the captured image. Further, for example, the imaging apparatus 101 supplies the generated captured image data to the integration server 102 (so-called uploading).
  • a subject also referred to as an imaging target
  • the imaging apparatus 101 supplies the generated captured image data to the integration server 102 (so-called uploading).
  • the accumulation server 102 performs processing related to accumulation and provision of captured image data. For example, the accumulation server 102 acquires and stores captured image data supplied from the imaging device 101. Further, for example, the integration server 102 generates a multi-view viewing playlist that is information used for reproducing captured image data.
  • the multi-view viewing playlist is information used for retrieval (selection) and reproduction control of captured image data, and retrieval (selection) of a plurality of captured image data generated by imaging the same imaging target from different positions. ) And a list of information available for playback control. More specifically, in the multi-view appreciation playlist, information regarding captured image data is collected for each imaging target (for example, a person, an object, an area, and the like). Therefore, for example, when searching (selecting) and playing back captured image data in which a desired imaging target is reflected, the user uses this multi-view viewing playlist to perform search (selection) and playback control. It can be made easier.
  • a user who views a captured image views a captured image showing a desired imaging target. That is, in many cases, what is reflected in a captured image is most important for a user who views the captured image.
  • a large number of imaging data obtained by imaging a large number of imaging targets is often shared.
  • conventional playlists, metadata, and the like hardly contain information about the imaging target, and it is difficult to perform a search based on the imaging target.
  • the search and reproduction control based on the imaging target can be performed more easily by using the above-described multi-view viewing playlist, user satisfaction can be improved.
  • the accumulation server 102 provides captured image data to the terminal device 103 (downloading, streaming, etc.). For example, the accumulation server 102 supplies the generated multi-view viewing playlist to the terminal device 103, and performs playback control based on the multi-view viewing playlist. That is, the accumulation server 102 supplies the captured image data requested based on the supplied multi-view viewing playlist to the terminal device 103.
  • the terminal device 103 performs processing related to reproduction of captured image data shared using the accumulation server 102. For example, the terminal device 103 acquires a multi-view viewing playlist from the integration server 102, specifies desired captured image data based on the multi-view viewing playlist, and requests the specified captured image data from the integration server 102. To do. When desired captured image data is supplied from the accumulation server 102 in response to the request, the terminal device 103 acquires the captured image data, reproduces it, and displays it on a monitor or the like.
  • FIG. 2 is a block diagram illustrating a main configuration example of the imaging apparatus 101.
  • the imaging apparatus 101 includes an imaging unit 121, a metadata generation unit 122, a metadata addition unit 123, and a communication unit 124.
  • the imaging unit 121 includes a captured image generation function such as an image sensor, for example, and performs processing related to imaging of an imaging target.
  • the metadata generation unit 122 performs processing related to generation of metadata of captured image data.
  • the metadata adding unit 123 performs a process related to adding metadata to captured image data.
  • the metadata adding unit 123 performs processing related to upload of captured image data to which metadata is added.
  • FIG. 3 is a block diagram illustrating a main configuration example of the integrated server 102.
  • the integrated server 102 includes a CPU (Central Processing Unit) 151, a ROM (Read Only Memory) 152, a RAM (Random Access Memory) 153, a bus 154, an input / output interface 160, an input unit 161, and an output.
  • the CPU 151, the ROM 152, and the RAM 153 are connected to each other via a bus 154.
  • An input / output interface 160 is also connected to the bus 154.
  • An input unit 161 to a drive 165 are connected to the input / output interface 160.
  • the input unit 161 includes arbitrary input devices such as a keyboard, a mouse, a touch panel, an image sensor, a microphone, a switch, and an input terminal.
  • the output unit 162 includes an arbitrary output device such as a display, a speaker, and an output terminal.
  • the storage unit 163 includes an arbitrary storage medium such as a hard disk, a RAM disk, a nonvolatile memory such as an SSD (Solid State Drive) or a USB (Universal Serial Bus) (registered trademark) memory.
  • the communication unit 164 is, for example, any communication standard such as Ethernet (registered trademark), Bluetooth (registered trademark), USB, HDMI (registered trademark) (High-Definition Multimedia Interface), IrDA or the like, wired or wireless, or both. Communication interface.
  • the drive 165 drives a removable medium 171 having an arbitrary storage medium such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.
  • Processing is performed by the CPU 151 loading, for example, a program stored in the ROM 152 or the storage unit 163 into the RAM 153 and executing it.
  • the RAM 153 also appropriately stores data necessary for the CPU 151 to execute various processes.
  • FIG. 4 is a functional block diagram illustrating an example of functions that the integrated server 102 has.
  • the integrated server 102 implements various functions as shown in FIG. 4 when the CPU 151 executes a program or the like.
  • the accumulation server 102 includes, as functional blocks, an accumulation unit 181, a captured image database 182, an imaging state analysis unit 183, a playlist generation unit 184, a multi-view viewing playlist database 185, and a playlist provision unit. 186 and a captured image providing unit 187.
  • the accumulation unit 181 performs processing related to the accumulation of captured image data and its metadata.
  • the accumulation unit 181 is realized by the CPU 151 executing a program or the like or controlling the communication unit 164 or the like via the input / output interface 160.
  • the captured image database 182 performs processing related to storage and management of information such as captured image data and metadata.
  • the captured image database 182 is realized, for example, when the CPU 151 executes a program or the like or controls the storage unit 163 or the like via the input / output interface 160.
  • the imaging state analysis unit 183 performs processing related to analysis of the imaging state.
  • the imaging state analysis unit 183 is realized by, for example, the CPU 151 executing a program or the like.
  • the playlist generation unit 184 performs processing related to generation of a multi-view viewing playlist.
  • the playlist generation unit 184 is realized, for example, when the CPU 151 executes a program or the like.
  • the multi-view viewing playlist database 185 performs processing related to storage and management of the multi-view viewing playlist.
  • the multi-view viewing playlist database 185 is realized, for example, when the CPU 151 executes a program or the like, or controls the storage unit 163 or the like via the input / output interface 160.
  • the playlist providing unit 186 performs processing related to providing a multi-view viewing playlist.
  • the playlist providing unit 186 is realized, for example, when the CPU 151 executes a program or the like, or controls the communication unit 164 or the like via the input / output interface 160.
  • the captured image providing unit 187 performs processing related to provision of captured image data.
  • the captured image providing unit 187 is realized, for example, when the CPU 151 executes a program or the like or controls the communication unit 164 or the like via the input / output interface 160.
  • FIG. 5 is a block diagram illustrating a main configuration example of the terminal device 103.
  • the terminal device 103 includes a CPU 201, a ROM 202, a RAM 203, a bus 204, an input / output interface 210, an input unit 211, an output unit 212, a storage unit 213, a communication unit 214, and a drive 215.
  • the CPU 201, the ROM 202, and the RAM 203 are connected to each other via a bus 204.
  • An input / output interface 210 is also connected to the bus 204.
  • An input unit 211 to a drive 215 are connected to the input / output interface 160.
  • the input unit 211 includes arbitrary input devices such as a keyboard, a mouse, a touch panel, an image sensor, a microphone, a switch, and an input terminal.
  • the output unit 212 includes an arbitrary output device such as a display, a speaker, and an output terminal, for example.
  • the storage unit 213 includes an arbitrary storage medium such as a hard disk, a RAM disk, a non-volatile memory such as an SSD (Solid State Drive) or a USB (Universal Serial Bus) (registered trademark) memory.
  • the communication unit 164 is, for example, any communication standard such as Ethernet (registered trademark), Bluetooth (registered trademark), USB, HDMI (registered trademark) (High-Definition Multimedia Interface), IrDA or the like, wired or wireless, or both. Communication interface.
  • the drive 215 drives a removable medium 221 having an arbitrary storage medium such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.
  • Processing is performed by the CPU 201 loading, for example, a program stored in the ROM 202 or the storage unit 213 into the RAM 203 and executing the program.
  • the RAM 203 also appropriately stores data necessary for the CPU 201 to execute various processes.
  • FIG. 6 is a functional block diagram illustrating an example of functions that the terminal device 103 has.
  • the terminal device 103 implements various functions as shown in FIG. 6 when the CPU 201 executes a program or the like.
  • the terminal device 103 includes a playlist acquisition unit 231, an image selection processing unit 232, a captured image request unit 233, a captured image acquisition unit 234, and a playback unit 235 as functional blocks.
  • the playlist acquisition unit 231 performs processing related to acquisition of a multi-view viewing playlist.
  • the playlist acquisition unit 231 is realized, for example, when the CPU 201 executes a program or the like, or controls the communication unit 214 or the like via the input / output interface 210.
  • the image selection processing unit 232 performs processing related to selection of captured images based on the multi-view viewing playlist.
  • the image selection processing unit 232 is realized, for example, when the CPU 201 executes a program or the like, or controls the communication unit 214 or the like via the input / output interface 210.
  • the captured image request unit 233 performs processing related to a request for a captured image to be reproduced.
  • the captured image request unit 233 is realized, for example, when the CPU 201 executes a program or the like, or controls the communication unit 214 or the like via the input / output interface 210.
  • the captured image acquisition unit 234 performs processing related to acquisition of a captured image.
  • the captured image acquisition unit 234 is realized, for example, when the CPU 201 executes a program or the like, or controls the communication unit 214 or the like via the input / output interface 210.
  • the playback unit 235 is a device that performs processing related to playback of captured image data.
  • the reproducing unit 235 is realized by the CPU 201 executing a program or the like or controlling the output unit 212 or the like via the input / output interface 210.
  • the imaging unit 121 of the imaging apparatus 101 is operated by the photographer at an event venue or the like in step S101 to capture an imaging target (subject).
  • the imaging unit 121 photoelectrically converts incident light with an image sensor or the like to generate captured image data, and supplies it to the metadata adding unit 123.
  • the metadata generation unit 122 collects information such as imaging settings and surrounding states (imaging environment), and generates metadata including them.
  • the content of this metadata is arbitrary.
  • the metadata generation unit 122 supplies the generated metadata to the metadata addition unit 123.
  • step S103 the metadata adding unit 123 adds the metadata generated by the process of step S102 to the captured image data generated by the process of step S101.
  • the metadata adding unit 123 supplies the captured image data to which the metadata is added to the communication unit 124.
  • step S104 the communication unit 124 communicates with the accumulation server 102, and supplies (transmits) the captured image data to which the metadata is added by the process in step S103 to the accumulation server 102 at a predetermined timing.
  • the captured image data to which the metadata is added is transmitted to the integrated server 102 via a transmission path such as the Internet.
  • step S ⁇ b> 121 the accumulation unit 181 of the accumulation server 102 controls the communication unit 164 to communicate with the imaging apparatus 101, and captured image data (imaging with metadata added) supplied (transmitted) from the imaging apparatus 101. Acquire (receive) image data. That is, the accumulation unit 181 collects captured image data to which metadata is added.
  • step S122 the captured image database 182 stores and manages the captured image data (and its metadata) acquired by the processing in step S121 in the storage unit 163.
  • the captured image accumulation process ends.
  • the captured image integration process as described above is executed for each imaging device used for imaging at the event venue.
  • the execution timing of this captured image integration process is arbitrary. For example, it may be after the event ends or during the event. Further, for example, it may be executed at a predetermined time, may be periodically executed at a predetermined time interval, or may be executed based on an instruction from a photographer or the like. Also good. Further, for example, it may be executed when a predetermined event occurs, such as generating new captured image data.
  • the accumulation server 102 executes a multi-view appreciation playlist generation process to generate a multi-view appreciation playlist.
  • An example of the flow of the multi-view appreciation playlist generation process will be described with reference to the flowchart of FIG.
  • the imaging state analysis unit 183 of the integrated server 102 analyzes the imaging state in step S141. Details of this analysis processing will be described later.
  • step S142 the playlist generation unit 184 generates a multi-view viewing playlist based on the analysis result. Details of the playlist generation processing will be described later.
  • this multi-view appreciation playlist generation process ends.
  • the execution timing of this multi-view appreciation playlist generation process is arbitrary. For example, it may be executed at a predetermined time, may be periodically executed at predetermined time intervals, or may be executed based on an instruction from a server administrator or the like. Good. Further, for example, it may be executed when a predetermined event occurs, such as acquiring new captured image data.
  • the imaging situation analysis unit 183 analyzes the imaging situation such as the imaging position at the time of imaging, the position of the imaging target, the imaging direction viewed from the imaging target, and the like on the captured image data. This analysis can be performed based on arbitrary information.
  • the imaging state analysis unit 183 refers to the following information.
  • Image pickup position Information regarding the position (image pickup position) of the image pickup apparatus 101 at the time of image pickup may measure its own position using a GPS (Global Positioning System) system at the time of imaging. Further, for example, positioning may be performed using communication with a wireless LAN (Local Area Network) access point or a base station, or various sensors such as an acceleration sensor may be used. The positioning method is arbitrary. In this case, the accumulation server 102 performs analysis with reference to the positioning result (coordinates and the like).
  • the information related to the imaging position may be other than the positioning result (coordinates, etc.).
  • it may be identification information (seat number or the like) or position information (coordinates or the like) of the seat where the image is taken.
  • a position (seat) where an imager or the like has taken an image may be specified based on a seating chart indicating a seat layout.
  • the accumulation server 102 performs analysis with reference to seat identification information (such as a seat number) and position information (such as coordinates).
  • Such information regarding the imaging position may or may not be included in the metadata of the captured image data.
  • it may be supplied to the accumulation server 102 as data different from the captured image data and metadata.
  • the integrated server 102 may analyze the captured image and specify the imaging position.
  • the accumulation server 102 may identify the imaging position by comparing the captured image with a reference image prepared in advance.
  • a marker for specifying a position may be installed at the event venue, and the accumulation server 102 may specify the imaging position based on the marker reflected in the captured image.
  • the accumulation server 102 performs analysis with reference to the imaging position (coordinates, etc.) specified by itself.
  • devices other than the imaging device 101 and the integrated server 102 may specify the imaging position.
  • Imaging time This is information relating to the time (such as the start time and the end time) when imaging was performed and the time from the start to the end. For example, it is possible to generate time information for starting and ending imaging using a timer function in which the imaging apparatus 101 is built.
  • the imaging device 101 may acquire time information from an external device such as a server.
  • the accumulation server 102 performs analysis with reference to this time information.
  • the captured image data can be more accurately synchronized.
  • the synchronization is not accurately achieved unless the times are synchronized and the reproduction of a plurality of captured image data is switched, for example, when simply searching for captured image data having similar imaging times. In many cases, there is no problem. That is, it is only necessary to synchronize the time information between the imaging devices 101 with accuracy according to the usage method.
  • communication with a server may be performed at the time of imaging. Information regarding the time and time when such imaging is performed may or may not be included in the metadata of the captured image data.
  • the imaging device 101 may measure the distance from the imaging device 101 to the imaging target.
  • the method of distance measurement is arbitrary.
  • Such information regarding the distance to the imaging target may or may not be included in the metadata of the captured image data.
  • it may be supplied to the accumulation server 102 as data different from the captured image data and metadata.
  • the integrated server 102 may analyze the captured image and measure the distance from the imaging device 101 to the imaging target.
  • the accumulation server 102 may perform distance measurement by comparing a captured image with a reference image prepared in advance. Of course, devices other than the imaging device 101 and the integrated server 102 may perform this distance measurement.
  • the accumulation server 102 performs analysis with reference to the distance measurement results (distance information, etc.) obtained in this way.
  • Imaging direction Information regarding the imaging direction of the imaging apparatus 101 may specify the imaging direction using an acceleration sensor, an electronic compass, or the like. Such information regarding the imaging direction may or may not be included in the metadata of the captured image data. For example, it may be supplied to the accumulation server 102 as data different from the captured image data and metadata. Further, for example, the accumulation server 102 may compare the captured image with a reference image prepared in advance to specify the imaging direction. The accumulation server 102 performs analysis with reference to the imaging direction specified in this way.
  • -Zoom level This is information related to the zoom setting (angle of view setting) when the imaging apparatus 101 performs imaging. Such information regarding the imaging direction may or may not be included in the metadata of the captured image data. For example, it may be supplied to the accumulation server 102 as data different from the captured image data and metadata. Further, for example, the accumulation server 102 may compare the captured image with a reference image prepared in advance to specify the zoom level. The accumulation server 102 performs analysis with reference to the zoom level specified in this way.
  • Imaging state analysis unit 183 refers to at the time of analysis is arbitrary. Any one of the various types of information described above may be referred to, or information other than those described above may be referred to. A plurality of pieces of information may be referred to.
  • the imaging state analysis unit 183 may refer to venue information that is information about an event venue where imaging is performed.
  • the content of the venue information is arbitrary, but may include, for example, imaging target positional relationship information indicating the positional relationship of various areas in the event venue, venue shape information indicating the shape and layout of the event venue, and the like. Good.
  • the image pickup target position relation information includes, for example, an image pickup position area that is an area where the image pickup apparatus 101 can be placed in an event venue, an image pickup target area that can be an image pickup target, and an image pickup target area.
  • This is information indicating the positional relationship of an imaging target detail area, which is an area where a specific imaging target exists.
  • FIG. 9 shows an example of an event such as an athletic meet held in the school yard.
  • the imaging target position relationship information 250 shown in FIG. 9 indicates an imaging position area 251, an imaging target area 252, an imaging target detail area 253-1, an imaging target detail area 253-2, an imaging target detail area 253-3, and the like. ing.
  • imaging target detail area 253-1, the imaging target detail area 253-2, and the imaging target detail area 253-3 are referred to as the imaging target detail area 253 when it is not necessary to distinguish between them. How much information in the imaging target detail area 253 is assumed in advance depends on the nature of the event.
  • imaging target position relationship information 250 it is possible to limit the position of the imaging device 101 and the imaging target in the venue to some extent, so it is easier to specify the imaging position, the position of the imaging target, the imaging direction, and the like. Can be.
  • the imaging target position relationship information 250 includes the positions, imaging directions, and the like of the imaging device 261-1 and the imaging device 261-2 that capture the imaging target region 252 and generate a reference image. .
  • Venue shape information is information about the shape of the venue. For example, the size, shape, coordinates, etc. of the venue are shown. Further, the layout and coordinates of the seat may be shown. By using such venue shape information, it is possible to specify an imaging position, an imaging target position, an imaging direction, and the like in the venue. For example, when coordinates are indicated as the imaging position, the imaging position in the venue can be specified by comparing the coordinates with the coordinates of the venue shape information. Further, for example, when a seat number is indicated as the imaging position, the imaging position (seat position) in the venue can be specified by comparing the seat number with the seating chart of the venue shape information.
  • step S141 in FIG. 8 the imaging state analysis unit 183 analyzes the imaging state such as the imaging position, the imaging target, and the imaging direction as viewed from the imaging target from the above-described various information.
  • the imaging state analysis unit 183 analyzes the imaging state such as the imaging position, the imaging target, and the imaging direction as viewed from the imaging target from the above-described various information.
  • the imaging state analysis unit 183 acquires data to be analyzed (for example, metadata of the captured image data to be analyzed) from the captured image database 182 in step S161. In step S162, the imaging state analysis unit 183 determines whether the metadata includes imaging position information indicating the imaging position. If it is determined that it is included, the process proceeds to step S163.
  • data to be analyzed for example, metadata of the captured image data to be analyzed
  • the imaging state analysis unit 183 specifies the imaging position in the “imaging target position relationship”, that is, the imaging position in the event hall, based on information (for example, GPS information) related to the imaging position included in the metadata. . Further, the venue information may be referred to. For example, the imaging state analysis unit 183 may identify the imaging position in the event venue by comparing coordinates indicating the imaging position with venue shape information (layout and coordinates in the venue). When the process of step S163 ends, the process proceeds to step S165.
  • step S164 the imaging state analysis unit 183 specifies the imaging position in the “imaging target position relationship”, that is, the imaging position in the event hall, based on various information.
  • the imaging state analysis unit 183 specifies the imaging position based on imaging position information (for example, coordinates and seat number) input by the photographer or the like.
  • the venue information may be referred to.
  • the imaging state analysis unit 183 may identify the imaging position in the event venue by comparing the captured image with a reference image whose imaging position and imaging direction are known.
  • step S165 the imaging state analysis unit 183 determines whether or not imaging target position information indicating the position of the imaging target is included in the metadata. If it is determined that it is included, the process proceeds to step S166.
  • the imaging state analysis unit 183 determines the position of the imaging target in the “imaging target positional relationship” (or based on the information related to the position of the imaging target included in the metadata (for example, the imaging direction and the distance to the imaging target) (or Range), that is, the position (or range) of the imaging target in the event venue.
  • the imaging state analysis unit 183 specifies an imaging target (thing, person, or region) based on information such as an imaging direction and a distance to the imaging target included in the captured image data and metadata. Specify the location in the event venue to be imaged. Further, the venue information may be referred to.
  • the imaging state analysis unit 183 identifies the position (or range) of the imaging target in the event venue by comparing the imaging direction and the distance to the imaging target with venue shape information (layout and coordinates in the venue). You may do it.
  • the process of step S166 ends, the process proceeds to step S168.
  • step S167 the imaging state analysis unit 183 specifies the position (or range) of the imaging target in the “imaging target positional relationship” based on various information.
  • the imaging state analysis unit 183 based on imaging target position information (for example, coordinates) input by the photographer or the like, the position (or range) of the imaging target, that is, the position (or range) of the imaging target in the event venue. Is identified. Further, the venue information may be referred to.
  • the imaging state analysis unit 183 may identify the position of the imaging target in the event venue by comparing the captured image with a reference image whose imaging position and imaging direction are known.
  • the imaging state analysis unit 183 determines the imaging position specified by the process of step S163 or the process of step S164, and the position (or range of the imaging target) specified by the process of step S166 or the process of step S167. Based on (center), the positional relationship between the imaging apparatus 101 and the imaging target is specified, and further, the imaging direction viewed from the imaging target is specified from the positional relationship. Further, the venue information may be referred to. For example, the imaging state analysis unit 183 may identify the position of the imaging target in the event venue by comparing the captured image with a reference image whose imaging position and imaging direction are known.
  • the captured image database 182 stores information indicating the imaging position, the position (or range) of the imaging target, and the imaging direction viewed from the imaging target, obtained as described above, as captured image data to be processed. Is stored as metadata of the captured image data and managed.
  • step S170 the imaging state analysis unit 183 determines whether or not all captured image data has been analyzed. If it is determined that there is unprocessed captured image data, the process proceeds to step S171. In step S171, the imaging state analysis unit 183 updates the analysis target to the next captured image data (changes the analysis target). When the process of step S171 ends, the process returns to step S161, and the subsequent processes are repeated.
  • the integration server 102 analyzes the imaging state of all the captured image data stored in the captured image database 182 by executing each process of step S161 to step S171 on each captured image data. If it is determined in step S170 that all captured image data has been analyzed, the analysis process ends, and the process returns to FIG.
  • EndX (Y) indicates the end time (for example, UTC time).
  • xxX (Y), yyX (Y), and zzX (Y) are coordinates in the “imaging target positional relationship” (FIG. 9)
  • xyX (Y) and zX (Y) are imaging directions (camera positions) viewed from the coordinates. Direction).
  • the coordinate origin and the reference direction of the angle are determined according to the “imaging target positional relationship”. For example, in the example of FIG. 9, the center is the coordinate origin, the reference of xyX (Y) is directly above, and zX (Y) (The standard is the horizontal plane, etc.
  • the numerical value representing the coordinates is an integer value in millimeters.
  • the playlist generation unit 184 groups the captured image data for each imaging target detail area in step S191.
  • the playlist generation unit 184 generates a multi-view viewing playlist for each group, that is, for each imaging target detail area.
  • the pre-list generation process ends, and the process returns to FIG.
  • the imaging target detail area 253 may be set in addition to the imaging target area 252 as a part of the imaging target positional relationship information 250 in advance.
  • the imaging target detail area 253 may be automatically formed from metadata given to each video as a specific result of the imaging target described above.
  • three types of grouping are described. It is assumed that the imaging target detail area 253 is defined as a circle (or a hemisphere), and the position expression on the playlist is based on the center position.
  • the imaging target detail area 282 includes all the imaging target positions of the imaging devices 101-1 to 101-8. That is, in this case, all of the captured image data captured by the imaging devices 101-1 to 101-8 are grouped into one group.
  • each imaging device 101 performs imaging at the same position and direction as in FIG. In this case, as shown in FIG. 15, since the imaging device 101-4, the imaging device 101-6, and the imaging device 101-8 are in the vicinity of each other, the imaging is performed so as to include all these imaging targets.
  • the target detail area 284-1 may be set, and grouping may be performed by the imaging target detail area 284-1.
  • the imaging object details are included so as to include all these imaging objects.
  • An area 284-2 may be set, and grouping may be performed by the imaging target detail area 284-2. Note that the size of the imaging target detail area set in this way is arbitrary. As in the example of FIG. 15, when a plurality of imaging target detailed areas are set, their sizes do not have to be unified (an imaging target detailed area having a different size from the other can be set).
  • (1) and (2) are grouped based on a pre-designated imaging target detail area.
  • (3) those in which the position of the imaging target of each captured image data is included in a certain range are grouped.
  • such grouping of captured image data is performed at certain time intervals. For example, every small event in the entire event. In that case, a certain imaging device 101 does not always capture one imaging target (imaging target detailed area) during that time, but it also captures a grouping target area even during a part of that period. If it is, the captured image data is included in the group. In that case, an identifier of a group belonging to each predetermined time is added as metadata to each captured image data. In this way, group information (information indicating a group of captured image data) is generated. An example of this group information is shown in FIG.
  • the group element is a preset imaging target detail area (imaging target detail area 282 in FIG. 13 and imaging target detail area 253 in FIG. 14), or metadata of each captured image data.
  • imaging target detail area for example, imaging target detail area 284-1 and imaging target detail area 284-2 in FIG. 15
  • cx (n), cy (n), and xz (n) indicate the coordinates of the center of the circular area (that is, the imaging target detail area) including the imaging target position of each grouped captured image data, and radius Represents the radius.
  • the gpref attribute of object_group indicates the id of the ⁇ group> element to which each captured image data belongs as a result of grouping.
  • startX (Y) and endX (Y) indicate the start and end time (for example, UTC time) of the section in which the captured image data belongs to each group.
  • the s-angle values (xyX (Y), zX (Y)) indicate the imaging direction (imaging position direction, angle on the horizontal plane and angle above and below the horizontal plane) as seen from the center of the imaging target detail area of the group. Show. Therefore, the value of the angle attribute given to the ⁇ period (n)> element added in advance to each captured image data does not necessarily match.
  • the ⁇ period> element and the ⁇ object_group> element described above are child elements of the same ⁇ photographing data> element and can be described together, but are omitted here for simplification.
  • step S212 the playlist generation unit 184 acquires the metadata of the image (i) to be processed (for example, the metadata of the i-th registered captured image data) from the captured image database 182.
  • step S213 the playlist generation unit 184 determines whether there is an imaging target detail area including the coordinates of the jth section. If it is determined that it exists, the process proceeds to step S214.
  • step S214 the playlist generation unit 184 assigns the jth section of the image (i) to the corresponding group.
  • the process proceeds to step S216. If it is determined in step S213 that there is no imaging target detailed area including the coordinates of the jth section, the process proceeds to step S215.
  • step S215 the playlist generation unit 184 records the position information of the jth section of the image (i) as the other imaging target (n). When the process of step S215 ends, the process proceeds to step S216.
  • step S216 the playlist generation unit 184 determines whether the value of the variable j is the number of sections of the image (i). If it is determined that the value of the variable j has not reached the number of sections of the image (i), the process proceeds to step S217.
  • step S217 the playlist generation unit 184 increments the values of the variable j and the variable n by “+1” (j ++, n ++). When the process of step S217 ends, the process returns to step S213, and the subsequent processes are repeated. As described above, the processing from step S213 to step S217 is repeatedly executed, and when it is determined in step S216 that the value of the variable j has reached the number of sections of the image (i), the processing proceeds to step S218.
  • step S218 the playlist generation unit 184 determines whether the value of the variable i is the number of captured image data. If it is determined that the value of the variable i has not reached the number of captured image data, the process proceeds to step S219.
  • step S219 the playlist generation unit 184 increments the value of the variable i by “+1” (i ++). When the process of step S219 ends, the process returns to step S212, and the subsequent processes are repeated. As described above, the processing from step S212 to step S219 is repeatedly executed, and when it is determined in step S218 that the value of the variable i has reached the number of captured image data, the processing proceeds to FIG.
  • step S221 of FIG. 18 the playlist generation unit 184 groups other target positions (1) to (n) in the vicinity range and sets an unspecified imaging target detail area.
  • step S223 the playlist generation unit 184 acquires the metadata of the image (i) to be processed (for example, the metadata of the captured image data registered for the i-th) from the captured image database 182.
  • step S224 the playlist generation unit 184 determines whether or not the i-th section has been assigned to the group. If it is determined that no assignment has been made, the process proceeds to step S225.
  • step S225 the playlist generation unit 184 determines whether there is an unspecified imaging target detail area including the coordinates of the jth section. If it is determined that it exists, the process proceeds to step S226.
  • step S226 the playlist generation unit 184 assigns the jth section of the image (i) to the corresponding group. When the process of step S226 ends, the process proceeds to step S227.
  • step S224 If it is determined in step S224 that the i-th section has already been assigned to the group, the process proceeds to step S227. If it is determined in step S225 that there is an unspecified imaging target detail area including the coordinates of the jth section, the process proceeds to step S227.
  • step S227 the playlist generation unit 184 determines whether or not the value of the variable j is the number of sections of the image (i). If it is determined that the value of the variable j has not reached the number of sections of the image (i), the process proceeds to step S228. In step S228, the playlist generation unit 184 increments the value of the variable j by “+1” (j ++). When the process of step S228 ends, the process returns to step S224 and the subsequent processes are repeated. As described above, the processing from step S224 to step S228 is repeatedly executed, and when it is determined in step S227 that the value of the variable j has reached the number of sections of the image (i), the processing proceeds to step S229.
  • step S229 the playlist generation unit 184 determines whether the value of the variable i is the number of captured image data. If it is determined that the value of the variable i has not reached the number of captured image data, the process proceeds to step S230. In step S230, the playlist generation unit 184 increments the value of the variable i by “+1” (i ++). When the process of step S230 ends, the process returns to step S223, and the subsequent processes are repeated. As described above, the processing from step S223 to step S230 is repeatedly executed. When it is determined in step S229 that the value of the variable i has reached the number of captured image data, the image grouping processing ends, and the processing is as shown in FIG. Return to.
  • the playlist generation unit 184 When the grouping of the captured image data is performed as described above, the playlist generation unit 184 generates a multi-view viewing playlist for each group by the process of step S192 in FIG. For example, in the case of (3) above, a video belonging to a video group is selected by the group identifier (gpref attribute value of object_group) added to each captured image data in the video metadata 271 of FIG.
  • the imaging direction in the section which is imaging the imaging target detailed area of the group of each video is determined.
  • the direction (imaging direction) in which each captured image is captured is the direction viewed from the center of the region.
  • the direction reference (0 degree direction) an orientation based on the ground axis can be used, but it can also be defined on the drawing of the imaging target positional relationship information.
  • the imaging direction on the horizontal plane of each captured image is determined by an angle in a clockwise direction in which the upward direction in FIG. 9 is 0 degree.
  • the imaging positions of all the captured images are within a certain height range from the plane, but the imaging positions are arranged more three-dimensionally, for example, in the example of FIG.
  • the information indicating the image pickup direction may be expressed by two values, an angle on the horizontal plane and an angle from the horizontal plane to the up and down direction. .
  • Multi-view viewing playlist> A multi-view viewing playlist is generated as described above. This multi-view viewing playlist can be generated for the entire event or for each small event. When a playlist for the entire event is generated and the above-described group configuration changes for each small event, the time may be divided for each small event.
  • An example of the multi-view viewing playlist is shown in FIG.
  • the multi-view viewing playlist 321 shown in FIG. 20 is described in XML (Extensible Markup Language) format.
  • this multi-view viewing playlist 321 includes, for example, the following elements and information as information that can be used for reproducing captured image data (information that can be used to select captured image data to be reproduced): Has attribute.
  • MultiviewPlaylist Indicates a multi-view viewing playlist.
  • BaseURL Indicates the base URL from which captured image data is acquired.
  • Pereod @ start Indicates the imaging start time of the section (small event time).
  • Pereod @ duration Indicates the length of the section (small event time).
  • ObjectGroup @ position Indicates the position (X, Y, Z coordinates) of the imaging target in the imaging target positional relationship drawing.
  • Representation @ angle Indicates the imaging angle (imaging direction) of this captured image viewed from the imaging target.
  • xy represents an angle w of the horizontal plane
  • z represents an angle with the horizontal plane.
  • URL Indicates the URL from which each captured image data is acquired. Used in combination with the base URL specified by MultiviewPlaylist.BaseURL. Inactive: The time when the captured image data is not capturing the imaging target (that is, the period not included in the captured image data) is specified by separating with a comma. Specify relative to Time @ start based on timescale.
  • the multi-view viewing playlist 321 information (Representation) that can be used for reproduction of each captured image data obtained by capturing each imaging target, for a plurality of imaging targets, is the imaging target (ObjectGroup ) Are grouped (grouped). These pieces of information are generated for a predetermined period (duration) such as a small event, and include information indicating the start time (Pereod @ start) and the length (Pereod @ duration) of the period. Further, the multi-view appreciation playlist 321 is a list of information that can be used for searching, selecting, and reproducing captured image data for each predetermined period (small event) for a plurality of periods (events). Including.
  • the multi-view viewing playlist can be expressed as an extension of MPEGDASH MPD.
  • An example is shown in FIGS.
  • the contents of the multi-view appreciation playlist 331 shown in FIG. 21, the multi-view appreciation playlist 332 shown in FIG. 22, and the multi-view appreciation playlist 333 shown in FIG. are the same.
  • a ⁇ MultiviewGroup> element that includes information indicating the imaging direction viewed from the imaging target, listed for each area to be imaged, is newly defined, and the group as the element By enumerating ⁇ AdaptationSet> belonging to the ⁇ views> element, a group of Adaptation ⁇ ⁇ Sets to be referenced for each imaging target (area) can be shown.
  • Each ⁇ views> element has the coordinates (position attribute) of the center of the target imaging region and the r attribute representing the range, that is, the radius.
  • the value of the r attribute is the value of the radius attribute of the ⁇ group> element generated as each video metadata.
  • each Adaptation Set has a plurality of Representations with different encoding and bit rates generated from the same captured image data, it is possible to perform adaptive bit rate streaming playback of each image captured from a plurality of angles.
  • each ⁇ MultiviewGroup> element can refer to a different Adaptation Set.
  • the Inactive element is described as a child element (or attribute) of the Adaptation Set for a section in which the imaging target that is the target of the specific Multiview Group is not shown.
  • the multi-view viewing playlist database 185 stores and manages the multi-view viewing playlist.
  • the captured image data stored in the captured image database 182 is shared by the terminal device 103 (the user). That is, the accumulation server 102 provides captured image data to the terminal device 103. At that time, the accumulation server 102 provides the above-described multi-view appreciation playlist to the terminal device 103, and selects captured image data based on the multi-view appreciation playlist. In other words, the terminal device 103 (the user) selects desired captured image data based on the multi-view viewing playlist supplied from the accumulation server 102 and requests it from the accumulation server 102.
  • the playlist providing unit 186 of the accumulation server 102 acquires the multi-view viewing playlist from the multi-view viewing playlist database 185 in step S251, and transmits it to the terminal via the communication unit 164. Supply to device 103.
  • the playlist acquisition unit 231 of the terminal device 103 controls the communication unit 214 to acquire the multi-view viewing playlist in step S261.
  • the exchange of the multi-view viewing playlist may be performed at an arbitrary timing or may be performed based on a request from the terminal device 103.
  • the playlist providing unit 186 selects a desired multi-view viewing playlist and supplies it to the terminal device 103. May be.
  • the playlist providing unit 186 may select a multi-view viewing playlist recommended for the terminal device 103 (user) and supply the playlist to the terminal device 103.
  • the terminal device 103 (the user) requests a desired multi-view viewing playlist, and the playlist providing unit 186 reads the requested multi-view viewing playlist from the multi-view viewing playlist database 185. May be supplied to the terminal device 103.
  • step S262 the image selection processing unit 232 controls the output unit 212 to display a GUI (Graphical User Interface) using the multi-view appreciation playlist supplied from the accumulation server 102 on the monitor, and allows the user to display the GUI.
  • the captured image is selected based on the viewpoint appreciation playlist.
  • an image selection GUI 350 as shown in FIG. 25 may be displayed.
  • the image selection GUI 350 shows an imaging target detail area 351 in the captured image provided by the user, and each captured image whose imaging target detail area 351 is an imaging target is the center of the imaging target detail area 351. It is shown in each imaging direction viewed from the viewpoint. In the example of FIG.
  • the captured image 352 is a captured image uploaded by the user, and the captured image 353 and the captured image 354 are captured images uploaded by others.
  • the image selection processing unit 232 sets the selected captured image as a captured image to be viewed (requested from the integrated server 102). .
  • the captured image is selected for an area different from the imaging target detail area as the imaging target in the uploaded captured image.
  • an area selection GUI configured as shown in FIGS. 13 and 14 may be displayed.
  • an image selection GUI as shown in FIG. 25 may be displayed for the selected area.
  • an image selection GUI as shown in FIG. 25 may be displayed for the imaging target detail area 282 shown in FIG.
  • the captured image request unit 233 requests the captured image data of the selected captured image from the integrated server 102 by controlling the communication unit 214 in step S263. .
  • the captured image providing unit 187 of the integrated server 102 controls the communication unit 164 and accepts the request in step S252, the captured image data is read from the captured image database 182 and is transmitted to the terminal via the communication unit 164. Supply to device 103.
  • step S263 the captured image acquisition unit 234 of the terminal device 103 controls the communication unit 214 to acquire the captured image data.
  • step S264 the reproduction unit 235 reproduces the captured image data.
  • a large number of integrations are performed by using metadata added to the captured image data at the time of imaging, and a positional relationship between an imaging-capable area and an imaging target prepared in advance.
  • the captured image data is classified for each imaging target (a person, an object, or a fixed range around it), and those captured images are for a specific imaging target (or the center if it has a fixed range).
  • a multi-view appreciation playlist with information on which image is taken from which direction can be generated, and a captured image captured by a plurality of imaging devices 101 including the one captured by the user based on the playlist. It becomes possible to easily select a desired one from the above. Thereby, the value of collecting and sharing the captured image data captured by a large number of viewers can be increased.
  • image data has been described as an example, but information to be collected and shared is arbitrary.
  • audio data may be collected and shared.
  • a plurality of types of data such as audio data and image data may be accumulated and shared. That is, by applying the present technology, the user can more easily select desired content.
  • the captured image data is grouped for each imaging target detail area.
  • this grouping can be performed on the basis of an arbitrary one as long as it relates to the imaging target.
  • the captured image data may be grouped for each imaging target (person, object, region).
  • each processing unit (imaging unit 121 to communication unit 124) of the imaging device 101, the CPU 151 of the integrated server 102, the CPU 201 of the terminal device 103, and the like execute the software. What is necessary is just to make it have the structure as a computer which can be performed. Examples of the computer include a computer incorporated in dedicated hardware and a general-purpose computer capable of executing an arbitrary function by installing various programs.
  • the integrated server 102 has the configuration described with reference to FIG. 3, and as described above, the CPU 151 loads a program stored in the storage unit 163, for example, through the input / output interface 160 and the bus 154.
  • the above-described series of processing is executed by software by being loaded into the RAM 153 and executed.
  • This program can be provided by being recorded on a removable medium 171 as a package medium, for example.
  • the program can be installed in the storage unit 163 via the input / output interface 160 by attaching the removable medium 171 to the drive 165.
  • This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.
  • the program can be received by the communication unit 164 and installed in the storage unit 163.
  • this program can be installed in the ROM 152 or the storage unit 163 in advance.
  • the integrated server 102 has been described. However, the same applies to other devices, and the same configuration may be used, and the program may be executed as described above. Note that a part of the series of processes described above can be executed by hardware, and the other can be executed by software.
  • the various metadata described above may be transmitted or recorded in any form as long as it is associated with the captured image data.
  • the term “associate” means, for example, that one data can be used (linked) when one data is processed. That is, the data associated with each other may be collected as one data, or may be individual data.
  • the information associated with the captured image data may be transmitted on a transmission path different from the captured image data (or at a different timing). Further, for example, information associated with captured image data may be recorded on a recording medium different from the captured image data (or another recording area of the same recording medium).
  • the “association” may be a part of the data, not the entire data. For example, an image and information corresponding to the image may be associated with each other in an arbitrary unit such as a plurality of frames, one frame, or a part of the frame.
  • the present technology may be applied to any configuration that constitutes an apparatus or a system, such as a processor as a system LSI (Large Scale Integration), a module using a plurality of processors, a unit using a plurality of modules, and the unit. It can also be implemented as a set to which other functions are added (that is, a partial configuration of the apparatus).
  • a processor as a system LSI (Large Scale Integration)
  • a module using a plurality of processors a unit using a plurality of modules
  • the unit such as a set to which other functions are added (that is, a partial configuration of the apparatus).
  • the system means a set of a plurality of constituent elements (devices, modules (parts), etc.), and it does not matter whether all the constituent elements are in the same casing. Accordingly, a plurality of devices housed in separate housings and connected via a network and a single device housing a plurality of modules in one housing are all systems. .
  • the configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units).
  • the configurations described above as a plurality of devices (or processing units) may be combined into a single device (or processing unit).
  • a configuration other than that described above may be added to the configuration of each device (or each processing unit).
  • a part of the configuration of a certain device (or processing unit) may be included in the configuration of another device (or other processing unit). .
  • the present technology can take a configuration of cloud computing in which one function is shared and processed by a plurality of devices via a network.
  • the above-described program can be executed in an arbitrary device.
  • the device may have necessary functions (functional blocks and the like) so that necessary information can be obtained.
  • each step described in the above flowchart can be executed by one device or can be executed by a plurality of devices.
  • the plurality of processes included in the one step can be executed by being shared by a plurality of apparatuses in addition to being executed by one apparatus.
  • a plurality of processes included in one step can be executed as a process of a plurality of steps.
  • the processing described as a plurality of steps can be collectively executed as one step.
  • the program executed by the computer may be executed in a time series in the order described in this specification for the processing of the steps describing the program, or in parallel or called. It may be executed individually at a necessary timing. That is, as long as no contradiction occurs, the processing of each step may be executed in an order different from the order described above. Furthermore, the processing of the steps describing this program may be executed in parallel with the processing of other programs, or may be executed in combination with the processing of other programs.
  • An information processing apparatus including a generation unit that generates a multi-view viewing playlist including a list of information that can be used to reproduce a plurality of captured image data generated by imaging the same imaging target from different positions.
  • the information processing apparatus according to (1) wherein the information usable for reproducing the captured image data is information usable for selecting the captured image data to be reproduced.
  • the information processing apparatus according to (1) or (2), wherein information usable for reproducing the captured image data includes information indicating a position of the imaging target.
  • the information processing apparatus includes information indicating an acquisition destination of the captured image data.
  • the information processing apparatus includes information indicating an acquisition destination of the captured image data.
  • information usable for reproducing the captured image data includes information indicating a period not included in the captured image data.
  • the multi-view viewing playlist includes, for a plurality of imaging targets, a list for each imaging target of information that can be used for reproducing the captured image data. Processing equipment.
  • the information processing apparatus according to any one of (1) to (7), wherein the multi-view viewing playlist is generated for a predetermined period and includes information indicating a start time and a length of the period.
  • the multi-view viewing playlist includes a list for each imaging target of information that can be used for reproduction of the captured image data for each predetermined period for a plurality of periods. .
  • the generation unit generates the multi-view viewing playlist as an MPD (Media Presentation Description) of MPEG-DASH (Moving Picture Experts Group phase-Dynamic Adaptive Streaming over HTTP).
  • MPD Media Presentation Description
  • MPEG-DASH Motion Picture Experts Group phase-Dynamic Adaptive Streaming over HTTP
  • the information processing apparatus includes: Group the captured image data based on the imaging target; The information processing apparatus according to any one of (1) to (13), wherein a list of information that can be used for reproduction of the captured image data is generated for each group.
  • the generation unit includes: All the captured image data is grouped into one group Group the captured image data using a preset area, or Or the information processing device according to (14), wherein the captured image data is grouped according to a position of each imaging target.
  • An analysis unit for analyzing the imaging state is further provided, The information processing apparatus according to any one of (1) to (15), wherein the generation unit is configured to generate the multi-view viewing playlist based on a result of analysis by the analysis unit.
  • the information processing apparatus analyzes a state of imaging based on metadata of captured image data and information related to a venue.
  • An information processing method for generating a multi-view viewing playlist including a list of information that can be used to reproduce a plurality of captured image data generated by imaging the same imaging target from different positions.
  • 100 image providing system 101 imaging device, 102 integration server, 103 terminal device, 121 imaging unit, 122 metadata generation unit, 123 metadata addition unit, 124 communication unit, 181 integration unit, 182 imaging image database, 183 imaging status analysis Unit, 184 playlist generation unit, 185 multi-view viewing playlist database, 186 playlist provision unit, 187 captured image provision unit, 231 playlist acquisition unit, 232 image selection processing unit, 233 captured image request unit, 234 captured image acquisition Part, 235 reproduction part

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Television Signal Processing For Recording (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

La présente invention concerne un dispositif et un procédé de traitement d'informations qui permettent à un utilisateur de sélectionner plus facilement des contenus souhaités. La présente invention permet la génération d'une liste de lecture de visualisation à points de vue multiples qui comprend une liste d'informations utilisables dans la reproduction d'une pluralité d'éléments de données d'images capturées générés par l'imagerie du même sujet d'imagerie à partir de positions mutuellement différentes. Par exemple, des informations utilisables dans la reproduction de données d'images capturées peuvent être des informations utilisables dans la sélection des données d'images capturées à reproduire. La présente invention peut être appliquée à un dispositif de traitement d'informations, à un dispositif de communication, à un serveur de distribution d'images, à un dispositif de codage et analogue.
PCT/JP2018/002379 2017-02-10 2018-01-26 Dispositif et procédé de traitement d'informations WO2018147089A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2017022703 2017-02-10
JP2017-022703 2017-02-10

Publications (1)

Publication Number Publication Date
WO2018147089A1 true WO2018147089A1 (fr) 2018-08-16

Family

ID=63107405

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2018/002379 WO2018147089A1 (fr) 2017-02-10 2018-01-26 Dispositif et procédé de traitement d'informations

Country Status (1)

Country Link
WO (1) WO2018147089A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020137876A1 (fr) * 2018-12-26 2020-07-02 シャープ株式会社 Dispositif de génération, dispositif de transmission de données tridimensionnelles, et dispositif de reproduction de données tridimensionnelles
WO2022091215A1 (fr) * 2020-10-27 2022-05-05 Ａｍａｔｅｌｕｓ株式会社 Dispositif de distribution de vidéo, système de distribution de vidéo, procédé de distribution de vidéo et programme

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011078008A (ja) * 2009-10-01 2011-04-14 Nippon Hoso Kyokai <Nhk> コンテンツ共有装置、コンテンツ編集装置、コンテンツ共有プログラム、およびコンテンツ編集プログラム
WO2014007083A1 (fr) * 2012-07-02 2014-01-09 ソニー株式会社 Appareil de transmission, procédé de transmission et appareil de réseau
WO2015109228A1 (fr) * 2014-01-16 2015-07-23 Qualcomm Incorporated Opération en direct robuste de dash
JP2017504234A (ja) * 2013-11-20 2017-02-02 グーグル インコーポレイテッド マルチビューオーディオおよびビデオインタラクティブプレイバック

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011078008A (ja) * 2009-10-01 2011-04-14 Nippon Hoso Kyokai <Nhk> コンテンツ共有装置、コンテンツ編集装置、コンテンツ共有プログラム、およびコンテンツ編集プログラム
WO2014007083A1 (fr) * 2012-07-02 2014-01-09 ソニー株式会社 Appareil de transmission, procédé de transmission et appareil de réseau
JP2017504234A (ja) * 2013-11-20 2017-02-02 グーグル インコーポレイテッド マルチビューオーディオおよびビデオインタラクティブプレイバック
WO2015109228A1 (fr) * 2014-01-16 2015-07-23 Qualcomm Incorporated Opération en direct robuste de dash

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
"Information technology-Dynamic adaptive streaming over HTTP (DASH)- Part 1: Media presentation description and segment formats", ISO/IEC, 2012OMICRON 04.01 , ISO/IEC 23009-1 : 2012 (E, pages 6 - 28 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020137876A1 (fr) * 2018-12-26 2020-07-02 シャープ株式会社 Dispositif de génération, dispositif de transmission de données tridimensionnelles, et dispositif de reproduction de données tridimensionnelles
WO2022091215A1 (fr) * 2020-10-27 2022-05-05 Ａｍａｔｅｌｕｓ株式会社 Dispositif de distribution de vidéo, système de distribution de vidéo, procédé de distribution de vidéo et programme
JPWO2022091215A1 (fr) * 2020-10-27 2022-05-05
JP7208695B2 (ja) 2020-10-27 2023-01-19 Ａｍａｔｅｌｕｓ株式会社 映像配信装置、映像配信システム、映像配信方法、及びプログラム

Similar Documents

Publication Publication Date Title
US11837260B2 (en) Elastic cloud video editing and multimedia search
US11381739B2 (en) Panoramic virtual reality framework providing a dynamic user experience
US11516557B2 (en) System and method for enhanced video image recognition using motion sensors
US9418703B2 (en) Method of and system for automatic compilation of crowdsourced digital media productions
US11196788B2 (en) Method and system for aggregating content streams based on sensor data
US20150312528A1 (en) Method and mechanism for coordinated capture and organization of multimedia data
US9483228B2 (en) Live engine
US20160261930A1 (en) Video-providing method and video-providing system
JP6187811B2 (ja) 画像処理装置、画像処理方法、及び、プログラム
JP2017517789A (ja) ビデオコンテンツの時間ベース及び地理的ナビゲーションのための方法、装置並びにシステム
CN103999470A (zh) 合并多个记录视频时间线的系统
WO2018147089A1 (fr) Dispositif et procédé de traitement d&#39;informations
US10664225B2 (en) Multi vantage point audio player
CN105814905A (zh) 用于使使用信息在装置与服务器之间同步的方法和系统
US20150304724A1 (en) Multi vantage point player
US10375456B2 (en) Providing highlights of an event recording
JP6060085B2 (ja) コンテンツ管理装置、コンテンツ管理方法、プログラム、およびコンテンツの表示方法
KR101843025B1 (ko) 카메라워크 기반 영상합성 시스템 및 영상합성방법
US20150256762A1 (en) Event specific data capture for multi-point image capture systems
JP6640130B2 (ja) 順応性のあるクラウド編集とマルチメディア検索
US10911839B2 (en) Providing smart tags
Bailer et al. Multi-sensor concert recording dataset including professional and user-generated content
van Deventer et al. Media orchestration between streams and devices via new MPEG timed metadata
US20220053248A1 (en) Collaborative event-based multimedia system and method
EP3142116A1 (fr) Procédé et dispositif pour capturer une vidéo dans une acquisition communautaire

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18750961

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18750961

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP