WO2024134861A1 - 移動端末、情報処理装置、および情報処理方法 - Google Patents

移動端末、情報処理装置、および情報処理方法 Download PDF

Info

Publication number
WO2024134861A1
WO2024134861A1 PCT/JP2022/047550 JP2022047550W WO2024134861A1 WO 2024134861 A1 WO2024134861 A1 WO 2024134861A1 JP 2022047550 W JP2022047550 W JP 2022047550W WO 2024134861 A1 WO2024134861 A1 WO 2024134861A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
information
unit
terminal
location
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/047550
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
健二 鳴島
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Micware Co Ltd
Original Assignee
Micware Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Micware Co Ltd filed Critical Micware Co Ltd
Priority to JP2024565530A priority Critical patent/JP7688473B2/ja
Priority to PCT/JP2022/047550 priority patent/WO2024134861A1/ja
Publication of WO2024134861A1 publication Critical patent/WO2024134861A1/ja
Priority to US19/242,974 priority patent/US12591401B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/66—Remote control of cameras or camera parts, e.g. by remote control devices
    • H04N23/661—Transmitting camera control signals through networks, e.g. control via the Internet
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H04N5/91—Television signal processing therefor
    • H04N5/92—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Definitions

  • the present invention relates to a mobile terminal that can record video while moving around indoors.
  • Patent Document 1 Conventionally, there has been technology that provides users with image data taken with an in-vehicle camera in a location and environment of the user's choice (see, for example, Patent Document 1).
  • the mobile terminal of the first invention is a mobile terminal that includes a position acquisition unit that acquires relative position information that identifies the relative position of the mobile terminal indoors, a shooting unit that shoots a video, a configuration unit that associates the relative position information or absolute position information acquired using the relative position information with the video, acquires an additional video, and a video output unit that outputs the additional video.
  • This configuration makes it possible to provide a mobile terminal that can capture video that can be used appropriately while moving around indoors.
  • the mobile terminal of the second invention further includes an attribute value acquisition unit that acquires one or more camera attribute values including directional information indicating the direction in which the image capture unit captures images, and the configuration unit is a mobile terminal that acquires an additional video in which the one or more camera attribute values acquired by the attribute value acquisition unit are also associated with the video.
  • This configuration allows for proper provision of video footage shot while moving around indoors.
  • the position acquisition unit is a mobile terminal equipped with absolute position acquisition means for acquiring absolute position information of the mobile terminal, and relative position acquisition means for acquiring relative position information when absolute position information cannot be acquired.
  • This configuration allows for proper provision of video footage shot while moving around indoors.
  • the mobile terminal of the fourth invention is a mobile terminal that further includes, compared to any one of the first to third inventions, a position transmission unit that transmits position information, which is relative position information or absolute position information obtained using the relative position information, to an information processing device, a movement instruction receiving unit that receives a transmission instruction from the information processing device, and a movement video transmission unit that transmits a video in response to the transmission instruction.
  • a position transmission unit that transmits position information, which is relative position information or absolute position information obtained using the relative position information, to an information processing device
  • a movement instruction receiving unit that receives a transmission instruction from the information processing device
  • a movement video transmission unit that transmits a video in response to the transmission instruction.
  • This configuration allows for proper provision of video footage shot while moving around indoors.
  • the mobile terminal of the fifth invention is a mobile terminal according to any one of the first to third inventions, further comprising a mobile inquiry receiving unit that receives an inquiry having location identification information that identifies an indoor location, a mobile determination unit that determines whether the relative location information or absolute location information obtained using the relative location information matches the inquiry, and a mobile video transmission unit that transmits a video when the mobile determination unit determines that the information matches the inquiry.
  • This configuration allows for proper provision of video footage shot while moving around indoors.
  • the information processing device of the sixth invention is an information processing device that includes an inquiry receiving unit that receives an inquiry having location identification information that identifies an indoor location from a user terminal, a video acquisition unit that acquires a video that matches the inquiry and is transmitted from a mobile terminal that activates the indoor location, and a video transmission unit that transmits the video to the user terminal.
  • This configuration provides a platform that allows appropriate use of videos captured by mobile devices moving around indoors.
  • the information processing device of the seventh invention further comprises a terminal management unit in which terminal information including location information identifying the location of the mobile terminal is stored in association with each of one or more mobile terminals, a location receiving unit that receives location information from each of one or more mobile terminals, which is relative location information or absolute location information obtained using the relative location information, a location storage unit that stores the location information in the terminal management unit in association with the mobile terminal, and a terminal determination unit that determines the mobile terminal corresponding to the location information corresponding to the location identified by the location identification information included in the query, and the video acquisition unit is an information processing device that acquires video captured by the mobile terminal determined by the terminal determination unit.
  • a terminal management unit in which terminal information including location information identifying the location of the mobile terminal is stored in association with each of one or more mobile terminals
  • a location receiving unit that receives location information from each of one or more mobile terminals, which is relative location information or absolute location information obtained using the relative location information
  • a location storage unit that stores the location information in the terminal management unit in association with
  • This configuration provides a platform that allows appropriate use of videos captured by mobile devices moving around indoors.
  • the information processing device of the eighth invention is an information processing device that further includes a terminal management unit that stores terminal information, which is information about the mobile terminal, in association with each of one or more mobile terminals, and a query transmission unit that transmits queries to each of one or more mobile terminals, in comparison with the sixth invention, and the video acquisition unit receives videos from the mobile terminals in response to the transmission of the queries.
  • This configuration provides a platform that allows appropriate use of videos captured by mobile devices moving around indoors.
  • the information processing device of the ninth invention is an information processing device according to any one of the sixth to eighth inventions, further comprising a video composition unit that acquires videos taken by a plurality of mobile terminals, temporally combines the plurality of videos acquired by the video acquisition unit, or spatially fuses the plurality of videos acquired by the video acquisition unit, to compose a single video, and the video transmission unit transmits the video composed by the video composition unit to the user terminal.
  • This configuration provides a platform that allows appropriate use of videos captured by mobile devices moving around indoors.
  • the information processing device of the tenth invention is an information processing device according to any one of the sixth to ninth inventions, further comprising a rights holder processing unit that corresponds the video acquired by the video acquisition unit to a rights holder identifier that identifies the rights holder of the video, and performs rights holder processing, which is processing related to the rights holder identified by the rights holder identifier associated with the video.
  • This configuration allows for appropriate processing regarding video rights holders.
  • the information processing device of the eleventh invention is an information processing device in which the rights holder processing unit is equipped with a reward means for performing reward processing, which is processing for providing rewards to each rights holder identified by a rights holder identifier, in comparison with the tenth invention.
  • This configuration makes it possible to provide compensation to the rights holder of videos captured by the mobile terminal.
  • the information processing device of the twelfth invention is an information processing device according to the eleventh invention, in which the compensation means performs compensation processing, which is a process of determining a compensation for the previous rights holder using a video attribute value corresponding to the video and providing the compensation.
  • This configuration makes it possible to provide appropriate compensation to the rights holder of videos captured by the mobile terminal.
  • the information processing device of the thirteenth invention is an information processing device in which, compared to the tenth invention, the rights holder processing unit is equipped with a first preservation means that performs a first preservation process in which the video is stored in association with a set of attribute values that correspond to the video.
  • This configuration allows videos to be preserved.
  • the information processing device of the fourteenth invention is an information processing device in which, compared to the tenth invention, the rights holder processing unit is equipped with a second security means that performs a second security process in which the video is stored in association with a rights holder identifier corresponding to the video.
  • This configuration ensures that the rights holder of the video is the appropriate rights holder.
  • the information processing device of the fifteenth invention is an information processing device in which the rights holder processing unit is equipped with a third security means for storing the video in association with a rights holder identifier that identifies the user of the user terminal, in comparison with the tenth invention.
  • This configuration allows the rights holder of a video to become the rights holder who needs that video.
  • the information processing device of the sixteenth invention is an information processing device according to any one of the thirteenth to fifteenth inventions, in which the rights holder processing unit is provided with a fourth preservation means that performs a fourth preservation process that accumulates preservation information, including access information for accessing the video, in the blockchain.
  • This configuration makes it possible to preserve management information for videos that need to be preserved.
  • the mobile terminal of the present invention can provide appropriate video footage captured while moving around indoors.
  • Block diagram of information system A Block diagram of the information processing device 1 A flowchart illustrating an example of the operation of the information processing device 1. A flowchart illustrating an example of the terminal determination process A flowchart illustrating an example of the moving image fusion process. A flowchart illustrating an example of the image fusion process. A flowchart illustrating an example of the maintenance process. A flowchart illustrating an example of the fourth maintenance process. A flowchart illustrating an example of the compensation process.
  • a flowchart for explaining an example of the operation of the mobile terminal 2 A flowchart illustrating an example of a position acquisition process
  • a flowchart for explaining an example of the operation of the user terminal 3 A diagram showing the terminal management table Block diagram of information system B in embodiment 2
  • Block diagram of the information processing device 4 A flowchart illustrating an example of the operation of the information processing device 4.
  • a flowchart for explaining an example of the operation of the mobile terminal 5 Overview of the computer system according to the above embodiment. Block diagram of the computer system
  • a mobile terminal that acquires relative position information of the mobile terminal indoors, associates a captured video with the relative position information, and outputs the video.
  • a mobile terminal that outputs one or more camera attribute values, including directional information indicating the shooting direction, in association with a video.
  • a mobile terminal that transmits acquired location information to an information processing device and transmits a captured video to the information processing device in response to a transmission instruction received from the information processing device.
  • an information processing device uses location information received from a mobile terminal to determine a mobile terminal that shoots a video that matches an inquiry received from a user terminal, instructs the mobile terminal to transmit the video, receives the video from the mobile terminal, and transmits the video to the user terminal that sent the inquiry.
  • an information processing device that temporally combines videos received from multiple mobile terminals 2 to form a single video, or spatially fuses the videos to form a single video, and transmits the single video to a user terminal.
  • an information processing device that performs rights holder processing, which is processing related to the rights holder of the output video, will be described.
  • the rights holder processing is, for example, a compensation process described below, and various types of preservation processing described below.
  • information X being associated with information Y means that information Y can be obtained from information X, or information X can be obtained from information Y, and the method of association is not important.
  • Information X and information Y may be linked, may exist in the same buffer, information X may be included in information Y, or information Y may be included in information X, etc.
  • FIG. 1 is a conceptual diagram of an information system A in this embodiment.
  • Information system A includes an information processing device 1, one or more mobile terminals 2, and one or more user terminals 3.
  • the information processing device 1 is a server that provides videos transmitted by one or more mobile terminals 2 to a user terminal 3.
  • the information processing device 1 is, for example, a so-called server, such as a cloud server or an ASP server, but the type is not important.
  • the information processing device 1 may also be a device that constitutes a blockchain.
  • a video may be, for example, a video of 60 frames per second or 30 frames per second, but it may also be a collection of multiple still images captured with an interval of a specified time (for example, one minute) or more, or a collection of multiple still images captured when specified conditions are met.
  • the mobile terminal 2 is a terminal that can move.
  • the mobile terminal 2 is a terminal that is installed on a moving object and that takes video.
  • the mobile terminal 2 is, for example, a smartphone, a tablet terminal, a camera with a communication function, glasses with a camera, or a smartwatch with a camera.
  • installation usually means that it is fixed, but it may also be considered to include being in contact with or held.
  • a mobile object is an object that moves, for example, equipment that moves underground, or a living thing.
  • a living thing is usually a person, but it may also be a dog or cat, etc.
  • the user terminal 3 is a terminal used by a user.
  • a user is someone who watches videos or needs videos.
  • the user terminal 3 may have the functions of a mobile terminal 2.
  • the user terminal 3 may be a terminal of a user who provides videos.
  • the user terminal 3 may be, for example, a smartphone, a tablet terminal, a so-called personal computer, a navigation terminal, or the like, and the type is not important.
  • the information processing device 1 and one or more mobile terminals 2, and the information processing device 1 and one or more user terminals 3, can typically communicate with each other via a network such as the Internet.
  • FIG. 2 is a block diagram of information system A in this embodiment.
  • FIG. 3 is a block diagram of information processing device 1.
  • the information processing device 1 comprises a storage unit 11, a receiving unit 12, a processing unit 13, and a transmitting unit 14.
  • the storage unit 11 comprises a terminal management unit 111.
  • the receiving unit 12 comprises a location receiving unit 121, and an inquiry receiving unit 122.
  • the processing unit 13 comprises a location accumulation unit 131, a terminal determination unit 132, a video acquisition unit 133, a video composition unit 134, and a rights holder processing unit 135.
  • the rights holder processing unit 135 comprises a first maintenance means 1351, a second maintenance means 1352, a third maintenance means 1353, a fourth maintenance means 1354, and a remuneration means 1355.
  • the transmitting unit 14 comprises a video transmission unit 141.
  • the mobile terminal 2 comprises a movement storage unit 21, a movement receiving unit 22, a movement processing unit 23, a movement transmitting unit 24, and a movement output unit 25.
  • the movement receiving unit 22 comprises a movement instruction receiving unit 221.
  • the movement processing unit 23 comprises a position acquisition unit 231, a photographing unit 232, an attribute value acquisition unit 233, and a configuration unit 234.
  • the position acquisition unit 231 comprises absolute position acquisition means 2311 and relative position acquisition means 2312.
  • the movement transmitting unit 24 comprises a position transmission unit 241 and a movement video transmission unit 242.
  • the movement output unit 25 comprises a video output unit 251.
  • the user terminal 3 includes a user storage unit 31 , a user reception unit 32 , a user processing unit 33 , a user transmission unit 34 , a user reception unit 35 , and a user output unit 36 .
  • the storage unit 11 stores various types of information.
  • the various types of information include, for example, terminal information (described later), videos, and video attribute values associated with the videos.
  • Video attribute values are the attribute values of a video.
  • Video attribute values are, for example, environmental information.
  • Environmental information is information about the environment in which a video was shot.
  • Environmental information is, for example, location information, time information, weather information, temperature information, and seasonal information, which will be described later.
  • Time information is information that specifies the time when a video was shot. The time when a video was shot may be around the time when the video was shot, and accuracy is not required.
  • Time information is, for example, the time, year/month/date/time, year/month/date/time/minute, year/month/date/time/minute/second, year/month/date, and month/day. In other words, the granularity of the time indicated by the time information does not matter.
  • Weather information is information that specifies the weather at the location (or region) where the video was shot. Weather information is, for example, “sunny,” “rain,” “snow,” or “cloudy.” Temperature information is information that specifies the temperature at the location where the video was shot. Temperature information is, for example, “25 degrees” or “above 30 degrees.” Seasonal information is information that specifies the season at the location where the video was shot. Examples of seasonal information are “spring,” “summer,” “early summer,” and "winter.”
  • the terminal management unit 111 stores one or more pieces of terminal information.
  • the terminal information corresponds to the mobile terminal 2.
  • the terminal information is information related to the mobile terminal 2.
  • the terminal information here typically includes a terminal identifier that is an identifier of the mobile terminal 2, location information, and terminal communication information.
  • the mobile terminal 2 here is typically a terminal that can transmit video.
  • the terminal identifier is information that identifies the mobile terminal 2.
  • the terminal identifier may be a rights holder identifier that identifies the rights holder who is the user of the mobile terminal 2.
  • the terminal identifier is, for example, the ID of the mobile terminal 2, the user identifier of the user of the mobile terminal 2, the name of the mobile terminal 2, the IP address of the mobile terminal 2, or the MAC address of the mobile terminal 2.
  • a rights holder is someone who has some rights regarding the video captured by the mobile terminal 2.
  • a rights holder is, for example, the owner of the video, the copyright holder of the video, the owner of the mobile terminal 2 that captured the video, or the person with the right to dispose of the video.
  • a rights holder is, for example, the initial rights holder of the video.
  • the rights holder is usually the owner of the mobile terminal 2, but it can be anyone who has rights to the video captured by the mobile terminal 2.
  • the rights holder identifier may be a terminal identifier.
  • the rights holder identifier is, for example, the rights holder's ID, the rights holder's name, the rights holder's email address, or the rights holder's phone number.
  • the rights holder's ID is, for example, a user identifier.
  • Location information is information that specifies the position of the mobile terminal 2.
  • the location information is, for example, absolute location information of the mobile terminal 2, or relative location information of the mobile terminal 2.
  • Absolute location information is, for example, (latitude, longitude) or (movement, longitude, altitude).
  • Relative location information is information that specifies a location relative to a reference point.
  • Relative location information is, for example, information that specifies a relative location indoors.
  • Relative location information is, for example, a relative location indoors, or an area identifier and a relative location indoors.
  • a relative location indoors is, for example, (x coordinate value, y coordinate value) relative to a reference point (0,0), or (x coordinate value, y coordinate value, h coordinate value) relative to a reference point (0,0,0).
  • the h coordinate value is altitude (height).
  • the area identifier is information that identifies the area corresponding to the relative position information.
  • An area usually has a size, but it may be a single location.
  • the area identifier is the ID of an indoor area, or the name of an indoor area (hereinafter referred to as the area name, where appropriate).
  • the area name is, for example, the name of an underground mall, the name of a station, the name of a building, or the name of a street.
  • the terminal communication information is information for communicating with the mobile terminal 2.
  • the terminal communication information is, for example, the IP address of the mobile terminal 2, the MAC address of the mobile terminal 2, and the ID of the communication app installed on the mobile terminal 2.
  • the receiving unit 12 receives various information and instructions from the mobile terminal 2 or the user terminal 3.
  • the various information and instructions are, for example, location information, inquiries, videos, and additional videos described below.
  • the receiving unit 12 may receive video from one or more mobile terminals 2 in pairs with the location information. In such a case, it is preferable for the receiving unit 12 to receive the video continuously. Continuous reception means, for example, receiving periodically or receiving constantly. The interval at which the video is received periodically may vary slightly.
  • the location receiving unit 121 receives location information from one or more mobile terminals 2.
  • Location information is information that identifies the location of a mobile terminal 2.
  • the location information received by the location receiving unit 121 is, for example, relative location information, a region identifier and relative location information, or absolute location information acquired using relative location information.
  • the location receiving unit 121 normally receives location information in association with a terminal identifier.
  • the location receiving unit 121 continuously receives the latest location information from one or more mobile terminals 2.
  • the inquiry receiving unit 122 receives inquiries from one or more user terminals 3.
  • a query can be said to be an instruction to transmit a video.
  • a query has location-specific information.
  • a query may have absolute location information.
  • a query may have one or more dynamic attribute values. Note that one or more dynamic attribute values contained in a query usually constitute a condition linked with AND.
  • the location identification information has, for example, a floor identifier.
  • the floor identifier is information that identifies a floor.
  • the floor identifier is, for example, "3rd floor,” “basement,” “2nd basement floor,” or "rooftop.”
  • the processing unit 13 performs various types of processing. For example, the various types of processing are performed by the location accumulation unit 131, the terminal determination unit 132, the video acquisition unit 133, the video composition unit 134, and the rights holder processing unit 135.
  • the processing unit 13 may store the videos continuously received by the receiving unit 12 in association with the location information and the terminal identifier.
  • the location accumulation unit 131 accumulates the location information received by the location receiving unit 121 from the mobile terminal 2 in the terminal management unit 111 in association with the mobile terminal 2.
  • the location accumulation unit 131 accumulates the location information in the terminal management unit 111, for example, in pairs with a terminal identifier.
  • the terminal determination unit 132 determines one or more mobile terminals 2 that correspond to the location information corresponding to the location specified by the location specification information included in the received inquiry.
  • the terminal determination unit 132 may determine one or more mobile terminals 2 that correspond to the location information corresponding to the absolute location information included in the received inquiry.
  • the terminal determination unit 132 determines one or more mobile terminals 2 that correspond to the location information included in the inquiry and the location information that satisfies the approximation condition. It is preferable that such location information is the latest location information of each mobile terminal 2 in the terminal management unit 111.
  • the approximation condition is that the two locations are close to each other.
  • the approximation condition is that the location specified by the location information is close to the location indicated by the location information.
  • the approximation condition is that the distance between the location specified by the location information and the location indicated by the location information is within a threshold or smaller than a threshold. Note that the distance is usually a physical distance, but it may also be travel time.
  • the terminal determination unit 132 determines the mobile terminal 2 as follows. (1) When the location-specific information includes a location name
  • the terminal determination unit 132 refers to a location name database and obtains relative location information corresponding to the location name included in the query. Next, the terminal determination unit 132 determines, for example, one or more mobile terminals 2 corresponding to the relative location information that satisfies the approximate condition with respect to the location indicated by the relative location information.
  • a location name database is stored in the storage unit 11 or an external device.
  • the location information received by the location receiving unit 121 is relative location information.
  • the location name database has one or more pairs of a location name and relative location information.
  • the location name database may have one or more pairs of an area identifier, a location name, and relative location information.
  • the location name database may have one or more pairs of an area identifier, a floor identifier, a location name, and relative location information.
  • the terminal determination unit 132 determines, for example, one or more mobile terminals 2 that correspond to location information that matches the area identifier included in the inquiry and satisfies the approximation condition with the relative location information included in the inquiry.
  • the location information received by the location receiving unit 121 is the area identifier and the relative location information.
  • the terminal determination unit 132 refers to a point name database and acquires relative position information corresponding to the point name included in the query.
  • such relative position information is information that specifies a relative position on a floor identified by a floor identifier in an area identified by an area identifier.
  • the terminal determination unit 132 determines, for example, one or more mobile terminals 2 that correspond to location information that matches the area identifier included in the inquiry, matches the floor identifier included in the inquiry, and satisfies the approximation condition with the acquired relative location information.
  • the location information received by the location receiving unit 121 is the area identifier, floor identifier, and relative location information.
  • the position accumulation unit 131 acquires absolute position information for the received relative position information from the relative position information received by the position receiving unit 121, for example, using absolute position information of a reference point for the relative position information. The position accumulation unit 131 then accumulates the acquired absolute position information in the terminal management unit 111, pairing it with the terminal identifier of the mobile terminal 2. Through this process, absolute position information indicating the latest position of each of one or more mobile terminals 2 is stored in the terminal management unit 111.
  • the terminal determination unit 132 refers to the terminal management unit 111 and determines one or more mobile terminals 2 that correspond to the absolute location information included in the query and that satisfy the approximation conditions.
  • the video acquisition unit 133 acquires videos that match the query and are sent from the mobile terminal 2 that is activated indoors.
  • the video acquisition unit 133 may acquire videos captured by each of multiple mobile terminals 2. For example, the video acquisition unit 133 receives a video that matches the query from one mobile terminal 2. Then, when the one mobile terminal 2 moves and no longer matches the query, the video acquisition unit 133 receives a video that matches the query from another mobile terminal 2.
  • the video acquisition unit 133 acquires, for example, a video captured by the mobile terminal 2 determined by the terminal determination unit 132.
  • the video acquisition unit 133 acquires, for example, terminal communication information corresponding to each of the one or more mobile terminals 2 determined by the terminal determination unit 132 from the terminal management unit 111.
  • the video acquisition unit 133 transmits, for example, a transmission instruction to the mobile terminal 2 corresponding to each of the one or more pieces of terminal communication information.
  • the video acquisition unit 133 receives video from each of the one or more mobile terminals 2 in response to the transmission of the transmission instruction.
  • the transmission instruction may be transmitted by the transmission unit 14.
  • the reception of the video may be performed by the reception unit 12.
  • the transmission instruction is an instruction to transmit the captured video.
  • the video acquisition unit 133 acquires from the storage unit 11 a video that is paired with the terminal identifier of the mobile terminal 2 determined by the terminal determination unit 132. In this case, the mobile terminal 2 continuously transmits the captured video to the information processing device 1.
  • the video composition unit 134 for example, combines multiple videos acquired by the video acquisition unit 133 in time to compose one video.
  • one video combined in time is one video made by connecting multiple original videos.
  • multiple videos that are the source of one video combined in time are shot, it is preferable that they are continuous in time, but they may be separated in time.
  • the time when one video that is the source of one video was shot may be, for example, 8:20, and the day when another video that is the source of the one video was shot may be, for example, 8:25.
  • One video combined in time is appropriately referred to as a combined video.
  • the video composition unit 134 also composes a single video by, for example, spatially fusing each of the multiple videos acquired by the video acquisition unit 133.
  • a single spatially fused video is a video that is constructed by constructing a frame using some or all of the frames that compose each of the multiple original videos for each of the multiple frames that compose each of the multiple original videos, and then connecting the constructed frames in time. Note that at least one of the frames that compose the single video is a frame that contains some or all of the frames of each of the multiple original videos.
  • a single spatially fused video is appropriately referred to as a fused video.
  • the video composition unit 134 for example, combines the multiple videos acquired by the video acquisition unit 133 in the order of the time information corresponding to the videos to compose a single video.
  • the video composition unit 134 acquires parts of the multiple videos acquired by the video acquisition unit 133, and combines the parts of the videos in order to compose a single video.
  • Combining multiple videos in time usually means combining parts of the videos taken by multiple mobile terminals 2 in the order of the time they were taken.
  • Combining parts of videos in order may also mean passing parts of the videos in order to the video transmission unit 141.
  • Combining in time order corresponding to the videos means combining the videos in the time order indicated by the time information corresponding to the videos, or combining the videos in the order in which they were received.
  • Combining multiple videos in time to compose a single video should result in a single video that the user can see.
  • the video composition unit 134 for example, spatially fuses parts of multiple videos acquired by the video acquisition unit 133, each of which has different position information associated with it, to compose a single video.
  • the video composition unit 134 for example, composes a single frame using some or all of the frames that compose each of the multiple videos acquired by the video acquisition unit 133, and connects the frames in time to compose a single video.
  • the process of spatially fusing frames included in each of a plurality of moving images is, for example, the following processes (a) and (b).
  • the video composition unit 134 performs, for example, a process of matching the direction and scale of each of the multiple frames that are the subject of the spatial connection process. Next, the video composition unit 134 detects, for example, identical areas in each of the multiple frames. Next, the video composition unit 134 performs, for example, a process of overlapping multiple frames that have the same area to compose one wide-area frame. Note that it is possible to detect the same area in each of the multiple frames using publicly known technology. (b) Machine learning-based methods
  • the video composition unit 134 for example, provides multiple frames and a learning model to a module that performs machine learning prediction processing, executes the module, and obtains a single frame over a wide area.
  • the learning model is obtained by providing multiple training data sets, each of which has multiple frames as explanatory variables and a single wide-area frame composed of the multiple frames as a target variable, to a module that performs machine learning learning processing, and then executing the module.
  • the learning model may also be called a learner, a classifier, a classification model, etc.
  • the machine learning algorithm deep learning is preferable, but any algorithm such as random forest is also acceptable.
  • various machine learning functions such as the TensorFlow library and the random forest module of the R language, as well as various existing libraries, can be used.
  • Rights holder processing unit 135 performs rights holder processing.
  • Rights holder processing is processing related to a rights holder identified by a rights holder identifier associated with a video.
  • Rights holder processing unit 135 performs rights holder processing, which is processing in response to the video sending unit 141 sending a video, and is processing related to a rights holder identified by a rights holder identifier associated with the video.
  • Rights holder processing is, for example, a first security processing described below, a second security processing described below, a third security processing described below, a fourth security processing described below, and a compensation processing described below.
  • the rights holder identifier corresponding to a video is, for example, a rights holder identifier corresponding to one or more videos that are the basis of the constructed video, or an identifier of a user who requested the video.
  • the user who requested the video is the user of the user terminal 3 that sent the inquiry.
  • the second preservation means 1352 performs a second preservation process in which a video composed by the video composition unit 134 is stored in association with a rights holder identifier corresponding to one or more videos that are the basis of the video.
  • the first preservation means 1351 or the second preservation means 1352 may store a video composed by the video composition unit 134 in association with a set of attribute values corresponding to one or more videos that were the basis of the video, and with a rights holder identifier corresponding to one or more videos that were the basis of the video.
  • the third security means 1353 stores a video composed by the video composition unit 134 in association with a rights holder identifier that identifies the user of the user terminal 3.
  • the user of the user terminal 3 here is the person who watches the video.
  • the user terminal 3 here is, for example, the terminal that sent the inquiry.
  • the destination where a video is stored is, for example, the storage unit 11, but it may also be another device, another device that constitutes the blockchain, etc.
  • the fourth preservation means 1354 performs a fourth preservation process that accumulates preservation information.
  • the preservation information includes access information for accessing one of the accumulated videos.
  • the order of accumulation of a video and the fourth preservation process for the preservation information corresponding to that video does not matter.
  • the fourth preservation means 1354 performs a fourth preservation process, for example, of storing preservation information including access information for accessing a video that has been composed and stored by the video composition unit 134 in the blockchain.
  • the fourth security means 1354 stores the security information in the blockchain. That is, it is preferable that the fourth security means 1354 stores the security information in the distributed ledger of the blockchain. It is preferable that the fourth security means 1354 registers the security information as an NFT (non-fungible token). It is preferable that the fourth security means 1354 registers the security information in a distributed file system in an IPFS (Inter Planetary File System) network.
  • NFT non-fungible token
  • IPFS Inter Planetary File System
  • the preservation information is information for ensuring the originality of a video. It can be said that the preservation information is heading information for a video.
  • the preservation information is, for example, access information and one or more video attribute values. It is preferable that the preservation information has, for example, one or more rights holder identifiers.
  • the preservation information has multiple rights holder identifiers, the rights holder of the video may be shared, and the multiple rights holder identifiers may be rights holder history information.
  • the rights holder history information is a collection of rights holder identifiers and is information that indicates the history of the rights holder.
  • the fourth preservation process can guarantee the originality of the preservation information of a registered video.
  • the access information is information for accessing a video.
  • the access information is information that identifies the storage destination of the video.
  • the access information is, for example, a URL or a URI.
  • the preservation information includes information (which may be called a flag) indicating whether the video can be provided to a third party.
  • the flag may be, for example, information indicating that the video can be viewed by a third party, information indicating that the video can be sold, or information indicating that neither viewing nor selling is permitted.
  • the compensation means 1355 performs compensation processing for each rights holder identified by a rights holder identifier corresponding to one or more videos that are the basis of a video composed by the video composition unit 134.
  • the reward process is a process of giving a reward.
  • the reward process is, for example, a process of increasing points managed in pair with one or more rights holder identifiers associated with a video.
  • the reward process is, for example, a process of making a payment to a rights holder identified by one or more rights holder identifiers associated with a video.
  • the reward process is, for example, a process of transmitting a video or other content to a user terminal 3 of a rights holder identified by one or more rights holder identifiers associated with a video.
  • the reward process need only give a benefit to a rights holder identified by one or more rights holder identifiers associated with a video, and the content of the benefit is not important.
  • the reward may be, for example, money, points, goods, content, etc., and the content is not important.
  • the compensation means 1355 performs compensation processing, which is a process of acquiring one or more video attribute values corresponding to one or more videos that are the basis of a video transmitted by the video transmission unit 141, determining compensation for each of multiple rights holders using the one or more video attribute values, and awarding the compensation.
  • the one or more video attribute values here are, for example, the amount of video data, the duration of the video, the number of frames in the video, and the resolution of the video.
  • the reward means 1355 preferably performs reward processing, which is a process of acquiring a reward amount corresponding to a service identifier that identifies a service provided to the target video, and giving a reward corresponding to the reward amount.
  • the service identifier is, for example, "view” or "purchase.”
  • the storage unit 11 stores the reward amount corresponding to the service identifier, or information for determining the reward amount corresponding to the service identifier.
  • the reward means 1355 performs reward processing, which is, for example, a process of acquiring a reward amount using one or more pieces of information from one or more video attribute values and one or more service identifiers, and granting a reward corresponding to the reward amount.
  • reward processing is, for example, a process of acquiring a reward amount using one or more pieces of information from one or more video attribute values and one or more service identifiers, and granting a reward corresponding to the reward amount.
  • an arithmetic formula or a correspondence table corresponding to each of the multiple service identifiers is stored in the storage unit 11.
  • the arithmetic formula is a formula that calculates the reward amount using one or more video attribute values as parameters.
  • the correspondence table is a table having multiple pieces of correspondence information that manage the reward amounts corresponding to one or more video attribute values.
  • the reward means 1355 normally performs a process of charging a user who has enjoyed a service related to the target video with a reward.
  • the process of charging a user with a reward is, for example, a process of charging the user with the amount of reward acquired.
  • the process of charging a user with a reward is, for example, a process of charging the user with the amount of reward acquired and the amount of profit obtained by the operator of the information processing device 1.
  • the process of charging a user with a reward is, for example, a process of deducting points corresponding to the user who has enjoyed the service, a payment process using the user's credit card number, etc.
  • the transmission unit 14 transmits various types of information or instructions to the mobile terminal 2 or the user terminal 3.
  • the various types of information or instructions are, for example, video, a transmission instruction, and a stop instruction.
  • a stop instruction is an instruction to stop the transmission of video.
  • the video sending unit 141 transmits the video composed by the video composition unit 134 to the user terminal 3.
  • the video composed by the video composition unit 134 may be the video acquired by the video acquisition unit 133 itself.
  • the mobile storage unit 21 stores various types of information.
  • the various types of information are, for example, video, attribute value sets, rights holder identifiers, and camera attribute values.
  • the camera attribute values are the attribute values of the camera.
  • the camera attribute values are, for example, the angle of view, directional information, and resolution.
  • One or more still images (which may also be called fields or frames) that make up a video are associated with, for example, one or more video attribute values that make up an attribute value set.
  • the one or more video attribute values may correspond to all still images, to some of the still images, or to multiple still images.
  • the mobile receiving unit 22 receives various types of information.
  • the various types of information are, for example, a transmission instruction and a stop instruction.
  • the mobile instruction receiving unit 221 receives a transmission instruction from the information processing device 1.
  • the movement processing unit 23 performs various types of processing.
  • the various types of processing are, for example, processing performed by a position acquisition unit 231, a photographing unit 232, an attribute value acquisition unit 233, and a configuration unit 234.
  • the position acquisition unit 231 acquires relative position information that identifies the relative position of the mobile terminal 2 indoors.
  • the position acquisition unit 231 acquires absolute position information that indicates the position of the mobile terminal 2 outdoors.
  • the absolute position acquisition means 2311 acquires absolute position information of the mobile terminal 2.
  • the absolute position acquisition means 2311 acquires absolute position information, for example, by a GPS receiver. It is preferable that the absolute position acquisition means 2311 stores the acquired latest absolute position information in the mobile storage unit 21.
  • the relative position acquisition means 2312 acquires relative position information of the mobile terminal 2. It is preferable that the relative position acquisition means 2312 stores the acquired latest relative position information in the mobile storage unit 21.
  • the relative position acquisition means 2312 acquires relative position information when the absolute position acquisition means 2311 is no longer able to acquire absolute position information.
  • the relative position acquisition means 2312 acquires relative position information when the mobile terminal 2 moves from outdoors to indoors.
  • the relative position acquisition means 2312 acquires relative position information of the mobile terminal 2, for example, by LiDAR.
  • the relative position acquisition means 2312 receives signals from, for example, three or more signal transmitters, and calculates relative position information using the strength of each of the three or more signals and the position information of each of the three or more signal transmitters.
  • the relative position acquisition means 2312 receives relative position information from, for example, a signal transmitter (for example, an RFID tag or a beacon) that transmits relative position information.
  • a signal transmitter for example, an RFID tag or a beacon
  • the relative position acquisition means 2312 receives an area identifier, for example, from a signal transmitter installed indoors.
  • the relative position acquisition means 2312 receives an area identifier and a floor identifier, for example, from a signal transmitter installed indoors.
  • the relative position acquisition means 2312 receives a location name, for example, from a signal transmitter installed indoors. Note that the technology by which the relative position acquisition means 2312 acquires the relative position information that identifies an indoor position is not important.
  • the image capturing unit 232 captures video.
  • the image capturing unit 232 captures video while the mobile terminal 2 is moving. It is preferable that the captured video be associated with location information acquired by the location acquisition unit 231.
  • the image capture unit 231 stores the captured video in the mobile storage unit 21.
  • the image capture unit 231 stores video in the mobile storage unit 21, which has a limited storage capacity, it is preferable that the image capture unit 231 overwrites the area in which the old video is stored with the new video.
  • the structure of the mobile storage unit 21 is a ring buffer structure.
  • the attribute value acquisition unit 233 acquires one or more camera attribute values including directional information indicating the direction in which the image capture unit 232 captures images. It is preferable that the attribute value acquisition unit 233 acquires one or more video attribute values including the camera attribute value.
  • the attribute value acquisition unit 233 acquires, for example, one or more video attribute values.
  • One or more video attribute values may be referred to as an attribute value set.
  • the attribute value acquisition unit 233 acquires, for example, time information from a clock (not shown) during shooting.
  • the attribute value acquisition unit 233 acquires time information, for example, constantly, at predetermined intervals, or when acquisition conditions are met.
  • the attribute value acquisition unit 233 acquires time information from a clock (not shown) during shooting, and acquires seasonal information corresponding to the time information.
  • the attribute value acquisition unit 233 acquires weather information, for example, during shooting. Note that even if the shooting location is indoors, the weather information may be information about the weather in the same area as the indoor location.
  • the configuration unit 234 associates the acquired position information with the video and acquires the additional video.
  • the configuration unit 234, for example, associates the relative position information or absolute position information acquired using the relative position information with the video and acquires the additional video.
  • the configuration unit 234, for example, associates the absolute position information acquired by the absolute position acquisition means 2311 with the video and acquires the additional video.
  • the configuration unit 234 acquires an additional video in which one or more camera attribute values acquired by the attribute value acquisition unit 233 are also associated with the video.
  • the mobile transmission unit 24 transmits various types of information to the information processing device 1.
  • the various types of information are, for example, location information, video, and additional video.
  • the mobile transmission unit 24 may continuously transmit video captured by the image capture unit 232 to the information processing device 1. In such cases, the video is usually associated with location information.
  • the mobile transmission unit 24 stops transmitting the video when a stop instruction is received.
  • the position transmission unit 241 transmits the position information acquired by the position acquisition unit 231 to the information processing device 1.
  • the position information is, for example, relative position information or absolute position information acquired using the relative position information.
  • the location transmission unit 241 continuously transmits the location information to the information processing device 1. Continuously may include always or periodically. Periodically means that the transmission interval may vary slightly. There is no restriction on the frequency or interval at which the location transmission unit 241 transmits the location information.
  • the moving video sending unit 242 sends a video in response to the moving instruction receiving unit 221 receiving a transmission instruction. It is preferable for the moving video sending unit 242 to send an additional video.
  • the movement output unit 25 outputs various types of information.
  • the various types of information are, for example, videos and sets of attribute values.
  • Output is a concept that includes displaying on a monitor, projecting using a projector, printing on a printer, outputting sound, sending to an external device, storing on a recording medium, and passing on the results of processing to other processing devices or other programs.
  • the video output unit 251 outputs a video. It is preferable that the video output unit 251 outputs an additional video.
  • the output here may be storage in the moving storage unit 21.
  • the user storage unit 31 stores various types of information.
  • the various types of information include, for example, user identifiers, videos, and inquiries.
  • the user reception unit 32 receives various instructions and information.
  • the various instructions and information are, for example, inquiries and purchase instructions. Note that inquiries may include purchase instructions.
  • a purchase instruction is an instruction to purchase a video.
  • the purchase instruction is associated with a user identifier.
  • the purchase instruction usually includes information that identifies the video.
  • the purchase instruction has, for example, a video identifier.
  • the purchase instruction includes, for example, an inquiry.
  • the purchase instruction includes, for example, purchase conditions.
  • the purchase conditions are, for example, the purchase amount.
  • the purchase conditions include, for example, information that identifies the rights period.
  • the means for inputting various instructions and information can be anything, such as a touch panel, keyboard, mouse, or menu screen.
  • the user processing unit 33 performs various types of processing.
  • the various types of processing are processing for creating a data structure for transmitting the various instructions and information received by the user receiving unit 32.
  • the various types of processing are processing for creating a structure for transmitting the information received by the user receiving unit 35.
  • the user transmission unit 34 transmits various instructions and information to the information processing device 1.
  • Various instructions and information are, for example, inquiries and purchase instructions.
  • the user receiving unit 35 receives various information and instructions.
  • the various information and instructions are, for example, video and additional video. Note that video can be considered a concept that includes additional video.
  • the user output unit 36 outputs various types of information.
  • the various types of information include, for example, videos and additional videos.
  • output is a concept that includes displaying on a display, projecting using a projector, printing on a printer, outputting sound, sending to an external device, storing on a recording medium, and passing on the processing results to other processing devices or other programs, etc.
  • the storage unit 11, the terminal management unit 111, the mobile storage unit 21, and the user storage unit 31 are preferably non-volatile recording media, but can also be realized using volatile recording media.
  • information may be stored in the storage unit 11, etc. via a recording medium, information transmitted via a communication line, etc. may be stored in the storage unit 11, etc., or information inputted via an input device may be stored in the storage unit 11, etc.
  • the receiving unit 12, the location receiving unit 121, the inquiry receiving unit 122, the movement receiving unit 22, the movement instruction receiving unit 221, and the user receiving unit 35 are typically realized by wireless or wired communication means, but may also be realized by means of receiving broadcasts.
  • the processing unit 13, position accumulation unit 131, terminal determination unit 132, video acquisition unit 133, video composition unit 134, rights holder processing unit 135, first security means 1351, second security means 1352, third security means 1353, fourth security means 1354, reward means 1355, movement processing unit 23, position acquisition unit 231, attribute value acquisition unit 233, composition unit 234, and user processing unit 33 can usually be realized by a processor, memory, etc.
  • the processing procedure of the processing unit 13, etc. is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may also be realized by hardware (dedicated circuit).
  • the processor may be a CPU, MPU, GPU, etc., and the type does not matter.
  • Transmitting unit 14 video transmitting unit 141, mobile transmitting unit 24, location transmitting unit 241, mobile video transmitting unit 242, and user transmitting unit 34 are typically realized by wireless or wired communication means, but may also be realized by broadcasting means.
  • the absolute position acquisition means 2311 is realized, for example, by a GPS receiver.
  • the relative position acquisition means 2312 is realized, for example, by LiDAR.
  • the relative position acquisition means 2312 is realized, for example, by a communication means and software, etc.
  • the image capture unit 232 includes a camera capable of capturing video. Note that any type of camera can be used.
  • the movement output unit 25 and the video output unit 251 can be realized, for example, by an output device such as a display or a speaker, or by a combination of driver software for an output device and an output device.
  • the movement output unit 25 and the video output unit 251 can be realized, for example, by a processor or a memory.
  • the user reception unit 32 can be realized by a device driver for an input means such as a touch panel or keyboard, or control software for a menu screen, etc.
  • the user output unit 36 may or may not include an output device such as a display, a speaker, etc.
  • the user output unit 36 may be realized by driver software for an output device, or a combination of driver software for an output device and the output device, etc. ⁇ Operation>
  • the location receiving unit 121 determines whether or not location information has been received from the mobile terminal 2. If location information has been received, the process proceeds to S402. If location information has not been received, the process proceeds to S403. The received location information is associated with the terminal identifier.
  • the location accumulation unit 131 associates the location information received in S401 with the terminal identifier and accumulates it in the terminal management unit 111. Return to S401.
  • the position accumulation unit 131 When the position accumulation unit 131 receives relative position information, it may obtain absolute position information from the absolute position information of the reference point and the relative position information, and may associate the absolute position information with the terminal identifier and accumulate it in the terminal management unit 111.
  • the inquiry receiving unit 122 determines whether or not an inquiry has been received from the user terminal 3. If an inquiry has been received, the process proceeds to S404, and if an inquiry has not been received, the process returns to S401. Note that an inquiry is usually associated with the user identifier of the user who sent the inquiry.
  • the terminal determination unit 132 determines one or more mobile terminals 2 that satisfy the approximation condition for the location identification information included in the query.
  • An example of such a terminal determination process will be described using the flowchart in FIG. 5. Note that a mobile terminal 2 that satisfies the approximation condition is a mobile terminal 2 that corresponds to location information that satisfies the approximation condition.
  • the video acquisition unit 133 assigns 1 to counter i.
  • the video acquisition unit 133 determines whether the i-th mobile terminal 2 is present among the mobile terminals 2 determined in S404. If the i-th mobile terminal 2 is present, the process proceeds to S407; if not, the process proceeds to S409.
  • the video acquisition unit 133 acquires the terminal communication information of the i-th mobile terminal 2 from the terminal management unit 111.
  • the video acquisition unit 133 uses the terminal communication information to send a video to the i-th mobile terminal 2.
  • the receiving unit 12 judges whether or not video has been received from one or more mobile terminals 2. If video has been received, the process proceeds to S410, and if video has not been received, the process returns to S409. Note that it is preferable to wait until video has been received from all mobile terminals 2 that have sent transmission instructions.
  • the video composition unit 134 determines whether or not multiple videos were received in S409. If multiple videos were received, the process proceeds to S411; if one video was received, the process proceeds to S412.
  • the video composition unit 134 performs processing to spatially blend multiple videos.
  • An example of such video blending processing is described using the flowchart in FIG. 6.
  • the video composition unit 134 places the acquired video in a buffer for transmitting videos.
  • the video transmission unit 141 transmits the video in the buffer to the user terminal 3 that has sent the inquiry.
  • the terminal determination unit 132 obtains the latest location information of each of the one or more mobile terminals 2 determined in S404 from the terminal management unit 111.
  • the location receiving unit 121 normally receives location information continuously and at all times from one or more mobile terminals 2, and the location storage unit 131 stores the latest location information in the terminal management unit 111 in association with the terminal identifier.
  • the terminal determination unit 132 determines whether the location specification information included in the inquiry and the latest location information of one or more mobile terminals 2 satisfy the approximation condition. If the approximation condition is satisfied, the process proceeds to S409; if the approximation condition is not satisfied, the process proceeds to S415.
  • video is received from each mobile terminal 2 determined in S404 as long as the latest location information of that mobile terminal 2 satisfies the approximation condition. Then, video reception from a mobile terminal 2 whose latest location information no longer satisfies the approximation condition is terminated.
  • the transmission unit 14 transmits a stop instruction to the mobile terminal 2 whose latest location information no longer satisfies the approximation condition.
  • the rights holder processing unit 135 performs preservation processing.
  • An example of preservation processing is explained using the flowchart in FIG. 8.
  • the processing unit 13 determines whether or not to end the process of transmitting the video to the user terminal 3. If the process of transmitting the video is to be ended, the process returns to S401. If the process is not to be ended, the process returns to S404.
  • processing ends when the power is turned off or an interrupt occurs to end processing.
  • the terminal determination unit 132 acquires location identification information from the received inquiry.
  • the terminal determination unit 132 assigns 1 to counter i.
  • the terminal determination unit 132 determines whether the i-th terminal information exists in the terminal management unit 111. If the i-th terminal information exists, the process proceeds to S504; if not, the process proceeds to S509.
  • the terminal determination unit 132 obtains the latest location information contained in the i-th terminal information from the terminal management unit 111.
  • the terminal determination unit 132 determines whether the location specification information acquired in S501 and the location information acquired in S504 satisfy the approximation condition. If the approximation condition is satisfied, the process proceeds to S506, and if the approximation condition is not satisfied, the process proceeds to S508.
  • the terminal determination unit 132 acquires the terminal communication information contained in the i-th terminal information from the terminal management unit 111 and stores it in a buffer (not shown).
  • the terminal determination unit 132 calculates the distance between the position indicated by the location identification information acquired in S501 and the position indicated by the position information acquired in S504.
  • the terminal determination unit 132 sorts the mobile terminals 2 using the distance as a key. It then returns to the upper-level process. Note that sorting the mobile terminals 2 is equivalent to sorting the terminal communication information.
  • the terminal determination unit 132 may determine only a predetermined number of mobile terminals 2 or less.
  • the predetermined number is, for example, "1" or "2.”
  • the terminal determination unit 132 may determine only a predetermined number of mobile terminals 2, for example, in ascending order of the distance acquired in S507.
  • the video composition unit 134 acquires reference location information.
  • the reference location information is, for example, the location information indicated by the location identification information included in the received inquiry.
  • the reference location information is, for example, the location information of the mobile terminal 2 that is closest to the location indicated by the location identification information.
  • the video composition unit 134 assigns 1 to counter i.
  • the video composition unit 134 determines whether the i-th time (i-th frame) exists in the video. If the i-th time exists, the process proceeds to S604; if not, the process proceeds to S610.
  • the video composition unit 134 acquires a reference image.
  • the reference image is an image in the video transmitted by the mobile terminal 2 that is closest to the position indicated by the point identification information.
  • the first reference image is the first frame in the video transmitted by the mobile terminal 2.
  • the video composition unit 134 judges whether or not the jth other image exists. If the jth other image exists, the process proceeds to S607, and if not, the process proceeds to S609.
  • the other image is a frame in a video transmitted from another mobile terminal 2. Note that the reference image and the other image are usually synchronized.
  • the video composition unit 134 performs image fusion processing.
  • An example of the image fusion processing is explained using the flowchart in FIG. 7.
  • the video composition unit 134 temporally combines the multiple fusion images fused in S607 to compose a fusion video, which is temporarily stored in a buffer (not shown). The process returns to the upper level process.
  • the video composition unit 134 acquires a base image.
  • the first base image is the reference image acquired in S602.
  • the base image is also the base image updated in S607 in the previous loop from S606 to S608.
  • the video composition unit 134 acquires area information that identifies the area covered by the base image acquired in S701.
  • the area information is, for example, a collection of position information of vertices that make up a polygon, or a collection of position information of two diagonal corners that make up a rectangle.
  • the video composition unit 134 acquires the jth other image in S606.
  • the video composition unit 134 acquires area information that identifies the area covered by the other images acquired in S703.
  • the video composition unit 134 uses the area information of the base image and the area information of the other image to determine the area in which the other image is to be placed relative to the base image, places the other image in that area, and obtains the updated base image. It then returns to the upper-level process.
  • the rights holder processing unit 135 stores the transmitted video in association with a set of attribute values corresponding to one or more videos that were the basis for the transmitted video.
  • the rights holder processing unit 135 stores the video in a pair with a rights holder identifier that identifies one or more rights holders.
  • the rights holder identifier here is, for example, one or more rights holder identifiers of the videos that are the basis of the stored video.
  • the rights holder identifier here is, for example, a single rights holder identifier that identifies the user who made the inquiry.
  • the rights holder processing unit 135 also stores the video in, for example, the storage unit 11 or a device other than the information processing device 1.
  • the device other than the information processing device 1 may be a device that constitutes a blockchain.
  • the fourth maintenance means 1354 performs the fourth maintenance process.
  • An example of the fourth maintenance process was explained using the flowchart in FIG. 9.
  • the rights holder processing unit 135 determines whether the i-th video that was the basis for the stored video exists. If the i-th video exists, the process proceeds to S805; if not, the process returns to the upper level process.
  • the compensation means 1355 performs compensation processing.
  • An example of the compensation processing will be described with reference to the flowchart in FIG. 10.
  • the compensation processing here is a process of paying compensation to the rights holder of the i-th video that was the basis for the accumulated video.
  • the rights holder processing unit 135 determines whether or not to change the rights holder of the i-th video that was the basis of the stored video. If the rights holder is to be changed, the process proceeds to S807; if the rights holder is not to be changed, the process proceeds to S808.
  • the decision as to whether or not to change the rights holder may be based on a flag associated with the i-th video, may be predetermined, or the rights holder may be changed if the inquiry includes "information indicating a request to change the rights holder.”
  • the rights holder processing unit 135 acquires the user identifier of the user terminal 3. This user identifier becomes the new rights holder identifier.
  • the first preservation means 1351 stores the i-th video that was the source of the stored video.
  • the fourth preservation means 1354 performs a fourth preservation process on the i-th video that was the source of the stored videos. An example of the fourth preservation process was explained using the flowchart in FIG. 9.
  • the fourth security means 1354 acquires access information that identifies the storage destination of the video.
  • a video may be, for example, a fused video or a combined video.
  • the fourth preservation means 1354 acquires a set of attribute values corresponding to the stored video. If the stored video is a video composed of multiple original videos, the set of attribute values corresponding to the video is the set of attribute values for each of the multiple original videos.
  • the fourth security means 1354 constructs security information having the access information acquired in S901, the attribute value set acquired in S902, and the rights holder identifier of the video. Note that if a new rights holder identifier has been acquired, the fourth security means 1354 constructs security information including, for example, the new rights holder identifier and the original rights holder identifier. (S904) The fourth maintenance unit 1354 accumulates the maintenance information constructed in S903, and then returns to the upper level process.
  • preservation information for a video corresponding to the preservation information to be stored has been stored, this preservation information is overwritten onto the preservation information constructed in S903. This makes it possible to manage, for example, the transition of rights holders of the video.
  • the fourth preservation means 1354 stores the preservation information in, for example, a blockchain.
  • the compensation means 1355 acquires one or more rights holder identifiers for the target video.
  • the compensation means 1355 may also acquire the rights holder identifiers of the previous rights holders of the target video.
  • the reward means 1355 acquires a set of attribute values for the target video.
  • the reward means 1355 acquires a service identifier that identifies the service provided for the target video.
  • the service identifier is, for example, "view” or "purchase.”
  • the reward means 1355 obtains the reward amount using one or more pieces of information from the attribute value set obtained in S1002 and the service identifier obtained in S1003.
  • the compensation means 1355 acquires a compensation amount for each rights holder identifier. If rights holder history information including multiple rights holder identifiers has been acquired, the compensation means 1355 may acquire a compensation amount for each rights holder identifier.
  • the compensation means 1355 acquires video attribute values corresponding to each of the multiple videos that were the basis for the video transmitted by the video transmission unit 141, and uses the video attribute values to determine the amount of compensation for each of the multiple rights holders. It is preferable that the compensation means 1355, for example, determines a larger amount of compensation the greater the amount of data, video duration, or number of frames of the original video used in the video transmitted by the video transmission unit 141. It is preferable that the compensation means 1355, for example, determines a larger amount of compensation the greater the resolution of the original video used in the video transmitted by the video transmission unit 141.
  • the reward means 1355 performs processing to give a reward to the rights holder identified by the rights holder identifier obtained in S1001, in the amount of reward obtained in S1004.
  • the remuneration means 1355 performs a process of charging the user who has enjoyed the service related to the target video with a remuneration. The process returns to the upper level process.
  • the target video is usually a video transmitted to the user terminal 3.
  • the profits earned by the operator of the information processing device 1 may be acquired and accumulated.
  • the mobile terminal 2 determines whether or not a start instruction has been received. If a start instruction has been received, the process proceeds to S1102, and if not, the process proceeds to S1111.
  • the start instruction may be, for example, an instruction to launch a video transmission application or to power on the mobile terminal 2.
  • the location acquisition unit 231 performs location acquisition processing.
  • An example of the location acquisition processing will be described with reference to the flowchart in FIG. 12. Note that the location acquisition processing is processing for acquiring current location information of the mobile terminal 2.
  • the attribute value acquisition unit 233 acquires one or more video attribute values excluding location information.
  • the configuration unit 234 obtains the terminal identifier from the movement storage unit 21.
  • the configuration unit 234 configures the information to be transmitted, which is information having location information, one or more video attribute values, and a terminal identifier.
  • the location transmission unit 241 transmits the information configured in S1105 to the information processing device 1.
  • the image capture unit 232 captures and acquires video.
  • the composition unit 234 composes an additional video having a video, location information, one or more video attribute values, and a terminal identifier.
  • the composition unit 234 stores the additional video in the moving storage unit 21.
  • the moving video transmission unit 242 determines whether or not to transmit the video to the information processing device 1. If a video is to be transmitted, the process proceeds to S1109. If a video is not to be transmitted, the process proceeds to S1110. Note that when a video is to be transmitted, for example, the transmission flag may be "ON”. Also, the initial value of the transmission flag is "OFF".
  • the moving video transmission unit 242 transmits the additional images accumulated in S1107 to the information processing device 1. Note that there is no limit to the amount of video transmitted at one time.
  • the movement processing unit 23 judges whether or not to end the transmission of the position information, etc. If the transmission is to be ended, the process returns to S1101, and if the transmission is not to be ended, the process returns to S1102. Note that the transmission of the position information, etc. is ended, for example, when an end instruction is received from the user.
  • the movement instruction receiving unit 221 determines whether or not a video transmission instruction has been received from the information processing device 1. If a transmission instruction has been received, the process proceeds to S1102; if not, the process returns to S1101.
  • the mobile receiving unit 22 determines whether or not a stop instruction has been received from the information processing device 1. If a stop instruction has been received, the process proceeds to S1114; if not, the process returns to S1101.
  • processing ends when the power is turned off or an interrupt occurs to end processing.
  • the absolute position acquisition means 2311 acquires absolute position information and attempts to store the absolute position information in a buffer (not shown).
  • the position acquisition unit 231 judges whether or not absolute position information was acquired in S1201. If absolute position information was acquired, the process returns to the upper level process, and if it was not acquired, the process proceeds to S1203. Note that if absolute position information was acquired in S1201 and the relative position acquisition means 2312 has been started, it is preferable for the movement processing unit 23 to stop the processing of the relative position acquisition means 2312.
  • the position acquisition unit 231 starts the relative position acquisition means 2312. Note that if the relative position acquisition means 2312 has already been started, the position acquisition unit 231 does nothing here.
  • the relative position acquisition means 2312 acquires the relative position information and stores the relative position information in a buffer (not shown).
  • the position acquisition unit 231 uses the relative position information acquired in S1204 to determine whether or not to acquire absolute position information. If absolute position information is to be acquired, the process proceeds to S1206, and if absolute position information is not to be acquired, the process returns to the upper level process. Note that absolute position information is acquired, for example, when absolute position information that serves as the basis for the relative position information exists in the movement storage unit 21 or a buffer not shown.
  • the position acquisition unit 231 acquires absolute position information that serves as a reference, and uses the absolute position information and the acquired relative position information to calculate absolute position information that corresponds to the relative position information, and accumulates the absolute position information in a buffer (not shown). The process returns to the upper level process.
  • the user reception unit 32 determines whether or not a query has been received. If a query has been received, the process proceeds to S1302, and if not, the process proceeds to S1307.
  • the reception of a query is, for example, the reception of a user input.
  • the reception of a query is, for example, the input of a user instruction for a specific location on the indoor map output to the mobile terminal 2.
  • the user processing unit 33 composes a query to be sent.
  • the user sending unit 34 associates the query with the user identifier and sends it to the information processing device 1.
  • the user receiving unit 35 determines whether or not information has been received from the information processing device 1. If information has been received, the process proceeds to S1304; if information has not been received, the process returns to S1303.
  • the user processing unit 33 determines whether the information received in S1303 includes a video. If it does include a video, the process proceeds to S1305; if it does not include a video, the process proceeds to S1306.
  • the user reception unit 32 determines whether or not a purchase instruction has been received from the user. If a purchase instruction has been received, the process proceeds to S1308, and if not, the process returns to S1301.
  • the purchase instruction may include, for example, an inquiry.
  • the user processing unit 33 composes a purchase instruction to be sent.
  • the user sending unit 34 associates the purchase instruction with the user identifier and sends it to the information processing device 1.
  • the user receiving unit 35 determines whether or not information has been received from the information processing device 1. If information has been received, the process proceeds to S1310, and if information has not been received, the process returns to S1309.
  • the information may be, for example, a video, information indicating that the transfer of rights holders has been completed, and a set of attribute values for the video.
  • the user processing unit 33 uses the received information to construct information to be output.
  • the user output unit 36 outputs the information. Return to S1301.
  • the terminal management table is a table for managing terminal information.
  • the terminal management table manages one or more records having an "ID”, “terminal identifier", “location information”, and “terminal communication information”.
  • the "location information” has an "area identifier” and "location details”.
  • the "area identifier” is an identifier of the area corresponding to the relative location information. If the area identifier is "-”, the “location details” is absolute location information. If the area identifier is information other than "-”, the “location details” is relative location information.
  • the "terminal communication information” is information used when communicating with the mobile terminal 2.
  • a large number of mobile terminals 2 that are taking videos perform the following operations and continuously transmit the latest location information to the information processing device 1.
  • the GPS receiver in the absolute position acquisition means 2311 of the mobile terminal 2 acquires absolute position information and attempts to store the absolute position information in a buffer (not shown).
  • the position acquisition unit 231 judges whether or not the absolute position information has been acquired. If the absolute position information has not been acquired, the relative position acquisition means 2312 is activated. Then, the communication module in the relative position acquisition means 2312 receives an area identifier from a signal transmitter present in that area. Also, the LiDAR in the relative position acquisition means 2312 acquires relative position information in that area. Next, the position acquisition unit 231 composes position information having an area identifier and relative position information. Note that if the absolute position acquisition means 2311 of the mobile terminal 2 has acquired the absolute position information of the mobile terminal 2, the relative position acquisition means 2312 is not activated. The position acquisition unit 231 repeats the acquisition of such position information.
  • the location transmission unit 241 of the mobile terminal 2 continuously transmits the latest location information to the information processing device 1 in association with the terminal identifier.
  • the location receiving unit 121 of the information processing device 1 continuously receives the latest location information associated with the terminal identifiers from the multiple mobile terminals 2.
  • the location storage unit 131 stores the location information in the terminal management table in association with the terminal identifiers.
  • the user reception unit 32 of the user terminal 3 receives the query " ⁇ area identifier> underground mall A relative position information (x a , y a )".
  • the user processing unit 33 composes the query to be transmitted.
  • the user transmission unit 34 transmits the query to the information processing device 1 in association with the user identifier.
  • the inquiry receiving unit 122 of the information processing device 1 receives an inquiry from the user terminal 3.
  • the terminal determination unit 132 refers to the terminal management table (FIG. 14) and determines a mobile terminal 2 that satisfies the approximation condition for the point identification information " ⁇ area identifier> underground mall A relative position information (x a , y a )" included in the inquiry. That is, the terminal determination unit 132 determines that the area identifier "underground mall A" matches and the position details (x 1 , y 1 ) whose distance from the relative position information (x a , y a ) is within a threshold value satisfies the approximation condition.
  • the terminal determination unit 132 acquires the terminal communication information "communication destination 1" from the terminal management table (FIG. 14). It is assumed that the terminal determination unit 132 determines that the area identifier "underground mall A" matches but the position details (x 2 , y 2 ) whose distance from the relative position information (x a , y a ) is greater than a threshold value does not satisfy the approximation condition. That is, the terminal determination unit 132 does not acquire the terminal communication information "notification destination 2" from the terminal management table (FIG. 14).
  • the video acquisition unit 133 sends a video transmission instruction to the mobile terminal 2 with the terminal identifier "T001" that can communicate using the terminal communication information "communication destination 1".
  • the movement instruction receiving unit 221 of the mobile terminal 2 receives an instruction to send a video from the information processing device 1.
  • the position acquiring unit 231 performs a position acquisition process.
  • the attribute value acquiring unit 233 acquires one or more video attribute values (e.g., time, temperature, etc.).
  • the configuration unit 234 acquires a terminal identifier from the movement storage unit 21.
  • the configuration unit 234 configures information to be sent, which is information having location information, one or more video attribute values, and a terminal identifier.
  • the position transmitting unit 241 transmits the configured information to the information processing device 1.
  • the configuration unit 234 configures an additional video having a video, location information, one or more video attribute values, and a terminal identifier.
  • the configuration unit 234 accumulates the additional video in the movement storage unit 21.
  • the movement video transmitting unit 242 transmits the additional image to the information processing device 1. This transmission of the video continues for a predetermined time or more.
  • the receiving unit 12 of the information processing device 1 receives the additional video from the mobile terminal 2.
  • the video composition unit 134 places the acquired video in a buffer for transmitting videos.
  • the video transmission unit 141 transmits the video in the buffer to the user terminal 3 that has sent the inquiry. The above transmission of videos continues for a predetermined period of time or more.
  • the user terminal 3 receives the video from the information processing device 1 and outputs it. This output of the video continues for a predetermined time or more. This allows the user to watch a video of a predetermined location in underground mall A (e.g., a store that the user wants to visit). By watching this video, the user can find out, for example, whether or not there is a queue in front of the store.
  • underground mall A e.g., a store that the user wants to visit.
  • the terminal determination unit 132 determines that the location information of the mobile terminal 2 no longer satisfies the approximation condition for the location identification information included in the inquiry from the user.
  • the rights holder processing unit 135 performs the preservation process described using the flowchart in FIG. 8. As a result, the user of the user terminal 3 that sent the video can receive a reward. In addition, the video is properly preserved.
  • the terminal determination unit 132 of the information processing device 1 determines that the area identifier "underground mall A" matches the latest location information paired with the terminal identifier "T002", and the location details whose distance from the relative location information (x a , y a ) is within the threshold value satisfy the approximation condition.
  • the terminal determination unit 132 acquires the terminal communication information "communication destination 2" from the terminal management table ( FIG. 14 ).
  • the video acquisition unit 133 sends a video transmission instruction to the mobile terminal 2 with the terminal identifier "T002" that can communicate using the terminal communication information "communication destination 2".
  • the movement instruction receiving unit 221 of the mobile terminal 2 receives an instruction to send a video from the information processing device 1.
  • the position acquiring unit 231 performs a position acquisition process.
  • the attribute value acquiring unit 233 acquires one or more video attribute values (e.g., time, temperature, etc.).
  • the configuration unit 234 acquires a terminal identifier from the movement storage unit 21.
  • the configuration unit 234 configures information to be sent, which is information having location information, one or more video attribute values, and a terminal identifier.
  • the position transmitting unit 241 transmits the configured information to the information processing device 1.
  • the configuration unit 234 configures an additional video having a video, location information, one or more video attribute values, and a terminal identifier.
  • the configuration unit 234 accumulates the additional video in the movement storage unit 21.
  • the movement video transmitting unit 242 transmits the additional image to the information processing device 1. This transmission of the video continues for a predetermined time or more.
  • the receiving unit 12 of the information processing device 1 receives the additional video from the mobile terminal 2.
  • the video composition unit 134 places the acquired video in a buffer for transmitting videos.
  • the video transmission unit 141 transmits the video in the buffer to the user terminal 3 that has sent the inquiry. The above transmission of videos continues for a predetermined period of time or more.
  • the user terminal 3 then receives and outputs the video from the information processing device 1. This output of the video continues for a predetermined period of time or more. This allows the user to sequentially view the videos transmitted from multiple mobile terminals 2. By viewing the videos, the user can, for example, know the situation in front of a store for a long period of time.
  • this embodiment can provide a mobile terminal that can capture video that can be used appropriately while moving around indoors.
  • this embodiment can provide a platform that allows appropriate use of videos captured by mobile terminals moving around indoors.
  • each device in this embodiment may be realized by software.
  • This software may be distributed by software download or the like.
  • This software may also be recorded on a recording medium such as a CD-ROM and distributed. This also applies to the other embodiments in this specification.
  • the information processing device 1 determines the mobile terminal 2 to which a video matching the user's inquiry can be transmitted. In the present embodiment, the mobile terminal determines whether or not to transmit a video in response to the user's inquiry.
  • FIG. 15 is a block diagram of information system B in this embodiment.
  • FIG. 16 is a block diagram of information processing device 4.
  • Information system B includes an information processing device 4, one or more mobile terminals 5, and one or more user terminals 3.
  • the information processing device 4 is a server that provides the video transmitted by one or more mobile terminals 5 to the user terminal 3.
  • the information processing device 4 is, for example, a so-called server, such as a cloud server or an ASP server, but the type is not important.
  • the information processing device 4 may also be a device that constitutes a blockchain.
  • the mobile terminal 5 is a terminal installed on a moving object and is a terminal that shoots video.
  • the mobile terminal 5 is, for example, a smartphone, a tablet terminal, a camera with a communication function, glasses with a camera, or a smart watch with a camera.
  • the information processing device 4 and one or more mobile terminals 5, and the information processing device 4 and one or more user terminals 3 are typically capable of communicating via a network such as the Internet.
  • the information processing device 4 includes a storage unit 41, a receiving unit 42, a processing unit 43, and a transmitting unit 14.
  • the storage unit 41 includes a terminal management unit 411.
  • the receiving unit 42 includes an inquiry receiving unit 122.
  • the processing unit 43 includes a video acquisition unit 433, a video composition unit 134, and a rights holder processing unit 135.
  • the mobile terminal 5 includes a movement storage unit 21, a movement receiving unit 52, a movement processing unit 53, a movement transmitting unit 54, and a movement output unit 25.
  • the movement processing unit 53 includes a position acquisition unit 231, a shooting unit 232, an attribute value acquisition unit 233, a configuration unit 234, and a movement determination unit 531.
  • the movement transmitting unit 54 includes a movement video transmission unit 542.
  • the various types of information include, for example, terminal information (described later), videos, and video attribute values associated with the videos.
  • the terminal management unit 411 stores one or more pieces of terminal information.
  • the terminal information corresponds to the mobile terminal 5.
  • the terminal information is information related to the mobile terminal 5.
  • the terminal information here typically includes a terminal identifier, which is an identifier of the mobile terminal 2, and terminal communication information.
  • the receiving unit 42 receives various information and instructions from the mobile terminal 2 or the user terminal 3.
  • the various information and instructions are, for example, inquiries, videos, and additional videos.
  • the processing unit 43 performs various types of processing. For example, the various types of processing are performed by the video acquisition unit 433, the video composition unit 134, and the rights holder processing unit 135.
  • the video acquisition unit 433 acquires videos that match the query and are sent from the mobile terminal 5 that is activated indoors.
  • the video acquisition unit 433 normally receives video from the mobile terminal 5 in response to the transmission of an inquiry. For example, the video acquisition unit 433 receives video captured by each of the multiple mobile terminals 5 from the mobile terminal 5.
  • the mobile terminal 5 that transmits the video is the mobile terminal 5 that corresponds to the location information that matches the inquiry.
  • the movement receiving unit 52 receives various types of information.
  • the various types of information are, for example, inquiries.
  • the movement instruction receiving unit 221 receives inquiries from the information processing device 1.
  • the movement processing unit 53 performs various types of processing.
  • the various types of processing are, for example, processing performed by the position acquisition unit 231, the photography unit 232, the attribute value acquisition unit 233, the configuration unit 234, and the movement determination unit 531.
  • the movement determination unit 531 determines whether or not the location information acquired by the location acquisition unit 231 matches the inquiry received by the movement receiving unit 52. More specifically, the movement determination unit 531 determines whether or not the location information acquired by the location acquisition unit 231 satisfies an approximation condition for the point-specific information contained in the inquiry received by the movement receiving unit 52. Note that the location information acquired by the location acquisition unit 231 is, for example, relative location information or absolute location information acquired using the relative location information.
  • the mobile transmission unit 54 transmits various types of information to the information processing device 1.
  • the various types of information are, for example, attribute value sets, videos, and additional videos.
  • the moving video sending unit 542 sends a video when the moving determination unit 531 determines that the video matches the query. Note that the video sent here may be an additional video.
  • the storage unit 41 and the terminal management unit 411 are preferably non-volatile recording media, but can also be realized using volatile recording media.
  • the process by which information is stored in the storage unit 41, etc. is not important.
  • information may be stored in the storage unit 41, etc. via a recording medium, information transmitted via a communication line, etc. may be stored in the storage unit 41, etc., or information inputted via an input device may be stored in the storage unit 41, etc.
  • the receiving unit 42 and the mobile receiving unit 52 are typically implemented using wireless or wired communication means, but may also be implemented using means for receiving broadcasts.
  • the processing unit 43, movement processing unit 53, and movement determination unit 531 can usually be realized by a processor, memory, etc.
  • the processing procedures of the processing unit 43, etc. are usually realized by software, and the software is recorded in a recording medium such as a ROM. However, they may also be realized by hardware (dedicated circuitry).
  • the processor may be a CPU, MPU, GPU, etc., and the type is not important.
  • the video acquisition unit 433 is usually implemented using a wireless or wired communication means, but may also be implemented using a means for receiving broadcasts.
  • the mobile transmitting unit 54 and the mobile video transmitting unit 542 are usually realized by wireless or wired communication means, but may also be realized by broadcasting means. ⁇ Operation>
  • the inquiry receiving unit 122 determines whether or not an inquiry has been received from the user terminal 3. If an inquiry has been received, the process proceeds to S1702; if an inquiry has not been received, the process returns to S1701.
  • the video acquisition unit 433 assigns 1 to counter i.
  • the video acquisition unit 433 determines whether or not the i-th terminal information exists in the terminal management unit 411. If the i-th terminal information exists, the process proceeds to S1704, and if not, the process proceeds to S1706.
  • the existence of terminal information means that there is a mobile terminal 5 that can potentially transmit video.
  • the video acquisition unit 433 sends a query to the mobile terminal 5 corresponding to the i-th terminal information.
  • the receiving unit 42 determines whether or not additional video has been received from one or more mobile terminals 5. If additional video has been received, the process proceeds to S1707; if additional video has not been received, the process proceeds to S1708.
  • the processing unit 43 accumulates the one or more received additional videos in a buffer (not shown). Return to S1706.
  • the processing unit 43 judges whether or not a timeout has occurred. If a timeout has occurred, the process proceeds to S1709, and if not, the process returns to S1706. Note that a timeout occurs when a time equal to or greater than a threshold has elapsed since the transmission of the inquiry. The threshold time is the time to wait for reception of video.
  • the receiving unit 42 determines whether additional video has been received from one or more mobile terminals 5. If additional video has been received within the time until the timeout, the process proceeds to S410; if not, the process proceeds to S1710.
  • the transmission unit 14 transmits an error message to the user terminal 3.
  • the error message is information to the effect that the video corresponding to the inquiry could not be obtained.
  • processing ends when the power is turned off or an interrupt occurs to end processing.
  • the mobile receiving unit 52 determines whether or not an inquiry has been received from the information processing device 4. If an inquiry has been received, the process proceeds to S1102; if an inquiry has not been received, the process returns to S1101.
  • the location acquisition unit 231 acquires location information.
  • An example of such location acquisition processing was explained using the flowchart in FIG. 12.
  • the movement determination unit 531 obtains the location-specific information included in the inquiry.
  • the movement determination unit 531 determines whether the location information acquired in S1102 and the point identification information acquired in S1803 satisfy the approximation condition. If the approximation condition is satisfied, the process proceeds to S1102, and if the approximation condition is not satisfied, the process proceeds to S1101.
  • the movement processing unit 53 determines whether or not to end the process. If the process is to be ended, the process returns to S1101. If the process is not to be ended, the process returns to S1102.
  • the terminal management unit 411 of the information processing device 4 stores a terminal management table having the attributes "ID,” “terminal identifier,” and “terminal communication information" from the terminal management table shown in FIG. 14.
  • the user reception unit 32 of the user terminal 3 receives the query " ⁇ area identifier> underground mall A relative position information (x a , y a )".
  • the user processing unit 33 composes a query to be transmitted.
  • the user transmission unit 34 transmits the query to the information processing device 1 in association with the user identifier.
  • the inquiry receiving unit 122 of the information processing device 1 receives an inquiry from the user terminal 3.
  • the video acquisition unit 433 transmits the inquiry to a large number of mobile terminals 5 in the terminal management table (FIG. 14).
  • the mobile receiving unit 52 of each mobile terminal 5 receives the inquiry from the information processing device 4.
  • the position acquisition unit 231 of each mobile terminal 5 acquires position information by the process described using the flowchart in FIG. 12.
  • the movement determination unit 531 of each mobile terminal 5 acquires the location identification information included in the inquiry.
  • the movement determination unit 531 also determines whether the acquired location information and the acquired location identification information satisfy an approximate condition.
  • the position acquisition unit 231 of the mobile terminal 5 that satisfies the approximation conditions performs a position acquisition process.
  • the attribute value acquisition unit 233 of the mobile terminal 5 acquires one or more video attribute values (e.g., time, temperature, etc.).
  • the configuration unit 234 of the mobile terminal 5 acquires a terminal identifier from the mobile storage unit 21.
  • the configuration unit 234 of the mobile terminal 5 configures information to be transmitted, which is information having position information, one or more video attribute values, and a terminal identifier.
  • the configuration unit 234 configures an additional video having a video, position information, one or more video attribute values, and a terminal identifier.
  • the configuration unit 234 accumulates the additional video in the mobile storage unit 21.
  • the mobile video transmission unit 542 of the mobile terminal 5 transmits the additional image to the information processing device 1. Such transmission of the video continues for a predetermined time or more.
  • the receiving unit 12 of the information processing device 1 receives the additional video from the mobile terminal 2.
  • the video composition unit 134 places the acquired video in a buffer for transmitting videos.
  • the video transmission unit 141 transmits the video in the buffer to the user terminal 3 that has sent the inquiry. The above transmission of videos continues for a predetermined period of time or more.
  • the user terminal 3 receives the video from the information processing device 1 and outputs it. This output of the video continues for at least a predetermined time. This allows the user to watch a video of a predetermined location in underground mall A (e.g., a store that the user wants to visit). By watching this video, the user can find out, for example, whether or not there is a queue in front of the store.
  • underground mall A e.g., a store that the user wants to visit.
  • the movement determination unit 531 determines that the location information of the mobile terminal 5 no longer satisfies the approximation condition for the point identification information included in the query. Next, the mobile terminal 5 stops pausing the video.
  • the position acquisition unit 231 of the mobile terminal 5 performs position acquisition processing. Furthermore, the attribute value acquisition unit 233 of the mobile terminal 5 acquires one or more video attribute values (e.g., time, temperature, etc.). Furthermore, the configuration unit 234 of the mobile terminal 5 acquires a terminal identifier from the movement storage unit 21. The configuration unit 234 of the mobile terminal 5 configures information to be transmitted, which is information having position information, one or more video attribute values, and a terminal identifier. The configuration unit 234 configures an additional video having a video, position information, one or more video attribute values, and a terminal identifier. The configuration unit 234 accumulates the additional video in the movement storage unit 21. Furthermore, the movement video transmission unit 542 of the mobile terminal 5 transmits the additional image to the information processing device 1. Such transmission of the video continues for a predetermined time or more.
  • the attribute value acquisition unit 233 of the mobile terminal 5 acquires one or more video attribute values (e.g., time, temperature, etc.). Furthermore, the configuration unit 234 of the mobile terminal
  • the receiving unit 12 of the information processing device 1 receives the additional video from the mobile terminal 2.
  • the video composition unit 134 places the acquired video in a buffer for transmitting videos.
  • the video transmission unit 141 transmits the video in the buffer to the user terminal 3 that has sent the inquiry. The above transmission of videos continues for a predetermined period of time or more.
  • the user terminal 3 receives the video from the information processing device 1 and outputs it. The output of the video continues for a predetermined time or more. This allows the user to watch the video of a predetermined point in the underground mall A for a long period of time.
  • Specific Example 2 It is assumed that a user other than the user in the specific example 1 inputs a query “ ⁇ area identifier> Building B ⁇ location name> Store S” into his/her user terminal 3 .
  • the user reception unit 32 of the user terminal 3 receives the inquiry " ⁇ area identifier> Building B ⁇ location name> Store S”.
  • the user processing unit 33 composes an inquiry to be transmitted.
  • the user transmission unit 34 transmits the inquiry to the information processing device 1 in association with the user identifier.
  • the inquiry receiving unit 122 of the information processing device 1 receives an inquiry from the user terminal 3.
  • the terminal determination unit 132 refers to a location name database (not shown) and acquires relative position information ( xS , yS ) paired with " ⁇ area identifier> Building B ⁇ location name> Store S.”
  • the terminal determination unit 132 refers to the terminal management table (FIG. 14) and determines a mobile terminal 2 that satisfies the approximation condition for the point identification information " ⁇ area identifier> Building B relative position information ( xS , yS )" indicated by the inquiry.
  • the terminal determination unit 132 determines that the detailed position ( x4 , y4 ) whose area identifier "Building B" matches and whose distance from the relative position information ( xS , yS ) is within the threshold satisfies the approximation condition.
  • the terminal determination unit 132 acquires the terminal communication information "Destination 4" from the terminal management table (FIG. 14). It is assumed that the terminal determination unit 132 determines that the detailed position ( x3 , y3 ) whose area identifier "Building B" matches but whose distance from the relative position information ( xS , yS ) is greater than the threshold does not satisfy the approximation condition.
  • the terminal determination unit 132 does not acquire the terminal communication information "Destination 3" from the terminal management table (FIG. 14).
  • the video acquisition unit 133 transmits a video transmission instruction to the mobile terminal 2 with the terminal identifier "T004" that can communicate using the terminal communication information "communication destination 4".
  • the movement instruction receiving unit 221 of the mobile terminal 2 receives a video transmission instruction from the information processing device 1.
  • the position acquiring unit 231 performs a position acquisition process.
  • the attribute value acquiring unit 233 acquires one or more video attribute values (for example, time, temperature, etc.).
  • the configuration unit 234 acquires a terminal identifier from the movement storage unit 21.
  • the configuration unit 234 configures information to be transmitted, which is information having position information, one or more video attribute values, and a terminal identifier.
  • the position transmitting unit 241 transmits the configured information to the information processing device 1.
  • the configuration unit 234 configures an additional video having a video, position information, one or more video attribute values, and a terminal identifier.
  • the configuration unit 234 accumulates the additional video in the movement storage unit 21.
  • the movement video transmitting unit 242 transmits the additional image to the information processing device 1. Such transmission of the video continues for a predetermined time or more.
  • the receiving unit 12 of the information processing device 1 receives the additional video from the mobile terminal 2.
  • the video composing unit 134 places the acquired video in a buffer for transmitting videos.
  • the video transmitting unit 141 transmits the video in the buffer to the user terminal 3 that has transmitted the inquiry.
  • the above transmission of the video continues for a predetermined time or more.
  • the user terminal 3 receives the video from the information processing device 1 and outputs it. This output of the video continues for a predetermined time or more. This allows the user to see a video of the surroundings of store S in building B. By watching this video, the user can know, for example, whether or not there is a queue in front of store S.
  • this embodiment can provide a mobile terminal that can capture video that can be used appropriately while moving around indoors.
  • this embodiment can provide a platform that allows appropriate use of videos captured by mobile terminals moving around indoors.
  • FIG. 19 also shows the appearance of a computer that executes the programs described in this specification to realize the information processing device 1, mobile terminal 2, user terminal 3, information processing device 4, and mobile terminal 5 of the various embodiments described above.
  • the above-mentioned embodiments can be realized by computer hardware and a computer program executed thereon.
  • FIG. 19 is an overview of this computer system 300
  • FIG. 20 is a block diagram of system 300.
  • computer system 300 includes computer 301, which includes a CD-ROM drive, keyboard 302, mouse 303, and monitor 304.
  • computer 301 in addition to CD-ROM drive 3012, computer 301 includes MPU 3013, bus 3014 connected to CD-ROM drive 3012 etc., ROM 3015 for storing programs such as a boot-up program, RAM 3016 connected to MPU 3013 for temporarily storing application program instructions and providing temporary storage space, and hard disk 3017 for storing application programs, system programs, and data.
  • computer 301 may further include a network card that provides connection to a LAN.
  • a program that causes computer system 300 to execute the functions of information processing device 1 and the like of the above-mentioned embodiment may be stored on CD-ROM 3101, inserted into CD-ROM drive 3012, and then transferred to hard disk 3017.
  • the program may be sent to computer 301 via a network (not shown) and stored on hard disk 3017.
  • the program is loaded into RAM 3016 when executed.
  • the program may be loaded directly from CD-ROM 3101 or the network.
  • the program does not necessarily have to include an operating system (OS) that causes the computer 301 to execute the functions of the information processing device 1 of the above-mentioned embodiment, or a third-party program, etc.
  • OS operating system
  • the program only needs to include an instruction portion that calls appropriate functions (modules) in a controlled manner to obtain the desired results. How the computer system 300 operates is well known, and a detailed description will be omitted.
  • the steps of transmitting information and receiving information do not include processing performed by hardware, such as processing performed by a modem or interface card in the transmission step (processing that can only be performed by hardware).
  • the computer that executes the above program may be a single computer or multiple computers. In other words, it may perform centralized processing or distributed processing.
  • multiple communication means present in one device may be realized physically by one medium.
  • each process may be realized by centralized processing in a single device, or may be realized by distributed processing in multiple devices.
  • the mobile terminal of the present invention has the effect of being able to appropriately use videos captured by a mobile terminal moving around indoors, and is useful as a terminal for providing videos, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Mobile Radio Communication Systems (AREA)
  • Telephonic Communication Services (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
PCT/JP2022/047550 2022-12-23 2022-12-23 移動端末、情報処理装置、および情報処理方法 Ceased WO2024134861A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2024565530A JP7688473B2 (ja) 2022-12-23 2022-12-23 移動端末、情報処理装置、および情報処理方法
PCT/JP2022/047550 WO2024134861A1 (ja) 2022-12-23 2022-12-23 移動端末、情報処理装置、および情報処理方法
US19/242,974 US12591401B2 (en) 2022-12-23 2025-06-19 Mobile terminal, information processing device and information processing method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/047550 WO2024134861A1 (ja) 2022-12-23 2022-12-23 移動端末、情報処理装置、および情報処理方法

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US19/242,974 Continuation US12591401B2 (en) 2022-12-23 2025-06-19 Mobile terminal, information processing device and information processing method

Publications (1)

Publication Number Publication Date
WO2024134861A1 true WO2024134861A1 (ja) 2024-06-27

Family

ID=91587894

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/047550 Ceased WO2024134861A1 (ja) 2022-12-23 2022-12-23 移動端末、情報処理装置、および情報処理方法

Country Status (3)

Country Link
US (1) US12591401B2 (https=)
JP (1) JP7688473B2 (https=)
WO (1) WO2024134861A1 (https=)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005175554A (ja) * 2003-12-08 2005-06-30 Ikuo Kondo 映像出力装置およびそのプログラム
JP2007067487A (ja) * 2005-08-29 2007-03-15 Hitachi Ltd 情報処理システム、および位置情報提示方法
JP2014127987A (ja) * 2012-12-27 2014-07-07 Sony Corp 情報処理装置および記録媒体
EP2827259A1 (en) * 2013-07-16 2015-01-21 Alcatel Lucent Method for providing image data for a location
JP2016134859A (ja) * 2015-01-21 2016-07-25 株式会社ナイルワークス 動画配信システム
KR102115098B1 (ko) * 2018-11-30 2020-05-25 서울과학기술대학교 산학협력단 실내 공간용 파노라마 영상 생성 시스템 및 방법
CN112291596A (zh) * 2019-07-25 2021-01-29 北京小米移动软件有限公司 图像数据的展示方法及装置、电子设备、存储介质

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010132677A1 (en) * 2009-05-13 2010-11-18 Rutgers, The State University Vehicular information systems and methods
JP7272244B2 (ja) 2019-11-22 2023-05-12 トヨタ自動車株式会社 画像データ配信システム

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005175554A (ja) * 2003-12-08 2005-06-30 Ikuo Kondo 映像出力装置およびそのプログラム
JP2007067487A (ja) * 2005-08-29 2007-03-15 Hitachi Ltd 情報処理システム、および位置情報提示方法
JP2014127987A (ja) * 2012-12-27 2014-07-07 Sony Corp 情報処理装置および記録媒体
EP2827259A1 (en) * 2013-07-16 2015-01-21 Alcatel Lucent Method for providing image data for a location
JP2016134859A (ja) * 2015-01-21 2016-07-25 株式会社ナイルワークス 動画配信システム
KR102115098B1 (ko) * 2018-11-30 2020-05-25 서울과학기술대학교 산학협력단 실내 공간용 파노라마 영상 생성 시스템 및 방법
CN112291596A (zh) * 2019-07-25 2021-01-29 北京小米移动软件有限公司 图像数据的展示方法及装置、电子设备、存储介质

Also Published As

Publication number Publication date
US12591401B2 (en) 2026-03-31
JP7688473B2 (ja) 2025-06-04
JPWO2024134861A1 (https=) 2024-06-27
US20250348262A1 (en) 2025-11-13

Similar Documents

Publication Publication Date Title
US20210056762A1 (en) Design and generation of augmented reality experiences for structured distribution of content based on location-based triggers
TWI617931B (zh) 適地性空間物件遠距管理方法與系統
US20130095855A1 (en) Method, System, and Computer Program Product for Obtaining Images to Enhance Imagery Coverage
US9804811B2 (en) System and method for printing location-based, customized data
KR101282292B1 (ko) 증강현실을 이용한 광고 서비스 시스템 및 그 방법
JP2017207469A (ja) ルート・ロギング・システムおよび方法、並びにモバイルクライアント装置
JP2018029290A (ja) 動画再生装置、動画再生システム、および動画再生方法
KR20150126289A (ko) 증강현실 기반 소셜 네트워크 서비스 정보를 제공하는 내비게이션 장치와 메타데이터 처리장치 및 그 방법
EP3388958B1 (en) Method and system for managing viewability of location-based spatial object
KR102197392B1 (ko) 관심지점 리스트에 대한 구독과 리뷰 정보에 기초하여 보상을 제공하는 방법 및 시스템
JP7373823B2 (ja) 情報処理装置、情報システム、情報処理方法、およびプログラム
CN110503450A (zh) 兴趣点媒体内容管理方法与系统
JP7688473B2 (ja) 移動端末、情報処理装置、および情報処理方法
CN109672798A (zh) 信息处理装置以及存储媒体
KR102105231B1 (ko) 정보 공유 시스템 및 방법
JP7667380B2 (ja) 情報処理装置、移動端末、ユーザ端末、および情報処理方法
JP7738782B2 (ja) 情報処理装置、ユーザ端末、および情報処理方法
JP7746610B2 (ja) 情報処理装置、ユーザ端末、および情報処理方法
JP7667921B2 (ja) 情報処理装置、および情報処理方法
KR20210109759A (ko) 차량용 컨텐츠 기반 경로 정보 제공 방법 및 그를 수행하기 위한 장치
JP7681812B2 (ja) 情報処理装置、ユーザ端末、および情報処理方法
WO2024224502A1 (ja) 情報処理装置、および情報処理方法
KR20210003516A (ko) 증강현실을이용한광고서비스시스템
JP7028488B1 (ja) 管理サーバ
WO2024161605A1 (ja) 情報処理装置、ユーザ端末、および情報処理方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22969244

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024565530

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22969244

Country of ref document: EP

Kind code of ref document: A1