WO2024084663A1 - 情報処理装置、および情報処理方法 - Google Patents

情報処理装置、および情報処理方法 Download PDF

Info

Publication number
WO2024084663A1
WO2024084663A1 PCT/JP2022/039141 JP2022039141W WO2024084663A1 WO 2024084663 A1 WO2024084663 A1 WO 2024084663A1 JP 2022039141 W JP2022039141 W JP 2022039141W WO 2024084663 A1 WO2024084663 A1 WO 2024084663A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
information
unit
fixed
terminal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/039141
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
健二 鳴島
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Micware Co Ltd
Original Assignee
Micware Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Micware Co Ltd filed Critical Micware Co Ltd
Priority to PCT/JP2022/039141 priority Critical patent/WO2024084663A1/ja
Priority to EP22962771.6A priority patent/EP4607911A1/en
Priority to JP2024551159A priority patent/JP7670937B2/ja
Publication of WO2024084663A1 publication Critical patent/WO2024084663A1/ja
Priority to US19/071,752 priority patent/US20250200716A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H04N5/765—Interface circuits between an apparatus for recording and another apparatus
    • H04N5/77—Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/79—Processing of colour television signals in connection with recording
    • H04N9/80—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/82—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only
    • H04N9/8205—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only involving the multiplexing of an additional signal and the colour video signal
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Definitions

  • the present invention relates to an information processing device that processes video captured by a mobile terminal and video captured by a fixed camera.
  • JP 2008-154192 A (paragraph [0140], FIG. 24, etc.)
  • the information processing device of the first invention is an information processing device comprising: a video acquisition unit that acquires mobile videos, which are videos captured by a mobile terminal and correspond to an attribute value set including one or more pieces of environmental information including location information specifying the shooting location or time information specifying the shooting time, and transmitted from a mobile terminal; and a fixed video, which is videos captured by a fixed camera and correspond to an attribute value set including one or more pieces of environmental information including location information or time information, and transmitted from a fixed terminal equipped with a fixed camera whose shooting location is fixed; a video composition unit that temporally combines the mobile videos and the fixed videos to compose a single video, or composes a single video having multiple frames by spatially fusing at least a portion of the frames of the mobile video and at least a portion of the frames of the fixed video; and a video transmission unit that transmits the single video composed by the video composition unit.
  • a video acquisition unit that acquires mobile videos, which are videos captured by a mobile terminal and correspond to an attribute value set including one or more pieces of
  • videos captured by a mobile device and videos captured by a fixed camera can be used to create a single useful video.
  • the information processing device of the second invention is an information processing device in which, compared to the first invention, the video acquisition unit is an information processing device that acquires a video corresponding to a priority type indicating a priority type of video among the fixed video and the moving video when there are fixed video and moving video corresponding to location information indicating positions close enough to each other to satisfy the first location condition, and the video acquisition unit is a multiple video satisfying an adoption condition that is a condition for the original videos that constitute one video.
  • the information processing device of the third invention is an information processing device according to the first or second invention, further comprising a receiving unit that receives moving videos from a mobile terminal and fixed videos from a fixed terminal equipped with a fixed camera, the video acquisition unit acquires the moving videos and fixed videos received by the receiving unit, and the receiving unit receives the moving videos from a mobile terminal that has accepted the user's permission according to a usage mode flag for the videos stored in the mobile terminal.
  • a useful video can be created by using moving video and fixed video based on the wishes of the mobile terminal user.
  • the information processing device of the fourth invention is an information processing device in which, compared to the third invention, the usage mode flag is information indicating whether or not there is a "non-provisional desire to use" the moving video sent from the mobile terminal, and the receiving unit receives the moving video from the mobile terminal only if the mobile terminal accepts the user's permission when the usage mode flag stored in the mobile terminal is information indicating "there is a non-provisional desire to use” the moving video.
  • a useful video can be created by using moving video and fixed video based on the wishes of the mobile terminal user.
  • the information processing device of the fifth invention is an information processing device according to any one of the first to fourth inventions, further comprising a rights holder processing unit that corresponds to a rights holder identifier that identifies the rights holder of the moving video or fixed video, and performs rights holder processing related to the rights holder identified by the rights holder identifier that corresponds to a video composed by the video composition unit.
  • This configuration allows for appropriate processing regarding video rights holders.
  • the rights holder identifier corresponding to one video is a rights holder identifier corresponding to each of the multiple videos that were the basis for the one video
  • the rights holder processing unit is an information processing device equipped with a reward means that performs reward processing to give rewards to each rights holder identified by the rights holder identifier corresponding to each of the multiple videos that were the basis for the one video.
  • This configuration makes it possible to provide compensation to the rights holder of videos captured by the mobile terminal.
  • the information processing device of the seventh invention is an information processing device that, compared to the sixth invention, has a remuneration means that acquires video attribute values corresponding to each of the multiple videos that are the basis of the single video transmitted by the video transmission unit, and performs a remuneration process that uses the video attribute values to determine remuneration for each of the multiple rights holders and grants the remuneration.
  • This configuration makes it possible to provide appropriate compensation to the rights holder of videos captured by the mobile terminal.
  • the information processing device of the eighth invention is an information processing device according to the fifth invention, in which the rights holder processing unit is provided with a first preservation means for performing a first preservation process in which a single video composed by the video composition unit is stored in association with a set of attribute values corresponding to each of the multiple videos that were the source of the single video.
  • This configuration makes it possible to preserve a video made up of multiple images.
  • the information processing device of the ninth invention is an information processing device according to the fifth invention, further comprising a second security means for performing a second security process in which the rights holder processing unit stores a single video composed by the video composition unit in association with a rights holder identifier corresponding to each of the multiple videos that were the basis of the single video.
  • This configuration ensures that the rights holder of a video made up of multiple images is the appropriate rights holder.
  • the information processing device of the tenth invention is an information processing device according to the fifth invention, further comprising an inquiry receiving unit that receives an inquiry from a user terminal regarding environmental information on the environment in which the mobile terminal captured the video, the video acquisition unit acquires multiple videos corresponding to the inquiry received by the inquiry receiving unit, and the rights holder processing unit has a third preservation means that stores one video composed by the video composition unit in association with a rights holder identifier that identifies the user of the user terminal.
  • the rights holder of a video from multiple images can be identified as the rights holder who needs that video.
  • the information processing device of the eleventh invention is an information processing device according to the eighth invention, in which the rights holder processing unit is provided with a fourth preservation means that performs a fourth preservation process that accumulates preservation information, including access information for accessing a video, in a blockchain.
  • This configuration makes it possible to preserve management information for videos that need to be preserved.
  • another information processing device of the present invention is an information processing device in which, compared to the first or second invention, the video acquisition unit sequentially acquires multiple videos that meet one or more of the following conditions: a location condition related to the location information associated with the video, and a time condition related to the time information associated with the video.
  • videos captured by mobile devices and videos captured by fixed cameras can be appropriately used to create a single useful video.
  • a useful video can be constructed and provided by using videos captured by a mobile terminal and videos captured by a fixed camera.
  • Block diagram of information system A Block diagram of the information processing device 1 Block diagram of the fixed terminal 3 A flowchart illustrating an example of the operation of the information processing device 1.
  • a flowchart for explaining an example of an attribute value set acquisition process A flowchart for explaining an example of the moving object attribute value tag acquisition process.
  • a flowchart illustrating an example of the moving image fusion process A flowchart illustrating an example of the image fusion process.
  • a flowchart illustrating an example of the registered video search process A flowchart for explaining a first example of the unregistered video search process.
  • FIG. 2 shows the mobile terminal management table.
  • a diagram showing the fixed terminal management table A diagram showing the registered video management table Overview of the computer system Block diagram of the computer system
  • an information processing device uses videos captured by one or more fixed cameras and videos acquired by cameras possessed by one or more mobile terminals to create a single video that is combined in time, or a single video that is fused in space.
  • queries for selecting multiple videos include, for example, a query using location information specified by a user terminal, a query using one or more pieces of location information received from a target terminal to be monitored, a query using a destination set in a navigation terminal, and a query using route information.
  • the conditions for adopting the original videos that create a single video include, for example, conditions related to one or more pieces of information among location information and time information.
  • a single video that is linked together in time is a single video that is made by connecting multiple original videos.
  • multiple original videos that are the basis for a single video that is linked together in time are shot, it is preferable that they are consecutive in time, but they may be separated in time.
  • the date on which one original video was shot may be, for example, August 20th
  • the date on which the other original video was shot may be, for example, October 21st.
  • a spatially fused video is a video that is constructed by constructing a frame using some or all of the frames that make up each of the multiple original videos, and then connecting the multiple frames that have been constructed in time. At least one of the frames that make up the single video contains some or all of the frames of each of the multiple original videos.
  • a type of video to be prioritized is determined between fixed video captured by a fixed camera and mobile video captured by a camera of a mobile terminal, and the information processing device will be described in which the prioritized type of video is used preferentially.
  • an information processing device that performs post-processing to convert a single video into a stereoscopic three-dimensional video and provides the stereoscopic three-dimensional video.
  • the single video that is the subject of post-processing here may be only a portion of the video that satisfies the processing conditions.
  • the usage method of a video differs according to a usage mode flag set by the rights holder of the video that is the source of a video.
  • the usage mode flag is, for example, a flag indicating the presence or absence of a "non-provisional request for permanent use.” For example, if the request is a "non-provisional request for permanent use," a process of inquiring of the rights holder is performed before transmitting the video.
  • an information processing device that performs rights holder processing, which is processing related to the rights holder of a video to be output, will be described.
  • the rights holder processing is, for example, a compensation processing described below, and various types of preservation processing described below.
  • an information processing device that handles videos to which one or more tags obtained by analyzing the videos or the like are added will be described. Note that the process of adding tags is performed, for example, by an information processing device, a mobile terminal, or a fixed terminal, but may also be performed by other devices.
  • an information processing device that performs processing for a user of a mobile terminal or a user who needs a video when a video desired by the user exists but cannot be transmitted. Note that a video cannot be transmitted when, for example, the power of the mobile terminal is OFF.
  • information X being associated with information Y means that information Y can be obtained from information X, or information X can be obtained from information Y, and the method of association is not important.
  • Information X and information Y may be linked, may exist in the same buffer, information X may be included in information Y, or information Y may be included in information X, etc.
  • FIG. 1 is a conceptual diagram of an information system A in this embodiment.
  • Information system A includes an information processing device 1, one or more mobile terminals 2, one or more fixed terminals 3, one or more user terminals 4, one or more target terminals 5, and one or more navigation terminals 6.
  • the information processing device 1 is a server that provides a video to a user terminal 4 using videos transmitted by one or more mobile terminals 2 and one or more fixed terminals 3.
  • the information processing device 1 is, for example, a cloud server or an ASP server, but the type is not important.
  • the information processing device 1 may also be a device that constitutes a blockchain.
  • the video here refers to images captured by the mobile terminal 2 or the fixed terminal 3.
  • the time interval between capture of the multiple still images contained in the video does not matter.
  • the video may be, for example, a video of 60 frames per second or a video of 30 frames per second, but it may also be a collection of multiple still images captured with an interval of a specified time (for example, one minute) or more, or a collection of multiple still images captured when specified conditions are met.
  • the mobile terminal 2 is a terminal that is installed on a moving object and that captures video.
  • the mobile terminal 2 is, for example, a drive recorder, a smartphone, a tablet terminal, or a camera with a communication function.
  • "installed” usually means “fixed,” but it may also be considered to include “contacting” or “holding.”
  • the mobile terminal 2 may also have a driving means such as an engine or a means of movement such as wheels.
  • a mobile body is an object that moves.
  • a mobile body is a terrestrial mobile device, a marine mobile device, an undersea mobile device, an aerial mobile device, a space mobile device, or a living thing.
  • Ground mobile equipment includes, for example, motor vehicles, bicycles, rickshaws, human-powered vehicles such as play cars, trains, steam locomotives and other railroad vehicles, rides in amusement parks, and commercial vehicles in factories and other facilities.
  • Ground mobile equipment can also be non-human-riding vehicles, such as various robots for commercial or play equipment, radio-controlled toys, and other such play equipment.
  • Motor vehicles include, for example, passenger cars, trucks, buses, taxis, and motorcycles.
  • Examples of marine vehicles include various types of ships, jet skis, surfboards, rowboats, swim rings, rafts, etc.
  • underwater mobile devices examples include submarines, diving robots, and diving equipment such as aqualungs.
  • Aerial vehicles include, for example, various aircraft, helicopters, gliders, parachutes, balloons, kites, etc.
  • Space vehicles include, for example, rockets, space probes, and artificial satellites.
  • Living things include, for example, humans and non-human moving objects.
  • Non-human moving objects include, for example, birds, mammals, reptiles, amphibians, fish, insects, and various other living things.
  • the fixed terminal 3 is a terminal that is installed at a fixed location and has a photographing function.
  • the fixed terminal 3 has a photographing means and a communication means.
  • the photographing means is, for example, a so-called surveillance camera.
  • the fixed terminal 3 has, for example, a surveillance camera installed in a city.
  • the fixed terminal 3 has, for example, a surveillance camera installed in a building such as a private home or a building.
  • the user terminal 4 is a terminal used by a user.
  • a user is someone who watches videos or needs videos.
  • the user terminal 4 may have the functions of a mobile terminal 2.
  • the user terminal 4 may be a terminal of a user who provides videos.
  • the user terminal 4 may also be a navigation terminal 6, which will be described later.
  • the target terminal 5 is a terminal that transmits location information.
  • the target terminal 5 is, for example, a terminal provided in the monitored object.
  • the monitored object is, for example, a living thing or an object.
  • a living thing is, for example, a person or an animal such as a pet.
  • a person is, for example, a child or an elderly person.
  • An object to be monitored is, for example, an object that should not be stolen, such as a car, a motorcycle, or a bicycle. However, the monitored object can be anything.
  • the navigation terminal 6 is a so-called navigation terminal that provides the user with information about the user's current location and guidance to the destination to which the user is heading.
  • the navigation terminal 6 is a terminal that is installed in a ground mobile device or held by a person.
  • the information processing device 1 and one or more mobile terminals 2 are typically capable of communicating with each other via a network such as the Internet. Furthermore, the information processing device 1 and one or more fixed terminals 3 are typically capable of communicating with each other via a network such as the Internet. Furthermore, the information processing device 1 and one or more user terminals 4 are typically capable of communicating with each other via a network such as the Internet. Furthermore, the information processing device 1 and one or more target terminals 5 are typically capable of communicating with each other via a network such as the Internet. Furthermore, the information processing device 1 and one or more navigation terminals 6 are typically capable of communicating with each other via a network such as the Internet. Furthermore, the user terminal 4 and the target terminal 5, and the user terminal 4 and the navigation terminal 6 are typically capable of communicating with each other via a network such as the Internet.
  • Fig. 2 is a block diagram of an information system A in this embodiment.
  • Fig. 3 is a block diagram of an information processing device 1.
  • Fig. 4 is a block diagram of a fixed terminal 3. ⁇ Example of configuration of each device>
  • the information processing device 1 includes a storage unit 11, a receiving unit 12, a processing unit 13, and a transmitting unit 14.
  • the storage unit 11 includes a mobile terminal management unit 111, and a fixed terminal management unit 112.
  • the receiving unit 12 includes a moving information receiving unit 121, an inquiry receiving unit 122, and a collection receiving unit 123.
  • the processing unit 13 includes a moving information accumulation unit 131, a collection accumulation unit 132, a video acquisition unit 133, a video composition unit 134, a video processing unit 135, and a right holder processing unit 136.
  • the video acquisition unit 133 includes a moving video acquisition means 1331, and a fixed video acquisition means 1332.
  • the right holder processing unit 136 includes a first maintenance means 1361, a second maintenance means 1362, a third maintenance means 1363, a fourth maintenance means 1364, and a reward means 1365.
  • the transmitting unit 14 includes a video transmission unit 141, a status transmission unit 142, and a needs transmission unit 143.
  • the mobile terminal 2 includes a mobile storage unit 21, a mobile receiving unit 22, a mobile processing unit 23, and a mobile transmitting unit 24.
  • the mobile processing unit 23 includes a photographing unit 231, a tag acquiring unit 232, and a mobile information acquiring unit 233.
  • the mobile transmitting unit 24 includes a mobile information transmitting unit 241, a mobile video transmitting unit 242, and a collection transmitting unit 243.
  • the fixed terminal 3 includes a fixed storage unit 31, a fixed processing unit 33, a fixed receiving unit 32, and a fixed transmitting unit 34.
  • the fixed processing unit 33 includes a fixed camera 331.
  • the user terminal 4 includes a user storage unit 41 , a user reception unit 42 , a user processing unit 43 , a user transmission unit 44 , a user reception unit 45 , and a user output unit 46 .
  • the storage unit 11 constituting the information processing device 1 stores various types of information.
  • the various types of information are, for example, mobile terminal information (described later), fixed terminal information (described later), attribute value sets (described later), mobile information (described later), video, and management information (described later).
  • the attribute value sets are mobile attribute value sets or fixed attribute value sets.
  • the management information is information used to monitor the person being watched.
  • the management information includes a user identifier for sending video to the user terminal 4 of the person watching the person, and a target identifier for identifying the target terminal 5 of the person being watched.
  • the mobile terminal management unit 111 stores one or more pieces of mobile terminal information.
  • the mobile terminal information is information related to the mobile terminal 2.
  • the mobile terminal information includes information that identifies the video currently held by the mobile terminal 2.
  • the mobile terminal information includes a mobile terminal identifier and a set of mobile attribute values.
  • the mobile terminal information may be associated with the video.
  • the mobile terminal identifier may be included in the set of mobile attribute values.
  • the mobile terminal identifier is information that identifies the mobile terminal 2.
  • the mobile terminal identifier may be a rights holder identifier that identifies the rights holder who is the user of the mobile terminal 2.
  • the mobile terminal identifier is, for example, the ID of the mobile terminal 2, the user identifier of the user of the mobile terminal 2, the name of the mobile terminal 2, the IP address of the mobile terminal 2, or the MAC address of the mobile terminal 2.
  • a rights holder is someone who has some right relating to a video.
  • a rights holder is, for example, the owner of the video, the copyright holder of the video, the owner of the mobile terminal 2 that shot the video, or the owner of the fixed terminal 3 that shot the video.
  • a rights holder is the initial rights holder of the video.
  • a rights holder is usually the holder of the mobile terminal 2 or the owner of the fixed terminal 3, but it can be anyone who has the rights to the video shot by the mobile terminal 2 or the video shot by the owner of the fixed terminal 3.
  • the rights holder identifier is an identifier of the rights holder of the video.
  • the rights holder identifier may be a terminal identifier.
  • the rights holder identifier is, for example, the rights holder's ID, the rights holder's name, the rights holder's email address, or the rights holder's phone number.
  • the rights holder ID is, for example, a user identifier.
  • a moving attribute value set is a set of one or more moving video attribute values.
  • a moving video attribute value is an attribute value of a moving video.
  • a moving video attribute value is, for example, environmental information.
  • a moving video attribute value is, for example, a tag.
  • a moving video attribute value is usually a dynamic attribute value that can change dynamically, but may also be a static attribute value that does not change dynamically.
  • Environmental information is information about the environment in which the video was shot. Examples of environmental information include location information, direction information, camera information, time information, weather information, temperature information, and season information.
  • Location information is information that identifies the shooting location. The shooting location is the location of the camera taking the video. Location information is, for example, (latitude, longitude), (latitude, longitude, altitude). However, location information may also be an area identifier that identifies an area on a map, an address, a road identifier that identifies a road, a lane identifier that identifies a lane on a road, etc.
  • Directional information is information that identifies the direction of shooting. Directional information is, for example, an angle from true north.
  • Camera information is information about the camera.
  • Time information is information that identifies the time when the video was shot.
  • the time when the video was shot may be a time around the time when the video was shot, and accuracy is not required.
  • Time information is, for example, time, year/month/date/time, year/month/date/time/minute, year/month/date/time/minute/second, year/month/date, and month/day. In other words, the granularity of the time indicated by the time information does not matter.
  • Weather information is information that specifies the weather at the location where the video was shot when it was shot.
  • weather information examples are “sunny,” “rain,” “snow,” and “cloudy.”
  • Temperature information is information that specifies the outside air temperature at the location where the video was shot when it was shot. Examples of temperature information are “25 degrees” and “above 30 degrees.”
  • Season information is information that specifies the season at the location where the video was shot when it was shot. Examples of season information are “spring,” “summer,” “early summer,” and “winter.”
  • a tag is information that identifies the characteristics of a video.
  • a tag is, for example, information resulting from analyzing a video.
  • a tag is, for example, information resulting from analyzing one or more moving object attribute values.
  • a tag is, for example, information resulting from analyzing multiple moving object attribute values in a time series.
  • a mobile object attribute value is an attribute value related to a mobile object.
  • a mobile object attribute value is, for example, information related to movement that a mobile object can acquire while moving.
  • a mobile object attribute value is, for example, CAN (Controller Area Network) data, or information indicating the use of an airbag.
  • CAN data is, for example, speed, engine RPM, or brake status.
  • a tag is, for example, "accident,” “traffic jam,” “dangerous driving,” “speeding,” or the name of an object shown in the video (for example, "person,” “bear,” or the name of a famous person).
  • the moving video attribute values of location information, direction information, time information, weather information, temperature information, and season information are dynamic attribute values.
  • camera information is a static attribute value.
  • the fixed terminal management unit 112 stores one or more pieces of fixed terminal information.
  • the fixed terminal information is information relating to the fixed terminal 3.
  • the fixed terminal information includes a fixed terminal identifier and a set of fixed attribute values.
  • the fixed terminal information may be associated with a video.
  • the fixed terminal identifier may be included in the set of fixed attribute values.
  • the fixed terminal identifier is information that identifies the fixed terminal 3.
  • the fixed terminal identifier may be a rights holder identifier that identifies the rights holder who is the user of the fixed terminal 3.
  • the fixed terminal identifier is, for example, the ID of the fixed terminal 3, the name of the fixed terminal 3, the IP address of the fixed terminal 3, or the MAC address of the fixed terminal 3.
  • the fixed attribute value set is a set of one or more fixed video attribute values.
  • a fixed video attribute value is an attribute value of a fixed video.
  • Fixed video attribute values are, for example, environmental information and tags.
  • Fixed video attribute values are dynamic attribute values that can change dynamically, or static attribute values that do not change dynamically.
  • the static attribute values of the fixed terminal 3 are, for example, location information and camera information.
  • the dynamic attribute values of the fixed terminal 3 are, for example, time information, weather information, temperature information, and season information.
  • the receiving unit 12 receives various information and instructions from the mobile terminal 2, the fixed terminal 3, the user terminal 4, the target terminal 5, or the navigation terminal 6.
  • the various information and instructions are, for example, movement information, position information, inquiries, attribute value sets, and videos.
  • the receiving unit 12 receives video from the mobile terminal 2. Such video is called a moving video.
  • the receiving unit 12 receives a moving video from the mobile terminal 2 that has accepted the user's permission, according to the usage mode flag stored in the mobile terminal 2.
  • the usage mode flag is information that specifies the stance of the rights holder of the moving video when a third party uses the moving video.
  • the usage mode flag is, for example, information indicating that "there is a desire for permanent use” or information indicating that "temporary use is acceptable.” Note that permanent use is not temporary use.
  • the usage mode flag is, for example, information indicating that "permission from the rights holder is obtained for third parties to use the moving video" or information indicating that "third parties may freely use the moving video.”
  • the receiving unit 12 receives the moving video from the mobile terminal 2 only if the mobile terminal 2 accepts the user's permission.
  • the receiving unit 12 receives video captured by the fixed camera 331 from the fixed terminal 3. Such video is called a fixed video.
  • the video received by the receiving unit 12 is associated with a rights holder identifier that identifies the rights holder of the video.
  • the video received by the receiving unit 12 is associated with, for example, one or more video attribute values.
  • the video received by the receiving unit 12 is, for example, a video taken of a parking lot.
  • the video received by the receiving unit 12 is, for example, a video showing a child who is being watched over. However, the location from which the video received by the receiving unit 12 is obtained does not matter.
  • the movement information receiving unit 121 receives movement information from the mobile terminal 2 when the mobile terminal 2 starts moving.
  • the start time is preferably immediately after the start, but may be after a predetermined time (e.g., one minute) has elapsed since the start of the movement.
  • the movement information is information that specifies that the mobile terminal 2 is moving.
  • the information that specifies that the mobile terminal 2 is moving may be information that specifies that the mobile terminal 2 has started moving.
  • the movement here may mean that the mobile terminal 2 is about to start moving, or that the mobile terminal 2 is currently moving.
  • the movement information is, for example, a movement start flag and a terminal identifier.
  • the movement start flag is information that indicates that the mobile terminal 2 is about to start moving.
  • the movement start means, for example, that the engine has been turned on and that the mobile terminal 2 has started moving.
  • the terminal identifier is an identifier of the mobile terminal 2 that has started moving.
  • the terminal identifier may be the same as the rights holder identifier.
  • the inquiry receiving unit 122 receives an inquiry regarding environmental information.
  • the inquiry receiving unit 122 typically receives an inquiry from the user terminal 4.
  • the inquiry receiving unit 122 may also receive an inquiry from the target terminal 5 or the navigation terminal 6.
  • a query regarding environmental information is a query that includes environmental information as a condition.
  • the environmental information here is information about the environment in which the mobile terminal 2 or the fixed terminal 3 shot a video.
  • the query is a request for a video shot by the mobile terminal 2 or the fixed terminal 3.
  • the query is, for example, SQL, but the format and data structure, etc., are not important.
  • the inquiry receiving unit 122 receives an inquiry that includes, for example, location information.
  • the inquiry receiving unit 122 receives an inquiry that includes, for example, location information from the user terminal 4.
  • the inquiry receiving unit 122 sequentially receives, for example, each of the multiple pieces of location information transmitted from the target terminal 5.
  • the inquiry receiving unit 122 may receive each of the multiple pieces of location information from the target terminal 5, or may receive the information from a user terminal 4 or the like that has received the information from the target terminal 5.
  • the inquiry receiving unit 122 receives an inquiry including, for example, location information that identifies a destination set on the navigation terminal 6.
  • the inquiry receiving unit 122 may receive the location information from the navigation terminal 6, may receive it from a user terminal 4 that received it from the navigation terminal 6, or may receive it from a user terminal 4 that sent it to the navigation terminal 6 to set the destination.
  • the inquiry receiving unit 122 receives an inquiry including route information that includes multiple pieces of location information, for example.
  • the inquiry receiving unit 122 receives an inquiry including route information from, for example, the user terminal 4 or the navigation terminal 6.
  • the route information contained in the inquiry is, for example, information specifying the route traveled by the user watching the video, or route information set in the navigation terminal 6 or the user terminal 4 with a navigation function. It is preferable that the route information has time information corresponding to each of the multiple pieces of position information.
  • the time information is, for example, information specifying the time when the user was at the location specified by the position information.
  • the distance interval between each of the multiple pieces of position information contained in the route information does not matter.
  • the time when the user was at the location specified by the position information is the time when the video was filmed at that location.
  • the set receiving unit 123 receives the movement attribute value set from the mobile terminal 2. It is preferable that the set receiving unit 123 receives the movement attribute value set from the mobile terminal 2 when the movement of the mobile terminal 2 ends.
  • the movement attribute value set here is information that identifies the video stored in the mobile terminal 2. Note that the movement attribute value set received by the set receiving unit 123 is associated with the identifier of the mobile terminal 2 (e.g., terminal identifier, rights holder identifier).
  • the set receiving unit 123 receives, for example, a set of fixed attribute values associated with the identifier of the fixed terminal 3 from the fixed terminal 3.
  • the set receiving unit 123 receives, for example, dynamic attribute values that can change dynamically from among the fixed video attribute values in the fixed attribute value set.
  • the dynamic attribute values here are, for example, time information, weather information, temperature information, seasonal information, and tags. It is preferable that the set receiving unit 123 does not receive location information and camera information from the fixed terminal 3.
  • the location information and camera information of the fixed terminal 3 are static attribute values, and are preferably stored in advance in the fixed terminal management unit 112.
  • the processing unit 13 performs various types of processing.
  • the various types of processing are, for example, processing performed by the movement information storage unit 131, the collection storage unit 132, the video acquisition unit 133, the video composition unit 134, the video processing unit 135, and the rights holder processing unit 136.
  • the movement information storage unit 131 stores the movement information received by the movement information receiving unit 121 in association with the mobile terminal 2.
  • Association with the mobile terminal 2 means, for example, association with a rights holder identifier or a terminal identifier.
  • the set storage unit 132 stores the mobile attribute value set received by the set receiving unit 123 in the mobile terminal management unit 111.
  • the set storage unit 132 usually stores the attribute value set in association with the mobile terminal 2 that has transmitted the mobile attribute value set.
  • Association with the mobile terminal 2 means, for example, association with the rights holder identifier or terminal identifier of the mobile terminal 2.
  • the set storage unit 132 stores the fixed attribute value set received by the set receiving unit 123 in the fixed terminal management unit 112.
  • the set storage unit 132 normally stores the attribute value set in association with the fixed terminal 3 that transmitted the fixed attribute value set.
  • Association with the fixed terminal 3 means, for example, association with the rights holder identifier or terminal identifier of the fixed terminal 3.
  • the video acquisition unit 133 acquires moving videos captured by one or more mobile terminals 2.
  • the video acquisition unit 133 also acquires fixed videos captured by one or more fixed terminals 3.
  • the video acquisition unit 133 normally acquires moving videos and fixed videos received by the receiving unit 12.
  • the moving video acquired by the video acquisition unit 133 is usually associated with a moving attribute value set.
  • the moving attribute value set includes one or more pieces of environmental information. It is preferable that the one or more pieces of environmental information include location information that specifies the shooting location or time information that specifies the shooting time. It is also preferable that the moving video is associated with a rights holder identifier or a terminal identifier of the mobile terminal 2.
  • the fixed video acquired by the video acquisition unit 133 is usually associated with a fixed attribute value set.
  • the fixed attribute value set includes one or more pieces of environmental information. It is preferable that the one or more pieces of environmental information include location information that specifies the shooting location or time information that specifies the shooting time. It is also preferable that the fixed video is associated with a rights holder identifier or a terminal identifier of the fixed terminal 3.
  • the video acquisition unit 133 acquires a video that corresponds to a priority type when there are multiple videos that satisfy the adoption conditions, for example, including fixed videos and moving videos.
  • the video acquisition unit 133 acquires a video that corresponds to a priority type from among the fixed videos and moving videos when there are multiple videos that satisfy the adoption conditions, for example, including fixed videos and moving videos that correspond to location information indicating positions close enough to satisfy the first location condition.
  • the video acquisition unit 133 may use one or more video attribute values of the multiple videos to determine the video that will ultimately be adopted. For example, when there are multiple videos that satisfy the adoption conditions, the video acquisition unit 133 determines the video with the highest resolution.
  • the adoption conditions are conditions for the original videos that compose one video. It is preferable that the adoption conditions include one or more of the position conditions or time conditions.
  • the position conditions are conditions related to the position information associated with the video.
  • the position condition is a video associated with position information whose distance from the position indicated by the position information included in the inquiry received by the inquiry receiving unit 122 is within a threshold or less than a threshold.
  • the time condition is a condition related to the time information associated with the video.
  • the time condition is a video associated with time information indicating the time closest to the current time.
  • the position condition is a first position condition or a second position condition.
  • one video is a video that is ultimately provided to a user, and is, for example, a combined video or a fusion video.
  • the first location condition is a condition related to location information.
  • the first location condition is that the location information is in an approximate relationship with the location specified by the target location information.
  • An approximate relationship is closeness, for example, a distance within a threshold, a distance less than a threshold, a travel time within a threshold, or a travel time less than a threshold.
  • the target location information is, for example, location information included in a query, received location information, or location information included in route information included in a query.
  • the first position condition is usually a condition that assumes that an image of the position specified by the target position information is captured.
  • the first position condition may include conditions other than position information.
  • the first position condition may be a condition related to position information and time information.
  • the first position condition is, for example, that the position information and time information satisfy an approximation condition for the pair of target position information and time information.
  • the position information and time information that satisfy an approximation condition for the pair of target position information and time information are, for example, position information indicating a position whose distance is within a threshold, whose distance is less than a threshold, whose travel time is within a threshold, or whose travel time is less than a threshold, from the position specified by the target position information, and time information whose time difference is within a threshold or less than a threshold, from the time specified by the target time information.
  • the first position condition may be a condition related to position information and the traveling direction of a moving object.
  • the first position condition may be a condition related to position information and camera direction information.
  • the second position condition is position information that is in an approximate relationship to the reference position or reference area of the comparison target.
  • An approximate relationship here means that the distance from the reference position is within a threshold, the distance from the reference position is less than a threshold, or the position is within the reference area.
  • the reference area is, for example, a specific area such as a parking lot.
  • the comparison target is the location where the video is synthesized, for example, a specific location such as a parking lot.
  • the priority type is information that identifies the type of video that is prioritized between fixed video and moving video.
  • the priority type is, for example, either "fixed video” or "moving video”. It is preferable that the priority type is "fixed video”.
  • the video acquisition unit 133 sequentially acquires multiple videos that meet one or more of the position conditions and time conditions, for example.
  • Such multiple videos include one or more moving videos and one or more fixed videos.
  • the video acquisition unit 133 acquires, for example, multiple videos corresponding to inquiries received by the inquiry receiving unit 122.
  • the video acquisition unit 133 acquires, for example, multiple videos corresponding to location information that satisfies a first location condition using the location information received by the inquiry receiving unit 122.
  • the video acquisition unit 133 sequentially uses each of the multiple pieces of location information received by the inquiry receiving unit 122 to sequentially acquire multiple videos corresponding to the location information that satisfies the first location condition.
  • the video acquisition unit 133 sequentially acquires, for example, multiple videos corresponding to location information that satisfies the first location condition using the location information received by the inquiry receiving unit 122.
  • the video acquisition unit 133 acquires, for example, multiple videos corresponding to location information that satisfies multiple first location conditions using multiple pieces of location information contained in the route information.
  • the video acquisition unit 133 acquires, for example, multiple videos corresponding to location information that satisfies a first location condition using each of multiple pieces of location information and time information contained in the route information.
  • the video acquisition unit 133 acquires a plurality of videos in which, for example, the location information associated with each of the plurality of videos satisfies a second location condition, and the time information associated with each of the plurality of videos satisfies a time condition.
  • the second location condition is, for example, that the location information is information indicating a location within a specified area (for example, a specific parking lot).
  • the time condition is, for example, that the time information is within a threshold value (for example, within 30 seconds) of the current time or a time earlier than the threshold value.
  • the multiple videos acquired by the video acquisition unit 133 typically include one or more moving videos and one or more fixed videos. However, the multiple videos acquired by the video acquisition unit 133 may all be moving videos or all be fixed videos.
  • the video acquisition unit 133 acquires videos that satisfy the accumulation conditions. It is also preferable that the videos acquired by the video acquisition unit 133 are videos that correspond to an attribute value set and a rights holder identifier.
  • the storage conditions are conditions for storing videos.
  • the storage conditions are to meet the conditions specified by the inquiry.
  • the storage conditions are to meet the conservation conditions described below.
  • the storage conditions may be the same as the adoption conditions.
  • the device in which the video to be acquired is stored may be the mobile terminal 2, the information processing device 1, or another device not shown.
  • the other device may be, for example, a device that constitutes a blockchain.
  • the video acquisition unit 133 acquires one or more videos corresponding to the inquiry from the mobile terminal 2 or the fixed terminal 3. For example, the video acquisition unit 133 acquires one or more videos that are paired with an attribute value set that satisfies a condition related to environmental information included in the inquiry from the mobile terminal 2 or the fixed terminal 3.
  • the video acquisition unit 133 interprets a query, detects a fixed terminal 3 that can provide a video that matches the query, and receives a video captured by the fixed terminal 3 from the fixed terminal 3. For example, the video acquisition unit 133 acquires a terminal identifier that pairs with location information that matches a first location condition for the location information included in the query from the fixed terminal management unit 112, transmits a video transmission instruction to the fixed terminal 3 identified by the terminal identifier, and receives a video corresponding to the video transmission instruction from the fixed terminal 3.
  • the video acquisition unit 133 acquires a terminal identifier that pairs with location information and direction information that match a first location condition for the location information and direction information included in the query from the fixed terminal management unit 112, transmits a video transmission instruction to the fixed terminal 3 identified by the terminal identifier, and receives a video from the fixed terminal 3.
  • a video transmission instruction is usually an instruction to transmit a video that is currently being shot, but it may also be an instruction to transmit a video that was previously shot by the fixed terminal 3 and is stored in the fixed terminal 3 or another device not shown.
  • the video transmission instruction may also be an inquiry. In other words, the inquiry may be an instruction to transmit a video that is currently being shot.
  • the video acquisition unit 133 sends a query to one or more mobile terminals 2, and receives one or more videos corresponding to the query from one or more mobile terminals 2.
  • This process is called unregistered video search processing.
  • the unregistered video search processing is processing for acquiring videos that match a condition from unregistered videos that are stored in the mobile terminal 2.
  • the unregistered video search processing is processing for acquiring videos that match a query from unregistered videos that are stored in the mobile terminal 2, for example.
  • the video acquisition unit 133 interprets a query and determines that there is no fixed terminal 3 that can provide a video that matches the query, it preferably transmits the query to one or more mobile terminals 2 and receives one or more videos corresponding to the query from the one or more mobile terminals 2.
  • the video acquisition unit 133 acquires a video corresponding to the inquiry from among videos taken by one or more mobile terminals 2 corresponding to the movement information, for example.
  • the video acquisition unit 133 transmits an inquiry to each of one or more mobile terminals 2 corresponding to the movement information, and receives a video that matches the inquiry from the mobile terminal 2.
  • the mobile terminal 2 corresponding to the movement information is a mobile terminal 2 that is moving and is capable of transmitting a video.
  • the video acquisition unit 133 determines one or more sets of attribute values that match the inquiry, and acquires videos corresponding to each of the one or more sets of attribute values from the storage unit 11.
  • the video acquisition unit 133 refers to the mobile terminal management unit 111, determines one or more videos corresponding to the inquiry, and acquires the one or more videos from the mobile terminal 2.
  • the process of determining a video corresponding to a query by referring to the mobile terminal management unit 111 is to determine an attribute value set that matches the query from among one or more attribute value sets held by the mobile terminal management unit 111.
  • This process is called a registered video search process.
  • the registered video search process is a process of searching registered videos for videos that match a condition.
  • the registered video search process is, for example, a process of searching registered videos for videos that match a query. Note that a registered video is a video for which a first preservation process described below or a second preservation process described below has been performed.
  • the video acquisition unit 133 refers to the mobile terminal management unit 111, determines the video corresponding to the inquiry, determines whether or not the video is in a state where it can be transmitted from the mobile terminal 2, and acquires the video from the mobile terminal 2 only if it is in a state where it can be transmitted.
  • the video acquisition unit 133 attempts to communicate with the mobile terminal 2, and if it can receive information from the mobile terminal 2, it determines that the mobile terminal 2 is in a state in which it can transmit video. For example, the video acquisition unit 133 determines whether or not mobile information corresponding to the mobile terminal 2 is stored in the storage unit 11, and if it is stored, it determines that the mobile terminal 2 is in a state in which it can transmit video. Note that a state in which video can be transmitted is, for example, a state in which the power of the mobile terminal 2 or the mobile object corresponding to the mobile terminal 2 is ON (for example, a state in which the engine of a car, which is a mobile object, is ON). In addition, the method by which the video acquisition unit 133 determines whether or not it is in a state in which it can transmit video is not important.
  • the video acquisition unit 133 refers to the fixed terminal management unit 112, determines one or more fixed videos corresponding to the inquiry, and acquires the fixed videos from the fixed terminal management unit 112.
  • the video acquisition unit 133 acquires videos corresponding to a mobile attribute value set that satisfies the conservation condition from the mobile terminal 2.
  • the video acquisition unit 133 acquires videos corresponding to a fixed attribute value set that satisfies the conservation condition from the fixed terminal 3. Through this process, videos can be acquired automatically.
  • a conservation condition is a condition for storing video.
  • a conservation condition is a condition related to an attribute value set. It is preferable that a conservation condition is a condition related to one or more tags.
  • a conservation condition is that the attribute value set includes a tag indicating "accident", that the attribute value set includes a tag indicating "traffic jam", or that the attribute value set includes a tag indicating a specific location.
  • a tag indicating a specific location is, for example, the name of a specific parking lot, a specific place name, the name of a specific scenic spot, or the name of a specific landscape.
  • the preservation condition is that the video corresponds to an attribute value set that satisfies a predetermined condition for an attribute value set corresponding to one or more videos stored in a predetermined device or area (e.g., storage unit 11).
  • the predetermined condition is, for example, that an attribute value possessed by the attribute value set does not exist in the attribute value set corresponding to one or more videos stored in the predetermined device or area.
  • the video acquisition unit 133 may acquire a processed image obtained by, for example, processing a video captured by the mobile terminal 2 or the fixed terminal 3.
  • a processed image is, for example, an around-view image.
  • An around-view image is an image displayed on an around-view monitor.
  • An around-view image is an image that appears to be captured from directly above an area including a moving object.
  • the moving video acquisition means 1331 performs the process of acquiring moving videos among the processes performed by the video acquisition unit 133.
  • the fixed video acquisition means 1332 performs the process of acquiring fixed videos among the processes performed by the video acquisition unit 133.
  • the video composition unit 134 combines the moving video and the fixed video in terms of time or space to compose a single video.
  • the video composition unit 134 composes one video using multiple videos acquired by the video acquisition unit 133. Compiling one video may involve sequentially passing parts of videos that appear as one video to the user to the video transmission unit 141. Note that the multiple videos include moving videos and fixed videos. Also, one video may be a collection of multiple partial videos with gaps between transmissions.
  • the moving image composing unit 134 performs (1) temporal combination of moving images, or (2) spatial combination of moving images, or both (1) and (2).
  • the video composition unit 134 for example, combines one or more moving videos and one or more fixed videos in time to compose a single video.
  • the video composition unit 134 temporally combines multiple videos that have different time information associated with each of the multiple videos acquired by the video acquisition unit 133 to compose a single video.
  • Combining multiple videos temporally to compose a single video may involve passing the multiple videos in sequence to the video transmission unit 141. In other words, combining multiple videos temporally to compose a single video is sufficient as long as the result is a single video that appears to the user.
  • the video composition unit 134 for example, combines the multiple videos acquired by the video acquisition unit 133 in a time order corresponding to the videos to compose one video.
  • the video composition unit 134 acquires parts of the multiple videos acquired by the video acquisition unit 133, and combines the parts of the videos in order to compose one video.
  • Combining multiple videos in time usually means combining parts of the videos taken by the mobile terminal 2 or the fixed terminal 3 in order.
  • Combining parts of the videos in order may also mean passing parts of the videos to the video transmission unit 141 in order.
  • Combining in a time order corresponding to the videos means combining the videos in a time order indicated by the time information corresponding to the videos, or combining the videos in the order in which the videos were received.
  • the video composition unit 134 for example, spatially fuses parts of multiple videos acquired by the video acquisition unit 133, each of which has different positional information associated with it, to compose a single video.
  • the video composition unit 134 for example, composes a single frame using some or all of the frames that compose each of the multiple videos acquired by the video acquisition unit 133, and connects the multiple frames in time to compose a single video.
  • the video composition unit 134 composes a video having multiple frames by spatially blending, for example, at least some of the frames of the moving video and at least some of the frames of the fixed video.
  • the video composition unit 134 processes each of the multiple videos acquired by the video acquisition unit 133 to compose one video.
  • the video composition unit 134 for example, spatially connects a directly above frame that combines frames of a moving video and a fixed image to compose one frame, and composes one video having one frame.
  • the video composition unit 134 for example, synthesizes multiple around-view images to compose a wide-area around-view image.
  • around-view is a registered trademark.
  • the multiple original videos that the video composition unit 134 uses to compose a single video include moving videos and fixed videos.
  • the process of spatially fusing frames included in each of a plurality of moving images is, for example, the following processes (a) and (b).
  • the video composition unit 134 performs, for example, a process of matching the direction and scale of each of the multiple frames that are the subject of the spatial connection process. Next, the video composition unit 134 detects, for example, identical areas in each of the multiple frames. Next, the video composition unit 134 performs, for example, a process of overlapping multiple frames that have the same area to compose one wide-area frame. Note that it is possible to detect the same area in each of the multiple frames using publicly known technology. (b) Machine learning-based methods
  • the video composition unit 134 for example, provides multiple frames and a learning model to a module that performs machine learning prediction processing, executes the module, and obtains a single frame over a wide area.
  • the learning model is obtained by providing multiple training data sets, each of which has multiple frames as explanatory variables and a single wide-area frame composed of the multiple frames as a target variable, to a module that performs machine learning learning processing, and then executing the module.
  • the learning model may also be called a learner, a classifier, a classification model, etc.
  • the machine learning algorithm is preferably deep learning, but any algorithm such as random forest can be used.
  • various machine learning functions such as the TensorFlow library and the random forest module of the R language, as well as various existing libraries, can be used.
  • the video processing unit 135 obtains a stereoscopic three-dimensional video from at least a part of a video constructed by the video construction unit 134.
  • a stereoscopic 3D video is a 3D video made up of illustrated frames. Note that the technology for acquiring stereoscopic 3D video from camera footage is publicly known, so a detailed explanation will be omitted (see the Internet URL "https://xtech.nikkei.com/atcl/nxt/column/18/01883/00004/").
  • the processing conditions are conditions for identifying partial videos from which a stereoscopic 3D video is obtained.
  • the learning model here is information acquired by providing a machine learning learning module with multiple pieces of teacher data including captured still images and stereoscopic 3D images made from the still images, and executing the learning module. Deep learning is a suitable machine learning algorithm, but other algorithms such as random forests may also be used.
  • the rights holder processing unit 136 performs rights holder processing.
  • Rights holder processing is processing related to the rights of a video.
  • Rights holder processing is, for example, processing related to a rights holder identified by a rights holder identifier corresponding to a video composed by the video composition unit 134.
  • Rights holder processing is, for example, processing related to a rights holder identified by a rights holder identifier corresponding to a stereoscopic 3D video acquired by the video processing unit 135.
  • Rights holder processing is, for example, a first security processing described below, a second security processing described below, a third security processing described below, a fourth security processing described below, and a compensation processing described below.
  • the rights holder processing unit 136 performs rights holder processing, which is processing in response to the video transmission unit 141 transmitting a video, and is processing related to the rights holder identified by the rights holder identifier associated with the video.
  • the video transmitted by the video transmission unit 141 is a video constructed by the video construction unit 134, or a stereoscopic three-dimensional video acquired by the video processing unit 135.
  • the rights holder identifier associated with a single video may be, for example, a rights holder identifier associated with each of the multiple videos that formed the single video, or an identifier of the user who requested the single video.
  • the rights holder processing unit 136 stores the videos acquired by the video acquisition unit 133 in association with the rights holder identifier. It is preferable that the rights holder processing unit 136 stores only the videos acquired by the video acquisition unit 133 as satisfying the storage conditions. It is also preferable that the rights holder processing unit 136 does not store videos that do not satisfy the storage conditions.
  • the rights holder identifier is the rights holder identifier of the original video, the rights holder identifier of a video, or the rights holder identifier of the stereoscopic 3D video.
  • the rights holder processing unit 136 acquires status information relating to that state.
  • Status information is information about a state in which a video cannot be sent.
  • Status information is, for example, information indicating the state of the mobile terminal 2 that holds the video. Status information is, for example, "A video exists, but it is not currently in a state where it can be sent to you," or "A video exists on Mr. XX's mobile terminal, but it is not currently in a state where it can be sent to you.”
  • Status information is, for example, information indicating that the power of the mobile terminal 2 is OFF, or information indicating that the power of the mobile terminal 2 is ON.
  • the rights holder processing unit 136 acquires the needs information, for example, when the video acquisition unit 133 determines that the mobile terminal 2 holding the video corresponding to the inquiry is in a state in which the video cannot be transmitted.
  • Needs information is information indicating that there is a need for a video. Examples of needs information include "Your XXX video has been requested by other users" or "Your XXX video has been requested by other users for XX yen.”
  • the first preservation means 1361 performs a first preservation process of storing a video constructed by the video construction unit 134 or a stereoscopic three-dimensional video acquired by the video processing unit 135 in association with an attribute value set corresponding to each of the multiple videos that are the source of the video.
  • the first preservation means 1361 may also perform a first preservation process of storing a video received from the mobile terminal 2 or the fixed terminal 3 in association with an attribute value set corresponding to the video.
  • the attribute value set corresponding to the video is a moving attribute value set corresponding to a moving video or a fixed attribute value set corresponding to a fixed video.
  • the second preservation means 1362 performs a second preservation process in which a video composed by the video composition unit 134 or a stereoscopic three-dimensional video acquired by the video processing unit 135 is stored in association with a rights holder identifier corresponding to each of the multiple videos that are the source of the video.
  • the first preservation means 1361 or the second preservation means 1362 may store a video constructed by the video construction unit 134 or a stereoscopic three-dimensional video acquired by the video processing unit 135 in association with a set of attribute values corresponding to each of the multiple videos that were the source of the video, and with a rights holder identifier corresponding to each of the multiple videos that were the source of the video.
  • the third security means 1363 stores the one video constructed by the video construction unit 134 or the stereoscopic 3D video acquired by the video processing unit 135 in association with a rights holder identifier that identifies the user of the user terminal 4.
  • the user of the user terminal 4 here is the person who views the one video or the stereoscopic 3D video.
  • the user terminal 4 here is, for example, the terminal that sent the inquiry.
  • the destination for storing a video or a stereoscopic three-dimensional video is, for example, the storage unit 11, but it may also be another device, such as another device that constitutes a blockchain.
  • the stored video is usually associated with a video identifier that identifies the video.
  • the fourth preservation means 1364 performs a fourth preservation process to store preservation information including access information for accessing one stored video or stereoscopic three-dimensional video.
  • the order of storing the video and performing the fourth preservation process on the preservation information corresponding to the video does not matter.
  • the fourth preservation means 1364 performs a fourth preservation process, for example, by storing preservation information including access information for accessing a video that has been composed and stored by the video composition unit 134 in the blockchain.
  • the fourth security means 1364 performs a fourth security process, for example, by storing security information including access information for accessing the stored stereoscopic 3D video acquired by the video processing unit 135 in the blockchain.
  • the fourth security means 1364 accumulates the security information in the blockchain. That is, it is preferable that the fourth security means 1364 accumulates the security information in the distributed ledger of the blockchain. It is preferable that the fourth security means 1364 registers the security information as an NFT (non-fungible token). It is preferable that the fourth security means 1364 registers the security information in a distributed file system in an IPFS (Inter Planetary File System) network.
  • NFT non-fungible token
  • IPFS Inter Planetary File System
  • the preservation information is information for ensuring the originality of a video. It can be said that the preservation information is heading information for a video.
  • the preservation information is, for example, access information and one or more video attribute values. It is preferable that the preservation information has, for example, one or more rights holder identifiers.
  • the preservation information has multiple rights holder identifiers, the rights holder of the video may be shared, and the multiple rights holder identifiers may be rights holder history information.
  • the rights holder history information is a collection of rights holder identifiers and is information that indicates the history of the rights holder.
  • the fourth preservation process can guarantee the originality of the preservation information of a registered video.
  • the access information is information for accessing a video.
  • the access information is information that identifies the storage destination of the video.
  • the access information is, for example, a URL or a URI.
  • the preservation information includes information (which may be called a flag) indicating whether the video can be provided to a third party.
  • the flag may be, for example, information indicating that the video can be viewed by a third party, information indicating that the video can be sold, or information indicating that neither viewing nor selling is permitted.
  • the compensation means 1365 performs compensation processing for each right holder identified by a right holder identifier corresponding to each of the multiple videos that were the basis for one video composed by the video composition unit 134, for example.
  • the compensation means 1365 performs compensation processing for each right holder identified by a right holder identifier corresponding to each of the multiple videos that are the basis for the stereoscopic 3D video acquired by the video processing unit 135, for example.
  • the reward process is a process of giving a reward.
  • the reward process is, for example, a process of increasing points managed in pair with one or more rights holder identifiers associated with a video.
  • the reward process is, for example, a process of making a payment to a rights holder identified by one or more rights holder identifiers associated with a video.
  • the reward process is, for example, a process of transmitting a video or other content to a user terminal 4 of a rights holder identified by one or more rights holder identifiers associated with a video.
  • the reward process need only give a benefit to a rights holder identified by one or more rights holder identifiers associated with a video, and the content of the benefit is not important.
  • the reward may be, for example, money, points, goods, content, etc., and the content is not important.
  • the compensation means 1365 performs compensation processing, which is a process of acquiring one or more video attribute values corresponding to each of the multiple videos that were the basis for the single video transmitted by the video transmission unit 141, determining compensation for each of the multiple rights holders using the one or more video attribute values, and awarding the compensation.
  • the one or more video attribute values here are, for example, the amount of video data, the duration of the video, the number of frames in the video, and the resolution of the video.
  • the reward means 1365 preferably performs reward processing, which is a process of acquiring a reward amount corresponding to a service identifier that identifies a service provided to the target video, and giving a reward corresponding to the reward amount.
  • the service identifier is, for example, "view” or "purchase.”
  • the storage unit 11 stores the reward amount corresponding to the service identifier, or information for determining the reward amount corresponding to the service identifier.
  • the reward means 1365 performs reward processing, which is, for example, a process of acquiring a reward amount using one or more pieces of information from one or more video attribute values and one or more service identifiers, and granting a reward corresponding to the reward amount.
  • reward processing is, for example, a process of acquiring a reward amount using one or more pieces of information from one or more video attribute values and one or more service identifiers, and granting a reward corresponding to the reward amount.
  • an arithmetic formula or a correspondence table corresponding to each of the multiple service identifiers is stored in the storage unit 11.
  • the arithmetic formula is a formula that calculates the reward amount using one or more video attribute values as parameters.
  • the correspondence table is a table having multiple pieces of correspondence information that manage the reward amounts corresponding to one or more video attribute values.
  • the compensation means 1365 typically performs a process of charging compensation to users who have enjoyed the service related to the target video.
  • the process of charging the reward is, for example, a process of charging the amount of reward acquired.
  • the process of charging the reward is, for example, a process of charging the amount of reward acquired and the amount of profit obtained by the operator of the information processing device 1.
  • the process of charging the reward is, for example, a process of deducting points corresponding to the user who enjoyed the service, a payment process using the user's credit card number, etc.
  • the transmission unit 14 transmits various types of information or instructions to the mobile terminal 2, the fixed terminal 3, the user terminal 4, or the navigation terminal 6.
  • the various types of information or instructions are, for example, videos, inquiries, status information, and needs information.
  • the video transmission unit 141 transmits, for example, one video that has been composed by the video composition unit 134.
  • the video transmission unit 141 transmits, for example, a stereoscopic three-dimensional video instead of or in addition to the one video.
  • the video sending unit 141 sends a video or a stereoscopic 3D video to the user terminal 4. It is preferable that the video sending unit 141 sends a video in response to receiving an inquiry.
  • the video sending unit 141 sends a video to the user terminal 4 or the navigation terminal 6, for example. Sending a video may mean that the video sending unit 141 sequentially sends parts of the video that have been passed to it.
  • the video transmission unit 141 transmits the video acquired by the video acquisition unit 133 to the user terminal 4.
  • the status transmission unit 142 transmits status information regarding the status to the user terminal 4. For example, the status transmission unit 142 transmits status information acquired by the rights holder processing unit 136 to the user terminal 4. Note that whether or not a video cannot be transmitted is determined by, for example, the video acquisition unit 133.
  • the needs transmission unit 143 transmits needs information to the user corresponding to the video, indicating that there is a need for the video. For example, the needs transmission unit 143 transmits the needs information acquired by the rights holder processing unit 136 to the user corresponding to the video.
  • Sending to a user corresponding to a video means, for example, sending the needs information by email to a notification destination indicated by the notification destination information paired with the rights holder identifier corresponding to the video.
  • Sending to a user corresponding to a video means, for example, sending the needs information by short message to a telephone number destination paired with the rights holder identifier corresponding to the video.
  • Sending to a user corresponding to a video means, for example, sending the needs information to a mobile terminal 2 paired with the rights holder identifier corresponding to the video.
  • the user corresponding to a video is usually the rights holder of the video.
  • the notification destination information is, for example, an email address, a telephone number, an IP address, or an ID.
  • the movement storage unit 21 constituting the mobile terminal 2 stores various types of information.
  • the various types of information are, for example, a moving video, a moving attribute value set, a rights holder identifier, movement information indicating the start of movement, a pair of attribute value tag conditions and tags, a pair of video tag conditions and tags, one or more conservation conditions, one or more pieces of acquisition information, and a usage mode flag.
  • the movement storage unit 21 typically stores one or more pairs of attribute value tag conditions and tags.
  • the movement storage unit 21 typically stores one or more pairs of video tag conditions and tags.
  • One or more still images (which may also be called fields or frames) that make up a moving video are associated with, for example, one or more video attribute values that make up a movement attribute value set.
  • the one or more video attribute values may correspond to all still images, to some of the still images, or to multiple still images.
  • An attribute value tag condition is a condition for acquiring a tag based on one or more mobile object attribute values.
  • An attribute value tag condition is a condition related to one or more mobile object attribute values. Examples of attribute value tag conditions are “the brake is applied suddenly,” “the degree of speed deceleration (acceleration) within a unit time is below a threshold (the brake is applied suddenly),” “the airbag is deployed,” “the time at or below a first speed has elapsed for a second hour or more (there is a traffic jam),” and “the location information corresponds to a specific name.” Examples of tags that are paired with attribute value tag conditions are “abnormal driving,” “accident,” “traffic jam,” and “specific location name.”
  • Video tag conditions are conditions related to videos. Video tag conditions are conditions based on the results of video analysis. Examples of video tag conditions are “having a still image of multiple cars touching each other (it is an accident)", “having a frame where the distance between multiple cars is zero (cars are touching each other)", “having a period in which a first threshold or more of cars are present within a specified distance that is equal to or greater than a second threshold (it is a traffic jam)", and “the cumulative amount of change in the distance between the center of gravity of the car in front and the lane per unit time is equal to or greater than a threshold (the car in front is swerving)”.
  • Tags that pair with video tag conditions are, for example, “accident”, “traffic jam", and “abnormal driving”.
  • the preservation condition is a condition for preserving a video.
  • the preservation condition is a condition related to a set of attribute values. For example, such a preservation condition is "the airbag has been deployed (an accident has occurred)" or "a specific tag has been added to the video (the specific tag is, for example, "accident” or “dangerous driving")”.
  • the preservation condition corresponds to, for example, the acquisition information.
  • the acquisition information is information that specifies the video to be acquired.
  • the acquisition information is information that specifies the video to be acquired when the preservation condition is satisfied.
  • such a preservation condition is "(1) the number of cars on the screen is equal to or greater than a first threshold value & (2) the driving speed of the cars is equal to or less than a second threshold value & the duration of (1) and (2) is equal to or greater than a third threshold value (traffic jam continues)".
  • the acquisition information is information that indicates "the video from one minute before the preservation condition is met until the preservation condition is no longer met after it is met”.
  • the mobile receiving unit 22 receives various types of information.
  • the various types of information include, for example, inquiries, needs information, videos taken by other mobile terminals 2, and videos taken by the fixed terminal 3.
  • the movement processing unit 23 performs various types of processing.
  • the various types of processing are, for example, processing performed by the photographing unit 231, the tag acquisition unit 232, and the movement information acquisition unit 233.
  • the movement processing unit 23 converts the information received by the movement receiving unit 22 into a data structure to be output.
  • the movement processing unit 23 detects, for example, the start of movement. Detection of the start of movement is, for example, detection of the power supply of the mobile terminal 2 being turned on, or detection of the engine being turned on of the moving object.
  • the movement processing unit 23 detects, for example, the end of movement. Detection of the end of movement is, for example, detection of the power supply of the mobile terminal 2 being turned off, or detection of the engine of the moving object being turned off.
  • the movement processing unit 23 determines an attribute value set that matches the query received by the movement receiving unit 22, and retrieves a video that pairs with the attribute value set from the movement storage unit 21.
  • the movement processing unit 23 acquires a video from the start of the movement to the end of the movement.
  • the movement processing unit 23 acquires a video from the start of the movement to the end of the movement, for example, by associating the video with the attribute value set acquired during the movement.
  • the movement processing unit 23 acquires an attribute value set during shooting.
  • the movement processing unit 23 stores the acquired movement attribute value set in the movement storage unit 21.
  • the movement processing unit 23 also associates the acquired movement attribute value set with a moving image, for example.
  • Associating with a moving image usually means associating with frames that make up the moving image. It is preferable that the attribute value set and the frames are synchronized in time.
  • the movement attribute value set is, for example, one or more pieces of environmental information.
  • the environmental information is, for example, location information, time information, weather information, temperature information, and season information.
  • the movement processing unit 23 acquires location information, for example, during shooting.
  • the movement processing unit 23 having the function of a GPS receiver acquires location information.
  • the movement processing unit 23 acquires location information, for example, constantly, at predetermined intervals, or when acquisition conditions are met.
  • An acquisition condition is a condition for acquiring information.
  • An acquisition condition is, for example, when an accident is detected, when a traffic jam is detected, or when there is a change in weather information.
  • the movement processing unit 23 acquires time information from a clock (not shown), for example, during shooting.
  • the movement processing unit 23 acquires time information, for example, constantly, at predetermined intervals, or when acquisition conditions are met.
  • the movement processing unit 23 acquires time information from a clock (not shown) and acquires seasonal information corresponding to the time information.
  • the movement processing unit 23 acquires weather information, for example, during shooting.
  • the movement processing unit 23 acquires weather information corresponding to location information, for example, from a server not shown.
  • the movement processing unit 23 acquires weather information, for example, constantly, at predetermined intervals, or when acquisition conditions are met.
  • the movement processing unit 23 acquires temperature information, for example, during shooting.
  • the movement processing unit 23 acquires temperature information corresponding to location information, for example, from a server not shown.
  • the movement processing unit 23 acquires temperature information, for example, from a temperature sensor installed on a moving object.
  • the movement processing unit 23 acquires temperature information, for example, constantly, at predetermined intervals, or when acquisition conditions are met.
  • the movement processing unit 23, judges whether the acquired movement attribute value set matches the preservation condition.
  • the movement processing unit 23 is equipped with a microphone, acquires sound information, and stores the sound information in association with the video acquired by the image capturing unit 231.
  • a function is, for example, the function of a drive recorder.
  • the movement processing unit 23 acquires the usage mode flag of the movement storage unit 21, and judges whether the usage mode flag is information indicating that the user is to be queried as to whether or not to transmit the movement video before transmitting the movement video. For example, when the usage mode flag is a value indicating "non-provisional request for actual use", the movement processing unit 23 outputs inquiry information that is information for inquiring as to whether or not to transmit the movement video.
  • the inquiry information is, for example, screen information (for example, a panel) for inquiring as to whether or not to transmit the movement video, or audio information for inquiring as to whether or not to transmit the movement video.
  • the image capturing unit 231 captures video. For example, the image capturing unit 231 starts capturing video after detecting the start of movement. It is also preferable that the image capturing unit 231 continues capturing video until it detects the end of movement.
  • the image capture unit 231 stores the captured video in the mobile storage unit 21.
  • the image capture unit 231 stores video in the mobile storage unit 21, which has a limited storage capacity, it is preferable that the image capture unit 231 overwrites the area in which the old video is stored with the new video.
  • the structure of the mobile storage unit 21 is a ring buffer structure.
  • the tag acquisition unit 232 acquires one or more tags for the video captured by the imaging unit 231 and associates them with the video.
  • the tag acquisition unit 232 analyzes a video captured by the imaging unit 231 and acquires one or more tags for the video.
  • the tag acquisition unit 232 acquires one or more tags, for example, using one or more moving object attribute values acquired during shooting by the shooting unit 231.
  • the moving object attribute values are, for example, CAN data.
  • the tag acquisition unit 232 determines one or more still images that match the video tag condition, and acquires a tag that pairs with the video tag condition.
  • the tag acquisition unit 232 may also associate the tag with the one or more still images. Note that the still images are frames that make up the video.
  • the tag acquisition unit 232 analyzes the frames of the video, recognizes multiple cars, and acquires the distance between each of the multiple cars. The tag acquisition unit 232 also acquires the number of cars with a distance between them within a threshold. The tag acquisition unit 232 also acquires the moving distance and frame rate of one car in multiple frames, and acquires the car's speed. The tag acquisition unit 232 then uses the number of cars with a distance between them within a threshold and the car's speed to determine whether the video tag condition is met. If the video tag condition is met, the tag acquisition unit 232 acquires the tag "traffic jam" that is paired with the video tag condition. The tag acquisition unit 232 may also associate the tag "traffic jam" with the analyzed frame.
  • the tag acquisition unit 232 determines one or more moving object attribute values that match an attribute value tag condition, and acquires a tag that pairs with the attribute value tag condition.
  • the tag acquisition unit 232 may also associate the tag with a video that pairs with the one or more moving object attribute values.
  • the tag acquisition unit 232 uses the speed history contained in the CAN data associated with each field of the video to detect CAN data that matches the attribute value tag condition, acquires the tag "traffic jam” that matches the attribute value tag condition, and associates the tag with the field associated with the CAN data.
  • the CAN data associated with each field of the video is CAN data acquired at the same time that the field was photographed.
  • the movement information acquisition unit 233 detects the movement of the mobile terminal 2, and acquires movement information, for example, at the start of the movement.
  • the movement information acquisition unit 233 acquires movement information, which is, for example, a rights holder identifier of the movement storage unit 21.
  • the movement information acquisition unit 233 acquires movement information of the movement storage unit 21, for example.
  • the movement information is, for example, a rights holder identifier, and information indicating that movement has started.
  • Detection of the movement of the mobile terminal 2 is, for example, detection of the engine being ON, or detection of the power being ON of the mobile terminal 2.
  • the movement transmission unit 24 transmits various types of information to the information processing device 1.
  • the various types of information are, for example, movement information, movement videos, and a set of movement attribute values.
  • the movement information transmission unit 241 typically transmits the movement information acquired by the movement information acquisition unit 233 to the information processing device 1 at the start of movement.
  • the movement video sending unit 242 sends the movement video captured by the shooting unit 231 to the information processing device 1.
  • the timing at which the movement video sending unit 242 sends the video does not matter. For example, after the movement processing unit 23 acquires a video that pairs with a movement attribute value set that matches the query received, the movement video sending unit 242 sends the video to the information processing device 1. For example, when the movement video sending unit 242 detects the end of movement, it sends the video that has been accumulated since the start of movement was detected to the information processing device 1. For example, when the movement video sending unit 242 determines that the conservation conditions are met, it sends the video corresponding to that determination to the information processing device 1.
  • the mobile transmission unit 24 does not transmit the sound information acquired by the mobile processing unit 23 even if it transmits the mobile video captured by the imaging unit 231 to the information processing device 1.
  • the sound information may be, for example, voice information of the driver or passengers in the mobile object, and transmitting such sound information to the information processing device 1 and providing it to the user terminal 4 or the like may threaten the privacy of the driver or passengers, and is therefore not appropriate.
  • the movement processing unit 23 inquires of the user of the mobile terminal 2 whether or not to send a movement video based on the usage mode flag, the movement video sending unit 242 sends the movement video only if information indicating that a movement video will be sent is received in response to the inquiry. Note that information indicating that a movement video will be sent or that a movement video will not be sent is received by a movement receiving unit (not shown) that the mobile terminal 2 has.
  • the set sending unit 243 sends the set of movement attribute values in the movement storage unit 21 to the information processing device 1.
  • the set sending unit 243 sends the set of movement attribute values in the movement storage unit 21 to the information processing device 1, for example, when the movement of the mobile terminal 2 ends.
  • the set of movement attribute values in the movement storage unit 21 is a set of movement attribute values for the video stored in the movement storage unit 21.
  • the set transmitting unit 243 may immediately transmit the attribute value set to the information processing device 1. In other words, the set transmitting unit 243 may transmit the moved attribute value set at any time.
  • the various types of information are, for example, a fixed video, a fixed attribute value set, a rights holder identifier, camera information, a pair of attribute value tag conditions and tags, a pair of video tag conditions and tags, one or more conservation conditions, and one or more pieces of acquired information.
  • the fixed storage unit 31 typically stores one or more pairs of video tag conditions and tags. It is preferable that the fixed attribute value set includes location information of the fixed terminal 3.
  • One or more still images (which may also be called fields or frames) that make up a fixed video are associated with, for example, one or more video attribute values that make up a fixed attribute value set.
  • the one or more video attribute values may correspond to all still images, to some of the still images, or to multiple still images.
  • Video tag conditions are conditions related to videos. Video tag conditions are conditions based on the results of video analysis. Examples of video tag conditions are “having a still image of multiple cars colliding (it is an accident)", “having a frame in which the distance between multiple cars is zero (cars are colliding)", and “the cumulative amount of change per unit time between the center of gravity of the car in front and the lane is above a threshold (the car in front is swerving)”. Tags that pair with video tag conditions are, for example, “accident” and "abnormal driving”.
  • the fixed receiving unit 32 receives various types of information.
  • the various types of information include, for example, inquiries and various instructions.
  • the fixed processing unit 33 performs various types of processing.
  • the various types of processing are, for example, processing performed by the fixed camera 331.
  • the fixed processing unit 33 configures information to be transmitted in response to an inquiry received by the fixed receiving unit 32.
  • the fixed processing unit 33 configures information to be transmitted by the fixed transmitting unit 34.
  • the transmitted information includes a fixed video. It is preferable that the transmitted information includes a fixed attribute value set and a rights holder identifier.
  • the fixed processing unit 33 acquires a fixed attribute value set during shooting.
  • the fixed processing unit 33 stores the acquired fixed attribute value set in the fixed storage unit 31.
  • the fixed processing unit 33 also, for example, associates the acquired fixed attribute value set with a moving image.
  • Associating with a moving image usually means associating with frames that make up the moving image. It is preferable that the fixed attribute value set and the frames are synchronized in time.
  • the fixed attribute value set is, for example, one or more pieces of environmental information.
  • the environmental information is, for example, time information, weather information, temperature information, and season information.
  • the fixed processing unit 33 acquires time information from a clock (not shown), for example, during shooting.
  • the fixed processing unit 33 acquires time information, for example, constantly, at predetermined intervals, or when acquisition conditions are met.
  • the fixed processing unit 33 for example, acquires time information from a clock (not shown) during shooting and acquires seasonal information corresponding to the time information.
  • the fixed processing unit 33 acquires weather information, for example, during shooting.
  • the fixed processing unit 33 acquires weather information corresponding to the location information of the fixed terminal 3, for example, from a server not shown.
  • the fixed processing unit 33 acquires weather information, for example, constantly, at predetermined intervals, or when acquisition conditions are met.
  • the fixed processing unit 33 acquires temperature information, for example, during shooting.
  • the fixed processing unit 33 acquires temperature information corresponding to the location information of the fixed terminal 3, for example, from a server not shown.
  • the fixed processing unit 33 acquires temperature information, for example, from a temperature sensor installed on a mobile object.
  • the fixed processing unit 33 acquires temperature information, for example, constantly, at predetermined intervals, or when acquisition conditions are met.
  • the fixed processing unit 33 judges whether the acquired fixed attribute value set matches the preservation condition.
  • the fixed processing unit 33 judges whether the acquired time-series fixed attribute value set matches the preservation condition.
  • the fixed processing unit 33 for example, if the preservation condition is met, acquires a video corresponding to the fixed attribute value set.
  • the fixed processing unit 33 for example, if the preservation condition is met, acquires a video corresponding to the preservation condition.
  • the fixed processing unit 33 for example, if the preservation condition is met, acquires acquisition information that pairs with the preservation condition, and acquires a video according to the acquisition information.
  • the fixed processing unit 33 is equipped with, for example, a microphone, acquires sound information, and stores the sound information in association with the fixed video acquired by the fixed camera 331.
  • the fixed processing unit 33 analyzes a fixed video and acquires one or more tags for the fixed video. Note that the process of acquiring such tags is the same as the process performed by the tag acquisition unit 232 described above.
  • Fixed camera 331 shoots and acquires video. Such video is fixed video.
  • Fixed camera 331 is a camera that does not move.
  • Fixed camera 331 is a camera that is installed in a fixed position.
  • Fixed camera 331 is a camera whose shooting position is fixed. Having a fixed shooting position does not necessarily mean that the shooting direction is also fixed, and includes the possibility that the shooting direction changes.
  • the fixed transmission unit 34 transmits various types of information to the information processing device 1.
  • the various types of information are, for example, fixed moving images, fixed attribute value sets, and sound information.
  • the user storage unit 41 that constitutes the user terminal 4 stores various types of information.
  • the various types of information include, for example, a user identifier and videos.
  • the user reception unit 42 receives various instructions and information. Examples of the instructions and information include inquiries and purchase instructions.
  • a purchase instruction is an instruction to purchase a video.
  • the purchase instruction is associated with a user identifier.
  • the purchase instruction usually includes information that identifies the video.
  • the purchase instruction has, for example, a video identifier.
  • the purchase instruction includes, for example, an inquiry.
  • the purchase instruction includes, for example, purchase conditions.
  • the purchase conditions are, for example, the purchase amount.
  • the purchase conditions include, for example, information that identifies the rights period.
  • the means for inputting various instructions and information can be anything, such as a touch panel, keyboard, mouse, or menu screen.
  • the user processing unit 43 performs various types of processing.
  • the various types of processing are processing for creating a data structure for transmitting various instructions and information received by the user receiving unit 42.
  • the various types of processing are processing for creating a structure for transmitting information received by the user receiving unit 45.
  • the user transmission unit 44 transmits various instructions and information to the information processing device 1.
  • the various instructions and information are, for example, inquiries, purchase instructions, location information received from the target terminal 5, and route information received from the navigation terminal 6.
  • the user receiving unit 45 receives various information and instructions.
  • the various information and instructions include, for example, inquiries, videos, and status information.
  • the user output unit 46 outputs various types of information.
  • the various types of information include, for example, video and status information.
  • output is a concept that includes displaying on a display, projecting using a projector, printing on a printer, outputting sound, sending to an external device, storing on a recording medium, and passing on the processing results to other processing devices or other programs, etc.
  • the target terminal 5 acquires and transmits location information that identifies the location of the target terminal 5.
  • the target terminal 5 transmits the location information to, for example, the information processing device 1 or the user terminal 4. It is preferable for the target terminal 5 to transmit the location information in pair with a target identifier.
  • the target identifier is information that identifies the subject of monitoring.
  • the target identifier is, for example, an ID, name, telephone number, email address, or MAC address of the target terminal 5.
  • the target terminal 5 acquires the location information by, for example, a GPS receiver, but the method of acquiring the location information is not important.
  • the navigation terminal 6 has the functions of a known navigation terminal.
  • the navigation terminal 6 accepts input of a destination, acquires the current position, and acquires route information from the current position to the destination.
  • the navigation terminal 6, for example, transmits the accepted destination.
  • the navigation terminal 6, for example, transmits the acquired route information.
  • the destination or route information is transmitted to, for example, the information processing device 1 or the user terminal 4.
  • the navigation terminal 6 may receive a destination from the user terminal 4, and acquire route information from the current position to the destination.
  • the storage unit 11, mobile terminal management unit 111, fixed terminal management unit 112, mobile storage unit 21, fixed storage unit 31, and user storage unit 41 are preferably non-volatile recording media, but can also be realized with volatile recording media.
  • information may be stored in the storage unit 11, etc. via a recording medium, information transmitted via a communication line, etc. may be stored in the storage unit 11, etc., or information inputted via an input device may be stored in the storage unit 11, etc.
  • the receiver 12, the mobile information receiver 121, the inquiry receiver 122, the collective receiver 123, the mobile receiver 22, and the user receiver 45 are typically implemented using wireless or wired communication means, but may also be implemented using means for receiving broadcasts.
  • the processing procedure of the processing unit 13, etc. is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may also be realized by hardware (dedicated circuit).
  • the processor is a CPU, MPU, GPU, etc., and the type does not matter.
  • It includes a transmission unit 14, a video transmission unit 141, a status transmission unit 142, and a needs transmission unit 143.
  • the mobile transmission unit 24, the mobile information transmission unit 241, the mobile video transmission unit 242, the collective transmission unit 243, the fixed transmission unit 34, and the user transmission unit 44 are typically realized by wireless or wired communication means, but may also be realized by broadcasting means.
  • the photographing unit 231 is realized by a camera.
  • the fixed camera 331 is a fixed camera.
  • the camera may be, for example, a CCD camera, a CMOS camera, a 3D camera, a LiDAR, or an omnidirectional camera, but the type is not important.
  • the user reception unit 42 can be realized by a device driver for an input means such as a touch panel or keyboard, or control software for a menu screen, etc.
  • the user output unit 46 may or may not include an output device such as a display, a speaker, etc.
  • the user output unit 46 may be realized by driver software for an output device, or a combination of driver software for an output device and the output device, etc. ⁇ Operation>
  • the movement information receiving unit 121 determines whether movement information associated with the rights holder identifier has been received from the mobile terminal 2. If movement information has been received, the process proceeds to S502; if not, the process proceeds to S503.
  • the movement information receiving unit 121 associates the movement information received in S501 with the rights holder identifier and stores it in the storage unit 11. Return to S501. Note that if the movement information is a rights holder identifier, the movement information receiving unit 121 stores the rights holder identifier.
  • the set receiving unit 123 determines whether or not a set of mobile attribute values corresponding to the right holder identifier has been received from the mobile terminal 2. If a set of mobile attribute values has been received, the process proceeds to S504; if not, the process proceeds to S506.
  • the set accumulation unit 132 accumulates the set of moved attribute values received in S503 in the set storage unit 111, in association with the rights holder identifier.
  • the processing unit 13 deletes the movement information corresponding to the rights holder identifier from the storage unit 11. Return to S501. Note that deleting the movement information is equivalent to "rewriting the movement information to information indicating that the video is not in a state where it can be transmitted.”
  • the receiving unit 12 determines whether or not a video or the like associated with the rights holder identifier has been received from the mobile terminal 2 or the fixed terminal 3. If a video or the like has been received, the process proceeds to S507; if not, the process proceeds to S510.
  • the processing unit 13 performs processing to acquire an attribute value set corresponding to the video received in S506.
  • An example of such an attribute value set acquisition processing will be described with reference to the flowchart in FIG. 6.
  • the attribute value set acquisition processing here may also be processing to acquire an attribute value set associated with the received video.
  • the video here is a moving video or a fixed video.
  • the attribute value set acquired here is a moving attribute value set or a fixed attribute value set.
  • the first maintenance means 1361 stores the video received in S506 in association with the attribute value set acquired in S507. Note that the first maintenance means 1361 stores the video in the storage unit 11, for example, but may store the video in another device, such as a device constituting a blockchain. The process in which the first maintenance means 1361 stores the video is the first maintenance process.
  • the fourth maintenance means 1364 performs the fourth maintenance process. Return to S501. An example of the fourth maintenance process will be explained using the flowchart in FIG. 9.
  • the inquiry receiving unit 122 determines whether or not an inquiry has been received. If an inquiry has been received, the process proceeds to S511, and if not, the process proceeds to S519.
  • the inquiry receiving unit 122 receives an inquiry from, for example, the user terminal 4 or the navigation terminal 6.
  • the navigation terminal 6 that sends the inquiry may be the user terminal 4.
  • the video acquisition unit 133 determines whether the received query is a real-time video search. If it is a real-time video search, the process proceeds to S512, and if it is not a real-time video search, the process proceeds to S515. This determination can be made by inspecting the received query.
  • a real-time video search is a process in which the mobile terminal 2 or fixed terminal 3 searches for videos that it is currently holding or is currently being shot.
  • the video acquisition unit 133 determines whether the received query requires spatial fusion of multiple videos. If spatial fusion is required, the process proceeds to S513. If spatial fusion is not required, the process proceeds to S514. This determination can be made by inspecting the received query.
  • Video fusion processing is a process of spatially fusing multiple videos to acquire and transmit a video having one or multiple frames. An example of the video fusion processing will be described using the flowchart in FIG. 10. Note that the video to be transmitted may be a stereoscopic three-dimensional video.
  • the video acquisition unit 133, video composition unit 134, video processing unit 135, etc. perform video combination processing. Go to S518.
  • the video combination processing is a process for transmitting a single video in which multiple videos are combined in time. An example of the video combination processing will be explained using the flowchart in FIG. 12.
  • the video acquisition unit 133 and the like perform a registered video search process.
  • An example of the registered video search process will be described with reference to the flowchart in FIG. 13. Note that there may be cases where the video acquisition unit 133 is unable to acquire a video.
  • the video acquisition unit 133 determines whether or not the video was acquired in S515. If the video was acquired, the process proceeds to S517; if the video was not acquired, the process proceeds to S518.
  • the video acquisition unit 133 performs unregistered video search processing.
  • An example of the unregistered video search processing will be described using the flowcharts in Figures 14 and 15. Note that there may be cases where the video acquisition unit 133 is unable to acquire a video.
  • the rights holder processing unit 136 performs preservation processing. Return to S501. An example of preservation processing will be explained using the flowchart in FIG. 16.
  • the receiving unit 12 determines whether or not a purchase instruction has been received from the user terminal 4. If a purchase instruction has been received, the process proceeds to S520; if not, the process returns to S501.
  • the video acquisition unit 133 acquires a video corresponding to the purchase instruction. Note that the video acquisition unit 133 receives the video corresponding to the purchase instruction from the mobile terminal 2 or the fixed terminal 3, for example. Also, the video acquisition unit 133 acquires the video corresponding to the purchase instruction from registered videos, for example. The video acquisition unit 133 may acquire the video corresponding to the purchase instruction from unregistered videos, for example.
  • the fourth security means 1364 acquires a user identifier corresponding to the user terminal 4 that sent the purchase instruction. This user identifier becomes the rights holder identifier for the purchased video.
  • the fourth security means 1364 performs the fourth security process using the user identifier acquired in S516.
  • An example of the fourth security process is described using the flowchart in FIG. 9.
  • the compensation means 1365 performs compensation processing for the original rights holder of the video being purchased. Return to S501. An example of compensation processing will be explained using the flowchart in FIG. 17.
  • processing ends when the power is turned off or an interrupt occurs to end processing.
  • the receiver 12 receives from each of one or more mobile terminals 2 location information that identifies the current location of the mobile terminal 2 in pairs with the identifier of the mobile terminal 2 (e.g., a rights holder identifier) and temporarily stores the information.
  • the frequency or time interval at which the receiver 12 receives the location information from the mobile terminal 2 and temporarily stores it is not important.
  • the receiving unit 12 may receive location information, for example, continuously, from the user terminal 4, the target terminal 5, or the navigation terminal 6.
  • the information processing device 1 may, for example, use the received location information to receive a video acquired by the mobile terminal 2 or a video acquired by the fixed terminal 3, and transmit it to the user terminal 4 or the navigation terminal 6. Note that the frequency or time interval at which the location information is continuously received does not matter.
  • the receiving unit 12 may receive the fixed attribute value set and the fixed video from the fixed terminal 3, paired with the identifier of the fixed terminal 3 (e.g., a rights holder identifier).
  • the processing unit 13 stores the fixed attribute value set and the fixed video in the fixed terminal management unit 112, paired with the identifier of the fixed terminal 3.
  • the movement processing unit 23 or the collection storage unit 132 or the fixed processing unit 33 acquires one or more pieces of environmental information etc. associated with the target video.
  • the target video is a video for which a video attribute value is to be acquired, such as a received video, a shot video, or a video being shot.
  • the target video is a moving video or a fixed video.
  • the environmental information etc. may be only environmental information, or may be environmental information and one or more moving object attribute values.
  • the movement processing unit 23 etc. acquires the rights holder identifier associated with the received video.
  • the movement processing unit 23 etc. judges whether or not to perform annotation processing on the video. If annotation processing is to be performed, the process proceeds to S604, and if not, the process proceeds to S611. It is assumed that whether or not annotation processing is to be performed is determined in advance. It is preferable to annotate the video on either the information processing device 1 or the terminal.
  • the terminal here is the mobile terminal 2 or the fixed terminal 3.
  • an annotation unit is the target for which annotations are obtained, and is, for example, one frame, a certain number of frames, or a video of a certain duration (for example, 5 seconds, 10 seconds).
  • the movement processing unit 23 etc. judges whether or not to use the moving object attribute value during annotation. If the moving object attribute value is to be used, the process proceeds to S607, and if not, the process proceeds to S608. Note that it is assumed that whether or not to use the moving object attribute value during annotation is determined in advance. Furthermore, the movement processing unit 23 etc. can use the moving object attribute value only when the target video is a moving video.
  • the movement processing unit 23 and the like perform a moving object attribute value tag acquisition process.
  • An example of the moving object attribute value tag acquisition process is described with reference to the flowchart in FIG. 7.
  • the movement processing unit 23 etc. judges whether or not to use video during annotation. If video is to be used, the process proceeds to S609, and if not, the process proceeds to S610. It is assumed that it is predetermined whether or not video is to be used during annotation.
  • the movement processing unit 23 and the like perform a video tag acquisition process.
  • An example of the video tag acquisition process is described using the flowchart in FIG. 8.
  • the movement processing unit 23 etc. constructs an attribute value set having one or more pieces of environmental information, one or more tags, and a rights holder identifier. It returns to the upper level process.
  • the video may be divided into multiple inspection units, and steps S601 to S611 may be performed for each inspection unit.
  • the movement processing unit 23 etc. judges whether the i-th attribute value tag condition exists. If the i-th attribute value tag condition exists, the process proceeds to S703, and if it does not exist, the process returns to the upper process.
  • the movement processing unit 23 etc. acquires one or more moving object attribute values to be used in determining the i-th attribute value tag condition.
  • the movement processing unit 23 etc. judges whether or not one or more moving object attribute values acquired in S704 satisfy the i-th attribute value tag condition. If the attribute value tag condition is satisfied, the process proceeds to S706, and if not, the process proceeds to S707.
  • the movement processing unit 23 etc. obtains a tag that pairs with the i-th attribute value tag condition, and associates the tag with the corresponding video.
  • Associating a tag with a video usually means associating the tag with a field that constitutes the video.
  • the movement processing unit 23 etc. judges whether the i-th video tag condition exists. If the i-th video tag condition exists, the process proceeds to S803, and if it does not exist, the process returns to the upper process.
  • the movement processing unit 23 etc. obtains the i-th video tag condition.
  • the movement processing unit 23 etc. acquires one or more video attribute values to be used to determine the i-th video tag condition.
  • the movement processing unit 23 etc. judges whether or not one or more video attribute values acquired in S804 satisfy the i-th attribute value tag condition. If the attribute value tag condition is satisfied, the process proceeds to S806, and if not, the process proceeds to S807.
  • the movement processing unit 23 etc. obtains a tag that pairs with the i-th attribute value tag condition, and associates the tag with the corresponding video.
  • Associating a tag with a video usually means associating the tag with one or more frames that pair with one or more video attribute values used to determine the attribute value tag condition.
  • the fourth security means 1364 acquires access information that identifies the storage destination of the video.
  • video may be, for example, a fusion video, a combined video, or a stereoscopic three-dimensional video.
  • the fourth preservation means 1364 acquires a set of attribute values corresponding to the stored video. If the stored video is a video composed of multiple original videos, the set of attribute values corresponding to the video is the set of attribute values for each of the multiple original videos.
  • the fourth security means 1364 constructs security information having the access information acquired in S901, the attribute value set acquired in S902, and the rights holder identifier of the video. Note that if a new rights holder identifier has been acquired, the fourth security means 1364 constructs security information including, for example, the new rights holder identifier and the original rights holder identifier. (S904) The fourth maintenance unit 1364 accumulates the maintenance information constructed in S903, and then returns to the upper level process.
  • preservation information for a video corresponding to the preservation information to be stored has already been stored, this preservation information is overwritten onto the preservation information constructed in S903. This makes it possible to manage, for example, the transition of rights holders of the video.
  • the fourth preservation means 1364 stores the preservation information in, for example, a blockchain.
  • the video acquisition unit 133 acquires reference location information.
  • the reference location information is, for example, location information included in the received inquiry.
  • the reference location information is, for example, location information of the user terminal 4 or the target terminal 5.
  • the reference location information is, for example, one piece of location information in the route information included in the inquiry.
  • the one piece of location information is the first piece of location information in the route information.
  • the location information is, for example, (latitude, longitude), but may also be a location name (for example, the name of a parking lot), a place name, etc.
  • the video acquisition unit 133 assigns 1 to counter i.
  • the video acquisition unit 133 determines whether or not there is an i-th terminal to which the video can be transmitted. If there is an i-th terminal, the process proceeds to S1004. If there is no i-th terminal, the process proceeds to S1008. Note that a terminal capable of transmitting video is usually a fixed terminal 3 or a mobile terminal 2 that is moving.
  • the mobile terminal 2 that is moving is the mobile terminal 2 that corresponds to the movement information stored in the storage unit 11.
  • the fixed terminal 3 is the fixed terminal 3 that corresponds to the fixed terminal information managed by the fixed terminal management unit 112. That is, the video acquisition unit 133 determines, for example, whether or not the i-th information exists among the movement information in the storage unit 11 or the fixed terminal information in the fixed terminal management unit 112.
  • the video acquisition unit 133 acquires the latest location information sent by the mobile terminal 2, which is the i-th terminal, or the location information included in the fixed terminal information, which is the i-th terminal.
  • the video acquisition unit 133 determines whether the location information acquired in S1004 satisfies the second location condition. If the second location condition is satisfied, the process proceeds to S1006; if the second location condition is not satisfied, the process proceeds to S1007.
  • the video acquisition unit 133 acquires an image from the i-th terminal, pairs the image with the rights holder identifier and location information of the i-th terminal, and temporarily stores the image in a buffer (not shown).
  • the video acquisition unit 133 sends, for example, a query including a type identifier that specifies the type of image requested (for example, "video”, “still image”, or “around view image”) to the i-th terminal, and receives an image corresponding to the query from the i-th mobile terminal 2.
  • a type identifier that specifies the type of image requested (for example, "video”, “still image”, or "around view image)
  • the video composition unit 134 calculates the difference (distance) between the position information for each image temporarily stored in a buffer (not shown) and the reference position information.
  • the video composition unit 134 sorts the images in ascending order using the difference as a key.
  • the video composition unit 134 acquires a reference image.
  • the reference image is an image that corresponds to the reference location information.
  • the reference image is, for example, an image sent from a user terminal 4 that is located at the location of the reference location information. Note that this image is of a type identified by a type identifier that specifies the type of image being requested.
  • the video composition unit 134 assigns 1 to counter j.
  • the video composition unit 134 determines whether or not a jth other image exists among the images sorted in S1008. If a jth other image exists, the process proceeds to S1012; if not, the process proceeds to S1014.
  • the other images are the images sorted in S1008.
  • the video composition unit 134 determines whether or not a priority image that can be substituted for the jth other image exists. If a priority image that can be substituted exists, the process proceeds to S1013; if not, the process proceeds to S1014.
  • a replaceable priority image is an image that captures an area equivalent to the area of the jth other image, and is an image that pairs with location information of a position whose distance from the position indicated by the location information of the jth other image is within a threshold.
  • a priority image is an image of a video (e.g., a fixed video) that corresponds to a priority type. Note that equivalent areas are, for example, those in which the proportion of overlapping areas is equal to or greater than a threshold, and those in which the size of the overlapping areas is equal to or greater than a threshold.
  • the video composition unit 134 checks whether an image is a priority image, for example, among images paired with position information of a position whose distance from the position indicated by the position information of the jth other image is within a threshold, in order of shortest distance.
  • the video composition unit 134 obtains a replaceable priority image.
  • the video composition unit 134 performs image fusion processing to fuse the reference image acquired in S1009 with the jth other image or the priority image acquired in S1013.
  • An example of the image fusion processing will be described using the flowchart in FIG. 11.
  • the video composition unit 134 composes an image that indicates the position indicated by the reference position information from the fusion image finally constructed in S1014, and temporarily stores the image in a buffer (not shown). This image is the fusion image.
  • the video sending unit 141 sends the fusion image constructed in S1016. Note that the video sending unit 141 may also send the fusion image finally constructed in S1014.
  • the destination of the fusion image is usually the user terminal 4 that sent the inquiry.
  • the video transmission unit 141 determines whether or not to end the transmission of the fusion image. If the transmission of the fusion image is to be ended, the process proceeds to S1019; if not, the process returns to S1001.
  • the video sending unit 141 determines to end the sending of the fusion image.
  • the video composition unit 134 combines the multiple transmitted fusion images over time to compose a fusion video, which is temporarily stored in a buffer (not shown).
  • the video processing unit 135 performs 3D processing on the fusion video constructed in S1019.
  • the process returns to the upper level process.
  • the 3D processing is a process in which image processing is performed on videos that satisfy the conditions, and a stereoscopic 3D video is constructed. An example of the 3D processing is explained using the flowchart in FIG. 18.
  • the video transmission unit 141 transmitted the fusion image.
  • the video transmission unit 141 may transmit the video that has been subjected to the three-dimensional processing in S1020.
  • the video composition unit 134 acquires a base image. Note that the first base image is the reference image acquired in S1009. The base image is also the base image updated in S1105 in the loop immediately prior to S1101 to S1105.
  • the video composition unit 134 acquires area information that identifies the area covered by the base image acquired in S1101.
  • the area information is, for example, a collection of position information of vertices that make up a polygon, or a collection of position information of two diagonal corners that make up a rectangle.
  • the video composition unit 134 acquires the image to be used for image fusion.
  • the other image to be used for image fusion is the j-th other image in S1011 or the priority image in S1013.
  • the video composition unit 134 acquires area information that identifies the area covered by the image acquired in S1103.
  • the video composition unit 134 uses the area information of the base image and the area information acquired in S1104 to determine the area in the base image where the image acquired in S1103 will be placed, places the other image in that area, and acquires an updated base image. It then returns to the upper process.
  • the two images on which image fusion processing is performed each have an overlapping area. Then, the video composition unit 134 fuses the two images based on the overlapping area to compose a single image. This image is the updated base image. Furthermore, image fusion processing of two images with overlapping areas is possible using publicly known technology.
  • the video acquisition unit 133 acquires the location information of the target.
  • the location information of the target is usually the location information contained in the query.
  • the location information of the target is, for example, information that identifies a location close to the location indicated by the location information corresponding to the video to be acquired.
  • the video acquisition unit 133 assigns 1 to counter i.
  • the video acquisition unit 133 determines whether or not there is an i-th terminal to which video can be transmitted, starting from a terminal that corresponds to the priority type (e.g., fixed terminal 3) to a terminal that does not correspond to the priority type (e.g., mobile terminal 2). If the i-th terminal is present, the process proceeds to S1204, and if not, the process proceeds to S1214.
  • the terminals that can transmit video are usually fixed terminals 3 and mobile terminals 2 that are moving.
  • the video acquisition unit 133 acquires the location information of the i-th terminal. Note that if the i-th terminal is a fixed terminal 3, the video acquisition unit 133 normally acquires the location information contained in the i-th fixed terminal information of the fixed terminal management unit 112. If the i-th terminal is a mobile terminal 2, the video acquisition unit 133 acquires the latest location information sent by the i-th terminal.
  • the video acquisition unit 133 determines whether the location information of the i-th terminal satisfies the first location condition for the target location information. If the first location condition is satisfied, the process proceeds to S1206, and if the first location condition is not satisfied, the process proceeds to S1210.
  • the video acquisition unit 133 acquires the video from the i-th terminal, pairs the video with the rights holder identifier and location information of the i-th terminal, and temporarily stores the video in a buffer (not shown).
  • the video acquisition unit 133 may send a query including a type identifier (e.g., "video” or “still image”) that specifies the type of image requested to the i-th terminal, and receive an image corresponding to the query from the i-th terminal, for example.
  • a type identifier e.g., "video” or “still image”
  • the video transmission unit 141 transmits the video acquired in S1206.
  • the video transmission unit 141 typically transmits the video to the user terminal 4 that has transmitted the inquiry.
  • the video acquisition unit 133 acquires the target location information. Note that, for example, if the target terminal 5 that transmits the location information is moving, the target location information acquired here differs from the location information acquired in S1201. However, if the target terminal 5 is not moving, if an inquiry having a fixed set of location information is received, etc., the target location information acquired here is the same as the location information acquired in S1201.
  • the video acquisition unit 133 acquires the location information of the i-th terminal. Return to S1205. Note that if the i-th terminal is a mobile terminal 2, the location information of the i-th terminal here is updated location information. If the i-th terminal is a fixed terminal 3, the location information of the i-th terminal here does not change.
  • the video composition unit 134 determines whether or not a transmitted video exists for the video acquired by the i-th terminal. If a transmitted video exists, the process proceeds to S1211, and if a transmitted video does not exist, the process proceeds to S1213.
  • the video composition unit 134 composes a combined video by combining the video acquired by the i-th terminal with the video that has already been transmitted.
  • the video composition unit 134 also adds the rights holder identifier of the i-th terminal to the combined video, and temporarily stores the combined video in a buffer (not shown).
  • the combined video is the video acquired by the i-th terminal.
  • the video acquisition unit 133 determines whether or not to end the video combination process. If so, proceed to S1215; if not, proceed to S1216.
  • the video acquisition unit 133 determines to end the video combination process.
  • the video acquisition unit 133 acquires the location information of the next target. For example, the video acquisition unit 133 acquires the next location information included in the route information of the query. Return to S1203.
  • the video composition unit 134 links the multiple combined videos composed in S1211 in the order in which they were acquired to compose a combined video to be stored. Next, the video composition unit 134 pairs the combined video with the multiple rights holder identifiers and location information of the videos that were the basis of the combined video, and temporarily stores the combined video in a buffer (not shown).
  • the video construction unit 134 does not perform the process of combining the combined videos.
  • the video processing unit 135 performs a three-dimensional process on the combined video created in S1216. The process returns to the upper level process.
  • An example of the three-dimensional process is described using the flowchart in FIG. 18.
  • the video acquisition unit 133 assigns 1 to counter i.
  • the video acquisition unit 133 determines whether the i-th target location information is present in the received query. If the i-th location information is present, the process proceeds to S1303; if not, the process proceeds to S1317.
  • the target location information is location information used to search for videos.
  • the target location information is, for example, one piece of location information in the received query, or each of multiple pieces of location information included in the route information in the received query.
  • the video acquisition unit 133 acquires the i-th target location information from the received inquiry.
  • the video acquisition unit 133 acquires all the preservation information. For example, the video acquisition unit 133 acquires the preservation information that exists on the blockchain.
  • the video acquisition unit 133 assigns 1 to counter j.
  • the video acquisition unit 133 determines whether the jth preservation information is present among the preservation information acquired in S1304. If the jth preservation information is present, the process proceeds to S1307; if not, the process proceeds to S1316.
  • the video acquisition unit 133 checks the jth security information and determines whether the video corresponding to the jth security information can be provided to a third party. If it can be provided, the process proceeds to S1308, and if it cannot be provided, the process proceeds to S1315. Note that being available means, for example, that it can be viewed, that it can be sold, or that it can be viewed and sold.
  • the video acquisition unit 133 assigns 1 to counter k.
  • the video acquisition unit 133 determines whether the kth search unit exists in the jth conservation information or the video corresponding to the jth conservation information. If the kth search unit exists, the process proceeds to S1310; if not, the process proceeds to S1315.
  • a search unit is a quantity of video that determines whether or not a query is satisfied.
  • Search unit videos are, for example, video from the start to the end of filming, video for a certain period of time, video until a predetermined event occurs (for example, video where the speed, which is a mobile object attribute value, goes from 0 to the next speed 0, video from leaving a residential location (for example, a parking lot at home) and returning to the residential location), and video of a specified period before and after a predetermined tag (for example, "accident” or "traffic jam") is attached.
  • the search unit is not important.
  • the video acquisition unit 133 acquires a set of video attribute values for the kth search unit from the jth conservation information.
  • the video acquisition unit 133 determines whether or not the set of video attribute values of the kth search unit for the ith target location information satisfies the first location condition. If the first location condition is satisfied, the process proceeds to S1312; if not, the process proceeds to S1314.
  • the video acquisition unit 133 acquires the video etc. corresponding to the j-th piece of security information and temporarily stores it in a buffer (not shown).
  • the video etc. is, for example, a collection of videos and video attribute values, or a video.
  • the video composition unit 134 determines whether or not multiple videos, etc. were acquired in S1312. If multiple videos, etc. were acquired, the process proceeds to S1318; if only one video, etc. was acquired, the process proceeds to S1319.
  • the video composition unit 134 combines the multiple videos stored in a buffer (not shown) in the order in which they were stored, composes a combined video, and stores the combined video in a buffer (not shown). Note that it is preferable for the video composition unit 134 to also associate the rights holder identifier of the videos that are the basis of the combined video and one or more video attribute values with the combined video and store them in a buffer (not shown).
  • the video sending unit 141 sends the combined video or the acquired video. It then returns to higher-level processing. It is preferable that such a video be associated with a rights holder identifier and one or more video attribute values.
  • the video transmission unit 141 may transmit the video. In this case, the process of S1319 is not necessary.
  • the video sending unit 141 typically sends the combined video or video to the user terminal 4 that sent the inquiry.
  • the first example of the unregistered video search process is a case in which the attribute value sets of the mobile terminal management unit 111 and the fixed terminal management unit 112 are used.
  • the video acquisition unit 133 assigns 1 to counter i.
  • the video acquisition unit 133 determines whether the i-th target location information is present in the received query. If the i-th location information is present, the process proceeds to S1403; if not, the process proceeds to S1420.
  • the video acquisition unit 133 acquires the i-th target location information from the received inquiry.
  • the video acquisition unit 133 assigns 1 to counter j.
  • the video acquisition unit 133 determines whether or not the jth attribute value set exists in the fixed attribute value set of the fixed terminal management unit 112 or the mobile attribute value set of the mobile terminal management unit 111. If the jth attribute value set exists, the process proceeds to S1406; if it does not exist, the process proceeds to S1419. Note that here, it is preferable for the video acquisition unit 133 to check from the fixed attribute value set of the fixed terminal management unit 112.
  • the video acquisition unit 133 assigns 1 to counter k.
  • the video acquisition unit 133 determines whether or not the kth search unit corresponding to the jth attribute value set exists. If the kth search unit exists, the process proceeds to S1408; if not, the process proceeds to S1414.
  • the video acquisition unit 133 acquires a set of video attribute values of the kth search unit in the jth attribute value set from the set storage unit 111. Note that there may be only one search unit in the attribute value set.
  • the video acquisition unit 133 determines whether the set of video attribute values acquired in S1408 satisfies the query. If it does, the process proceeds to S1410; if it does not, the process proceeds to S1414.
  • the video acquisition unit 133 acquires the rights holder identifier corresponding to the video.
  • the video acquisition unit 133 determines whether or not it is possible to acquire the video corresponding to the rights holder identifier acquired in S1410. If it is possible to acquire the video, the process proceeds to S1412; if it is not possible to acquire the video, the process proceeds to S1415.
  • video can be acquired when the terminal corresponding to the jth attribute value set is a fixed terminal 3, or when the terminal corresponding to the jth attribute value set is a mobile terminal 2 and movement information for that terminal exists (for example, when the power of the mobile terminal 2 is ON).
  • the video acquisition unit 133 acquires the video corresponding to the kth search unit from the terminal corresponding to the rights holder identifier acquired in S1410, and temporarily stores it in a buffer (not shown). Go to S1419.
  • the video acquisition unit 133 sends a request for a video corresponding to the kth search unit to the terminal, and receives the video corresponding to the request from the terminal.
  • the range of videos corresponding to the kth search unit does not matter.
  • the videos corresponding to the kth search unit may be all videos stored in the terminal, or only a portion of the videos.
  • the video acquisition unit 133 acquires status information and the like.
  • the status information and the like includes, for example, status information indicating that the terminal is a mobile terminal 2 and that its power supply is OFF.
  • the status transmission unit 142 transmits the status information etc. acquired in S1415 to the user who sent the inquiry.
  • the video acquisition unit 133 acquires needs information.
  • the needs transmission unit 143 transmits the needs information acquired in S1417 to the rights holder of the video. Return to S1414.
  • the video composition unit 134 determines whether or not multiple videos were acquired in S1412. If multiple videos were acquired, the process proceeds to S1421; if only one video was acquired, the process proceeds to S1422.
  • the video composition unit 134 combines the multiple videos stored in a buffer (not shown) in the order they were stored in S1412 to create a single combined video, which it then stores in a buffer (not shown).
  • the video sending unit 141 sends the combined video or a video. Return to upper-level processing. Note that the video sending unit 141 normally sends the combined video or a video to the user terminal 4 that sent the inquiry.
  • the second example of the unregistered video search process is a case where an inquiry is made to the fixed terminal 3 or the mobile terminal 2.
  • the video acquisition unit 133 assigns 1 to counter i.
  • the video acquisition unit 133 determines whether the i-th target location information is present in the received query. If the i-th location information is present, the process proceeds to S1503; if not, the process proceeds to S1512.
  • the video acquisition unit 133 acquires the i-th target location information from the received inquiry.
  • the video acquisition unit 133 assigns 1 to counter j.
  • the video acquisition unit 133 determines whether or not a jth terminal to which a query can be sent exists. If a jth terminal exists, the process proceeds to S1503; if a jth terminal does not exist, the process proceeds to S1511.
  • terminals that can send inquiries are terminals that are in a state where they can send videos, and are fixed terminals 3 or mobile terminals 2 whose movement information exists in the storage unit 11.
  • the video acquisition unit 133 selects as the jth terminal only a fixed terminal 3 that is installed at a position indicated by location information that matches the first location condition for the i-th target location information.
  • the video acquisition unit 133 searches for a terminal that transmits a query from among terminals that correspond to the priority type (usually a fixed terminal 3), and if there is no terminal that corresponds to the priority type that can transmit a query, it searches for a terminal that does not correspond to the priority type (usually a mobile terminal 2).
  • the video acquisition unit 133 transmits a query to the jth terminal.
  • the video acquisition unit 133 preferably transmits the query from a terminal corresponding to the priority type (usually, the fixed terminal 3), and if there is no terminal that can respond to the query, transmits the query to a terminal that is not of the priority type (usually, the mobile terminal 2).
  • the video acquisition unit 133 determines whether or not information has been received from the jth terminal. If information has been received, the process proceeds to S1508; if not, the process returns to S1507.
  • the video acquisition unit 133 determines whether or not a video is included in the information received in S1507. If a video is included, the process proceeds to S1509; if not, the process proceeds to S1510.
  • the video acquisition unit 133 temporarily stores the video etc. received in S1507 in a buffer (not shown). Then, the process proceeds to S1511.
  • the video etc. is, for example, a video, a set of attribute values, and a rights holder identifier.
  • the video composition unit 134 determines whether or not multiple videos were stored in S1509. If multiple videos were stored, the process proceeds to S1513; if only one video was acquired, the process proceeds to S1514.
  • the video composition unit 134 combines the multiple videos stored in a buffer (not shown) in the order they were stored in S1509 to create a single combined video, which it then stores in a buffer (not shown).
  • the video sending unit 141 sends the combined video or a video. Return to upper-level processing. Note that the video sending unit 141 normally sends the combined video or a video to the user terminal 4 that sent the inquiry.
  • the rights holder processing unit 136 stores the transmitted video in association with a set of attribute values corresponding to one or more videos that were the basis for the transmitted video.
  • the rights holder processing unit 136 stores videos in pairs with rights holder identifiers that identify one or more rights holders of the videos.
  • the rights holder identifiers here are, for example, one or more rights holder identifiers of the videos that are the basis of the stored videos.
  • the rights holder identifiers here are, for example, a single rights holder identifier that identifies the user who made the inquiry.
  • the rights holder processing unit 136 also stores the video in, for example, the storage unit 11 or a device other than the information processing device 1.
  • the device other than the information processing device 1 may be a device that constitutes a blockchain.
  • the fourth maintenance means 1364 performs the fourth maintenance process.
  • An example of the fourth maintenance process was explained using the flowchart in FIG. 9.
  • the rights holder processing unit 136 determines whether the i-th video that was the basis for the stored video exists. If the i-th video exists, the process proceeds to S1605; if not, the process returns to the upper level process.
  • the compensation means 1365 performs compensation processing.
  • An example of the compensation processing will be described with reference to the flowchart in FIG. 17.
  • the compensation processing here is a process of paying compensation to the rights holder of the i-th video that was the basis for the accumulated video.
  • the rights holder processing unit 136 determines whether or not to change the rights holder of the i-th video that was the basis of the stored video. If the rights holder is to be changed, the process proceeds to S1607; if the rights holder is not to be changed, the process proceeds to S1608.
  • the decision as to whether or not to change the rights holder may be based on a flag associated with the i-th video, may be predetermined, or the rights holder may be changed if the inquiry includes "information indicating a request to change the rights holder.”
  • the rights holder processing unit 136 acquires the user identifier of the user terminal 4. This user identifier becomes the new rights holder identifier.
  • the rights holder processing unit 136 determines whether the i-th video that was the source of the stored video has already been registered. If it has already been registered, the process proceeds to S1611. If it has not yet been registered, the process proceeds to S1609.
  • the first preservation means 1361 stores the i-th video that was the source of the stored video.
  • the fourth preservation means 1364 performs a fourth preservation process on the i-th video that was the source of the stored videos.
  • An example of the fourth preservation process was explained using the flowchart in FIG. 9.
  • the compensation means 1365 acquires one or more rights holder identifiers for the target video.
  • the compensation means 1365 may also acquire the rights holder identifiers of the previous rights holders for the target video.
  • the reward means 1365 obtains a set of attribute values for the target video.
  • the reward means 1365 acquires a service identifier that identifies the service provided for the target video.
  • the service identifier is, for example, "view” or "purchase.”
  • the reward means 1365 obtains the reward amount using one or more pieces of information from the attribute value set obtained in S1702 and the service identifier obtained in S1703.
  • the compensation means 1365 acquires a compensation amount for each rights holder identifier. If rights holder history information including multiple rights holder identifiers has been acquired, the compensation means 1365 may acquire a compensation amount for each rights holder identifier.
  • the compensation means 1365 acquires video attribute values corresponding to each of the multiple videos that were the basis for the video transmitted by the video transmission unit 141, and uses the video attribute values to determine the amount of compensation for each of the multiple rights holders. It is preferable that the compensation means 1365, for example, determines a larger amount of compensation the greater the amount of data, video duration, or number of frames of the original video used in the video transmitted by the video transmission unit 141. It is preferable that the compensation means 1365, for example, determines a larger amount of compensation the greater the resolution of the original video used in the video transmitted by the video transmission unit 141.
  • the reward means 1365 performs processing to give a reward to the rights holder identified by the rights holder identifier obtained in S1701, in the amount of reward obtained in S1704.
  • the compensation means 1365 performs a process of charging the user who has enjoyed the service related to the target video with a compensation. The process returns to the upper level process.
  • the target video is usually a video transmitted to the user terminal 4.
  • the profits earned by the operator of the information processing device 1 may be acquired and accumulated.
  • the video processing unit 135 assigns 1 to counter i.
  • the video processing unit 135 determines whether the i-th inspection unit exists in the target video. If the i-th inspection unit exists, the process proceeds to S1803, and if the i-th inspection unit does not exist, the process returns to the upper process.
  • the inspection unit here refers to the video to be inspected in order to determine at once whether or not the shot video to be processed should be made into a stereoscopic three-dimensional video.
  • the inspection unit may be, for example, a predetermined number of frames, a predetermined period of time, each video that is the source of the shot video to be processed, or the entire video to be processed.
  • the shot video to be processed may be, for example, a fused video or a combined video.
  • the video processing unit 135 acquires one or more video attribute values that are used to determine the processing conditions and that are paired with the video of the i-th inspection unit.
  • the one or more video attribute values may be, for example, tags, time information, weather information, temperature information, and season information.
  • the video processing unit 135 determines whether or not one or more video attribute values acquired in S1803 satisfy the processing conditions. If the processing conditions are satisfied, the process proceeds to S1805; if the processing conditions are not satisfied, the process proceeds to S1811.
  • the video processing unit 135 assigns 1 to counter j.
  • the video processing unit 135 determines whether or not the jth frame corresponding to the ith inspection unit exists. If the jth frame exists, the process proceeds to S1807; if the jth frame does not exist, the process proceeds to S1811.
  • the frame corresponding to the i-th inspection unit is usually a frame included in the i-th inspection unit, but it may also be a frame that precedes or follows the i-th inspection unit.
  • the frames to be made into the stereoscopic 3D video may include frames before and after the accident occurs.
  • the video processing unit 135 obtains the jth frame corresponding to the ith inspection unit.
  • the video processing unit 135 obtains a stereoscopic 3D image from the frames obtained in S1807.
  • the video processing unit 135 replaces the jth frame with the stereoscopic 3D image acquired in S1808.
  • the movement processing unit 23 judges whether or not the start of movement has been detected. If the start of movement has been detected, the process proceeds to S1902, and if not, the process proceeds to S1917. Note that the start of movement is determined, for example, by the engine of the moving object on which the mobile terminal 2 is installed being turned on, or the power supply of the mobile terminal 2 being turned on.
  • the movement information acquisition unit 233 acquires movement information.
  • the movement information transmission unit 241 transmits the movement information to the information processing device 1. Note that the movement information is stored in, for example, the movement storage unit 21.
  • the image capture unit 231 starts capturing images.
  • the image capture unit 231 acquires the video.
  • the movement processing unit 23 acquires one or more video attribute values and associates the one or more video attribute values with the video acquired in S1904.
  • the movement processing unit 23 acquires, for example, location information, time information, weather information, temperature information, and season information.
  • the movement processing unit 23 also acquires, for example, one or more moving object attribute values (for example, CAN data).
  • the tag acquisition unit 232 determines whether or not to acquire a moving object attribute value tag. If a moving object attribute value tag is to be acquired, the process proceeds to S1907; if not, the process proceeds to S1909. Note that whether or not to acquire a moving object attribute value tag is determined in advance.
  • the tag acquisition unit 232 performs a mobile object attribute value tag acquisition process.
  • An example of the mobile object attribute value tag acquisition process was explained using the flowchart in FIG. 7. Note that there may be cases where the tag cannot be acquired at this point.
  • the tag acquisition unit 232 associates the one or more tags with the video acquired in S1904.
  • the tag acquisition unit 232 associates one or more tags with a video and stores them in the mobile storage unit 21. This allows the user of the mobile terminal 2 to search for videos, for example, using the tags as a key.
  • the tag acquisition unit 232 determines whether or not to acquire a video tag. If a video tag is to be acquired, the process proceeds to S1910; if not, the process proceeds to S1912. Note that whether or not a video tag is to be acquired is determined in advance.
  • the tag acquisition unit 232 performs video tag acquisition processing.
  • An example of the video tag acquisition processing has been described using the flowchart in FIG. 8. Note that there may be cases where the tag cannot be acquired at this point.
  • the tag acquisition unit 232 associates the one or more tags with the video acquired in S1904.
  • the tag acquisition unit 232 associates one or more tags with a video and stores them in the mobile storage unit 21. This allows the user of the mobile terminal 2 to search for videos, for example, using the tags as a key.
  • the movement processing unit 23 determines whether or not the end of the movement has been detected. If the end of the movement has been detected, the process proceeds to S1913; if not, the process returns to S1904.
  • the movement processing unit 23 detects the end of movement by, for example, detecting that the engine is turned off, that the power of the mobile terminal 2 is turned off, or by detecting arrival at the destination.
  • the movement processing unit 23 obtains the rights holder identifier from the movement storage unit 21.
  • the movement processing unit 23 constructs an attribute value set having one or more video attribute values acquired in S1905, one or more moving object attribute values and tags acquired in S1907, and one or more tags acquired in S1910. Note that the attribute value set does not have any information that was not acquired above.
  • the set sending unit 243 associates the attribute value set constructed in S1914 with the rights holder identifier and sends it to the information processing device 1.
  • the moving video transmission unit 242 etc. performs video transmission processing. Return to S1901. An example of the video transmission processing will be described with reference to the flowchart in FIG. 20.
  • the video transmission processing is processing for automatically transmitting videos that meet the preservation conditions.
  • the mobile receiving unit 22 determines whether or not an inquiry has been received from the information processing device 1. If an inquiry has been received, the process proceeds to S1918; if not, the process returns to S1901.
  • the movement processing unit 23 performs a terminal image acquisition process corresponding to the inquiry received in S1917.
  • An example of the terminal image acquisition process is described using the flowchart in FIG. 21.
  • the terminal image acquisition process is a process of acquiring a video that matches the inquiry from among the videos stored in the movement storage unit 21.
  • the movement processing unit 23 determines whether or not an image was acquired in S1918. If an image was acquired, the process proceeds to S1920; if not, the process proceeds to S1921.
  • the moving video transmission unit 242 transmits images, etc. to the information processing device 1.
  • images, etc. are, for example, an image, a set of attribute values, and a rights holder identifier.
  • images are, for example, videos, still images, and around-view images.
  • the moving video transmission unit 242 transmits error information.
  • the error information is information indicating that the video could not be acquired.
  • a set of video attribute values which are all or a part of one or more video attribute values acquired in S1905, the moving object attribute values and tags acquired in S1907, and the tags acquired in S1910, to the information processing device 1 each time a video is acquired, at predetermined intervals, or when a predetermined condition is met (for example, the tag "accident” or "traffic jam” is acquired).
  • the video transmission process (processing of S1916) is performed after the end of movement is detected, but the video transmission process may be performed continuously.
  • Performing the video transmission process continuously means, for example, performing the video transmission process (S1916) immediately after S1911. In this way, even if the storage area for videos in the storage unit 11 is small, videos that meet the preservation conditions can be preserved.
  • processing ends when the power is turned off or an interrupt occurs to end processing.
  • the movement processing unit 23 acquires a set of attribute values from the movement storage unit 21.
  • the movement processing unit 23 determines whether the i-th search unit exists. If the i-th search unit exists, the process proceeds to S2004; if it does not exist, the process returns to the upper process.
  • the movement processing unit 23 obtains a set of video attribute values for the i-th search unit.
  • the movement processing unit 23 assigns 1 to counter j.
  • the movement processing unit 23 determines whether the jth conservation condition exists. If the jth conservation condition exists, the process proceeds to S2007; if it does not exist, the process proceeds to S2010.
  • the movement processing unit 23 determines whether the set of video attribute values acquired in S2004 satisfies the jth conservation condition. If the jth conservation condition is satisfied, the process proceeds to S2008; if not, the process proceeds to S2011.
  • the movement processing unit 23 acquires the video etc. of the i-th search unit.
  • the video etc. of the i-th search unit is, for example, the video of the i-th search unit and a set of attribute values corresponding to the video.
  • the video of the i-th search unit may include, for example, the video before the i-th search unit and the video after the i-th search unit.
  • the moving video transmission unit 242 transmits the video etc. acquired in S2008 to the information processing device 1.
  • the movement processing unit 23 uses the received inquiry to determine whether or not to acquire a video. If a video is to be acquired, the process proceeds to S2102; if a still image or the like is to be acquired, the process proceeds to S2108.
  • the movement processing unit 23 checks the type identifier (e.g., "video”, “still image”, “around view image”) included in the received inquiry and determines whether or not to acquire a video.
  • type identifier e.g., "video”, “still image”, “around view image
  • the movement processing unit 23 determines whether the i-th search unit exists. If the i-th search unit exists, the process proceeds to S2104; if it does not exist, the process returns to the upper process.
  • the movement processing unit 23 obtains a set of video attribute values for the i-th search unit.
  • the movement processing unit 23 determines whether the set of video attribute values of the i-th search unit matches the query. If it matches the query, the process proceeds to S2106; if it does not match, the process proceeds to S2107.
  • the movement processing unit 23 acquires the video etc. of the i-th search unit.
  • the video etc. is, for example, a collection of videos and video attribute values.
  • the movement processing unit 23 acquires the latest composite image or the latest still image. It then returns to the upper process.
  • a composite image is an image that is synthesized using still images captured by multiple cameras installed on a moving object.
  • the composite image is, for example, an around-view image.
  • a still image is an image captured by one camera installed on a moving object.
  • the technology for constructing an around-view image is publicly known, so a detailed description will be omitted.
  • the fixed camera 331 starts capturing images.
  • the trigger for starting capturing images is, for example, turning on the power or receiving an instruction from the user.
  • the fixed processing unit 33 acquires the video captured by the fixed camera 331 and temporarily stores it in a buffer (not shown).
  • the fixed processing unit 33 acquires one or more video attribute values.
  • the one or more video attribute values are, for example, time information, weather information, temperature information, and season information.
  • the fixed processing unit 33 associates the video acquired in S2202 with one or more video attribute values acquired in S2203.
  • the fixed processing unit 33 judges whether or not to acquire a video tag. If a video tag is to be acquired, the process proceeds to S2206. If a video tag is not to be acquired, the process proceeds to S2208. Note that whether or not a video tag is to be acquired is determined in advance.
  • the fixed processing unit 33 acquires a video tag. Note that an example of such video tag acquisition processing is the same as the processing in the flowchart in FIG. 8.
  • the fixed processing unit 33 associates the video acquired in S2202 with one or more tags acquired in S2206.
  • the fixed processing unit 33 judges whether or not to end the processing. If it is judged that the processing should be ended, the processing ends, and if it is judged that the processing should not be ended, the process proceeds to S2209. Note that the fixed processing unit 33 judges that the processing should be ended, for example, when the power is turned OFF or when an end instruction is received.
  • the fixed receiving unit 32 determines whether or not an inquiry has been received from the information processing device 1. If an inquiry has been received, the process proceeds to S2210; if not, the process proceeds to S2202.
  • the fixed processing unit 33 acquires a video corresponding to the inquiry.
  • An example of such a fixed video acquisition process is described using the flowchart in FIG. 23.
  • the fixed transmission unit 34 transmits the fixed video etc. acquired in S2210 to the information processing device 1.
  • the fixed video etc. is, for example, a fixed video, one or more video attribute values, and a rights holder identifier.
  • the fixed processing unit 33 determines whether or not to end transmission of the fixed video. If the transmission is to be ended, the process returns to S2202. If the transmission is not to be ended, the process returns to S2210.
  • the fixed processing unit 33 judges whether or not to acquire a past fixed video, which is a fixed video that has already been stored. If a past fixed video is to be acquired, the process proceeds to S2302. If a past fixed video is not to be acquired, the process proceeds to S2202. The past videos are stored in a buffer (not shown).
  • the fixed processing unit 33 determines to acquire a past fixed video.
  • the fixed processing unit 33 assigns 1 to counter i.
  • the fixed processing unit 33 determines whether the i-th search unit exists in the fixed video stored in the fixed storage unit 31. If the i-th search unit exists, the process proceeds to S2304, and if the i-th search unit does not exist, the process returns to the upper process.
  • the fixed processing unit 33 obtains the attribute value set corresponding to the i-th search unit from the fixed storage unit 31.
  • the fixed processing unit 33 determines whether the attribute value set acquired in S2304 matches the received query. If it matches the query, the process proceeds to S2306; if it does not match the query, the process proceeds to S2307.
  • the fixed processing unit 33 acquires the fixed video etc. corresponding to the i-th search unit.
  • the fixed video etc. is, for example, a fixed video, an attribute value set, and a rights holder identifier.
  • the fixed video here may be the video of the i-th search unit, or may include a video that is temporally before and after the video of the i-th search unit based on the video of the i-th search unit.
  • the user reception unit 42 determines whether or not an inquiry has been received. If an inquiry has been received, the process proceeds to S2402; if not, the process proceeds to S2407.
  • accepting an inquiry includes, for example, accepting user input, receiving location information from the target terminal 5, and receiving destination or route information from the navigation terminal 6.
  • the user processing unit 43 composes a query to be sent.
  • the user sending unit 44 associates the query with the user identifier and sends it to the information processing device 1.
  • the user receiving unit 45 determines whether or not information has been received from the information processing device 1. If information has been received, the process proceeds to S2404; if information has not been received, the process returns to S2403.
  • the user processing unit 43 determines whether the information received in S2403 includes a video. If it does include a video, the process proceeds to S2405. If it does not include a video, the process proceeds to S2406.
  • the user output unit 46 outputs the other information that was received. Return to S2401. Note that the other information is, for example, information indicating that the video could not be acquired.
  • the user processing unit 43 determines whether it is time to make an inquiry. If it is time to make an inquiry, the process proceeds to S2408. If it is not time to make an inquiry, the process proceeds to S2409.
  • the timing of the inquiry is, for example, when a destination is set on the user terminal 4, which is a navigation terminal, or when the user terminal 4 installed in the vehicle detects congestion.
  • the user processing unit 43 performs a query generation process. Go to S2402. An example of the query generation process is explained using the flowchart in FIG. 25.
  • the user reception unit 42 determines whether or not a purchase instruction has been received from the user. If a purchase instruction has been received, the process proceeds to S2410, and if not, the process returns to S2401.
  • the purchase instruction may include, for example, an inquiry.
  • the user processing unit 43 composes a purchase instruction to be sent.
  • the user sending unit 44 associates the purchase instruction with the user identifier and sends it to the information processing device 1.
  • the user receiving unit 45 determines whether or not information has been received from the information processing device 1. If information has been received, the process proceeds to S2412, and if information has not been received, the process returns to S2411.
  • the information may be, for example, a video, information indicating that the transfer of rights holders has been completed, and a set of attribute values for the video.
  • the user processing unit 43 uses the received information to construct information to be output.
  • the user output unit 46 outputs the information. Return to S2401.
  • processing ends when the power is turned off or an interrupt occurs to end processing.
  • the user processing unit 43 searches for a route using the destination and the current location, and obtains route information.
  • the user processing unit 43 acquires one or more pieces of point information that are points on the route specified by the route information acquired in S2503 and that satisfy the extraction conditions.
  • the point information is usually location information included in the route information.
  • the user processing unit 43 constructs a query using one or more pieces of location information acquired in S2504. The process returns to the upper level process. For example, the user processing unit 43 constructs a query such as, "Search for videos that are within the next hour, have a tag of either "traffic jam” or "accident” added, and are paired with an attribute value set that includes one or more pieces of location information.”
  • the user processing unit 43 may obtain the current location and construct a query including the current location.
  • a query is a request to send a video corresponding to a specific tag (e.g., "accident” or "traffic jam") in an area surrounding the current location (e.g., a distance within a threshold).
  • the target terminal 5 acquires location information.
  • the target terminal 5 transmits the location information to the information processing device 1 or the user terminal 4 registered in the target terminal 5.
  • the target terminal 5 acquires and transmits the location information, for example, by receiving an instruction from the holder of the target terminal 5 (for example, a child being watched over).
  • the target terminal 5 has, for example, a GPS receiver and acquires the location information by the GPS receiver, but the method of acquiring the location information is not important.
  • the user terminal 4 registered in the target terminal 5 is, for example, the guardian of the holder of the target terminal 5.
  • the navigation terminal 6 accepts a destination from a user. Next, the navigation terminal 6 performs the inquiry generation process described in Fig. 25. Next, the navigation terminal 6 performs the processes from S2402 to S2405 and S2406 in Fig. 24. That is, the navigation terminal 6 accepts the destination, acquires route information, makes an inquiry to the information processing device 1 using the route information, and receives from the information processing device 1 videos of one or more points on the route indicated by the route information, and outputs them.
  • the navigation terminal 6 accepts the destination, acquires route information, makes an inquiry to the information processing device 1 using the route information, and receives from the information processing device 1 videos of one or more points on the route indicated by the route information, and outputs them.
  • a mobile terminal management table having the structure shown in FIG. 26 is stored in the mobile terminal management unit 111 of the information processing device 1.
  • the mobile terminal management table is a table that manages information about one or more mobile terminals.
  • the mobile terminal management table is a table that manages mobile terminals 2 that transmit mobile videos. Note that a mobile terminal 2 that is not managed in the mobile terminal management table may also transmit videos to the information processing device 1.
  • the mobile terminal management table manages one or more records having an "ID,” “terminal identifier,” “video information,” “movement information,” “registration flag,” and “provision flag.”
  • Video information is information about the captured video, and has a “frame identifier” and a “video attribute value.”
  • the "video attribute value” has “environmental information” and “tag.”
  • “Environmental information” has “location information,” “direction information,” “camera information,” “time information,” “weather information,” and “temperature information.”
  • “Camera information” has “angle of view” and “resolution.”
  • “Environmental information” is information about the environment around the mobile terminal 2 when the video was captured, etc.
  • Tags include “accident,” “traffic jam,” and “dangerous driving.” In other words, here, a video can be given one or more tags, for example, “accident,” “traffic jam,” and “dangerous driving.”
  • ID is information that identifies the record.
  • Terminal identifier is an identifier of the mobile terminal 2, and is the same as the rights holder identifier that identifies the rights holder of the video at the time of transmitting the video.
  • Full identifier is the ID of a frame that constitutes the video. A frame may also be called a field or a still image.
  • Location information is (latitude, longitude) here.
  • Direction information is the shooting direction of the camera, and here, the angle in a clockwise direction from true north. In other words, if the shooting direction of the camera is due east, the direction information is “90 degrees”.
  • Angle of view is the angle of view of the camera.
  • “Resolution” is the resolution of the camera.
  • “Time information” is here, the year, month, day, hour, minute, and second.
  • "Weather information” is, for example, “sunny", “rain”, “cloudy”, or “snow”.
  • "Temperature information” is the temperature (°C) outside the moving object.
  • a value of "1” for “accident” indicates that a tag indicating that an accident has occurred is attached to the corresponding frame.
  • a value of "1” for "traffic jam” indicates that a tag indicating that a traffic jam has occurred is attached to the corresponding frame.
  • a “Dangerous Driving” value of "1” indicates that a tag indicating that the car or the like in front is driving dangerously has been attached to the corresponding frame.
  • “Provision Flag” of “1” indicates that viewing of the video is permitted.
  • “Provision Flag” of "2” indicates that sale of the video (transfer of rights holder) is permitted.
  • the fixed terminal management unit 112 of the information processing device 1 also stores a fixed terminal management table having the structure shown in FIG. 27.
  • the fixed terminal management table is a table that manages information about one or more fixed terminals.
  • the fixed terminal management table is a table that manages fixed terminals 3 that transmit fixed videos.
  • the fixed terminal management table manages one or more records that have an "ID,” “terminal identifier,” “location information,” “direction information,” and “camera information.”
  • the "camera information” includes "angle of view” and "resolution.”
  • a registered video management table having the structure shown in FIG. 28 is stored on the blockchain.
  • the registered video management table is a table that manages registered videos.
  • the registered video management table is a table that manages preservation information.
  • Registered videos are, for example, videos provided to users in response to inquiries from the users, and videos that satisfy preservation conditions. In other words, here, videos that match inquiries received by the information processing device 1 and videos that satisfy preservation conditions are automatically accumulated.
  • the registered video management table manages one or more records having an "ID,” “video identifier,” “access information,” “rights holder identifier,” “rights registration date,” “video information,” and “provision flag.” Registered videos are stored in the information processing device 1, another device, or the blockchain. Each record in the registered video management table is considered to be preservation information here.
  • Specific examples 1 to 4 are examples of using real-time video.
  • Specific example 5 is an example of using video captured in the past.
  • Specific example 1 is a case where, in order to watch over a subject (e.g., a child or a wandering elderly person), a combined video is sent to a user terminal 4 of a user (e.g., a guardian) related to the subject, by combining fixed images taken by one or more fixed terminals 3 and moving videos taken by one or more mobile terminals 2 installed on each of the mobile bodies.
  • a subject e.g., a child or a wandering elderly person
  • a combined video is sent to a user terminal 4 of a user (e.g., a guardian) related to the subject, by combining fixed images taken by one or more fixed terminals 3 and moving videos taken by one or more mobile terminals 2 installed on each of the mobile bodies.
  • Specific example 2 is a case where a combined video that combines one or more fixed images and one or more moving videos is output to the user terminal 4 in response to an inquiry using route information corresponding to a destination set in the user terminal 4 or the navigation terminal 6.
  • Specific example 3 is a case where the user terminal 4 or navigation terminal 6 detects a traffic jam, obtains a combined video that allows the user to understand the reason for the traffic jam, and outputs it to the user terminal 4 or navigation terminal 6.
  • the combined video here is a video that combines one or more fixed images with one or more moving videos.
  • Specific example 4 is a case where a fusion video is output that allows a user to grasp the situation in a parking lot so that the user can find an available space in the parking lot.
  • the fusion video here is a video that is constructed by fusing one or more fixed images with one or more moving videos. In other words, one or more fixed terminals 3 are installed in the parking lot.
  • Specific example 5 is a case where a combined video that combines registered videos is output to the user terminal 4 using route information that identifies a route the user has traveled in the past (e.g., a route traveled).
  • the combined video here is a video that combines one or more fixed images with one or more moving videos.
  • the storage unit 11 of the information processing device 1 stores management information having a user identifier (e.g., the IP address of the user terminal 4) for sending a video to a user terminal 4 of a guardian P, and a target identifier "T001" that identifies a target terminal 5 of a child A of the guardian P.
  • a user identifier e.g., the IP address of the user terminal 4
  • T001 target identifier
  • the receiving unit 12 of the information processing device 1 periodically receives location information that identifies the location of the mobile terminal 2 from each of one or more mobile terminals 2, paired with the terminal identifier, and stores the location information in the mobile terminal management table ( Figure 26) paired with the terminal identifier.
  • the video acquisition unit 133 determines the fixed terminal 3 or the first mobile terminal 2 corresponding to the location information that satisfies the first location condition for the location indicated by the latest location information of the target terminal 5, by the operation described in the flowchart of FIG. 12. Note that here, it is assumed that the priority video is a "fixed video".
  • the video acquisition unit 133 then checks the fixed terminal management table (FIG. 27) to see whether there is location information, direction information, and angle of view that satisfy the first location condition (the location indicated by the latest location information of the target terminal 5 has been photographed) for the latest location information of the target terminal 5. If there is no location information, etc. that satisfies the first location condition in the fixed terminal management table (FIG. 27), it determines whether the latest location information, direction information, and angle of view of each mobile terminal 2 in the mobile terminal management table satisfy the first location condition for the latest location information of the target terminal 5.
  • the video acquisition unit 133 determines that the fixed terminal management table ( Figure 27) does not contain location information that satisfies the first location condition, and therefore no videos from any of the fixed terminals 3 can be used.
  • the video acquisition unit 133 has determined a first terminal (here, a mobile terminal 2 mounted on an automobile).
  • the video acquisition unit 133 receives a video from the first mobile terminal 2.
  • child A appears in the video.
  • a video is usually associated with the location information and the first rights holder identifier of the mobile terminal 2, which is the first terminal.
  • the received video is a video that was sent immediately after being shot by the mobile terminal 2.
  • the video sending unit 141 sends the video received by the video acquiring unit 133 to the user terminal 4 of guardian P, who is identified by the user identifier that pairs with the target person identifier "T001.”
  • the video acquisition unit 133 temporarily stores the video received from the first terminal in a buffer (not shown) by pairing it with the first rights holder identifier and location information of the first terminal.
  • the video acquisition unit 133 receives at least the video received from the first terminal while the location information of the target terminal 5 of child A and the location information of the first terminal, etc., satisfy the first location condition, and the video transmission unit 141 transmits the video to the user terminal 4 of parent P.
  • the video acquisition unit 133 determines that the location information of the target terminal 5 of Child A and the location information of the first terminal, etc., no longer satisfy the first location condition (Child A is no longer in the image). This is because the mobile terminal 2 has moved and is no longer in a position to capture an image of Child A, who is walking home.
  • the video acquisition unit 133 again tries to determine, by referring to the fixed terminal management table (FIG. 27), the location information of a second terminal that satisfies the first location condition for the latest location information of the target terminal 5.
  • the video acquisition unit 133 determines that the area being photographed based on the location information (FX 1 , FY 1 ), direction information "Fd 1 ", and angle of view "FD 1 " in the fixed terminal management table (FIG. 27) satisfies the first location condition for the latest location information of the target terminal 5.
  • the video acquisition unit 133 then transmits a video transmission instruction to the fixed terminal 3 identified by the terminal identifier "U101" and receives the fixed video from the fixed terminal 3.
  • the video acquisition unit 133 then pairs the video with the second rights holder identifier "U101" of the second terminal (the fixed terminal 3) and location information, and temporarily stores the video in a buffer (not shown).
  • the video sending unit 141 sends the fixed video received by the video acquisition unit 133 from the second terminal to the guardian P's user terminal 4.
  • the video acquisition unit 133 determines that the position information of Child A's target terminal 5 and the position information, direction information, and angle of view of the second terminal no longer satisfy the first position condition (Child A is no longer in the picture).
  • the video acquisition unit 133 again attempts to determine the location information of a third terminal whose latest location information satisfies the first location condition by referring to the fixed terminal management table in FIG. 27 for the latest location information of the target terminal 5, but it is assumed that there is no location information of a third terminal (fixed terminal 3) that satisfies the first location condition.
  • the video acquisition unit 133 refers to FIG. 26 and attempts to determine the location information, etc. of a mobile terminal 2 whose latest location information satisfies the first location condition with respect to the latest location information of the target terminal 5, but there is no location information, etc. of a mobile terminal 2 that satisfies the first location condition.
  • the video transmission unit 141 may transmit information to the user terminal 4 of the guardian P that the video containing child A cannot be transmitted, or may stop transmitting the video or information to the user terminal 4.
  • the user terminal 4 sequentially receives and outputs the mobile video acquired by the mobile terminal 2 and the fixed video acquired by the fixed terminal 3. This allows the guardian P to see how the child A returns home.
  • a third terminal a mobile terminal 2 approaches child A, and the video acquisition unit 133 refers to FIG. 26 and determines the location information, etc. of the third terminal that satisfies the first location condition for the latest location information of the target terminal 5.
  • the video acquisition unit 133 was unable to detect the location information, etc. of the fixed terminal 3 that satisfies the first location condition for the latest location information of the target terminal 5.
  • the video acquisition unit 133 acquires the video from the third terminal.
  • the video acquisition unit 133 pairs the video with the third rights holder identifier and location information of the third terminal, and temporarily stores the video in a buffer (not shown).
  • the video sending unit 141 sends the video received by the video acquisition unit 133 from the third terminal to the user terminal 4 of guardian P, who is identified by the user identifier that pairs with the target identifier "T001.”
  • user terminal 4 receives and outputs the video acquired by the third terminal.
  • parent P was unable to see Child A's return home for a short time, but by outputting the video, he or she is once again able to see Child A's return home.
  • Child A's target device 5 is powered off (until Child A returns home), allowing Parent B to monitor Child A's return home from school.
  • the compensation means 1365 also performs the compensation process described above for the rights holders identified by the rights holder identifiers of the first mobile terminal 2, the fixed terminal 3 which is the second terminal, and the third mobile terminal 2, which provided the video to parent B.
  • the video composition unit 134 also combines the videos sent from each mobile terminal 2 and fixed terminal 3 in the order they were sent to compose a combined video.
  • the rights holder processing unit 136 also stores the combined video in a pair with a rights holder identifier, which is the identifier of guardian P.
  • the rights holder processing unit 136 also stores the combined video in association with a set of attribute values that correspond to one or more videos that are the source of the combined video.
  • the fourth maintenance means 1364 acquires access information that specifies where the combined video is stored.
  • the fourth maintenance means 1364 also acquires a set of attribute values that correspond to the stored combined video.
  • the fourth maintenance means 1364 then constructs maintenance information that has the acquired access information, the acquired set of attribute values, and the rights holder identifier of the video.
  • the fourth maintenance means 1364 then stores the constructed maintenance information in the blockchain.
  • the combined video can be used to monitor a subject who has a target terminal 5.
  • a reward can be given to the rights holder who provided the video that is the basis for the combined video used for monitoring.
  • the combined video can be appropriately managed.
  • a user inputs a destination to a user terminal 4 having a navigation function.
  • a user reception unit 42 of the user terminal 4 receives the destination.
  • a user processing unit 43 acquires a current position.
  • the user processing unit 43 acquires route information from the current position to the destination. It is assumed here that the route information includes multiple pieces of position information.
  • the user transmission unit 44 of the user terminal 4 automatically transmits an inquiry including the route information to the information processing device 1 in response to the acquisition of the route information by the user processing unit 43.
  • the inquiry receiving unit 122 of the information processing device 1 receives the inquiry.
  • the video acquisition unit 133 and the video composition unit 134 perform video combination processing as follows.
  • the video acquisition unit 133 acquires, from among the multiple pieces of position information contained in the route information, the position information (first position information) of the first intersection on the route specified by the route information. In other words, it is preferable for the video acquisition unit 133 to acquire a video using only a portion of the position information contained in the received route information. It is preferable for the video acquisition unit 133 to acquire a video using only a portion of the position information contained in the received route information that satisfies a specific condition.
  • the specific condition is, for example, that the position information is information indicating the position of an intersection, and that the distance from the previously used position information is equal to or greater than a threshold.
  • the video acquisition unit 133 determines the position information, direction information, and angle of view of the first terminal that satisfies the first position condition for the acquired position information.
  • the first terminal is the fixed terminal 3 or the mobile terminal 2.
  • the video acquisition unit 133 preferably searches the fixed terminal management table ( Figure 27) according to the priority type "fixed terminal", and if the position information, etc. of the first terminal cannot be determined, it searches the mobile terminal management table ( Figure 26).
  • the video acquisition unit 133 acquires the video from the determined first terminal, pairs the video with the first rights holder identifier and first location information, etc. of the first terminal, and temporarily stores the video in a buffer (not shown).
  • the video transmission unit 141 transmits the acquired video to the user terminal 4.
  • the video acquisition unit 133 acquires second location information of the next intersection that is closer to the destination than the location indicated by the first location information of the previously acquired intersection.
  • the video acquisition unit 133 determines the location information of a second terminal that satisfies the first location condition for the second location information.
  • the second terminal is a fixed terminal 3 or a mobile terminal 2.
  • the video acquisition unit 133 acquires the video from the determined second terminal, pairs the video with the second rights holder identifier and second location information of the second terminal, and temporarily stores the video in a buffer (not shown).
  • the video transmission unit 141 transmits the acquired video to the user terminal 4.
  • the video acquisition unit 133 acquires third location information of the next intersection that is closer to the destination than the location indicated by the second location information.
  • the video acquisition unit 133 determines the location information, etc. of a third terminal that satisfies the first location condition for the third location information. Then, the video acquisition unit 133 acquires a video from the determined third terminal, pairs the video with the third rights holder identifier of the third terminal and the third location information, etc., and temporarily stores the video in a buffer (not shown). Next, the video transmission unit 141 transmits the acquired video to the user terminal 4.
  • the information processing device 1 repeats the above process until it transmits a video corresponding to the nth location information that identifies the destination.
  • the user receiving unit 45 of the user terminal 4 sequentially receives the first video, the second video, the third video, ... the nth video.
  • the user output unit 46 sequentially outputs the first video, the second video, the third video, ... the nth video.
  • the reward means 1365 of the information processing device 1 performs reward processing to reward the first rights holder, the second rights holder, ..., and the nth rights holder identified by the first rights holder identifier for providing the video.
  • the video composition unit 134 also combines the videos sent from the multiple terminals in the order they were sent to compose a combined video.
  • the combined video is a video in which, for example, the mobile video sent from the mobile terminal 2 and the fixed video sent from the fixed terminal 3 are combined in time in the order they were received.
  • the rights holder processing unit 136 also stores the combined video in a pair with a rights holder identifier, which is the identifier of the user of the user terminal 4. In other words, in this case, the rights holder of the combined video is the user.
  • the rights holder processing unit 136 also stores the combined video in association with a set of attribute values that correspond to one or more videos that are the source of the combined video.
  • the fourth security means 1364 acquires access information that specifies where the combined video is stored.
  • the fourth security means 1364 also acquires a set of attribute values that correspond to the stored combined video.
  • the fourth security means 1364 then constructs security information that has the acquired access information, the acquired set of attribute values, and the rights holder identifier of the video.
  • the fourth security means 1364 then stores the constructed security information.
  • videos transmitted from multiple mobile terminals 2 are combined in time, at least in appearance, to create a combined video, which allows the user to check the route to the destination in order of proximity from the current position, thereby providing support for driving a vehicle such as an automobile.
  • the combined video can be appropriately stored and managed.
  • the navigation terminal 6 detects a traffic jam on a route to a destination, for example.
  • the function of detecting a traffic jam is possible using known technology.
  • the navigation terminal 6 acquires route information having one or more pieces of location information specifying a road with a traffic jam, and transmits an inquiry including the route information and a user identifier stored in the navigation terminal 6 to the information processing device 1.
  • the user identifier is an identifier of the user terminal 4 that receives a video transmitted from the fixed terminal 3 or the mobile terminal 2 located at the position of the road with a traffic jam, and is, for example, an IP address.
  • the route information is information that specifies one or more roads with a traffic jam.
  • the user terminal 4 is, for example, a terminal of a passenger in the front passenger seat.
  • the inquiry receiving unit 122 of the information processing device 1 receives the inquiry including the route information and the user identifier.
  • the video acquisition unit 133 and the video composition unit 134 perform video combination processing as follows.
  • the video acquisition unit 133 acquires the location information that is closest to the current location from among the location information included in the route information.
  • the location information that is closest to the current location is, for example, the first location information among the location information included in the route information.
  • the video acquisition unit 133 acquires the last piece of location information that follows the location information and is within the threshold value of the adjacent location information. The video acquisition unit 133 sets this location information as the target location information to be used for video acquisition.
  • the video acquisition unit 133 determines a first terminal, such as location information, that satisfies a first location condition for the target location information.
  • the first terminal is a fixed terminal 3 or a mobile terminal 2.
  • the video acquisition unit 133 acquires the video from the terminal, pairs the video with the first rights holder identifier of the first terminal and location information, etc., and temporarily stores the video in a buffer (not shown).
  • the video acquisition unit 133 receives video from the first terminal and temporarily stores it in a buffer (not shown) until the location information, etc. of the first terminal no longer satisfies the first location condition.
  • the video transmission unit 141 also sequentially transmits the acquired videos to the user terminal 4.
  • the video acquisition unit 133 determines a second terminal, such as location information, that satisfies the first location condition for the target location information.
  • the first terminal is a fixed terminal 3 or a mobile terminal 2.
  • the video acquisition unit 133 acquires the video from the second terminal, pairs the video with the second rights holder identifier and location information, etc. of the second terminal, and temporarily stores the video in a buffer (not shown).
  • the video acquisition unit 133 receives video from the second mobile terminal 2 and temporarily stores it in a buffer (not shown) until the location information, etc. of the second terminal no longer satisfies the first location condition.
  • the video transmission unit 141 also sequentially transmits the acquired videos to the user terminal 4.
  • the user terminal 4 receives the video from the information processing device 1 and outputs it.
  • the user of the user terminal 4 can continuously know the traffic congestion situation.
  • the user can also know the point where the traffic congestion will be resolved.
  • the information processing device 1 performs a similar process using route information identifying the second and subsequent locations.
  • the user terminal 4 can also receive and output video for understanding the congestion situation at the second and subsequent locations. Note that, if there are two or more congested locations, it is preferable that the user terminal 4 automatically or at the user's instruction switch between the locations to receive and output the video.
  • the traffic congestion situation on the route to the destination can be understood using video transmitted from multiple terminals including the fixed terminal 3 and the mobile terminal 2.
  • the inquiry receiving unit 122 of the information processing device 1 receives the inquiry.
  • the video acquisition unit 133 determines from the received inquiry that spatial fusion of multiple videos is necessary.
  • the video acquisition unit 133 and the video composition unit 134 perform video fusion processing as follows.
  • the video acquisition unit 133 acquires the reference location information contained in the received inquiry.
  • the video acquisition unit 133 determines one or more mobile terminals 2 that correspond to location information that satisfies a second location condition with respect to the reference location information.
  • the video acquisition unit 133 transmits an instruction to transmit a current around-view image to one or more terminals.
  • Each of the multiple terminals is a fixed terminal 3 or a mobile terminal 2.
  • the second location condition here is that the location indicated by the location information is present in the area of the parking lot that includes the reference location information. It is assumed that each of the one or more mobile terminals 2 is continuously acquiring around-view images. It is also assumed that one or more fixed terminals 3 are installed on the ceiling of the parking lot, capturing an image directly below, and acquiring around-view images.
  • the video acquisition unit 133 receives from each of one or more terminals an around-view image that corresponds to the terminal's location information.
  • the video composition unit 134 then calculates the difference (distance) between the position information paired with each received around-view image and the reference position information.
  • the video composition unit 134 then sorts the around-view images in ascending order using the difference as a key.
  • the video composition unit 134 acquires a reference image.
  • the reference image is the around-view image included in the query.
  • the video composition unit 134 then performs the process described using the flowchart in FIG. 11, and uses the position information to place each of the around-view images received from the other terminals in the appropriate position relative to the reference image in the sorted order, thereby composing a fusion image.
  • the video composition unit 134 also composes an image that clearly indicates the position indicated by the reference position information (the position of the user terminal 4 that sent the query) in the finally constructed fusion image. This image is also a fusion image.
  • the video transmission unit 141 transmits the constructed fusion image to the user terminal 4.
  • the user terminal 4 receives and outputs the fusion image.
  • the information processing device 1 also repeats the above process, and as a result, transmits a fusion video including multiple fusion images to the user terminal 4.
  • the user terminal 4 then receives and outputs the fusion video.
  • images transmitted from one or more other mobile terminals 2 or fixed terminals 3 can be fused to assist in finding vacant spaces in a parking lot.
  • the information processing device 1 fuses around-view images.
  • the information processing device 1 may fuse multiple normal camera images or multiple images captured by an omnidirectional camera using position information corresponding to each image to construct a fused image and transmit the fused image to the user terminal 4.
  • images from a fixed terminal 3 installed at an event venue such as a live performance may be fused with images from user terminals 4 held by members of the audience who have gathered at the event venue, and images of areas that could not be captured by the user terminals 4 held by the audience may be supplemented with images from the fixed terminals 3.
  • User terminal 4 accepts the inquiry and transmits the inquiry to information processing device 1. Note that this inquiry is for obtaining a video of the driving course traveled by the user on a sunny day.
  • the inquiry receiving unit 122 of the information processing device 1 receives the inquiry.
  • the video acquisition unit 133 determines that the received inquiry is not a real-time video search.
  • the video acquisition unit 133 etc. perform a registered video search process as follows.
  • the video acquisition unit 133 acquires the first location information contained in the route information in the received inquiry.
  • the video acquisition unit 133 acquires all the preservation information from the registered video management table ( Figure 28). In other words, the video acquisition unit 133 acquires, for example, the preservation information that exists on the blockchain.
  • the video acquisition unit 133 acquires the access information "Address 01" that pairs with the determined security information.
  • the video acquisition unit 133 acquires a video using the access information "Address 01".
  • the video acquisition unit 133 cuts out from the video corresponding to the access information "Address 01” a video up to the point where the second or subsequent location information in the route information of the received inquiry satisfies (is the same as or is close to) the first location condition as the location information of the security information, etc. In other words, the video acquisition unit 133 cuts out and acquires from the video corresponding to the access information "Address 01" a video up to the point where the user deviates from the route of the driving course traveled by the user.
  • the video acquisition unit 133 acquires second location information contained in the route information contained in the received inquiry, which is location information when the video corresponding to the access information "Address 01" deviates from the driving course.
  • the video acquisition unit 133 acquires access information (for example, "Address X") that pairs with the determined security information.
  • access information for example, "Address X”
  • the video acquisition unit 133 acquires a second video using the access information "Address X”.
  • the video acquisition unit 133 cuts out from the second video corresponding to the access information "Address X” a video up to the point where the location information beyond the second location information contained in the route information contained in the received inquiry satisfies the location information of the security information.
  • the video acquisition unit 133 cuts out and acquires from the video corresponding to the access information "Address X" a video up to the point where the user deviates from the route of the driving course traveled by the user.
  • the video acquisition unit 133 performs the above process until it uses the last location information contained in the route information in the received inquiry.
  • the video composition unit 134 combines the multiple videos acquired by the video acquisition unit 133 in the order in which they were acquired (in the order of the route) to compose a combined video. It is preferable that the multiple videos that are the basis for the combined video include fixed videos and moving videos.
  • the video sending unit 141 sends the combined video to user B's user terminal 4.
  • the user terminal 4 receives and outputs the combined video.
  • the reward means 1365 of the information processing device 1 acquires the rights holder identifiers (e.g., "U001") of each of the original multiple videos used in the combined video, and performs reward processing to give rewards to the rights holders identified by each of the multiple rights holder identifiers.
  • the rights holder identifiers e.g., "U001”
  • the rights holder processing unit 136 also performs the various preservation processes described above on the combined video.
  • route information about the route traveled by the user can be used to create and output a combined video by combining multiple videos taken along that route in the past.
  • a useful video can be constructed and provided by using a video captured by the mobile terminal 2 and a video captured by the fixed terminal 3.
  • a video captured by the mobile terminal 2 and a video captured by the fixed terminal 3 can be temporally combined to create and provide a single useful video.
  • a video captured by the mobile terminal 2 and a video captured by the fixed terminal 3 can be spatially merged to create and provide a single useful video.
  • a useful video can be constructed and provided by using moving video and fixed video based on the intentions of the user of the mobile terminal 2.
  • the camera used to capture the video can be an omnidirectional camera or the like, and the type is not important.
  • the processing in this embodiment may be realized by software.
  • This software may be distributed by software download or the like.
  • This software may also be recorded on a recording medium such as a CD-ROM and distributed. This also applies to the other embodiments in this specification.
  • the software that realizes information system A in this embodiment is a program such as the following.
  • this program causes a computer to execute a video acquisition step of acquiring a moving video, which is a video captured by a mobile terminal and corresponds to an attribute value set including one or more pieces of environmental information including location information specifying the shooting location or time information specifying the shooting time, and transmitted from the mobile terminal, and acquiring a fixed video, which is a video captured by a fixed camera with a fixed shooting location and corresponds to an attribute value set including one or more pieces of environmental information including location information or time information, and transmitted from a fixed terminal equipped with the fixed camera; a video composition step of temporally combining the moving video and the fixed video to compose a single video, or of compos- ing a single video having two or more frames by spatially fusing at least a portion of the frames of the moving video and at least a portion of the frames of the fixed video; and a video transmission step of transmitting the single video composed in the video composition step.
  • a moving video which is a video captured by a mobile terminal and corresponds to an attribute value set including one or
  • FIG. 29 also shows the appearance of a computer that executes the programs described in this specification to realize the information processing device 1, user terminal 4, target terminal 5, navigation terminal 6, etc., of the various embodiments described above.
  • the above-mentioned embodiments can be realized by computer hardware and computer programs executed thereon.
  • FIG. 29 is an overview of this computer system 300
  • FIG. 30 is a block diagram of system 300. Note that the mobile terminal 2 and fixed terminal 3 are equipped with cameras.
  • computer system 300 includes computer 301, which includes a CD-ROM drive, keyboard 302, mouse 303, and monitor 304.
  • computer 301 includes, in addition to CD-ROM drive 3012, MPU 3013, bus 3014 connected to CD-ROM drive 3012 etc., ROM 3015 for storing programs such as a boot-up program, RAM 3016 connected to MPU 3013 for temporarily storing instructions for application programs and providing temporary storage space, and hard disk 3017 for storing application programs, system programs, and data.
  • computer 301 may further include a network card that provides connection to a LAN.
  • the program that causes the computer system 300 to execute the functions of the information processing device 1 of the above-mentioned embodiment may be stored on the CD-ROM 3101, inserted into the CD-ROM drive 3012, and then transferred to the hard disk 3017.
  • the program may be sent to the computer 301 via a network (not shown) and stored on the hard disk 3017.
  • the program is loaded into the RAM 3016 when executed.
  • the program may be loaded directly from the CD-ROM 3101 or the network.
  • the program does not necessarily have to include an operating system (OS) that causes the computer 301 to execute the functions of the information processing device 1 of the above-mentioned embodiment, or a third-party program, etc.
  • OS operating system
  • the program only needs to include an instruction portion that calls appropriate functions (modules) in a controlled manner to obtain the desired results. How the computer system 300 operates is well known, and a detailed description will be omitted.
  • the steps of transmitting information and receiving information do not include processing performed by hardware, such as processing performed by a modem or interface card in the transmission step (processing that can only be performed by hardware).
  • the computer that executes the above program may be a single computer or multiple computers. In other words, it may perform centralized processing or distributed processing.
  • multiple communication means present in one device may be realized physically by one medium.
  • each process may be realized by centralized processing in a single device, or may be realized by distributed processing in multiple devices.
  • the information processing device 1 has the effect of being able to use videos captured by a mobile terminal and videos captured by a fixed camera to create and provide a single useful video, and is useful as a server that provides videos, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Mobile Radio Communication Systems (AREA)
PCT/JP2022/039141 2022-10-20 2022-10-20 情報処理装置、および情報処理方法 Ceased WO2024084663A1 (ja)

Priority Applications (4)

Application Number Priority Date Filing Date Title
PCT/JP2022/039141 WO2024084663A1 (ja) 2022-10-20 2022-10-20 情報処理装置、および情報処理方法
EP22962771.6A EP4607911A1 (en) 2022-10-20 2022-10-20 Information processing device and information processing method
JP2024551159A JP7670937B2 (ja) 2022-10-20 2022-10-20 情報処理装置、および情報処理方法
US19/071,752 US20250200716A1 (en) 2022-10-20 2025-03-06 Information processing device and information processing method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/039141 WO2024084663A1 (ja) 2022-10-20 2022-10-20 情報処理装置、および情報処理方法

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US19/071,752 Continuation US20250200716A1 (en) 2022-10-20 2025-03-06 Information processing device and information processing method

Publications (1)

Publication Number Publication Date
WO2024084663A1 true WO2024084663A1 (ja) 2024-04-25

Family

ID=90737202

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/039141 Ceased WO2024084663A1 (ja) 2022-10-20 2022-10-20 情報処理装置、および情報処理方法

Country Status (4)

Country Link
US (1) US20250200716A1 (https=)
EP (1) EP4607911A1 (https=)
JP (1) JP7670937B2 (https=)
WO (1) WO2024084663A1 (https=)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008032739A1 (en) * 2006-09-12 2008-03-20 Panasonic Corporation Content imaging device
JP2008154192A (ja) 2006-11-22 2008-07-03 Sony Corp 画像表示システム、表示装置、表示方法
JP2020088855A (ja) * 2018-11-16 2020-06-04 協栄精工株式会社 ゴルフダイジェスト作成システム、移動撮影ユニットおよびダイジェスト作成装置
JP2022110177A (ja) * 2021-01-16 2022-07-29 協栄精工株式会社 ゴルフダイジェスト作成システム、移動撮影ユニットおよびダイジェスト作成装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008032739A1 (en) * 2006-09-12 2008-03-20 Panasonic Corporation Content imaging device
JP2008154192A (ja) 2006-11-22 2008-07-03 Sony Corp 画像表示システム、表示装置、表示方法
JP2020088855A (ja) * 2018-11-16 2020-06-04 協栄精工株式会社 ゴルフダイジェスト作成システム、移動撮影ユニットおよびダイジェスト作成装置
JP2022110177A (ja) * 2021-01-16 2022-07-29 協栄精工株式会社 ゴルフダイジェスト作成システム、移動撮影ユニットおよびダイジェスト作成装置

Also Published As

Publication number Publication date
JP7670937B2 (ja) 2025-04-30
EP4607911A1 (en) 2025-08-27
JPWO2024084663A1 (https=) 2024-04-25
US20250200716A1 (en) 2025-06-19

Similar Documents

Publication Publication Date Title
AU2016208929B2 (en) Systems and methods for providing information for an on-demand service
JP6773579B2 (ja) マルチメディアキャプチャ用のシステム及び方法
US20220012776A1 (en) Apparatus and method for using connected vehicles as an advertisement platform
CN113452927A (zh) 用于自主交通工具用户的增强的社交媒体体验
CN108985543A (zh) 共乘管理装置、共乘管理方法及存储介质
WO2018225069A1 (en) Digitizing and mapping the public space using collaborative networks of mobile agents and cloud nodes
US20190385452A1 (en) Method and system for vehicle location
JP7407246B2 (ja) サービス提供装置、サービス提供方法およびサービス提供プログラム
JP7717434B2 (ja) 情報処理装置、移動端末、情報処理方法、およびプログラム
US11487286B2 (en) Mobile object system that provides a commodity or service
JP7670937B2 (ja) 情報処理装置、および情報処理方法
JP7803736B2 (ja) 配信サーバー、プログラム及び配信システム
JP7667921B2 (ja) 情報処理装置、および情報処理方法
JP7733427B2 (ja) 情報システム
WO2024180655A1 (ja) 情報処理装置、および情報処理方法
WO2024201927A1 (ja) 情報処理装置、ユーザ端末、および情報処理方法
JP7667380B2 (ja) 情報処理装置、移動端末、ユーザ端末、および情報処理方法
JP2005227991A (ja) 情報提供方法及び装置
US20240426621A1 (en) Vehicle navigation methods to maximize operator's intent
JP2024119425A (ja) 情報処理装置、及び、方法
JP2022141511A (ja) 情報処理システム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22962771

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024551159

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 2501002413

Country of ref document: TH

WWE Wipo information: entry into national phase

Ref document number: 2022962771

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2022962771

Country of ref document: EP

Effective date: 20250520

WWP Wipo information: published in national office

Ref document number: 2022962771

Country of ref document: EP