WO2021033443A1 - 情報処理装置、情報処理方法、プログラム及び情報処理システム - Google Patents

情報処理装置、情報処理方法、プログラム及び情報処理システム Download PDF

Info

Publication number
WO2021033443A1
WO2021033443A1 PCT/JP2020/026527 JP2020026527W WO2021033443A1 WO 2021033443 A1 WO2021033443 A1 WO 2021033443A1 JP 2020026527 W JP2020026527 W JP 2020026527W WO 2021033443 A1 WO2021033443 A1 WO 2021033443A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
information
information processing
image data
dimensional model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/026527
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
嵩明 加藤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Corp
Original Assignee
Sony Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Corp filed Critical Sony Corp
Priority to US17/635,085 priority Critical patent/US12002158B2/en
Priority to JP2021540657A priority patent/JP7559761B2/ja
Priority to CN202080057389.7A priority patent/CN114270404A/zh
Publication of WO2021033443A1 publication Critical patent/WO2021033443A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G06T17/10—Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00—Payment architectures, schemes or protocols
    • G06Q20/04—Payment circuits
    • G06Q20/06—Private payment circuits, e.g. involving electronic currency used among participants of a common payment scheme
    • G06Q20/065—Private payment circuits, e.g. involving electronic currency used among participants of a common payment scheme using e-cash
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q20/00—Payment architectures, schemes or protocols
    • G06Q20/38—Payment protocols; Details thereof
    • G06Q20/389—Keeping log of transactions for guaranteeing non-repudiation of a transaction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/17—Terrestrial scenes taken from planes or by drones
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/10—Recognition assisted with metadata

Definitions

  • This technology relates to information processing devices, information processing methods, programs, and information processing systems that use blockchain.
  • Patent Document 1 describes that at a construction site or the like, an aerial image of a land to be surveyed by a drone (unmanned airplane) is analyzed to generate 3D data or the like showing the topographical shape of the land. ..
  • the situation changes over time, such as the number of materials, their storage space, earth and sand, and the shape of the building to be constructed. It is desirable to regularly monitor and manage changes in circumstances in order to carry out accurate and prompt construction work.
  • the purpose of this technology is to provide an information processing device, an information processing method, a program, and an information processing system capable of grasping the situation in a certain area.
  • the information processing apparatus includes an image acquisition unit, a three-dimensional model generation unit, and a block generation unit.
  • the image acquisition unit acquires image information having image data linked to each other and position information at the time of acquisition of the image data.
  • the three-dimensional model generation unit generates a three-dimensional model using the image information.
  • the block generation unit generates a new block connected to a block chain formed by connecting blocks in which image information used when generating the three-dimensional model is recorded.
  • An image use determination unit for verifying whether or not the image information is approved as a transaction is further provided, and the three-dimensional model generation unit may generate a three-dimensional model using the image information approved as the transaction. Good.
  • the image use determination unit may determine whether or not to use the image information approved as a transaction in the three-dimensional model generation.
  • the image usage determination unit uses the image data of the usage determination target for the three-dimensional model generation based on the similarity of the image data of the usage determination target to the image data used for the three-dimensional model generation. May be determined.
  • the image use determination unit may sort a plurality of the above image information using the above position information.
  • the three-dimensional model generation unit may generate the three-dimensional model using a plurality of the image information sorted using the position information.
  • the information processing method acquires unauthenticated image information having image data linked to each other and position information at the time of acquisition of the image data, and obtains the image.
  • a three-dimensional model is generated using the information, and a new block is generated which is connected to a block chain in which the blocks in which the image information used at the time of generating the three-dimensional model is recorded are connected.
  • the program according to one form of the present technology uses the step of acquiring the image information having the image data linked to each other and the position information at the time of acquiring the image data, and the image information.
  • the information processing device is subjected to a process including a step of generating a three-dimensional model and a step of generating a new block connected to a block chain formed by connecting blocks in which image information used when generating the above three-dimensional model is recorded. Let it run.
  • the information processing system includes a plurality of information processing devices.
  • the information processing apparatus includes a three-dimensional model generation unit that generates a three-dimensional model using image information that has image data linked to each other and position information at the time of acquisition of the image data, and a generation of the three-dimensional model. It is provided with a block generation unit that generates a new block to be connected to a block chain formed by connecting blocks in which image information used at times is recorded.
  • FIG. 1 is a schematic view showing the configuration of the information processing system 1.
  • a three-dimensional map of a certain area of a municipality unit is generated, and this three-dimensional map can be used by, for example, a civil engineering construction company or a company that provides a game using the three-dimensional map. It has become.
  • a plurality of companies that agree on the use of a three-dimensional map of a certain area may form an alliance to form a group.
  • a three-dimensional map is a three-dimensional model that includes geospatial information.
  • the three-dimensional map is generated and updated by a plurality of information processing devices (miners) based on a plurality of images provided by an unspecified number of image providers (end users).
  • the images used to generate the 3D map are recorded on the blockchain.
  • a civil engineering construction company which is a user of a 3D map, can accurately grasp the situation of a construction site, such as the number of materials, storage space, and the state of earth and sand, by looking at the 3D map that is updated from time to time. it can.
  • a company that provides a game can utilize the three-dimensional map for producing a game using the three-dimensional map.
  • the miner receives a reward from the user of the three-dimensional map for the generation of the three-dimensional map, and the end user for the provision of the image. The reward will be described later.
  • the information processing system 1 includes a minor group 6 composed of a plurality of information processing devices (minors) 2, a group 3 composed of a plurality of companies 30A to 30D having an alliance, and a plurality of mobile terminals ( It has an end user group 7 composed of end users) 4.
  • a minor group 6 composed of a plurality of information processing devices (minors) 2
  • a group 3 composed of a plurality of companies 30A to 30D having an alliance
  • a plurality of mobile terminals It has an end user group 7 composed of end users 4
  • FIG. 1 in order to distinguish each mobile terminal, reference numerals 4A to 4C are attached, but when it is not particularly necessary to distinguish them, they will be referred to as mobile terminals 4.
  • the number of mobile terminals 4 is shown as 3
  • the number of companies with which the alliance is formed is 4
  • the number of information processing devices (minor) 2 is shown as 3. Not limited to numbers.
  • the mobile terminal 4 is possessed by an unspecified number of end users.
  • the end user is an image provider who provides image data obtained by capturing an area to be generated by a three-dimensional map, and is, for example, a general consumer.
  • the end user can upload the image information of the image taken by the mobile terminal 4 to the image / three-dimensional map database (hereinafter, referred to as the image / three-dimensional map DB) 26 described later. Details of the mobile terminal 4 will be described later.
  • the plurality of information processing devices 2 are configured to be able to communicate with each other by P2P (Peer to Peer). Further, each information processing device 2 is configured to be able to communicate with the mobile terminal 4.
  • the information processing device 2 verifies the unauthenticated image uploaded to the image / three-dimensional map DB 26, and performs a verification work of whether or not to approve it as a transaction.
  • the image data approved as a transaction is image data that can be used for generating a three-dimensional map, and is image data that can be used as a consideration for providing the image data.
  • the information processing device 2 generates a three-dimensional map using an image approved as a transaction.
  • the information processing device 2 performs a mining process for recording the image used at the time of generating the three-dimensional map on the blockchain. That is, the information processing device 2 is a minor. Further, the information processing device 2 stores a part (fragmentary three-dimensional map) of the three-dimensional map generated by itself and another information processing device 2 in the image / three-dimensional map DB 26. Details of the information processing device 2 will be described later.
  • Group 3 consists of a plurality of companies 30A to 30D. Each company 30A to 30D can use a three-dimensional map. Each company 30A to 30D owns an information processing device (not shown) that acquires a fragmentary three-dimensional map stored in the information processing device (minor) 2 and integrates one three-dimensional map. Group 3 pays a reward to a minor who is the owner of the information processing device 2 that generates a three-dimensional map and an end user who is a user of the mobile terminal 4 who is an image provider. Miners are rewarded, for example, when the 3D map is updated. More specifically, the miner who generates a 3D map using images, discovers the nonce first, and generates the block connected to the blockchain is paid. In addition, the end user is paid a reward when, for example, the provided image data is used when generating a three-dimensional map.
  • FIG. 2 is a block diagram showing a configuration example of the mobile terminal 4 and the information processing device 2 which are a part of the information processing system 1.
  • the electronic device possessed by the end user may be a mobile terminal such as a smartphone or a PDA (Personal Digital Assistant), a digital camera, a video, or the like, and has at least an imaging unit and a position acquisition unit. It suffices if the image data and the position information acquired by the position acquisition unit can be uploaded to the image / three-dimensional map DB 26 directly or indirectly via another information terminal.
  • a mobile terminal 4 such as a smartphone as an electronic device possessed by an end user will be given.
  • the image information will be described here as an example, video information may be used.
  • the mobile terminal 4 has an application program that can upload the image information of the image taken by itself to the image / three-dimensional map DB26 and receive a reward when the image information is used for three-dimensional map generation. It has been downloaded in advance.
  • the mobile terminal 4 includes an image sensor 41 as an image pickup unit, a GPS (Global Positioning System) 42 as a position information acquisition unit, and a geomagnetic sensor 43 as an attitude information acquisition unit.
  • the image sensor 41 acquires image data by a shooting operation by the end user.
  • the GPS 42 detects the position information of the mobile terminal 4.
  • the GPS 42 measures its position based on triangulation using radio waves from artificial satellites orbiting the earth.
  • the position information indicates the position of the mobile terminal (end user) 4 when the image data is acquired, and the position information and the image data are associated with each other.
  • the information acquired by the GPS 42 includes date and time information such as a date and a time stamp, as well as position information such as altitude when the longitude, latitude, and altitude are used as sea level standards.
  • the geomagnetic sensor 43 detects the attitude information of the mobile terminal 4.
  • the posture information is the photographed image orientation in the true direction, and is the photographed direction information seen from the mobile terminal 4.
  • the display unit 44 displays an image or the like taken by the image sensor 41.
  • the display unit 44 is composed of an image display panel such as a liquid crystal display or an organic EL display.
  • the communication unit 45 communicates with the information processing device 2 via the wireless network.
  • the communication unit 45 transmits image information including image data and position information to the information processing device 2.
  • the transmitted image information is stored in the image / three-dimensional map DB 26.
  • the input operation unit 46 is composed of keys, buttons, a touch panel, and the like. An image is uploaded or the like by an end user's input operation using the input operation unit 46.
  • the end user who owns the mobile terminal 4 can provide image information (image data with position information) of the image acquired by himself / herself, and can receive a reward by using the image information at the time of generating a three-dimensional map. Further, in the image information acquired by the mobile terminal 4 and uploaded to the image / three-dimensional map DB 26, in addition to the image data and the position information, it is specified by which mobile terminal 4 the image associated with these is acquired. Mobile terminal information (end user information) may be included so that it can be performed. As a result, the end user who provided the image used for the three-dimensional map generation can be identified, and the end user who contributed to the three-dimensional map generation can be paid a reward.
  • the uploaded image information may include date and time information indicating the date and time of shooting acquired by GPS 42 in association with the image data. Since the timing of uploading by each mobile terminal 4 may differ from that at the time of image data acquisition, the date and time information is included in the image information, so that when the three-dimensional map is generated, the date and time information is used in addition to the position information. Data can be sorted into multiple groups. Sorting will be described later. When the date and time information is included, the image data is stored in the image / three-dimensional map DB 26 in time series based on the date and time information.
  • the information processing device 2 has a control unit 20, an image / three-dimensional map DB 26, and a recording ledger DB 27.
  • the information processing device 2 includes a program that performs image data usage determination processing, three-dimensional map generation processing using image information, and mining processing for recording image data used during three-dimensional map generation on a blockchain. ..
  • the image use determination process includes a verification operation of whether to approve the image data as a transaction and a determination operation of whether to use the image data for three-dimensional map generation.
  • the program is pre-installed in a storage unit (not shown).
  • the image / three-dimensional map DB 26 contains image information uploaded from the mobile terminal (end user) 4, a fragmentary three-dimensional map generated by itself, and a fragmentary tertiary generated by another information processing device (minor). Stores the original map.
  • the fragmentary three-dimensional map stored in the image / three-dimensional map DB 26 may be divided into fragment information after being integrated into the whole as a three-dimensional map (three-dimensional model).
  • a fragmentary three-dimensional map once integrated as a three-dimensional map and then divided into fragment information may be distributed to each information processing device (minor).
  • the image information includes at least position information and image data associated with each other, and in the present embodiment, end user information associated with the position information and image data are further included.
  • the recording ledger DB27 stores the image information used for generating the three-dimensional map in a blockchain.
  • the data stored in the recording ledger DB 27 will be referred to as blockchain data.
  • the blockchain data stored in the recording ledger DB 27 is similarly held by all of the information processing devices (miners) 2 participating in the three-dimensional map generation.
  • the blockchain data is generated by itself or another information processing device (minor) 2, and is updated as needed.
  • FIG. 3 is a conceptual diagram of the blockchain 5.
  • the blockchain 5 is configured by connecting each block 51 with a chain 52.
  • the block 51 stores transaction data, a nonce value, a hash value of the immediately preceding block, and a reward destination.
  • the transaction data is the image information used at the time of generating the three-dimensional map, and is the image data and the position information associated with each other.
  • the destination of the reward corresponds to the minor information that found the nonce.
  • the blockchain data is a recording ledger of image information used when generating a three-dimensional map.
  • the blockchain data contains information on the miners who contributed to the generation of the 3D map. Further, the blockchain data includes the image information used at the time of generating the three-dimensional map, and the image information stored in the image / three-dimensional map DB 26 includes the information of the mobile terminal 4 (end user information) in association with it. Therefore, the blockchain data indirectly includes the information of the end user who provided the image that contributed to the generation of the three-dimensional map. In this way, the blockchain data is not only a recording ledger of images used at the time of 3D map generation, but also a recording ledger of rewards paid to miners and end users who have contributed to 3D map generation.
  • the transaction data stored in the block 51 is the image data and the position information associated with each other, but in addition to this, the end user information may be included in the transaction data.
  • the image data stored in the image / 3D map DB26 is divided into two statuses: authenticated (verification completed) and unauthenticated (verification unworked).
  • authenticated verification completed
  • unauthenticated verification unworked
  • the image data that has been verified by the image use determination unit 22 of the control unit 20 described later to verify whether or not to approve it as a transaction is given an authenticated status, and the image data that has not yet been verified Is given an unauthenticated status.
  • the image data to which the authenticated status is given is divided into image data approved as a transaction and image data not approved as a transaction.
  • each information processing device 2 has a recording ledger DB 27, and each information processing device 2 has common blockchain data.
  • the control unit 20 controls a series of processes in the three-dimensional map generation.
  • the control unit 20 includes an image acquisition unit 21, an image use determination unit 22, a three-dimensional map generation unit 23 as a three-dimensional model generation unit, and a block generation unit 24.
  • the image acquisition unit 21 acquires image data to which an unauthenticated status is given from the image / three-dimensional map DB 26.
  • the image use determination unit 22 performs verification work for determining whether or not to approve the image data acquired by the image acquisition unit 21 as a transaction.
  • the verification work result is transmitted to the image / three-dimensional map DB26 and stored.
  • the determination of whether or not to approve as a transaction is a determination of whether or not the image is suitable for generating a three-dimensional map. For example, an overexposed image, an out-of-black image, an out-of-focus image, etc. are not images suitable for three-dimensional map generation because the object to be photographed is unclear. Therefore, the image use determination unit 22 uses these images as transactions. Not approve.
  • the tone of the bright part may be lost and a phenomenon called whiteout may occur in the image.
  • the image use determination unit 22 does not approve the image data as a transaction when the whiteout pixels in which such whiteout occurs are present in a predetermined ratio or more with respect to the number of pixels of the entire image.
  • a part of the screen such as night or shade is extremely dark or the difference in brightness in the screen is large, the tone of the dark part of the captured image may be lost and black crushing may occur, which is a state in which the image appears black. is there.
  • the image use determination unit 22 does not approve the image data as a transaction when the black crushed pixels in which such black crushing occurs are present in a predetermined ratio or more with respect to the number of pixels of the entire image.
  • the whiteout pixel determination and the blackout pixel determination using the luminance threshold value described in JP-A-2008-148180 can be used. Then, if the overexposed pixels or the underexposed pixels are present in a predetermined ratio or more with respect to the number of pixels of the entire image, the image data is not approved as a transaction, and if not, the image data is approved as a transaction.
  • JP-A-2010-217954 can be used, and whether or not the image use determination unit 22 approves the image data as a transaction based on the blur degree detection result. Is determined.
  • the image use determination unit 22 sorts a plurality of image data approved as a transaction into a plurality of groups based on the associated position information.
  • the position accuracy that can be determined by the GPS 42 is about several tens of meters to several hundreds of meters.
  • the image use determination unit 22 outputs the group sorted based on the position information to the three-dimensional map generation unit 23. Since the three-dimensional map generation unit 23 generates a three-dimensional map using a plurality of image data of each group sorted based on the position information, the three-dimensional map can be efficiently generated.
  • the image use determination unit 22 may determine whether or not to use the image data sorted based on the position information at the time of generating the three-dimensional map. For example, the image use determination unit 22 may determine that the image data having a high degree of similarity to the image data used at the time of generating the three-dimensional map is not used. For example, when multiple images of the same scene are shot continuously from the same place without moving in the same shooting direction with one mobile terminal, the obtained image is statistical information such as brightness histogram, average brightness value, and color distribution. It is possible to obtain images with a high degree of similarity with almost the same composition with little change in. When there are a plurality of image data having such a high degree of similarity, not all the image data is required for the three-dimensional map generation.
  • the image use determination unit 22 determines that the image data having a high degree of similarity to the image data used when generating the three-dimensional map is recognized as a transaction but is not used for the three-dimensional map generation. .. This makes it possible to prevent the occurrence of excessive uploads by the end user for the purpose of reward. For example, if the end user who provided the image data is paid according to the number of uploaded images regardless of whether or not it is used when generating the three-dimensional map, the end user has a high degree of similarity to the reward purpose. There is a possibility to upload multiple image data.
  • Whether or not the similarity is high is determined by, for example, how much the image data contributes to the 3D map generation by adding the image data at the time of generating the 3D map. More specifically, how much the number of 3D points increases when the 3D map is generated by adding the image data to be used for comparison to the image data to be compared that is determined to be used when the 3D map is generated. And so on. If the amount of increase in the three-dimensional points is a certain amount or more, the similarity can be determined to be low, and if it is lower than a certain amount, the similarity can be determined to be high. In addition, for the image data to be compared and the image data for use determination, the respective brightness histograms, average brightness, color distribution, image frequency, edge strength, etc. are acquired as reference parameters, and the similarity is obtained by comparing these. You may judge. In addition, the degree of similarity may be determined by extracting the points of interest of the image data to be compared and the image data to be determined to be used, and comparing them
  • the image use determination unit 22 may determine that the image data having almost no similarity to the image data used when generating the three-dimensional map is not used. For example, even if two end users shoot at the same position, if one shooting direction is east and the other shooting direction is west, and so on, the shot images have the same position information. Even so, the images taken by the two are completely different, and the image data has almost no similarity. That is, even if there are a plurality of image data sorted into the same group based on the position information, the shooting directions may be opposite.
  • the image use determination unit 22 compares a plurality of image data having the same position information, detects a common landmark, for example, and for the image data on which the same landmark is projected, the landmark and its surroundings are three-dimensional. It is determined that the image data used for map generation and other image data are not used for three-dimensional map generation of the landmark and its surroundings as image data having almost no similarity. The image determined not to be used can be used to generate a three-dimensional map of another area.
  • one group sorted based on the position information is further divided into a plurality of small groups by the above-mentioned method using landmarks, etc., and each small group is targeted by using the image data belonging to each small group. You may want to generate a three-dimensional model of the area of.
  • the image use determination unit 22 has insufficient information for three-dimensional map generation. If there is, it may be determined that the three-dimensional map generation using the image data classified into the group is not performed at this time.
  • the three-dimensional map generation unit 23 is portable when each image data is acquired by using a plurality of image data approved as a transaction by the image use determination unit 22 and determined to be used for three-dimensional map generation. After estimating the position and shooting direction of the terminal 4, a fragmentary three-dimensional map is generated and reconstructed based on the plurality of image data. The reconstructed and updated fragmentary 3D map is stored in the image / 3D map DB26. A reward is paid to the end user who provided the image data used when generating the 3D map. That is, the determination result by the image use determination unit 22 determines whether or not the end user is rewarded.
  • Existing technology can be used to generate a three-dimensional map using a plurality of image data.
  • an image-based 3D modeling method that restores the three-dimensional information contained in the image from one or a plurality of images or a time-series image to create a three-dimensional model can be used.
  • Image-based 3D modeling captures the optical image of a 3D scene mapped to a 2D plane as image data, and reverse-maps the 2D image data to the 3D scene to create the original 3D scene. It restores the information of.
  • FIG. 4 is a conceptual diagram illustrating an example of generating a three-dimensional model using a plurality of images. As shown in FIG. 4, the shooting position and shooting direction in which each image was acquired are estimated from the plurality of images 1, image 2, and image 3, and the images 1, image 2, and images taken from each estimated shooting position are estimated. A three-dimensional model of the object A that satisfies all of 3 is generated.
  • the block generation unit 24 generates a new block 51 to be connected to the blockchain 5, and performs a block generation process to connect the new block 51 to the blockchain 5. Further, the block generation unit 24 verifies and approves whether or not the nonce value discovered by the other information processing device (minor) 2 is correct.
  • the block generation unit 24 stores the image data and position information (transaction data) associated with each other used at the time of generating the three-dimensional map in the new block 51, and is a nonce value for connecting to the previous block 51.
  • the nonce value is found, it is verified by another information processing device (minor) 2, and when it is approved, the block generation unit 24 stores the destination of the mining reward in the new block 51, and the blockchain 5 is new.
  • the blocks 51 are connected.
  • the destination of the mining reward is the information of the miner who found the nonce value, and when the new block 51 is connected to the immediately preceding block 51, the miner who found the nonce value can receive the reward.
  • the hash value of the previous block (the previous hash value in the figure), the nonce value for connecting to the previous block 51, the transaction data (image data and position information linked to each other), and the reward.
  • the destination is stored.
  • the new block 51 is concatenated, and the updated blockchain data is stored in the recording ledger DB 27.
  • the information processing device 2 performs image data use determination processing, three-dimensional map generation processing using image information, and mining processing for recording image data used at the time of three-dimensional map generation on a blockchain.
  • FIG. 5 is a flow chart of information processing in the information processing device 2.
  • 6 to 8 are examples of display screens displayed on the mobile terminal 4 owned by the end user.
  • image information (image data and position information linked to each other) is acquired by the mobile terminal 4 possessed by the end user.
  • the example shown in FIG. 6 is a schematic plan view of the mobile terminal 4 in which the images 47A to 47C acquired by the different mobile terminals 4A to 4C are projected on the display unit 44. Both images are images of the exterior of the same hotel, and were taken from different shooting directions.
  • the image 47A acquired by the mobile terminal 4A shown in FIG. 6A is a photograph of the front of the hotel from the front side of the hotel.
  • the image 47B acquired by the mobile terminal 4B shown in FIG. 6B is taken at a position higher than that of the mobile terminal 4A, and is taken by looking down at the hotel from an oblique left.
  • the image 47C acquired by the mobile terminal 4C shown in FIG. 6C was acquired at the same altitude position as the mobile terminal 4A, and was taken by looking up at the hotel from an oblique right side.
  • a comment asking whether or not to upload the image / three-dimensional map DB 26 is displayed.
  • the end user inputs whether or not to upload according to the comment.
  • the image information is stored in the image / three-dimensional map DB 26 of each information processing device 2.
  • the image acquisition unit 21 acquires the image information (image data and position information associated with each other) stored in the image / three-dimensional map DB 26 (S1).
  • the image use determination unit 22 performs verification work of the acquired image data and determines whether or not to approve it as a transaction, and further, whether or not to use the image approved as a transaction for three-dimensional map generation. Is determined (S2).
  • the three-dimensional map generation unit 23 generates a three-dimensional map based on the image determined to be used for the three-dimensional map generation (S3).
  • the block generation unit 24 has already generated a three-dimensional map by another information processing device 2 and a nonce value has been found, the nonce value is verified and approved.
  • the block generator 24 When the nonce value is first discovered by itself generating a three-dimensional map, the block generator 24 generates a new block after being verified and approved by another information processing device 2, and blocks this new block. It is connected to the chain (S4).
  • the updated fragmentary three-dimensional map is stored in the image / three-dimensional map DB 26 of the information processing device 2.
  • the updated blockchain data is stored in all the recording ledgers DB27 of the information processing device (minor) 2 participating in the three-dimensional map generation.
  • the information about the image used for the three-dimensional map generation processing is recorded in the blockchain, and it is difficult to be tampered with retroactively, so that a highly reliable three-dimensional map can be generated.
  • FIG. 7 is an example of a display screen of the mobile terminal 4 when one user uploads, for example, three images.
  • the shooting position information, the shooting date / time information, and the reward information associated with the image are displayed on the right side of each image uploaded by the end user.
  • the reward amount is displayed in the reward information column.
  • the image in the middle of FIG. 7 is an example of an image that has a high degree of similarity to the top image and was not used for three-dimensional map generation. In this case, since the reward is not paid, the reward amount of 0 yen is displayed as the reward information.
  • the image at the bottom of FIG. 7 shows an example during usage determination by the image usage determination unit 22, and the characters processing are displayed in the reward information column. In this way, the end user can confirm the presence or absence of the reward according to the determination result of the image use determination unit 22 from the display image of the mobile terminal 4.
  • the three-dimensional map generation unit 23 estimates the position and shooting direction (attitude) of the mobile terminal 4 when each image data is acquired based on the plurality of image data at the time of generating the three-dimensional map. ..
  • FIG. 8 is an example of a confirmation screen of the three-dimensional map generation result.
  • a figure showing the area where the three-dimensional model was generated this time the position of the mobile terminal 4 when the image data used for generating the area was acquired, and the shooting direction (attitude) on the three-dimensional map ( White circles and triangles) are superimposed and displayed.
  • the white circles indicate the shooting positions of the mobile terminals 4A to 4C when the captured images 47A to 47C shown in FIG.
  • the embedded triangle indicates the shooting direction.
  • the shooting direction is the direction from the apex in contact with the white circle toward the opposite side of the apex.
  • the positions and shooting directions (attitudes) of these mobile terminals 4A to 4C are estimated based on a plurality of image data acquired by each mobile terminal when the three-dimensional map is generated.
  • the area filled with dots shown on the front side of the hotel indicates an area in which a three-dimensional image is generated using the image data acquired by the mobile terminals 4A to 4C.
  • the estimated shooting position and shooting direction are displayed graphically, and the area where the three-dimensional map is generated can be seen from the image data.
  • the confirmation screen can be confirmed by the end user, for example.
  • the number of images used for three-dimensional map generation is set to three for convenience, the number is not limited to this.
  • a highly reliable three-dimensional map is generated by the end user group and the minor group using the blockchain, and the user of the three-dimensional map uses the three-dimensional map to generate the area. You can grasp the situation. Further, the three-dimensional map can be updated at any time, and the user of the three-dimensional map can grasp the change in the situation of the area at any time by using the three-dimensional map.
  • a construction company which is a user of a 3D map, can grasp changes in the situation over time such as the number of materials, earth and sand, and the shape of a building by regularly monitoring the 3D map of a construction site. , Can perform accurate and quick construction work.
  • the blockchain is used to record the image data used at the time of generating the three-dimensional map and the minor and the end user who contributed to the generation of the three-dimensional map. It is possible to grasp the reward status to the end user. It should be noted that the effects described in the present specification are merely examples and are not limited, and other effects may be obtained.
  • the image information uploaded from the end user is image data with position information, but in addition to this, attitude information obtained from, for example, a geomagnetic sensor is uploaded. It may be included in the image information.
  • attitude information obtained from, for example, a geomagnetic sensor is uploaded. It may be included in the image information.
  • the plurality of image data can be grouped for each shooting direction by using the posture information, and the work of removing images having low similarity can be reduced. It can be unnecessary. As a result, a three-dimensional map can be efficiently generated.
  • the attitude information may be acquired by using SLAM (Simultaneous Localization and Mapping) that calculates the self-position from the feature point information of the captured image. This eliminates the need for estimating the shooting direction (posture) of the mobile terminal using a plurality of image data when generating a three-dimensional map.
  • SLAM Simultaneous Localization and Mapping
  • the position information acquisition unit may be acquired by using Wi-Fi (registered trademark).
  • each information processing device (minor) has an image / three-dimensional map DB
  • a plurality of groups including a plurality of information processing devices (minors) may be provided, and each group may share one image / three-dimensional map DB.
  • the image information and the fragmentary three-dimensional map generated by each of the information processing devices (miners) belonging to the same group are stored in one image / three-dimensional map DB.
  • each information processing device (minor) may have a DB for storing image information, and the three-dimensional map may be stored in one DB shared by a plurality of information processing devices (minor).
  • the DB in which the three-dimensional map is stored may be managed by, for example, a group 3 that uses the three-dimensional map.
  • a reward is paid to the miner who discovers the nonce value, and the reward is paid to the end user who provided the image used for the 3D map generation.
  • image data approved as a transaction may be set so that a reward is paid even if the image data has a high degree of similarity and is not used for three-dimensional map generation.
  • the reward amount is set lower than the reward for the image data used for generating the three-dimensional map.
  • the reward for the image data approved as such a transaction but not used for the 3D map generation may be stopped.
  • the timing of stopping the reward may be set in advance for a limited period such as 3 months from the start of map generation, or may be set according to the degree of completion of 3D map generation.
  • a reward is paid to the miners, and the reward may be distributed among the miners.
  • one blockchain data is configured to exist in one group 3, but the present invention is not limited to this.
  • a blockchain may be generated for each village, and a plurality of blockchains may be generated for each municipality for one group 3. You may.
  • the information processing device 2 is configured to determine an area where image data for generating a three-dimensional map is insufficient based on the generated three-dimensional map, and notify the end user of the area. May be good. As a result, it is possible to encourage the end user to actively acquire the image data in the area where the image data is insufficient, and it is possible to promote the generation of the three-dimensional map.
  • An image acquisition unit that acquires image information having image data linked to each other and position information at the time of acquisition of the above image data, and an image acquisition unit.
  • a 3D model generator that generates a 3D model using the above image information
  • An information processing device including a block generator for generating a new block connected to a blockchain formed by connecting blocks for recording image information used when generating the above three-dimensional model.
  • the three-dimensional model generation unit is an information processing device that generates a three-dimensional model using image information approved as the transaction.
  • the image use determination unit is an information processing device that determines whether or not to use the image information approved as a transaction in the generation of a three-dimensional model. (4) The information processing device according to (3) above. Whether or not the image usage determination unit uses the image data of the usage determination target for the three-dimensional model generation based on the similarity of the image data of the usage determination target to the image data used for the three-dimensional model generation. Information processing device that determines. (5) The information processing device according to any one of (2) to (4) above. The image use determination unit is an information processing device that sorts a plurality of the above image information using the above position information. (6) The information processing device according to (5) above.
  • the three-dimensional model generation unit is an information processing device that generates the three-dimensional model using a plurality of the image information sorted by using the position information. (7) Acquire unauthenticated image information having image data linked to each other and position information at the time of acquisition of the above image data. A three-dimensional model is generated using the above image information, An information processing method for generating a new block connected to a blockchain formed by connecting blocks in which image information used when generating the above three-dimensional model is recorded.
  • the three-dimensional model generation unit that generates a three-dimensional model using the image data associated with each other and the image information having the position information at the time of acquiring the image data, and the image information used at the time of generating the three-dimensional model are
  • An information processing system including a plurality of information processing devices, including a block generation unit that generates a new block connected to a block chain formed by connecting recorded blocks.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Accounting & Taxation (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • Geometry (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Graphics (AREA)
  • Finance (AREA)
  • Tourism & Hospitality (AREA)
  • Evolutionary Computation (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Medical Informatics (AREA)
  • Remote Sensing (AREA)
  • Economics (AREA)
  • Human Resources & Organizations (AREA)
  • Marketing (AREA)
  • Primary Health Care (AREA)
  • Processing Or Creating Images (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
PCT/JP2020/026527 2019-08-21 2020-07-07 情報処理装置、情報処理方法、プログラム及び情報処理システム Ceased WO2021033443A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US17/635,085 US12002158B2 (en) 2019-08-21 2020-07-07 Information processing apparatus, information processing method and information processing system
JP2021540657A JP7559761B2 (ja) 2019-08-21 2020-07-07 情報処理装置、情報処理方法、プログラム及び情報処理システム
CN202080057389.7A CN114270404A (zh) 2019-08-21 2020-07-07 信息处理装置、信息处理方法、程序和信息处理系统

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019-151196 2019-08-21
JP2019151196 2019-08-21

Publications (1)

Publication Number Publication Date
WO2021033443A1 true WO2021033443A1 (ja) 2021-02-25

Family

ID=74661096

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/026527 Ceased WO2021033443A1 (ja) 2019-08-21 2020-07-07 情報処理装置、情報処理方法、プログラム及び情報処理システム

Country Status (4)

Country Link
US (1) US12002158B2 (https=)
JP (1) JP7559761B2 (https=)
CN (1) CN114270404A (https=)
WO (1) WO2021033443A1 (https=)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12395313B1 (en) * 2020-08-28 2025-08-19 Response Images, LLC Methods and sensor array platforms for creating and monitoring cryptographic blockchain images and executing programs based on monitored images
CN117853191B (zh) * 2023-12-04 2025-05-06 中交(厦门)电子商务有限公司 一种基于区块链技术的商品交易信息共享方法及系统

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016208347A (ja) * 2015-04-24 2016-12-08 国立大学法人 筑波大学 著作物保護支援装置
WO2019072311A2 (en) * 2018-12-29 2019-04-18 Alibaba Group Holding Limited PARTICIPATORY PRODUCTION BASED ON A CHAIN OF BLOCKS OF MAPPING APPLICATIONS

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10025447B1 (en) * 2015-06-19 2018-07-17 Amazon Technologies, Inc. Multi-device user interface
JP6425353B2 (ja) * 2016-02-01 2018-11-21 三菱マテリアルテクノ株式会社 空中写真測量および三次元立体モデル生成用標識、空中写真測量方法
CN107045650B (zh) * 2016-10-25 2021-06-11 罗轶 一种基于区块链的网约车系统
US20180165585A1 (en) * 2016-12-09 2018-06-14 Cognitive Scale, Inc. Method for Providing Procurement Related Cognitive Insights Using Blockchains
US10663303B2 (en) * 2017-06-12 2020-05-26 Panasonic Intellectual Property Management Co., Ltd. System and method for dynamically authenticating map data using blockchains
KR20190023274A (ko) * 2017-08-28 2019-03-08 주식회사 케이티 블록체인을 이용한 위치 인증 장치 및 방법
JP7180061B2 (ja) * 2017-09-13 2022-11-30 富士フイルムビジネスイノベーション株式会社 情報処理装置、情報処理方法及びプログラム
CN109325368A (zh) * 2018-09-11 2019-02-12 云量区块链(深圳)有限公司 基于区块链的城市实景管理方法、电子设备及存储介质
US11762968B2 (en) * 2021-07-20 2023-09-19 P Tech, Llc Encryption, security, and video optimization

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016208347A (ja) * 2015-04-24 2016-12-08 国立大学法人 筑波大学 著作物保護支援装置
WO2019072311A2 (en) * 2018-12-29 2019-04-18 Alibaba Group Holding Limited PARTICIPATORY PRODUCTION BASED ON A CHAIN OF BLOCKS OF MAPPING APPLICATIONS

Also Published As

Publication number Publication date
JP7559761B2 (ja) 2024-10-02
US20220292769A1 (en) 2022-09-15
JPWO2021033443A1 (https=) 2021-02-25
CN114270404A (zh) 2022-04-01
US12002158B2 (en) 2024-06-04

Similar Documents

Publication Publication Date Title
TWI483215B (zh) 根據相關３ｄ點雲端資料強化影像資料
KR102038127B1 (ko) 무인비행체를 이용한 시설물 관리 장치 및 방법
CN116917929A (zh) 用于遥感中的超分辨率图像处理的系统和方法
US20180262789A1 (en) System for georeferenced, geo-oriented realtime video streams
US10402942B2 (en) Multiple georeferenced aerial image crop analysis and synthesis
KR102129408B1 (ko) 공공 지도 또는 외부 지도와 매칭되는 무인 비행체에 의해 촬영된 이미지의 레이어로부터 측량 데이터를 획득하는 방법 및 장치
KR101871764B1 (ko) 가상현실 기반의 시설관리 플랫폼 구축 시스템
CN114299236B (zh) 倾斜摄影测量空地融合实景建模方法、装置、产品及介质
US10110817B2 (en) Image processing device and method, and program for correcting an imaging direction
Rijsdijk et al. Unmanned aerial systems in the process of juridical verification of cadastral border
KR102548454B1 (ko) 태양광패널의 설치 위치 및 설치 각도 추천 방법
CN108474657A (zh) 一种环境信息采集方法、地面站及飞行器
US20210264666A1 (en) Method for obtaining photogrammetric data using a layered approach
CN112399084A (zh) 无人机航拍方法、装置、电子设备及可读存储介质
CN117197388B (zh) 一种基于生成对抗神经网络和倾斜摄影的实景三维虚拟现实场景构建方法及系统
CN111275396A (zh) 基于无人机采集变更图斑照片的新方法
JP7559761B2 (ja) 情報処理装置、情報処理方法、プログラム及び情報処理システム
US11272081B2 (en) Systems and methods for real-time compositing of video content
KR20100029725A (ko) 촬영영상에서 추출된 촬영객체를 이용한 영상정보 관리 시스템 및 그 방법
CN112665554B (zh) 一种正射影像的生成方法与系统
CN108108396B (zh) 一种飞行器航拍图片拼接管理系统
CN109034214B (zh) 用于生成标记的方法和装置
CN113470051B (zh) 图像分割方法、计算机终端及存储介质
CN119071431B (zh) 无人机全景巡检方法、装置、系统及电子设备
Spicer et al. Producing usable simulation terrain data from UAS-collected imagery

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20853945

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021540657

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20853945

Country of ref document: EP

Kind code of ref document: A1