JP7228298B1 - Information processing system, information processing method, program, mobile object, management server - Google Patents

Abstract

An information processing capable of generating result data according to comparison data between 3D point cloud model data representing at least components in a structure and 3D point cloud sensing data acquired by a moving object. Provide systems, etc.
Kind Code: A1 An information processing system or the like of the present invention presents three-dimensional point cloud model data obtained by converting at least three-dimensional model data relating to components in a structure into point groups, and results of sensing inside the structure by a sensor. a comparison unit that generates comparison data by comparing the three-dimensional point cloud sensing data with each other; and a result data generation unit that generates result data based on the comparison data. The three-dimensional point cloud model data is generated by moving a virtual sensor within the three-dimensional model data to obtain point cloud data, thereby forming a point cloud.
[Selection drawing] Fig. 1

Description

The present invention relates to an information processing system, an information processing method, a program, a mobile object, and a management server.

In recent years, drones and unmanned aerial vehicles (UAV: Unmanned Aerial Vehicles)
l Vehicle) and other flying vehicles (hereinafter collectively referred to as "flying vehicles") and unmanned ground vehicles (U
Autonomously controllable mobile bodies such as GV (Unmanned Ground Vehicle) have begun to be used in industry. Under these circumstances, in Patent Document 1, LIDAR
discloses a system that acquires a point cloud of the surrounding environment and flies indoors.

Japanese Patent Application Laid-Open No. 2020-131930

However, in Patent Document 1, the point cloud data acquired by the flying object is an operator terminal (
Although it is disclosed that the data is transferred to the user terminal), effective use of the point cloud data is not described.

The present invention has been made in view of such a background. It is an object of the present invention to provide an information processing system or the like capable of generating result data.

An information processing system or the like of the present invention for solving the above problems includes three-dimensional point group model data obtained by converting at least three-dimensional model data relating to components in a structure into point groups, and sensors sensing the inside of the structure. 3D point cloud sensing data indicating a result, a comparison unit that generates comparison data by comparing them with each other, and a result data generation unit that generates result data based on the comparison data.

According to the present invention, in particular, information capable of generating result data according to comparison data between at least three-dimensional point cloud model data representing components in a structure and three-dimensional point cloud sensing data acquired by a moving object. A processing system and the like can be provided.

1 is a diagram showing the configuration of an information processing system according to an embodiment of the present invention; FIG. 2 is a block diagram showing the hardware configuration of a management server in FIG. 1; FIG. 2 is a block diagram showing the hardware configuration of the user terminal in FIG. 1; FIG. FIG. 2 is a block diagram showing the hardware configuration of the moving body in FIG. 1; FIG. 2 is a block diagram showing functions of a management server in FIG. 1; FIG. FIG. 2 is a block diagram showing functions of the moving body of FIG. 1; It is a flow chart of the information processing method concerning an embodiment of the invention. It is a display example of three-dimensional model data according to the embodiment of the present invention. It is a display example of three-dimensional point cloud model data according to the embodiment of the present invention. It is a display example of three-dimensional point group sensing data according to the embodiment of the present invention. It is an example of a display at the time of fitting three-dimensional data according to the embodiment of the present invention. It is a display example of comparison data according to the embodiment of the present invention.

The contents of the embodiments of the present invention are listed and explained. An information processing system or the like according to an embodiment of the present invention has the following configuration.
[Item 1]
A comparison in which three-dimensional point group model data obtained by converting at least three-dimensional model data relating to components within a structure into a point group and three-dimensional point group sensing data indicating the results of sensing the inside of the structure by a sensor are compared with each other. a comparison unit that generates data;
a result data generation unit that generates result data based on the comparison data;
An information processing system characterized by:
[Item 2]
The three-dimensional point cloud model data is generated by moving a virtual sensor within the three-dimensional model data to acquire point cloud data,
The information processing system according to claim 1, characterized by:
[Item 3]
The comparison data includes difference information between the 3D point cloud model data and the 3D point cloud sensing data.
3. The information processing system according to claim 1 or 2, characterized in that:
[Item 4]
further comprising a data update unit that generates update data for the three-dimensional model data based on the difference information;
4. The information processing system according to claim 3, characterized by:
[Item 5]
The data updating unit estimates a structure according to the difference information, and generates the update data for changing structure information indicating the structure in the three-dimensional model data.
5. The information processing system according to claim 4, characterized by:
[Item 6]
If the data updating unit can estimate that a predetermined structure exists but cannot estimate the structure information of the structure, the unknown model data indicating that placed on the data,
6. The information processing system according to claim 5, characterized by:
[Item 7]
If the data updating unit can estimate the existence of a predetermined structure but cannot estimate the structure information of the structure, the unknown model data of the shape estimated based on the difference information is updated to the tertiary model data. placed on the original model data,
6. The information processing system according to claim 5, characterized by:
[Item 8]
The texture of the unknown model data is a photographed image acquired when sensing by the sensor,
8. The information processing system according to claim 6 or 7, characterized by:
[Item 9]
The comparison data is self-position estimation information in the three-dimensional point cloud model data estimated based on self-position estimation information in the three-dimensional point cloud sensing data.
9. The information processing system according to any one of claims 1 to 8, characterized by:
[Item 10]
3D point cloud model data obtained by converting at least 3D model data of components in a structure into a point group, and 3D point cloud sensing data indicating the result of sensing the inside of the structure by a sensor. generating comparative data compared to each other;
generating result data based on the comparison data by a result data generator;
including,
An information processing method by a computer, characterized by:
[Item 11]
3D point cloud model data obtained by converting at least 3D model data of components in a structure into a point group, and 3D point cloud sensing data indicating the result of sensing the inside of the structure by a sensor. generating comparative data compared to each other;
generating result data based on the comparison data by a result data generator;
A program characterized by causing a computer to execute
[Item 12]
A comparison in which three-dimensional point group model data obtained by converting at least three-dimensional model data relating to components within a structure into a point group and three-dimensional point group sensing data indicating the results of sensing the inside of the structure by a sensor are compared with each other. a comparison unit that generates data;
a result data generation unit that generates result data based on the comparison data;
A moving body characterized by:
[Item 13]
A comparison in which three-dimensional point group model data obtained by converting at least three-dimensional model data relating to components within a structure into a point group and three-dimensional point group sensing data indicating the results of sensing the inside of the structure by a sensor are compared with each other. a data update unit that generates update data for the three-dimensional model data based on the data;
A management server characterized by:

<Details of Embodiment>
Embodiments of an information processing system, an information processing method, a program, a mobile object, and a management server according to embodiments of the present invention will be described below. In the accompanying drawings, the same or similar elements are denoted by the same or similar reference numerals and names, and duplicate descriptions of the same or similar elements may be omitted in the description of each embodiment. Also, the features shown in each embodiment can be applied to other embodiments as long as they are not mutually contradictory.

<Configuration>
As shown in FIG. 1, the information processing system in this embodiment includes a management server 1,
It has one or more user terminals 2 , one or more mobile bodies 4 (for example, flying bodies or running bodies), and one or more mobile body storage devices 5 . The management server 1, the user terminal 2, the mobile unit 4, and the mobile unit storage device 5 are connected so as to be able to communicate with each other via a network. Note that the illustrated configuration is only an example, and the configuration is not limited to this, and for example, a configuration that is carried by the user without having the mobile unit storage device 5 may be employed.

<Management Server 1>
FIG. 2 is a diagram showing the hardware configuration of the management server 1. As shown in FIG. Note that the illustrated configuration is an example, and other configurations may be employed.

As illustrated, a management server 1 is connected to a user terminal 2, a mobile unit 4, and a mobile unit storage device 5 to form part of this system. The management server 1 may be, for example, a general-purpose computer such as a workstation or personal computer, or may be logically realized by cloud computing.

The management server 1 includes at least a processor 10 , a memory 11 , a storage 12 , a transmission/reception section 13 , an input/output section 14 and the like, which are electrically connected to each other through a bus 15 .

The processor 10 is an arithmetic device that controls the overall operation of the management server 1, controls transmission and reception of data between elements, executes applications, and performs information processing necessary for authentication processing. For example, the processor 10 is a CPU (Central Processing Unit).
it) and/or a GPU (Graphics Processing Unit), which executes programs for this system stored in the storage 12 and developed in the memory 11 to perform each information processing.

The memory 11 is a DRAM (Dynamic Random Access Memory
y) and other volatile storage devices, and flash memory and HDD (Hard
Auxiliary memory composed of a non-volatile memory device such as a Disc Drive). The memory 11 is used as a work area or the like for the processor 10, and also contains a BIOS (Basic Input/Output System) executed when the management server 1 is started,
and stores various setting information.

The storage 12 stores various programs such as application programs. A database storing data used for each process may be constructed in the storage 12 .

The transmission/reception unit 13 connects the management server 1 to the network. In addition, the transmission/reception unit 13
Bluetooth® and BLE (Bluetooth Low Energy
) near field communication interface.

The input/output unit 14 is an information input device such as a keyboard and a mouse, and an output device such as a display.

A bus 15 is commonly connected to the above elements and transmits, for example, address signals, data signals and various control signals.

<User terminal 2>
The user terminal 2 shown in FIG. 3 also includes a processor 20, memory 21, storage 22,
A transmission/reception unit 23, an input/output unit 24, and the like are provided, and these are electrically connected to each other through a bus 25. FIG. Since the function of each element can be configured in the same manner as the management server 1 described above, detailed description of each element will be omitted.

<Moving body 4>
The moving object 4 is a known moving object including flying objects such as drones and unmanned aerial vehicles, and traveling objects such as unmanned ground vehicles, and particularly autonomously controllable moving objects. As a specific example of the moving body 4, the traveling body 4 will be described below. FIG. 4 is a block diagram showing the hardware configuration of the running body 4. As shown in FIG. Controller 41 may have one or more processors, such as programmable processors (eg, central processing units (CPUs)).

Also, the controller 41 is an arithmetic device that controls the operation of the entire mobile body 4, controls transmission and reception of data between elements, executes applications, performs information processing necessary for authentication processing, and the like. The controller 41 has a memory 411 and can access the memory. Memory 411 stores logic, code, and/or program instructions executable by controller 41 to perform one or more steps. The controller 41 may also include sensors 412 such as an inertial sensor (acceleration sensor, gyro sensor), GPS sensor, proximity sensor (for example, rider).

Memory 411 may include, for example, separable media or external storage devices such as SD cards and random access memory (RAM). Data acquired from cameras/sensors 42 may be communicated directly to memory 411 and stored. For example, still image/moving image data captured by a camera or the like may be recorded in the built-in memory or the external memory. , the user terminal 2, or the mobile storage device 5. The camera 42 may be installed on the traveling body 4 via a gimbal.

The controller 41 includes a control module (not shown) configured to control the state of the vehicle 4 . For example, the control module may determine the spatial arrangement, velocity, and/or of the vehicle.
Alternatively, in order to adjust the acceleration, the traveling part of the traveling body (motor 45, wheels 46, etc.) is controlled. A motor 45 powered by a battery 48 rotates a wheel 46 to generate a propulsion force for the vehicle. A control module can control one or more of the states of the cameras/sensors 42 . Then, it is possible to configure so as to perform autonomous movement control based on movement route information stored in a movement route information storage unit 480, which will be described later. In the example of FIG. 4, the motor 45 is provided for each wheel 46, but instead of this, it may be common for two front and rear wheels (that is, two motors 45 for front and rear), or four front and rear wheels. The wheels may be common (that is, one motor 45), and these are not limited.

The controller 41 controls one or more external devices (e.g., transmitter/receiver (propo) 49,
data from the management server 1, terminal, display device or other remote controller) and/or
Alternatively, it can communicate with the transmitting/receiving unit 47 configured to receive. Transceiver 49 may use any suitable means of communication, such as wired or wireless communication.

For example, the transceiver 47 utilizes one or more of a local area network (LAN), a wide area network (WAN), infrared, wireless, WiFi, point-to-point (P2P) networks, telecommunications networks, cloud communications, and the like. can do.

The transmitting/receiving unit 47 transmits and/or can receive.

The cameras/sensors 42 according to the present embodiment include inertial sensors (acceleration sensors, gyro sensors), GPS sensors, proximity sensors (for example, LiDAR (Light Detect
on And Ranging), or a vision/image sensor (eg, camera).

5 and 6 are block diagrams illustrating functions implemented in the management server 1 and the mobile unit 4, respectively. In the embodiment of the present invention, for example, three-dimensional 3D point cloud model data obtained by converting model data (e.g., BIM data, CAD data, etc.) into point groups, and 3D point cloud sensing data (e.g., SLAM data, etc.) that indicates the results of sensing inside a structure using sensors. are compared with each other to generate comparison result information, and to generate result data according to the comparison result information.

In this embodiment, the management server 1 includes a communication unit 110, a 3D point cloud model data generation unit 120, a data update unit 130, an information output unit 140, a 3D model data storage unit 150.
, and an acquired data storage unit 160 . Furthermore, the moving body 4 includes a communication unit 410, a three-dimensional point cloud sensing data acquisition unit 420, a comparison unit 430, a comparison result data transmission unit 440, a self-position estimation unit 450, a three-dimensional data storage unit 460, and a generated data storage unit 470. , and a moving route information storage unit 480 . Note that various functional units may be implemented by the processor 10 or the controller 41 according to the capabilities of the processor 10 of the management server 1 or the controller 41 of the mobile unit 4.
You may implement|achieve in any structure of.

The communication unit 110 communicates with the user terminal 2 , the mobile unit 4 and the mobile unit storage device 5 . The communication unit 110 also functions as a reception unit that receives various requests, data, and the like from the user terminal 2 and the like.

The 3D point cloud model data generation unit 120 converts 3D model data including structure information (including dimension information) about structures in the structure and layout information about the layout of the structures into Generate 3D point cloud model data. The three-dimensional model data and the three-dimensional point cloud model data are data represented by a three-dimensional orthogonal coordinate system with an arbitrary set point as the origin (reference point). stored and managed.

The three-dimensional model data is a three-dimensional model data that shows the arrangement of components in a structure and has dimensional information, created based on data created with CAD (Computer-Aided Design) design software, etc. any data,
For example, BIM (Building Information Modeling) data, CIM (Construction Information Modeling) data
data, CAD data, GML data such as City-GML, 3D model data reconstructed from BIM data, etc. original model data, etc.), three-dimensional model data reconstructed by combining multiple types of three-dimensional model data (for example, BIM data and City-GML data, etc.), or two-dimensional design It may be three-dimensional model data obtained by generating a structure having a predetermined height based on diagram data.

The method of generating the 3D point cloud model data may be any method as long as it is possible to generate 3D point cloud model data related to components within the structure from the 3D model data. By moving a virtual mobile object 4 equipped with a virtual sensor (for example, a virtual LiDAR) within the structure of the model data, 3D point cloud model data relating to the components within the structure may be generated. As a result, in theory, it is possible to generate point cloud data close to the point cloud sensing data when the inside of the structure is actually measured by the sensor of the moving body 4 . Another method for generating 3D point cloud model data is to convert the 3D model data into point groups evenly at predetermined intervals. may be arranged to form point groups, or point groups may be formed using a known point group forming technique (conversion technique to point group data).

Here, in view of the flow of processing in this embodiment, the functional units of the moving body 4 will be explained once.

The communication unit 410 communicates with the management server 1 , the user terminal 2 and the mobile storage device 5 . The communication unit 410 also functions as a reception unit that receives various requests, data, etc. from the user terminal 2 or the like.

The three-dimensional point group sensing data acquisition unit 420 is a three-dimensional point group sensing data generated by actually measuring the inside of the structure that is the target of the three-dimensional model data with a sensor (for example, LiDAR) mounted on the moving body 4. Get data. The three-dimensional point group sensing data is data represented by a three-dimensional orthogonal coordinate system with an arbitrary set point as the origin (reference point), and is stored and managed in the three-dimensional data storage unit 460. .

The comparison unit 430 compares, for example, the 3D point cloud model data generated by the 3D point cloud model data generation unit 120 and the 3D point cloud sensing data, and generates comparison data. The three-dimensional model data is stored in the three-dimensional data storage unit 460 from the management server 1 in advance and managed, and the comparison data is stored in the generated data storage unit 470 and managed, for example.

The comparison method may be any method as long as comparison data can be generated by comparing point groups, and may be an existing technique for comparison between point groups. Perform fitting of 3D point cloud data (including scale adjustment if the scales of both 3D point cloud data are different), and compare 3D point cloud sensing data based on points of 3D point cloud model data, for example. It is also possible to generate comparison data for points with a predetermined deviation width, or for points where the presence or absence of points is different in the first place. The space is separated by voxels of different heights, and points of the 3D scan point cloud data exist in the voxels where there were no points in the 3D point cloud model data. good too. That is, the comparison data may be differential point cloud data, which is point cloud data in which the difference between the two is greater than or equal to a predetermined value. As mentioned above, when the scales of both point cloud data are different, fitting is performed so that the scales match. Therefore, it is possible to determine that there is no change in the presence or absence of a point even if there is a deviation of the point.
Errors due to different scales are reduced even if fitting for scale matching is not performed.

Further, the comparison of the point cloud data by the comparison unit 430 may be performed by comparing the 3D point cloud sensing data after moving along a predetermined movement route with the 3D point cloud model data. It may be a comparison between the acquired partial 3D point cloud sensing data and the 3D point cloud model data. In the latter case of comparison, the comparison data may include self-position estimation information in the 3D point cloud model data estimated based on self-position estimation information in the 3D point cloud sensing data.

The result data transmission unit 440 generates result data based on the comparison data generated by the comparison unit 430 via the communication unit 410, and transmits the result data to the outside (especially the management server 1 and the user terminal 2). The result data may be, for example, the comparison data used as it is, or the comparison data edited by deleting unnecessary information (for example, information whose deviation width is smaller than a predetermined value) from the comparison data. There may be.

Here, the functional units of the management server 1 will be described again in view of the flow of processing in this embodiment.

The data update unit 130 generates update data for updating the 3D model data based on the result data received from the moving object 4, and updates the 3D model data according to the update data. The result data may be stored and managed in the acquired data storage unit 160 . More specifically, the data updating unit 130 uses, for example, a learning model for estimating structure information in the 3D model data from the shape of the point cloud, or uses a learning model to estimate the shape of the structure in the 3D model data. Based on the point cloud data of the result data (especially the difference point cloud data), the configuration information of the configuration that has changed ( In particular, in addition to estimating what the component is (such as component identification information and component type information), it is also possible to determine which component has changed in comparison with the 3D model data (number of components). increase/decrease or change of location). Then, based on this determination result, the data updating unit 130 adjusts the structure information of the three-dimensional model data to change, for example, adds or deletes the model data of the structure for which the change is determined, or changes the position of the structure. to change or update. As a result, it is possible to update the three-dimensional model data to reflect the state of the components in the structure at the time of sensing by the moving body 4 .

Note that the data updating unit 130 updates the configuration (configuration information) based on the result data as described above.
When estimating , it is not possible to estimate the component information (especially information that identifies the type of the component, such as component identification information and component type information), but it is possible to estimate that some component exists. In such a case, for example, unknown model data of a predetermined shape indicating that the estimation could not be performed (for example, a three-dimensional shape such as a cylinder or square prism) may be arranged on the three-dimensional model data. However, the unknown model data of the shape (including approximate shape) of the unknown structure estimated based on the point cloud data of the result data (especially the difference point cloud data) may be placed on the three-dimensional model data. , the difference point cloud data (particularly, the point cloud data increased as a result of the comparison) may be superimposed on the 3D model data, for example, by coloring to indicate that the point cloud could not be estimated. Furthermore, the textures of the unknown model data mentioned above are treated as textures that are different from other components (for example, textures with conspicuous single colors, textures with symbols such as question mark, text "unknown", etc.) Alternatively, for example, a photographed image of the position of an unknown structure obtained when point cloud data is obtained by sensing with a sensor may be used as the texture of the unknown model data. As a result, when checking the 3D model data, the locations that need to be confirmed on site become clear.

The information output unit 140 outputs three-dimensional model data, three-dimensional point cloud model data, three-dimensional point cloud sensing data stored in the three-dimensional data storage unit 150, result data and photographed data stored in the acquired data storage unit 160 ( output information to be transmitted to the user terminal 2 or the moving body 4 based on the moving route information and self-location information stored in the moving-related information storage unit 170; In the present embodiment, for example, output information for displaying to visualize three-dimensional model data and movement route information on the user terminal 2, output information for displaying the current position based on the current self-position information, acquisition Browsing 3D data (for example, 3D model data or 3D point cloud data) with marks such as symbols that serve as a link for browsing acquired information corresponding to the position associated with the location information of the information. It may generate output information for enabling. Then, by receiving an operation input for selecting the link on the user terminal 2 on the user terminal 2, the corresponding obtained information may be displayed. At this time,
When using 3D data related to BIM data as 3D data, it is possible to have information for each component that constitutes 3D data in association with BIM data, so it is possible to link each component (for example, link generation) becomes possible.

An information processing method (particularly, a method for updating three-dimensional model data) according to the present embodiment will be described with reference to FIG. FIG. 7 illustrates a flowchart of information processing according to this embodiment. 8-11 illustrate each three-dimensional data.

First, the management server 1 converts 3D model data (see FIG. 8, for example, 3D model data generated from BIM data) about the structure of the structure into point groups by the 3D point cloud model data generation unit. Three-dimensional point cloud model data (see FIG. 9) is generated (S101). Then, the generated 3D point cloud model data is transmitted to the moving body 4 via the communication unit 101 (
S102). The 3D point cloud model data received by the mobile object 4 may be stored in the 3D data storage unit 460, for example. Note that S102 may be executed at any timing as long as it is executed before the comparison process is performed in the moving body 4 .

Next, the moving body 4 stores the movement-related information storage unit 17 in order to actually measure the inside of the structure using a sensor.
Based on the movement route information stored in 0 and the control of the controller 41, autonomous movement is started (S103). Then, the moving body 4 obtains the three-dimensional point cloud sensing data generated by the sensor (for example, LiDAR) during movement by the three-dimensional point cloud sensing data acquisition unit 420 (as illustrated in FIG. 10, (S104). It should be noted that the moving route information is, for example, the user terminal 2
It may be information related to a movement route generated by a user by means of a controller, or information storing a movement route obtained by moving the mobile body 4 using an external controller such as a propo. In addition, moving body 4
The movement of is not limited to the autonomous movement described above, and may be made to move in accordance with the input operation of the user terminal 2 or an external controller.

Next, the moving object 4 compares the acquired 3D point cloud sensing data with the 3D point cloud model data to generate comparison data during at least either during movement or after the end of movement by the comparison unit 430. (S105). During movement, the comparison data is obtained by comparing the partial 3D point cloud sensing data and the 3D point cloud model data, and the 3D point cloud estimated based on the self-position estimation information in the 3D point cloud sensing data. Self-position estimation information in the original point cloud model data may be included. In addition, the comparison data is generated by, for example, comparing both three-dimensional point cloud data (both three-dimensional point cloud data are superimposed in the illustration of FIG. 11) after the end of movement, and for example, a predetermined Comparison data on points with a gap between points and points with different presence or absence of points in the first place, that is, point cloud data where the difference between the two is a predetermined amount or more (for example,
Difference point cloud data 600 (see FIG. 12), which is a group of point data that do not exist in the same voxel or newly exist in the same voxel, may be generated. Furthermore, the comparison data may be displayed superimposed on at least one of the 3D point cloud model data and the 3D point cloud sensing data, as illustrated in FIG. 12 .

Next, the moving object 4 transmits the result data transmission unit 440 via the communication unit 410 to the comparison unit 4
30 to generate result data based on the comparison data generated by the external (particularly the management server 1
or user terminal 2) (S107). As described above, the result data may be obtained by using the comparison data as it is, or by deleting unnecessary information (for example, information whose deviation width is smaller than a predetermined value) from the comparison data. It may be edited data. Result data is not limited to data based on single comparison data corresponding to three-dimensional point cloud sensing data sensed from a predetermined position, but data summarizing multiple comparison data at multiple positions sensed while the moving body 4 is moving. (For example, difference point cloud data obtained by superimposing a plurality of difference point cloud data 600 corresponding to a plurality of three-dimensional point cloud sensing data at a plurality of positions). Note that such a summarizing process may be performed in step S107 in the moving body 4, or may be performed in the management server 1 in step S108, which will be described later.

Next, the management server 1 updates the three-dimensional model data by the update unit 130 based on the received result data (S108). The updating is performed by estimating the structure information of the structure that has changed based on the result data, and adjusting the structure information of the 3D model data to the change determined from the result data, for example, updating the structure whose change has been determined. Add, delete, or change the position of model data.

Next, the management server 1 uses the information output unit 140 to transmit output information for displaying at least predetermined information on the user terminal 2 (for example, as described above, display for visualizing three-dimensional model data and moving route information). output information for, output information for displaying the current location based on the current self-location information, a symbol that becomes a link for viewing the acquired information corresponding to the location associated with the location information of the acquired information, etc. output information for enabling viewing of the three-dimensional data marked with ) is generated (S109). The generated output information may then be transmitted to the user terminal 2 or the mobile object 4 .

In this way, the present invention provides information capable of generating result data according to comparison data between at least three-dimensional point cloud model data representing components in a structure and three-dimensional point cloud sensing data acquired by a moving object. A processing system and the like can be provided.

In the above-described embodiment, the sensing inside the structure by the moving body 4 is taken as a specific example. 3D point cloud sensing data may be generated by using other functional units (communication unit 410, 3D point cloud sensing data acquisition unit 420, comparison unit 430
, comparison result data transmission unit 440, self-position estimation unit 450, three-dimensional data storage unit 460, generated data storage unit 470, moving route information storage unit 480, etc.), the terminal to which the sensor device is directly or communicatively connected (including the user terminal 2) or the processor and memory of the management server 1 connected via the terminal. In that case, the movement start step of S103 may be a step of activating the sensor and moving the worker.

In the above-described embodiment, the acquisition of information inside the structure by the moving body 4 was taken as a specific example. Using the 3D model data including the data that the 3D model data is converted into a point group and comparing the 3D point cloud model data and the 3D point cloud sensing data, the same results as in the structure can be obtained. It is possible to generate data, acquire self-position estimation information, and update the configuration outside the structure (for example, a signboard, an antenna, an extended configuration, etc.) by the updating unit 130 . Furthermore, the treatments for both the inside and outside of the structure may be combined.

In addition, the mobile body 4 may further include a device, equipment, or the like that is used to inspect the inner wall and/or the outer wall of the structure for the presence or absence of a predetermined event. More specifically, imaging devices (visible light cameras, infrared cameras, metal detectors, ultrasonic measuring devices, etc.), keying devices, etc., detection devices (
(metal detectors), sound collectors, odor measuring instruments, gas detectors, air pollution measuring instruments, detectors (devices for detecting cosmic rays, radiation, electromagnetic waves, etc.), etc. can be employed.

Further, the embodiments may be implemented, for example, during security and surveillance within a structure, and may further include devices, equipment, etc. used for security and surveillance. More specifically, security and monitoring structures such as imaging devices (visible light cameras, infrared cameras, night vision cameras, metal detectors, ultrasonic measuring devices, etc.) and sensor devices (motion sensors, infrared sensors, etc.) Any device necessary for imaging and detecting anomalies in the system, intruders, etc., can be employed.

The mobile body of the present invention can be suitably used as a mobile body for photography equipped with a camera or the like, and can also be used in security fields, infrastructure monitoring, surveying, sports venues, factories, warehouses, and other buildings and structures. It can also be used in various industries such as inspection and disaster response.

The above-described embodiments are merely examples for facilitating understanding of the present invention, and are not intended to limit and interpret the present invention. It goes without saying that the present invention can be modified and improved without departing from its spirit, and that equivalents thereof are included in the present invention.

1 Management Server 2 User Terminal 4 Mobile 5 Mobile Storage Device

Claims (10)

3D point group model data obtained by converting 3D model data including at least structure model data corresponding to structures within a structure into a point group; a comparison unit that compares the data with each other to generate comparison data;
Based on the comparison data, the types of structures existing only in the 3D point cloud sensing data are estimated, and structure model data corresponding to the estimated types of structures are newly added to the 3D model data. a data update unit that generates update data for
with
the construct model data is not point cloud data;
An information processing system characterized by: 3D point group model data obtained by converting 3D model data including at least structure model data corresponding to structures within a structure into a point group; a comparison unit that compares the data with each other to generate comparison data;
estimating a type of structure that is a difference between the three-dimensional point cloud model data and the three-dimensional point cloud sensing data based on the comparison data, and constructing structure model data corresponding to the estimated type of structure; a data update unit that generates update data for reflecting the to the three-dimensional model data;
with
The structure model data is not point cloud data,
If the data updating unit can estimate that a predetermined structure exists, but cannot determine the type of the structure, the unknown model data of the shape estimated based on the comparison data is added to the three-dimensional model data. Deploy,
An information processing system characterized by: The comparing unit generates three-dimensional point cloud model data obtained by converting three-dimensional model data including at least structure model data corresponding to structures in a structure into point groups, and three-dimensional point cloud model data indicating the result of sensing inside the structure by a sensor. generating comparison data by comparing the original point cloud sensing data with each other;
Based on the comparison data, the data updating unit estimates the types of structures existing only in the 3D point group sensing data, and updates the structure model data corresponding to the estimated types of structures to the 3D model. generating updated data to add new to the data;
the construct model data is not point cloud data;
An information processing method by a computer, characterized by: The comparing unit generates three-dimensional point cloud model data obtained by converting three-dimensional model data including at least structure model data corresponding to structures in a structure into point groups, and three-dimensional point cloud model data indicating the result of sensing inside the structure by a sensor. generating comparison data by comparing the original point cloud sensing data with each other;
Based on the comparison data, the data updating unit estimates the type of structure that is the difference between the three-dimensional point cloud model data and the three-dimensional point cloud sensing data, and corresponds to the estimated type of structure. generating update data for reflecting structure model data to the three-dimensional model data;
The structure model data is not point cloud data,
If it is possible to estimate the presence of a predetermined structure by the step of generating the update data, but if the type of the structure cannot be determined, the shape of the unknown model data estimated based on the comparison data is transferred to the three-dimensional model. further comprising placing on the data;
An information processing method by a computer, characterized by: The comparing unit generates three-dimensional point cloud model data obtained by converting three-dimensional model data including at least structure model data corresponding to structures in a structure into point groups, and three-dimensional point cloud model data indicating the result of sensing inside the structure by a sensor. generating comparison data by comparing the original point cloud sensing data with each other;
Based on the comparison data, the data updating unit estimates the types of structures existing only in the 3D point group sensing data, and updates the structure model data corresponding to the estimated types of structures to the 3D model. generating update data for new additions to the data;
on the computer, and
the construct model data is not point cloud data;
A program characterized by The comparing unit generates three-dimensional point cloud model data obtained by converting three-dimensional model data including at least structure model data corresponding to structures in a structure into point groups, and three-dimensional point cloud model data indicating the result of sensing inside the structure by a sensor. generating comparison data by comparing the original point cloud sensing data with each other;
Based on the comparison data, the data updating unit estimates the type of structure that is the difference between the three-dimensional point cloud model data and the three-dimensional point cloud sensing data, and corresponds to the estimated type of structure. a step of generating update data for reflecting structure model data to the three-dimensional model data;
on the computer, and
The structure model data is not point cloud data,
In the case where it is possible to estimate the presence of a predetermined structure by the step of generating the update data, but the type of the structure cannot be determined, the shape of the unknown model data estimated based on the comparison data is transferred to the three-dimensional model. further comprising placing on the data;
A program characterized by 3D point group model data obtained by converting 3D model data including at least structure model data corresponding to structures within a structure into a point group; a comparison unit that compares the data with each other to generate comparison data;
Based on the comparison data, the types of structures existing only in the 3D point cloud sensing data are estimated, and structure model data corresponding to the estimated types of structures are newly added to the 3D model data. a data update unit that generates update data for
the construct model data is not point cloud data;
A moving body characterized by: 3D point group model data obtained by converting 3D model data including at least structure model data corresponding to structures within a structure into a point group, and 3D point group sensing indicating results of sensing inside the structure by a sensor. a comparison unit that compares the data with each other to generate comparison data;
estimating a type of structure that is a difference between the three-dimensional point cloud model data and the three-dimensional point cloud sensing data based on the comparison data, and constructing structure model data corresponding to the estimated type of structure; a data updating unit that generates update data for reflecting in the three-dimensional model data,
The structure model data is not point cloud data,
If the data updating unit can estimate that a predetermined structure exists, but cannot determine the type of the structure, the unknown model data of the shape estimated based on the comparison data is added to the three-dimensional model data. Deploy,
A moving body characterized by: 3D point group model data obtained by converting 3D model data including at least structure model data corresponding to structures within a structure into a point group; based on the comparative data obtained by comparing the data and the three-dimensional point group sensing data with each other; a data update unit that generates update data to be newly added to the model data;
the construct model data is not point cloud data;
A management server characterized by: 3D point group model data obtained by converting 3D model data including at least structure model data corresponding to structures within a structure into a point group, and 3D point group sensing indicating results of sensing inside the structure by a sensor. estimating a type of structure that is a difference between the three-dimensional point cloud model data and the three-dimensional point group sensing data based on comparison data obtained by comparing the data with each other; a data updating unit that generates update data for reflecting corresponding structure model data in the three-dimensional model data;
The structure model data is not point cloud data,
If the data updating unit can estimate that a predetermined structure exists, but cannot determine the type of the structure, the unknown model data of the shape estimated based on the comparison data is added to the three-dimensional model data. Deploy,
A management server characterized by:

Images