WO2023199477A1

WO2023199477A1 - Information processing system, mobile body, information processing method, and program

Info

Publication number: WO2023199477A1
Application number: PCT/JP2022/017830
Authority: WO
Inventors: 成史大畑
Original assignee: 株式会社センシンロボティクス
Priority date: 2022-04-14
Filing date: 2022-04-14
Publication date: 2023-10-19
Also published as: JP2023157826A; JPWO2023199477A1; JP7170369B1

Abstract

[Problem] To provide an information processing system and the like that make it possible to set a movement path for an autonomous mobile body that includes a movement path that spans the inside and outside of a structure. [Solution] This information processing system comprises: a three-dimensional composite point group data generation unit that generates three-dimensional composite point group data by combining three-dimensional structure interior point group data that indicates at least the arrangement of things inside a structure and three-dimensional structure exterior point group data that indicates at least the arrangement of things outside the structure; and a movement path generation unit that, on the basis of the three-dimensional composite point group data, generates movement path information that indicates a movement path for a mobile body that spans the inside and outside of the structure.

Description

Information processing system, mobile object, information processing method, program

The present invention relates to an information processing system, a mobile object, an information processing method, and a program.

In recent years, autonomous control of flying objects (hereinafter collectively referred to as "flying objects") such as drones and unmanned aerial vehicles (UAVs), and running objects such as unmanned ground vehicles (UGVs) has been increasing. Possible mobile objects are beginning to be used in industry. Under these circumstances, Patent Document 1 discloses a system in which a flying object sequentially photographs an object at a plurality of preset waypoints.

Japanese Patent Application Publication No. 2014-089160

However, the technology disclosed in Patent Document 1 uses GNSS (global navigation satellite system) outdoors for self-position estimation, and creates a movement route for a mobile body based on latitude and longitude information, which is difficult to use when moving indoors. A similar method cannot be used for the movement path of the body.

In addition, when generating a movement route for a moving object indoors (for example, inside a structure such as a building), a technique such as Visual SLAM (Simultaneous Localization and Mapping) is used to manually control the movement of a moving object. One possible method is to obtain three-dimensional indoor information in advance based on sensor information from a sensor installed in the vehicle, and then allow the user to set a travel route based on this information. However, the methods for creating outdoor travel routes and the methods for creating indoor travel routes are different, and the desire to create flight routes that span inside and outside of structures has not been sufficiently studied. .

The present invention was made in view of this background, and an object of the present invention is to provide an information processing system and the like that can set a travel route for an autonomous mobile body, including a travel route that spans inside and outside of a structure.

The main invention of the present invention for solving the above problems is to provide three-dimensional structure internal point cloud data indicating at least the arrangement of components inside the structure, and three-dimensional structure internal point cloud data indicating at least the arrangement of the components outside the structure. a three-dimensional composite point cloud data generation unit that generates three-dimensional composite point cloud data by compositing the original structure external point cloud data with each other; A moving route generation unit that generates moving route information indicating a moving route that spans inside and outside of an object.

According to the present invention, in particular, it is possible to provide an information processing system and the like that can set a travel route for an autonomous mobile body, including a travel route that spans inside and outside of a structure.

1 is a diagram showing the configuration of an information processing system according to an embodiment of the present invention. FIG. 2 is a block diagram showing the hardware configuration of the management server in FIG. 1. FIG. FIG. 2 is a block diagram showing the hardware configuration of the user terminal in FIG. 1. FIG. FIG. 2 is a block diagram showing the hardware configuration of the aircraft shown in FIG. 1. FIG. 2 is a block diagram showing the functions of each component in FIG. 1. FIG. FIG. 3 is a diagram illustrating the association of point group data outside the structure and point group data inside the structure according to the embodiment of the present invention. 3 is a flowchart of a travel route generation method according to an embodiment of the present invention.

The contents of the embodiments of the present invention will be listed and explained. An information processing system etc. according to an embodiment of the present invention has the following configuration.
[Item 1]
Three-dimensional structure internal point group data indicating at least the arrangement of components inside the structure and three-dimensional structure external point group data indicating at least the arrangement of components outside the structure are combined with each other. a three-dimensional composite point cloud data generation unit that generates three-dimensional composite point cloud data;
a moving route generation unit that generates moving route information indicating a moving route for a moving body that spans inside and outside of the structure based on the three-dimensional composite point group data;
An information processing system characterized by:
[Item 2]
The three-dimensional structure internal point cloud data is point cloud data in which at least the model surface of the three-dimensional structure internal model data indicating at least the arrangement of components inside the structure is formed into a point cloud.
The information processing system according to item 1, characterized in that:
[Item 3]
The three-dimensional structure internal model data is BIM data or CIM data.
The information processing system according to item 2, characterized in that:
[Item 4]
The point cloud data outside the three-dimensional structure is point cloud data in which at least the model surface of the three-dimensional outside structure model data indicating at least the arrangement of components outside the structure is converted into a point cloud.
The information processing system according to any one of items 1 to 3, characterized in that:
[Item 5]
The three-dimensional structure external model data is CityGML data,
The information processing system according to item 4, characterized in that:
[Item 6]
At least one of the point group data inside the three-dimensional structure and the point group data outside the three-dimensional structure is point group data obtained by moving a sensor within the structure in advance.
The information processing system according to item 1, characterized in that:
[Item 7]
At least one of the three-dimensional structure internal point group data and the three-dimensional structure external point group data is point group data obtained by moving a virtual sensor within the corresponding three-dimensional structure model data.
The information processing system according to item 1, characterized in that:
[Item 8]
Three-dimensional structure internal point group data indicating at least the arrangement of components inside the structure and three-dimensional structure external point group data indicating at least the arrangement of components outside the structure are combined with each other. a movement path generation unit that generates movement path information indicating a movement path for a moving object that spans inside and outside of the structure, based on the generated three-dimensional composite point group data;
A mobile object characterized by:
[Item 9]
The three-dimensional composite point cloud data generation unit generates three-dimensional structure internal point cloud data indicating at least the arrangement of components inside the structure, and three-dimensional structure external point cloud data indicating at least the arrangement of components outside the structure. a step of composing the point cloud data with each other to generate three-dimensional composite point cloud data;
a step of generating movement path information indicating a movement path for the mobile body spanning inside and outside of the structure, based on the three-dimensional composite point group data, by a movement path generation unit;
An information processing method characterized by causing a computer to execute.
[Item 10]
Three-dimensional structure internal point group data indicating at least the arrangement of components inside the structure and three-dimensional structure external point group data indicating at least the arrangement of components outside the structure are combined with each other. A 3D composite point cloud data generation function that generates 3D composite point cloud data,
a travel route generation function that generates travel route information indicating a travel route for a mobile object that spans inside and outside of the structure based on the three-dimensional composite point group data;
A program characterized by realizing this on a computer.

<Details of embodiment>
Embodiments of information processing systems and the like according to embodiments of the present invention will be described below. In the accompanying drawings, the same or similar elements are given the same or similar reference numerals and names, and redundant description of the same or similar elements may be omitted in the description of each embodiment. Furthermore, features shown in each embodiment can be applied to other embodiments as long as they do not contradict each other.

<Configuration>
As shown in FIG. 1, the information processing system according to the present embodiment includes a management server 1, one or more user terminals 2, and one or more mobile objects 4 (for example, a flying object, a traveling object, etc.). It has the above mobile object storage device 5. The management server 1, the user terminal 2, the mobile object 4, and the mobile object storage device 5 are connected to each other via a network so that they can communicate with each other. Note that the illustrated configuration is an example, and is not limited to this. For example, a configuration that does not include the movable body storage device 5 and is carried by the user may be used.

<Management server 1>
FIG. 2 is a diagram showing the hardware configuration of the management server 1. Note that the illustrated configuration is an example, and other configurations may be used.

As illustrated, a management server 1 is connected to a user terminal 2, a mobile object 4, and a mobile object storage device 5, and constitutes a part of this system. The management server 1 may be a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing.

The management server 1 includes at least a processor 10, a memory 11, a storage 12, a transmitting/receiving section 13, an input/output section 14, etc., which are electrically connected to each other via a bus 15.

The processor 10 is an arithmetic device that controls the overall operation of the management server 1, controls the transmission and reception of data between each element, and performs information processing necessary for application execution and authentication processing. For example, the processor 10 is a CPU (Central Processing Unit) and/or a GPU (Graphics Processing Unit), and executes programs for this system stored in the storage 12 and developed in the memory 11 to perform various information processing. .

The memory 11 includes a main memory configured with a volatile storage device such as a DRAM (Dynamic Random Access Memory), and an auxiliary memory configured with a non-volatile storage device such as a flash memory or an HDD (Hard Disc Drive). . The memory 11 is used as a work area for the processor 10, and also stores a BIOS (Basic Input/Output System) executed when the management server 1 is started, various setting information, and the like.

The storage 12 stores various programs such as application programs. A database storing data used for each process may be constructed in the storage 12.

The transmitting/receiving unit 13 connects the management server 1 to the network. Note that the transmitting/receiving unit 13 may include a short-range communication interface of Bluetooth (registered trademark) and BLE (Bluetooth Low Energy).

The input/output unit 14 is information input devices such as a keyboard and mouse, and output devices such as a display.

The bus 15 is commonly connected to each of the above elements and transmits, for example, address signals, data signals, and various control signals.

<User terminal 2>
The user terminal 2 shown in FIG. 3 also includes a processor 20, a memory 21, a storage 22, a transmitting/receiving section 23, an input/output section 24, etc., which are electrically connected to each other through a bus 25. Since the functions of each element can be configured in the same manner as the management server 1 described above, a detailed explanation of each element will be omitted.

The user terminal 2 is, for example, an information processing device such as a personal computer or a tablet terminal, but may also be configured by a smartphone, a mobile phone, a PDA, or the like. In particular, the input/output unit 24 is composed of a display, a keyboard, and a mouse when the user terminal 2 is composed of a personal computer, and is composed of a touch panel etc. when the user terminal 2 is composed of a smartphone or a tablet terminal. Ru.

<Mobile object 4>
The moving object 4 is a known moving object including a flying object such as a drone or an unmanned aerial vehicle, a running object such as an unmanned ground vehicle, and is particularly a moving object that can be autonomously controlled. As a specific example of the moving object 4, the flying object 4 will be explained below. FIG. 4 is a block diagram showing the hardware configuration of the flying object 4. As shown in FIG. Flight controller 41 may include one or more processors, such as a programmable processor (eg, a central processing unit (CPU)).

Additionally, the flight controller 41 has a memory 411 and can access the memory. Memory 411 stores logic, code, and/or program instructions executable by the flight controller to perform one or more steps. Further, the flight controller 41 may include sensors 412 such as an inertial sensor (acceleration sensor, gyro sensor), a GPS sensor, a proximity sensor (eg, lidar), and the like.

The memory 411 may include, for example, a separable medium or external storage device such as an SD card or random access memory (RAM). Data acquired from cameras/sensors 42 may be communicated directly to and stored in memory 411. For example, still image/video data taken with a camera etc. may be recorded in the built-in memory or external memory, but is not limited to this. It may be recorded in either the user terminal 2 or the mobile storage device 5. The camera 42 may be installed on the flying object 4 via a gimbal 43.

Flight controller 41 includes a control module (not shown) configured to control the state of the aircraft. For example, the control module may be configured to adjust the spatial position, velocity, and/or acceleration of an air vehicle with six degrees of freedom (translational motion x, y, and z, and rotational motion θ _x , θ _y , and θ _z ). , and controls the propulsion mechanism (motor 45, etc.) of the aircraft via an ESC 44 (Electric Speed Controller). A propeller 46 is rotated by a motor 45 supplied with power from a battery 48, thereby generating lift of the flying object. The control module can control one or more of the states of the mounting section and sensors.

Flight controller 41 also transmits and/or transmits data to one or more external devices (e.g., transceiver 49, management server 1, user terminal 2, display device, or other remote controller). Alternatively, it can communicate with a transmitter/receiver 47 configured to be able to receive data from an external device. Transceiver 49 may use any suitable communication means, such as wired or wireless communication.

Furthermore, the flight controller 41 not only functions to control the state of the flying object, such as controlling the state of the flying object described above, but also executes application programs in response to instructions from external devices (particularly the user terminal 2). , various functions related to data processing may be realized, for example, functions corresponding to a movement route generation section 420 and a movement instruction section 430, which will be described later, may be executed. Note that the flight controller 41 may have a function related to data processing so that it can be used for both the mobile object state control function and the data processing function, but instead of this, a separate processor (control unit) dedicated to the data processing function may be provided. You can also do this.

The transmitter/receiver 47 uses, for example, one or more of a local area network (LAN), wide area network (WAN), infrared rays, wireless, WiFi, point-to-point (P2P) network, telecommunications network, cloud communication, etc. can do.

The transmitting/receiving unit 47 transmits and/or transmits one or more of data acquired by the camera/sensors 42, processing results generated by the flight controller 41, predetermined control data, user commands from a terminal or remote controller, etc. or can be received.

The cameras/sensors 42 according to the present embodiment include an inertial sensor (acceleration sensor, gyro sensor), a GPS sensor, a proximity sensor (for example, LiDAR (Light Detection And Ranging), etc.), or a vision/image sensor (for example, a camera). may include.

<Functions of user terminal 2>
FIG. 5 is a block diagram illustrating functions implemented in the user terminal 2. As shown in FIG. In an embodiment of the present invention, at least a point group indicating the arrangement of internal components (e.g., inner walls, columns, ceilings, windows, doors, stairs, internal equipment, etc.) of a structure (e.g., a building such as a building) is included. Point cloud data inside a three-dimensional structure, and at least structures outside the structure (e.g., external walls, roofs, windows, doors, stairs, external equipment, roads, railways, stations, street lights, bus stops, bridges, tunnels, topography) It has various functional units to generate three-dimensional composite point cloud data by compositing three-dimensional structure external point cloud data including point clouds indicating the arrangement of plants, vegetation, water bodies, etc.). Note that some or all of the various functional units may be realized by an information processing device (processor, control unit) installed in at least one of the management server 1 and the mobile body 4.

In the present embodiment, the user terminal 2 includes a communication section 210, a three-dimensional structure point cloud data generation section 220, a three-dimensional composite point cloud data generation section 230, and a storage section 270. Furthermore, the storage unit 270 includes various databases such as a three-dimensional data storage unit 271.

The communication unit 210 communicates with the management server 1, the mobile body 4, and the mobile body storage device 5. The communication unit 210 also functions as a reception unit that receives various requests, data, etc. from the management server 1, the mobile unit 4, and the mobile unit storage device 5.

The three-dimensional structure point cloud data generation unit 220 generates three-dimensional structure internal model data indicating at least the arrangement of components inside the structure, and three-dimensional structure external model data indicating at least the arrangement of components outside the structure. , three-dimensional structure internal point group data and three-dimensional structure external point group data are generated by converting at least the model surface into a point group. Each 3D model data (model data inside a 3D structure, model data outside a 3D structure, etc.), each 3D point cloud data (point group data inside a 3D structure, point group data outside a 3D structure, etc.) are data expressed in a three-dimensional coordinate system, and are stored in the three-dimensional data storage unit 271 by being read from the management server 1 or the mobile object 4, created by an application on the user terminal 2, etc. managed.

The three-dimensional structure internal model data may be any type of three-dimensional structural internal model data that indicates at least the arrangement of components within the structure and has dimensional information. -Aided Design) A model created based on data created with design software, including BIM (Building Information Modeling) data, CIM (Construction Information Modeling) data, and CAD data. , reconstructed from BIM data, etc. It may be three-dimensional model data or the like, or it may be three-dimensional model data obtained by generating a structure having a predetermined height based on two-dimensional blueprint data. Note that, as described above, the three-dimensional structure internal model data indicates the arrangement of the components inside the structure, but it does not necessarily have to indicate the arrangement of only the components inside the structure; for example, It may also include the arrangement of components outside the structure related to the structure, such as an outer wall. Moreover, reconstruction of the three-dimensional model data, etc. may be executed in the processor of the management server 1 or the user terminal 2, or may be executed outside the management server 1 or the user terminal 2 and acquired into the inside. The three-dimensional structure internal model data can be data expressed by a three-dimensional orthogonal coordinate system with an arbitrary set point as the origin (reference point), but for example, the horizontal direction is latitude and longitude coordinates, The data may be expressed by a three-dimensional coordinate system in which the vertical direction is the altitude coordinate, but this is not necessarily the case.

The three-dimensional outside-structure model data may be any type of three-dimensional outside-structure model data that indicates at least the arrangement of components outside the structure and has dimensional information, but in particular data on the ground surface and on the ground surface. It is preferable that the model includes a model that virtually represents the surface shape of at least a part of the structure in a three-dimensional three-dimensional space, for example, three-dimensional city model data such as CityGML (Generalized Markup Language), CityJson, GeoTIFF, etc. It may also be three-dimensional city model data stored in a three-dimensional city model database outside this system. The three-dimensional external structure model data may be data expressed by a three-dimensional coordinate system in which the horizontal direction is latitude and longitude coordinates and the vertical direction is altitude coordinates.

The method for generating three-dimensional point cloud model data (point cloud data inside a three-dimensional structure, point cloud data outside a three-dimensional structure, etc.) by the three-dimensional structure point cloud data generation unit 220 is to generate a structure from the three-dimensional model data described above. Any method can be used as long as it is possible to generate three-dimensional point cloud model data that is a point cloud of model surfaces related to components inside or outside the structure. Three-dimensional point cloud model data regarding components inside or outside the structure is generated by moving a virtual moving body 4 equipped with a virtual sensor (for example, virtual LiDAR) inside or outside the structure. You may. As a result, it is theoretically possible to generate point cloud data close to point cloud sensing data obtained when the inside or outside of the structure is actually measured using the sensor of the moving body 4. Other methods of generating 3D point cloud model data include forming 3D model data into a point cloud evenly at predetermined intervals, or if the 3D model data is polygon data, points are placed at each vertex. may be arranged to form a point cloud, or may be formed into a point cloud using a known point cloud technology (conversion technology to point cloud data).

The three-dimensional composite point cloud data generation unit 230 may generate three-dimensional composite point cloud data by combining the three-dimensional structure internal point cloud data and the three-dimensional structure external point cloud data. The three-dimensional structure internal point cloud data and the three-dimensional structure external point cloud data may be three-dimensional point cloud data generated by the three-dimensional structure point cloud data generation unit 220 as described above, or three-dimensional point cloud data At least one of the original point cloud data inside the structure and the point cloud data outside the three-dimensional structure is three-dimensional point cloud data obtained by moving a moving body 4 equipped with a sensor (for example, a LiDAR sensor) in advance. It's okay.

The method of generating three-dimensional point cloud data by the three-dimensional composite point cloud data generation unit 230 (i.e., the synthesis method) may be any method as long as it can be synthesized. The two may be aligned (and scaled) based on the above, and the three-dimensional point cloud data of both may be combined as an integrated three-dimensional composite point cloud data at the determined position, or using known point cloud composition techniques. The three-dimensional point group data may be combined with each other using . However, as a specific example in this embodiment, the synthesis method differs depending on whether the point group data inside the three-dimensional structure is expressed in the latitude/longitude/height coordinate system like the point group data outside the three-dimensional structure.

As a specific example of the synthesis method, a case will be explained in which point group data inside a three-dimensional structure and point group data outside a three-dimensional structure are both expressed in a common three-dimensional coordinate system such as a latitude-longitude-height coordinate system. . In such a case, since each point data of both three-dimensional point group data is expressed by latitude, longitude, and height coordinates, the combination is executed by arranging them in the same virtual space. Although both three-dimensional point cloud data are expressed in latitude, longitude, and height coordinates, when the moving object 4 actually moves within a structure, the position information of the moving object 4 is expressed in latitude, longitude, and height coordinates. Since it is difficult to obtain such data, the three-dimensional composite point group data may be converted in advance to a three-dimensional orthogonal reference coordinate system having a predetermined reference point in the virtual space as the origin. Alternatively, the management server 1, the user terminal 2, or the moving body 4 is equipped with a coordinate conversion unit 240 that performs the conversion, and converts the position information of the moving body 4 acquired within the structure into latitude, longitude, and height coordinates. You can do it like this.

Another specific example of the synthesis method is that the point cloud data inside the three-dimensional structure and the point cloud data outside the three-dimensional structure are not in a common three-dimensional coordinate system (for example, the point cloud data inside the three-dimensional structure is A case will be described in which the three-dimensional structure external point group data is not in the longitude/height coordinate system but in the latitude/longitude/height coordinate system. In such a case, for example, as shown in Figure 6, for the same structure, the point cloud data inside the three-dimensional structure and the point cloud data outside the three-dimensional structure are Set the corresponding points at ). The corresponding points may be, for example, the four corners of the first floor (ground surface) of a building such as a square building, or may be characteristic points on the floor of another structure. Note that as long as a corresponding point with clear height information can be set, the corresponding point is not limited to the floor surface, and may be any characteristic point. For example, the corresponding points may be set manually on the user terminal 2 based on user operations, or automatically based on shape analysis technology in a known structure model or corner point detection technology in a point cloud model. May be set.

Then, the corresponding points P1 to P4 of the point group data outside the structure are converted into coordinate values expressed in a three-dimensional orthogonal coordinate system with the reference point O as the origin, and both corresponding points are By determining the positional relationship information (coordinate transformation information) that minimizes the distance between the Combine both 3D point cloud data. In addition, since the positional relationship information of both three-dimensional point cloud data can be obtained, if necessary, the point cloud data inside the structure can be changed to the latitude, longitude, and height coordinate system based on the latitude, longitude, and height coordinate system of the point cloud data outside the structure. The coordinates of the system may also be given. Further, the corresponding points may be set on the three-dimensional model data, and the corresponding point data may be set again as the corresponding points after point cloud formation.

Here, when corresponding points are set in the 3D structure internal point cloud data and the 3D structure external point cloud data for the floor surface of the same structure as described above, both the corresponding points are not necessarily the same. Alternatively, the positions may not be approximate. In other words, if corresponding points are set with the inner wall as a reference in the 3D structure internal point cloud data, and corresponding points are set with the outer wall as the reference in the 3D structure external point cloud data, then at both corresponding points, There will be a shift and the positions will not be the same or similar. Therefore, after setting multiple corresponding points in the point cloud data inside the 3D structure and the point cloud data outside the 3D structure, the center of gravity is calculated, and the center of gravity is set as the corresponding point to combine both 3D point cloud data. Positional relationship information may be determined by matching. Furthermore, based on the corresponding points that target the floor surface, etc. (hereinafter referred to as "first corresponding points") and the corresponding points that correspond to the center of gravity (hereinafter referred to as "second corresponding points"), Positional relationship information may be obtained by matching three-dimensional point cloud data, and in particular, positional relationship information is determined by weighting which degree of matching is given priority between the first corresponding point and the second corresponding point using a weighting coefficient. You may also ask for In view of the possibility that the above-mentioned deviation may occur, the weighting may be such that the weighting for the second corresponding point is greater than the weighting for the first corresponding point.

In this way, the three-dimensional structure internal point cloud data and the three-dimensional structure external point cloud data, which are different from each other, are synthesized and the three-dimensional composite point cloud data indicating the arrangement of components inside and outside the structure is converted into unified three-dimensional coordinates. It becomes possible to express it by a system. As a result, for example, when waypoints are set based on three-dimensional composite point cloud data displayed on the user terminal 2, even if the travel route straddles the inside and outside of a structure, a unified three-dimensional It becomes possible to specify positions such as generating travel routes based on the original coordinate system.

<Functions of mobile object 4>
FIG. 5 is a block diagram further illustrating functions implemented in the mobile body 4. In the embodiment of the present invention, based on the generated three-dimensional composite point group data, movement route information for the moving body spanning inside and outside the structure is generated, and the moving body 4 is instructed to move on the movement route. It has various functional parts for this purpose. Note that some or all of the various functional units may be realized by an information processing device (processor, control unit) installed in at least one of the management server 1 and the user terminal 2.

In the present embodiment, the moving body 4 includes a movement route generation section 420, a movement instruction section 430, a coordinate conversion section 440, and a storage section 470. Furthermore, the storage unit 470 includes various databases such as a travel route information storage unit 471 and a travel information storage unit 472.

For example, the movement route generation unit 420 sequentially generates one or more waypoint information from a start point to an end point, or generates arbitrary points based on the user's selection operation on the three-dimensional composite point group data displayed on the user terminal 2. The waypoint information may be set in an arbitrary order, and travel route information may be generated by a known method based on the waypoint information, and stored and managed in the travel route information storage unit 471, or three-dimensional composite point cloud data for example, calculates a travel route in which waypoint information that can obtain information on specific or all components within the structure is set, and stores this as travel route information in the travel route information storage unit 471, It may also be managed.

Note that the movement route may be generated, for example, by setting the position of the moving body storage device 5 as the movement start position and the movement end position, and passing through each waypoint, or conversely, Alternatively, the configuration may be such that the position where the aircraft is carried by the user is set as the movement start position, or the user collects the aircraft at the movement end position. Based on the information (for example, position information, storage status information, storage machine information, etc.) of the mobile body storage device 5 managed in the storage unit of the mobile body storage device 5, the mobile body storage device 5 selected as the movement start position or movement end position is It may also be configured to be generated as a movement route including the location.

The movement instruction unit 430 refers to the movement route information stored in the movement route information storage unit 471 and transmits a signal instructing the movement of the mobile body 4 to the mobile body 4 according to the coordinates indicated by the movement route information. Further, the movement instruction unit 430 uses the coordinate conversion unit 440 (which may have the same function as the coordinate conversion unit 240) to convert, for example, the movement route information stored in the movement route information storage unit 471 and the above-mentioned positional relationship information. (coordinate conversion information) and convert the coordinates indicated by the movement route information into a three-dimensional orthogonal coordinate system (for example, the three-dimensional orthogonal reference coordinate system mentioned above) used to control the movement of the moving object 4. It may be configured to instruct movement of the body 4.

The movement information storage unit 472 stores parameters used when the movement route generation unit 420 generates a movement route, the movement instruction unit 430 instructs the autonomously controlled moving body 4 to move on the movement route, etc. It stores information, information acquired during movement, etc. acquired on the movement route. Examples of specific parameters include moving speed, flight altitude (if the moving object 4 is a flying object), overlap rate of captured images, information acquired during movement (for example, image information, video information, etc.), etc. including.

<Example of movement route generation method>
With reference to FIG. 7, the travel route generation method according to this embodiment will be described, including the operation of the information processing system according to this embodiment. FIG. 7 illustrates a flowchart of the movement route generation method according to the present embodiment. Although this flowchart exemplarily shows a configuration in which an application is started on the user terminal 2, the configuration is not limited to this. The configuration may include an input/output device and allow various settings and the like.

First, the user starts an application including a three-dimensional composite point cloud data generation function, for example on the user terminal 2 (SQ101). This application may be stored in the user terminal 2, for example, or may be software (so-called SaaS) provided from the management server 1 or another server (not shown) connected via a network. A login screen may be displayed as necessary, and a configuration may be adopted in which, for example, a login ID and password are requested.

Next, the user creates a new travel plan (SQ102). For example, you can set the "plan name", "area name", "address", etc., acquire and display the 3D composite point cloud data within the target structure on the user terminal 2, and create a new movement. Start creating a plan.

Next, the user generates a travel route for the movement of the mobile object 4 (SQ103). For example, one or more waypoint information (for example, expressed in the latitude and longitude coordinate system on the user terminal 2) is set by the user's selection operation for the three-dimensional composite point cloud data displayed on the user terminal 2. . Then, the three-dimensional composite point cloud data and waypoint information are transmitted to the mobile body 4, and the mobile body 4 uses a known method (for example, between each of the four waypoints) based on the three-dimensional composite point cloud data and the waypoint information. (Connecting with a straight line), movement route information is generated.

Next, the user instructs the moving body 4 to start moving (SQ104). For example, with reference to the movement route information storage section 471 and the movement information storage section 472, movement of the mobile object 4 for purposes such as inspection, security, construction progress management, etc. is executed.

Next, the user instructs the user terminal 2 to output the acquired information (SQ105). For example, information on the route along which the mobile object 4 actually traveled may be displayed superimposed on the three-dimensional composite point cloud data displayed on the user terminal 2. In addition, the acquired information (still images, moving images, audio, and other information) acquired by the mobile object 4 on the movement route can be displayed, and the positions associated with the position information of the acquired information (especially waypoints) can be displayed. A mark such as a symbol that serves as a link for viewing acquired information corresponding to location information) may also be attached. Then, by selecting the link on the user terminal 2, the corresponding acquired information may be displayed.

As described above, the present invention can provide an information processing system and the like that can set a travel route for an autonomous mobile body, including a travel route that spans inside and outside of a structure. Note that instead of the above-described configuration in which the mobile object 4 moves autonomously in SQ104, it may be configured to simply create an optimal movement route and display it on the user terminal 2 for confirmation. Based on this, it becomes possible to display on the user terminal 2 of another user a movement route that serves as a model for when another user manually moves the mobile object 4 within the structure using a radio or the like.

Further, in the above-described embodiment, the acquisition of information inside and outside the structure by the moving object 4 was taken as a specific example, but it may also be an inspection of the structure, and the presence or absence of a predetermined event on the inner wall and/or outer wall of the structure may be used. It may also be equipped with equipment, equipment, etc. used for inspecting. More specifically, imaging devices (visible light cameras, infrared cameras, metal detectors, ultrasonic measuring devices, etc.), keying devices, detection devices (metal detectors), sound collection devices, odor measuring devices, gas detection devices, etc. All devices necessary to know the condition of a structure to be inspected having an inner wall or an outer wall, such as an air contamination measuring device, a detection device (device for detecting cosmic rays, radiation, electromagnetic waves, etc.), etc., can be employed.

Further, the embodiment may be, for example, security or monitoring inside a structure, and may include devices, equipment, etc. used for security or monitoring. More specifically, structures to be guarded and monitored, such as imaging devices (visible light cameras, infrared cameras, night vision cameras, metal detectors, ultrasonic measuring instruments, etc.) and sensor devices (motion sensors, infrared sensors, etc.) All devices necessary to image and detect abnormalities, intruders, etc. can be employed.

The mobile object of the present invention can be suitably used as a mobile object for photographing equipped with a camera, etc., and can also be used in the security field, infrastructure monitoring, surveying, and in buildings and structures such as sports venues, factories, warehouses, etc. It can also be used in various industries such as inspection and disaster response.

The embodiments described above are merely illustrative to facilitate understanding of the present invention, and are not intended to be interpreted as limiting the present invention. It goes without saying that the present invention can be modified and improved without departing from its spirit, and that the present invention includes equivalents thereof.

1 Management server 2 User terminal 4 Mobile object 5 Mobile object storage device

Claims

Three-dimensional structure internal point group data indicating at least the arrangement of components inside the structure and three-dimensional structure external point group data indicating at least the arrangement of components outside the structure are combined with each other. a three-dimensional composite point cloud data generation unit that generates three-dimensional composite point cloud data;
a moving route generation unit that generates moving route information indicating a moving route for a moving body that spans inside and outside of the structure based on the three-dimensional composite point group data;
An information processing system characterized by:
The three-dimensional structure internal point cloud data is point cloud data in which at least the model surface of the three-dimensional structure internal model data indicating at least the arrangement of components inside the structure is formed into a point cloud.
The information processing system according to claim 1, characterized in that:
The three-dimensional structure internal model data is BIM data or CIM data.
The information processing system according to claim 2, characterized in that:
The point cloud data outside the three-dimensional structure is point cloud data in which at least the model surface of the three-dimensional outside structure model data indicating at least the arrangement of components outside the structure is converted into a point cloud.
The information processing system according to any one of claims 1 to 3, characterized in that:
The three-dimensional structure external model data is CityGML data,
5. The information processing system according to claim 4.
At least one of the point group data inside the three-dimensional structure and the point group data outside the three-dimensional structure is point group data obtained by moving a sensor within the structure in advance.
The information processing system according to claim 1, characterized in that:
At least one of the three-dimensional structure internal point group data and the three-dimensional structure external point group data is point group data obtained by moving a virtual sensor within the corresponding three-dimensional structure model data.
The information processing system according to claim 1, characterized in that:
Three-dimensional structure internal point group data indicating at least the arrangement of components inside the structure and three-dimensional structure external point group data indicating at least the arrangement of components outside the structure are combined with each other. a movement path generation unit that generates movement path information indicating a movement path for a moving object that spans inside and outside of the structure, based on the generated three-dimensional composite point group data;
A mobile object characterized by:
The three-dimensional composite point cloud data generation unit generates three-dimensional structure internal point cloud data indicating at least the arrangement of components inside the structure, and three-dimensional structure external point cloud data indicating at least the arrangement of components outside the structure. a step of composing the point cloud data with each other to generate three-dimensional composite point cloud data;
a step of generating movement path information indicating a movement path for the mobile body spanning inside and outside of the structure, based on the three-dimensional composite point group data, by a movement path generation unit;
An information processing method characterized by causing a computer to execute.
Three-dimensional structure internal point group data indicating at least the arrangement of components inside the structure and three-dimensional structure external point group data indicating at least the arrangement of components outside the structure are combined with each other. A 3D composite point cloud data generation function that generates 3D composite point cloud data,
a travel route generation function that generates travel route information indicating a travel route for a mobile object that spans inside and outside of the structure based on the three-dimensional composite point group data;
A program characterized by realizing this on a computer.