JP7239744B2 - Information processing system - Google Patents

Description

The present invention relates to the creation of 3D maps.

In Patent Document 1, a difference between map data generated based on an aerial image taken by an unmanned aerial vehicle and the original map data is calculated, and the calculated difference is used for overall deformation (parallel movement, scaling, rotation, There is disclosed a technique for generating map data in which the difference is suppressed by generating aerial photography parameters that improve the position, route, attitude, etc. at the time of aerial photography when showing shear, etc.).

JP 2019-028807 A

When flying a flying object such as a drone autonomously, there are cases where a flight path is determined so as not to collide with obstacles based on a three-dimensional map showing the arrangement of objects existing in a real three-dimensional space. Objects existing in the three-dimensional space change daily due to new construction or demolition. In order to update a 3D map, it is necessary to photograph an area where there are objects that change daily. You will have to fly in vain.
SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to efficiently perform photographing for updating a 3D map.

In order to achieve the above objects, the present invention provides a storage unit for storing a three-dimensional map showing the arrangement of objects existing in a three-dimensional space, an output acquisition unit that acquires an output from a sensor capable of detecting the arrangement of objects in the three-dimensional space; and based on the acquired output, changes in the arranged objects occur compared with the three-dimensional map. An information processing system is provided that includes a specifying unit that specifies a changed area, and a photographing instruction unit that instructs a second flying object to photograph the specified changed area.

According to the present invention, it is possible to efficiently perform photography for updating a 3D map.

A diagram showing an example of the overall configuration of a 3D map system according to an embodiment. A diagram showing an example of a hardware configuration of a server device A diagram showing an example of the hardware configuration of a drone A diagram showing an example of the hardware configuration of a photography drone Diagram showing the functional configuration realized by each device A diagram showing an example of an identified change area A diagram showing an example of a no-fly space A diagram showing an example of the operation procedure of each device in the update process

[1] Embodiment FIG. 1 shows an example of the overall configuration of a 3D map system 1 according to an embodiment. The 3D map system 1 is a system that generates and updates a 3D map using an aircraft. A 3D map is a map that three-dimensionally represents objects such as topography, buildings, and plants that exist in real space. A 3D map is used, for example, to determine a flight route that avoids obstacles such as buildings and trees when making a flight plan for an aircraft to fly through space.

The three-dimensional map system 1 includes a network 2, a server device 10, drones 20-1, 20-2, 20-3, . 30. A network 2 is a communication system including a mobile communication network, the Internet, etc., and relays data exchange between devices accessing the system. The network 2 is accessed by the server device 10 through wired communication (or wireless communication), and by a plurality of drones 20 through wireless communication.

In this embodiment, the drone 20 is a rotorcraft-type flying object that flies by rotating one or more rotor blades. Each drone 20 has a function of autonomously flying along a planned flight route, and flies for the purpose of delivering packages or taking photographs, for example. Each drone 20 includes a sensor for detecting obstacles, detects obstacles present in the flight path, and controls flight to avoid collisions.

Although the flight path is designed to avoid obstacles, flying objects such as birds or other drones may appear as obstacles. Other obstacles may appear, such as newly constructed buildings, newly installed overhead lines, and trees that have grown to intervene in the flight path. Flying objects are temporary obstacles, but buildings and the like are permanent obstacles. Once those permanent obstacles are reflected in the cubic map, subsequent flight paths are determined to avoid the obstacles.

The server device 10 performs processing for reflecting obstacles detected during flight by the drone 20 on the 3D map. The server device 10 instructs the photography drone 30 to photograph the detected obstacle. The photographing drone 30 has a function of photographing the surroundings, and photographs designated obstacles. The imaging drone 30 transmits the captured image of the obstacle to the server device 10 . The server device 10 reflects obstacles on the 3D map based on the transmitted video.

FIG. 2 shows an example of the hardware configuration of the server device 10. As shown in FIG. The server device 10 may be physically configured as a computer device including a processor 11, a memory 12, a storage 13, a communication device 14, a bus 15, and the like. Note that in the following description, the term "apparatus" can be read as a circuit, device, unit, or the like.

Also, one or more of each device may be included, or some devices may not be included. The processor 11, for example, operates an operating system to control the entire computer. The processor 11 may be configured by a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic device, registers, and the like.

For example, the baseband signal processing section and the like may be implemented by the processor 11 . The processor 11 also reads programs (program codes), software modules, data, etc. from at least one of the storage 13 and the communication device 14 to the memory 12, and executes various processes according to the read programs. As the program, a program that causes a computer to execute at least part of the operations described in the above embodiments is used.

Although it has been explained that the various processes described above are executed by one processor 11, they may be executed by two or more processors 11 simultaneously or sequentially. Processor 11 may be implemented by one or more chips. Note that the program may be transmitted from a network via an electric communication line. Memory 12 is a computer-readable recording medium.

The memory 12 may be composed of at least one of, for example, ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), and the like. The memory 12 may also be called a register, cache, main memory (main storage device), or the like. The memory 12 can store executable programs (program codes), software modules, etc. for implementing the wireless communication method according to an embodiment of the present disclosure.

The storage 13 is a computer-readable recording medium, for example, an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disc, a magneto-optical disc (e.g., a compact disc, a digital versatile disc, a Blu-ray disk), smart card, flash memory (eg, card, stick, key drive), floppy disk, magnetic strip, and/or the like.

The storage 13 may also be called an auxiliary storage device. The aforementioned storage medium may be, for example, a database, server or other suitable medium including at least one of memory 12 and storage 13 . The communication device 14 is hardware (transmitting/receiving device) for communicating between computers via at least one of a wired network and a wireless network.

For example, the above-described transmitting/receiving antenna, amplifier section, transmitting/receiving section, transmission line interface, etc. may be implemented by the communication device 14 . The transceiver may be physically or logically separate implementations for the transmitter and receiver. Each device such as the processor 11 and the memory 12 is connected by a bus 15 for communicating information. The bus 15 may be configured using a single bus, or may be configured using different buses between devices.

FIG. 3 shows an example of the hardware configuration of the drone 20. As shown in FIG. Drone 20 may be physically configured as a computer device including processor 21, memory 22, storage 23, communication device 24, flight device 25, sensor device 26, bus 27, and the like. Hardware with the same name shown in FIG. 2, such as the processor 21, is the same kind of hardware as in FIG. 2, although there are differences in performance, specifications, and the like.

The communication device 24 has a function of communicating with the network 2 (for example, a wireless communication function using radio waves in the 2.4 GHz band). The flight device 25 is a device that includes a motor, a rotor, and the like, and allows the aircraft to fly. The flying device 25 can move itself in all directions in the air and can make itself stationary (hovering). The sensor device 26 is a device having a group of sensors that acquire information necessary for flight control.

The sensor device 26 includes, for example, a position sensor that measures the position (latitude and longitude) of the drone and the direction in which the drone is facing (the front direction of the drone is determined, and the determined front direction and an altitude sensor for measuring the altitude of the aircraft itself. Further, the sensor device 26 includes a speed sensor that measures the speed of its own machine.

The sensor device 26 also includes an inertial measurement sensor (IMU (Inertial Measurement Unit)) that measures triaxial angular velocities and tridirectional accelerations. The sensor device 26 also includes a sensor for detecting an obstacle that may collide with the drone 20 (hereinafter referred to as an "obstacle detection sensor"). The sensor device 26 includes, as an obstacle detection sensor, a ranging sensor that emits infrared rays, millimeter waves, or the like and measures the distance and direction to surrounding obstacles.

FIG. 4 shows an example of the hardware configuration of the imaging drone 30. As shown in FIG. The imaging drone 30 is physically configured as a computer device including a processor 31, a memory 32, a storage 33, a communication device 34, a flight device 35, a sensor device 36, a camera 37, a bus 38, and the like. may be Hardware with the same name shown in FIG. 2, such as the processor 31, is the same kind of hardware as in FIG. 2, although there are differences in performance and specifications. The camera 37 includes an image sensor, optical system parts, and the like, and photographs an object in the direction in which the lens is directed.

Each of the above devices includes hardware such as microprocessors, digital signal processors (DSPs), ASICs (Application Specific Integrated Circuits), PLDs (Programmable Logic Devices), and FPGAs (Field Programmable Gate Arrays). may consist of Also, each of the above devices may be realized by the hardware in part or all of each functional block. For example, processor 11 may be implemented using at least one such piece of hardware.

Each function of each device included in the 3D map system 1 is performed by the processor by loading predetermined software (program) onto hardware such as each processor and memory, and controlling communication by each communication device. or by controlling at least one of data reading and writing in memory and storage.

FIG. 5 shows the functional configuration realized by each device. The server device 10 includes a 3D map storage unit 101, a location information acquisition unit 102, a change area identification unit 103, a no-fly instruction unit 104, an area photography instruction unit 105, an area image acquisition unit 106, and a 3D map update unit. A building schedule acquisition unit 108 is provided. The drone 20 includes a flight control section 201 and a placement information measurement section 202 . The imaging drone 30 includes an instruction reception unit 301 , a flight control unit 302 and an area imaging unit 303 .

The flight control unit 201 of the drone 20 controls the flight of the drone 20 using the measurement results of each sensor included in the sensor device 26 . The placement information measurement unit 202 of the drone 20 measures information indicating the placement of objects existing in the three-dimensional space (hereinafter referred to as “placement information”). The arrangement information measurement unit 202 measures, for example, position information indicating the position of the aircraft in a three-dimensional space, and obstacle information indicating the distance from the aircraft to surrounding obstacles and the direction in the three-dimensional space as location information. do.

The arrangement information indicates, as the arrangement of an object (obstacle in this embodiment), a position that is moved by a distance indicated by the obstacle information in the direction indicated by the obstacle information from the position indicated by the position information. In other words, the placement of the object indicated by the placement information is indicated by the position in the three-dimensional space of the portion of the object measured by the sensor of the drone 20 . The location information represents the location of the object by, for example, latitude, longitude and height.

Since the arrangement information is also used for flight control, the arrangement information measuring section 202 repeatedly measures the arrangement information and supplies the arrangement information to the flight control section 201 each time it is measured. The flight control unit 201 controls the flight of the own aircraft so as to avoid obstacles indicated by the supplied arrangement information. In addition, the placement information measuring unit 202 repeatedly measures the placement information and transmits the placement information to the server device 10 each time the measurement is performed.

The 3D map storage unit 101 of the server device 10 stores a 3D map showing the arrangement of objects existing in a 3D space. The 3D map storage unit 101 is an example of the "storage unit" of the present invention. The 3D map storage unit 101 stores, as a 3D map, a set of surface coordinates of objects existing in a real 3D space, for example, in a 3D coordinate system set in the real 3D space. Objects represented on the 3D map include objects whose positions are fixed without moving (hereinafter referred to as “fixed objects”) such as the ground, water surfaces, buildings, and plants.

For example, objects that change shape but do not move, such as plants or buildings under construction, are included in fixed objects. On the other hand, moving objects such as animals and vehicles (hereinafter referred to as "moving objects") are not included in the objects represented on the 3D map. The arrangement information acquisition unit 102 of the server device 10 receives the arrangement information transmitted from the server device 10 by a sensor capable of detecting the arrangement of surrounding objects in the three-dimensional space provided in the drone 20 flying in the three-dimensional space. (hereafter referred to as "arrangement detection sensor").

The drone 20 is an example of the "first flying object" of the present invention, and the placement information acquisition section 102 is an example of the "output acquisition section" of the present invention. In this embodiment, a sensor for detecting obstacles that may collide with the drone 20 is used as the placement detection sensor. Specifically, the arrangement detection sensor of the present embodiment includes a positioning sensor (a position sensor and an altitude sensor included in the sensor device 26 in FIG. 3) that detects position information indicating the position of the aircraft itself, and a It is a ranging sensor that detects obstacle information indicating distance and direction.

The arrangement information acquisition unit 102 acquires each piece of arrangement information repeatedly transmitted from the drone 20, and supplies the acquired arrangement information to the changed area identification unit 103 each time it is acquired. Based on the supplied layout information, i.e., the output of the sensor acquired by the layout information acquisition unit 102, the changed region identifying unit 103 compares the 3D map with a region where a change occurs in the arranged object (hereinafter referred to as " (referred to as the “region of change”). The changed region specifying unit 103 is an example of the "specifying unit" of the present invention.

Changes in objects that occur in the change area include changes in objects that are placed only in the 3D map (that is, only in the past) and are not placed in the current real space, and changes in objects that are placed in the current real space but are not placed in the 3D map. includes changes in objects that have not been placed (ie, in the past) and objects that have been placed in the solid map (ie, in the past) and are also placed in the current real space.

When the arrangement information is supplied, the changing area specifying unit 103 first determines whether the object whose arrangement is indicated by the supplied arrangement information is a fixed object or a moving object. If the arrangement indicated by a plurality of pieces of arrangement information for the same object changes, the changed area specifying unit 103 determines that the object is a moving object and does not specify the changed area. Also, if the arrangement of these objects has not changed, the changed region specifying unit 103 determines that the object is a fixed object, and specifies the changed region.

The changed area specifying unit 103 reads the 3D map from the 3D map storage unit 101 when specifying the changed area. The changed area specifying unit 103 calculates the coordinates in the three-dimensional coordinate system used in the 3D map for the object (obstacle in this embodiment) whose arrangement is indicated by the arrangement information. Since the arrangement information expresses the arrangement of objects by latitude, longitude and height as described above, the changing area specifying unit 103 calculates the coordinates indicated by these latitudes, longitudes and heights.

The changed area specifying unit 103 determines whether or not the calculated coordinates of the object are included in the set of coordinates of the read 3D map. Identify as a change area.
FIG. 6 shows an example of identified change regions. FIG. 6 shows a three-dimensional space A1 viewed from vertically above.

It is assumed that the three-dimensional space A1 is partitioned into squares, and a three-dimensional map is represented by a set of coordinates of intersection points. Actually, the height difference is also represented on the three-dimensional map, but in FIG. 6, the height difference is not taken into consideration. FIG. 6 shows the drone 20 that has flown along the route B1. The drone 20 was scheduled to fly along the planned flight path C1, but the flight path C1 is blocked by a building 3 that was not shown on the three-dimensional map.

The drone 20 measures arrangement information indicating the arrangement of the measurement points D1 measured by the positioning sensor and the distance measuring sensor in the building 3, which is an obstacle. The drone 20 flies along a route B2 that avoids obstacles, and continues to measure the location information of the measurement point D1 while flying. In the example of FIG. 6, the drone 20 measures the arrangement information of six measurement points D1. The measurement locations D1 include only locations of the building 3 that can be measured from the route B2 side.

The changed area specifying unit 103 determines that the coordinates of the object calculated for the measurement point D1 are not included in the set of coordinates of the read 3D map, and changes the area including the measurement point D1 and the area adjacent to the measurement point D1. It is specified as area E1. As shown in FIG. 6, the changed area E1 is determined based on the placement information measured by the drone 20, so in the example of FIG.

As described above, the changing area specifying unit 103 specifies, as a changing area, an area in which an obstacle exists indicated by the arrangement information output from the arrangement detection sensor. As a result, the drone 20 having the function of avoiding obstacles can be used to identify the changed area. After specifying the changed region, the changed region specifying unit 103 supplies change information indicating the specified changed region to the no-fly instruction unit 104 . The change information includes, for example, arrangement information (information indicating latitude, longitude, and height) of the measurement point D1 used to specify the changed area, and a set of coordinates representing the specified changed area E1 (each of the four sides of each changed area E1). coordinates).

When change information is supplied, that is, when an area with a new object not shown on the 3D map is specified as a change area, the flight prohibition instruction unit 104 prohibits flight in the space where the object exists. Command the drone 20. The no-flying instruction unit 104 is an example of the "no-flying instruction unit" of the present invention. The no-fly instruction unit 104 generates, for example, the coordinates of the object indicated by the supplied change information, i.e., instruction data indicating a range of a certain distance from the coordinates of the obstacle as the no-fly space, and transmits the instruction data to each drone 20 . .

FIG. 7 shows an example of a no-fly space. In FIG. 7, the change area E1 specified in the example of FIG. 6 is represented. The no-fly instruction unit 104 defines the smallest rectangular area that includes the changing area E1 inside (the outermost grid does not include the changing area E1) as the no-flying space F1. As shown in FIG. 6, the changing area E1 may not be an area that includes the entire building 3, so the flight prohibition space F1 is set larger than the changing area E1.

The flight control unit 201 of the drone 20 controls the flight of the drone 20 to avoid the prohibited flight space F1 when the drone 20 flies on a flight route passing through the prohibited flight space F1 indicated by the transmitted instruction data. If the 3D map is up-to-date and building 3 is represented, a flight plan that avoids building 3 is created, but before building 3 is represented on the 3D map, the flight plan is shown in FIG. A flight plan that collides with the building 3 may be created like the flight route C1.

Even so, if the drone 20 has an avoidance function to avoid obstacles, it can avoid collision with the building 3, but the avoidance function may be defective, so the possibility of collision with the building 3 is reduced. not 0. In this embodiment, by instructing the prohibition of flight as described above, the probability of the drone 20 colliding with a new object such as the building 3 is reduced regardless of whether the 3D map is up-to-date. be able to.

It should be noted that, in this embodiment, the changing region specifying unit 103 does not specify changing regions for moving objects. Therefore, the no-fly instruction unit 104 also does not instruct no-fly when the new object not shown on the 3D map is a moving object. If the new object is a moving object, it will soon disappear from the position where the location information was measured, so prohibiting flight is useless. By not prohibiting moving objects from flying, such waste can be prevented.

After specifying the changed region, the changed region specifying unit 103 supplies change information indicating the specified changed region to the region photographing instructing unit 105 . The region photographing instruction unit 105 instructs the photographing drone 30 to photograph the changed region specified by the changed region specifying unit 103 . The imaging drone 30 is an example of the "second flying object" of the present invention, and the area imaging instruction section 105 is an example of the "imaging instruction section" of the present invention.

When the changing region E1 shown in FIG. 6 is identified, the region photographing instruction unit 105 transmits to the photographing drone 30 instruction data for instructing photographing of the changing region E1 indicated by the supplied change information. The instruction receiving unit 301 of the imaging drone 30 receives an instruction indicated by the transmitted instruction data, that is, an instruction for imaging the changing area by the area imaging instruction unit 105 . The instruction receiving unit 301 notifies the flight control unit 302 of the received instruction.

The flight control unit 302 controls the flight of the aircraft based on the notified instruction, and causes the aircraft to fly to the instructed change area. Further, the flight control unit 302 detects obstacles based on the measurement results of the obstacle detection sensor, and causes the aircraft to fly so as to avoid the detected obstacles. The instruction receiving unit 301 also notifies the area imaging unit 303 of the received instruction. When the own machine reaches the changed area, the area photographing unit 303 photographs the changed area based on the notified instruction.

For example, when photographing of the changing region E1 is instructed, the region photographing unit 303 photographs the direction of the changing region E1 at a plurality of locations while traveling around the changing region E1. At this time, the imaging drone 30 may collide with the building 3 if it gets too close to the changing area E1, but it flies while avoiding collision with the building 3 under the control of the flight control unit 302 . Therefore, while the imaging drone 30 flies around including the building 3 that is not included in the changing area E1, the area capturing unit 303 captures the changing area E1.

The region photographing unit 303 generates image data in which the position (latitude, longitude, height), direction, and distance to the object in the image, which are measured at the time of photographing, are associated with the photographed image of the changing region E1. and transmits it to the server device 10 . The area image acquiring unit 106 of the server device 10 acquires the transmitted image data as data representing the image of the changing area captured by the imaging drone 30 . The area image acquiring unit 106 supplies the acquired image data to the 3D map updating unit 107 .

The 3D map update unit 107 updates the 3D map based on the supplied image data, that is, the image of the changing area photographed according to the instruction of the area photographing instruction unit 105 . The 3D map updater 107 is an example of the "updater" of the present invention. When the image data of the changed area is supplied, the 3D map updating unit 107 updates the position (latitude, longitude, height), the direction, and the distance to the object in the image indicated by the supplied image data. , calculate the latitude, longitude and height of the surface of the object (eg building 3) in the image.

The 3D map updating unit 107 calculates coordinates in the 3D coordinate system used in the 3D map, corresponding to the calculated latitude, longitude and height of the object in the image. The 3D map update unit 107 reads out the latest 3D map from the 3D map storage unit 101 and reflects the calculated coordinates of the object in the read 3D map. For example, the 3D map updating unit 107 updates the z coordinate to the coordinate of the object in the portion where the x coordinate and the y coordinate are common to the coordinate of the object in the set of coordinates indicated by the 3D map.

For example, when coordinates (x1, y1, z1) are represented on the 3D map and coordinates of the object are (x1, y1, z2), the 3D map updating unit 107 updates (x1, y1, z1) are updated to (x1, y1, z2). If the object has vertical walls like a building, then (x1, y1, z2), (x1, y1, z3), (x1, y1, z4), . It is represented by a plurality of coordinates with different z-coordinates.

In this case, the 3D map updating unit 107 updates a location, which is one coordinate in the 3D map, with a plurality of coordinates. By updating the 3D map as described above, when the drone 20 flies near an object that has newly appeared in the actual space and detects it as an obstacle, the object is reflected in the 3D map and a new 3D map is created. A map-based flight plan can create a flight path that avoids the object.

In the present embodiment, the changing region specifying unit 103 does not specify changing regions for moving objects. In this case, an instruction for photographing to the photographing drone 30 is not issued. If the new object is a moving object, it will soon disappear from the position where the arrangement information was measured, so even if a photographing instruction is given, it will be useless. Such waste can be prevented by not issuing a photographing instruction for a moving object.

Further, the region photographing instruction unit 105 instructs the photographing drone 30 to repeatedly fly for photographing the changing region specified by the changing region specifying unit 103 . When the new object that appears in the changed area is a building, even if the changed area is photographed once and the 3D map is updated, the shape and size of the building may change as the construction progresses. Therefore, by repeatedly issuing photographing instructions, the 3D map can be continuously updated until a new object such as a building is completed.

The changed area specifying unit 103 also supplies change information indicating the specified changed area to the construction plan acquisition unit 108 . The construction schedule acquisition unit 108 acquires schedule information indicating a schedule for completion of construction of a building when an area in which a building under construction exists is specified as a changed area. The construction schedule acquisition unit 108 is an example of the "schedule acquisition unit" of the present invention. The building schedule acquisition unit 108 stores, for example, the scheduled completion date of the building in association with the position of the building in the region represented by the three-dimensional map.

When the change information is supplied, the building schedule acquisition unit 108 acquires, as schedule information, the scheduled completion date of the building located in the position of the change area indicated by the supplied change information. Note that the construction schedule acquisition unit 108 may inquire about the scheduled completion date of a pre-registered external system for managing buildings, and acquire the scheduled completion date sent in response to the inquiry as schedule information. The construction schedule acquisition unit 108 supplies the acquired schedule information to the area photographing instruction unit 105 .

The area photographing instruction unit 105 instructs photographing if the current date is before the scheduled completion date of the building indicated by the supplied schedule information, and instructs photographing if the current date is after the scheduled completion date. do not instruct In this way, the area photographing instruction unit 105 instructs the photographing drone 30 to finish photographing the changed area when the construction end time indicated by the schedule information acquired by the construction schedule acquiring unit 108 comes. This can prevent unnecessary photography after the building is completed.

Each device included in the 3D map system 1 performs update processing for updating the 3D map based on the above configuration.
FIG. 8 shows an example of the operation procedure of each device in the update process. The operation procedure of FIG. 8 is started, for example, when the drone 20 flies along the planned flight route. First, the drone 20 (placement information measurement unit 202) measures the placement information indicating the placement of objects existing in the three-dimensional space (step S11), and transmits the measured placement information to the server device 10 (step S12).

The server device 10 (placement information acquisition unit 102) acquires the transmitted placement information as an output of the placement detection sensor (step S13). Next, the server device 10 (changed area identification unit 103) identifies changed areas in which the arranged objects are changed in comparison with the 3D map based on the acquired arrangement information (step S14). Subsequently, the server device 10 (area imaging instruction unit 105) transmits instruction data for instructing imaging of the specified changing area to the imaging drone 30 (step S15).

The imaging drone 30 (flight control unit 302) controls the flight of the drone based on the instruction indicated by the transmitted instruction data, and flies the drone to the instructed change area (step S21). Next, when the imaging drone 30 (area imaging unit 303) reaches the changing area, it takes an image of the changing area based on the instruction indicated by the instruction data (step S22), and sends the image data of the imaged changing area to the server. It is transmitted to the device 10 (step S23).

The server device 10 (area image acquiring unit 106) acquires the transmitted image data as data representing the image of the changing area captured by the imaging drone 30 (step S24). Then, the server device 10 (the 3D map updating unit 107) updates the 3D map based on the image of the changed area indicated by the transmitted image data (step S25).

In this embodiment, even if the imaging drone 30 does not search for the changed area, the drone 20 flying for a purpose other than searching for the changed area detects an obstacle to identify the changed area, and the changed area is photographed. and updating of the 3D map. As a result, photographing for updating the 3D map can be performed more efficiently than when the photographing drone 30 flies to search for the changed area.

[2] Modifications The embodiment described above is merely an example of implementation of the present invention, and may be modified as follows. Moreover, the embodiment and each modification may be combined as necessary. When combining the embodiment and each modification, prioritize each modification (order to determine which is given priority when conflicting events occur when implementing each modification). good too.

[2-1] Aircraft In the embodiment, a rotorcraft-type aircraft was used as the aircraft, but the invention is not limited to this. The flying object may be, for example, a fixed-wing (airplane type) type flying object, a helicopter type flying object, or a VTOL (Vertical Take-Off and Landing Aircraft) type flying object.

[2-2] Layout Detection Sensor In the embodiment, a sensor for detecting obstacles was used as the layout detection sensor, but the layout detection sensor is not limited to this. For example, an image sensor may be used as the placement detection sensor. In that case, the drone 20 is equipped with a digital camera and captures an image formed on the image sensor of the digital camera.

In this case, the changed area specifying unit 103 specifies, as a changed area, an area where the object recognized from the image output from the image sensor is different from the object shown on the 3D map. The image captured by the digital camera includes not only the flight path of the drone 20 but also objects existing around it. Therefore, not only new objects on the flight path, but also changed areas that exist within the range that can be captured by the drone 20 can be reflected in the 3D map.

[2-3] Changed region In the embodiment, the changed region specifying unit 103 specified a region where a new object appeared as a changed region. may be specified as A missing object is, for example, a demolished building, relocated equipment, or a felled tree.

[2-4] Apparatus for realizing each function The apparatus for realizing each function shown in FIG. 5 is not limited to the apparatus described above. For example, the function realized by the server device 10 may be realized by the drone 20 or the imaging drone 30 . Further, the drone 20 may implement the functions implemented by the imaging drone 30 . In this case, the drone 20 is instructed to photograph the changed area, and the drone 20 goes to photograph the changed area unless there is an original flight schedule for transportation or the like. In either case, it is sufficient that the three-dimensional map system 1 as a whole implements each function shown in FIG.

[2-5] Shooting instruction target In the embodiment, the area shooting instruction unit 105 instructs the shooting drone 30, which is a specific drone, to shoot an area. You can change it. For example, when there are a plurality of drones capable of photographing the changing area identified by the changing area identifying unit 103, the area photographing instruction unit 105 instructs a drone that satisfies a predetermined condition for proximity to the changing area to photograph. It is assumed to be a drone that

The predetermined condition is, for example, a condition that only drones closest to the change area satisfy. In addition, the condition may be satisfied by the drones closest to the change area up to the N (N is a natural number) drones, or the condition may be satisfied by the drones whose distance to the change area is less than the threshold. good too. Regardless of whether the pointing target is the imaging drone 30 or the drone 20, the position of the drone may be the position of the office where the drone is located, or the position of the drone during flight.

When the closeness of the distance from the changing area is used to determine the pointing target, the changing area can be captured more quickly than when the distance from the changing area is not considered. In addition, for example, when there are a plurality of drones capable of photographing the changed regions specified by the changed region specifying unit 103, the region shooting instruction unit 105 also sets the degree of freedom of the allowable flight path to a predetermined degree. A drone that satisfies the conditions is designated as a drone that instructs photography.

For example, it is assumed that the degree of freedom of the flight path is higher for a drone that has fewer restrictions based on its flight purpose. Constraints based on the purpose of flight, for example, include many restrictions for drones that transport packages, but fewer restrictions for drones that finish transporting packages and return to bases. In addition, the degree of freedom of the flight path may be higher for a drone with a longer flight distance or a drone with a higher obstacle avoidance capability.

In short, a drone with fewer flight route options or less flight time has more restrictions, and a drone with more flight route options or more flight time has fewer restrictions. The predetermined condition is, for example, a condition that only drones with the highest degree of freedom of allowed flight paths satisfy. In addition, the condition may be satisfied by the N (N is a natural number) drones having the highest degree of freedom of allowed flight paths.

When the degree of freedom of the flight path is used to determine the referent, compared to when the degree of freedom of the flight path is not considered, it is easier to take a flight path for imaging the changing area, and the imaging of the changing area is easier. can be done more reliably. In addition, according to this modification, compared to the case where the same drone is always instructed to shoot, it is possible to shoot the changing area more quickly or more reliably.

[2-6] Category of the Invention The present invention provides a three-dimensional map system 1 including an information processing device such as the server device 10 described above, the information processing device, and flying objects such as the drone 20 and the imaging drone 30. It can also be regarded as an information processing system such as Further, the present invention can be regarded as an information processing method for realizing processing executed by an information processing apparatus, and can also be regarded as a program for causing a computer that controls the information processing apparatus to function. The program regarded as the present invention may be provided in the form of a recording medium such as an optical disc storing the program,
The program may be provided in a form such that it is downloaded to a computer via a network such as the Internet, and the downloaded program is installed and made available.

[2-7] Functional Blocks The block diagrams used in the description of the above embodiments show blocks in functional units. These functional blocks (components) are realized by any combination of at least one of hardware and software. Also, the method of implementing each functional block is not particularly limited.

That is, each functional block may be implemented using one device that is physically or logically coupled, or directly or indirectly using two or more devices that are physically or logically separated (e.g. , wired, wireless, etc.) and may be implemented using these multiple devices. A functional block may be implemented by combining software in the one device or the plurality of devices.

Functions include judging, determining, determining, calculating, calculating, processing, deriving, investigating, searching, checking, receiving, transmitting, outputting, accessing, resolving, selecting, choosing, establishing, comparing, assuming, expecting, assuming, Broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping, assigning, etc. can't For example, a functional block (component) that makes transmission work is called a transmitting unit or a transmitter. In either case, as described above, the implementation method is not particularly limited.

[2-8] Handling of input/output information, etc. Input/output information, etc. may be stored in a specific location (for example, memory), or may be managed using a management table. Input/output information and the like can be overwritten, updated, or appended. The output information and the like may be deleted. The entered information and the like may be transmitted to another device.

[2-9] Judgment method Judgment may be made by a value represented by 1 bit (0 or 1), by a boolean value (Boolean: true or false), or by a numerical value may be performed by a comparison of (eg, a comparison with a predetermined value).

[2-10] Processing Procedures, etc. The processing procedures, sequences, flowcharts, etc. of each aspect/embodiment described in the present disclosure may be rearranged in order as long as there is no contradiction. For example, the methods described in this disclosure present elements of the various steps using a sample order, and are not limited to the specific order presented.

[2-11] Handling of input/output information, etc. Input/output information, etc. may be stored in a specific location (for example, memory), or may be managed in a management table. Input/output information and the like can be overwritten, updated, or appended. The output information and the like may be deleted. The entered information and the like may be transmitted to another device.

[2-12] Software Software, whether referred to as software, firmware, middleware, microcode, hardware description language or otherwise, includes instructions, instruction sets, code, code segments, program code, programs , subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, and the like.

Software, instructions, information, etc. may also be sent and received over a transmission medium. For example, the software uses wired technology (coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), etc.) and/or wireless technology (infrared, microwave, etc.) to create websites, Wired and/or wireless technologies are included within the definition of transmission medium when sent from a server or other remote source.

[2-13] Information, Signals Information, signals, etc. described in this disclosure may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may refer to voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these. may be represented by a combination of

[2-14] "judgment", "decision"
As used in this disclosure, the terms "determining" and "determining" may encompass a wide variety of actions. "Judgement", "determining" are, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiring (eg, lookup in a table, database, or other data structure), ascertaining as "judged" or "determined", and the like.

Also, "judgment" and "determination" are used for receiving (e.g., receiving information), transmitting (e.g., transmitting information), input, output, access (accessing) (for example, accessing data in memory) may include deeming that a "judgment" or "decision" has been made. In addition, "judgment" and "decision" are considered to be "judgment" and "decision" by resolving, selecting, choosing, establishing, comparing, etc. can contain. In other words, "judgment" and "decision" may include considering that some action is "judgment" and "decision". Also, "judgment (decision)" may be read as "assuming", "expecting", "considering", or the like.

[2-15] Meaning of "based on" The phrase "based on" as used in this disclosure does not mean "based only on," unless explicitly stated otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

[2-16] "Different"
In the present disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean that "A and B are different from C". Terms such as "separate,""coupled," etc. may also be interpreted in the same manner as "different."

[2-17] "and", "or"
In the present disclosure, regarding configurations that can be implemented with either “A and B” or “A or B,” the configuration described in one expression may be used as the configuration described in the other expression. For example, when "A and B" are described, they may be used as "A or B" as long as they are not inconsistent with other descriptions and practicable.

[2-18] Variations of Aspects, etc. Each aspect/embodiment described in the present disclosure may be used alone, may be used in combination, or may be used by switching according to execution. In addition, notification of predetermined information (for example,
Notification of “being X”) is not limited to being performed explicitly, but may be performed implicitly (for example, by not notifying the predetermined information).

Although the present disclosure has been described in detail above, it should be apparent to those skilled in the art that the present disclosure is not limited to the embodiments described in this disclosure. The present disclosure can be practiced with modifications and variations without departing from the spirit and scope of the present disclosure as defined by the claims. Accordingly, the description of the present disclosure is for illustrative purposes and is not meant to be limiting in any way.

Reference Signs List 1 3D map system 10 Server device 20 Drone 30 Shooting drone 101 3D map storage unit 102 Location information acquisition unit 103 Changing area identification unit 104 No flight instruction unit 105 Region photographing instruction unit 106 Region image acquisition unit 107 3D map update unit 108 Building schedule acquisition unit 201 Flight control unit 202 Layout information measurement unit 301 Instruction reception unit 302 Flight control unit , 303 . . . area photographing unit.

Claims (8)

a storage unit that stores a 3D map showing the arrangement of objects existing in a 3D space;
an output acquisition unit that acquires an output from a sensor capable of detecting the arrangement of surrounding objects in the three-dimensional space, provided in the first flying object that flies in the three-dimensional space;
a specifying unit that specifies a changed area in which a change occurs in an object arranged in comparison with the 3D map, based on the obtained output;
a photographing instruction unit that instructs the second flying object to photograph the identified change area ;
a schedule acquisition unit for acquiring schedule information indicating a schedule for completion of construction of the building when an area in which the building under construction exists is specified as the changed area;
with
The photographing instruction unit instructs the second flying object to repeatedly fly for photographing the specified changing area, and when the construction completion time indicated by the acquired schedule information comes, the changing area is photographed. instructing the second flying object to finish photographing
Information processing system. 2. The information processing system according to claim 1, further comprising an updating unit that updates the 3D map based on the image captured by the instruction. When there are a plurality of flying objects capable of photographing the specified change area, the photographing instruction unit specifies either the closeness of the distance from the change area or the degree of freedom of the allowed flight path. 3. The information processing system according to claim 1 or 2, wherein the second flying object is the flying object that satisfies the condition. the sensor is a sensor for detecting an obstacle that may collide with the first flying object;
The information processing system according to any one of claims 1 to 3, wherein the specifying unit specifies, as the changed area, an area in which the obstacle is present indicated by the output. The information processing system according to claim 4, wherein the photographing instruction unit does not issue the instruction when the identified obstacle is a moving object. the sensor is an image sensor;
4. The specifying unit according to any one of claims 1 to 3, wherein the specifying unit specifies a region in which an object recognized from an image that is an output of the image sensor is different from an object shown on the 3D map as the changed region. Information processing system. 7. A prohibition instructing unit that instructs the flying object to prohibit flight in the space where the object exists when a region containing a new object not shown in the 3D map is specified as the changed region. The information processing system according to any one of. The information processing system according to claim 7, wherein the prohibition instructing unit does not instruct the prohibition of flight when the new object is a moving object.

Images