WO2021166845A1 - Information processing device, information processing method, and program - Google Patents

Abstract

The present invention addresses the problem of accurately identifying a position even indoors.　An image data acquisition unit 111 of a position identifying device 1, which estimates a position of a drone D on the basis of an image in which a marker in real space was captured as a subject from the position of the drone D and a distance from the position of the drone D to a wall, acquires data on the image. A relative position estimation unit 112 estimates, on the basis of the image of the marker included in the image corresponding to the data, a relative position including at least a distance, a direction, and an angle with respect the marker. A distance acquisition unit 113 acquires the distance from the position of the drone D to the wall. A spatial information acquisition unit 114 acquires information about the arrangement of a plurality of objects including the marker and the wall as spatial information. A position identifying unit 115 estimates the position of the drone D, on the basis of the relative position with respect to the marker, the distance to the wall, and the spatial information. Accordingly, the problem is solved.

Description

Information processing equipment, information processing methods, and programs

The present invention relates to an information processing device, an information processing method, and a program.

Conventionally, in a guidance system that guides an unmanned aircraft to land at a predetermined point, by having a display unit that displays an index from a designated landing position to a predetermined position, the index imaged from the imaging unit of the unmanned aircraft can be used. Based on this, a technique for landing an unmanned air vehicle has been proposed (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2018-206089

However, in the prior art including the technique described in Patent Document 1, the index is displayed on the display unit in association with the position of the unmanned aircraft, and the display unit itself that controls the display is costly.
In addition, there is a technique for performing takeoff and landing control based on position information specified by using GPS (Global Positioning System). However, there is a problem that the accuracy of the position information specified by using GPS deteriorates at a position where GPS radio waves cannot be directly received, such as indoors.
That is, in indoors where GPS cannot be used, for example, it was possible to deal with it by using indicators at a plurality of positions, but in this case, there was a trade-off relationship between cost and accuracy.

An object of the present invention is to accurately specify the position even indoors.

In order to achieve the above object, the information processing device of one aspect of the present invention is
An information processing device that estimates a predetermined position based on an image of a first object in the real space captured from a predetermined position as a subject to be imaged and a distance from the predetermined position with respect to the second object in the real space. hand,
An image data acquisition means for acquiring the first data of the image, and
Based on the image of the first object included in the image corresponding to the first data, a relative position including at least one of a distance, a direction, and an angle from the predetermined position with respect to the first object is estimated. Relative position estimation means and
A distance acquisition means for acquiring the distance from the predetermined position with respect to the second object, and
Spatial information acquisition means for acquiring information on the arrangement of the first object and a plurality of objects including the second object in the real space as spatial information.
A position specifying means for specifying the predetermined position based on the relative position from the predetermined position with respect to the first object, the distance from the predetermined position with respect to the second object, and the spatial information.
To be equipped.

Each of the information processing methods and programs of one aspect of the present invention is a method and program corresponding to the information processing device of one aspect of the present invention.

According to the present invention, the position can be accurately specified even indoors.

It is a figure which shows the outline of the maneuvering control system including the drone provided with the position specifying device which concerns on one Embodiment of this invention. It is a block diagram which shows an example of the hardware composition of the position identification apparatus provided in the drone of FIG. It is a functional block diagram which shows an example of the functional structure of the position specifying apparatus of FIG. It is a figure which shows an example of the situation where various information is acquired by the drone provided with the position identification device of FIG. It is a figure which shows the example different from FIG. 4 among the example of the situation where various information is acquired by the drone provided with the position identification apparatus of FIG. It is a figure which shows each of the example of the image acquired by the position identification apparatus of FIG. 3 in the situation shown in each of FIGS. 4 and 5. FIG. 3 is a diagram showing an example of a situation in which the position specifying device of FIG. 3 controls a region to be imaged in the situation of FIG. It is a figure which shows an example of the marker which is the object of the image acquired by the position specifying apparatus of FIG. FIG. 5 is a flowchart illustrating an example of a flow of position identification processing executed by the position identification device having the functional configuration of FIG. It is a figure which shows the example different from FIG. 4 and FIG. It is a figure which shows the example different from FIG.4, FIG.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Hereinafter, the information processing system according to the present invention will be described with reference to the embodiment, and the information processing system including the small unmanned aerial vehicle (hereinafter referred to as “drone”) D capable of moving in three-dimensional space will be described with reference to the drawings. In the figure, the same reference numerals are given to the same elements, and duplicate description will be omitted. However, the subject of the term "drone" in the present invention is not limited to a small unmanned aerial vehicle that can move in three-dimensional space. That is, for example, a drone that is a traveling vehicle that can move in a two-dimensional space is also included.

In the following description, unless otherwise specified, the directions defined as follows are used.
That is, hereinafter, in the case of assuming a propeller type drone D in a state of flying alone as in Patent Document 1 described above, the direction is such that the main propeller rotates to generate a thrust against gravity. The axis passing through the center of gravity of the drone excluding cargo etc. is called the axis DZ. Here, when the rotation speed of the main propeller is increased from the state where the drone D is hovering at a certain position, the drone increases in altitude and rises. This direction is appropriately referred to as "the direction in which the axis DZ is positive" and the opposite is appropriately referred to as "the direction in which the axis DZ is negative". Therefore, according to the above definition, the axis DZ is fixed to the drone D regardless of the attitude of the drone D.

Further, in the following description, a three-dimensional Cartesian coordinate system including the following axes X, Y, and Z is used. That is, the axis Z of the three-dimensional Cartesian coordinate system is taken in the direction of the axis DZ of the drone D arranged in the attitude of the normal flight state. Further, the direction opposite to the direction in which gravity acts is defined as the direction of the axis Z. Also, take an axis X and an axis Y that are orthogonal to each other so as to be orthogonal to the axis Z. At this time, the directions of the axes X and Y are defined so that the three-dimensional coordinate system using the axes X, Y, and Z is a right-handed system. That is, when the drone D moves without changing its height, the drone D flies in the XY plane including the axis X and the axis Y.
Hereinafter, based on the above definition, the direction in which the height increases is referred to as "the direction in which the axis Z is positive", and the opposite is referred to as "the direction in which the axis Z is negative". Further, the directions of the axis X are referred to as "the direction in which the axis X is positive" and "the direction in which the axis X is negative", and the directions of the axis Y are referred to as "the direction in which the axis Y is positive" and "the direction in which the axis Y is negative". They are called "axis Y is in the negative direction" respectively.

FIG. 1 is a diagram showing an outline of a maneuvering control system including a drone including a position specifying device according to an embodiment of the present invention.

In the maneuvering control system shown in FIG. 1, a drone D equipped (mounted) with a position specifying device 1 and a driver terminal 2 operated by a driver U are connected via wireless communication. Here, the drone D and the operator terminal 2 are connected to each other via a predetermined network N such as the Internet (not shown). The network N includes not only the Internet and mobile carrier networks, but also short-range wireless communications such as NFC (registered trademark) and Bluetooth (registered trademark).

Further, as shown in FIG. 1, the drone D provided with the position specifying device 1 acquires position information by receiving a GPS signal transmitted from a GPS (Global Positioning System) satellite GS.
That is, normally, the drone D receives the GPS signal transmitted from the GPS satellite GS and uses the position information of the drone D itself acquired for autonomous control, or transmits the position information to the operator terminal 2. .. The drone D is not limited to GPS, and can use positioning systems using various satellites such as the quasi-zenith satellite system in Japan.

The operator terminal 2 is a terminal composed of a smartphone or the like and used by the operator U to operate the drone D. By operating the driver terminal 2, the operator U can directly control the drone D, as well as issue a takeoff / landing command, a preset predetermined work command, and the like.

However, as described above, the accuracy of the position information specified by using GPS may deteriorate indoors or the like where GPS radio waves cannot be directly received. That is, for example, the accuracy of the position information of the drone D itself based on the GPS signal is insufficient in a place where GPS radio waves hardly reach indoors or in a place near a large building (wall) even outdoors. There was something. Furthermore, the accuracy of the position information of the drone D itself based on the GPS signal has an upper limit on the accuracy of the position information of the drone D itself due to the standard of the satellite positioning system such as GPS.
The position identification device 1 of the present embodiment is provided with the configuration described later, so that the position of the drone D itself can be determined indoors, in a place near a large building (wall), or with an accuracy not limited to the GPS standard. Can be identified.

FIG. 2 is a block diagram showing an example of the hardware configuration of the position specifying device provided in the drone of FIG.

The position specifying device 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, an output unit 16, and an input unit. A 17, a storage unit 18, a communication unit 19, and a drive 20 are provided.

The CPU 11 executes various processes according to the program recorded in the ROM 12 or the program loaded from the storage unit 18 into the RAM 13.
Data and the like necessary for the CPU 11 to execute various processes are also appropriately stored in the RAM 13.

The CPU 11, ROM 12 and RAM 13 are connected to each other via the bus 14. An input / output interface 15 is also connected to the bus 14. An output unit 16, an input unit 17, a storage unit 18, a communication unit 19, and a drive 20 are connected to the input / output interface 15.

The output unit 16 is composed of a display, a speaker, and the like, and outputs various information as images and sounds.

The input unit 17 is composed of a keyboard, a mouse, etc., and inputs various information.

The storage unit 18 is composed of a hard disk, a DRAM (Dynamic Random Access Memory), or the like, and stores various data.
The communication unit 19 communicates with another device (such as the operator terminal 2 in the example of FIG. 1) via the network N including the Internet.

A removable media 31 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately mounted on the drive 20. The program read from the removable media 31 by the drive 20 is installed in the storage unit 18 as needed.
Further, the removable media 31 can also store various data stored in the storage unit 18 in the same manner as the storage unit 18.

Although not shown, the operator terminal 2 in FIG. 1 has basically the same configuration as the hardware configuration shown in FIG. 2 except for the following points. Therefore, the description of the hardware configuration of the operator terminal 2 will be omitted.

Hereinafter, the functional configuration of the position specifying device 1 shown in FIG. 3 will be described with reference to an example of specifying the position of the drone D itself in the position specifying device 1 provided in the drone D shown in FIG.

As described above, the position identification device 1 specifies the position of the drone D itself as a series of processes for identifying the position of the drone D itself (hereinafter, referred to as "position identification process"). Can be executed.
In order to realize such position identification processing, the steering control system of FIG. 1 has a functional configuration as shown in FIG.

FIG. 3 is a functional block diagram showing an example of the functional configuration of the position specifying device of FIG.
FIG. 4 is a diagram showing an example of a situation in which various information is acquired by a drone equipped with the position identification device of FIG.

When the position identification process is executed, the position identification device 1, the image pickup unit 41, the distance measuring unit 42, the drive unit 43, and the image pickup unit drive unit 44 provided in the drone D function.

As described above, the position specifying device 1 provided in the drone D identifies the position of the drone D based on various information. That is, the position specifying device 1 in the present embodiment is an information processing device provided in the drone D and executing information processing related to control. Therefore, in FIG. 3, the drone D is illustrated as a configuration including (mounting) the position specifying device 1.

First, the imaging unit 41, the distance measuring unit 42, the driving unit 43, and the imaging unit driving unit 44 included in the drone D, which acquires various data with the position specifying device 1 and controls each unit, will be described.

The image pickup unit 41 includes a lens, an optical element such as a CCD or CMOS, and a control unit thereof, and captures an image or video. Further, although details will be described later with reference to FIG. 5, the image pickup unit 41 includes a set of a plurality of lenses and optical elements capable of capturing images of a plurality of angles of view.

The distance measuring unit 42 includes various signal transmitting devices, various sensors, a control unit thereof, and the like, and measures the distance from the distance measuring unit 42 to the point to be measured. Specifically, for example, a light wave range finder using a light wave such as a radar range finder or a radio wave range finder using a radio wave is adopted as the distance measuring unit 42. Furthermore, a device capable of acquiring a two-dimensional or three-dimensional structure to be measured can also be adopted as the distance measuring unit 42. That is, for example, a radar or LiDAR (Light Detection and Ringing) can be adopted as the distance measuring unit 42.

The drive unit 43 is driven by using the supplied energy. The drone D can move in space by driving the drive unit 43 or the like. A motor driven by using electric energy and an engine driven by using chemical energy such as gasoline are both examples of the driving unit 43.

The image pickup unit drive unit 44 can drive the above-mentioned image pickup unit 41 with a gimbal or the like. That is, although the details will be described later with reference to FIG. 7, the image pickup unit drive unit 44 drives a gimbal having a plurality of rotation axes to drive a gimbal having a plurality of rotation axes to drive the gimbal having the plurality of rotation axes to drive the gimbal having the plurality of rotation axes to drive the gimbal having the plurality of rotation axes, respectively. The area imaged by the imaging unit 41 can be changed by rotating the direction of.

Before explaining each functional block that functions in the position specifying device 1, first, the drone D of the example shown in FIG. 4, the imaging unit 41 and the distance measuring unit 42 included in the drone D, and the marker to be imaged are described. , The arrangement with the wall or the like to be measured for the distance will be described.

In the example of FIG. 4, when the position identification device 1 provided in the drone D executes the position identification process, the drone D flying in the space adopting the above-mentioned three-dimensional Cartesian coordinate system has the axis Y in the positive direction. This is an example shown from.
In the example of FIG. 4, the wall W is arranged in the direction in which the axis X is negative with respect to the drone D. Further, the ground G is arranged in the direction in which the axis Z is negative with respect to the drone D. Here, the marker 3 is installed on the ground G. An example of a specific shape of the marker 3 will be described later with reference to FIGS. 6 and 8.
The drone D includes an imaging unit 41 that images a direction in which the axis Z is negative. The two dotted lines V1-a and V1-b indicate the angle of view of the imaging unit 41 provided in the drone D. That is, the imaging unit 41 images the region between the intersection of the dotted line V1-a and the ground G and the intersection of the dotted line V1-b and the ground G. That is, in FIG. 4, the imaging unit 41 captures an image including the image of the marker 3. Although not shown, it goes without saying that the angle of view of the imaging unit 41 also extends in the direction of the axis Y.
Further, the drone D includes a distance measuring unit 42 that measures a distance in a direction in which the axis X is negative. Specifically, for example, the distance D1 is the distance between the point WP on the wall W and the point S1 on the distance measuring unit 42. That is, the distance measuring unit 42 provided in the drone D measures the distance D1 between the point WP and the point S1 as the distance in the direction in which the axis X is negative.

Hereinafter, each functional block that functions in the position specifying device 1 of FIG. 3 will be described with reference to FIG.

In the CPU 11 of the position identification device 1, the image data acquisition unit 111, the relative position estimation unit 112, the distance acquisition unit 113, the spatial information acquisition unit 114, the position identification unit 115, the attitude control unit 116, and the imaging area control Unit 117 and are functioning.

The image data acquisition unit 111 acquires image data captured from the position of the drone D with the marker 3 in the real space as the object to be imaged. Specifically, the image data acquisition unit 111 acquires image data from the image pickup unit 41 included in the drone D via the communication unit 19. As described above, this image data includes an image of the marker 3 in which the axis Z of the drone D is installed on the ground in the negative direction and is captured.

The relative position estimation unit 112 determines a relative position including at least one of a distance, a direction, and an angle from the position of the drone D with respect to the marker 3 based on the image of the marker 3 included in the image corresponding to the image data. presume. That is, the relative position estimation unit 112 estimates the position relative to the position of the drone D with respect to the marker 3 based on the image of the marker 3 included in the image corresponding to the image data acquired by the image data acquisition unit 111. do.

Here, the relative position is a second position grasped as a set of coordinate values in a coordinate system having an origin at the first position. On the other hand, the absolute position is a position grasped as a set of coordinate values with respect to one common coordinate system. That is, for example, when a three-dimensional Cartesian coordinate system is used as the coordinate system, the relative position from the first position to the second position is the first from the coordinate value (set) of the absolute position of the second position. It is possible to adopt the value obtained by subtracting (a set) of the coordinates of the absolute position of the position.
However, in the present specification, the relative position from the first position to the second position will be described as being determined by a distance, a direction, and an angle. That is, a three-dimensional polar coordinate system is used as the coordinate system representing the relative position. That is, the relative position from the first position to the second position includes the distance, the azimuth angle, and the elevation / depression angle. The azimuth is an angle that determines the relative position of the XY plane (horizontal plane). That is, the azimuth corresponds to the azimuth (direction) seen from the drone D. The elevation / depression angle is an angle in the vertical direction with respect to the horizontal.

As will be described in detail later, the relative position represented by the three-dimensional polar coordinate system estimated by the relative position estimation unit 112 has the distance, azimuth angle, and elevation / depression angle depending on the arrangement of the image of the marker 3 in the captured image. Each accuracy changes. Under certain conditions, it is conceivable that only one of the distance, direction, and angle is highly accurate. Therefore, in the following description, the state in which the relative position is estimated is not only the state in which the three-dimensional relative position is accurately obtained, but also any one of the distance, the azimuth angle, and the elevation / depression angle is predetermined. It will be described as a state estimated by the accuracy of.

Hereinafter, a method in which the relative position estimation unit 112 estimates the relative position of the drone D with respect to the marker 3 based on the image of the marker 3 included in the image corresponding to the data acquired by the image data acquisition unit 111. , FIG. 4 will be described as appropriate.

The relative position estimation unit 112 determines the distance D2 from the position of the drone D with respect to the marker 3 based on the size of the marker 3 included in the image corresponding to the data acquired by the image data acquisition unit 111. (See FIG. 4) is estimated. When the marker 3 is placed on the ground and the distance between the marker 3 and the drone D is sufficiently large, the distance D2 as a relative position corresponds to the altitude of the so-called drone D.
In the example of FIG. 4, the marker 3 is located directly below the drone D. Based on the image corresponding to the data acquired by the image data acquisition unit 111, the relative position estimation unit 112 sets the distance D2 as the relative position between the point MP on the marker 3 and the point S2 on the image pickup unit 41. Estimate the distance.
Although not shown, the drone D usually includes an altimeter including a rangefinder in a direction in which the axis Z is negative. Therefore, the relative position estimation unit 112 may use the altitude information measured by the altimeter provided in the drone D as the distance D2 from the marker 3.

Further, the relative position estimation unit 112 is based on the position where the marker 3 is included in the image corresponding to the data acquired by the image data acquisition unit 111 and the posture of the drone D. The azimuth and elevation / depression angles from the position of the drone D with respect to 3 are estimated. That is, although not shown, the drone D usually includes a gyro sensor or the like for acquiring the angle of the drone D's own posture. Further, the area of the image captured by the imaging unit 41 changes due to the change in the posture of the drone D. That is, the relative position estimation unit 112 can estimate the azimuth angle and the elevation / depression angle after grasping which region the imaging unit is capturing based on the posture of the drone D.
Even when the marker 3 is not imaged with a sufficient size, the relative position estimation unit 112 can estimate the azimuth angle and the elevation / depression angle with a predetermined accuracy when it can be grasped that the marker 3 is a marker 3.

The distance acquisition unit 113 acquires the distance from the position of the drone D with respect to the wall W. That is, the distance acquisition unit 113 acquires the distance D1 measured by the distance measurement unit 42.
As described above, in the example of FIG. 4, the distance measuring unit 42 provided in the drone D measures the distance D1 between the point WP and the point S1 as the distance in the direction in which the axis X is negative. The distance acquisition unit 113 can acquire the distance D1 from the distance measuring unit 42 provided in the drone D via the communication unit 19.

The spatial information acquisition unit 114 acquires information on the arrangement of a plurality of objects including the marker 3 and the wall W in the real space as spatial information. That is, the spatial information acquisition unit 114 acquires spatial information from the operator terminal 2.

Here, the spatial information is information on the arrangement of a plurality of objects including the marker 3 and the wall W in the real space. Specifically, for example, the spatial information includes various objects including an object with which the drone D may come into contact, a ground G on which the marker 3 is arranged, and an object whose distance can be measured by the distance measuring unit 42. Information that includes placement information.
In the example of FIG. 4, the spatial information includes information on the arrangement of the wall W, information on the arrangement of the ground G, and information on the arrangement of the marker 3. That is, for example, information such as "where the wall W exists at the coordinate of the axis X" is included in the spatial information. Further, for example, information such as "where the ground G exists at the coordinate (altitude) of the axis Z" is included in the spatial information. Further, for example, information such as "where the absolute coordinates are where the marker 3 exists" is included in the spatial information.

The position specifying unit 115 identifies the position of the drone D based on the relative position of the drone D with respect to the marker 3, the distance from the position of the drone D with respect to the wall W, and the spatial information.
That is, the position specifying unit 115 is acquired by the relative position including the distance D2 with respect to the marker 3 estimated by the relative position estimation unit 112, the distance D1 to the wall W acquired by the distance acquisition unit 113, and the spatial information acquisition unit 114. The position of the drone D is specified based on the spatial information.

Specifically, for example, the spatial information includes the absolute coordinates of the marker 3. Therefore, the position specifying unit 115 can specify the absolute coordinates of the position of the drone D based on the relative position estimated by the relative position estimating unit 112 and the spatial information. The position specifying unit 115 can improve the accuracy of the specified position based on the distance D1 with the wall W acquired by the distance acquisition unit 113 and the spatial information including the position of the wall W.
That is, for example, as described above, the accuracy of the relative coordinates changes depending on the arrangement of the image of the marker 3 in the captured image. For example, in the example of FIG. 4, when the distance D2 is large and the accuracy of the azimuth angle and the elevation / depression angle is poor, the coordinates of the drone D on the XY plane in absolute coordinates have a large error. Even in such a case, the position specifying unit 115 can improve the accuracy of the specified position based on the distance D1 from the wall W and the spatial information including the position of the wall W. Furthermore, if the coordinates of the drone D on the XY plane in absolute coordinates have a large error, the drone D may come into contact with the wall W whose axis X is arranged in the negative direction. However, the drone D of the present embodiment can directly measure the distance D1 with respect to the wall W. As a result, the autonomously controlled drone D can be prevented from coming into contact with the wall W.

The attitude control unit 116 controls the attitude and position of the drone D based on the position of the drone D specified by the position specifying unit 115 and the distance D1 from the wall W. That is, the attitude control unit 116 can control the drive unit 43 via the communication unit 19.
Thereby, as described above, it is possible to control the distance D1 with respect to the wall W so as not to be shorter than a predetermined distance. That is, it is possible to prevent the drone D from coming into contact with the wall W.
Further, for example, as will be described later, when the marker 3 protrudes from the region imaged by the imaging unit 41 and is imaged, the posture can be controlled so that the entire marker 3 is imaged. As a result, the relative position estimation unit 112 can estimate the relative position with the marker 3. An example in which the marker 3 protrudes from the region imaged by the imaging unit 41 and is imaged will be described later with reference to FIGS. 5 and 6.

The imaging area control unit 117 controls to change the area imaged by the imaging unit 41. That is, for example, the imaging unit 41 can be turned by a gimbal including an imaging unit driving unit 44. The image pickup area control unit 117 can control to change the image pickup area by controlling the image pickup unit drive unit 44 via the communication unit 19. An example in which the imaging region control unit 117 changes the imaging region will be described later with reference to FIG. 7.

As described above, the functional configuration of the position specifying device 1 shown in FIG. 3 has been described with reference to an example of specifying the position of the drone D itself in the position specifying device 1 provided in the drone D shown in FIG.
Hereinafter, an example in which the marker 3 protrudes from the region imaged by the imaging unit 41 and is imaged will be described with reference to FIGS. 5 and 6.

FIG. 5 is a diagram showing an example different from that of FIG. 4 among an example of a situation in which various information is acquired by a drone equipped with the position specifying device of FIG.
FIG. 6 is a diagram showing each of the examples of images acquired by the position identifying device of FIG. 3 in the situations shown in FIGS. 4 and 5, respectively.

In the example of FIG. 5, similarly to FIG. 4, when the position identification device 1 provided in the drone D executes the position identification process, the drone D flying in the space adopting the above-mentioned three-dimensional Cartesian coordinate system is displayed. This is an example in which the axis Y is shown from the positive direction.
In the drone D in the example of FIG. 5, the axis DZ is not parallel to the axis Z. That is, the drone D is in a situation where the aircraft is tilted due to being fanned by the wind or moving horizontally in the direction of the axis X.
Here, the drone D in the example of FIG. 5 includes two imaging units 41-1 and 41-2. The two dotted lines V2-a and V2-b indicate the angle of view of the imaging unit 41-1 provided in the drone D. The first imaging unit 41-1 images an angle of view basically similar to that of the imaging unit 41 in FIG. Further, the two dotted lines V3-a and V3-b indicate the angle of view of the imaging unit 41-2 provided in the drone D. That is, the imaging unit 41-2 captures an image having a wider angle of view than the angle of view of the imaging unit 41-1.

Here, an example of an image captured by the imaging unit 41 of FIG. 4 and an image captured by the imaging unit 41-1 of FIG. 5 will be described.
The figures shown in the situations A and B in FIG. 6 are examples of images taken from the drone D flying in each of the situations of FIGS. 4 and 5, respectively, in which the axis Z is shown from the positive direction. Is.
The figure shown in the A situation of FIG. 6 is an example of an image captured by the imaging unit 41 of FIG. That is, the image data acquisition unit of FIG. 3 acquires the image shown in the situation A of FIG. 5 in the situation of FIG.
The figure shown in the B situation of FIG. 6 is an example of an image captured by the imaging unit 41-1 of FIG. That is, the image data acquisition unit of FIG. 3 acquires the image shown in the situation B of FIG. 5 in the situation of FIG.

That is, the image data acquisition unit 111 can further acquire data of another image having a different angle of view from the image captured from the position of the drone D.
Further, the relative position estimation unit 112 acquires the position of the drone D and the relative position of the marker 3 based on the image of the marker 3 included in at least one of the images corresponding to the data of the plurality of images. can do.

In the situation of FIG. 4, the drone D is not tilted, and the marker 3 is included in the angle of view indicated by the dotted lines V1-a and V1-b. Therefore, the example of the image captured by the imaging unit 41 shown in the situation A in FIG. 6 includes the ground G and the entire marker 3 arranged on the ground G.
In the situation of FIG. 5, the drone D has an inclined body, and the marker 3 protrudes from the angle of view indicated by the dotted lines V2-a and V2-b. Therefore, the example of the image captured by the imaging unit 41-1 shown in the B situation of FIG. 6 includes the ground G and a part of the marker 3 arranged on the ground G.

In this way, when the body of the drone D is tilted, the marker 3 may protrude from the image captured by the imaging unit 41 or the imaging unit 41-1. An image captured by the imaging unit 41-2 is used in such a case.
As described above, the image pickup unit 41-2 provided in the drone D captures an image having an angle of view indicated by the two dotted lines V3-a and V3-b. Looking at FIG. 5, the entire marker 3 is included in the angle of view indicated by the two dotted lines V3-a and V3-b. That is, the image pickup unit 41-2 provided in the drone D captures an image including the entire marker 3. The relative position estimation unit 112 cannot estimate the relative position based on the data of the image captured by the imaging unit 41-2 by estimating the relative position based on the data of the image captured by the imaging unit 41-2. Even in the case, the relative position can be estimated.

Furthermore, if the drone D is provided with a plurality of imaging units having different angles of view, the following uses are possible.
As described above, when the posture of the drone D changes at high speed due to the gust, for example, the posture of the drone D may not be corrected in time. At this time, by providing the imaging unit 41-2 having a wide angle of view, it is possible to capture an image including the entire marker 3 even when the posture changes. As a result, the position specifying device 1 can specify the position even when the attitude changes, and the drone D can fly safely.

Next, an example in which the imaging unit driving unit 44 drives the imaging unit 41-1 with a gimbal or the like will be described with reference to FIG. 7.
FIG. 7 is a diagram showing an example of a situation in which the position identifying device of FIG. 3 controls a region to be imaged in the situation of FIG.
In the example of FIG. 7, similarly to FIGS. 4 and 5, when the position identification device 1 provided in the drone D executes the position identification process, the drone flies in the space adopting the above-mentioned three-dimensional Cartesian coordinate system. This is an example in which D is shown from the direction in which the axis Y is positive.

The example of FIG. 7 is an example in which the imaging unit 41-1 is driven by the imaging unit driving unit 44 of the drone D of the example of FIG. Here, the two dotted lines V2-a and V2-b are the angles of view of the imaging unit 41-1 provided in the drone D, but the two dotted lines V2-a and V2-b in the example of FIG. 7 are It is shown at a different position from the example of FIG. That is, in the example of FIG. 7, the direction of imaging by the imaging unit 41-1 is changed by driving the imaging unit 41-1 by the imaging unit driving unit 44 of the drone D of the example of FIG. Is shown.

The image pickup area control unit 117 can control to change the real space area to be imaged as an image. That is, the image pickup area control unit 117 can control the region to be imaged by the image pickup unit 41-1 by driving the gimbal or the like of the image pickup unit 41-1 via the image pickup unit drive unit 44. can. Although it depends on the number of pixels of the image captured by the imaging unit 41 or the like, when estimating the relative position, the accuracy of the relative position is usually improved based on the image captured by the imaging unit 41-1 having a narrow angle of view. do. That is, the accuracy of the relative position is improved based on the image of the marker 3 captured by the camera having a narrow angle of view. That is, the accuracy of relative position estimation is improved by driving the gimbal or the like of the imaging unit 41-1 by the imaging region control unit 117 via the imaging unit driving unit 44.

However, when landing, when the drone D moves in the direction of the axis Z, the imaging unit 41-1 having a narrow angle of view may fly in a region where the entire marker 3 cannot be imaged. In this case, the relative position estimation unit 112 can estimate the relative position based on the image captured by the image pickup unit 41-2 having a wide angle of view.
In this way, by providing the imaging units 41-1, 41-2 and the like for capturing images of a plurality of angles of view, the area in which the relative position can be estimated becomes wide when the drone D flies.
Furthermore, by controlling the area imaged by the image pickup units 41-1, 41-2, etc. by the image pickup area control unit 117, it is possible to estimate the relative position regardless of the posture and position of the drone D. The area becomes wider.

Here, the structure of the marker 3 will be described with reference to an example of an image of the captured marker 3 shown in the situation A of FIG. 6 and an example of another marker 3 shown in FIG.
FIG. 8 is a diagram showing an example of a marker that is an object to be imaged of an image acquired by the position specifying device of FIG.

First, an example of an image of the captured marker 3 shown in the situation A in FIG. 6 will be described.
The marker 3 in the example of the situation A in FIG. 6 is provided with a two-dimensional bar code-like pattern consisting of an arrangement of white-painted squares and black-painted squares inside a square frame with a black border.
The pattern provided by the marker 3 has the following features.

The shape of the marker 3 shown in the situation A in FIG. 6 is not rotationally symmetric with respect to the axis parallel to the axis Z. As a result, the relative position estimation unit 112 can estimate the azimuth angle with respect to the marker 3 from the image captured by the marker 3.

Although not shown, the marker 3 can have the following patterns.
By providing various patterns, the marker 3 can include meanings such as high-precision control and instructions. Specifically, for example, the marker 3 can include a pattern including information on which of the plurality of markers 3 is the marker 3 (hereinafter, referred to as “marker ID”). As a result, the position specifying unit 115 can specify the position based on the arrangement of the plurality of markers 3, the marker ID of the marker 3 being imaged, and the relative position of the marker 3 being imaged.
Further, the marker 3 is provided with a pattern for compensating for the lack of information included in the marker 3 by using a predetermined algorithm so that information such as the azimuth angle and the marker ID can be acquired even when the entire marker 3 cannot be imaged. be able to.
Further, the marker 3 may include a pattern including information on GPS coordinates in which the marker is installed. This makes it possible to authenticate whether the marker is installed at the correct position. The position specifying device 1 may collate the GPS coordinates with the marker ID of the marker 3 being imaged by acquiring a set of information of the marker ID and the GPS coordinates installed on the marker 3.

Next, an example of another marker 3 shown in FIG. 8 will be described.
The marker 3 in the example of FIG. 8 is composed of a marker 3-a having the same structure as the marker 3 in the example of FIG. 6 and a square marker 3-b having a large black border as compared with the marker 3-a. Will be done.
The pattern included in the marker 3 in the example of FIG. 8 has the following features.

By providing a square marker 3-b having a large black border as compared with the marker 3-a, when the marker 3 protrudes from the image captured by the imaging unit 41, the resistance to the protrusion of the marker 3 Can be given. That is, for example, when the drone D disturbs the attitude of the aircraft due to a gust or the like, even when a part of the marker 3 protrudes from the image captured by the imaging unit 41, a large black-edged square marker 3- b is likely to be included in the image captured by the imaging unit 41. As a result, the imaging region control unit 117 can control the region to be imaged by the imaging unit 41 so that the marker 3-a is imaged.

Also, for example, the drone D flying at a high altitude may not be able to grasp the detailed pattern of the relatively small marker 3-a from the captured image. In such a case, the drone D approaches the marker 3 with the target of the square marker 3-b having a large black border as compared with the marker 3-a, so that the detailed pattern of the marker 3-a can be grasped. At that point, the marker ID and the like can be obtained from the marker 3-a, and it can be confirmed that the landing point is correct.

As described above, the relative position estimation unit 112 determines the distance, direction, and direction from the position of the drone D with respect to the marker 3 based on at least one of the large structure of the marker 3 and the structure smaller than the large structure. Relative positions including at least one of the angles can be analyzed.

FIG. 9 is a flowchart illustrating an example of the flow of the position identification process executed by the position identification device having the functional configuration of FIG.

When the position identification device 1 provided in the drone D executes the position identification process, the position identification process is started and the following processes S11 to S15 are executed.

In step S11, the image data acquisition unit 111 acquires image data captured from the position of the drone D with the marker 3 in the real space as the object to be imaged.

In step S12, the relative position estimation unit 112 determines the distance, direction, and angle from the position of the drone D with respect to the marker 3 based on the image of the marker 3 included in the image corresponding to the image data acquired in step S11. Estimate the relative position including at least one of them.

In step S13, the distance acquisition unit 113 acquires the distance from the position of the drone D with respect to the wall W.

In step S14, the spatial information acquisition unit 114 acquires information on the arrangement of a plurality of objects including the marker 3 and the wall W in the real space as spatial information.

In step S15, the position specifying unit 115 is the relative position from the drone D with respect to the marker 3 estimated in step S12, the distance from the position of the drone D with respect to the wall W acquired in step S13, and the position is acquired in step S14. The position of the drone D is specified based on the spatial information.
As a result, the position identification process is completed.

The embodiment of the maneuvering control system including the drone including the position specifying device to which the present invention is applied has been described above. However, the embodiment to which the present invention is applied may be, for example, as follows.

For example, the marker 3 has been described as a two-dimensional bar code-shaped marker in the above-described embodiment, but the marker 3 is not particularly limited thereto. Hereinafter, an example of the marker 3 having a different form will be described with reference to FIGS. 10 and 11.

For example, the position specifying device 1 can also be used in the example shown in FIG.
FIG. 10 is a diagram showing an example different from FIGS. 4 and 5 in an example of a situation in which various information is acquired by a drone equipped with the position specifying device of FIG.

In the example of FIG. 10, when the position identification device 1 provided in the drone D executes the position identification process, the drone D flying in the space adopting the above-mentioned three-dimensional Cartesian coordinate system has the axis Y in the positive direction. This is an example shown from. Although details will be described later, in the example of FIG. 10, the marker 3 is an example in which the string-shaped marker 3-R is adopted.
In the example of FIG. 10, the wall W1 is arranged in the direction in which the axis X is positive with respect to the drone D. Further, the wall W2 is arranged in the direction in which the axis X is negative with respect to the drone D. The ceiling C is arranged in the direction in which the axis Z is positive with respect to the drone D. Further, the ground G is arranged in the direction in which the axis Z is negative with respect to the drone D. Here, a string-shaped marker 3-R is hung from the ceiling C and installed.
That is, the space in the example of FIG. 10 is a space closed by the ceiling C, the walls W1 and W2, and the ground G.

The drone D in the example of FIG. 10 is drawn with the position specifying device 1, the imaging unit 41, and the distance measuring unit 42, as in the example of FIG. Here, unlike the example of FIG. 4, the imaging unit 41 is imaging the string-shaped markers 3-R hanging from the ceiling C instead of the markers 3 arranged on the ground.
Even in this case, the relative position estimation unit 112 can estimate the relative position based on the image of the marker 3-R imaged by the image pickup unit 41. As described above, as the marker 3, not only the marker 3 installed on the floor but also the string-shaped marker 3-R hung from the ceiling C can be adopted.
The markers 3-R may be a prominently colored rope or a rod built from the ground G. Although such markers 3-R are easy to install, they are effective even in a cylindrical building such as a chimney or a building with high symmetry. Furthermore, although not shown, a plurality of markers 3-R are prepared, and a string shape having a different color for each of the walls W1 and W2 and the wall in which the axis Y (not shown) is in the positive direction and the axis Y is in the negative direction. By hanging the markers 3-R of the above, it is possible to identify the wall. The position of the drone D can be specified by identifying which wall it is by using the markers 3-R.

Also, for example, the position specifying device 1 can be used in the example shown in FIG.

FIG. 11 is a diagram showing an example different from FIGS. 4, 5, and 9 among an example of a situation in which various information is acquired by a drone equipped with the position identification device of FIG.

In the example of FIG. 11, when the position identification device 1 provided in the drone D executes the position identification process, the drone D flying in the space adopting the above-mentioned three-dimensional Cartesian coordinate system has the axis Y in the positive direction. This is an example shown from. Although details will be described later, in the example of FIG. 10, the marker 3 is an example in which the string-shaped marker 3-R is adopted.
In the example of FIG. 11, the wall W3-a is arranged in the direction in which the axis X is positive with respect to the drone D. Further, the wall W3-b is arranged in the direction in which the axis X is negative with respect to the drone D. Further, the ground G is arranged in the direction in which the axis Z is negative with respect to the drone D. Here, a string-shaped marker 3-R is hung from the ceiling C and installed. Here, in FIG. 11, the wall W3-a and the wall W3-b are drawn by drawing an arc and connecting them. That is, the space in the example of FIG. 11 is a chimney-shaped space.

In the example of FIG. 11, a string-shaped marker 3-R is hung and arranged along the wall W3-a.

Here, the imaging unit 41 included in the drone D images the string-shaped markers 3-R.
At this time, the image captured by the imaging unit 41 includes only the region corresponding to the angle indicated by the angle θ shown in FIG. Furthermore, when the chimney is long, the angle θ is about 90 degrees, and for the drone D, the image is taken as if the markers 3-R exist almost directly above. That is, the image captured by the imaging unit 41 includes the image of the markers 3-R as a large image by the length of the string-shaped markers 3-R.
As described above, in the example of FIG. 11, the markers 3-R are imaged as a relatively large image. In this case, assuming that the axis Z is imaged in the positive direction by the imaging unit 41, the drone D is grasped as if there is a straight line indicating a certain radius with respect to the cross section of the circular chimney. As a result, the relative position estimation unit 112 can use the markers 3-R to specify the relative position in the rotation direction in the chimney symmetrical to the cylinder. In the example of FIG. 11, the markers 3-R may have a rod-like shape protruding from the wall surface. Further, the markers 3-R may be a light such as an LED.
As described above, the marker 3 adopts not only the flat marker having the two-dimensional bar code-like pattern shown in FIGS. 6 and 8 but also the string-shaped or rod-shaped marker shown in FIGS. 10 and 11. be able to.

As described above, the marker 3 may be not only a square but also a rectangular parallelepiped or a straight line (which can be said to be an elongated rectangle). Further, the marker 3 may include two or more lights such as LEDs, and may be drawn with a luminous paint or the like. This makes it easy to image the marker 3 even in a dark place, and the position specifying device 1 can specify the position.
Further, when flying to a high altitude, a rectangular frame is suitable because the accuracy of relative position estimation by image recognition can be improved.

Further, for example, in the description of FIG. 4 and the like, the marker 3 is arranged on the ground G, and the distance measuring unit 42 measures the distance to the wall W, but the present invention is not particularly limited to this. Specifically, for example, the marker 3 may be installed on the wall W, and the distance measuring unit 42 may measure the distance to the ground G. Even in this case, the position specifying device 1 can specify the position.

The drone D is a small unmanned aerial vehicle that can move in three-dimensional space, but the drone D is not particularly limited to this. That is, for example, the drone D may be a vehicle or the like that moves on the ground.
For example, when the position identification device 1 is mounted on a vehicle moving on the ground, the distance between the imaging unit 41 and the marker 3 becomes short when the vehicle recognizes the marker 3 installed on the floor. Such a double marker is suitable. Furthermore, in this case, if the marker 3 is drawn with the luminous paint, it emits light when the vehicle passes over it, so that it is easy to be imaged, and the relative position can be easily estimated by image recognition.

Further, for example, in the above description of the embodiment, the drone D is assumed to include the imaging unit 41. Further, the image pickup unit 41 includes a lens, an optical element such as a CCD or CMOS, and a control unit thereof, and is intended to capture an image or video. However, it is not limited to this.
That is, for example, the drone D may have a depth sensor. Specifically, for example, the drone D may have various depth sensors such as a depth camera, a radar, and a LiDAR that can acquire three-dimensional depth information. In this case, the position specifying device 1 acquires the measurement result of the depth sensor as depth information. In this case, the position specifying device 1 can specify the position based on the depth information acquired through the depth sensor instead of the image data acquired by the imaging unit 41.
When a depth sensor is used, it is preferable that the marker has a shape having a predetermined unevenness. That is, the position specifying device 1 can recognize the position in the same manner as the example of the marker 3 of the above-described embodiment by recognizing the marker having a predetermined unevenness included in the depth information. Further, as the shape of the marker, the shape shown in FIGS. 6 and 8 can be used even when the depth sensor is used.
Furthermore, the drone D may have both an imaging unit 41 and a depth sensor. In this case, the position specifying device 1 can specify the position based on the image data captured by the imaging unit 41 and the depth information acquired via the depth sensor. With such a configuration, the marker for using the imaging unit 41 and the marker for using the depth sensor can be used properly depending on the installation position of the marker.
Furthermore, as the marker, a marker having both features such as color and fluorescence for using the imaging unit 41 and predetermined unevenness for using the depth sensor can be adopted. This makes it a marker that can be used by both the image pickup unit 41 and the drone D having either the depth sensor.

Further, for example, in the description of the above-described embodiment, the distance acquisition unit 113 acquires the distance measured by the distance measurement unit 42, but the distance acquisition unit 113 is not particularly limited to this. That is, for example, it is sufficient for the distance measuring unit 42 to be able to acquire the distance from the drone D to an object other than a marker or the like in the real space.
Specifically, for example, the distance acquisition unit 113 can acquire the distance based on the position of the drone D and the angle with another object. That is, for example, at a certain time t1, it is assumed that the position specifying device 1 has been able to specify the position of the drone D by an arbitrary method including GPS. Next, at time t2, which is a time after time t1, the distance acquisition unit 113 analyzes the data of the image captured by the imaging unit such as a camera, so that another object exists as seen from the drone D. Obtain the angle to be used (for example, azimuth and elevation / depression angle). Further, the distance acquisition unit 113 estimates the distance between the drone D and the other object based on the angle and spatial information in which the other object exists when viewed from the drone D. As a result, the distance acquisition unit 113 can acquire the distance between the drone D and another object. Needless to say, the above-mentioned angle may be corrected and used based on the attitude information (tilt and direction of the aircraft) of the drone D.
Furthermore, in the above case, the distance acquisition unit 113 can acquire a variation (amount of change in position) of the position of another object by performing image analysis on the image data. This makes it possible to estimate and obtain the distance between the drone D and another object.

Further, for example, the present embodiment can also exert the following effects.
Specifically, for example, when both the relative position with the marker and the distance with another object such as a wall can be acquired, the accuracy of the position specified by the position specifying device 1 simply specifies the position based on the marker. Compared to doing, it improves.
Further, for example, the relative position of the floor (floor on which the marker is installed) with the marker can be acquired, and the position of another object can be estimated from the relative position with the marker and the spatial information.
Hereinafter, for example, a case where another object is a steel tower will be specifically described. The technology related to the flight of the drone D around the steel tower is an important technology in applications such as inspection of the steel tower using the drone D.
First, it is difficult to constantly measure the distance between the drone D and another object, a steel tower, with a distance measuring sensor capable of measuring a distance from one point. Further, for example, when having a structure that repeats a similar structure, it may be difficult to measure an accurate distance even with a depth sensor or the like capable of three-dimensional distance measurement. Even in such a case, the position identification device 1 improves the accuracy of position identification based on the information obtained as a result of movement such as orbiting around the tower while measuring the distance to the tower. Can be done. However, it may not be possible to obtain the relative position with the marker while orbiting the tower. Therefore, the position specifying device 1 prevents the steel tower from colliding with the drone D based on the measured distance, and obtains information on the distance between the towers for the laps and the markers acquired a plurality of times during the laps. The position can be specified based on the relative position information. As a result, the position specifying device 1 can compress the error by, for example, processing such as averaging the results of a plurality of measurements.
Furthermore, there are cases where grass grows around the tower. In this case, when the altitude above ground level is measured using a simple distance sensor, it is not possible to specify at which height of the tower the drone D is located. However, by arranging the marker on the ground near the steel tower as described above, the position specifying device 1 can specify the position (elevation to ground level) based on the relative position with the marker. Furthermore, the accuracy of the position relative to the marker deteriorates as the distance from the marker increases. However, by using the distance from the steel tower, the position specifying device 1 can improve the accuracy of specifying the position based on the distance from the steel tower, and can also prevent contact with the steel tower.

Further, for example, in the description of the above-described embodiment, the imaging unit 41 is configured to include a set of a plurality of lenses and optical elements capable of capturing images of a plurality of angles of view. Not limited. That is, for example, it goes without saying that the drone D has a plurality of imaging units and may be capable of capturing images having a plurality of angles of view.
Hereinafter, the effect of being able to capture images with a plurality of angles of view will be described.
Specifically, for example, when there are images having different angles of view (telephoto degree), as described with reference to FIG. 7, even when the posture of the drone D is disturbed, an image having a wide angle of view can be used. It is more likely that the marker will be included and imaged. As a result, the position identification device 1 can improve the accuracy of position identification. Further, by correcting the posture, it is possible to capture the image as an image having a narrow angle of view and improve the accuracy of the position specified by the position specifying device 1.
Further, for example, the region imaged by the imaging unit 41 may be variable. That is, for example, the lens and the set of the image sensor of the image pickup unit 41 may be movable so as to be able to take an image in various directions. As a result, it is possible to take an image for specifying the position, or to take an image of a steel tower as an inspection target in the above example. Furthermore, even when another object in the vicinity moves (for example, a power transmission line swaying by the wind), the movable imaging unit makes it possible to always include the other object within the angle of view.

Further, for example, in the above description of the embodiment, the position specifying device 1 is assumed to be mounted on the drone D, but the present invention is not particularly limited to this. That is, for example, the position identification device 1 may be provided on the ground so that the image captured from the drone D and the distance information may be transmitted to the position identification device 1 on the ground. As a result, for example, it is not necessary to equip the drone D with hardware for executing high-load data processing, and the cost can be reduced.

Further, for example, the above-mentioned series of processes can be executed by hardware or software.
In other words, the functional configuration of FIG. 3 is merely an example and is not particularly limited.
That is, it suffices if the steering control system is provided with a function capable of executing the above-mentioned series of processes as a whole, and what kind of functional block is used to realize this function is not particularly limited to the example of FIG. Further, the location of the functional block is not particularly limited to FIG. 3, and may be arbitrary. For example, the functional block of the position specifying device 1 may be transferred to the operator terminal 2 or the like.
Further, one functional block may be configured by a single piece of hardware, a single piece of software, or a combination thereof.

Further, for example, when a series of processes are executed by software, a program constituting the software is installed on a computer or the like from a network or a recording medium.
The computer may be a computer embedded in dedicated hardware.
Further, the computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose smartphone or a personal computer in addition to a server.

Further, for example, a recording medium containing such a program is not only composed of a removable medium (not shown) distributed separately from the device main body in order to provide the program to the operator U, but is also preliminarily incorporated in the device main body. It is composed of a recording medium or the like provided to the operator U in the state of being in the state.

In the present specification, the steps for describing a program to be recorded on a recording medium are not necessarily processed in chronological order, but also in parallel or individually, even if they are not necessarily processed in chronological order. It also includes the processing to be executed.
Further, in the present specification, the term of the system means an overall device composed of a plurality of devices, a plurality of means, and the like.

In other words, the information processing apparatus to which the present invention is applied can take various embodiments having the following configurations.

That is, the information processing device to which the present invention is applied (for example, the position specifying device 1 in FIG. 3) is
An image of a first object (for example, marker 3) in real space captured from a predetermined position (for example, the position of drone D) as a subject to be imaged, and a second object (for example, wall W) in the real space. An information processing device that estimates the predetermined position based on the distance from the predetermined position.
An image data acquisition means for acquiring the first data of the image (for example, the image data acquisition unit 111 in FIG. 3) and
Based on the image of the first object included in the image corresponding to the first data, a relative position including at least one of a distance, a direction, and an angle from the predetermined position with respect to the first object is estimated. Relative position estimation means (for example, relative position estimation unit 112 in FIG. 3) and
A distance acquisition means for acquiring the distance from the predetermined position with respect to the second object (for example, the distance acquisition unit 113 in FIG. 3) and
Spatial information acquisition means (for example, spatial information acquisition unit 114 in FIG. 3) for acquiring information on the arrangement of the first object and a plurality of objects including the second object in the real space as spatial information.
A position specifying means (for example, FIG. 3) that estimates the predetermined position based on the relative position from the predetermined position with respect to the first object, the distance from the predetermined position with respect to the second object, and the spatial information. Positioning part 115) and
An information processing device equipped with the above is sufficient.

As a result, the information processing device can improve the accuracy of the specified position not only based on the relative position estimated based on the image of the first object but also based on the distance to the second object. Furthermore, by setting an object that has a risk of contact with a moving body or the like as the second object, the risk of direct contact with the second object can be reduced.

Further, the image data acquisition means further acquires second data of another image having an angle of view different from that of the image captured from the predetermined position.
The relative position estimation means is based on the image of the first object contained in at least one of the image corresponding to the first data and the second data and each of the other images, and the predetermined position. And the relative position with respect to the first object can be obtained.

Further, it is possible to further include an imaging region control means that controls to change the region of the real space imaged as at least one of the image and the other image.

Further, the relative position estimating means determines the distance, direction, and distance from the predetermined position with respect to the first object based on at least one of a large structure of the first object and a structure smaller than the large structure. And relative positions including at least one of the angles can be analyzed.

D ... Drone, 1 ... Position identification device, 2 ... Operator terminal, 11 ... CPU, 111 ... Image data acquisition unit, 112 ... Relative position estimation unit, 113 ... Distance acquisition unit, 114 ... Spatial information acquisition unit, 115 ... Position identification unit, 116 ... Attitude control unit, 117 ... Imaging area control unit, 19 ... Communication unit, 41 ... Imaging Unit, 42 ... Distance measuring unit, 43 ... Drive unit, 44 ... Imaging unit drive unit

Claims (6)

1. An information processing device that estimates a predetermined position based on an image of a first object in the real space captured from a predetermined position as a subject to be imaged and a distance from the predetermined position with respect to the second object in the real space. hand,
   An image data acquisition means for acquiring the first data of the image, and
   Based on the image of the first object included in the image corresponding to the first data, a relative position including at least one of a distance, a direction, and an angle from the predetermined position with respect to the first object is estimated. Relative position estimation means and
   A distance acquisition means for acquiring the distance from the predetermined position with respect to the second object, and
   Spatial information acquisition means for acquiring information on the arrangement of the first object and a plurality of objects including the second object in the real space as spatial information.
   A position specifying means for specifying the predetermined position based on the relative position from the predetermined position with respect to the first object, the distance from the predetermined position with respect to the second object, and the spatial information.
   Information processing device equipped with.
2. The image data acquisition means further acquires second data of another image having an angle of view different from that of the image captured from the predetermined position.
   The relative position estimation means is based on the image of the first object contained in at least one of the image corresponding to the first data and the second data and each of the other images, and the predetermined position. And the relative position with respect to the first object,
   The information processing device according to claim 1.
3. Further comprising an imaging region control means for controlling to change the region of the real space imaged as at least one of the image and the other image.
   The information processing device according to claim 2.
4. The relative position estimating means is based on at least one of a large structure of the first object and a structure smaller than the large structure, and the distance, direction, and angle from the predetermined position with respect to the first object. Analyze relative positions including at least one of
   The information processing device according to any one of claims 1 to 3.
5. An information processing device that estimates the predetermined position based on the image of the first object in the real space captured from the predetermined position as the object to be imaged and the distance from the predetermined position to the second object in the real space is executed. In the information processing method
   An image data acquisition step for acquiring the first data of the image, and
   Based on the image of the first object included in the image corresponding to the first data, a relative position including at least one of a distance, a direction, and an angle from the predetermined position with respect to the first object is estimated. Relative position estimation step and
   A distance acquisition means for acquiring the distance from the predetermined position with respect to the second object, and
   A spatial information acquisition step for acquiring information on the arrangement of the first object and a plurality of objects including the second object in the real space as spatial information.
   A position estimation step for estimating the predetermined position based on the relative position from the predetermined position with respect to the first object, the distance from the predetermined position with respect to the second object, and the spatial information.
   Information processing methods including.
6. A computer that estimates the predetermined position based on the image of the first object in the real space captured from the predetermined position as the object to be imaged and the distance from the predetermined position to the second object in the real space.
   An image data acquisition step for acquiring the first data of the image, and
   Based on the image of the first object included in the image corresponding to the first data, a relative position including at least one of a distance, a direction, and an angle from the predetermined position with respect to the first object is estimated. Relative position estimation step and
   A distance acquisition means for acquiring the distance from the predetermined position with respect to the second object, and
   A spatial information acquisition step for acquiring information on the arrangement of the first object and a plurality of objects including the second object in the real space as spatial information.
   A position estimation step for estimating the predetermined position based on the relative position from the predetermined position with respect to the first object, the distance from the predetermined position with respect to the second object, and the spatial information.
   A program that executes control processing including.

Images