WO2019003492A1

WO2019003492A1 - Control device, flying body, and control program

Info

Publication number: WO2019003492A1
Application number: PCT/JP2018/006001
Authority: WO
Inventors: 皓正高塚; 一希笠井; 正寛小原; 慶之谷
Original assignee: オムロン株式会社
Priority date: 2017-06-27
Filing date: 2018-02-20
Publication date: 2019-01-03
Also published as: TWI693959B; TW201904643A; JP2019008676A; JP6988197B2

Abstract

The present invention addresses the problem of suitably projecting an image to a target position. A control device (10) controls a mobile body (1) that is provided with a projection unit (3). The control device (10) is provided with: a picked up image acquisition unit (16) that acquires a picked up image (8); and a projection control unit (6), which refers to the picked up image (8), and controls the position of the mobile body (1) and/or attitude of the mobile body (1) and/or direction of the projection unit (3).

Description

Controller, aircraft, and control program

One aspect of the present invention relates to a control device, an aircraft, and a control program.

BACKGROUND ART Conventionally, there is known a technique for projecting an image by a moving body provided with a projection unit. Patent Document 1 discloses a technique for presenting information using a projector mounted on an unmanned aerial vehicle.

Japanese Patent Publication No. "Japanese Unexamined Patent Publication No. 2017-76084 (April 20, 2017)"

However, in the prior art as described above, it is not easy to project an image to a target position by the projection unit provided on the moving body.

Therefore, one embodiment of the present invention aims to preferably project an image to a target position.

In order to solve the above-mentioned subject, a control device concerning mode 1 of the present invention is a control device which controls a mobile provided with a projection part, and a captured image including a projection image which the projection part projected is included The projection image acquisition unit to acquire, and the projection control unit that controls at least one of the position of the movable body, the attitude of the movable body, and the direction of the projection unit with reference to the imaged image. It is characterized by

According to the above configuration, it is possible to realize a control device that projects an image suitably to a target position.

In the control device according to aspect 2 of the present invention, the projection control unit extracts a projection position of the projection image and a target projection position from the captured image, and the projection position projected by the moving object is the target The projection position projected by the moving body may be controlled to be a projection position.

According to the above configuration, it is possible to realize a control device that projects an image such that the projection position projected by the moving object is the target projection position.

Further, in the control device according to aspect 3 of the present invention, the projection control unit may determine the correction amount of the projection position by calculating the difference between the projection position of the current projection image and the target projection position. Good.

Further, in the control device according to the fourth aspect of the present invention, the projection control unit is configured to, from the captured image, a first feature point indicating a projection position of the projection image at present and a second feature indicating a target projection position. The correction amount may be determined by extracting points respectively and calculating the difference in position between the first feature point and the second feature point.

According to the above configuration, control for projecting the image more accurately by calculating the difference between the first feature point and the second feature point of the target projection position from the current projection position to obtain the correction amount The device can be realized.

Further, the control device according to aspect 5 of the present invention may control the projection position projected by the moving body with reference to a plurality of captured images captured at different times by the same imaging device.

According to the above configuration, the correction amount of the projection position can be calculated from the temporal change of the projection image.

Moreover, the control apparatus which concerns on aspect 6 of this invention may control the projection position which the said mobile body projects by controlling the position or attitude | position of the said mobile body.

According to the above configuration, the control device can control the projection position by controlling the position or attitude of the moving body.

Moreover, the control apparatus which concerns on the aspect 7 of this invention may control the projection position which the said mobile body projects by controlling the projection direction of the said projection part.

According to the above configuration, the control device can control the projection position by controlling the projection direction of the projection unit.

Further, in the control device according to aspect 8 of the present invention, the captured image acquisition unit may acquire, as the captured image, a captured image captured by an imaging device included in the moving body.

According to said structure, a control apparatus can control a projection position using the captured image which the imaging device with which a mobile body was equipped imaged.

Further, in the control device according to aspect 9 of the present invention, the captured image acquisition unit may acquire, as the captured image, a captured image captured by an imaging device provided other than the moving body.

According to the above configuration, the control device can control the projection position using the captured image captured by the imaging device provided other than the mobile object.

Further, the control device according to aspect 10 of the present invention may control a flying object as the moving object.

According to the above configuration, it is possible to realize a control device that projects an image to a target position using a flying object as a moving object.

Further, in the control device according to the eleventh aspect of the present invention, the image projected by the projection unit may include information for assisting the work by the worker.

According to the above configuration, it is possible to realize a control device that presents information for assisting the worker at an accurate position.

The flying object according to aspect 12 of the present invention is a flying object including a projection unit, and a captured image acquisition unit that acquires a captured image including a projection image projected by the projection unit; The projection control unit controls at least one of the position of the flight object, the attitude of the flight object, and the direction of the projection unit.

According to the above configuration, at least one of the position of the flying object, the attitude of the flying object, and the direction of the projection unit can be controlled to realize the flying object that projects an image to a target position.

A control program according to aspect 13 of the present invention is a control program for causing a computer to function as the control device according to one aspect of the present invention, and functions the computer as the captured image acquisition unit and the projection control unit. Is a control program to

According to the above configuration, the same effect as that of the above aspect 1 can be obtained.

According to one aspect of the present invention, it is possible to realize a control device that projects an image suitably to a target position.

It is a figure which shows schematic structure of the unmanned aerial vehicle concerning Embodiment 1 of this invention. BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram which shows schematic structure of the unmanned aerial vehicle concerning Embodiment 1 of this invention. It is a sequence diagram which shows the flow of the projection position correction process which concerns on Embodiment 1 of this invention. It is a figure showing example 1 of projection position amendment concerning embodiment 1 of the present invention. It is a figure which shows Example 2 of the projection position correction | amendment which concerns on Embodiment 1 of this invention. It is a block diagram which shows schematic structure of the projection system which concerns on Embodiment 2 of this invention. It is a sequence diagram which shows the flow of the projection position correction process which concerns on Embodiment 2 of this invention.

Hereinafter, embodiments of the present invention will be described in detail based on the drawings. However, the configuration described in the present embodiment is not intended to limit the scope of the present invention to only that, unless specifically described otherwise, and is merely an illustrative example. Moreover, about the member which has the function same as the member shown to each embodiment for convenience of explanation, the same code | symbol is attached | subjected and the description is abbreviate | omitted suitably.

Embodiment 1
(Configuration of the UAV 1)
FIG. 1 is a view showing an outline of an unmanned aerial vehicle 1 (mobile body, flight body) according to a first embodiment of the present invention. In addition, a mobile body is not limited to the unmanned aerial vehicle 1, It can also be set as a self-propelled robot etc. which are not a flight body according to a use.

When projection is performed using an aircraft including the unmanned aerial vehicle 1, it is generally difficult to project to a target position. However, according to the control device 10 according to the present embodiment, it is possible to project an image to a target position even using a flying object.

As shown in FIG. 1, the unmanned aerial vehicle 1 includes blades 2a to 2d, a camera 7 (imaging device), and a projector 3 (projector). Although not shown in FIG. 1, a motor for driving the blades 2a to 2d is present inside the unmanned aerial vehicle 1, and controls the flight of the unmanned aerial vehicle 1.

The projector 3 projects the image 8. In the following, when simply referred to as an "image", the image projected by the projector 3 is referred to. The image 8 may include information for assisting the work by the worker. The information for assisting the work is, for example, an image for assisting the worker to pick an object smoothly in the warehouse in the present embodiment. The projector 3 accurately projects the image 8 to the specified position, so that it is possible to accurately present the position of the object in the warehouse to the operator via the image 8.

In addition, the projector 3 may project information including an instruction to another apparatus, a work robot, and the like, as well as an instruction to a human worker. In that case, the information projected by the projector 3 may be configured to include an instruction for the other device or the other work robot as optical information.

The information projected by the projector 3 is not particularly limited as long as it is information using light, in other words, information transmitted by light. Further, the wavelength of light to be used is not particularly limited. For example, the light may be light outside the visible region, infrared light, ultraviolet light, or the like. However, when an instruction for a worker who is a human being is included in the information projected by the projector 3, the light preferably includes a visible light region.

In the present embodiment, the camera 7 is an optical detection element, and performs imaging in a visible light region, for example. However, the wavelength range which the camera 7 picks up is not limited in particular, as long as the camera 7 can detect the light used for the projector 3 to project information, it may detect the light outside the visible range.

(Configuration of the UAV 1)
The configuration of the unmanned aerial vehicle 1 will be described with reference to FIG. FIG. 2 is a block diagram showing the configuration of the unmanned aerial vehicle 1. As shown in FIG. 2, the unmanned aerial vehicle 1 includes blades 2a to 2d, projectors 3 (projection units), motors 4a to 4d, a control device 10, a projection information acquisition unit 14, and an imaging device 18. The control device 10 includes a projection information storage unit 13, a captured image acquisition unit 16, and a projection control unit 6. The projection control unit 6 includes a captured image processing unit 11, a correction amount calculation unit 17, a flying object control unit 12, and a projector control unit 25.

In the present embodiment, the unmanned aerial vehicle 1 acquires projection information, which is information about the image 8 to be projected, from the external device or the like by the projection information acquisition unit 14. As an example, the projection information also includes projection position designation information for designating the projection position of the image 8. However, this does not limit the present embodiment, and the projection information acquisition unit 14 may It is good also as composition acquired separately from a device etc. Here, the projection position specification information is information for specifying a projection position on which the projection image is to be projected. The control device 10 controls the position and the projection direction of the unmanned aerial vehicle 1 with reference to the projection position designation information.

Also, the projection information may be information that changes with time according to the situation of the unmanned aerial vehicle 1.

The projection information acquisition unit 14 supplies the acquired projection information to the projection information storage unit 13. The projection information storage unit 13 stores projection information.

The captured image processing unit 11 supplies the projection information acquired from the projection information storage unit 13 to the projector 3 (projection unit). The projector 3 projects the image 8.

The camera 7 (imaging device) acquires an image including a projected image of the image 8 as a captured image. Hereinafter, in the case of “captured image”, it refers to an image captured by the camera 7. Depending on the imaging direction of the camera 7 and the projection direction of the projector 3, the “captured image” may not show a projection image of the image projected by the projector 3 at all.

The imaging device 18 supplies a captured image to the captured image acquisition unit 16 of the control device 10. The captured image acquisition unit 16 supplies the captured image acquired from the camera 7 to the captured image processing unit 11 of the projection control unit 6. The captured image processing unit 11 refers to the projection information and extracts the current projection position and the target projection position of the image 8 from the captured image. Here, the target projection position is a projection position designated by the projection position designation information. Specific extraction processing of the projection position by the captured image processing unit 11 will be described later. In the control as an example, the unmanned aerial vehicle 1 is moved to a rough position with reference to the map in the warehouse, and then the target projection position is extracted from the captured image. In such control, the target projection position is not included in the captured image before moving to the rough position, and the target projection position is included in the captured image after moving to the rough position. It will be.

The captured image processing unit 11 supplies the current projection position and the target projection position extracted from the captured image to the correction amount calculation unit 17. The correction amount calculation unit 17 calculates the difference between the current projection position and the target projection position, and calculates the correction amount of the projection position. Specific calculation processing of the correction amount will be described later. The “difference” between the current projection position and the target projection position indicates the magnitude and direction of the shift for moving the current projection position to the target projection position.

The correction amount calculation unit 17 supplies the calculated correction amount to the aircraft control unit 12 and the projector control unit 25. The flying body control unit 12 controls the motors 4a to 4d with reference to the correction amount to control the drive of the blades 2a to 2d to control the position or attitude of the unmanned aerial vehicle 1. Further, the projector control unit 25 controls the projection position of the image 8 by controlling the projection direction of the projector 3. Note that controlling the "posture" of a moving object means directing the moving object in a target projection direction, and also directing a self-propelled robot that is not a flying object in a target projection direction. .

As described above, the control of the projection direction of the projector 3 may be performed by controlling only the projection direction of the projector 3 itself without moving the position or posture of the unmanned aerial vehicle 1. Conversely, the control of the projector 3 itself is unmanned The position or attitude of the unmanned aerial vehicle 1 may be controlled without changing the relative projection direction to the aircraft 1. In the case of a configuration in which the relative projection direction of the projector 3 itself is not changed, the projector control unit 25 is not an essential configuration of the projection control unit 6.

The flying object control unit 12 and the projector control unit 25 determine the control method of the unmanned aerial vehicle 1 and the projector 3 in consideration of the performance of the unmanned aerial vehicle, the projected position and the ease of holding the projected attitude.

The flow of the projection position correction process by the unmanned aerial vehicle 1 will be described with reference to FIG. FIG. 3 is a flowchart showing the flow of the projection position correction process.

(Step S004)
First, in step S 004, the projector 3 provided in the unmanned aerial vehicle 1 projects the image 8 based on the projection information supplied from the captured image processing unit 11.

(Step S006)
Subsequently, in step S006, the camera 7 acquires an image including the projected image of the image 8 projected in step S004 as a captured image. The camera 7 supplies the captured image to the captured image acquisition unit 16.

(Step S008)
Subsequently, in step S008, the captured image acquisition unit 16 acquires a captured image acquired by the camera 7. The captured image acquisition unit 16 supplies the acquired captured image to the captured image processing unit 11.

(Step S010)
Subsequently, in step S010, the captured image processing unit 11 extracts the current projection position and the target projection position from the captured image. The captured image processing unit 11 supplies the current projection position and the target projection position to the correction amount calculation unit 17. Specific extraction processing of the projection position by the captured image processing unit 11 will be described later.

(Step S012)
Subsequently, in step S012, the correction amount calculation unit 17 calculates the difference between the current projection position and the target projection position.

(Step S014)
Subsequently, in step S014, the correction amount calculation unit 17 determines whether the difference between the current projection position and the target projection position is within a predetermined value.

If the difference between the current projection position and the target projection position is within the predetermined value, the correction amount calculation unit 17 determines that the projection onto the target projection position has been completed, and the projection position correction processing ends. If the difference between the current projection position and the target projection position is larger than a predetermined value, then the process of step S016 is performed. In addition, what is necessary is just to set the said "fixed value" according to the use by which the control apparatus 10 is used. As an example, a configuration is conceivable in which the correction amount calculation unit 17 calculates the correction amount with one pixel in the captured image as the “constant value”.

(Step S016)
Subsequently, in step S016, the correction amount calculation unit 17 calculates the correction amount of the projection position so as to minimize the difference between the current projection position and the target projection position. Note that “minimizing” the difference is not necessarily limited to a perfect match between the current projection position and the target projection position. If the difference between the current projection position and the target projection position can be projected so as to be within the "constant value" described in the explanation of step S014, the projection control unit 6 determines that the projection has been performed with sufficient accuracy.

(Step S018)
Subsequently, in step S018, the correction amount calculation unit 17 supplies the correction amount obtained in step S016 to the aircraft control unit 12 and the projector control unit 25.

(Step S020)
Subsequently, in step S020, the flying object control unit 12 controls the motors 4a to 4d with reference to the correction amount supplied from the correction amount calculation unit 17 to project the projection position closer to the target projection position. Control the position. The projector control unit 25 controls the projection position so that the projection position approaches the target projection position by controlling the projection direction of the projector 3 with reference to the correction amount supplied from the correction amount calculation unit 17. The control of the unmanned aerial vehicle 1 and the projector 3 has been described above, and thus the description thereof is omitted here.

Subsequently, the process returns to step S 004, and projection is performed again from the projection position after position correction. Subsequently, the processes of steps S006 to S014 are sequentially performed. In step S014, if the difference between the current projection position and the target projection position is within a predetermined value, the projection control unit 6 determines that the projection onto the target projection position has been completed, and ends the projection position correction process. If the difference between the current projection position and the target projection position is larger than a predetermined value, the projection control unit 6 sequentially performs each processing of steps S016 to S020, and returns to step S004 to perform processing again.

The number of times of repeating the processes of steps S 004 to S 020 is not particularly limited, and the process can be performed until the difference between the current projection position and the target projection position becomes within a predetermined value. Further, even when the position of the unmanned aerial vehicle 1 is shifted after the position correction, the position correction can be performed by repeating the processing of S 004 to S 020 again.

In the above description, the camera 7 acquires a single captured image in step S 004, and the correction amount calculation unit 17 calculates the correction amount with reference to the single captured image. A plurality of captured images may be acquired at different times. In this case, while changing the projection position and the projection direction, the correction amount calculation unit 17 refers to the projection positions of the projection images in a plurality of captured images captured at different times by the same camera 7. By extracting the change over time, the correction amount can be calculated. More specifically, for example, the correction amount calculation unit 17 refers to the plurality of captured images captured while changing the projection direction and the projection position, and the projection position and the target projection position of the projection image in the plurality of captured images The direction and the magnitude of the correction can be calculated by analyzing the relative positional relationship with and the time change thereof. As described above, it is possible to calculate the direction and magnitude of correction by extracting the direction and magnitude of the deviation of the projection position by trying to rotate or translate the unmanned aerial vehicle 1.

(Example 1 of projection position correction)
FIG. 4 is a diagram for explaining an example 1 of the projection position correction by the projection control unit 6. In this example, the projector 3 projects one point as an image. The camera 7 acquires a captured image including the current projection position 20 as the point. The captured image processing unit 11 extracts the current projection position 20 as a first feature point from the captured image supplied from the camera 7, extracts the target projection position 21 as a second feature point, and performs projection for the projection. The marks 22a to 22d are also extracted as feature points. Here, the landmarks for projection refer to one or more landmarks to be referred to for specifying a value in the world coordinate system of the projection position 20. A mark for projection is, for example, arranged near the target projection position.
Here, in order to simplify the description, it is assumed that all the marks 22a to 22d for projection are present on the same plane and the coordinates are known in advance. The calculation process is not limited. Also, the projection is performed on a plane, and the current projection position 20 and the target projection position 21 are also on the same plane as the marks 22a to 22d for projection, but this is also the correction amount calculation in this embodiment. It does not limit the process.

The correction amount calculation unit 17 specifies specific values of the actual coordinates (x, y, z) of the current projection position 20 of the point using the coordinates of the marks 22a to 22d for projection. The specific values of the actual coordinates (x ', y', z ') of the target projection position 21 are predetermined by the projection control unit 6 by referring to the projection position designation information. After specifying the specific value of (x, y, z), the correction amount calculation unit 17 calculates the correction amount according to, for example, the method of Calculation Example 1 or 2 described later.

Further, in the present embodiment, in order to simplify the description, the correction of the projection position is made by the translational motion or the rotational motion of the unmanned aerial vehicle 1, but this does not limit the present embodiment.

Note that the method of projection position correction is not limited to the above example. As another method, for example, there is a method in which the correction amount calculation unit 17 calculates the correction amount using a method of directly measuring the correction amount from the image processing or the measurement result of the 3D sensor.

(Calculation example 1: A method of obtaining only translational amount ΔT without considering rotational movement)
One of the calculation examples in which the correction amount calculation unit 17 calculates the correction amount under the conditions described in FIG. 4 will be described.

In the following description, it is assumed that camera coordinates and projector coordinates are calibrated, and the relationship between these two coordinates (how one coordinate is transformed to the other) is known. The camera coordinate system is a coordinate system in which the position of the CCD or lens of the camera 7 is the origin and the optical axis direction of the camera 7 is the Z axis, and the X axis Y axis is set by the right hand system or the left hand system. The projector coordinate system is a coordinate system in which the position of the mirror or light source of the projector 3 is the origin, and the optical axis direction of the projector 3 is the Z axis, and the X axis Y axis is set by the right hand system or the left hand system. Moreover, a world coordinate system is a coordinate provided in the space which the unmanned aircraft 1 moves, and it is a three-dimensional coordinate system provided in the warehouse which the unmanned aircraft 1 moves in this embodiment.

In the following, a transformation matrix from camera coordinates to projector coordinates is expressed as R _pc and T _pc . Here, the transformation matrix R _pc represents rotation, and the transformation matrix T _pc represents translation. Similarly, transformation matrices from the world coordinate system to the camera coordinate system are expressed as R _cw and T _cw . Here, the transformation matrix R _cw represents rotation, and the transformation matrix T _cw represents translation.

When the above notation is used, the correction amount calculation unit 17 uses the world coordinates (x _w , y _w , z _w ) and the following equation 1 to obtain projector coordinates (x _p , y _p , z _p ) can be converted.

The correction amount calculation unit 17 uses the equation 1 to calculate the coordinates (x ′, y ′, z ′) of the target projection position in the world coordinate system and the coordinates (x, y, z) of the current projection position. The coordinates of the target projection position in the projector coordinate system (x _g , y _g , z _g ) and the coordinates of the current projection position in the projector coordinate system (x _a , y _a , z _a ) are converted.

Subsequently, the correction amount calculating section 17, coordinates of the target projection position of the projector coordinate system _{_{_{(x g, y g, z}}} g) and the coordinates of the current projection position in the projector coordinate system _{_{_{(x a, y a, z}}} a The transformation matrices R ^* and T ^* are determined by substituting)) into Equation 2 below. Here, the transformation matrix R ^* represents rotation, and T ^* represents translation.

Generally, although there are a plurality of R ^* and T ^* that satisfy the condition of Equation 2, in this calculation example, only the translation amount ΔT is determined without considering the rotational movement of the unmanned aerial vehicle 1. In other words, in the present example, the correction amount calculation unit 17 sets R ^* as a unit matrix and calculates only the correction amount T ^* = ΔT. The correction amount ΔT calculated by the correction amount calculation unit 17 is supplied to the flying object control unit 12, and the flight object control unit 12 causes the unmanned aircraft to translate by the correction amount indicated by the correction amount to project to the target projection position. It can be performed.

(Calculation example 2: A method of determining only the rotation amount R without considering translational movement)
Next, a method of determining only the amount of rotation R without considering the translational movement of the unmanned aerial vehicle 1 will be described as Calculation Example 2 of the correction amount.

First, as in Calculation Example 1, the coordinates (x ', y', z ') of the target projection position in the world coordinate system and the coordinates (x, y, z) of the current projection position are the target projection in the projector coordinate system. The coordinates (x _g , y _g , z _g ) of the position and the coordinates (x _a , y _a , z _a ) of the current projection position in the projector coordinate system are converted, and substituted into Expression 2 in the same manner as Calculation Example 1.

In this calculation example, it is assumed that T ^* is 0 and translational movement of the unmanned aerial vehicle 1 is not considered. The angle between the two projection position vectors (x _g , y _g , z _g ) and (x _a , y _a , z _a ) in the projector coordinate system, and the rotational axis of the rotational movement of the unmanned aerial vehicle 1 are as follows: Equations 3 and 4 can be used.

The correction amount calculation unit 17 calculates the correction amount R ^* = ΔR by applying the rotation axis and the rotation angle obtained using Equation 3 and Equation 4 to the Rodriguez rotation formula.

The correction amount ΔR calculated by the correction amount calculation unit 17 is supplied to the flying object control unit 12, and the flight object control unit 12 projects the unmanned aircraft to the target projection position by rotating the unmanned aircraft by the correction amount indicated by the correction amount. It can be performed.

(Example 2 of projection position correction)
FIG. 5 is a diagram for explaining an example 2 of the projection position correction. FIG. 5A shows an image 24 projected by the projector 3. FIG. 5 (b) is an example of a captured image captured by the camera 7, which includes the current projected position 20 of the point and the target projected position 21.

The camera 7 acquires a captured image including the current projection position 20 of the point. The captured image processing unit 11 determines from the captured image supplied from the camera 7 the feature points 23a to 23d (first feature points) in the current projection position 20 of the image 24, and a mark for projection (second Feature points) 22a to 22d are extracted.

The captured image processing unit 11 associates each of the feature points 23a to 23d with each of the marks 22a to 22d. More specifically, the correction amount calculation unit 17 obtains a homography matrix from a set of corresponding points, and uses the method of Zhang and the like to determine the homography matrix, and the rotation matrix R (= ΔR) and the translation matrix It decomposes into T (= ΔT). Thereby, the correction amount calculation unit 17 can obtain the correction amounts ΔR and ΔT. The conversion from camera coordinates to projector coordinates may be performed using the above-described equation 1.

Second Embodiment
The second embodiment will be described in detail below with reference to FIG. In the following description, the same reference numerals are assigned to members already described in the above embodiment, and the description thereof is omitted, and only differences from the above embodiment will be described.

(Configuration of control system)
FIG. 6 is a block diagram showing a schematic configuration of a projection system 5 according to a second embodiment. In the present embodiment, the projection system 5 includes the unmanned aerial vehicle 1 and a server 60.

The unmanned aerial vehicle 1 includes blades 2a to 2d, projectors 3, motors 4a to 4d, a control device 10, a projection information acquisition unit 14, and a correction amount reception unit 9. The control device 10 includes a projection information storage unit 13, a captured image processing unit 11 a, a flying object control unit 12, and a projector control unit 25. The projector 3 is not an essential component of the control device 10.

The server 60 includes a projection control unit 6 a, a camera 7, a captured image acquisition unit 16, and a correction amount transmission unit 19. The projection control unit 6 a includes a captured image processing unit 11 b and a correction amount calculation unit 17.

In the present embodiment, the camera 7 provided in the server 60 captures an image projected by the projector 3 provided in the unmanned aerial vehicle 1, and the server 60 extracts and corrects the projection position described in the first embodiment. The amount calculation process is performed, and the calculated correction amount is transmitted from the server 60 to the unmanned aerial vehicle 1.

The flow of the projection position correction process by the projection system 5 according to the present embodiment will be described with reference to FIG. FIG. 7 is a sequence diagram showing a flow of projection position correction processing. In the following, an example in which steps S 004 to S 020 are performed only once has been described, but the present embodiment is not limited to this. As described in the first embodiment, steps S 004 to S 020 may be repeated multiple times until the difference between the target projection position and the current projection position is within a certain range.

(Step S004)
As in the first embodiment, the projector 3 provided in the unmanned aerial vehicle 1 projects the image 8.

(Step S006)
Subsequently, the camera 7 provided in the server 60 acquires an image including the projection image of the image 8 projected in step S004 as a captured image. The camera 7 supplies the captured image to the captured image acquisition unit 16.

(Step S008)
Subsequently, in step S008, the captured image acquisition unit 16 included in the server 60 acquires a captured image acquired by the camera 7. The captured image acquisition unit 16 supplies the acquired captured image to the captured image processing unit 11.

(Step S010)
Subsequently, in step S010, the captured image processing unit 11 extracts the current projection position and the target projection position from the captured image. The captured image processing unit 11 supplies the current projection position and the target projection position to the correction amount calculation unit 17. The specific extraction process of the projection position by the captured image processing unit 11 is the same as that of the first embodiment.

(Step S016)
Subsequently, in step S016, the correction amount calculation unit 17 calculates the correction amount of the projection position so as to minimize the difference between the current projection position and the target projection position. The specific content of step S016 is the same as that of the first embodiment.

(Step S018)
Subsequently, in step S018, the correction amount calculation unit 17 sends the correction amount obtained in step S016 to the correction amount transmission unit 19. The correction amount transmission unit 19 transmits the correction amount to the correction amount reception unit 9 of the unmanned aerial vehicle 1.

(Step S019)
The correction amount reception unit 9 receives the correction amount from the correction amount transmission unit 19, and supplies the correction amount to the aircraft control unit 12 and the projector control unit 25.

(Step S020)
Subsequently, in step S020, the flying object control unit 12 controls the motors 4a to 4d with reference to the correction amount to control the projection position so that the projection position approaches the target projection position. The projector control unit 25 controls the projector 3 by controlling the projector 3 with reference to the correction amount so as to bring the projection position close to the target projection position. The control of the aircraft 1 and the projector 3 is the same as in the first embodiment.

In the present embodiment, the camera 7 is provided in addition to the unmanned aerial vehicle 1. More specifically, the camera 7 is provided on the server 60 side. For this reason, it is also possible to provide one server 60 for each area, and to perform projection position correction of a plurality of unmanned aerial vehicles in the area by one server 60. Further, in the present embodiment, since the projection position extraction process and the correction amount calculation process are performed in the server 60, the control device 10 can be realized with a relatively simple configuration.

[Example of software implementation]
The control block of the control device 10 may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by software.

In the latter case, the control device 10 includes a computer that executes instructions of a program that is software that implements each function. The computer includes, for example, one or more processors, and a computer readable recording medium storing the program. Then, in the computer, the processor reads the program from the recording medium and executes the program to achieve the object of the present invention. For example, a CPU (Central Processing Unit) can be used as the processor. As the above-mentioned recording medium, a tape, a disk, a card, a semiconductor memory, a programmable logic circuit or the like can be used besides “a non-temporary tangible medium”, for example, a ROM (Read Only Memory). In addition, a RAM (Random Access Memory) or the like for developing the program may be further provided. The program may be supplied to the computer via any transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. Note that one aspect of the present invention can also be realized in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.

The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims, and embodiments obtained by appropriately combining the technical means disclosed in the different embodiments. Is also included in the technical scope of the present invention.

1 Unmanned aerial vehicle (mobile)
3 Projector (Projector)
6, 6a Projection controller 7 Camera (projector)
8 image 10 control device 11 captured image processing unit 12 flying object control unit 17 correction amount calculation unit 16 image acquisition unit 25 projector control unit

Claims

A control device for controlling a movable body provided with a projection unit, the control device comprising:
A captured image acquisition unit that acquires a captured image including a projection image projected by the projection unit;
A control apparatus comprising: a projection control unit configured to control at least one of a position of the movable body, an attitude of the movable body, and a direction of the projection unit with reference to the captured image.
The projection control unit extracts a projection position of the projection image and a target projection position from the captured image, and the mobile body projects so that the projection position projected by the mobile body becomes the target projection position. The control device according to claim 1, which controls a projection position.
The control device according to claim 2, wherein the projection control unit determines a correction amount of the projection position by calculating a difference between a projection position of the current projection image and a target projection position.
The projection control unit extracts, from the captured image, a first feature point indicating a current projection position of the projection image and a second feature point indicating a target projection position, and the first feature point and the second feature point The control device according to claim 3, wherein the correction amount is determined by calculating a difference in position between two feature points.
The control according to any one of claims 1 to 3, wherein a projection position projected by the moving body is controlled with reference to a plurality of captured images captured at different times by the same imaging device. apparatus.
The control device according to any one of claims 1 to 5, wherein a projection position projected by the movable body is controlled by controlling a position or an attitude of the movable body.
The control device according to any one of claims 1 to 6, wherein a projection position projected by the movable body is controlled by controlling a projection direction of the projection unit.
The control device according to any one of claims 1 to 7, wherein the captured image acquisition unit acquires, as the captured image, a captured image captured by an imaging device included in the moving body.
The control device according to any one of claims 1 to 7, wherein the captured image acquisition unit acquires, as the captured image, a captured image captured by an imaging device provided other than the moving body.
The control device according to any one of claims 1 to 6, characterized in that a flying object is controlled as the moving object.
The control device according to any one of claims 1 to 10, wherein the image projected by the projection unit includes information for assisting a worker's work.
An aircraft having a projection unit,
A captured image acquisition unit that acquires a captured image including a projection image projected by the projection unit;
A control device provided with a projection control unit for controlling at least one of the position of the flight object, the attitude of the flight object, and the direction of the projection unit with reference to the captured image Characterized flying body.
A control program for causing a computer to function as the control device according to any one of claims 1 to 11, wherein the control program causes the computer to function as the captured image acquisition unit and the projection control unit.