TWI693959B - Control device, flying body and recording medium - Google Patents

Abstract

The present invention appropriately projects the image toward the target position. A control device (10) is a control device (10) that controls a moving body (1) including a projection unit (3), and includes a captured image acquisition unit (16) to acquire a captured image (8); and The projection control unit (6) refers to the captured image (8) and controls at least one of the position of the moving body (1), the posture of the moving body (1), and the direction of the projection unit (3).

Description

Control device, flying body and recording medium

An aspect of the present invention relates to a control device, a flying body, and a recording medium.

In the past, there has been known a technique for projecting an image by a moving body equipped with a projection unit. Patent Document 1 discloses a technique for prompting information using a projector mounted on an unmanned aircraft.

[Prior Art Literature] [Patent Literature]

[Patent Document 1] Japanese Patent Publication Gazette "Japanese Patent Laid-Open No. 2017-76084 (published on April 20, 2017)"

However, in the prior art as described above, it is not easy to project an image toward the target position by the projection unit provided in the mobile body.

Therefore, an aspect of the present invention aims to appropriately project an image toward a target position.

In order to solve the above-mentioned problems, the control device of the first aspect of the present invention is a control device that controls a mobile body provided with a projection unit, and is characterized by comprising a captured image acquisition unit that acquires an image including a projection image projected by the projection unit An image; and a projection control unit that refers to the captured image and controls at least one of the position of the moving body, the posture of the moving body, and the direction of the projection unit.

According to the above configuration, a control device that appropriately projects an image toward a target position can be realized.

In addition, in the control device according to the second aspect of the present invention, the projection control unit may extract the projection position and the target projection position of the projected image from the captured image, and perform projection projection using the moving body The manner in which the position becomes the target projection position controls the projection position at which the mobile body performs projection.

According to the above configuration, it is possible to realize a control device that projects an image so that the projection position at which the moving object projects becomes the target projection position.

In addition, in the control device according to the third aspect of the present invention, the projection control unit may calculate the difference between the current projection position of the projected image and the target projection position, thereby determining the correction amount of the projection position.

According to the above configuration, it is possible to realize a control device that projects an image so that the projection position at which the moving object projects becomes the target projection position.

In addition, in the control device according to the fourth aspect of the present invention, the projection control unit may separately extract the first feature point indicating the current projection position of the projection image and the target projection position from the captured image The second feature point and calculate the difference between the position of the first feature point and the second feature point, thereby determining the correction amount.

According to the above configuration, it is possible to realize a control device that calculates the difference between the first feature point and the second feature point of the target projection position from the current projection position and calculates the correction amount, thereby projecting the image with higher accuracy.

In addition, the control device according to the fifth aspect of the present invention may refer to a plurality of captured images captured by the same imaging device at different times to control the projection position at which the moving body projects.

According to the above configuration, the correction amount of the projection position can be calculated based on the temporal change of the projected image.

In addition, the control device according to the sixth aspect of the present invention may control the position or posture of the moving body, thereby controlling the projection position at which the moving body projects.

According to the above configuration, the control device can control the projection position by controlling the position or posture of the moving body.

In addition, the control device according to the seventh aspect of the present invention may control the projection direction of the projection unit, thereby controlling the projection position at which the mobile body performs projection.

According to the above configuration, the control device can control the projection position by controlling the projection direction of the projection unit.

In addition, in the control device according to the eighth aspect of the present invention, the captured image acquisition unit may acquire the captured image captured by the imaging device included in the moving body as the captured image.

According to the above configuration, the control device can control the projection position using the captured image captured by the imaging device included in the moving body.

In addition, in the control device of the ninth aspect of the present invention, the captured image The acquisition unit may acquire a captured image captured by an imaging device provided other than the moving body as the captured image.

According to the above configuration, the control device can control the projection position using the captured image captured by the imaging device provided other than the moving body.

In addition, the control device according to the tenth aspect of the present invention may control the flying body as the moving body.

According to the above configuration, it is possible to realize a control device that projects an image toward a target position using a flying body as a moving body.

In addition, in the control device according to the eleventh aspect of the present invention, the image projected by the projection unit may include information to assist the operator's work.

According to the above configuration, it is possible to realize a control device which presents information to assist the operator at the correct position.

In addition, the flying body in the form 12 of the present invention is a flying body including a projection unit, and includes a control device including a captured image acquisition unit that acquires a captured image including a projection image projected by the projection unit; And the projection control unit refers to the captured image, and controls at least one of the position of the flying object, the posture of the flying object, and the direction of the projection unit.

According to the above configuration, it is possible to realize a flying object that controls at least one of the position of the flying object, the attitude of the flying object, and the direction of the projection unit, and projects an image toward the target position.

In addition, the recording medium of the thirteenth aspect of the present invention is a computer-readable recording medium, and its recording is useful to use a computer as the captured image acquisition unit and A control program for the projection control unit to function, the control program is a control program for causing a computer to function as a control device according to an aspect of the present invention.

According to the above-mentioned configuration, the same effect as in the first embodiment can be obtained.

According to one aspect of the present invention, a control device that appropriately projects an image toward a target position can be realized.

1: Unmanned aircraft (moving body)

2a~2d: rotor

3: Projector (projection section)

4a~4d: motor

5: projection system

6, 6a: Projection control department

7: Camera (shooting device)

8: Image

9: Correction amount receiving section

10: Control device

11, 11a, 11b: Captured image processing unit

12: Aircraft control section (projection position indication section)

13: Projection Information Storage Department

14: Projection Information Acquisition Department

16: Captured image acquisition section

17: Correction amount calculation unit

19: Correction amount sending unit

20: Current projection position

21: Target projection position

22a~22d: mark

23a~23d: characteristic points

24: Image

25: Projector control department

60: server

S004~S020: Steps

FIG. 1 is a diagram showing a schematic configuration of an unmanned aircraft according to Embodiment 1 of the present invention.

2 is a block diagram showing a schematic configuration of an unmanned aircraft according to Embodiment 1 of the present invention.

3 is a sequence diagram showing the flow of the projection position correction process according to Embodiment 1 of the present invention.

4 is a diagram showing an example 1 of projection position correction according to Embodiment 1 of the present invention.

5(a) and 5(b) are diagrams showing a second example of correction of the projection position according to Embodiment 1 of the present invention.

6 is a block diagram showing a schematic configuration of a projection system according to Embodiment 2 of the present invention.

7 is a sequence diagram showing the flow of a projection position correction process according to Embodiment 2 of the present invention.

Hereinafter, embodiments of the present invention will be described in detail based on the drawings. but Yes, the configuration described in the present embodiment is not intended to limit the scope of the present invention to this, and is only an illustrative example unless there is a specific description in particular. In addition, for the convenience of explanation, members having the same functions as those shown in the embodiments are denoted by the same symbols, and descriptions thereof are appropriately omitted.

[Embodiment 1]

(Composition of UAV 1)

FIG. 1 is a diagram showing an outline of an unmanned aircraft 1 (moving body, flying body) according to Embodiment 1 of the present invention. In addition, the moving body is not limited to the unmanned aircraft 1, and may be a robot such as a self-propelled type that is not a flying body, depending on the application.

When projecting using a flying body including the unmanned aircraft 1, it is usually difficult to project toward the target position. However, according to the control device 10 of this embodiment, even if a flying object is used, an image can be projected toward the target position.

As shown in FIG. 1, the drone 1 includes a rotor 2 a to a rotor 2 d, a camera 7 (imaging device), and a projector 3 (projection unit). In addition, although not shown in FIG. 1, the motors that drive the rotors 2 a to 2 d exist inside the unmanned aircraft 1 and control the flight of the unmanned aircraft 1.

Projector 3 projects image 8. In addition, when only called "image" below, it refers to the image which the projector 3 projects. The image 8 may also contain information to assist the operator's work. The information used to assist the operation, for example, in the present embodiment, means that the auxiliary operator can smoothly select the image of the object in the warehouse. With the projector 3 projecting the image 8 correctly to the designated position, the operator can accurately present the position of the object in the warehouse through the image 8.

In addition, the projector 3 may project information including not only instructions to the operator who is a human but also instructions to other devices, a robot for operation, or the like. In this case, it suffices to have a configuration in which the instructions to other devices or other operation robots are included as optical information in the information to be projected by the projector 3.

The information to be projected by the projector 3 is not limited as long as it is information using light, in other words, as long as it is information conveyed by light. In addition, the wavelength of light used is not particularly limited. For example, the light may be light outside the visible range, or infrared light, ultraviolet light, or the like. However, when the information projected by the projector 3 includes an instruction to the operator as a person, the light preferably includes a visible light region.

In the present embodiment, the camera 7 is an optical detection element, and performs imaging in the visible light region, for example. However, the wavelength range in which the camera 7 shoots is not particularly limited, and the camera 7 only needs to be able to detect the light used for the projection information of the projector 3, and can also detect the light outside the visible area.

(Composition of UAV 1)

The configuration of the drone 1 will be described with reference to FIG. 2. FIG. 2 is a block diagram showing the configuration of the drone 1. As shown in FIG. 2, the drone 1 includes a rotor 2 a to a rotor 2 d, a projector 3 (projection unit), a motor 4 a to a motor 4 d, a control device 10, a projection information acquisition unit 14, and a camera 7 (imaging device). The control device 10 includes a projection information storage unit 13, a captured image acquisition unit 16, and a projection control unit 6. The projection control unit 6 includes a captured image processing unit 11, a correction amount calculation unit 17, an aircraft control unit 12, and a projector control unit 25.

In the present embodiment, the unmanned aircraft 1 obtains projection information as information about the image 8 to be projected from an external device or the like through the projection information acquisition unit 14. As an example, the projection information also includes projection position specification information that specifies the projection position of the image 8, but it is not limited to this embodiment, and the projection information acquisition unit 14 may be configured to separately acquire projection position specification information from other devices, etc. . Here, the projection position specification information refers to information that specifies the projection position at which the projection image is projected. The control device 10 controls the position and projection direction of the drone 1 with reference to the projection position specification information.

In addition, the projection information may be information that changes with time in combination with the condition of the unmanned aircraft 1.

The projection information acquired by the projection information acquisition unit 14 is supplied to the projection information storage unit 13. The projection information storage unit 13 stores projection information.

The captured image processing unit 11 supplies the projection information acquired from the projection information storage unit 13 to the projector 3 (projection unit). Projector 3 projects image 8.

The camera 7 (imaging device) acquires an image including the projected image of the image 8 as a captured image. Hereinafter, when referred to as “captured image”, it refers to an image captured by the camera 7. Furthermore, depending on the shooting direction of the camera 7 and the projection direction of the projector 3, there may be a case where the projected image of the image projected by the projector 3 is not reflected in the "captured image" at all.

The camera 7 (imaging device) supplies the captured image to the captured image acquisition unit 16 of the control device 10. The captured image acquisition unit 16 supplies the captured image acquired from the camera 7 to the captured image processing unit 11 of the projection control unit 6. Where the image was taken The processing unit 11 refers to the projection information and extracts the current projection position and the target projection position of the image 8 from the captured image. Here, the target projection position refers to the projection position specified by the projection position specification information. The specific extraction process using the projection position of the captured image processing unit 11 will be described later. Furthermore, in the control as an example, after referring to the map in the warehouse and moving the drone 1 to a rough position, the target projection position is extracted from the captured image. In this control, the target projection position is not included in the captured image before moving to the approximate position, and the target projection position is included in the captured image after moving to the approximate position.

The captured image processing unit 11 supplies the current projection position and the target projection position extracted from the captured image to the correction amount calculation unit 17. The correction amount calculation unit 17 calculates the difference between the current projection position and the target projection position, and calculates the correction amount of the projection position. The specific calculation process of the correction amount will be described later. Furthermore, the "difference" between the current projection position and the target projection position refers to the magnitude and direction of the deviation used to move the current projection position toward the target projection position.

The correction amount calculation unit 17 supplies the calculated correction amount to the flying object control unit 12 and the projector control unit 25. The flying body control unit 12 refers to the correction amount and controls the motors 4 a to 4 d to control the driving of the rotors 2 a to 2 d to control the position or attitude of the unmanned aircraft 1. In addition, the projector control section 25 controls the projection direction of the projector 3, thereby controlling the projection position of the image 8. Furthermore, the control of the “posture” of the moving body refers to the direction of projection of the moving body toward the target, and also includes the direction of the projection of the target such as a self-propelled robot that is not a flying body.

In this way, the projection direction of the projector 3 can be controlled by not making the drone The position or posture of 1 is moved to control only the projection direction of the projector 3 itself. Conversely, the unmanned aircraft 1 can also be controlled by not changing the relative projection direction of the projector 3 relative to the unmanned aircraft 1 Position or posture. In addition, in the case of a configuration that does not change the relative projection direction of the projector 3 itself, the projector control unit 25 is not an essential configuration of the projection control unit 6.

The flying object control unit 12 and the projector control unit 25 determine the control method of the unmanned aircraft 1 and the projector 3 in consideration of the performance of the drone, the ease of maintaining the projection position and the projection posture, and the like.

With reference to FIG. 3, the flow of the projection position correction process using the drone 1 will be described. 3 is a flowchart showing the flow of projection position correction processing.

(Step S004)

First, in step S004, the projector 3 included in the drone 1 projects the image 8 based on the projection information supplied from the self-photographed image processing unit 11.

(Step S006)

Next, in step S006, the camera 7 obtains a projected image including the image 8 projected in step S004 as a captured image. The camera 7 supplies the captured image to the captured image acquisition unit 16.

(Step S008)

Then, in step S008, the captured image acquisition unit 16 acquires the captured image acquired by the camera 7. The captured image acquisition unit 16 supplies the acquired captured image to the captured image processing unit 11.

(Step S010)

Then, in step S010, the captured image processing unit 11 extracts the current projection position and the target projection position from the captured image. The captured image processing unit 11 supplies the current projection position and the target projection position to the correction amount calculation unit 17. The specific extraction process using the projection position of the captured image processing unit 11 will be described later.

(Step S012)

Then, in step S012, the correction amount calculation unit 17 calculates the difference between the current projection position and the target projection position.

(Step S014)

Then, in step S014, the correction amount calculation unit 17 determines whether the difference between the current projection position and the target projection position is within a fixed value.

If the difference between the current projection position and the target projection position is within a fixed value, the correction amount calculation unit 17 determines that projection to the target projection position has been performed, and the projection position correction process ends. If the difference between the current projection position and the target projection position is greater than the fixed value, then the process of step S016 is performed. In addition, the "fixed value" may be set according to the purpose of using the control device 10. As an example, a configuration in which the correction amount calculation unit 17 calculates the correction amount using one pixel in the captured image as a "fixed value" may be considered.

(Step S016)

Then, in step S016, the correction amount calculation unit 17 calculates the correction amount of the projection position so as to minimize the difference between the current projection position and the target projection position. Furthermore, the "minimization" of the difference is not necessarily limited to the fact that the current projection position exactly matches the target projection position. As long as the difference between the current projection position and the target projection position If the projection is performed within the "fixed value" described in the description of step S014, the projection control unit 6 determines that the projection has been performed with sufficient accuracy.

(Step S018)

Then, in step S018, the correction amount calculation unit 17 supplies the correction amount obtained in step S016 to the flying object control unit 12 and the projector control unit 25.

(Step S020)

Then, in step S020, the flying object control unit 12 refers to the correction amount supplied from the correction amount calculation unit 17 and controls the motors 4a to 4d, thereby controlling the projection position so that the projection position approaches the target projection position. The projector control unit 25 refers to the correction amount supplied from the correction amount calculation unit 17 and controls the projection direction of the projector 3, thereby controlling the projection position so that the projection position approaches the target projection position. The control of the unmanned aerial vehicle 1 and the projector 3 has been described above, so the description is omitted here.

Then, the process returns to step S004, and projection is performed again from the corrected projection position. Then, the processes from step S006 to step S014 are sequentially performed. In step S014, if the difference between the current projection position and the target projection position is within a fixed value, the projection control unit 6 determines that projection to the target projection position has been performed, and ends the projection position correction process. If the difference between the current projection position and the target projection position is larger than the fixed value, the projection control unit 6 sequentially performs each process from step S016 to step S020, and returns to step S004 again for processing.

The number of times to repeat the processing from step S004 to step S020 is not particularly limited, and it can be performed until the difference between the current projection position and the target projection position becomes within a fixed value. In addition, when the position deviation of the drone 1 after the position correction is repeated, it is repeated again From S004 to S020, the position can be corrected.

In the above description, the camera 7 acquires one captured image in step S004, and the correction amount calculation unit 17 calculates the correction amount by referring to one captured image. However, the camera 7 may be different from each other. Acquire multiple captured images at the same time. In this case, the correction amount calculation unit 17 refers to changing the projection position or the projection direction, and extracts the projection by the projection position of the projected image among the plurality of captured images captured at the same time by the same camera 7. The position changes over time, from which the correction amount can be calculated. More specifically, for example, the correction amount calculation unit 17 refers to a plurality of captured images obtained by changing the projection direction or the projection position and shooting, and the projection position of the projected image among the plurality of captured images is relative to the target projection position Analyze the positional relationship and its time change, so that the direction and size of the correction can be calculated. In this way, try to rotate or move the unmanned aircraft 1 and extract the direction and magnitude of the deviation of the projection position, thereby calculating the corrected direction and magnitude.

(Example 1 of projection position correction)

4 is a diagram for explaining Example 1 of correction of the projection position by the projection control unit 6. In this example, the case where the projector 3 projects a point as an image is taken as an example. The camera 7 acquires a captured image including the current projection position 20 at that point. The captured image processing unit 11 extracts the current projection position 20 from the captured image supplied by the camera 7 as the first feature point, and extracts the target projection position 21 as the second feature point. The symbols 22a to 22d used for projection are also Extract as feature points. Here, the symbol used for projection refers to one or more symbols referred to for specifying the value in the world coordinate system of the projection position 20. As an example, the symbol used for projection is placed on the target Near the projection location.

Here, in order to simplify the description, it is assumed that the symbols 22a to 22d used for projection are all present on the same plane and the coordinates have been identified in advance, but this does not limit the correction amount calculation processing in this embodiment. In addition, the plane is projected, and the current projection position 20 and the target projection position 21 are also located on the same plane as the symbols 22a to 22d used for projection, but it is not limited to the calculation of the correction amount in this embodiment. deal with.

The correction amount calculation unit 17 uses the coordinates of the symbols 22a to 22d for projection to specify the specific values of the actual coordinates (x, y, z) of the current projection position 20 of the point. Furthermore, the specific values of the actual coordinates (x', y', z') of the target projection position 21 are determined in advance by the projection control unit 6 by referring to the projection position specification information. After the correction amount calculation unit 17 specifies the specific value of (x, y, z), for example, the correction amount is calculated according to the method of calculation example 1 or calculation example 2 described later.

In addition, in this embodiment, in order to simplify the description, the correction of the projection position is assumed to be performed by the parallel movement or the rotary movement of the unmanned aircraft 1, but it is not limited to this embodiment.

Furthermore, the method of correcting the projection position is not limited to the above example. As another method, for example, the correction amount calculation unit 17 calculates the correction amount using a method such as directly measuring the correction amount based on image processing or a measurement result in a three-dimensional (three-dimensional) sensor.

(Calculation example 1: The method of calculating the parallel advance △T without considering the rotation motion)

One of the calculation examples of the correction amount calculation unit 17 for calculating the correction amount under the conditions described in FIG. 4 will be described.

In the following description, it is assumed that the camera coordinates and the projector coordinates have been calibrated, and the relationship between the two coordinates is known (how to change one coordinate to another coordinate). In addition, the camera coordinate system means that the position of the charge coupled device (CCD) or lens of the camera 7 is set as the origin, and the direction of the optical axis of the camera 7 is set as the Z axis. Set the coordinate system of X axis and Y axis with the left hand system. The projector coordinate system means that the position of the mirror or light source of the projector 3 is set as the origin, and the direction of the optical axis of the projector 3 is set as the Z axis, and the X axis and the Y axis are set by the right-handed system or the left-handed system Coordinate system. In addition, the world coordinate system refers to a coordinate provided in a space where the drone 1 moves, and in this embodiment, a three-dimensional coordinate system installed in a warehouse where the drone 1 moves.

In the following, the transformation matrix from the camera coordinate to the projector coordinate is expressed as R _pc and T _pc . Here, the transformation matrix R _pc expresses rotation, and the transformation matrix T _pc expresses progress. Similarly, the transformation matrix from the world coordinate system to the camera coordinate system is expressed as R _cw and T _cw . Here, the transformation matrix R _cw expresses rotation, and the transformation matrix T _cw expresses progress.

When using the expression method described above, the correction amount calculation unit 17 can convert the world coordinates (x _w ,y _w ,z _w ) to the projector coordinates (x _p ,y by using the following formula 1 _p , z _p ).

The correction amount calculation unit 17 uses the above formula 1 to convert the coordinates (x', y', z') of the target projection position in the world coordinate system and the coordinates (x, y, z) of the current projection position into a projector The coordinates of the target projection position in the coordinate system (x _g , y _g , z _g ), and the coordinates of the current projection position in the projector coordinate system (x _a , y _a , z _a ).

Then, the correction amount calculation unit 17 combines the coordinates of the target projection position in the projector coordinate system (x _g , y _g , z _g ) and the coordinates of the current projection position in the projector coordinate system (x _a , y _a , z _a ) is substituted into Equation 2 below to determine the transformation matrix R ^* and the transformation matrix T ^* . Here, the transformation matrix R ^* expresses rotation, and the transformation matrix T ^* expresses parallel.

Generally, there are a plurality of R ^* and T ^* satisfying the condition of Equation 2, but in this calculation example, the rotation amount of the unmanned aircraft 1 is not considered, and only the advancing amount ΔT is obtained. In other words, in this example, the correction amount calculation unit 17 sets R ^* to the identity matrix and calculates only the correction amount T ^* =ΔT. The correction amount ΔT calculated by the correction amount calculation unit 17 is supplied to the flying body control unit 12, and the flying body control unit 12 causes the unmanned aircraft to move in parallel only by the correction amount shown by the correction amount, whereby the projection position toward the target can be achieved Perform projection.

(Calculation example 2: The method of calculating the rotation amount R without considering the parallel motion)

Next, as a calculation example 2 of the correction amount, a method of obtaining only the rotation amount R without considering the parallel motion of the unmanned aircraft 1 will be described.

First, as in Calculation Example 1, the coordinates of the target projection position in the world coordinate system (x', y', z') and the coordinates of the current projection position (x, y, z) are converted into the projector coordinate system The coordinates of the target projection position in (x _g , y _g , z _g ), and the coordinates of the current projection position in the projector coordinate system (x _a , y _a , z _a ), and substitute them in the same way as in Calculation Example 1. To formula 2.

In this calculation example, T ^{* is} set to 0, and the parallel movement of the UAV 1 is not considered. The angle between the vector (x _g , y _g , z _g ) and the vector (x _a , y _a , z _a ) of the two projection positions in the projector coordinate system, and the axis of rotation of the unmanned aircraft 1 can be It is calculated using the following formula 3 and formula 4.

The correction amount calculation unit 17 applies the rotation axis and the rotation angle obtained using Equations 3 and 4 to the Rodrigues' Rotation Formula, thereby calculating the correction amount R ^* =ΔR.

The correction amount ΔR calculated by the correction amount calculation unit 17 is supplied to the flying body control unit 12, and the flying body control unit 12 causes the drone to rotate only by the correction amount indicated by the correction amount, whereby the projection position toward the target can be achieved Perform projection.

(Example 2 of projection position correction)

5(a) and 5(b) are diagrams for explaining Example 2 of projection position correction. FIG. 5(a) is an image 24 projected by the projector 3. FIG. 5(b) is an example of a captured image captured by the camera 7, and includes the current projection position 20 and the target projection position 21 of the point.

The camera 7 acquires the captured image of the current projection position 20 including the point. The captured image processing unit 11 extracts the feature point 23a to the feature point 23d (first feature point) in the current projection position 20 of the image 24 from the captured image supplied by the camera 7, and the symbol for projection ( Second characteristic point) 22a to symbol (second characteristic point) 22d.

The captured image processing unit 11 associates each of the feature points 23 a to 23 d with each of the symbols 22 a to 22 d. More specifically, the correction amount calculation unit 17 obtains a homography matrix based on the corresponding point group, and decomposes the obtained homography matrix using Zhang's Method or the like Form a rotation matrix R (=△R) and a parallel matrix T (=△T). With this, the correction amount calculation unit 17 can obtain the correction amount ΔR and the correction amount ΔT. In addition, the conversion from the camera coordinates to the projector coordinates can be performed using Equation 1 above.

[Embodiment 2]

Hereinafter, the second embodiment will be described in detail with reference to FIG. 6. In the following description, the members already explained in the above-mentioned embodiment are denoted by the same reference symbols, and the description is omitted, and points different from the above-mentioned embodiment will be described.

(Configuration of control system)

6 is a block diagram showing a schematic configuration of the projection system 5 of the second embodiment. In this embodiment, the projection system 5 includes the drone 1 and the server 60.

The drone 1 includes a rotor 2a to a rotor 2d, a projector 3, a motor 4a to a motor 4d, a control device 10, a projection information acquisition unit 14, and a correction amount receiving unit 9. The control device 10 includes a projection information storage unit 13, a captured image processing unit 11a, a flying object control unit 12, and a projector control unit 25. Furthermore, the projector 3 is not an essential configuration of the control device 10.

The server 60 includes a projection control unit 6a, a camera 7, a captured image acquisition unit 16, and a correction amount transmission unit 19. The projection control unit 6a includes a captured image processing unit 11b and a correction amount calculation unit 17.

In this embodiment, the following configuration is adopted: the camera 7 included in the server 60 takes an image projected by the projector 3 included in the drone 1, and the projection position described in Embodiment 1 is performed on the server 60 Extraction processing and correction amount calculation processing, the calculated correction amount is sent from the server 60 to the unmanned aircraft 1.

The flow of projection position correction processing using the projection system 5 of this embodiment will be described with reference to FIG. 7. 7 is a sequence diagram showing the flow of projection position correction processing. In addition, an example in which steps S004 to S020 are performed only once will be described below, but the present embodiment is not limited to this. As described in Embodiment 1, steps S004 to S020 may be repeated multiple times until the difference between the target projection position and the current projection position becomes within a fixed value.

(Step S004)

As in the first embodiment, the projector 3 included in the drone 1 projects the image 8.

(Step S006)

Then, the camera 7 included in the server 60 acquires an image including the projected image of the image 8 projected in step S004 as a captured image. The camera 7 supplies the captured image to the captured image acquisition unit 16.

(Step S008)

Then, in step S008, the captured image acquisition unit 16 included in the server 60 acquires the captured image acquired by the camera 7. The captured image acquisition unit 16 supplies the acquired captured image to the captured image processing unit 11.

(Step S010)

Then, in step S010, the captured image processing unit 11 extracts the current projection position and the target projection position from the captured image. The captured image processing unit 11 supplies the current projection position and the target projection position to the correction amount calculation unit 17. The specific extraction process using the projection position of the captured image processing unit 11 is the same as in the first embodiment.

(Step S012)

Then, in step S012, the correction amount calculation unit 17 calculates the difference between the current projection position and the target projection position.

(Step S016)

Then, in step S016, the correction amount calculation unit 17 calculates the correction amount of the projection position so as to minimize the difference between the current projection position and the target projection position. The specific content of step S016 is the same as in the first embodiment.

(Step S018)

Then, in step S018, the correction amount calculation unit 17 sends the correction amount obtained in step S016 to the correction amount transmission unit 19. The correction amount sending unit 19 will modify The positive amount is sent to the correction amount receiving unit 9 of the unmanned aircraft 1.

(Step S019)

The correction amount receiving unit 9 receives the correction amount from the correction amount transmitting unit 19 and supplies it to the flying object control unit 12 and the projector control unit 25.

(Step S020)

Then, in step S020, the flying object control unit 12 controls the motors 4a to 4d with reference to the correction amount, thereby controlling the projection position so that the projection position approaches the target projection position. The projector control unit 25 controls the projector 3 with reference to the correction amount, thereby controlling the flying body 1 and the projector 3 so that the projection position approaches the target projection position. The control of the flying body 1 and the projector 3 is the same as in the first embodiment.

In this embodiment, the camera 7 is installed outside the drone 1. More specifically, the camera 7 is provided on the server 60 side. Therefore, one server 60 may be provided in each area, and the configuration of the projection positions of a plurality of unmanned aircrafts in the area may be corrected by one server 60. In addition, in the present embodiment, since the projection position extraction process and the correction amount calculation process are performed in the server 60, the control device 10 can be realized with a relatively simple configuration.

[Realization example using software]

The control block of the control device 10 may be realized by a logic circuit (hardware) formed in an integrated circuit (Integrated Circuit (Integrated Circuit, IC) chip) or the like, or may be realized by software.

In the latter case, the control device 10 is equipped with a computer that executes commands as a program of software that realizes each function. This computer has, for example, more than one processing And a computer-readable recording medium that stores the program. Moreover, in the computer, the processor reads the program from the recording medium and executes it, thereby achieving the purpose of the invention. As the processor, for example, a central processing unit (Central Processing Unit, CPU) can be used. As the recording medium, in addition to "non-transitory tangible media" such as read-only memory (Read Only Memory, ROM), etc., magnetic tape, magnetic disk, memory card, semiconductor memory, programmable logic can be used Circuit etc. In addition, a random access memory (RAM) that expands the program may be further provided. In addition, the program can also be supplied to the computer via any transmission medium (communication network, broadcast wave, etc.) that can transmit the program. Furthermore, an aspect of the present invention can also be realized in the form of a data signal embedded in a carrier wave embodied in the program by electronic transmission.

The present invention is not limited to the above-mentioned embodiments, and various changes can be made within the scope shown in the application. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the present invention. Within the scope of technology.

1: Unmanned aircraft (moving body)

2a~2d: rotor

3: Projector (projection section)

4a~4d: motor

6: Projection control department

7: Camera (shooting device)

10: Control device

11: Shooting image processing section

12: Aircraft control section (projection position indication section)

13: Projection Information Storage Department

14: Projection Information Acquisition Department

16: Captured image acquisition section

17: Correction amount calculation unit

25: Projector control department

Claims (12)

A control device is a control device that controls a moving body provided with a projection unit, characterized by comprising: a captured image acquisition unit that acquires a captured image including a projection image projected by the projection unit; and a projection control unit, With reference to the captured image, at least one of the position of the moving body, the posture of the moving body, and the direction of the projection unit are controlled, and the projection control unit extracts the The projection position of the projected image and the target projection position, and the projection position at which the mobile body projects the projection is controlled so that the projection position at which the mobile body projects the projection becomes the target projection position, and the projection control unit calculates the current projection position The difference between the projection position of the projected image and the target projection position determines the correction amount of the projection position. The control device according to item 1 of the patent application scope, wherein the control device controls a projection position at which the moving body performs projection with reference to a plurality of captured images captured by the same shooting device at different times from each other . The control device according to item 1 of the patent application range, wherein the captured image acquisition unit acquires a captured image captured by a capturing device other than the moving body as the captured image. The control device according to item 1 of the patent application scope, wherein the projection control unit extracts, from the captured image, a first feature point indicating the current projection position of the projection image and a target feature indicating the target projection position The second feature point, and calculates the difference between the positions of the first feature point and the second feature point, thereby determining the correction amount. The control device according to any one of items 1 to 4 of the patent application scope, which controls the position or posture of the moving body, thereby controlling the projection position at which the moving body projects. The control device according to any one of items 1 to 4 of the patent application scope, which controls the projection direction of the projection section, thereby controlling the projection position at which the movable body projects. The control device according to any one of claims 1 to 4, wherein the captured image acquisition unit acquires a captured image captured by an imaging device included in the moving body as the captured image Like. The control device as described in item 1 or 2 of the patent application scope, which controls the flying body as the moving body. The control device according to any one of claims 1 to 4, wherein the image projected by the projection section includes information to assist the operator's work. A flying body comprising a projection unit, characterized by comprising a control device, the control device comprising: a captured image acquisition unit that acquires a captured image including a projection image projected by the projection unit; and The projection control unit refers to the captured image and controls at least one of the position of the flying object, the posture of the flying object, and the direction of the projection unit. The projection control unit selects from the captured image Extract the projection position of the projected image and the target projection position, and the projection position projected by the moving body becomes the The method of the target projection position controls the projection position at which the mobile body projects, and the projection control unit calculates the difference between the current projection position of the projected image and the target projection position, thereby determining the correction amount of the projection position. The flying body as described in item 10 of the patent application range, wherein the control device controls the projection position of the moving body for projection by referring to a plurality of captured images captured at different times by the same shooting device . A computer-readable recording medium that records a control program that enables a computer to function as the captured image acquisition unit and the projection control unit. The control program is used to make the computer as a patent application The control program in which the control device according to any one of items 1 to 4 functions.

Images