JP2017224094A - Control device, movable body, control method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem in which quick control of the attitude of a body part may be required on the basis of a change in the attitude of the body part when a displacement part is displaced.SOLUTION: A control device comprises: a first acquisition part that acquires first information indicating the position of a displacement destination of a displacement part that is displaceably held by a body part; a prediction part that predicts the attitude of the body part at a moment when the displacement part is displaced to the position of the displacement destination on the basis of the first information; and a control part that controls the attitude of the body part on the basis of the attitude of the body part predicted by the prediction part.SELECTED DRAWING: Figure 6

Description

  The present invention relates to a control device, a moving body, a control method, and a program.

Patent Document 1 discloses a camera system. The camera system includes an external unit and a main unit. When the camera is turned on, unit information is exchanged between the main unit and the external unit. The unit information consists of model number, unit type, manufacturer type, firmware version number, and information on the presence or absence of firmware on the main body side.
Patent Document 1 JP 2011-239301 A

  If the displacement part hold | maintained so that a displacement with respect to a main-body part displaces, the gravity center position of a main-body part and a displacement part may change. Thereby, the attitude | position of a main-body part may change. In some cases, it is required to quickly control the posture of the main body based on the change in the posture of the main body.

  In one aspect, the control device includes: a first acquisition unit that acquires first information indicating a position of a displacement destination of a displacement unit that is held displaceably with respect to the main body unit; and the displacement unit is based on the first information. A prediction unit that predicts the posture of the main body after being displaced to the position of the displacement destination, and a control unit that controls the posture of the main body based on the posture of the main body predicted by the prediction unit.

  The prediction unit may predict the posture of the main body after the displacement unit is displaced to the displacement destination position based on the mass of the displacement unit and the first information.

  The control unit controls the posture of the main body unit based on the posture of the main body unit predicted by the prediction unit so that a change in the posture of the main body unit caused by the displacement unit being displaced to the position of the displacement destination is suppressed. It's okay.

  The prediction unit predicts the position of the displacement unit at each of a plurality of timings within a period until the displacement unit is displaced from the position before displacement to the position at the displacement destination, and the posture of the main body unit for each of the predicted plurality of positions. The control unit may control the posture of the main body at each of a plurality of timings based on the posture of the main body predicted by the prediction unit with respect to the position of the displacement portion at each timing.

  Based on the target value of the posture of the main body, the detected value of the posture of the main body by the sensor for acquiring the posture of the main body, and the posture of the main body predicted by the prediction unit, the control unit You may control your attitude.

  The displacement part may include a movable lens.

  The first information may include information indicating the position of the displacement destination of the movable lens corresponding to the focal length of the lens system including the movable lens.

  The first information may include information indicating the position of the displacement destination of the movable lens corresponding to the focus position of the lens system including the movable lens.

  A storage unit that stores the position of the center of gravity of the lens device including the lens system in association with a plurality of positions of the movable lens is further provided, and the prediction unit stores the position in association with the displacement destination position of the movable lens indicated by the first information. The posture of the main body after the movable lens is displaced to the position of the displacement destination of the movable lens indicated by the first information may be predicted based on the center-of-gravity position stored in the unit and the mass of the lens device.

  The movable lens is included in the lens system of the interchangeable lens that can be attached to and detached from the main body. You may provide the 2nd acquisition part which acquires the 2nd information which shows the gravity center position of a lens, and the mass of an interchangeable lens.

  The displacement unit may be an imaging device.

  The imaging device may be provided with a variable orientation relative to the main body, and the displacement of the displacement unit may be a change in the orientation of the imaging device.

  The imaging device can be attached to and detached from the main body, and the control device includes a second acquisition unit that acquires second information indicating the center of gravity of the imaging device and the mass of the imaging device from the imaging device attached to the main body. You may be prepared.

  In one mode, a mobile object is provided with the above-mentioned control device and a main part.

  In one aspect, the control method includes obtaining a first information indicating a position of a displacement destination of a displacement portion that is held displaceably with respect to the main body, and the displacement portion is a displacement destination based on the first information. A step of predicting a posture of the main body after being displaced to a position, and a step of controlling the posture of the main body based on the posture of the main body predicted in the step of predicting.

  In one aspect, a program causes a computer to execute the control method.

  The above summary of the present invention does not enumerate all of the features of the present invention. A sub-combination of these feature groups can also be an invention.

It is a figure which shows an example of a structure of a system. An example of the functional block of UAV100 is shown. An example when the lens apparatus 160 is in the tele end state is shown. An example of lens information stored in the memory 163 is shown. An example of gravity center information is shown. An example of the block diagram regarding the attitude | position control of UAV100 in UAV100 is shown. It is a flowchart which shows an example of the control flow which controls the attitude | position of UAV100. It is a flowchart which shows an example of the control flow after the UAV control part 104 receives 1st information until it outputs 2nd information to PID control. An example when the direction of the imaging device 190 is changed is shown. An example of imaging unit information stored in the memory 146 is shown.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Not all combinations of features described in the embodiments are essential for the solution of the invention.

  The claims, the description, the drawings, and the abstract include matters subject to copyright protection. The copyright owner will not object to any number of copies of these documents as they appear in the JPO file or record. However, in other cases, all copyrights are reserved.

  FIG. 1 is a diagram illustrating an example of a system configuration. The system comprises an unmanned aerial vehicle (UAV) 100. The UAV 100 includes a UAV main body 101, a gimbal 110, and an imaging device 190. The UAV main body 101 has a plurality of rotor blades including a rotor blade 108a and a rotor blade 108b. A plurality of rotor blades included in the UAV main body 101 may be collectively referred to as rotor blades 108. The imaging device 190 includes an imaging unit 140 and a lens device 160. The UAV 100 is an example of a moving body that moves with an object. The moving body is a concept including, in addition to UAV, other aircraft that moves in the air, vehicles that move on the ground, ships that move on the water, and the like.

  The UAV 100 flies by controlling the rotation of the rotor blade 108. The UAV 100 flies using, for example, four rotary wings 108. The number of rotor blades 108 is not limited to four. The UAV 100 may be a fixed wing aircraft that does not have the rotor blades 108.

  The gimbal 110 supports the imaging unit 140 and the lens device 160 so that the posture of the imaging device 190 with respect to the gimbal 110 can be changed. The gimbal 110 may support the imaging device 190 rotatably about at least one axis. For example, the gimbal 110 supports the imaging device 190 so as to be rotatable about the pitch axis, the roll axis, and the yaw axis. The gimbal 110 may hold the imaging unit 140 or the lens device 160.

  The imaging unit 140 generates and records image data of an optical image formed through the lens device 160. The lens device 160 is a so-called interchangeable lens. The lens device 160 can be attached to and detached from the imaging unit 140.

  FIG. 2 shows an example of functional blocks of the UAV 100. The UAV 100 includes a UAV main body 101, a gimbal 110, and an imaging device 190. The UAV main body 101 includes a communication interface 102, a UAV control unit 104, a sensor 120, a memory 106, a driver 107, a detection unit 105, a motor 109, and a rotating blade 108.

  The driver 107 and the motor 109 function as a drive unit that moves the UAV 100. The driver 107 drives the motor 109. The motor 109 drives the rotary blade 108. One motor 109 is provided for each of the plurality of rotor blades 108. The driver 107 controls the rotational speed of each drive shaft of the motor 109. The rotation speed is, for example, the number of rotations per unit time. The rotary blades 108 are rotated by the rotation of the drive shaft of the corresponding motor 109. The UAV 100 flies by rotating the rotary wing 108.

  The communication interface 102 communicates with an external transmitter. The communication interface 102 receives various commands from a remote transmitter. The UAV control unit 104 controls the flight of the UAV 100 according to the command received from the transmitter. The UAV control unit 104 controls the gimbal 110, the imaging unit 140, and the lens device 160.

  The UAV control unit 104 may be configured by a microprocessor such as a CPU or MPU, a microcontroller such as an MCU, or the like. The memory 106 stores programs necessary for the UAV control unit 104 to control the gimbal 110, the imaging unit 140, and the lens device 160. The memory 106 may be a computer-readable recording medium and may include at least one of flash memory such as SRAM, DRAM, EPROM, EEPROM, and USB memory. The memory 106 may be provided in the housing of the UAV 100. The memory 106 may be provided so as to be removable from the housing of the UAV 100. The UAV control unit 104 may cause the UAV 100 to function as a control device. In this case, a program for adjusting the attitude of the UAV 100 is read from the memory 106 and executed. The UAV control unit 104 may cause the UAV 100 to function as a control device.

  The detection unit 105 detects the situation of the UAV 100 or the situation around the UAV 100. The detecting unit 105 detects, for example, a position that includes the latitude, longitude, and altitude of the UAV 100 and a direction corresponding to the nose direction of the UAV 100. The detection unit 105 may include an imaging device for sensing that images the surroundings of the UAV 100.

  The UAV control unit 104 includes a confirmation unit 188. The confirmation unit 188 executes initialization before the flight of the UAV 100. The confirmation unit 188 performs operation confirmation of the driver 107, the motor 109, the rotary blade 108, and the detection unit 105. The confirmation unit 188 may perform operation confirmation of the driver 107, the motor 109, the rotor 108, and the detection unit 105 before the flight of the UAV 100. The confirmation unit 188 may perform operation confirmation of various sensors that detect the state of the UAV 100. The period in which the initialization is executed is an example of a period from when the power is turned on to the UAV 100 until the UAV 100 starts moving.

  The sensor 120 is a sensor for acquiring the attitude of the UAV main body 101. The sensor 120 may detect the attitude of the gimbal 110. The sensor 120 may be a gyro sensor. The detected value of the posture by the sensor 120 is output to the UAV control unit 104. The UAV control unit 104 controls the UAV main body 101 and the gimbal 110 based on the detected attitude value by the sensor 120.

  The gimbal 110 supports the imaging device 190 rotatably about at least one axis. The gimbal 110 may support the imaging device 190 rotatably about the yaw axis, the pitch axis, and the roll axis. The UAV control unit 104 outputs an operation command for the gimbal 110. The operation command includes a yaw angle target value, a pitch angle target value, and a roll angle target value. The yaw angle is a rotation angle of the imaging device 190 around the yaw axis. The pitch angle is a rotation angle of the imaging device 190 around the pitch axis. The roll angle is a rotation angle of the imaging device 190 around the roll axis. The gimbal 110 rotates the imaging device 190 around at least one of the yaw axis, the pitch axis, and the roll axis in accordance with the operation command acquired from the UAV control unit 104. The gimbal 110 controls the yaw angle, pitch angle, and roll angle to their respective target values. Thereby, the orientation of the imaging device 190 is controlled.

  The imaging unit 140 includes an imaging control unit 142, an imaging element 144, and a memory 146. The imaging control unit 142 may be configured by a microprocessor such as a CPU or MPU, a microcontroller such as an MCU, or the like. The imaging control unit 142 controls the imaging unit 140 and the lens device 160 in accordance with operation commands for the imaging unit 140 and the lens device 160 from the UAV control unit 104. The memory 146 may be a computer-readable recording medium, and may include at least one of flash memory such as SRAM, DRAM, EPROM, EEPROM, and USB memory. The memory 146 may be provided inside the housing of the imaging unit 140. It may be provided so as to be removable from the housing of the imaging unit 140.

  The imaging element 144 is held inside the housing of the imaging unit 140, generates image data of an optical image formed through the lens device 160, and outputs the image data to the imaging control unit 142. The imaging control unit 142 stores the image data output from the imaging element 144 in the memory 146. The imaging control unit 142 may output and store the image data in the memory 106 via the UAV control unit 104.

  The lens device 160 includes a lens control unit 162, a memory 163, a lens 164, a lens 166, and a lens 168. The lens 164, the lens 166, and the lens 168 form a lens system. The lens 164, the lens 166, and the lens 168 are disposed inside the lens barrel of the lens device 160. Part or all of the lens 164, the lens 166, and the lens 168 may be held to be displaceable along the optical axis. A lens that can be displaced along the optical axis is an example of a movable lens. The lens control unit 162 moves at least one of the lens 164, the lens 166, and the lens 168 along the optical axis in accordance with a command from the imaging control unit 142. An image formed by the lens included in the lens device 160 is captured by the imaging unit 140.

  In the present embodiment, an example in which the UAV 100 includes the UAV control unit 104, the imaging control unit 142, and the lens control unit 162 will be described. However, any one control unit may execute processing executed by two or three of the UAV control unit 104, the imaging control unit 142, and the lens control unit 162. The processing executed by the UAV control unit 104, the imaging control unit 142, and the lens control unit 162 may be executed by one control unit.

  In the UAV control unit 104, the first acquisition unit 170 acquires first information indicating the position of the displacement part of the displacement unit that is held displaceably with respect to the UAV main body 101. The movable lens included in the lens device 160 is an example of a displacement unit. The imaging device 190 is an example of a displacement unit. The UAV main body 101 is an example of a main body.

  The prediction unit 130 predicts the posture of the UAV main body 101 after the displacement unit is displaced to the displacement destination position based on the first information. The control unit 180 controls the posture of the UAV main body 101 based on the posture of the UAV main body 101 predicted by the prediction unit 130.

  The prediction unit 130 predicts the posture of the UAV main body 101 after the displacement unit is displaced to the displacement destination position based on the first information and the mass of the displacement unit. Based on the posture of the UAV main body 101 predicted by the prediction unit 130, the control unit 180 suppresses the change in the posture of the UAV main body 101 caused by the displacement unit being displaced to the displacement destination position. To control the attitude.

  The prediction unit 130 predicts the position of the displacement unit at each of a plurality of timings within a period until the displacement unit is displaced from the position before displacement to the position before displacement. The prediction unit 130 predicts the posture of the UAV main body 101 for each of the predicted plurality of positions. The control unit 180 controls the posture of the UAV main body 101 based on the posture of the UAV main body 101 predicted by the prediction unit 130 for the position of the displacement portion at each timing at each of a plurality of timings.

  The control unit 180 controls the posture of the UAV main body 101 based on the target value of the posture of the UAV main body 101, the detected value of the posture of the UAV main body 101 by the sensor, and the posture of the UAV main body 101 predicted by the prediction unit 130. To do. The control unit 180 performs feedforward control based on the attitude of the UAV main body 101 predicted by the prediction unit 130.

  When the displacement unit includes a movable lens, the first information may include information indicating the position of the displacement destination of the movable lens corresponding to the focal length of the lens system including the movable lens. The first information may be information indicating a change destination of the focal length. The first information may include information indicating the position of the displacement destination of the movable lens corresponding to the focus position of the lens system including the movable lens. The first information may be information indicating a change destination of the focus position.

  The memory 106 stores the gravity center position of the lens device including the lens system in association with the plurality of positions of the movable lens. The prediction unit 130 determines the displacement of the movable lens indicated by the first information based on the position of the center of gravity stored in the memory 106 in association with the position of the displacement destination of the movable lens indicated by the first information and the mass of the lens device. The posture of the UAV main body 101 after the movable lens is displaced to the previous position is predicted.

  When the movable lens is included in the lens system of the interchangeable lens that can be attached to and detached from the UAV main body 101, the second acquisition unit 172 determines the focal length and the focus position of the interchangeable lens from the interchangeable lens attached to the UAV main body 101. You may acquire the 2nd information which shows the gravity center position of the interchangeable lens corresponding to at least one, and the mass of an interchangeable lens. The second acquisition unit 172 may acquire the second information during a period in which initialization before the flight of the UAV 100 is being executed.

  When the imaging device 190 is variably provided with respect to the UAV main body 101, the imaging device 190 is an example of a displacement unit. The displacement of the displacement portion corresponds to a change in the orientation of the imaging device.

  The imaging device 190 may be detachable from the UAV main body 101. In this case, the second acquisition unit 172 may acquire second information indicating the position of the center of gravity of the imaging device 190 and the mass of the imaging device from the imaging device 190 mounted on the UAV main body 101. The second acquisition unit 172 may acquire the second information during a period in which initialization before the flight of the UAV 100 is being executed.

  FIG. 3 shows an example when the lens device 160 is in the telephoto end state. The state of the lens device 160 shown in FIG. 3 is different from the state of the lens device 160 shown in FIG. For example, FIG. 1 shows a state in which the lens device 160 is at the wide end. Except for this point, the state of the UAV 100 shown in FIG. 3 is the same as the state shown in FIG.

  The focal length of the lens device 160 is adjusted by displacing the movable lens included in the lens device 160. The focus position of the lens device 160 is adjusted by displacing the movable lens included in the lens device 160. When the movable lens is displaced, the position of the center of gravity of the lens device 160 changes. The position of the center of gravity of the lens device 160 changes depending on the displacement of the movable lens barrel and the lens frame that hold the movable lens, in addition to the displacement of the movable lens. When the center of gravity of the lens device 160 changes, the center of gravity of the UAV 100 changes.

  G1 indicates the position of the center of gravity of the UAV 100 when the UAV 100 is in the state shown in FIG. G2 indicates the position of the center of gravity of the UAV 100 when the UAV 100 is in the state of FIG. When the lens apparatus 160 is changed from the state shown in FIG. 1 to the tele end state, the center of gravity position of the UAV 100 changes from the position of G1 to the position of G2 due to the change of the center of gravity position of the lens apparatus 160. Therefore, even when the lens device 160 is in the tele end state, in order to prevent the posture of the UAV 100 from changing, for example, it is necessary to decrease the rotational speed of the rotary blade 108a and increase the rotational speed of the rotary blade 108b. There is.

  The control unit 180 controls the attitude of the UAV 100 by feedback control using the detected value of the attitude of the UAV 100 by the sensor 120. For example, the control unit 180 controls the attitude of the UAV 100 by PID control. Thereby, for example, when a change in the posture of the UAV 100 occurs, the rotational speed of the rotary blade 108 is increased or decreased so as to cancel the change in the posture of the UAV 100.

  The control unit 180 controls the attitude of the UAV 100 by feedforward control based on the displacement of the movable lens of the lens device 160 in addition to the feedback control. Specifically, the prediction unit 130 predicts a change in the posture of the UAV 100 using the amount of change in the center of gravity of the lens device 160 caused by the displacement of the movable lens and the mass of the lens device 160. The control unit 180 adds the feedforward compensation value corresponding to the predicted change in the posture of the UAV 100 to the input of the PID control in synchronization with the displacement of the movable lens of the lens device 160. As a result, the amount of change in the center of gravity of the lens device 160 can be calculated in advance, and the rotational speed of the rotary blade 108 can be quickly controlled. Therefore, as compared with the case where only the feedback control is used without using the feedforward control, the swing of the UAV 100 caused by the change in the center of gravity position of the lens device 160 can be suppressed.

  FIG. 4 shows an example of lens information stored in the memory 163. The lens information includes a lens type ID, mass information, and centroid information. The lens type ID is, for example, a model number of the lens. The mass information is information indicating the mass of the lens device 160. The barycentric information is information indicating the barycentric position of the lens device 160.

  FIG. 5 shows an example of centroid information. The centroid information includes a lens position and a centroid position corresponding to the lens position. When the lens device 160 has one movable lens, the lens position indicates the position of the one movable lens. The barycentric position indicates the barycentric position of the lens device 160 when there is a movable lens at the corresponding lens position. When the lens device 160 has two or more movable lenses, the lens position indicates the position of each of the two or more movable lenses. The barycentric position indicates the barycentric position of the lens device 160 when each movable lens is located at the position of each movable lens indicated by the corresponding lens position. As described above, the center-of-gravity information associates the center-of-gravity position of the lens device 160 with each of a plurality of lens positions.

  The barycentric position may be coordinates of the barycentric position of the lens device 160 when a predetermined reference position is the origin. The reference position may be, for example, the position of the imaging element 144 of the imaging unit 140. The reference position may be provided on the optical axis of the lens device 160. The barycentric position may be a distance from the image sensor 144 measured along the optical axis. When the lens device 160 is an interchangeable lens, the reference position may be provided within the mount surface of the imaging unit 140 that mounts the lens device 160.

  In the UAV control unit 104, the second acquisition unit 172 acquires the lens information stored in the memory 163 through the lens control unit 162 and the imaging control unit 142. For example, the second acquisition unit 172 stores the acquired lens information in the memory 106. The lens information stored in the memory 106 is used for processing by the prediction unit 130 and the control unit 180.

  FIG. 6 shows an example of a block diagram regarding the attitude control of the UAV 100 in the UAV 100. The block diagram shows the add-on point 200, transfer element 210, transfer element 220, transfer element 230, add-on point 202, transfer element 240, operator 250, transfer element 260, transfer element 270, transfer element 280, and add-on point. 204.

  In the block diagram of FIG. 6, the control target is the UAV 100 represented by the transfer element 230. The control amount is the angular velocity of the posture angle of the UAV 100 or the posture angle of the UAV 100. The angular velocity of the posture angle of the UAV 100 is detected by the sensor 120 represented by the transfer element 240. The state variable of the detection value by the sensor 120 is converted into the state variable of the posture angle by the operator 250 that is an integral element.

  R represents a state variable of the target value of the posture angle of the UAV 100. In addition point 200, the state variable of the target value is added. In addition point 200, the state variable of the output of operator 250 is subtracted. The transfer element 210 represents PID control of the control unit 180. Specifically, the transfer element 210 represents an operation in which the control unit 180 generates a drive signal for the driver 107 based on the error signal. The transfer element 220 is a transfer element that represents an operation in which the driver 107 generates a drive current of the motor 109 based on the drive signal. The transmission element 230 represents a phenomenon in which the angular velocity of the posture angle of the UAV 100 changes as the motor 109 is driven by the drive current and the rotary blade 108 rotates. The addition point 202 adds the state variable of the angular velocity of the posture angle generated by the action other than the rotary blade 108 to the state variable of the angular velocity of the posture angle of the UAV 100. The transmission element 210, the transmission element 220, the transmission element 230, the addition point 202, the transmission element 240, the operator 250, and the addition point 200 form a feedback control system.

  The transmission element 260 represents an operation in which the lens control unit 162 outputs first information for displacing the movable lens. When the movable lens is displaced according to the first information, the position of the center of gravity of the lens device 160 changes. The angular velocity of the posture angle of the UAV 100 changes due to the change in the position of the center of gravity of the lens device 160. The transfer element 270 represents a phenomenon in which the position of the center of gravity of the lens device 160 changes due to the output of the first information, and the angular velocity of the posture angle of the UAV 100 changes. In addition point 204, the state variable of the change amount of the angular velocity of the posture angle is added. The addition point 204 performs addition of a state variable indicating the amount of change in the angular velocity of the posture angle due to disturbance. Examples of the disturbance include a change in the posture angle of the UAV 100 due to wind or the like. The state variable output from the addition point 204 is added to the state variable of the angular velocity of the posture angle of the UAV 100 output from the transfer element 230 at the addition point 202.

  The transfer element 280 represents an operation in which the prediction unit 130 generates a predicted value of the change amount of the posture angle of the UAV 100 based on the first information. In addition point 200, the state variable which shows the predicted value which prediction part 130 generated is added. Thus, the state variable of the predicted value generated by the transfer element 280 is added to the input of the control loop as a feedforward compensation value.

  FIG. 7 is a flowchart illustrating an example of a control flow for controlling the attitude of the UAV 100. In S310, the lens control unit 162 outputs the first information to the movable lens. The first information is information indicating that the movable lens of the lens device 160 is displaced.

  For example, when the lens control unit 162 obtains a command to change the focal length of the lens device 160 from the imaging control unit 142, a future position that is a displacement destination position of one or more movable lenses included in the lens device 160, and a displacement timing. And decide. The future position of the movable lens is determined by the focal length. The displacement timing may be a time when the movable lens starts to be displaced. For example, the lens control unit 162 determines to start displacement of the movable lens after 30 ms. The displacement timing may be a time when the movable lens ends the displacement. The first information includes information indicating the future position of the movable lens and the displacement timing. The first information is also output to the UAV control unit 104 through the imaging control unit 142.

  In S320, the movable lens is displaced based on the first information. In S330, the control unit 180 outputs the second information to the PID control at the timing when the movable lens is displaced. The second information is information indicating the feedforward compensation value. An example of the process in S330 will be described with reference to FIG.

  In S340, the sensor 120 detects the angular velocity of the gimbal 110, thereby detecting the angular velocity of the posture angle of the UAV 100. In S350, the driver 107 controls the rotation speed of the rotary blade 108. For example, an error signal between the added value of the target value of the UAV 100 posture angle and the feedforward compensation value and the detected value of the posture angle of the UAV 100 by the sensor 120 is input to the PID control of the control unit 180. In the PID control of the control unit 180, a drive signal for the driver 107 is generated based on the error signal. When the driver 107 drives the motor 109 according to the drive signal, the rotational speed of the rotor blade 108 is controlled.

  FIG. 8 is a flowchart illustrating an example of a control flow from when the UAV control unit 104 receives the first information until it outputs the second information to the PID control. In S400, the first acquisition unit 170 receives the first information. In S410, the first acquisition unit 170 acquires the current position, future position, displacement speed, displacement start time, and attitude angle of the UAV 100 of the movable lens of the lens device 160. The first acquisition unit 170 acquires the current position and displacement speed of the movable lens from the lens control unit 162.

  In S410, the prediction unit 130 calculates the stop time of the movable lens. The stop time is the time when the displacement of the movable lens ends. The prediction unit 130 calculates the stop time of the movable lens based on the current position, future position, and displacement speed of the movable lens of the lens device 160.

  In S430, the prediction unit 130 determines whether the current time matches the stop time of the movable lens or whether the current time has passed the stop time of the movable lens. If the current time coincides with the stop time of the movable lens or if the current time has passed the stop time of the movable lens, the control flow ends. If the current time is before the stop time of the movable lens, the process proceeds to S440.

  In S440, the prediction unit 130 acquires the frame time of the main control loop. The main control loop is, for example, a feedback loop shown in FIG. The frame time is a control period in the feedback loop.

  In S450, the prediction unit 130 generates a predicted value of the change amount of the posture angle of the UAV 100 in each frame between the displacement start time and the stop time of the movable lens. For example, when the period from the displacement start time to the displacement end time is 10 ms and the frame time is 1 ms, the prediction unit 130 generates a predicted value of the change amount of the posture angle of the UAV 100 in each frame of 10 frames. . The predicted value of the change amount of the posture angle of the UAV 100 in each frame is an example of a feedforward compensation value in each frame.

  The prediction unit 130 predicts the change amount of the posture angle in each frame based on the change amount of the center of gravity position from the current center position of the lens device 160 and the mass of the lens device 160. The prediction unit 130 identifies the mass of the lens device 160 from the lens information shown in FIG. The prediction unit 130 identifies the amount of change in the center of gravity position of the lens device 160 using the center of gravity information shown in FIG. For example, the prediction unit 130 specifies the current center-of-gravity position of the lens device 160 based on the current position of the movable lens. The prediction unit 130 specifies the position of the center of gravity of the lens device 160 in each frame based on the position of the movable lens in each frame. The prediction unit 130 identifies the amount of change in the center of gravity position in each frame based on the position of the center of gravity of the lens device 160 in each frame and the current center of gravity position of the lens device 160.

  In S460, control unit 180 determines whether or not the current time is before the start time of displacement of the movable lens. If the current time is earlier than the displacement start time of the movable lens, the determination in S460 is repeated until the current time matches the displacement start time of the movable lens or the current time exceeds the displacement start time of the movable lens. If the current time matches the displacement start time of the movable lens or if the current time exceeds the displacement start time of the movable lens, the process proceeds to S470.

  In S470, the control unit 180 starts outputting the predicted value of the change amount of the posture angle in each frame predicted by the prediction unit 130 in S450. For example, the control unit 180 outputs the predicted value of the change amount of the posture angle in each frame predicted by the prediction unit 130 to the PID control in the corresponding frame. The frame that starts outputting the predicted value of the change amount of the posture angle may not be a frame corresponding to the displacement start time. The frame from which the output of the predicted value of the change amount of the posture angle may be a frame after the frame corresponding to the displacement start time. If the correspondence relationship with the frame in the feedback loop is maintained, the output of the predicted value of the change amount of the posture angle may be started from the frame after the frame corresponding to the displacement start time. When the control unit 180 outputs the predicted value of the change amount of the posture angle in the frame corresponding to the displacement stop time, the control unit 180 ends the control flow.

  FIG. 9 shows an example of a state in which the orientation of the imaging device 190 has changed. The state of the imaging device 190 illustrated in FIG. 9 is different from the state of the imaging device 190 illustrated in FIG. For example, FIG. 9 shows a state in which the pitch angle of the imaging device 190 has changed. Except for this point, the state of the UAV 100 shown in FIG. 9 is the same as the state shown in FIG.

  G3 indicates the position of the center of gravity of the UAV 100 when the UAV 100 is in the state shown in FIG. When the pitch angle of the imaging device 190 is changed from the state shown in FIG. 1 to the state shown in FIG. 9, the center of gravity of the UAV 100 changes from the position G1 to the position G3. Therefore, in order to prevent the posture of the UAV 100 from changing, for example, it is necessary to increase the rotational speed of the rotary blade 108a and decrease the rotational speed of the rotary blade 108b.

  The control unit 180 controls the posture of the UAV 100 by feedforward control based on a change in the orientation of the imaging device 190, similarly to the case where the position of the center of gravity of the lens device 160 changes. Specifically, the prediction unit 130 uses the amount of change in the center of gravity of the imaging unit 140 and the lens device 160 caused by the change in the orientation of the imaging unit 140, the mass of the imaging unit 140, and the mass of the lens device 160. , Predict changes in the posture of the UAV 100. As described above, the prediction unit 130 predicts the amount of change in the posture of the UAV 100 using the amount of change in the center of gravity of the image pickup device 190 caused by the change in the orientation of the image pickup device 190 and the mass of the image pickup device 190. The predicted value of the change amount of the posture of the UAV 100 is generated as a feedforward compensation value.

  The control unit 180 adds the generated feedforward compensation value to the input of PID control in synchronization with the change in the orientation of the imaging device 190. As a result, the amount of change in the center of gravity of the imaging device 190 can be calculated in advance, and the rotational speed of the rotary blade 108 can be quickly controlled. Therefore, it is possible to suppress the swing of the UAV 100 as compared with the case where only the feedback control is used without using the feedforward control.

  With reference to FIG. 9, the change in the gravity center position of the imaging device 190 caused by the change in the pitch angle has been described. The position of the center of gravity of the imaging device 190 changes not only due to the change in pitch angle but also due to the change in yaw angle. Therefore, the position of the center of gravity of the imaging device 190 changes depending on the combination of the pitch angle and the yaw angle. In the present embodiment, it is assumed that there is no change depending on the change of the roll angle. When the gravity center position of the imaging device 190 is on the roll axis, the gravity center position of the imaging device 190 does not change depending on the change of the roll angle.

  FIG. 10 shows an example of imaging unit information stored in the memory 146. The imaging unit information includes an imaging unit type ID, mass information, and barycentric position information. The imaging unit type ID is, for example, the model number of the imaging unit 140. The mass information is information indicating the mass of the imaging unit 140. The barycentric position information is information indicating the barycentric position of the imaging unit 140.

  The barycentric position may be coordinates expressed with a predetermined reference position as an origin. The reference position may be a grip position where the gimbal 110 grips the imaging unit 140, for example. The reference position may be provided on the pitch axis.

  In the UAV control unit 104, the second acquisition unit 172 acquires the imaging unit information stored in the memory 146 through the imaging control unit 142. For example, the second acquisition unit 172 stores the acquired imaging unit information in the memory 106. The imaging unit information stored in the memory 106 is used for processing by the prediction unit 130 and the control unit 180.

  Based on the lens information shown in FIGS. 4 and 5 and the imaging unit information shown in FIG. 10, the prediction unit 130 and the control unit 180 are imaging devices that combine the position, pitch angle, and yaw angle of the movable lens of the lens device 160. The center of gravity position of 190 can be calculated. The prediction unit 130 and the control unit 180 can calculate the mass of the imaging device 190 based on the lens information illustrated in FIG. 4 and the imaging unit information illustrated in FIG. The prediction unit 130 and the control unit 180 can calculate a change in the center of gravity position when at least one of the position of the movable lens of the lens device 160 and the orientation of the imaging device 190 is changed. Thereby, the prediction unit 130 and the control unit 180 can calculate a change in the center of gravity predicted when at least one of the position of the movable lens of the lens device 160 and the orientation of the imaging device 190 is changed. The prediction unit 130 and the control unit 180 can predict a change in posture angle of the UAV 100 or a change in angular velocity of the posture angle based on the calculated change in the center of gravity of the imaging device 190 and the mass of the imaging device 190.

  The control system corresponding to the change in the orientation of the imaging device 190 can be realized by adding a state variable and a transfer element related to the change in the orientation of the imaging device 190 to the control system shown in FIG.

  The control flow corresponding to the change in the orientation of the imaging device 190 can be realized by adding control related to the change in the orientation of the imaging device 190 in addition to the control relating to the displacement of the lens in the control flow shown in FIGS. For example, in the control flow shown in FIG. 7, information indicating the displacement of the movable lens and the direction of the imaging device 190 may be applied as the first information. In the control flow shown in FIG. 8, in addition to the position and displacement speed of the movable lens, control relating to the direction and direction of the imaging apparatus 190 may be applied.

  In the above embodiment, the lens device 160 is an interchangeable lens. However, the lens device 160 may be provided integrally with the imaging unit 140. In this embodiment, the imaging device 190 including the lens device 160 and the imaging unit 140 may be detachable from the gimbal 110.

  In the form in which the lens device 160 is provided integrally with the imaging unit 140, the imaging device 190 including the lens device 160 and the imaging unit 140 may not be detachable from the gimbal 110. The imaging device 190 may be fixed to the gimbal 110.

  The movable lens and imaging device 190 in the above embodiment is an example of a displacement unit. When the lens device 160 does not have a movable lens, or when the amount of change in the center of gravity position of the UAV 100 due to the displacement of the movable lens is smaller than a predetermined value, a mode in which only the imaging device 190 is a displacement portion can be adopted. When the orientation of the imaging device 190 is fixed, or when the amount of change in the center of gravity position of the UAV 100 due to the change in orientation of the imaging device 190 is smaller than a predetermined value, only the lens device 160 is a displacement part. it can.

  The displacement part may be a member other than the movable lens and the imaging device 190. For example, the displacement part may be a member such as a movable arm. A displacement part is not restricted to the member attached to UAV100. The displacement part may include a member gripped by the UAV 100. Any member can be considered as the displacement portion as long as the UAV 100 can be actively displaced.

  At least one of the steps shown in the above embodiments may be implemented by hardware or a program instructing related hardware. The program may be stored in a computer-readable recording medium. The recording medium may include at least one of a ROM, a magnetic disk, and an optical disk.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above embodiment. It is apparent from the description of the scope of claims that embodiments with such changes or improvements can be included in the technical scope of the present invention.

  The execution order of each process such as operation, procedure, step, and stage in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. ”And the like, and can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the specification, and the drawings, even if it is described using “first”, “next”, etc. for the sake of convenience, it means that it is essential to carry out in this order. It is not a thing. The description of “vertical” may be “intersecting”.

100 UAV
101 UAV main body 102 communication interface 104 UAV control unit 105 detection unit 106 memory 107 driver 108 rotor blade 109 motor 110 gimbal 120 sensor 130 prediction unit 140 imaging unit 142 imaging control unit 144 imaging element 146 memory 160 lens device 162 lens control unit 163 memory 164, 166, 168 Lens 190 Imaging device 170 First acquisition unit 172 Second acquisition unit 180 Control unit 188 Confirmation unit 200, 202, 204 Addition points 210, 220, 230, 240, 260, 270, 280
250 operator

100 UAV
101 UAV main body 102 communication interface 104 UAV control unit 105 detection unit 106 memory 107 driver 108 rotor blade 109 motor 110 gimbal 120 sensor 130 prediction unit 140 imaging unit 142 imaging control unit 144 imaging element 146 memory 160 lens device 162 lens control unit 163 memory 164, 166, 168 Lens 170 First acquisition unit 172 Second acquisition unit 180 Control unit 188 Confirmation unit
190 Imaging device 200, 202, 204 Addition point 210, 220, 230, 240, 260, 270, 280
250 operator

Claims (16)

A first acquisition unit for acquiring first information indicating a position of a displacement destination of the displacement unit held so as to be displaceable with respect to the main body unit;
A predicting unit that predicts the posture of the main body after the displacement unit is displaced to the position of the displacement destination based on the first information;
And a control unit that controls the posture of the main body based on the posture of the main body predicted by the prediction unit. The control device according to claim 1, wherein the prediction unit predicts a posture of the main body after the displacement unit is displaced to the position of the displacement destination based on the mass of the displacement unit and the first information. Based on the posture of the main body predicted by the prediction unit, the control unit suppresses a change in the posture of the main body caused by the displacement of the displacement portion being displaced to the position of the displacement destination. The control apparatus of Claim 2 which controls the attitude | position of a main-body part. The prediction unit predicts the position of the displacement unit at each of a plurality of timings within a period until the displacement unit is displaced from the position before displacement to the position of the displacement destination, and each of the plurality of predicted positions Predict the posture of the main body with respect to
The control unit controls the attitude of the main body at each of the plurality of timings based on the attitude of the main body predicted by the prediction unit with respect to the position of the displacement at each timing. 4. The control device according to any one of 3. The control unit includes a target value of the posture of the main body, a detected value of the posture of the main body by a sensor for acquiring the posture of the main body, and the posture of the main body predicted by the prediction unit. The control device according to claim 1, wherein the control device controls the posture of the main body based on the control device. The control device according to claim 1, wherein the displacement unit includes a movable lens. The control device according to claim 6, wherein the first information includes information indicating a displacement destination position of the movable lens corresponding to a focal length of a lens system including the movable lens. The control device according to claim 6 or 7, wherein the first information includes information indicating a displacement destination position of the movable lens corresponding to a focus position of a lens system including the movable lens. A storage unit that stores the position of the center of gravity of the lens device including the lens system in association with a plurality of positions of the movable lens;
The prediction unit is configured to associate the first information based on the position of the center of gravity stored in the storage unit in association with the position of the displacement destination of the movable lens indicated by the first information and the mass of the lens device. The control device according to claim 7, wherein the posture of the main body after the movable lens is displaced to a position of the displacement destination of the movable lens indicated by is predicted. The movable lens is included in a lens system of an interchangeable lens that can be attached to and detached from the main body.
The controller is
Second information is acquired from the interchangeable lens mounted on the main body to obtain second information indicating the position of the center of gravity of the interchangeable lens and the mass of the interchangeable lens corresponding to at least one of the focal length and the focus position of the interchangeable lens. The control device according to claim 6, further comprising an acquisition unit. The control device according to claim 1, wherein the displacement unit is an imaging device. The imaging device is provided with a variable orientation with respect to the main body,
The control device according to claim 11, wherein the displacement of the displacement unit is a change in a direction of the imaging device. The imaging device is detachable from the main body,
The controller is
The control device according to claim 11 or 12, further comprising: a second acquisition unit that acquires second information indicating a gravity center position of the imaging device and a mass of the imaging device from the imaging device mounted on the main body. A moving body comprising the control device according to claim 1 and the main body. Obtaining first information indicating a position of a displacement destination of a displacement portion held so as to be displaceable with respect to the main body portion;
Predicting the posture of the main body after the displacement portion is displaced to the position of the displacement destination based on the first information;
A control method comprising: controlling the posture of the main body based on the posture of the main body predicted in the predicting step.   A program for causing a computer to execute the control method according to claim 15.

Images