JP2019219874A - Autonomous moving and imaging control system and autonomous moving body - Google Patents

Abstract

To provide an autonomous moving and imaging control system and an autonomous moving body which autonomously, highly-precisely, and safely imaging, in a camera-mounted autonomous body, a predetermined imaging object by integrating a position control on the autonomous moving body and an AF control on the camera on the basis of shape information on an imaging object facility, output information from each position measuring unit and various sensors, and an instruction from each control unit.SOLUTION: An autonomous moving and imaging control system includes: object facility information 1 on an imaging object; a sensor unit 2 including various sensors for grasping a moving state; a control unit 3 that calculates an instruction value on the basis of various control rules; a motor driving unit 4 that performs drive control on a motor loaded on an unmanned aircraft 100; and a camera 5. A camera imaging unit 52 and a close-tracking control unit 36 are provided in order to surely maintain a focus on the imaging object. A PTZ control amount is calculated on the basis of a relative distance and a relative angle both obtained by a distance measuring sensor 25, and imaging is performed by the close-tracking control unit 36 while a movement along the imaging object is being performed.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an autonomous mobile imaging control system that performs movement control and imaging control of an autonomous mobile body equipped with a camera, and an autonomous mobile body.

  Conventionally, a helicopter is used by a cameraman to manually shoot a high-definition high-definition camera for images used for inspection of fine equipment at high places or large facilities where it is difficult for people to enter. It has been. However, such shooting methods are expensive, have a limited number of helicopters, take a long time from preparation to shooting, and where the process can be affected by the weather, etc. There was a problem that it was limited. For this reason, in recent years, attempts have been made to remotely or automatically shoot using an autonomously navigable unmanned aerial vehicle equipped with a camera, a so-called drone.

  For example, a method in which a drone is automatically navigated based on navigation control information such as a waypoint indicating a point on a flight route set in advance, and a camera is controlled based on imaging control information to shoot an image (see Patent Document 1) Also, a method has been proposed in which a flight route of an unmanned airplane and a route of a bogie carrying an unmanned airplane are calculated based on observation route information and equipment information (see Patent Document 2). Further, a method has been proposed in which an environment type is determined based on a plurality of sensor data mounted on an unmanned aerial vehicle, and the unmanned aerial vehicle is operated in a flight mode corresponding to the environment type (see Patent Document 3). I have.

  Further, in the conventional automatic unmanned aircraft navigation system, a user designates waypoints such as take-off points, waypoints, and landing points in advance to create navigation control information, and the unmanned aircraft is caused to fly according to the navigation control information. . For this reason, automatic navigation outside of the sight is automatically performed in poor environments where events that are not expected at the time of navigation control information generation, such as unknown obstacles, changes in the flight environment such as strong winds, and changes in the state of the shooting target, etc. It is difficult to navigate and shoot normally.

  If you still have to sail outside of the sight, make sure that there is a space around the flight path where the influence of the poor environment is small enough, or make sure that there is no contact even if there is a change in the flight environment or a change in the state of the shooting target. In addition, it is necessary to create navigation control information by taking a sufficient distance from the equipment. However, it is effective to approach the imaging target as much as possible to obtain a high-resolution image. Therefore, if the separation distance is set too large, it becomes more difficult to capture the imaging target with high definition.

  For this reason, one method of capturing high-resolution video while maintaining safe flight outside the eyes is to use a distance measurement sensor or camera attached to an unmanned aerial vehicle instead of securing a sufficient separation distance as described above. It is conceivable to incorporate a process of performing an evacuation operation autonomously upon detecting an error. In other words, integrated autonomous control that integrates detection technology, image processing technology, and flight control technology that incorporates multiple sensors and cameras on an unmanned aerial vehicle to achieve both minimum departure and maximum approach A system is conceivable.

JP-A-2017-078575 JP-A-2017-120538 JP-T-2017-501475

  However, while these advanced and innovative technologies are continuously modified and improved by the developers who are familiar with each specialized field as the technology progresses, it is necessary to reflect the modifications and improvements to the integrated autonomous control system. However, since a plurality of technologies are integrally assembled, there is a problem that the entire system must be integrally redeveloped each time.

  Also, when the object to be photographed is a thin object with a distance from the background, such as a transmission line or an overhead ground line, it is mounted on an unstable drone that flies with a sufficient separation distance to avoid contact There is a problem that it is difficult for a camera to keep focusing on an object to be photographed accurately using only an autofocus (AF) function of the camera.

  The present invention has been made in view of such circumstances, and a purpose thereof is, in an autonomous mobile body equipped with a camera, shape information of equipment to be photographed, output information of each positioning unit and various sensors, and An autonomous mobile photographing control system and an autonomous mobile object that autonomously photograph a predetermined object to be photographed with high definition and safety by integrating position control of the autonomous mobile object and AF control of a camera based on a command from each control unit. Is to provide.

  In order to solve the above problems, an embodiment of the present invention is an autonomous mobile imaging control system for controlling an autonomous mobile body equipped with a camera, the system including shape information of an imaging target and positional information of an imaging location. A facility information storage unit that stores target facility information; an IMU (Inertial Measurement Unit) unit that measures the attitude and orientation of the autonomous mobile body; and a horizontal and vertical position of the autonomous mobile body. A satellite positioning unit, an altitude measurement sensor that measures the vertical position of the autonomous mobile body, an obstacle measurement sensor that measures a separation distance between obstacles around the autonomous mobile body, and an An electromagnetic field sensor for measuring an electromagnetic field, and a distance measuring sensor for measuring a relative position and a relative angle between the autonomous moving body and a photographing object around the autonomous moving body. And a camera control unit that adjusts the focus of the camera based on the relative position to photograph the photographing object, and an image analysis unit that analyzes an image photographed by the camera, wherein the photographed image is photographed by the camera. An image analysis unit that combines a plurality of images of the imaging target to synthesize an image of the entire imaging target; an analysis recording unit that records an analysis result of the image captured by the camera in the image analysis unit; And a position control unit that controls the position of the autonomous mobile object based on the information of each of the sensors, and a communication unit that transmits the information of each of the units and each of the sensors by two-frequency and non-line-of-sight wireless communication. Features.

  In another aspect, the apparatus has target facility information including position information, shape information, and shooting position information of a shooting target to be shot by the camera, and information of an azimuth obtained from the IMU unit, the satellite positioning unit and the satellite positioning unit. The path from the self-position information including the horizontal position and the vertical position acquired from the altitude measurement sensor to the photographing start position indicated by the position information of the photographing place, the photographing target based on the target facility information It is characterized by further comprising a path forming unit that is generated so as to avoid collision with an object.

  In another aspect, the position control unit includes a route following command for moving along the route to the shooting start position, and an obstacle around the autonomous moving body and the autonomous moving body by the obstacle measurement sensor. When the separation distance from the object is shorter than a predetermined separation distance, the autonomous mobile generates a command to move away from obstacles around the autonomous mobile, and generates a command based on an integrated command obtained by integrating a plurality of the commands. It is characterized in that the position of the autonomous mobile body is controlled.

  In another aspect, the position control unit specifies a shooting target location and a shooting position of the shooting target from the own position information, the target facility information, and the relative distance and relative angle, and the camera shooting unit Is characterized in that the object to be photographed is photographed at the photographing position.

  In another aspect, when the electromagnetic field sensor measures a value equal to or greater than a predetermined value, the position control unit determines the position of the electromagnetic field generation source closest to the own position indicated by the own position information and the target facility information. And further generating a command to move away from the magnetic field generation source.

  In another aspect, the camera photographing unit, when the electromagnetic field sensor detects a value equal to or greater than a predetermined value, continues or stops imaging at a position where the value of the electromagnetic field sensor is equal to or less than a predetermined value. Features.

  In another aspect, the position control unit moves from the shooting start position to a shooting end position indicated by the position information of the shooting location while following the relative position with the shooting target so as to keep the relative position constant. And generating an instruction to perform the operation.

  In another aspect, it is characterized by confirming and recording that the entire photographing target has been photographed without missing, based on the target facility information, the image, the photographing start position information, and the photographing end position information. .

  In another aspect, the image analysis unit is configured to determine the target facility information, the image, the shooting start position information of the image, the shooting end position information based on the past of the shooting target in the past An image associated with the past photographing position information that is closer to the photographing position information is extracted from the photographing position information, and a specific difference indicating an abnormality is detected by comparison.

  In another aspect, the camera control unit is configured to, when photographing the specified photographing location of the photographing target by the target facility information, include a color, a material, and an image photographed immediately before that the target facility information has. It is characterized in that the ISO sensitivity, the aperture, the magnification and the shutter speed are adjusted based on the above, and the adjustment values of the camera are recorded.

  In another aspect, when the image analysis unit detects an abnormality, the image analysis unit outputs an abnormality detection signal including position information of an abnormal part to the position control unit and the camera photographing unit, and the position control unit includes: The camera photographing unit generates a command to move to the photographing position of the abnormal place, the photographing unit photographs the abnormal place at a higher magnification at the photographing position of the abnormal place, and starts photographing the abnormal place at a higher magnification. An end time, a shooting start position, a shooting end position, an adjustment value of the camera, and an image of the abnormal portion are recorded.

  In another aspect, the position control unit calculates an inclination angle of the image of the imaging target object with respect to each side of a frame of the image, and corrects a relative orientation between the autonomous mobile body and the imaging target object based on the inclination angle. It is characterized by doing.

  Further, in another aspect, when there is a plurality of the photographing objects, when the photographing of one of the photographing objects is completed, the path forming unit may include position information, shape information, and photographing information of the photographing object photographed by the camera. Target equipment information including the location information of the place, the information of the azimuth obtained from the IMU unit and the self-position information obtained from the satellite positioning unit and the altitude measurement sensor are used for other shooting targets whose shooting is not completed. The route to the shooting start position indicated by the information of the shooting location, the route to the next different shooting start position of the shooting target so as to avoid collision with the shooting target object based on the target facility information Is generated.

  In another aspect, each of the commands generated by the position control unit is a speed vector, and the integrated command is a combination of the speed vectors.

  In another aspect, the altitude measurement sensor includes at least one of a barometric pressure sensor and a LIDAR sensor.

  Another aspect is an autonomous mobile body that has a driving unit and can move autonomously, and includes a camera mounted on the autonomous mobile body, shape information of a shooting target, and position information of a shooting location. A facility information storage unit that stores target facility information including: an IMU unit that measures the attitude and orientation of the autonomous mobile body; a satellite positioning unit that measures the horizontal and vertical positions of the autonomous mobile body; An altitude measurement sensor that measures the vertical position of the autonomous mobile object, an obstacle measurement sensor that measures the distance between the autonomous mobile object and an obstacle around the autonomous mobile object, and an electromagnetic field sensor that measures the electromagnetic field around the autonomous mobile object A positioning sensor that measures a relative position between the autonomous mobile body and an object around the autonomous mobile body, and a position control unit that controls the driving unit to move the position of the autonomous mobile body based on the relative position. And the relative position A camera control unit that adjusts the focus of the camera based on the position and shoots the object to be shot, and an image analysis unit that analyzes an image shot by the camera, and a plurality of images shot by the camera. An image analysis unit that combines and synthesizes an image of the entire object to be photographed, an analysis recording unit that records an analysis result of an image photographed by the camera in the image analysis unit, and information of each unit and each sensor. A position control unit that controls the position of the autonomous mobile body based on the position, and a communication unit that transmits information of each unit and each sensor by dual-frequency and non-line-of-sight wireless communication are provided.

  The present invention relates to an autonomous mobile body equipped with a camera, based on shape information of equipment to be imaged, output information of each positioning unit and various sensors, and a command of each control unit, for position control of the autonomous mobile body and AF control of the camera. By integrating the above, it is possible to obtain a high-definition and safe image of a predetermined object to be photographed.

FIG. 1 is a functional configuration diagram of an autonomous mobile photography control system according to an embodiment of the present invention. FIG. 1 is a diagram illustrating an environment in which an autonomous mobile body equipped with an autonomous mobile imaging control system according to an embodiment is used. It is a figure showing the hardware composition in which the control part of the autonomous mobile photography control system concerning one embodiment of the present invention is mounted. It is a figure showing the outline of the command procedure of the autonomous mobile photography control system concerning one embodiment of the present invention. FIG. 4 is a diagram showing a control rule used in step 1 in the autonomous mobile photography control system according to one embodiment of the present invention. (A) is a diagram showing an example of generating a speed control command by integrating the high-speed position control output and the approach avoidance control output of the autonomous mobile photography control system according to one embodiment of the present invention, and (b) is FIG. 7 is a diagram showing a movement process from takeoff to a shooting target in which the shooting target can be detected by the distance measurement sensor in step 1 of the autonomous mobile shooting control system according to the embodiment of the present invention. FIG. 4 is a diagram showing a control rule used in step 2 in the autonomous mobile photography control system according to one embodiment of the present invention. (A) is a diagram showing an example of generating a speed control command by integrating the characteristic approach control output and the electromagnetic field avoidance control output of the autonomous mobile photography control system according to one embodiment of the present invention, and (b). FIG. 4 is a diagram showing a movement process to a shooting start position in step 2 of the autonomous mobile shooting control system according to one embodiment of the present invention. FIG. 4 is a diagram showing a control rule used in step 3 in the autonomous mobile photography control system according to one embodiment of the present invention. (A) is a diagram illustrating an example of generating a speed control command by integrating the approach tracking control output and the imaging control output of the autonomous mobile imaging control system according to an embodiment of the present invention, and (b) is a diagram illustrating an example. It is a figure which shows the process of image connection and abnormality detection in step 3 of the autonomous mobile imaging | photography control system which concerns on one Embodiment of this invention. It is a figure showing the control rule used at Step 3a in the autonomous mobile photography control system concerning one embodiment of the present invention. (A) shows an example in which an autonomous mobile photography control system according to an embodiment of the present invention integrates an approach following control output and a photography control output when an abnormality is detected to generate a speed control command, and (b) shows an example in which a speed control command is generated. It is a figure showing a process of an abnormal detection place in Step 3a of an autonomous mobile photography control system concerning one embodiment of the present invention. It is a figure showing the control rule used at Step 3b in the autonomous mobile photography control system concerning one embodiment of the present invention. (A) is a diagram showing an example of generating a speed control command by integrating the characteristic approach control output and the electromagnetic field avoidance control output of the autonomous mobile photography control system according to one embodiment of the present invention, and (b). Shows an example of step 3b of the autonomous mobile photography control system according to one embodiment of the present invention, and completes photography of an upper ground fault line between power transmission towers, and approaches a main subject which is a new photography target. FIG. 14 is a diagram showing an operation example in the case. It is a figure showing the control rule used at Step 3c in the autonomous mobile photography control system concerning one embodiment of the present invention. It is a figure showing a situation at the time of fixed equipment inspection in Step 3c of the autonomous mobile photography control system concerning one embodiment of the present invention. FIG. 6 is a diagram showing a control rule used in step 4 in the autonomous mobile photography control system according to one embodiment of the present invention. It is a figure which shows the moving process from an imaging position to landing in step 4 of the autonomous mobile imaging | photography control system which concerns on one Embodiment of this invention.

  Most of the objects to be photographed have pre-existing high-precision shape information that identifies the installation location, shape, characteristics, etc. In some cases, markers are arbitrarily attached so that the administrator of the target equipment can easily judge with sensors and cameras In some cases, it has been. Therefore, the autonomous mobile body stores in advance target equipment information including position information and shape information of the inspection object, and moves and shoots by combining the target equipment information with information from the sensor and the camera. Devise a control system.

  The present invention provides a high-speed position control law emphasizing high-speed movement, a high-precision position control law emphasizing high-precision flight, and a distance to an object to be photographed, in order to realize the above autonomous mobile photography control system. Control approach rule, approach tracking control rule that moves while keeping the distance to the shooting target constant, approach avoidance control rule that detects and avoids unknown obstacles, electromagnetic field that affects the movement of the autonomous mobile object And a command integration unit that integrates a control output based on each of the electromagnetic field avoidance control rules for detecting and avoiding the generation source from the target facility information. In addition, the image analysis unit includes an image analysis unit that analyzes an image captured by the camera, compares the captured image with a reference image such as a past image, and detects an abnormality, to determine whether the entire imaging target has been captured. These various control rules, the command integration unit, and the image analysis unit are recorded on a storage medium as a program, and the functions are realized by being executed.

(1) Various control rules in which known information is combined with sensor and camera information When an object to be photographed is photographed by a camera mounted on an autonomous mobile body such as an unmanned aerial vehicle, the same position is photographed periodically from the same direction. In many cases, the procedure is similar for the purpose of photographing even if there is a difference in the shape of the photographing target object of the same type. Therefore, by preparing known information relating to shooting, such as map information and object information, and information from sensors and cameras, it is possible to recognize the positional relationship between the self-position and the object, and therefore, the movement control information is determined in advance by a shooting procedure. And various control procedures can be automatically generated based on the control rules.

  In addition, even when the movement control information is not used, the avoidance operation, the approach operation, and the photographing operation are performed based on information from the sensor and the camera, so that safe and high-quality information can be obtained by enabling the autonomous moving body to process. Autonomous shooting control can be introduced.

(2) Parallel introduction by decentralization and encapsulation of various control rules and adoption of command integration unit In order to realize each function by individually configuring control rules according to the function to be given to the autonomous mobile object It is possible to use information from the sensors and cameras, equipment information, and peripheral information to be used to the maximum without waste.

  If multiple control rules are integrated into an integrated autonomous control rule, the format, update cycle, and timing of each information do not match. A large-scale filter for attaching, estimating, and synchronizing is separately required. However, by dispersing and encapsulating, the filter can be included in a control law, and waste of the filter can be reduced. If the commands of the navigation control rules are unified into speed vectors or the like, the commands obtained by different technologies can be integrated without synchronization. In addition, since each control rule can be individually modified and improved by a developer who is familiar with each technology, it is possible to always obtain autonomous high-speed movement performance employing the latest technology.

  Further, the processing cycle of the integrated autonomous control in which everything is integrally formed is governed by the calculation time of image processing or the like which takes the longest time for processing. However, in the present invention, by decentralizing and encapsulating the processing for each control law, those in which the processing of each control law such as a safe navigation control law is fast are processed at a high speed and high frequency by shortening the processing cycle, The reliability of autonomous high-speed movement can be improved.

  Hereinafter, embodiments of the present invention will be described in detail.

  FIG. 1 shows a functional configuration diagram of an autonomous mobile photography control system according to an embodiment of the present invention. FIG. 2 shows an environment in which an autonomous mobile body equipped with the autonomous mobile imaging control system according to the present embodiment is used. The autonomous mobile photographing control system according to the present embodiment controls an autonomous mobile body such as the unmanned aerial vehicle 100 and the camera 5 mounted thereon, and avoids an unknown obstacle 300 while determining a target facility 200 to be inspected. (A subject to be photographed) can be photographed.

  An autonomous mobile imaging control system according to an embodiment of the present invention includes target equipment information 1 including detailed shape information such as latitude, longitude, elevation information, and CAD data of a reference point of an imaging target to be inspected, A sensor unit 2 including various sensors used to grasp the state, a control unit 3 that calculates a command value based on various control rules, a motor drive unit 4 that performs drive control of a motor mounted on the unmanned aerial vehicle 100, A camera 5 is provided.

  The sensor unit 2 includes an IMU (Inertial Measurement Unit) 21 for measuring azimuth and acceleration, a satellite positioning unit 22, a pressure sensor 23 for measuring altitude, and an altitude LIDAR (light detection and ranging: Light Detection And Ranging). ) Section 24, a distance measuring sensor 25 for measuring a relative distance and a relative angle to an object to be photographed, an obstacle LIDAR section 26 for measuring a distance from an obstacle around the autonomous moving body, an infrared sensor 27, an ultrasonic sensor 28, An electromagnetic field sensor 29 for measuring an electromagnetic field is included. Note that the type of sensor is not limited to this, and a necessary sensor may be selected or added according to the required accuracy of position control and use environment.

  The control unit 3 performs various independent control rules, that is, attitude control 31, speed control 32, high-speed position control 33, high-precision position control 34, feature approach control 35, approach tracking control 36, approach avoidance control 37, and electromagnetic field avoidance control. 38, a photographing control 39, and a command integration unit 40 for integrating command values of those control rules. Of the control rules, the attitude control 31 and the speed control 32 may be the same as those used in a conventional unmanned aerial vehicle. However, since it is difficult to operate with sufficient accuracy by the conventional position control, a new high-speed position control 33, a high-accuracy position control 34, a feature approach control 35, an approach following control 36, an approach avoidance control 37, and The electromagnetic field avoidance control 38 is employed. Further, the plurality of control rules are integrated by the command integration unit 40. In addition, an image analysis unit 41 that analyzes an image captured by the camera 5 is provided.

  The camera 5 controls the camera 5 mounted on the unmanned aerial vehicle 100 to perform a pan / tilt / zoom (PTZ) operation, and adjusts shooting parameters such as AF, aperture, ISO sensitivity, and shutter speed. And a camera photographing section 52 for photographing the image.

  FIG. 3 shows a hardware configuration in which the control unit 3 of the autonomous mobile photography control system according to the embodiment of the present invention is mounted. The various controls 31 to 39, the command integration unit 40, and the image analysis unit 41 are realized as programs stored in the storage 303 and executed by the CPU 302. The memory 301 is a main memory such as a RAM. Note that the target facility information 1 may be recorded in the storage 303. The sensor unit 2, the motor driving unit 4, and the camera 5 are connected to the CPU 302 via an input / output interface (I / F) 308. Further, in the present invention, since each control law is decentralized and encapsulated, the GAP 304 and the DRAM 305 are used for the processing of the image analysis unit 41, and the mobile unit control unit 306 incorporating the conventional attitude control 31 and speed control 32 is used. May be used. The communication unit 307 can transmit and receive data such as transmitting information on the unmanned aerial vehicle 100 under autonomous control, receiving supplementary information for improving the accuracy of the satellite positioning unit 22, and if necessary. It is also possible to transmit the captured image to the outside, and to receive an external command and image analysis information.

  Hereinafter, an autonomous flight system of the unmanned aerial vehicle 100 such as a multi-copter will be described as an example of a procedure to be processed by the command integration unit 40 by taking an example of an overhead ground line inspection of a power facility required to take a high-definition image. . It should be noted that the present invention is also applicable to close-up photography by an unmanned aerial vehicle 100, even if the object to be photographed is a tower, a dam, a building, a bridge, a chimney, or other large-scale equipment, if the structure has high-precision information Is possible. Further, although an example in which the unmanned aerial vehicle 100 is used is shown here, the invention is not limited to the unmanned aerial vehicle, and another autonomous mobile body capable of position control may be used.

  FIG. 4 shows an outline of a command procedure of the autonomous mobile photography control system according to one embodiment of the present invention. Since the target equipment information 1 is held, the nearest target equipment 200 for performing inspection for inspection based on its own position and the remaining amount of the battery, the other target equipment 200 for which inspection is performed continuously, and the final target Route and procedure to a typical landing site can be proposed. Therefore, in the present invention, there is no need to manually set navigation control information such as waypoints (point information on a route).

  Hereinafter, the details of the order procedure will be described.

(Step 1) Takeoff / High-Speed Movement FIG. 5 shows a control rule used in step 1 in the autonomous mobile photography control system according to one embodiment of the present invention. In step 1, in addition to the attitude control 31 and the speed control 32, in order to take off the unmanned aerial vehicle 100 and make the unmanned aerial vehicle 100 approach the shooting position of the shooting target efficiently and at high speed, two high speed position control 33 and approach avoidance control 37 Adopt control rules. It suffices that the unmanned aerial vehicle 100 can enter the area where the sensor reliably detects in step 2 described below. However, since the flight time is limited by the battery capacity, the self-position information acquired by using the satellite positioning unit 22 before the flight starts. A flight route is automatically generated based on the location information of the imaging target equipment 200 acquired from the target equipment information 1.

  However, the take-off and landing points of the unmanned aerial vehicle 100 cannot always be set to a place convenient for flight according to laws and regulations, so that obstacles that can be determined from the target facility information 1 are routed so as to avoid collision. Is generated.

  While controlling to move along this flight path, the control which avoids the unknown obstacle 300 and the electromagnetic field detected by each sensor at the same time is combined.

  FIG. 6A shows an example of generating a speed control command by integrating the high-speed position control output and the approach avoidance control output of the autonomous mobile photography control system according to one embodiment of the present invention. FIG. 6B shows a moving process from the takeoff in step 1 of the autonomous mobile shooting control system according to the embodiment of the present invention to the shooting target from which the distance measurement sensor 25 can detect the shooting target. The high-speed position control 33 and the approach avoidance control 37 output commands to the command integrating unit 40 independently of each other. Since both of these outputs are velocity vectors, the command integrating unit 40 outputs the commands, for example, as shown in FIG. By combining the command values as shown in (1), an integrated speed command can be output. Therefore, it is possible to approach the vicinity of the imaging target while avoiding collision with the unknown obstacle 300.

  As shown in FIGS. 5 and 6 (a), when the aircraft starts taking off and taking off, it mainly ascends while measuring the altitude by the satellite positioning unit 22 or the atmospheric pressure sensor 23 or the altitude LIDAR unit 24 by the high-speed position control 33. A horizontal flight based on the attitude of the unmanned aerial vehicle 100 obtained indirectly from the attitude control 31 while measuring the horizontal position by the satellite positioning unit 22, or a combination of the horizontal flight and the shooting target along a flight procedure path set in advance. Move around the object. At this time, as shown in FIG. 6B, when there is an unknown obstacle 300 on a preset flight procedure route, the unknown obstacle 300 is measured as an obstacle to measure the separation distance from the obstacle around the autonomous mobile body. When the detection is performed by the LIDAR unit 26, the infrared sensor 27, and the ultrasonic sensor 28, the approach avoidance control 37 outputs an avoidance command which is a speed vector in a direction away from the unknown obstacle 300. The approach avoidance control 37 converts the separation distance measured by each sensor into a Gaussian function or a sigmoid function instead of proportional so that the unknown obstacle 300 can be avoided while securing a predetermined separation distance or more that is set in advance. Alternatively, the velocity vector may be decomposed into a horizontal component and a vertical component, and the weight of the horizontal component having poor response of the unmanned aerial vehicle 100 may be increased to some extent. Further, since the purpose of step 1 is to move safely and at high speed so that the distance measuring sensor 25 can stably detect the photographing target, even if the unmanned aerial vehicle 100 deviates from the flight path by the approach avoidance control 37, the flight path You do not need to go back.

(Step 2) Movement of Shooting Start Point FIG. 7 shows a control rule used in step 2 in the autonomous mobile shooting control system according to one embodiment of the present invention. When the magnitude of the high-speed position control output and the approach avoidance control output are sufficiently small in Step 1 and the unmanned aerial vehicle 100 approaches the position where the positioning sensor 25 can stably measure the separation distance from the imaging target, Step 2 Move to In step 2, based on the separation distance measured by the distance measurement sensor 25, the camera further approaches the shooting position of the shooting target. In step 2, it is required to maintain a highly accurate relative position in order to acquire a high-definition image in step 3, and to stop the unmanned aerial vehicle 100. Therefore, a high-precision position control 34, a characteristic approach control 35, and an electromagnetic field avoidance control 38 are used instead of the high-speed position control 33 and the approach avoidance control 37 used in step 1.

  FIG. 8A shows an example of generating a speed control command by integrating the characteristic approach control output and the electromagnetic field avoidance control output of the autonomous mobile imaging control system according to one embodiment of the present invention. FIG. 8B shows a process of moving to the shooting start position in step 2 of the autonomous mobile shooting control system according to the embodiment of the present invention.

  The positional accuracy between the object to be photographed and the unmanned aerial vehicle 100 required at the time of photographing may be different from the position accuracy information and the information output cycle obtained by the satellite positioning unit 22 because there is a possibility of colliding with the target equipment. The performance is insufficient as a sensor.

  Therefore, in the present invention, the relative position between the unmanned aerial vehicle 100 and the object to be photographed is measured with a measurement accuracy of centimeters or less by utilizing the target facility information 1 and the distance measuring sensor 25 capable of precise measurement in a short distance. , A high-precision position control 34 of several tens of centimeters or less is performed. In the characteristic approach control 35, the photographing target is specified and associated with the photographing start position by associating the photographing target's facility information included in the target facility information 1 with the photographing target's shape measured by the distance measuring sensor 25. Induce. In FIG. 2, the imaging target is assumed to be imaged from above for the sake of explanation, but the imaging direction of the imaging target can be set to an arbitrary imaging start position if there is no restriction on the position attached to the unmanned aerial vehicle 100. At this time, a safe approach start area is set in step 1 so that the photographing start position is not obstructed by a part of the photographing target equipment on the moving route. As another means, when the unknown obstacle 300 is abnormally approaching the imaging target equipment and it is not desired to make the unmanned aerial vehicle 100 detour, move to the vicinity of the imaging target equipment by the target equipment information 1 in step 2. It may assist in setting a route. As another means, an approach avoidance control 37 may be added in case an unknown obstacle 300 such as a bird's nest is to be avoided in the photographing target.

  In addition, when taking a picture in close proximity to the power transmission equipment in operation, a strong electromagnetic field is generated near the electric wire, so that the electric field may cause a failure in the electric and electronic circuit of the unmanned aerial vehicle 100, or the magnetic field may cause a problem in the IMU 21 There is a risk that the electronic compass being used loses its bearing. Therefore, in step 2, the electromagnetic field avoidance control 38 is introduced to ensure safe flight. Although the electromagnetic field sensor 29 can determine the magnitude of the electromagnetic field received by the unmanned aerial vehicle 100, it is difficult to identify the source. However, in addition to the measured value obtained from the electromagnetic field sensor 29, the electromagnetic field generation source can be specified based on the own position and the target facility information 1.

  In order to specify the electromagnetic field generation source, the output vector of the electromagnetic field avoidance command is derived from the eigenvalue and the eigenvector, the eigenvalue is a measurement value obtained from the electromagnetic field sensor 29, and the eigenvector is the target equipment information 1 and the distance measurement sensor. This is realized by obtaining a direction to be avoided with respect to the source of the electromagnetic field based on 25. For example, in FIG. 8, since an electromagnetic field is generated by a transmission line that applies power to the lower side of the unmanned aerial vehicle 100, when the strength of the electromagnetic field exceeds a predetermined threshold, the electromagnetic field is increased to secure a separation distance. A velocity vector in the direction is generated. When the magnitude of the magnetic field can be measured by the electronic compass provided in the IMU unit 21, the IMU unit 21 may be used as a magnetic field sensor. As another method, if the magnitude of the electromagnetic field of the transmission line can be known from the power flow information of the transmission line, it may be used via the communication unit 307.

(Step 3) Approach Tracking Imaging FIG. 9 shows a control rule used in step 3 in the autonomous mobile imaging control system according to one embodiment of the present invention. In step 3, since the camera has reached the shooting position of the shooting target object with high precision in step 2, the shooting control 39 obtains the camera driving unit 51 and the camera shooting unit 52 of the camera 5 by the distance measurement sensor 25. The PTZ control amount is calculated based on the relative position and a command to start photographing is output. At the same time, in the approach tracking control 36, similarly to the feature approach control 35 in step 2, the robot moves along the photographing object based on the relative position to the photographing object obtained by the distance measurement sensor 25 without depending on the satellite positioning information. While controlling the position. FIG. 10 naturally draws a flight trajectory along a catenary curve drawn by an electric wire in order to move with high accuracy along an overhead ground line. If the imaginary ground line of the object to be photographed fluctuates, it flies following the shaking.

  FIG. 10A shows an example of generating a speed control command by integrating the approach tracking control output and the shooting control output of the autonomous mobile shooting control system according to one embodiment of the present invention. FIG. 10B shows a process of image connection and abnormality detection in step 3 of the autonomous mobile photography control system according to the embodiment of the present invention.

  Note that it is also possible to adjust the posture within a range where stability can be maintained from an image captured by the camera 5. For example, when the imaginary ground wire is photographed from directly above, the image analysis unit 41 calculates the inclination angle of each image frame of the imaginary ground wire captured by the camera 5 with respect to each side of the frame, and is obtained from the image via the command integration unit 40. The relative angle between the angle of the imaging target and the yaw angle of the unmanned aerial vehicle 100 is determined, and transmitted to the attitude control 31 to control the yaw angle of the unmanned aerial vehicle 100, thereby changing the angle of the imaging target in the image vertically or horizontally. Can also be corrected.

  The approaching follow-up control 36 approaches the object to be photographed. However, the unmanned aerial vehicle 100 is constantly subjected to disturbance such as a crosswind, so that a certain degree of temporary position fluctuation always occurs. At the time of high-definition photographing by the camera 5, camera shake occurs due to a change in the position of the unmanned aerial vehicle 100. Therefore, a telephoto lens having a long focal length and a shallow depth of field tends to cause blurred images. Further, it is difficult to keep focusing on a thin electric wire or a small screw floating in the air by an ordinary camera's autofocus (AF) function because the area of the image reflected on the image is small. Therefore, in the present invention, in order to keep focusing on the photographing target reliably, the camera photographing unit 52 sets the relative position to the photographing target obtained from the target facility information 1 and the distance measurement sensor 25 in the approach tracking control 36. Adjust focus based on Alternatively, if the target facility is not clearly present around the unmanned aerial vehicle 100, the camera photographing unit 52 directly acquires the distance information output from the distance measurement sensor 25, and from there, to the photographing target as described later. May be extracted, and focus adjustment may be performed based on the extracted distance information.

  As another method, in order to extract only the relative position from the relative position obtained by the distance measuring sensor 25 to the object to be photographed, for example, from the measured relative position information, the shortest distance is determined. The relative distance between the distance measurement sensor 25 and the object to be photographed may be used. However, in this case, since the focus is adjusted to the point closest to the camera 5, when an object having a depth is photographed, a portion other than the near side is out of focus.

  As another method, when the shape is relatively large and can be determined by the distance measuring sensor 25, the relative position may be determined by excluding other relative position information, and the color of the object to be photographed has a characteristic. If it is out of focus, the approximate position can be determined by the image analysis unit 41, so that the relative distance can be clarified by limiting valid information among the relative position information obtained from the distance measurement sensor 25.

  As still another method, the approximate distance to the photographing target is specified from the target facility information 1 and the own position, and the distance closest to the specified distance among the distances measured by the distance measurement sensor 25 is determined. The relative distance between the distance measurement sensor 25 and the object to be photographed may be used.

  The relative distance between the object to be photographed and the camera 5 is determined based on the relative distance and the relative angle between the object to be photographed and the distance measuring sensor 25 and the relative position between the camera 5 and the distance measuring sensor 25 and the distance between the camera 5 and the distance measuring sensor 25. The calculation is performed based on the relative position of each of the sensors 25 and the arrangement state of the camera 5 and the distance measuring sensor 25.

  In order to obtain a high-definition image, in addition to precisely focusing as described above, the ISO sensitivity, aperture, and shutter speed must be appropriately adjusted according to the flight date and time, weather, color and material of the shooting target, etc. Need to adjust. In the present invention, since the photographing target can be specified in advance, the target equipment information 1 records the color and material of the photographing target and parameters to be corrected based on the evaluation of the image taken immediately before by the image analysis unit 41. The image quality can also be improved by readjusting the ISO sensitivity, aperture, and shutter speed.

  During shooting, disturbance such as a cross wind may be applied to the unmanned aerial vehicle 100, and it may not be possible to shoot the route and timing as scheduled, continuity may be lost in the obtained plurality of images, and a partial image of the shooting target may be obtained. Not always. For this reason, the image analysis unit 41, for example, connects the obtained plurality of images based on the feature points of the imaging target, checks whether there is no omission in acquiring the image of the imaging target, and if there is any omission, In order to take an image of the location, the approach follow-up control 36 is instructed.

  As another method, if the image analysis unit 41 does not have an imaginary ground line at both ends from the captured image of the imaginary ground line, or if the imaginary ground line is not reflected, the image analysis unit 41 immediately re-executes it. You may.

  The image obtained through the above-described processing includes, together with the image, a shooting start time, a shooting end time, a shooting start coordinate, a shooting end coordinate, positional information of the shooting target within the angle of view, and each adjustment value of the camera 5 in the storage 303. Is recorded to

  At this time, the image analysis unit 41 can recognize the abnormality by comparing with the past photographing result stored in the storage 303, but the recorded coordinates are completely different between the past image and the current image. It does not match. Therefore, an abnormal part is detected by associating the closest image from the recording group acquired in the past with the start / end coordinates of the acquired image. As another method, when the secular change of the photographing target is small, a sample image immediately after laying may be used as a comparison target. The image analysis unit 41 notifies the command integration unit 40 of an abnormality detection signal when the comparison indicates a large difference or a suspected abnormality such as discoloration. The abnormality detection signal includes target facility position information and shooting position information.

  Processing in the image analysis unit 41 such as image connection and detection of an abnormal part is performed by the CPU 302, GPU 304, and DRAM 305 that can be mounted on the unmanned aerial vehicle 100. Consumption tends to increase. Therefore, as another method, if the transmission band of the communication unit 307 is sufficient, the image data may be transmitted to an external computer in order to suppress power consumption, and the processing in the image analysis unit 41 may be performed externally.

  When detecting an electromagnetic field that affects the unmanned aerial vehicle 100 during shooting, the electromagnetic field avoidance control 38 determines an electromagnetic field generation source based on the target facility information 1 and the distance measurement sensor 25, as in step 2. An electromagnetic field avoidance control output is generated with the direction to be avoided as an eigenvector and the measurement value obtained from the electromagnetic field sensor 29 as an eigenvalue. At this time, since the electromagnetic field avoidance control output is a command contrary to the approach tracking control output, while the electromagnetic field avoidance control output is output, the approach tracking is performed at a different point from the relative position where the electromagnetic field avoiding control output is not affected by the electromagnetic field. continue. However, the camera control unit 51 adjusts the zoom magnification, and the angle of view changes accordingly, in order to ensure a constant image quality even in the state where the relative distance is different. Therefore, the image analyzing unit 41 instructs the command integrating unit 40 to adjust the following speed, and the command integrating unit 40 instructs the approach following control 36 to adjust the speed. As still another method, since the magnitude of the electromagnetic field generated from the electromagnetic field generation source is not constant, the photographing control 39 may be stopped so that photographing is performed again when the influence of the electromagnetic field has stopped. Note that, when an extremely large magnetic field is detected from the imposed electric wire during shooting, the electromagnetic field avoidance control output may increase and the shooting target may retreat outside the detection range of the distance measurement sensor 25. Then, the process proceeds to step 2.

  Further, even if the communication module 304 has difficulty in transmitting and receiving data to and from the outside due to the influence of the mountains and the unknown obstacle 300, the communication module 304 follows the route on which the unmanned aerial vehicle 100 waits or flies until the transmission and reception of data is restored. It may return autonomously. In addition, the battery voltage or capacity may be monitored in case of being affected by a strong wind or a magnetic field, or when the battery is greatly consumed due to frequent shift to step 3a to be described later. Then, the process proceeds to step 4.

(Step 3a) Detailed Shooting of Abnormal Location FIG. 11 shows a control rule used in step 3a in the autonomous mobile shooting control system according to one embodiment of the present invention. FIG. 12A shows an example of generating a speed control command by integrating an approach following control output and an imaging control output when an abnormality is detected in the autonomous mobile imaging control system according to an embodiment of the present invention, FIG. 12 (b) shows a process of photographing an abnormal detection point in step 3a of the autonomous mobile photographing control system according to one embodiment of the present invention. In step 3, when the command integration unit 40 receives notification information of abnormality detection from the image analysis unit 41, the process proceeds to step 3a. In step 3a, the command integration unit 40 instructs the approach following control 36 to stop at the location where the abnormality is detected, and the camera driving unit 51 from the imaging control 39 performs zooming so that a high-definition image can be captured. Outputs a command to perform readjustment on the high magnification side.

  Although the angle of view narrows when a clearer image is taken by enlarging the shooting target point by zooming, the location of the shooting target is reflected in the image because the magnitude of the temporary positional change of the unmanned aerial vehicle 100 does not change. The frequency of disappearance increases. Therefore, as shown in FIG. 12B, the shooting control 39 continues autonomous following by the approach following control 36 until a stable state in which a video suspected of being abnormal can be shot is obtained, and shooting becomes possible. Shooting is started when When the photographing is completed, the photographing control notifies the command integrating unit 40 of the completion, and the process proceeds to step 3 or 4 in response to the notification information.

(Step 3b) Movement between photographing points FIG. 13 shows a control rule of movement between photographing points used in step 3b in the autonomous mobile photographing control system according to one embodiment of the present invention. If there is not a single object to be photographed, control for moving to the next object to be photographed approaches the photographing start position of the object as in step 2. It should be noted that since the photographing target whose photographing is completed in step 3 becomes a new obstacle, the feature approach control 35 is a command to combine the departure output and the new approach output. As another method, since the original imaging target is a known obstacle, the high-speed position control 33 may be used for the separation output. As another method, the object to be photographed may be moved to the next object after a distance from the object once photographed.

  FIG. 14A shows an example of generating a speed control command by integrating the characteristic approach control output and the electromagnetic field avoidance control output of the autonomous mobile photography control system according to one embodiment of the present invention. 3) shows an example of step 3b, and shows an operation example in a case where the photographing of the overhead overhead ground line between the power transmission towers is completed and the vehicle approaches the main line which is a new photographing target. However, when approaching an electric wire to which power is being applied, the object to be photographed becomes an electromagnetic field generation source, and an electromagnetic field avoidance control output is generated in a direction away from the electric wire even during approach, so the overhead ground wire The approach speed is slower than the approach to. The output is adjusted so that the characteristic approach control output and the electromagnetic field avoidance control output are opposed to each other before an area where a strong electromagnetic field that may cause an abnormality in the unmanned aerial vehicle 100 is generated. By setting a position where the avoidance control output is in opposition to a new imaging position, it is possible to prevent the position from entering a region where a strong electromagnetic field of a predetermined value or more is generated.

(Step 3c) Inspection of Fixed Equipment FIG. 15 shows a control rule for inspecting fixed equipment used in step 3c in the autonomous mobile photography control system according to one embodiment of the present invention. FIG. 16 shows a state at the time of fixed facility inspection in step 3c of the autonomous mobile photography control system according to the embodiment of the present invention.

  In the case of taking a close-up image of a structure such as a steel tower, or equipment fixed thereto, for example, an insulator or an arc horn, at the same time as moving to the image capturing position, the relative distance to the camera driving unit 51 and the camera image capturing unit 52 is the same as in step 3. A shooting command is output together with information such as. The control rule to be operated is the same as that in step 3, but the object equipment information 1 is used more effectively because it is an object such as an overhead ground wire or electric wire whose position does not greatly change depending on the situation at the time of shooting. Thus, the position information of the unmanned aerial vehicle 100 can be corrected. Further, since the distance measurement sensor 25 detects a plurality of objects, the imaging control 39 can acquire a plurality of images by moving the focus of the camera 5.

(Step 4) Landing at Target Position FIG. 17 shows a control law for landing at the target position used in step 4 in the autonomous mobile photography control system according to one embodiment of the present invention. FIG. 18 shows a moving process from the photographing position to the landing in step 4 of the autonomous mobile photographing control system according to the embodiment of the present invention.

  Step 4 is a procedure selected when returning the unmanned aerial vehicle 100 to a target point, for example, a take-off point. When the photographing is completed or the remaining flight time is short, the unmanned aerial vehicle 100 When the autonomous photographing flight becomes difficult, or when a return command is received from the operator through the communication unit 307, the vehicle lands at an arbitrary point according to the same control rule as in step 1.

  In addition to the point where the unmanned aerial vehicle 100 took off, the landing point is determined in advance by setting a plurality of landing candidate points different from the takeoff point, so that the point suitable for the remaining flight time and the poor environmental condition according to the battery state is determined. The landing site can be changed as needed, and the safety can be further improved.

DESCRIPTION OF SYMBOLS 1 Target equipment information 2 Sensor part 3 Control part 4 Motor drive part 5 Camera 21 IMU part 22 Satellite positioning part 23 Atmospheric pressure sensor 24 Altitude LIDAR part 25 Distance measuring sensor 26 Obstacle LIDAR part 27 Infrared sensor 28 Ultrasonic sensor 29 Electromagnetic field sensor 31 attitude control 32 speed control 33 high-speed position control 34 high-precision position control 35 feature approach control 36 approach tracking control 37 approach avoidance control 38 electromagnetic field avoidance control 39 shooting control 40 command integration unit 41 image analysis unit 51 camera drive unit 52 camera shooting Unit 100 unmanned aerial vehicle 200 equipment to be photographed 300 unknown obstacle 301 memory 302 CPU
303 Storage 304 GPU
305 DRAM
306 Mobile unit control unit 307 Communication unit 308 Input / output I / F

In order to solve the above problems, an embodiment of the present invention is an autonomous mobile imaging control system for controlling an autonomous mobile body equipped with a camera, the system including shape information of an imaging target and positional information of an imaging location. A facility information storage unit that stores target facility information; an IMU (Inertial Measurement Unit) unit that measures the attitude and orientation of the autonomous mobile body; and a horizontal and vertical position of the autonomous mobile body. a satellite positioning unit for the altitude measuring sensor for positioning the vertical position of the autonomous moving body, and the obstacle measurement sensor that measures the distance between the autonomous moving body around the obstacle, the front Symbol autonomous moving body wherein a distance measuring sensor for measuring the relative position and the relative angle between the autonomous moving body around the shooting target, azimuth information acquired from the IMU unit, measuring the satellite The path from the self-position information including the horizontal position and the vertical position obtained from the unit and the altitude measurement sensor to the shooting start position indicated by the position information of the shooting location, based on the target facility information A route forming unit that generates so as to avoid collision with the shooting target, a route following command for moving to the shooting start position along the route, and the autonomous moving body and the When the separation distance from the obstacle around the autonomous moving body becomes shorter than a predetermined separation distance, the autonomous moving body generates a command to move away from the obstacle around the autonomous moving body, and integrates the plurality of commands. It includes the position control section for controlling the position of the autonomous moving body on the basis of the integrated command, a camera control unit for photographing a focus the imaging target by adjusting the camera based on the relative position It is characterized by having.

In another aspect, an image analysis unit that analyzes an image captured by the camera, an image analysis unit that combines a plurality of images captured by the camera and synthesizes an image of the entire imaging target, An analysis recording unit that records an analysis result of an image captured by the camera in the image analysis unit is further provided.

In another aspect, the apparatus further includes a communication unit that transmits information of each of the sensors and each of the units by dual-frequency and non-line-of-sight wireless communication .

In another aspect, the apparatus further comprises an electromagnetic field sensor that measures an electromagnetic field around the autonomous mobile body, wherein the position control unit is configured to, when a value equal to or more than a predetermined value is measured by the electromagnetic field sensor, obtain the position information. The position of the electromagnetic field generation source closest to the own position indicated by is specified from the target facility information, and a command to move away from the magnetic field generation source is further generated.

Another aspect is an autonomous mobile body that has a driving unit and can move autonomously, and includes a camera mounted on the autonomous mobile body, shape information of a shooting target, and position information of a shooting location. A facility information storage unit that stores target facility information including a target object, a facility information storage unit that stores target facility information including shape information of a photographing target and positional information of a photographing location, and measures a posture and an orientation of the autonomous mobile body. An IMU unit, a satellite positioning unit that measures the horizontal and vertical positions of the autonomous mobile object, an altitude measurement sensor that measures the vertical position of the autonomous mobile object, and obstacles around the autonomous mobile object. and the obstacle measurement sensor which measures the separation between the front Symbol autonomous moving body and the distance sensor measuring for measuring the relative position of the autonomous moving body surrounding objects, azimuth information acquired from the IMU unit, the satellite positioning Part and the altitude The path from the self-position information including the horizontal position and the vertical position acquired from the measurement sensor to the photographing start position indicated by the position information of the photographing place is defined by the photographing target object based on the target facility information. A route forming unit that generates so as to avoid collision with the vehicle, a route following command for moving to the shooting start position along the route, and the autonomous moving body and the surroundings of the autonomous moving body by the obstacle measurement sensor. When the separation distance from the obstacle is shorter than a predetermined separation distance, the autonomous moving body generates a command to move away from obstacles around the autonomous moving object, and generates an integrated command obtained by integrating a plurality of the commands. to a position control unit for controlling the position of the autonomous moving body based, characterized in that to adjust the focus of the camera based on the relative position and a camera control unit for photographing the photographic subject .
Another aspect is an autonomous mobile imaging control system that controls an autonomous mobile body equipped with a camera, and stores equipment information including target equipment information including shape information of an imaging target and position information of an imaging location. Unit, including a sensor for measuring the attitude and orientation of the autonomous mobile body, a sensor unit having a plurality of sensors, and the location information of the shooting location from the self-position information of the autonomous mobile body obtained from the sensor unit indicates A route forming unit that generates a route to a shooting start position based on the target facility information so as to avoid collision with the shooting target, and a route for moving to the shooting start position along the route A tracking command, and when the separation distance between the autonomous moving body and an obstacle around the autonomous moving body is shorter than a predetermined separation distance by the sensor unit, the autonomous moving body is moved around the autonomous moving body. A position control unit that generates a command to move away from the harmful object and controls the position of the autonomous mobile body based on an integrated command obtained by integrating a plurality of the commands, and adjusts the focus of the camera based on the relative distance to perform the shooting. A camera control unit for photographing the object.

In order to solve the above problems, an embodiment of the present invention is an autonomous mobile imaging control system for controlling an autonomous mobile body equipped with a camera, the system including shape information of an imaging target and positional information of an imaging location. A facility information storage unit that stores target facility information; an IMU (Inertial Measurement Unit) unit that measures the attitude and orientation of the autonomous mobile body; and a horizontal and vertical position of the autonomous mobile body. A satellite positioning unit, an altitude measurement sensor that measures the vertical position of the autonomous mobile object, an obstacle measurement sensor that measures a separation distance between obstacles around the autonomous mobile object, the autonomous mobile object, A distance measuring sensor for measuring a relative position and a relative angle with respect to an object to be photographed around the autonomous moving object; information on an azimuth obtained from the IMU unit; The path from the self-position information including the horizontal position and the vertical position obtained from the unit and the altitude measurement sensor to the shooting start position indicated by the position information of the shooting location, based on the target facility information A path forming unit for generating a collision with the object to be photographed, and an image analyzing unit for analyzing an image photographed by the camera, wherein a plurality of images photographed by the camera are connected to each other, An image analysis unit that synthesizes an image of the entire object, a path following command for moving to the shooting start position along the path, and the autonomous moving body and obstacles around the autonomous moving body by the obstacle measurement sensor A command relating to a plurality of flight controls, including a command to move the autonomous mobile body away from obstacles around the autonomous mobile body when the separation distance becomes shorter than a predetermined separation distance , and A plurality of commands including a command generated by the image analysis unit to move to a shooting position of an abnormal location detected from an image shot by the camera, and a command to correct a relative orientation between the autonomous moving body and the shooting target. of Directive imaging control, a position control unit for controlling the position and attitude of the autonomous moving body on the basis of the integrated command which integrates plural directive in response to the flight conditions, the focus of the camera based on the relative position A camera control unit for adjusting and photographing the object to be photographed.

Another aspect is an autonomous mobile body that has a driving unit and can move autonomously, and includes a camera mounted on the autonomous mobile body, shape information of a shooting target, and position information of a shooting location. A facility information storage unit that stores target facility information including a target object, a facility information storage unit that stores target facility information including shape information of a photographing target and positional information of a photographing location, and measures a posture and an orientation of the autonomous mobile body. An IMU unit, a satellite positioning sensor that measures the horizontal and vertical positions of the autonomous mobile object, an altitude measurement sensor that measures the vertical position of the autonomous mobile object, and obstacles around the autonomous mobile object. An obstacle measurement sensor for measuring a distance from the vehicle, a distance measurement sensor for measuring a relative position between the autonomous mobile body and an object around the autonomous mobile body, azimuth information obtained from the IMU unit, the satellite positioning sensor and The route from the self-position information including the horizontal position and the vertical position acquired from the altitude measurement sensor to the photographing start position indicated by the position information of the photographing location is obtained based on the target facility information. A path forming unit that generates so as to avoid collision with an object, and an image analyzing unit that analyzes an image captured by the camera, wherein a plurality of images captured by the camera are connected to each other, and An image analysis unit that synthesizes the images of the images, a path following instruction for moving to the shooting start position along the path, and the autonomous mobile body and the obstacle around the autonomous mobile body by the obstacle measurement sensor. command separation distance for a plurality of flight control comprising a command and that the autonomous moving body when it becomes shorter than the predetermined distance is away from the autonomous moving body around obstacles, and the image A plurality of shootings including a command generated by the analyzing unit to move to a shooting position of an abnormal location detected from an image shot by the camera, and a command to correct a relative orientation between the autonomous moving body and the shooting target. of Directive on control, and adjust a position control unit for controlling the position and attitude of the autonomous moving body on the basis of the integrated command which integrates plural directive in response to the flight conditions, the focus of the camera based on the relative position And a camera control unit for photographing the photographing object.
Another aspect is an autonomous mobile imaging control system that controls an autonomous mobile body equipped with a camera, and stores equipment information including target equipment information including shape information of an imaging target and position information of an imaging location. Unit, including a sensor for measuring the attitude and orientation of the autonomous mobile body, a sensor unit having a plurality of sensors, and the location information of the shooting location from the self-position information of the autonomous mobile body obtained from the sensor unit indicates A route to a shooting start position, a route forming unit that generates a collision with the shooting target based on the target facility information to avoid collision, an image analysis unit that analyzes an image shot by the camera, a velocity vector which is a path following command to move to the imaging start position along the path, distance between the autonomous moving body and the autonomous moving body around the obstacle by the sensor portion is of a predetermined Velocity vectors for a plurality of flight control comprising a velocity vector which is a command to the autonomous moving body when it becomes shorter than the interval distance away from the autonomous moving body around obstacles, and generated by the image analysis unit A speed vector that is a command to move to an imaging position of an abnormal location detected from an image captured by the camera, and a speed vector that is a command to correct the relative orientation between the autonomous mobile object and the imaging target. the photographing control of the velocity vectors for a position control unit for controlling the position and attitude 6 of the autonomous moving body based on a plurality of integrated commands the velocity vector was synthesized in accordance with the flight conditions, the camera based on the relative distance A camera control unit that adjusts the focus to photograph the photographing object.

In order to solve the above-described problems, in one embodiment of the present invention, an autonomous mobile body equipped with a camera and having a drive unit and capable of autonomous movement, including a transmission line and an overhead ground line of a power facility. In an autonomous mobile body that shoots the transmission line / overhead ground line while moving along, the target equipment information including the shape information of the shooting target including the transmission line / overhead ground line and the position information of the shooting location is stored in advance. An IMU (Inertial Measurement Unit) unit for measuring the attitude and orientation of the autonomous mobile unit, a satellite positioning sensor for positioning the horizontal and vertical positions of the autonomous mobile unit, and An altitude measurement sensor that measures the vertical position of the autonomous mobile body, an obstacle measurement sensor that measures the distance between obstacles around the autonomous mobile body, A distance measuring sensor for measuring the relative distance and relative angle between the transmission lines and ground wire around the autonomous moving body, an electromagnetic field sensor for measuring the electromagnetic field from the transmission lines and ground wire, captured by the camera An image analysis unit for analyzing the captured image, the image analysis unit for combining a plurality of images taken by the camera to synthesize an image of the entire transmission line / overhead ground wire, and a plurality of commands according to the flight situation A control unit for controlling a position and an attitude of the autonomous mobile body based on an integrated command obtained by integrating: a) azimuth information obtained from the IMU unit; and the horizontal information obtained from the satellite positioning sensor and the altitude measurement sensor. generate a route from the current position information including the direction of the position and the vertical position to the imaging start position indicated by the position information of the transmission lines and ground wire movement, until the imaging start position along said path A path following instruction because, if the distance between the autonomous moving body and the autonomous moving body around the obstacle or the transmission lines and ground wire by the obstacle measurement sensor becomes shorter than the predetermined distance An integrated command in which the autonomous mobile body integrates an obstacle avoidance around the autonomous mobile body or an approach avoidance control command to move away from the transmission line / overhead ground wire , b) the self-positioning information by the satellite positioning sensor, Based on the separation distance measured from the distance measurement sensor, the feature approach control command to approach the imaging start position of the transmission line / overhead ground line, and when a value of a predetermined value or more is measured by the electromagnetic field sensor, integrated command which integrates the electromagnetic field around control command away from power lines, ground wire, and, c) irrespective of the current position information by the satellite positioning sensor, distance to be measured from the distance measuring sensor Based on a close follow-up control command to approach the transmission lines and ground wire, produced by the image analysis unit, from the image captured by the camera, the relative orientation of the shooting target and the autonomous moving body a command correction to, among command to move the photographing position of the anomaly has been detected from an image captured by the camera, system that controls the position and attitude of the autonomous moving body on the basis of the integrated command which integrates instruction multiple A camera control unit that adjusts the focus of the camera based on the relative position and shoots the shooting target.

Further, in another aspect, prior Symbol control section, the case of measuring the value of a predetermined value or more in the electromagnetic field sensor, the relevant equipment the position of the nearest field generation source to the own position in which the current position information indicates An electromagnetic field avoidance control command that specifies from information and moves away from the magnetic field generation source is further generated.

In another aspect, the camera control unit adjusts a zoom magnification of the camera by following the autonomous moving object according to the electromagnetic field avoidance control command when the electromagnetic field sensor measures a value equal to or greater than a predetermined value. and, wherein the image analysis unit, the imaging control output to adjust the tracking speed is outputted to the control unit, the camera control unit, continued shooting at a position where the value of the electromagnetic field sensor is equal to or less than a predetermined value vinegar Rukoto and features.

In another aspect, the image analysis unit calculates an inclination angle of the image of the transmission line / overhead ground line with respect to each side of a frame, and based on the inclination angle, the autonomous mobile body and the transmission line / overhead ground line. And outputting a photographing control output for correcting the relative azimuth to the control unit .

Further, in another aspect, each of the instruction pre-Symbol control section generates is the velocity vector, the integrated command is characterized in that it is obtained by combining the velocity vector.

Claims (16)

An autonomous mobile shooting control system for controlling an autonomous mobile body equipped with a camera,
Equipment information storage unit that stores target equipment information including the shape information of the imaging target and the position information of the imaging location,
An IMU unit for measuring the attitude and orientation of the autonomous mobile body,
A satellite positioning unit that measures the horizontal and vertical positions of the autonomous mobile body,
An altitude measurement sensor that measures the vertical position of the autonomous mobile body,
An obstacle measurement sensor that measures a separation distance from an obstacle around the autonomous mobile body,
An electromagnetic field sensor that measures an electromagnetic field around the autonomous moving body,
A distance measuring sensor that measures a relative distance and a relative angle between the autonomous moving body and the imaging object around the autonomous moving body,
A camera control unit that adjusts the focus of the camera based on the relative distance and shoots the shooting target,
An image analysis unit that analyzes an image captured by the camera, an image analysis unit that combines a plurality of images captured by the camera and synthesizes an image of the entire imaging target,
An analysis recording unit that records an analysis result of an image captured by the camera in the image analysis unit,
A position control unit that controls the position of the autonomous mobile body based on the information of the sensors and the units;
A communication unit that transmits the information of each of the sensors and the respective units by dual-frequency and non-line-of-sight wireless communication,
An autonomous mobile photography control system comprising:   The shooting start position indicated by the position information of the shooting location from the self-position information including the azimuth information obtained from the IMU unit, the horizontal position and the vertical position obtained from the satellite positioning unit and the altitude measurement sensor. The autonomous mobile imaging control system according to claim 1, further comprising: a route forming unit configured to generate a route to the imaging target based on the target facility information so as to avoid collision with the imaging target. .   The position control unit is configured to determine a route following command for moving to the shooting start position along the route, and determine a distance between the autonomous moving body and an obstacle around the autonomous moving body by the obstacle measurement sensor. When the distance becomes shorter than the separation distance, the autonomous mobile generates a command to move away from an obstacle around the autonomous mobile, and controls the position of the autonomous mobile based on an integrated command obtained by integrating a plurality of the commands. The autonomous mobile photography control system according to claim 2, wherein The position control unit, from the own position information and the target equipment information and the relative distance and the relative angle, to specify the shooting target location and shooting position of the shooting target,
4. The autonomous mobile photography control system according to claim 3, wherein the camera control unit photographs the photography target at the photography position. 5.   When the electromagnetic field sensor measures a value equal to or greater than a predetermined value, the position control unit specifies the position of the electromagnetic field generation source closest to the own position indicated by the own position information from the target facility information, and The autonomous mobile radiographing control system according to claim 3 or 4, wherein a command for moving away from the source is further generated.   6. The camera control unit according to claim 5, wherein when the electromagnetic field sensor measures a value equal to or greater than a predetermined value, the camera controller continues or stops shooting at a position where the value of the electromagnetic field sensor is equal to or less than a predetermined value. An autonomous mobile photography control system as described.   The position control unit further issues a command to move from the photographing start position to a photographing end position indicated by the position information of the photographing place while following the relative distance and the relative angle with the photographing object so as to keep the relative distance and the relative angle constant. The autonomous mobile photography control system according to any one of claims 4 to 6, wherein the system is generated.   The method according to claim 7, wherein the target facility information, the image, the photographing start position information, and the photographing end position information are used to confirm and record that the entire photographing object has been photographed without falling off. Autonomous mobile photography control system.   The image analysis unit, based on the target equipment information, the image, the shooting start position information of the image, the shooting end position information, based on the past shooting position information of the shooting target previously shot, The autonomous mobile photography control system according to claim 8, wherein an image associated with the past photography location information close to the location information of the photography location is extracted and compared to detect a specific difference indicating an abnormality. .   The camera control unit is configured to, when photographing the specified photographing location of the photographing target based on the target facility information, determine an ISO sensitivity, an aperture, and a magnification based on a color, a material, and an image captured immediately before the target facility information. The autonomous mobile photography control system according to any one of claims 4 to 9, further comprising adjusting a shutter speed and a shutter speed, and recording the adjustment values of the cameras. The image analysis unit, when detecting an abnormality, outputs an abnormality detection signal including position information of an abnormal part to the position control unit and the camera control unit,
The position control unit generates a command to move to the imaging position of the abnormal location,
The camera control unit, at the imaging position of the abnormal location, captures the abnormal location at a higher magnification, the imaging start time, the imaging end time, the imaging start position, the imaging end position, the imaging end time at a higher magnification of the abnormal location The autonomous mobile radiographing control system according to claim 10, wherein an adjustment value of (i) and an image of the abnormal part are recorded.   The image analysis unit calculates an angle of inclination of each side of a frame of an image of the object to be photographed, and corrects a relative orientation between the autonomous mobile body and the object to be photographed based on the angle of inclination. Item 12. The autonomous mobile imaging control system according to any one of Items 8 to 11.   The path forming unit, when there is a plurality of the imaging target, when one imaging of the imaging target is completed, the information of the azimuth obtained from the IMU unit and the information obtained from the satellite positioning unit and the altitude measurement sensor A route to a shooting start position indicated by the position information of the shooting location of another shooting target whose shooting is not completed based on the own location information is avoided based on the target facility information to avoid a collision with the shooting target. The autonomous mobile photography control system according to any one of claims 4 to 12, wherein the autonomous mobile photography control system is generated as described above.   The autonomous device according to any one of claims 3 to 13, wherein each of the commands generated by the position control unit is a speed vector, and the integrated command is a combination of the speed vectors. Mobile shooting control system.   14. The autonomous mobile photography control system according to claim 1, wherein the altitude measurement sensor includes at least one of a barometric pressure sensor and a LIDAR sensor. An autonomous mobile body having a driving unit and capable of autonomously moving,
A camera mounted on the autonomous mobile body,
An IMU unit for measuring the attitude and orientation of the autonomous mobile body,
A satellite positioning sensor for positioning the horizontal and vertical positions of the autonomous mobile body,
An altitude measurement sensor that measures the vertical position of the autonomous mobile body,
An obstacle measurement sensor that measures a distance from an obstacle around the autonomous mobile body,
An electromagnetic field sensor that measures an electromagnetic field around the autonomous moving body,
A distance measuring sensor that measures a relative position and a relative angle between the autonomous moving body and a shooting target around the autonomous moving body,
A camera control unit that adjusts the focus of the camera based on the relative position and shoots the shooting target.
An image analysis unit that analyzes an image captured by the camera, an image analysis unit that combines a plurality of images captured by the camera and synthesizes an image of the entire imaging target,
An analysis recording unit that records an analysis result of an image captured by the camera in the image analysis unit,
A position control unit that controls the position of the autonomous mobile body based on the information of the sensors and the units;
A communication module that transmits the information of each of the sensors and the respective units by dual-frequency and non-line-of-sight wireless communication,
An autonomous mobile body comprising:

Images