WO2020195876A1 - 移動体およびその制御方法、並びにプログラム - Google Patents
移動体およびその制御方法、並びにプログラム Download PDFInfo
- Publication number
- WO2020195876A1 WO2020195876A1 PCT/JP2020/010740 JP2020010740W WO2020195876A1 WO 2020195876 A1 WO2020195876 A1 WO 2020195876A1 JP 2020010740 W JP2020010740 W JP 2020010740W WO 2020195876 A1 WO2020195876 A1 WO 2020195876A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- line segment
- unit
- control unit
- moving
- captured
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
- G05D1/106—Change initiated in response to external conditions, e.g. avoidance of elevated terrain or of no-fly zones
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U10/00—Type of UAV
- B64U10/10—Rotorcrafts
- B64U10/13—Flying platforms
- B64U10/14—Flying platforms with four distinct rotor axes, e.g. quadcopters
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/0202—Control of position or course in two dimensions specially adapted to aircraft
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U20/00—Constructional aspects of UAVs
- B64U20/80—Arrangement of on-board electronics, e.g. avionics systems or wiring
- B64U20/87—Mounting of imaging devices, e.g. mounting of gimbals
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U2101/00—UAVs specially adapted for particular uses or applications
- B64U2101/30—UAVs specially adapted for particular uses or applications for imaging, photography or videography
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U2201/00—UAVs characterised by their flight controls
- B64U2201/10—UAVs characterised by their flight controls autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
Definitions
- the present technology relates to a moving body and its control method and a program, and in particular, a moving body and its control method and a program capable of accurately detecting and avoiding a linear object which may be an obstacle in movement. Regarding.
- a moving object that flies autonomously such as a drone, recognizes the position of an object around the aircraft from an image captured by a stereo camera, and moves autonomously while avoiding obstacles.
- the baseline is a line segment connecting the optical centers of the two cameras constituting the stereo camera.
- Patent Document 1 discloses a technique of flying while maintaining a certain distance from an electric wire based on the magnitude of the current flowing through the electric wire and imaging the electric wire as an unmanned aerial vehicle that detects and flies the electric wire.
- Patent Document 1 can only handle electric wires during power transmission, and can only fly along the electric wires.
- This technology was made in view of such a situation, and makes it possible to accurately detect and avoid linear objects that can be obstacles in movement.
- the moving body on one side of the present technology has a line segment detection unit that detects a line segment in an image captured by at least one of the stereo cameras, and the stereo camera fixed in a direction orthogonal to the line segment. It is equipped with a control unit that moves the aircraft.
- the moving body detects a line segment in an captured image captured by at least one of the stereo cameras, and the stereo camera is oriented in a direction orthogonal to the line segment. Moves the fixed aircraft.
- the program of one aspect of the present technology is to move the computer to a line segment detection unit that detects a line segment in a captured image captured by at least one of the stereo cameras, and the stereo camera in a direction orthogonal to the line segment. Is for functioning as a control unit for moving the fixed aircraft.
- a line segment in an captured image captured by at least one of the stereo cameras is detected, and the aircraft to which the stereo camera is fixed is moved in a direction orthogonal to the line segment. ..
- the program can be provided by transmitting via a transmission medium or by recording on a recording medium.
- FIG. 1 is a plan view of a drone which is a moving body to which the present technology is applied.
- Drone 1 in FIG. 1 is a quad-type flying moving object having four rotors (rotor blades) 11.
- the drone 1 is a quad-type flight moving body having four rotors 11, but the drone 1 is not limited to this, and may be, for example, a multicopter having six or eight rotors 11. ..
- a plurality of cameras 13 are arranged on the main body 12 of the drone 1. More specifically, eight cameras 13A to 13H are arranged on the side surface of the outer peripheral portion of the main body portion 12, and one camera 14 is arranged on the bottom surface of the main body portion 12.
- Each of the cameras 13A to 13H provided on the side surface captures a subject captured within a predetermined viewing angle in the vertical and horizontal directions with the horizontal direction of the drone 1 as the imaging center.
- the camera 14 provided on the bottom surface captures a subject captured within a predetermined viewing angle in the vertical and horizontal directions with the lower portion facing the ground as the imaging center.
- each of the cameras 13A to 13H is not particularly distinguished, it is simply referred to as a camera 13.
- the number and arrangement of the cameras 13 are not limited to the example of FIG. 1, and can be arbitrarily determined.
- a stereo camera is composed of a pair of two cameras 13 arranged so that the optical axes are parallel to each other among the eight cameras 13 provided on the side surface.
- the cameras 13A and 13B form a stereo camera
- the cameras 13C and 13D form a stereo camera
- the cameras 13E and 13F form a stereo camera
- the cameras 13G and 13H form a stereo camera. .. From the two captured images (paired captured images) captured by the two cameras 13 that make up the stereo camera, the object existing around the drone 1 and the distance to that object are recognized by the principle of triangulation. Will be done.
- the stereo camera composed of the cameras 13A and 13B captures the traveling direction, detects the situation of obstacles in the traveling direction, and the like.
- the cameras 13C to 13H capture an image for detecting the situation of the entire surroundings of the drone 1.
- the drone 1 recognizes an object existing as an obstacle in the traveling direction based on two captured images captured by the stereo camera, autonomously flies while avoiding the obstacle, and moves to the destination.
- the destination is received by wireless communication or the like from a remote terminal (not shown).
- the baseline is a line segment connecting the optical centers of the two cameras 13 constituting the stereo camera.
- FIG. 2 shows a state in which the electric wire 15 is imaged with a stereo camera.
- the captured image L1 (hereinafter referred to as the left camera captured image L1) is an image captured by the left camera, which is one of the stereo cameras, and the captured image R1 (hereinafter referred to as the right camera captured image R1) is It is an image taken by the other right camera.
- the drone 1 is equipped with a control that enables detection of an object with little change in texture in the direction of a straight line parallel to the baseline, such as the electric wire 15 in FIG.
- the electric wire 15 shown in FIG. 2 when the aircraft is rotated and imaged with a stereo camera, the electric wire 15 is as shown in the left camera image L1'and the right camera image R1' in FIG. Can acquire an image tilted by a predetermined angle ⁇ .
- the electric wire 15 cannot be accurately detected because the image does not have a small change in texture in the direction of the straight line parallel to the baseline. it can.
- FIG. 3 is a block diagram relating to flight control of the drone 1.
- the drone 1 includes at least a controller 31, an RTK-GPS receiving unit 32, and an airframe driving unit 33.
- the controller 31 determines the current location and surrounding conditions of the drone 1 based on the images captured by the cameras 13 and 14 and the position information, speed information, time information, and the like detected by the RTK-GPS receiver 32. Recognize and control the flight (movement) of the drone 1.
- the controller 31 is composed of, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a microprocessor, or the like, and executes a program stored in a storage unit such as a ROM. Then, the control of the drone 1 is executed.
- the RTK-GPS receiver 32 receives both radio waves from GPS satellites, which are one of the GNSS (Global Navigation Satellite System), and radio waves from reference stations installed on the ground, and has an accuracy of several centimeters. Detects (positions) its current position with.
- the GNSS is not limited to GPS, and positioning signals of positioning satellites such as GLONASS (US), BeiDou (China), Galileo (EU), and quasi-zenith satellite Michibiki (Japan) may be used or used in combination.
- the RTK-GPS receiving unit 32 supplies the position information, speed information, time information, etc. of the drone 1 to the self-position estimation unit 43.
- the machine body drive unit 33 includes four rotors 11 and a motor 51 that drives each rotor 11.
- the machine body drive unit 33 moves the machine body or changes the posture of the machine body by changing the rotation speeds of the four rotors 11 according to the control from the controller 31.
- the controller 31 has an object recognition unit 41, a stereo ranging unit 42, a self-position estimation unit 43, a line segment detection unit 44, a rotation angle calculation unit 45, an occupied grid construction unit 46, and an action control unit 47.
- the object recognition unit 41 advances based on the captured image captured by one of the cameras 13 (monocular camera) of the two stereo cameras (for example, cameras 13A and 13B) that capture the traveling direction among the plurality of cameras 13. Detects (recognizes) an object in the direction. Among the objects included in the captured image, the object recognition unit 41 detects an elongated linear object such as an electric wire, an antenna, and a utility pole, and provides information (object) for identifying the object such as the position and size of the detected object. Specific information) is supplied to the line segment detection unit 44.
- a known method can be adopted as an algorithm for detecting an arbitrary object from an image, but for detecting an elongated linear object such as an electric wire or an antenna, for example, "Gubbi, Jayavardhana, Ashley Varghese, and P. Balamuralidhar”
- Machine Vision Applications (MVA) 2017 Fifteenth IAPR International Conference on. IEEE, 2017 can be adopted.
- the object recognition unit 41 detects an elongated linear object based on the captured image captured by the monocular camera, but based on the captured image captured by the stereo camera. , An elongated linear object may be detected.
- the stereo ranging unit 42 measures the distance with a stereo camera. Specifically, the stereo ranging unit 42 generates a parallax map from two captured images (paired captured images) captured by two cameras 13 arranged so that the optical axes are parallel, and occupies the parallax map. It is supplied to the lattice construction unit 46.
- the parallax map is an image in which a parallax amount corresponding to a distance in the depth direction of an object reflected in the captured image is added to one captured image of a pair of captured images in pixel units of the captured image.
- the parallax map is an image showing depth information corresponding to the captured image, and is also called a depth image.
- the self-position estimation unit 43 determines the current self-position of the drone 1 based on the position information and speed information of the drone 1 supplied from the RTK-GPS receiving unit 32 and the captured images supplied from the plurality of cameras 13 and 14. And estimate the posture. For example, when the self-position estimation unit 43 can receive radio waves from a positioning satellite or a base station, the self-position estimation unit 43 estimates the self-position based on the position information measured by the RTK-GPS reception unit 32, indoors, a tunnel, or the like. In places where radio waves cannot be received, feature points of captured images supplied from a plurality of cameras 13 and 14 are detected, and the self-position and orientation are estimated by Visual-SLAM (Simultaneous Localization and Mapping). The self-position estimation unit 43 supplies the detected self-position and posture to the occupied grid construction unit 46 and the behavior control unit 47.
- Visual-SLAM Simultaneous Localization and Mapping
- the drone 1 may further include inertial measurement sensors such as a gyro sensor, an acceleration sensor, a magnetic sensor, and a pressure sensor.
- the self-position estimation unit 43 also uses the sensor information to self-use. The position and orientation can be estimated with high accuracy.
- the line segment detection unit 44 converts the elongated linear object detected by the object recognition unit 41 into a line segment using the Hough transform, and detects the elongated linear object as a line segment.
- the detected line segment information is supplied to the rotation angle calculation unit 45 and the action control unit 47.
- the rotation angle calculation unit 45 calculates the angle ⁇ of the line segment detected by the line segment detection unit 44, and supplies the calculation result to the action control unit 47. For example, assuming that the line segment detected by the line segment detection unit 44 is the electric wire 15 in FIG. 2, the rotation angle calculation unit 45 calculates the rotation angle ⁇ on the image of the electric wire 15 in FIG. 2 and controls the action. It is supplied to the unit 47.
- the occupancy grid construction unit 46 folds the result of the parallax map supplied from the stereo ranging unit 42 in the time direction to show the presence or absence of obstacles in the three-dimensional space around the drone 1 (occupancy grid). ) Is constructed.
- the occupied grid construction unit 46 is also supplied with the position (self-position) and attitude of the drone 1 estimated by the self-position estimation unit 43.
- P (m p) of the formula (1) represents the prior probability in the captured image of the index p
- z p, g p) represents the noise characteristics of the sensor, if the sensor is a stereo camera For example, it corresponds to an error due to distance resolution.
- P (M) represents the prior probability of the three-dimensional space map M.
- a in FIG. 4 shows an example of a parallax map supplied from the stereo ranging unit 42.
- the distance information of each pixel is represented by an 8-bit gray value, and the brighter the brightness (white), the closer the distance.
- FIG. 4 shows an example of an occupied grid map constructed by the occupied grid construction unit 46.
- the action control unit 47 uses the self-position and posture supplied from the self-position estimation unit 43 and the occupancy grid map constructed by the occupancy grid construction unit 46 to target the drone 1 from the current position.
- a movement route to the ground is set, and each motor 51 of the machine body drive unit 33 is controlled according to the set movement route.
- the destination is transmitted from the remote terminal, received by a communication unit (not shown), and input to the behavior control unit 47.
- the behavior control unit 47 is a motor that rotates the machine body in the yaw direction when a line segment of an elongated linear obstacle such as the electric wire 15 in FIG. 2 is supplied from the line segment detection unit 44. 51 is controlled.
- the action control unit 47 rotates the machine body in the yaw direction until the angle ⁇ of the line segment supplied from the rotation angle calculation unit 45 becomes the angle R.
- the angle R is determined as follows. As shown in FIG. 5, the horizontal resolution of the camera 13 is width [pixel], and the horizontal and vertical pixels of the block when performing block matching for searching the corresponding points of the two captured images by the stereo camera. Assuming that the number is B [pixel], the angle R is calculated by the following equation (2).
- the angle R is an angle at which a predetermined object in the center of the image moves in the vertical direction by an amount corresponding to the block size of block matching in the captured image.
- the self-position estimation unit 43 performs self-position estimation. That is, the current position and attitude of the drone 1 are estimated (determined) based on the position information and speed information from the RTK-GPS receiver 32 and the captured images supplied from the plurality of cameras 13 and the cameras 14, and the behavior is controlled. It is supplied to the unit 47.
- step S12 the behavior control unit 47 determines whether or not the vehicle has arrived at the destination based on the current position of the drone 1. If it is determined in step S12 that the drone 1 has arrived at the destination, the flight control process ends.
- step S12 determines whether the drone 1 has not yet arrived at the destination. If it is determined in step S12 that the drone 1 has not yet arrived at the destination, the process proceeds to step S13, and the action control unit 47 is a passing point in the movement route to the final destination.
- a local destination within a predetermined distance from the current location hereinafter referred to as a local destination
- a movement route to the local destination is determined, and the movement is started. If the final destination exists within a predetermined distance from the current location, that destination becomes the local destination.
- the movement route to the local destination is determined by inputting the three-dimensional occupied grid map from the occupied grid construction unit 46 and calculating the cost when the drone 1 passes through a certain space.
- the cost represents the difficulty of passing the drone 1, and the closer to the obstacle, the higher the cost is set.
- semantic information such as electric wires and buildings is attached to the occupied grid map
- the cost can be changed according to the semantic information. For example, a high cost is set near an area recognized as an electric wire or a moving object such as a person. As a result, a movement route is determined so that the drone 1 takes a distance from an obstacle with a high cost.
- a known search algorithm such as A * algorithm, D * algorithm, RRT (Rapidly Exploring Random Tree), DWA (Dynamic Window Approach), etc. can be used for the search of the movement route.
- step S14 the RTK-GPS receiving unit 32 acquires GPS position information. More specifically, the RTK-GPS receiving unit 32 detects (positions) its own position by receiving both radio waves from GPS satellites and radio waves from a reference station installed on the ground.
- a plurality of cameras 13 perform imaging.
- two adjacent cameras 13 having the traveling direction as the imaging direction take an image in the traveling direction, and another camera 13 takes an image of the surroundings other than the traveling direction.
- steps S14 and S15 are processes that are continuously executed during the flight, and the GPS position information and the stereo camera image are sequentially updated according to the movement of the drone 1. Then, the occupied grid map by the occupied grid construction unit 46 is updated (reconstructed).
- step S16 the behavior control unit 47 determines whether an obstacle is detected in the traveling direction.
- step S16 If it is determined in step S16 that an obstacle is detected in the traveling direction, the process proceeds to step S17, and the drone 1 executes the obstacle avoidance action process.
- the details of the obstacle avoidance action processing will be described later with reference to FIG. 7.
- step S16 determines whether obstacle is detected in the traveling direction. If it is determined in step S16 that no obstacle is detected in the traveling direction, the process proceeds to step S18, and the drone 1 moves along the movement route set in step S13.
- step S17 or step S18 the process proceeds to step S19, and the action control unit 47 determines whether or not the self-position has arrived at the local destination.
- step S19 If it is determined in step S19 that the self-position has not yet arrived at the local destination, the process returns to step S14, and steps S14 to S19 described above are repeated.
- step S19 determines whether or not the self-position has arrived at the local destination. If it is determined in step S19 that the self-position has arrived at the local destination, the process returns to step S12, and it is determined again whether or not the user has arrived at the destination. The processes of steps S13 to S19 are repeated until it is determined that the aircraft has arrived at the destination in step S12, and when it is determined that the aircraft has arrived at the destination, the flight control process ends.
- step S17 which is executed when it is determined that an obstacle is detected in the traveling direction in step S16 of FIG. 6, will be described.
- the obstacle avoidance action process of FIG. 7 describes a case where an elongated linear object that is difficult to recognize as an object, such as the above-mentioned electric wire, antenna, and utility pole, is avoided among the obstacles.
- step S41 the action control unit 47 controls each motor 51 of the machine body drive unit 33 to decelerate the machine body.
- the drone 1 is moving in a forward leaning posture in the traveling direction, although the speed is slower than before decelerating.
- step S42 the object recognition unit 41 recognizes an object as an obstacle from the captured image captured in the traveling direction, and supplies the object identification information for identifying the recognized object to the line segment detection unit 44.
- step S43 the line segment detection unit 44 performs line segment conversion that converts the elongated linear object detected by the object recognition unit 41 into a line segment using the Hough transform. As a result, an elongated linear object is detected as a line segment.
- the detected line segment information is supplied to the rotation angle calculation unit 45 and the action control unit 47.
- the object recognition in step S42 and the line segment detection in step S43 are continuously executed until the obstacle is avoided, in other words, until the processing in step S52 is started.
- step S44 the action control unit 47 controls the motor 51 and moves the aircraft in the direction dir orthogonal to the detected line segment LL as shown in FIG.
- the direction dir orthogonal to the detected line segment LL as shown in FIG. 8
- the upward direction is selected instead of the ground direction, but it is also controlled by the occupied grid map around it.
- step S44 the drone 1 moves in the direction dir orthogonal to the detected line segment LL for a certain period of time or a certain distance.
- step S45 the action control unit 47 determines whether the obstacle can be avoided based on the detection result from the line segment detection unit 44, that is, whether the line segment corresponding to the electric wire or the like cannot be seen from the line of sight. To do.
- step S45 If it is determined in step S45 that the obstacle can be avoided, the process proceeds to step S52 described later.
- step S45 determines whether the obstacle could not be avoided. If it is determined in step S45 that the obstacle could not be avoided, the process proceeds to step S46, and the action control unit 47 controls the motor 51 to rotate the aircraft in the yaw direction.
- step S47 the rotation angle calculation unit 45 calculates the angle ⁇ of the line segment detected by the line segment detection unit 44, and supplies the calculation result to the action control unit 47.
- step S48 the action control unit 47 determines whether the angle ⁇ of the line segment is the angle R of the equation (2).
- step S48 If it is determined in step S48 that the angle ⁇ of the line segment is not the angle R, the process proceeds to step S46, and the processes of steps S46 to S48 are repeated. That is, the drone 1 rotates the airframe in the yaw direction until the angle ⁇ of the line segment becomes the angle R. As a result, as described with reference to FIG. 2, an elongated object such as an electric wire can be accurately detected by two captured images of a stereo camera.
- step S48 when it is determined in step S48 that the angle ⁇ of the line segment is the angle R, the process proceeds to step S49, and the stereo ranging unit 42 is a stereo camera in the traveling direction with the aircraft rotated.
- a parallax map is generated from the two captured images and supplied to the occupied grid construction unit 46.
- step S50 the occupied grid construction unit 46 indicates the presence or absence of obstacles in the three-dimensional space around the drone 1 by adding the result of the parallax map supplied from the stereo ranging unit 42 to the occupied grid map. Update (reconstruct) the occupied grid map.
- step S51 the action control unit 47 controls each motor 51 based on the updated occupied grid map to take an obstacle avoidance action.
- the drone 1 is moved in a direction of avoiding an elongated object such as an electric wire.
- the process proceeds to step S52.
- the local destination initially set by the obstacle avoidance action is performed. It will deviate from the movement route to.
- step S52 the action control unit 47 resets the local destination, determines the movement route to the local destination, controls each motor 51, and starts the movement.
- the method of resetting the local destination is the same as the setting of the local destination in step S13.
- step S17 the process proceeds from step S17 to step S19 of FIG. 6, and it is determined whether or not the self-position has arrived at the local destination. After that, it is as described with reference to FIG.
- an elongated linear object which is difficult to accurately recognize with a stereo camera, can be accurately recognized by rotating the aircraft in the yaw direction. You can avoid the object and fly to the destination accurately.
- it is determined whether the angle ⁇ of the line segment is the target angle R, and the machine is controlled to rotate in the yaw direction until the target angle R is reached. It may be a simple control that rotates the machine body in the yaw direction by a predetermined predetermined angle without determining whether or not.
- the drone flies while keeping a certain distance from the power line. Since the electric wire is bent and sways by the wind, it is difficult to grasp the detailed three-dimensional structure in advance. In addition, the altitude of the electric wire is high, and it is an environment in which the stereo camera tends to cause a deviation in self-position recognition.
- a marker or the like for recognizing the self-position may be attached to a fixed facility such as a steel tower, and the self-position recognition by the marker may be performed as an auxiliary.
- the camera 13 that captures the traveling direction is fixed to the main body 12 of the drone 1, and when the captured image is controlled to be tilted by the angle R, the entire drone 1 is rotated.
- a camera such as an FPV (First Person View) camera that can rotate the camera itself with respect to the main body 12 of the drone 1 can be used. In this case, only the camera is rotated, and the drone 1 does not need to be rotated.
- FPV First Person View
- the plurality of cameras 13 are arranged in the horizontal direction, but for example, a part (at least one) of the plurality of cameras 13 is arranged in the vertical direction.
- two vertically arranged cameras 13 may be used as stereo cameras. Since the stereo cameras arranged in the horizontal direction and the stereo cameras arranged in the vertical direction have different textures in the horizontal direction (horizontal direction) in the image, the detection and avoidance of the elongated object described above can be performed without rotating the camera 13. Is possible.
- the movement control of the present disclosure can be applied to the movement control of vehicles such as ordinary automobiles and trucks that are automatically driven.
- it is effective for recognizing line segments parallel to the baseline of a stereo camera such as guardrails and road signs.
- the series of flight control processes described above can be executed by hardware or software.
- the programs constituting the software are installed on the computer.
- the computer includes a microcomputer embedded in dedicated hardware and, for example, a general-purpose personal computer capable of executing various functions by installing various programs.
- FIG. 9 is a block diagram showing a configuration example of computer hardware that executes the above-mentioned series of flight control processes programmatically.
- a CPU Central Processing Unit
- ROM ReadOnly Memory
- RAM RandomAccessMemory
- An input / output interface 105 is further connected to the bus 104.
- An input unit 106, an output unit 107, a storage unit 108, a communication unit 109, and a drive 110 are connected to the input / output interface 105.
- the input unit 106 includes a keyboard, a mouse, a microphone, a touch panel, an input terminal, and the like.
- the output unit 107 includes a display, a speaker, an output terminal, and the like.
- the storage unit 108 includes a hard disk, a RAM disk, a non-volatile memory, and the like.
- the communication unit 109 is composed of a network interface or the like that performs wired communication or wireless communication.
- the communication unit 109 includes, for example, the Internet, a public telephone line network, a wide area communication network for wireless mobiles such as so-called 4G lines and 5G lines, WAN (Wide Area Network), LAN (Local Area Network), and Bluetooth (registered trademark). Communicate in accordance with the standard.
- the communication unit 109 complies with standards such as short-range wireless communication such as NFC (Near Field Communication), infrared communication, HDMI (registered trademark) (High-Definition Multimedia Interface), and USB (Universal Serial Bus). Wired communication and communication via a communication network or communication path of any communication standard are performed.
- the drive 110 drives a removable recording medium 111 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.
- the CPU 101 loads the program stored in the storage unit 108 into the RAM 103 via the input / output interface 105 and the bus 104 and executes the above-described series. Flight control processing is performed.
- the RAM 103 also appropriately stores data and the like necessary for the CPU 101 to execute various processes.
- the program executed by the computer can be recorded and provided on the removable recording medium 111 as a package medium or the like, for example. Programs can also be provided via wired or wireless transmission media such as local area networks, the Internet, and digital satellite broadcasting. In addition, the program can be pre-installed in the ROM 102 or the storage unit 108.
- this technology can have a cloud computing configuration in which one function is shared by a plurality of devices via a network and processed jointly.
- each step described in the above flowchart can be executed by one device or can be shared and executed by a plurality of devices.
- one step includes a plurality of processes
- the plurality of processes included in the one step can be executed by one device or shared by a plurality of devices.
- the present technology can have the following configurations.
- a line segment detector that detects a line segment in a captured image captured by at least one of the stereo cameras,
- a moving body including a control unit for moving the body to which the stereo camera is fixed in a direction orthogonal to the line segment.
- An object recognition unit that recognizes an object in the captured image is further provided.
- the moving body according to (2), wherein the line segment detecting unit detects the line segment by converting the object in the captured image into the line segment.
- a stereo ranging unit that generates a parallax map from the captured image captured by the stereo camera when the rotation angle of the line segment is the predetermined angle. It is further equipped with an occupied grid map construction unit that constructs an occupied grid map from the parallax map.
- the control unit After moving the aircraft, the control unit resets the local destination, which is the local destination of the aircraft, and moves the aircraft to the local destination after the resetting (1) to The moving body according to any one of (7).
- the moving body Detects line segments in captured images captured by at least one of the stereo cameras and A method for controlling a moving body in which the body to which the stereo camera is fixed is moved in a direction orthogonal to the line segment.
- Computer A line segment detector that detects a line segment in a captured image captured by at least one of the stereo cameras, A program for making the stereo camera function as a control unit for moving a fixed body in a direction orthogonal to the line segment.
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Mechanical Engineering (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
- Image Analysis (AREA)
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US17/438,942 US20220153411A1 (en) | 2019-03-25 | 2020-03-12 | Moving body, control method thereof, and program |
| JP2021509022A JP7476882B2 (ja) | 2019-03-25 | 2020-03-12 | 移動体およびその制御方法、並びにプログラム |
| CN202080021996.8A CN113678082B (zh) | 2019-03-25 | 2020-03-12 | 移动体、移动体的控制方法和程序 |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2019055967 | 2019-03-25 | ||
| JP2019-055967 | 2019-03-25 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2020195876A1 true WO2020195876A1 (ja) | 2020-10-01 |
Family
ID=72609331
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/JP2020/010740 Ceased WO2020195876A1 (ja) | 2019-03-25 | 2020-03-12 | 移動体およびその制御方法、並びにプログラム |
Country Status (4)
| Country | Link |
|---|---|
| US (1) | US20220153411A1 (cg-RX-API-DMAC7.html) |
| JP (1) | JP7476882B2 (cg-RX-API-DMAC7.html) |
| CN (1) | CN113678082B (cg-RX-API-DMAC7.html) |
| WO (1) | WO2020195876A1 (cg-RX-API-DMAC7.html) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2557715B (en) * | 2017-02-28 | 2019-04-17 | Matthew Russell Iain | Unmanned aerial vehicles |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0239276A (ja) * | 1988-07-28 | 1990-02-08 | Agency Of Ind Science & Technol | 障害物検出装置 |
| JP2010092279A (ja) * | 2008-10-08 | 2010-04-22 | Murata Machinery Ltd | 自律移動体及び自律移動体の移動制御方法 |
| JP2011179886A (ja) * | 2010-02-26 | 2011-09-15 | National Maritime Research Institute | 障害物検知装置及び障害物検知方法 |
| JP2017151499A (ja) * | 2016-02-22 | 2017-08-31 | 株式会社Ihi | 障害物回避方法および装置 |
| JP2019053501A (ja) * | 2017-09-14 | 2019-04-04 | Kddi株式会社 | 飛行装置、管理装置、飛行管理方法及びプログラム |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5606627A (en) * | 1995-01-24 | 1997-02-25 | Eotek Inc. | Automated analytic stereo comparator |
| US9185391B1 (en) * | 2014-06-17 | 2015-11-10 | Actality, Inc. | Adjustable parallax distance, wide field of view, stereoscopic imaging system |
| JP6673371B2 (ja) * | 2015-07-08 | 2020-03-25 | エスゼット ディージェイアイ テクノロジー カンパニー リミテッドSz Dji Technology Co.,Ltd | 可動物体を使用して障害物を検出する方法及びそのシステム |
| CN106054900B (zh) * | 2016-08-08 | 2018-11-09 | 电子科技大学 | 基于深度摄像头的机器人临时避障方法 |
| CN206445826U (zh) * | 2016-12-09 | 2017-08-29 | 南京理工大学 | 一种带电作业机器人数据通信系统 |
| CN107329490B (zh) * | 2017-07-21 | 2020-10-09 | 歌尔科技有限公司 | 无人机避障方法及无人机 |
| DE112018003986T5 (de) * | 2017-08-04 | 2020-04-16 | Sony Corporation | Steuervorrichtung, steuerverfahren, programm und mobileinheit |
| CN107656545A (zh) * | 2017-09-12 | 2018-02-02 | 武汉大学 | 一种面向无人机野外搜救的自主避障与导航方法 |
| CN108500992A (zh) * | 2018-04-09 | 2018-09-07 | 中山火炬高新企业孵化器有限公司 | 一种多功能的移动安防机器人 |
| CN108710376A (zh) * | 2018-06-15 | 2018-10-26 | 哈尔滨工业大学 | 基于多传感器融合的slam与避障的移动底盘 |
| CN109164825A (zh) * | 2018-08-13 | 2019-01-08 | 上海机电工程研究所 | 一种用于多旋翼无人机的自主导航避障方法及装置 |
-
2020
- 2020-03-12 JP JP2021509022A patent/JP7476882B2/ja active Active
- 2020-03-12 US US17/438,942 patent/US20220153411A1/en not_active Abandoned
- 2020-03-12 CN CN202080021996.8A patent/CN113678082B/zh active Active
- 2020-03-12 WO PCT/JP2020/010740 patent/WO2020195876A1/ja not_active Ceased
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0239276A (ja) * | 1988-07-28 | 1990-02-08 | Agency Of Ind Science & Technol | 障害物検出装置 |
| JP2010092279A (ja) * | 2008-10-08 | 2010-04-22 | Murata Machinery Ltd | 自律移動体及び自律移動体の移動制御方法 |
| JP2011179886A (ja) * | 2010-02-26 | 2011-09-15 | National Maritime Research Institute | 障害物検知装置及び障害物検知方法 |
| JP2017151499A (ja) * | 2016-02-22 | 2017-08-31 | 株式会社Ihi | 障害物回避方法および装置 |
| JP2019053501A (ja) * | 2017-09-14 | 2019-04-04 | Kddi株式会社 | 飛行装置、管理装置、飛行管理方法及びプログラム |
Also Published As
| Publication number | Publication date |
|---|---|
| JPWO2020195876A1 (cg-RX-API-DMAC7.html) | 2020-10-01 |
| CN113678082A (zh) | 2021-11-19 |
| JP7476882B2 (ja) | 2024-05-01 |
| US20220153411A1 (en) | 2022-05-19 |
| CN113678082B (zh) | 2024-07-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11721225B2 (en) | Techniques for sharing mapping data between an unmanned aerial vehicle and a ground vehicle | |
| US10599149B2 (en) | Salient feature based vehicle positioning | |
| US20210065400A1 (en) | Selective processing of sensor data | |
| US11709073B2 (en) | Techniques for collaborative map construction between an unmanned aerial vehicle and a ground vehicle | |
| US10936869B2 (en) | Camera configuration on movable objects | |
| US11288860B2 (en) | Information processing apparatus, information processing method, program, and movable object | |
| US10152059B2 (en) | Systems and methods for landing a drone on a moving base | |
| CN108139211B (zh) | 用于测量的装置和方法以及程序 | |
| CN111886626A (zh) | 信号处理设备和信号处理方法、程序及移动体 | |
| WO2017076929A1 (en) | Device and method for autonomous localisation | |
| JP2018109564A (ja) | 情報処理装置、情報処理方法、および情報処理プログラム | |
| CN111093907B (zh) | 机器人式运载工具的稳健导航 | |
| US20220290996A1 (en) | Information processing device, information processing method, information processing system, and program | |
| JP7476882B2 (ja) | 移動体およびその制御方法、並びにプログラム | |
| US12417643B2 (en) | Systems and methods for updating a current base model comprising a plurality of images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20777007 Country of ref document: EP Kind code of ref document: A1 |
|
| ENP | Entry into the national phase |
Ref document number: 2021509022 Country of ref document: JP Kind code of ref document: A |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 20777007 Country of ref document: EP Kind code of ref document: A1 |