JP2017227516A - Device, mobile body device, and method for measuring distance - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device by which information for safely moving a mobile body can be obtained.SOLUTION: The distance measuring device is a device equipped in a mobile body 10, and includes: a distance measuring system attached to the mobile body 10 by a gimbal for measuring distance information on the distance to an object; a detection system for detecting attitude change information as information on the relative attitude change of the distance measuring system and the mobile body 10 associated with moving of the mobile body 10; and a control operation unit 520 (correction system) for correcting the distance information using the attitude change information.SELECTED DRAWING: Figure 8

Description

  The present invention relates to a device, a mobile device, and a distance measuring method.

  2. Description of the Related Art In recent years, development of devices that perform ranging and imaging mounted on a moving body has been actively performed.

  For example, Patent Document 1 discloses an apparatus for performing imaging that is attached to a moving body via a gimbal.

  However, conventional devices such as the device disclosed in Patent Document 1 have room for improvement with respect to obtaining information for safely moving a moving object.

  The present invention is an apparatus mounted on a moving body, the ranging system attached to the moving body via a gimbal, and the relative attitude of the ranging system and the moving body accompanying the movement of the moving body An apparatus comprising: a detection system that detects posture change information that is information relating to a change; and a correction system that corrects the measurement distance of the distance measurement system using the posture change information.

  According to the present invention, information for safely moving a mobile object can be obtained.

It is a schematic diagram which shows an example of the mobile body apparatus of 1st Embodiment. It is a perspective view which shows the ranging system of FIG. It is a perspective view which shows typically an example of the gimbal of FIG. It is a schematic diagram of the stereo camera of a ranging system. It is a block diagram which shows an example of a structure of a mobile body apparatus. It is a block diagram which shows an example of the hardware constitutions of the ranging part of a ranging system. It is a block diagram which shows an example of the hardware constitutions of the control part of a ranging system. It is a block diagram which shows an example of the hardware constitutions of a moving body. It is a principle diagram of a stereo camera. It is a figure for demonstrating the parallax obtained with a stereo camera. FIG. 6 is a relationship diagram between parallax and distance obtained by a stereo camera. FIGS. 12A to 12D are diagrams (No. 1 to No. 4) showing the relationship between the distance image and the direction of the stereo camera and the direction of the moving body attached to the moving body via the gimbal, respectively. is there. 6 is a flowchart for explaining distance measurement processing 1; FIGS. 14A to 14C are diagrams (No. 1 to No. 3) for explaining the distance measuring method of the second embodiment, respectively. It is a figure for demonstrating the mobile body apparatus of 3rd Embodiment. It is a block diagram which shows the hardware constitutions of the mobile body of FIG. It is a block diagram which shows the hardware constitutions of the ranging system of FIG. 18A and 18B are diagrams for explaining the light projecting system and the light receiving system in FIG. 17, respectively. 10 is a flowchart for explaining distance measurement processing 2; It is a figure for demonstrating the modification of the ranging system using a TOF method. It is a figure for demonstrating an example of the ranging system using a pattern projection method.

[Introduction]
An embodiment of the present invention relates to a mobile device that can move independently by mounting a distance measuring device on the mobile as an example. In the mobile body device, the distance measuring device supports the movement of the mobile body by measuring the distance to an object (target object) existing around the mobile body.

  For example, if the mobile body is a mobile body that moves autonomously, the distance measuring device estimates the self-position of the mobile body by measuring the distance to an object (target object) that exists around the mobile body. The moving body can control the autonomous movement based on the self-position estimation result of the distance measuring device. Thereby, the moving body can move autonomously while avoiding obstacles.

  In the following embodiments, the moving body is an unmanned aerial vehicle that autonomously flies (for example, Drone or UAV (Unmanned aerial vehicle)), the ranging device estimates the self-position of the moving body, and the moving body performs the self-position estimation result. A mobile device that controls autonomous movement based on the above will be described as an example.

<< First Embodiment >>
Hereinafter, a first embodiment of the present invention will be described in detail with reference to the drawings.
[Basic configuration]
FIG. 1 is a schematic diagram illustrating an example of a mobile device 1 according to the first embodiment. As shown in FIG. 1, the moving body device 1 includes a moving body 10, a gimbal 20, a distance measuring system 100, and a posture driving unit 200. Here, the moving body 10 is an unmanned aerial vehicle. Hereinafter, an XYZ three-dimensional orthogonal coordinate system (hereinafter referred to as a “moving body coordinate system”) set to a moving body in which the longitudinal direction of the moving body 10 is the X-axis direction, the left-right direction is the Y-axis direction, and the height direction is the Z-axis direction. Will be described as appropriate. Here, the forward direction of the moving body 10 is the + X direction in the moving body coordinate system.

  The moving body 10 includes a body 11 and propellers 12 to 15 provided on the front, rear, left and right of the body 11. The moving body 10 is configured to fly when the propellers 12 to 15 are rotationally driven. The airframe 11 is provided with a gimbal 20, a distance measuring system 100, and four legs (16, 17, 18, 19). The four legs also play a role of protecting the distance measuring system 100 from coming into contact with the ground while supporting the aircraft 11 so as not to fall when the aircraft 11 lands on the ground.

  2 is a perspective view showing an example of the distance measuring system 100 of the first embodiment, FIG. 3 is a perspective view schematically showing the gimbal 20, and FIG. 4 is an example of the distance measuring system 100 schematically. FIG.

  2 to 4, the distance measuring system 100 includes a housing 120, a stereo camera 350 including imaging optical systems 121 </ b> A and 121 </ b> B and imaging elements 311 </ b> A and 311 </ b> B disposed in the housing 120, and a housing. And a mounting attachment 110 fixed to 120.

  The distance measuring system 100 includes a captured image captured by a CMOS (Complementary Metal Oxide Semiconductor) 311A that is an imaging device for the imaging optical system 121A and the imaging optical system 121A, and an imaging device for the imaging optical system 121B and the imaging optical system 121B. Ranging is performed by parallax calculation using the captured image captured by the CMOS 311B.

  The distance measuring system 100 is attached to the moving body 10 via the shaft 21, the attitude driving unit 200, and the gimbal 20 (see FIG. 1).

  Specifically, the attachment 110 fixed to the housing 120 is attached to one end (the end on the −Z side) of the shaft 21. The other end (+ Z side end) of the shaft 21 is held by the attitude driving unit 200. The attitude driving unit 200 is connected to the airframe 11 via the gimbal 20.

  The gimbal 20 has a gimbal mechanism that is a three-degree-of-freedom gyro having three swing axes orthogonal to each other, and a system including the distance measuring system 100, the shaft 21, and the attitude driving unit 200 is arbitrarily arranged with respect to the moving body 10. Connect in a swingable direction.

  In addition to the gimbal mechanism, the gimbal 20 has a three-axis gyro sensor 20a that simultaneously detects the angular velocities of the swing axes of the gimbal mechanism. The detection result of the three-axis gyro sensor 20a, that is, the inclination information of each sway axis of the gimbal mechanism is sent to the attitude driving unit 200. The gyro sensor is also called an angle sensor.

  Note that the gimbal may have three single-axis gyro sensors that detect the angular velocity of each swing shaft instead of the three-axis gyro sensor, or each combination of the two-axis gyro sensor and the single-axis gyro sensor. The angular velocity of the swing shaft may be detected.

  The surfaces including the optical axes of the imaging optical systems 121A and 121B of the stereo camera 350 are substantially perpendicular to the shaft 21 (see FIG. 4).

  The attitude driving unit 200 drives the distance measuring system 100 via the shaft 21 based on the detection result of the three-axis gyro sensor 20a.

  More specifically, the attitude driving unit 200 changes the relative attitude of the distance measuring system 100 with respect to the moving body 10 (for example, rotation around the X axis, Y axis, etc.) due to the operation of the gimbal 20 accompanying the movement (flight) of the moving body 10. The shaft 21 is driven so that the distance measuring system 100 follows the movement of the moving body 10. The attitude drive unit 200 has a small motor as a drive source.

  FIG. 5 is a block diagram illustrating an example of the configuration of the mobile device 1 according to the first embodiment. As shown in FIG. 5, the distance measuring system 100 includes a distance measuring unit 300 and a control unit 400. The distance measuring unit 300 has a configuration related to distance measurement by the stereo camera 350, and the control unit 400 has a configuration related to control of the distance measuring unit 300 and communication with the gimbal 20 and the moving body 10.

  FIG. 6 is a block diagram illustrating an example of a hardware configuration of the distance measuring unit 300 according to the first embodiment. The subject light that has passed through the imaging optical system 121A forms an image on the CMOS 311A, and the subject light that has passed through the imaging optical system 121B forms an image on the CMOS 311B. The CMOS 311A converts the optical image formed on the imaging surface into an electrical signal and outputs it as analog image data. The CMOS 311B converts the optical image formed on the imaging surface into an electrical signal and outputs analog image data. Output as. A CCD (Charge Coupled Device) may be employed as the image sensor instead of the CMOS 311A and the CMOS 311B.

  The image data output from the CMOS 311A has its noise component removed by a CDS (Correlated Double Sampling) circuit 312A, converted to a digital value by an A / D converter 313A, and output to the image processing circuit 314A. . The image data output from the CMOS 311B has its noise component removed by the CDS circuit 312B, converted to a digital value by the A / D converter 313B, and output to the image processing circuit 314B.

  Note that the timing of operation of the CMOSs 311A and 311B, the CDS circuits 312A and 312B, and the A / D converters 313A and 313B is controlled by a timing signal generator 316. Each of the image processing circuits 314A and 314B uses an SDRAM (Synchronous Dynamic Random Access Memory) 317 that temporarily stores image data, and performs RGB → YCrCb conversion processing, white balance control processing, contrast correction processing, edge enhancement processing, and color. Various image processing such as conversion processing is performed. The white balance control process is an image process that adjusts the color density of image information, the contrast correction process is an image process that adjusts the contrast of image information, and the edge enhancement process adjusts the sharpness of image information. The color conversion process is an image process for adjusting the hue of image information.

  The image data subjected to the image processing stored in the SDRAM 317 is compressed by the image compression / decompression circuit 318 and recorded in the memory card 319. The image compression / decompression circuit 318 compresses the image data output from the image processing circuits 314A and 314B and outputs the compressed image data to the memory card 319, and decompresses the image data read from the memory card 319 to the image processing circuits 314A and 314B. It is a circuit to output.

  The timing signal generator 316, the image processing circuits 314A and 314B, the image compression / decompression circuit 318, and the memory card 319 are controlled by a CPU (Central Processing Unit) 315. A ROM (Read Only Memory) 321 is a read-only memory that stores programs executed by the CPU 315, and a RAM (Random Access Memory) 322 is a work area that the CPU 315 uses in various processing steps, and stores various data. This is a readable / writable memory having an area and the like.

  The CPU 315 generates a parallax image by performing parallax calculation using the image data subjected to various image processing by the image processing circuit 314A and the image data subjected to various image processing by the image processing circuit 314B, that is, a stereo image. Then, based on the generated parallax image, the distance to the object shown in the parallax image is obtained. Note that the parallax is calculated for each pixel or each pixel block, and in the parallax image, the parallax information is associated with each pixel or each pixel block. A pixel block is a block composed of one or more pixels. Thereby, the parallax image can represent the three-dimensional position (three-dimensional coordinate) in the three-dimensional space of the object (target object) reflected in the parallax image. The main components of the distance measuring unit 300 are interconnected by a bus line 19.

  FIG. 7 is a block diagram illustrating an example of a hardware configuration of the control unit 400 according to the first embodiment. The control unit 400 includes a CPU 411, a RAM 412, a flash ROM 413, an I / O 414, and a network I / F 415.

  The CPU 411 controls the overall operation of the control unit 400 by executing the program 420 stored in the flash ROM 413. The program 420 may be distributed in a state stored in the memory card 416, or may be distributed by being downloaded from a server via a communication network such as a mobile phone network or a wireless local area network (LAN) network. Also good. The RAM 412 is used as a work area (a program and data are temporarily stored) when the CPU 411 executes the program.

  The CPU 411 performs self-position estimation based on the distance measurement result of the distance measurement unit 300 or causes the distance measurement unit 300 to perform distance measurement again. The I / O 414 is an input / output interface such as I2C or UART. The network I / F 415 is a communication device for communicating on an in-vehicle network such as Ethernet (registered trademark).

  FIG. 8 is a block diagram illustrating an example of a hardware configuration of the moving object 10 according to the first embodiment. The moving body 10 mainly includes a control device 519, a GPS antenna 512, a GPS receiver 525, and propeller motors 545A to 545D.

  The GPS antenna 512 and the GPS receiver 525 measure the absolute coordinates of the moving body 10. As the GPS antenna 512 and the GPS receiver 525, post-processing kinematic GPS or real-time kinematic GPS (RTK-GPS) is preferably used. RTK-GPS can measure with high accuracy.

  Propeller motors 545A to 545D are motors for rotationally driving propellers 12 to 15, respectively.

  The control device 519 mainly includes a control calculation unit 520, a storage unit 522, an imaging control unit 523, a flight control unit 526, an orientation sensor 527, a gyro unit 528, and a communication unit 529.

  The storage unit 522 is, for example, a RAM or a ROM, and includes a program storage unit and a data storage unit. The program storage unit includes programs such as an imaging program for controlling imaging of the stereo camera 350, a flight control program for autonomous flight, an attitude control program for controlling the attitude of the stereo camera 350, and a flight planning program. Stored. The data storage unit stores image data captured by the stereo camera 350 and data such as absolute coordinates (ground coordinates) acquired by the GPS antenna 512 and the GPS receiver 525.

  The control calculation unit 520 controls the overall operation of the moving body 10 based on a program stored in the storage unit 522.

  Here, the stereo camera 350 can capture a still image and a moving image, and can output the acquired image to the imaging control unit 523 as image data of a digital signal.

  The imaging control unit 523 performs control related to imaging of still images and moving images by the stereo camera 350. Distance information for each pixel or pixel block, that is, a distance image obtained from a parallax image is sent from the distance measurement unit 300 to the imaging control unit 523. The imaging control unit 523 can also obtain luminance information from the image data from the stereo camera 350.

  The direction sensor 527 detects the direction of the moving body 10 and outputs the detection result to the control calculation unit 520.

  The gyro unit 528 detects the attitude of the moving body 10 in the flight state, and outputs the detection result to the control calculation unit 520.

  Further, the detection result of the three-axis gyro sensor 20 a is also output to the control calculation unit 520. Further, the measurement result (distance image) of the distance measurement unit 300 is also output to the control calculation unit 520.

  The control calculation unit 520 performs calculation processing described later based on the measurement result of the distance measurement unit 300 and the detection result of the three-axis gyro sensor 20a, and outputs the calculation result to the flight control unit 526.

  The flight control unit 526 includes a self-position estimated by the control unit 400 of the distance measuring system 100, a direction of the moving body 10 detected by the direction sensor 527, a posture of the moving body 10 detected by the gyro unit 528, and a control calculation unit. The autonomous flight of the moving body 10 is controlled by controlling the driving of the propeller motors 545A to 545D based on the calculation result from 520 and the like.

  Note that the flight control unit 526 controls the autonomous flight of the mobile object 10 using the absolute coordinates acquired by the GPS device including the GPS antenna 512 and the GPS receiver 525 and the calculation result from the control calculation unit 520. good.

  However, although the GPS device can determine the absolute position, the result may become unstable or unobservable depending on the environment. Therefore, the merit of providing the orientation sensor 27, the gyro unit 528, etc. is great.

  The communication unit 529 communicates with the distance measurement system 100 to acquire self-position information indicating the self-position estimated by the distance measurement system 100, or image data captured by the stereo camera 350, a GPS antenna 512, and a GPS receiver. A function of transmitting the absolute coordinates of the moving body 10 measured by 525 wirelessly to a base station or the like is provided.

[Principles of ranging]
Conventionally, distance measuring devices including a distance measuring system including, for example, a stereo camera, a millimeter wave radar, a laser radar, and the like are known, and a distance to an object is measured using a distance measurement result of these distance measuring devices. A technique is known (see, for example, Patent Document 1 (Japanese Patent Laid-Open No. 2006-31101)).

  For example, a distance measuring system including a stereo camera takes an object to be measured with two cameras and calculates the distance to the object based on the principle of triangulation using the parallax generated between the two images obtained. To do.

  The principle of calculating the distance using parallax will be described with reference to FIGS. In FIG. 9, the measurement object 101 is photographed using a stereo camera 106 including two cameras 102 a and 102 b composed of optical systems (lenses and imaging elements) having the same configuration. The measurement object image 104a obtained through the lens 103a and the measurement object image 104b obtained through the lens 103b are shown on the two-dimensional imaging elements 105a and 105b, respectively. At this time, the difference in the position where the measurement object appears on the two-dimensional image sensor is the parallax Δ.

Here, the distance between the optical axes of the lenses 103a and 103b is called a base line length, which is B, the distance between the lens and the measurement object is A, the focal length of the lenses 103a and 103b is f, and A >> f. In some cases, the following (Equation 0) holds.
A = Bf / Δ (Expression 0)

  From the above (Equation 0), the base line length B and the focal length f of the lens are known, so that the distance A to the measurement object can be calculated by detecting the parallax Δ. As can be seen from the above (Equation 0), when the distance A is large (= far), the parallax Δ is small, and when the distance A is small (= close), the parallax Δ is large. The distance A obtained here is a distance in the direction perpendicular to the direction of the baseline length B.

  A method of obtaining the parallax Δ will be described with reference to FIG. When the measurement object 101 is shown on the two-dimensional image sensor 105b, the measurement object 101 is shown on the two-dimensional image sensor 105a at a position different from the two-dimensional image sensor 105b by a parallax Δ. A technique called “pattern matching” is used to detect how different they are. This is where the object exists in one image (referred to as a reference image, here the image of the two-dimensional image sensor 105b) in the other image (referred to as a comparative image, here the image of the two-dimensional image sensor 105a). In this search, correlation processing such as SAD (Sum of Absolute Difference) method is generally used.

  The correlation process is a process of determining a block most correlated with the template by moving a window having the same size as the template within the search range on the comparison image using a block (for example, 7 × 7 pixels) in the reference image as a template. It is. When searching for the same image (block), since the search is performed in order of 1 pixel, 2 pixels,..., While shifting one pixel at a time, the measurable distance increases as the number of search pixels increases.

  As an example, when the base line length B is 100 mm, the lens focal length is 10 mm, and the pixel size of the image sensor is 5 μm, the relationship between the distance A from the (Equation 0) to the measurement object and the parallax Δ is as shown in FIG. . When the distance A to the measurement object is 1300 mm, the parallax is 154 pixels, when the distance is 1000 mm, the parallax is 200 pixels, and when the distance is 700 mm, the parallax is 286 pixels.

  As described above, a distance measuring apparatus that measures the distance to an object using a stereo camera is already known.

[problem]
By the way, an unmanned aerial vehicle that autonomously flies as a moving body on which a current stereo camera is mounted has a role of capturing an image of an object with a stereo camera while flying, not just autonomously flying. Therefore, in order to obtain a stable image, a stereo camera is generally attached to a moving body via a video stabilization / tracking device called a gimbal device including a gimbal and a posture driving unit as in the first embodiment. It has become. With this gimbal device, the camera orientation can be stabilized against fine vibrations of the moving body or sudden changes in direction, and an easy-to-view image without screen blur can be provided.

  However, in other words, the stereo camera does not follow the movement of the moving body quickly. When a ranging system including a stereo camera is attached to a moving body via a gimbal device, a stable image can be obtained as an imaging means, but in a direction (traveling direction) that the moving body is originally facing as a ranging means. Although the distance should be measured, the distance in the direction in which the stereo camera is facing is slightly different from the direction in which the moving body is facing. Therefore, when the moving body changes direction, the direction of the stereo camera follows the direction of the moving body with a delay, so during that time the stereo camera is facing instead of the distance of the moving body (direction of travel). It will measure the distance in the direction.

  For example, in a mobile object that autonomously flies in a small space such as a room, the mobile object may be affected by a ranging error in the time from when the direction of the mobile object changes until the camera direction matches that of the mobile object. There is concern about collision with surrounding objects.

[Discussion]
The gimbal is configured to connect one device to another device so as to be rotatable around three orthogonal axes of, for example, Pitching, Rolling, and Yawing. (There is also a gimbal that can be rotated around its axis).

  When a stereo camera is attached to a moving body via a gimbal, an angle sensor (gyro sensor) detects a fine tilt at high speed due to vibration around each axis of the stereo camera relative to the moving body, and cancels the detected tilt. The attitude of the stereo camera relative to the moving body can be kept constant.

By feeding back the detected inclination to the parallax obtained by the stereo camera, it is possible to calculate the distance in the direction (traveling direction) in which the moving body is facing, not the distance in the direction in which the stereo camera is facing.
The principle will be described with reference to FIG.

  FIG. 12A shows a case where the moving body and the stereo camera face the same direction, that is, a case where the traveling direction of the moving body and the optical axis direction of the stereo camera coincide. Here, for convenience, the “optical axis of the stereo camera” is an intermediate axis between the optical axes of the two eyes (two imaging units) of the stereo camera. In this case, if there is a plane (one surface of the object) in front of the moving body, the stereo camera detects that there is a plane perpendicular to the traveling direction of the moving body. At this time, the distance image generated by the distance measuring unit including the stereo camera shown in FIG. 12A (hereinafter also referred to as “stereo distance measuring unit”) is uniform (constant density) over the entire plane. The “distance image” is a total of distance data for each pixel obtained from the parallax image generated by the stereo distance measuring unit.

  On the other hand, as shown in FIG. 12B, even when the moving body suddenly rotates (rapidly turns) in one direction (for example, counterclockwise) around an axis (for example, the vertical axis) perpendicular to the paper surface, Stereo cameras that are kept stable by the gimbal do not suddenly tilt. Therefore, the stereo camera still detects that there is a plane perpendicular to the traveling direction. At this time, the distance image generated by the stereo distance measuring unit shown in FIG. 12B is the same as the distance image shown in FIG.

  In this case, when viewed from the moving body, the plane is not perpendicular to the traveling direction but is inclined by θ. When the moving body proceeds as it is, the right portion near the plane of the moving body collides with the plane.

  Therefore, it is detected that the moving body is inclined by the angle θ, and the angle θ is added to each distance data generated by the stereo distance measuring unit, that is, the distance data is corrected using the angle θ.

  Then, as shown in FIG. 12C, the distance data of the plane output from the stereo distance measuring unit is converted into distance data in which the distance is shorter as it is closer to the right end of the moving body as viewed from the moving body, and the distance is longer as it is closer to the left end. It is possible to correct. At this time, the distance image generated by the stereo distance measuring unit shown in FIG. 12C has a light and dark gradation indicating that the darker portion has a shorter distance and the lighter portion has a longer distance.

  The stereo camera follows the moving body suddenly. When the sudden rotation of the moving body ends, the direction of the moving body and the direction of the stereo camera coincide as shown in FIG. 12D, and the moving body and the stereo camera face the same direction as in FIG. Become. At this time, the distance image generated by the stereo distance measuring unit shown in FIG. 12 (d) is the same as the distance image shown in FIG. 12 (c), and the angle θ is added to the distance data in FIG. 12 (c). Can be confirmed.

  In this way, distance data of the moving body can be obtained in real time by adding the difference between the direction of the moving body and the direction of the stereo camera to the distance data from the stereo camera.

  Although the distance data is not stable against vibration, there is no problem because the distance data is not viewed and observed by a person. A stable photographed image can be obtained by the gimbal, and a distance image can be obtained in real time, so that the moving body can fly with high accuracy without colliding with surrounding objects even in a narrow place such as a room.

  Here, as an example of the case where the direction of the stereo camera and the direction of the moving body are different from each other, the case where the moving body suddenly rotates has been described. However, for example, when the moving body suddenly accelerates or decelerates suddenly, It is assumed that the direction and the direction of the moving object are different. Also, for example, when the moving body 10 vibrates, when the moving body bends gently, when slowly accelerating, or when slowly decelerating, it is assumed that the direction of the stereo camera and the direction of the moving body are larger or smaller. Is done.

[Improvements]
In the following, for the sake of simplicity, it is assumed that the direction of the moving body 10 (the forward direction) coincides with the horizontal direction and that only gravity acts on the system including the distance measuring system 100, the shaft 21, and the attitude driving unit 200. When the initial state of the gimbal 20 is defined, the description will be made on the assumption that the direction of the moving body 10 and the direction of the stereo camera 350 match in the initial state.

  As described above, since the stereo camera 350 attached to the moving body 10 via the gimbal 20 follows the movement of the moving body 10 with a delay (slowly) by the posture driving unit 200, the traveling direction of the moving body 10 and the stereo There is a temporary shift (tilt) in the optical axis of the camera 350.

  Therefore, in the first embodiment, the detection result of the three-axis gyro sensor 20a of the gimbal 20 is the difference in the direction of the stereo camera 350 with respect to the direction of the moving body 10, that is, the optical axis of the stereo camera 350 with respect to the traveling direction of the moving body 10. Focusing on the fact that tilt information (tilt direction and tilt angle) is indicated, the tilt data is used to convert (correct) distance data (distance information) measured by the distance measuring unit 300 into distance data viewed from the moving body 10. . The method will be described below.

  First, a distance in which the z-axis is a direction parallel to the optical axis of the stereo camera 350 from two captured images of the stereo camera 350 attached to the moving body 10 via the gimbal 20 (captured images by the two CMOSs 311A and 311B). Represented by information (x, y, z), that is, a two-dimensional coordinate (x, y) in a plane orthogonal to the optical axis of the stereo camera 350 and a distance z from the stereo camera 350 at (x, y). A distance image including distance information for each pixel or pixel block (in the camera coordinate system) viewed from the stereo camera 350 is obtained.

When the tilt of the optical axis of the stereo camera 350 (hereinafter also referred to as “camera tilt”), which is the detection result of the three-axis gyro sensor 20a, with respect to the traveling direction of the moving body 10 is a rotation of the angle α around the x axis. The distance information (x ′, y ′, z ′) viewed from the moving body 10 is expressed by the following (Expression 1).

When the camera tilt is rotation of an angle β around the y axis, distance information (x ′, y ′, z ′) viewed from the moving body 10 is expressed by the following (Expression 2).

When the camera tilt is a rotation of an angle γ about the z axis, distance information (x ′, y ′, z ′) viewed from the moving body 10 is expressed by the following (Expression 3).

When the camera tilt is rotation of angles α, β, and γ around the x-axis, y-axis, and z-axis (combined rotation), distance information (x ′, y ′, z ′ viewed from the moving body 10). ) Is expressed by the following (formula 4).

  As can be seen from the above description, distance information (x, y, z) obtained from the distance measuring system 100 attached to the moving body 10 via the gimbal 20 was obtained from the detection result of the three-axis gyro sensor 20a. Using the optical axis of the stereo camera 350 and the inclination angles α, β, γ of the moving body in the traveling direction, distance information (x ′, y ′, z ′) viewed from the moving body 10, that is, distance information in the moving body coordinate system Can be converted to This conversion can be performed by a calculation process in the control calculation unit 520. Therefore, the “ranging device 1000” is configured including the ranging system 100, the three-axis gyro sensor 20a, and the control calculation unit 520.

  Thereby, even when the stereo camera 350 is attached to the moving body 10 via the gimbal 20, the distance information (x ′, y ′, z ′) is always based on the direction (traveling direction) of the moving body 10. It becomes possible to obtain in real time, and the moving body 10 can fly with high accuracy without colliding with surrounding objects even in a narrow place such as a room.

  In the initial state of the gimbal 20, when the direction of the stereo camera 350 is inclined with respect to the direction of the moving body 10, the distance data may be corrected in consideration of the inclination information.

  Hereinafter, distance measurement processing 1 for realizing the distance measurement method of the first embodiment will be described with reference to FIG. The flowchart in FIG. 13 follows a processing algorithm executed by the distance measuring unit 300 and the control device 519.

  In the first step S1, it is determined whether or not the moving body 10 is moving. Specifically, the presence or absence of movement of the moving body 10 is determined based on information from the GPS receiver 525 or the like. If the determination here is affirmed, the process proceeds to step S2, and if the determination is negative, the flow ends.

  In step S2, the stereo camera 350 images an object that falls within the angle of view.

  In the next step S3, a parallax image of the object is generated from two captured images (stereo images) of the stereo camera 350.

  In the next step S4, a distance image of the object is generated from the parallax image.

  In the next step S <b> 5, posture change information (α, β, γ) that is information related to a relative posture change of the distance measuring system 100 with respect to the moving body 10 is detected.

  In the next step S6, the distance image is corrected using the posture change information (α, β, γ). Specifically, the calculation using any of the above formulas (1) to (4) is performed on the distance information for each pixel or pixel block of the distance image.

  The corrected distance image is sent to the flight control unit 526 and used for controlling the mobile object 10.

  The distance measuring device 1000 according to the first embodiment described above is a distance measuring device (device) mounted on the moving body 10 and is attached to the moving body 10 via the gimbal 20 to obtain distance information to the object. A distance measuring system 100 to be measured, a detection system that detects posture change information that is information relating to a relative posture change between the distance measuring system 100 and the moving body 10 accompanying the movement of the moving body 10, and using the posture change information A control calculation unit 520 (correction system) that corrects the distance information. The “movement of the moving body 10” includes forward movement, backward movement, ascending movement, descending movement, turning movement, rotational movement (posture change), and vibration in an arbitrary direction.

  In this case, the measurement result of the distance measuring system 100 when the relative postures of the distance measuring system 100 and the moving object 10 change with the movement of the moving object 10 is corrected to the distance information viewed from the moving object 10. (Converted).

  As a result, according to the distance measuring apparatus 1000, information for safely moving the moving body 10 can be obtained.

  Further, it is preferable to use a stereo camera method for the distance measuring system.

  In this case, a stable luminance image by the gimbal 20 can be obtained from the stereo camera 350, and a real-time distance image from which the stability effect of the gimbal 20 is removed (as viewed from the moving body 10) can be obtained from the stereo camera 350. . That is, it is possible to achieve both stabilization of image quality and improved reliability of distance measurement using one stereo camera 350.

  Further, the posture change information is preferably information related to the inclination of the optical axis of the distance measuring system 100 with respect to the traveling direction of the moving body 10.

  The detection system preferably includes at least one gyro sensor that measures inclination information of the swing axis of the gimbal 20 accompanying the movement of the moving body 10.

  In this case, the posture change information can be easily obtained from the tilt information of the swing axis of the gimbal 20 used as control information for controlling the relative posture change of the distance measuring system 100 accompanying the movement of the moving body 10. .

  The detection system includes a gyro unit 528 (moving body posture detecting unit) that detects the posture of the moving body 10, and a ranging system posture detecting unit (for example, at least one gyro sensor) that detects the posture of the ranging system 100. , May be included.

  In this case, posture change information can be obtained using information related to the posture of the moving body 10 and information related to the posture of the distance measuring system. That is, posture change information can be obtained without using the detection result of the three-axis gyro sensor 20a.

  Specifically, the rotation angle around each axis of the world coordinate system that is the detection result of the gyro unit 528 is (φ1, σ1, ε1), and each axis of the world coordinate system that is the detection result of the ranging system posture detection unit. If the rotation angle is (φ2, σ2, ε2), the posture change information (α, β, γ) = (φ2-φ1, σ2-σ1, ε2-ε1).

  Furthermore, the detection system preferably includes an azimuth sensor 527 (moving body traveling direction detection unit) that detects the traveling direction of the moving body 10.

  In this case, posture change information can be obtained more accurately.

  Further, according to the mobile device 1 including the distance measuring system 100 and the mobile object 10 to which the distance measuring system 100 is attached via the gimbal 20, the mobile device that can move the mobile object safely is provided. Can be provided.

  The distance measuring method of the first embodiment is a distance measuring method using the distance measuring system 100 attached to the moving body 10 via the gimbal 20 from the first viewpoint. A step of measuring distance information to the object, a step of detecting posture change information related to a relative posture change between the distance measuring system 100 and the moving body 10 accompanying the movement of the moving body 10, and based on the posture change information And correcting the distance information.

  In this case, the measurement result of the distance measuring system when the posture of the distance measuring system 100 with respect to the moving object 10 is changed with the movement of the moving object 10 can be corrected to the distance information viewed from the moving object 10.

  As a result, according to the distance measuring system 100, information for safely moving the moving body 10 can be obtained.

  Here, the distance measuring system 100 of the first embodiment can also be regarded as an imaging device having a distance measuring function.

  Therefore, the imaging apparatus having a distance measuring function according to the first embodiment is an imaging apparatus (apparatus) mounted on the moving body 10 and is attached to the moving body 10 via a gimbal, and includes an imaging optical system and an imaging element ( For example, an imaging system including a stereo camera 350 having a plurality of (for example, two) imaging units each including CMOS and a parallax image generated from the imaging results of the plurality of imaging units and captured by the plurality of imaging units CPU 315 (calculation means) for calculating the distance information to the camera, a detection system for detecting posture change information, which is information related to a relative posture change of the moving body 10 and the imaging system accompanying the movement of the moving body 10, and posture change information And a control calculation unit 520 (correction unit) that corrects the distance information by using.

  In this case, the calculation result of the CPU 315 when the relative posture between the imaging system and the moving body 10 changes with the movement of the moving body 10 can be corrected (converted) into distance information viewed from the moving body 10.

  As a result, according to the imaging device of the first embodiment, it is possible to obtain information for safely moving the moving body 10.

  Moreover, it is preferable that the imaging device of 1st Embodiment is further provided with the imaging control part 523 (luminance information acquisition part) which acquires luminance information from the imaging result of a several imaging part.

  In this case, a stable luminance image by the gimbal 20 can be obtained from the stereo camera 350, and a real-time distance image from which the stability effect of the gimbal 20 is removed (as viewed from the moving body 10) can be obtained from the stereo camera 350. . That is, it is possible to achieve both stabilization of image quality and improved reliability of distance measurement using one stereo camera 350.

  Further, the posture change information is preferably information related to the inclination of the optical axis of the stereo camera 350 with respect to the traveling direction of the moving body 10.

  The detection system preferably includes at least one gyro sensor that detects inclination information of the swing axis of the gimbal 20 accompanying the movement of the moving body 10.

  In this case, the posture change information can be easily obtained from the tilt information of the swing axis of the gimbal 20 used as control information for controlling the relative posture change of the distance measuring system accompanying the movement of the moving body 10.

  The detection system includes a moving body posture detection unit (for example, at least one gyro sensor) that detects the posture of the moving body 10, and a stereo camera posture detection unit (for example, at least one gyro sensor) that detects the posture of the stereo camera 350. And may be included.

  In this case, posture change information can be obtained using information related to the posture of the moving body 10 and information related to the posture of the stereo camera 350.

  In addition, according to the moving body device including the imaging apparatus of the first embodiment and the moving body 10 to which the imaging apparatus is attached via the gimbal 20, in this case, the stereo camera 350 is accompanied with the movement of the moving body 10. The measurement result of the stereo camera 350 when the posture with respect to the moving body 10 is changed can be corrected to the distance information viewed from the moving body 10.

  As a result, according to the imaging device of the first embodiment, it is possible to obtain information for safely moving the moving body 10.

  From the second viewpoint, the distance measuring method according to the first embodiment is a distance measuring method using an imaging system including a stereo camera 350 that is attached to the moving body 10 via the gimbal 20 and has a plurality of imaging units. A step of generating parallax images from the imaging results of the plurality of imaging units, calculating distance information to an object imaged by the plurality of imaging units, and an imaging system and the moving body 10 associated with the movement of the moving body 10 A step of detecting posture change information, which is information relating to a relative posture change, and a step of correcting distance information using the posture change information.

  In this case, the distance information calculated when the relative posture of the imaging system and the moving body 10 changes as the moving body 10 moves can be corrected (converted) to the distance information viewed from the moving body 10. .

  As a result, according to the distance measuring method of the first embodiment, information for safely moving the moving body 10 can be obtained.

  As can be understood from the above description, the inventor can always obtain the distance in the direction in which the moving body is facing (traveling direction) even when the ranging system or the imaging system is attached to the moving body 10 via the gimbal 20. By making it possible to achieve both stable image capturing and reliable distance measurement, in particular, as a technology that can move the moving body 10 accurately and at high speed without colliding even in a narrow space such as a room, The first embodiment was invented.

  In the first embodiment, the control calculation unit 520 of the moving body 10 performs the calculation process of correcting the distance information using the posture change information (α, β, γ) or the like. For example, the CPU 315 of the distance measuring unit 300 or the CPU 411 of the control unit 400 may perform this. Also in this case, it is necessary to input posture change information (α, β, γ) or the like to the CPU 315 of the distance measuring unit 300 or the CPU 411 of the control unit 400.

  In the following, another embodiment of the present invention will be described. In the following embodiments, points different from the first embodiment will be mainly described, and members having the same configurations and functions as those in the first embodiment will be denoted by the same reference numerals and detailed description will be given. Omitted.

<< Second Embodiment >>
In the first embodiment, the difference (α, β, γ) between the orientation of the stereo camera 350 and the moving body is obtained from the detection result of the three-axis gyro sensor 20a. In the second embodiment, the stereo camera 350 is used. Obtained from stereo images.

  As shown in FIG. 1, the airframe 11 is supported by four legs 16, 17, 18, and 19. When the angle of view of the lens of the stereo camera 350 used in the distance measuring system 100 is a wide angle, the foot 16 and the foot 17 in front of the stereo camera 350 fall within the angle of view of the stereo camera 350.

  Therefore, two marks 51 and 52 are attached to the foot 16, and two marks 53 and 54 are attached to the foot 17, and mark detection is performed from an image (FIG. 14A) captured by the stereo camera 350. By doing so, it is possible to detect the direction of the legs 16 and 17 of the stereo camera 350 and the moving body 10 and the distance between the stereo camera 350 and the legs 16 and 17 of the moving body 10.

  The feet 16 and 17 move following the movement of the moving body 10 swiftly, but the distance measuring system 100 including the stereo camera 350 is stabilized by the gimbal 20 and therefore does not follow the movement of the moving body 10 quickly. .

  Therefore, the marks of the foot 16 and the foot 17 that fall within the angle of view of the stereo camera 350 move little by little by the movement of the moving body 10. If this movement is detected, the difference (α, β, γ) between the orientation of the stereo camera 350 and the moving body 10 can be detected.

  If the direction of the stereo camera 350 rotates (pitch) around the Y axis with respect to the direction of the moving body 10, the mark is shifted by ΔY in the Y axis direction as shown in FIG. If the orientation of the stereo camera 350 rotates (yaws) around the Z axis with respect to the orientation of the moving body 10, the mark shifts in the ΔX direction along the X axis as shown in FIG.

  Furthermore, since the stereo camera 350 can measure the distance to the foot 16 and the foot 17, not only the direction of the moving body 10 but also the relative position (distance) between the stereo camera 350 and the moving body 10 can be detected.

  The positional relationship among the stereo camera 350, the foot 16, and the foot 17 is a fixed relationship in the initial state (see FIG. 14A). Therefore, the distance between the mark 51 and the mark 53 that falls within the angle of view of the stereo camera 350 is a constant value in the initial state.

  However, if the distance from the stereo camera 350 to the feet 16 and 17 is changed by the operation of the gimbal 20, the distance from the stereo camera 350 to the marks 51 and 53 is changed. The parallax between the mark 51 and the mark 53 increases as the distance decreases, and the parallax between the mark 51 and the mark 53 decreases as the distance increases. Therefore, the distance from the stereo camera 350 to the foot 16 and the foot 17 can be determined from the value A in FIG. If the distance between the stereo camera and the legs 16 and 17 is greater or less than the initial state, the distance from the moving object 10 to the surrounding objects can be obtained by adding or subtracting the distance from the distance to the surrounding object. Can be measured accurately.

  As described above, it is possible to detect the difference in the direction of the stereo camera 350 and the direction of the moving body 10 and the difference in the distance from the foot image of the airframe 11 entering the angle of view of the stereo camera 350. This makes it possible to obtain distance information from the moving object in real time as much as the position of the stereo camera 350 can be obtained more accurately than when using the output of the three-axis gyro sensor 20a. It is possible to fly with high accuracy without colliding with things.

  Here, a substantially butterfly shape is used as a “mark” attached to the leg of the moving body 10, but the shape is not limited to this, and a shape of another shape may be used. Further, the place where the mark is attached is not limited to the leg of the moving body, but may be any part as long as it is a part of the moving body and falls within the angle of view of the stereo camera.

  In the second embodiment described above, the distance measuring system 100 includes an imaging system including the stereo camera 350 having a plurality of imaging units, and a CPU 315 (processing unit) that generates parallax images from the captured images of the plurality of imaging units. The control calculation unit 520 (detection system) obtains posture change information (α, β, γ) that is relative posture change information of the imaging system with respect to the moving body 10 based on the parallax image. Instead of the control calculation unit 520 as the detection system, the CPU 315 of the distance measuring unit 300 or the CPU 411 of the control unit 400 may obtain the posture change information (α, β, γ).

  In this case, posture change information can be obtained using the stereo camera 350.

  In the plurality of imaging units, it is preferable that at least a part of the moving body 10 falls within the angle of view.

  Further, it is preferable that at least one mark is attached to at least a part of the angle of view of the plurality of imaging units of the moving body 10.

  In the first and second embodiments, the ranging system and the imaging system having the ranging function use the stereo camera method, but the TOF method or the pattern projection method may be used.

  In addition, it is preferable that the plurality of imaging units further include a step of obtaining posture change information based on the parallax image when at least a part of the moving body 10 enters the angle of view.

  In this case, posture change information can also be obtained using the stereo camera 350. Specifically, posture change information can be obtained by comparing a captured image after the posture change of the stereo camera 350 with respect to the moving body 10 and an image (reference image) before the posture change.

  Note that in the second embodiment, the detection system includes an image of a portion of the captured image of the stereo camera 350 that falls within the angle of view of the stereo camera 350 of the moving body 10 and pixels of the reference luminance image of the portion acquired in advance. The posture change information may be obtained by using the sum of the difference of values.

  In this case, the deviation can be measured with a simple algorithm.

  In the second embodiment, the detection system is acquired in advance from a parallax image of a portion that falls within the angle of view of the stereo camera 350 of the moving body 10 obtained from captured images of a plurality of imaging units of the stereo camera 350. The posture change information may be obtained using the sum of the differences in the parallax values of the reference parallax image of the portion.

  In this case, posture change information can be measured robustly against illumination changes.

<< Third Embodiment >>
Hereinafter, as a mobile device according to the third embodiment of the present invention, a mobile device including a distance measuring device (or an imaging device) using the TOF method will be described.

  FIG. 15 schematically shows the entire configuration of the mobile device 3 including the mobile 10 ′ of the third embodiment.

  As shown in FIG. 15, the mobile device 3 replaces the ranging system 100 (or the imaging system having a ranging function) of the mobile device of the first and second embodiments with a ranging / imaging unit 600. It has a configuration.

  FIG. 16 is a block diagram illustrating an example of a hardware configuration of the moving body 10 ′ and the ranging / imaging unit 600.

  As shown in FIGS. 15 and 16, the distance measuring / imaging unit 600 includes distance measuring means 600 a including a laser radar 355 and image capturing means 600 b including a camera 524 (here, a monocular camera).

  The distance measuring means 600a and the image pickup means 600b are provided integrally in the vertical direction. Here, the distance measuring means 600a is on the upper side and the imaging means 600b is on the lower side, but the opposite may be possible. Further, the distance measuring means 600a and the imaging means 600b may each have a housing or a common housing.

  The distance image generated by the distance measuring means 600a using the laser radar 355 is sent to the control calculation unit 520.

  The luminance image generated by the imaging unit 600b using the camera 524 is sent to the imaging control unit 523. Note that the configuration of the imaging unit 600b is the same as that obtained by replacing the configuration of the distance measuring unit 300 including the stereo camera 350 shown in FIG. 6 with a monocular camera (that is, an imaging optical system, a CMOS, etc., one by one). Equivalent to.

  The attitude driving unit 541 of the third embodiment is based on the detection result of the three-axis gyro sensor 20a via the control calculation unit 520, and the distance measurement / imaging unit 600 with respect to the moving body 10 ′ accompanying the movement of the moving body 10 ′. The ranging / imaging unit 600 is driven so as to cancel the posture change. The configuration of the attitude driving unit 541 is the same as that of the attitude driving unit 200.

  FIG. 17 is a block diagram illustrating an example of a hardware configuration of the distance measuring unit 600a.

  In addition to the radar radar 355, the distance measuring means 600a has a measurement control unit, a time measurement circuit, and a distance calculation unit.

  The laser radar 355 includes an LD, a light projecting optical system that guides light from the LD, a light projecting system that includes an LD driving circuit that turns on the LD when a light emission start signal is input from the measurement control unit, and the light projecting system. A light receiving optical system for guiding light projected from the system and reflected by an object, a light receiving element for receiving light via the light receiving optical system), and a light receiving system including a processing circuit for processing an output current of the light receiving element; And a binarization circuit that binarizes the processed signal from the processing circuit. Note that a photodetector is configured including the light receiving element and the processing circuit.

  The binarized signal from the binarization circuit is output to the time measurement circuit as a detection signal. The time measurement circuit calculates a time difference between the light emission timing of the LD and the light reception timing of the light receiving element based on the light emission start signal from the measurement control unit and the detection signal from the binarization circuit, and outputs the time difference to the distance calculation unit. The distance calculation unit calculates the distance to the object from the time difference from the time measurement circuit.

  18A and 18B show the configuration of the light projecting system and the light receiving system of the laser radar 355. FIG.

  The laser radar 355 has a scanning mirror whose light projection system deflects and scans the light from the LD toward the effective scanning region.

  Here, a MEMS mirror that drives the mirror portion by a MEMS mechanism is used as the scanning mirror, but various modifications such as a polygon mirror that rotates a polygon mirror by a motor, and other galvanometer mirrors are possible.

  Here, a light detection unit (for example, a photodiode or a phototransistor) that detects light (scanning light) scanned by the MEMS mirror is provided in a light projection range (detection region) including an effective scanning region, and the measurement control unit The position of the scanning light (scanning position) is detected based on the output of the light detection unit, and the LD of the light projecting system is controlled based on the scanning position. When the MEMS mirror is provided with a deflection angle detection unit that detects the deflection angle of the mirror unit, the measurement control unit may detect the scanning position based on the output of the deflection angle detection unit.

  Further, as a scanning configuration, a light projecting system and a light receiving system are stacked in the Z-axis direction, and both are incident on the scanning mirror coaxially and scanned when viewed from the Z-axis direction as shown in FIG. As a result, the area where light is projected by the light projecting system and the area where light is received by the light receiving system are matched to realize stable object detection and distance measurement.

  As a modified example of the third embodiment, a configuration may be adopted in which scanning is performed with a scanning mirror only in the light projecting system, and the entire light scanning system is observed with the imaging optical system without using the scanning mirror.

  In this case, the installation space of the scanning mirror is not required in the light receiving system, and it is possible to achieve high speed driving or wide angle driving by reducing the size of the scanning mirror in the light projecting system.

  In FIG. 18, the coordinate axes are set so that the effective scanning area is the YZ plane, and the scanning mirror is not shown in FIG. 18, but can be scanned independently in the two-axis directions of the Y-axis direction and the Z-axis direction. It is configured. When the scanning mirror is replaced with a polygon mirror or the like, it is also possible to switch the scanning / detection region in the Z-axis direction by arranging a plurality of reflecting surfaces inclined at different angles with respect to the rotation axis.

  Hereinafter, distance measurement processing 2 for realizing the distance measurement method of the third embodiment will be described with reference to FIG. The flowchart in FIG. 19 follows a processing algorithm executed by the distance measurement / imaging unit 600 and the control device 519.

  In the first step S11, it is determined whether or not the moving body 10 is moving. Specifically, the presence or absence of movement of the moving body 10 is determined based on information from the GPS receiver 525 or the like. If the determination here is affirmed, the process proceeds to step S2, and if the determination is negative, the flow ends.

  In step S12, a laser radar 355 is used to generate a distance image of the object existing in the effective scanning area.

  In the next step S13, posture change information, which is information relating to the relative posture change of the distance measurement / imaging unit 600 with respect to the moving body 10, is obtained from the detection results (α, β, γ) of the three-axis gyro sensor 20a.

  In the next step S14, the distance image is corrected using the posture change information. Specifically, the calculation using any of the above formulas (1) to (4) is performed on the distance information for each pixel or pixel block of the distance image.

  The corrected distance image is sent to the flight control unit 526 and used for controlling the moving body 10 '.

  Note that, as an example, the light projecting system 131 emits light whose intensity is modulated at a predetermined frequency, as in a modification of the distance measuring system using the TOF (Time-Of-Flight) method shown in FIG. good. Here, the light receiving system 132 includes a single two-dimensional imaging device capable of measuring a phase difference and an imaging optical system. At this time, the light receiving system 132 images light reflected from the imaging object and having a phase shift. Then, the calculation unit 133 compares the light emitted from the light projecting system 131 with the light imaged by the light receiving system 132, and obtains a depth map based on the time difference or the phase difference.

  In the first to third embodiments, the attitude driving unit controls the attitude of the distance measuring system and the imaging system to follow the direction of the moving object. However, the distance image viewed from the moving object is accurately obtained. From the viewpoint of generating, the posture driving unit is not essential. That is, even if the direction of the moving body and the direction of the ranging system or imaging system change continuously or constantly, by correcting the distance image using the posture change information (α, β, γ), This is because the viewed distance image can be generated with high accuracy. In this case, the three-axis gyro sensor 20a is not essential. That is, in the apparatus of the present invention, the gimbal can be configured only by the gimbal mechanism.

  In the first to third embodiments, a stereo camera or radar radar is used for the distance measuring system. However, the present invention is not limited to this, and for example, a millimeter wave radar or an infrared sensor can be used.

  In addition, not only laser radar but also a lidar (light detection and ranging) using TOF in general (including a light source that is not a laser) can be used for the distance measuring system.

  A distance projection system or an imaging system having a distance measurement function may use a pattern projection method.

  As an example, FIG. 21 shows a configuration example of a distance measuring system using a pattern projection method. Here, the light projecting system 131 emits a certain structured light. The structured light is light suitable for a method known as the structured light method, and includes, for example, striped light and matrix light. The light receiving system 132 images structured light that has been reflected and deformed by the imaging object. Then, the arithmetic unit 133 compares the light emitted from the light projecting system 131 with the light imaged by the light receiving system 132, and obtains a depth map based on the triangulation method. Here, the focal length of the light receiving system 132 is referred to as the focal length of the ranging system for convenience.

  The imaging system having a ranging system or a ranging function may include a plurality of same or different types of ranging units.

  For example, a camera for shooting (monocular camera or stereo camera) and a stereo camera for distance measurement may be provided side by side.

  In addition, the distance measuring system and the imaging system only need to have a plurality of imaging units, and may be, for example, a combination of two monocular cameras.

  Further, a plurality of systems including at least one ranging system and an imaging system having at least one ranging function may be attached to the moving body via a plurality of gimbals, respectively.

  Moreover, you may attach the ranging system and imaging system of the said 1st-3rd embodiment to moving bodies, such as a robot which can move autonomously, a vehicle, a ship, and a manned aircraft via a gimbal. Also in this case, the same effect as the first to third embodiments can be obtained.

  The gimbal that connects the moving body to the ranging system or the imaging system having the ranging function is not limited to the gimbal having 3 degrees of freedom, but may be a gimbal having 2 degrees of freedom or 1 degree of freedom. When a gimbal with 2 degrees of freedom is used, it is sufficient to provide at least one gyro sensor that detects the angular velocities of two swing axes orthogonal to each other. When a gimbal with 1 degree of freedom is used, A single gyro sensor for detecting the angular velocity may be provided.

  The numerical values, shapes, etc. used in the description of the above embodiments can be appropriately changed without departing from the spirit of the present invention.

  Below, the thought process which the inventor invented the said 1st-3rd embodiment is demonstrated.

  First, patent document 1 (Unexamined-Japanese-Patent No. 2014-62789) is demonstrated. In Patent Document 1, a camera for shooting and a stereo camera are attached to a moving body via a gimbal, so that a comprehensive observation of the crop is performed with the camera for shooting and a local observation of the crop is performed with the stereo camera. It is an object.

  In order to achieve this purpose, a wide range of crops are observed comprehensively using a camera for photography attached to the moving body. It is stationary (hovering) above, extending the shaft, bringing a stereo camera closer, measuring the height of the crop, observing the growth state, and performing local observation. Thus, it is disclosed that a camera for shooting and a stereo camera are attached to a moving body via a gimbal.

  However, in Patent Document 1, the stereo camera is only used for observing crops, and is not used for measuring the distance to surrounding objects because the mobile body flies autonomously.

  For this reason, in particular, in a mobile body that autonomously flies in a narrow space such as a room, there is a problem that it collides with surrounding objects due to a ranging error due to a gimbal tracking delay time.

  In order for a mobile body to fly autonomously without colliding with surrounding objects, it is necessary to fly while measuring the distance to the surrounding objects in real time using a three-dimensional measuring device (specifically, a stereo camera). . In order to measure the distance of surrounding objects, it is desirable that the angle of view of the stereo camera be as wide as possible so that the distance of a wide range of objects can be measured.

  However, as described above, when a stereo camera is attached to a gimbal device that is a video stabilization / follow-up device, a stable image can be obtained, but the camera does not follow the movement of the moving object in real time. The distance in the direction in which the camera is facing is measured instead of the direction in which the moving body is facing (the traveling direction).

  Therefore, as a result of intensive studies, the inventor has found a method for calculating the distance by adding the difference between the direction of the stereo camera and the direction of the moving object to the parallax obtained by the distance measuring unit including the stereo camera. For example, by detecting the tilt of the rocking axis of the gimbal in real time and calculating the distance information using the detection result, an accurate distance can be output in real time using a stereo camera attached to the moving body via the gimbal. it can. Thereby, the mobile body can fly autonomously without colliding with surrounding objects.

  DESCRIPTION OF SYMBOLS 1, 3 ... Mobile body apparatus 10, 10 '... Mobile body, 20 ... Gimbal, 20a ... Three-dimensional gyro sensor (detection system), 100 ... Ranging system, 315 ... CPU (processing part), 600b ... Imaging system, 520 ... Control operation unit (correction system), 523 ... Imaging control unit (luminance information acquisition unit), 528 ... Gyro unit (moving body posture detection unit), 600 ... Ranging / imaging unit (ranging system), 1000 ... Measurement Distance device (device).

JP 2014-62789 A

Claims (16)

A device mounted on a moving body,
A ranging system attached to the mobile body via a gimbal;
A detection system for detecting posture change information, which is information relating to a relative posture change between the distance measuring system and the moving body accompanying the movement of the moving body;
A correction system that corrects the measurement distance of the distance measurement system using the posture change information.   The apparatus according to claim 1, wherein the posture change information is information relating to an inclination of an optical axis of the distance measuring system with respect to a traveling direction of the moving body.   The apparatus according to claim 1, wherein the detection system includes at least one gyro sensor that measures inclination information of a swing shaft of the gimbal accompanying the movement of the moving body. The detection system is
A moving body posture detection unit for detecting the posture of the moving body;
The apparatus according to claim 1, further comprising a ranging system attitude detection unit that detects an attitude of the ranging system. The ranging system is
A plurality of imaging units;
A processing unit that generates a parallax image from the captured images of the plurality of imaging units,
The apparatus according to claim 1, wherein the detection system obtains the posture change information based on the parallax image.   The apparatus according to claim 5, wherein at least a part of the moving body falls within an angle of view in the plurality of imaging units.   The apparatus according to claim 6, wherein at least one mark is attached to the at least part.   The apparatus according to claim 5, further comprising a luminance information acquisition unit that acquires luminance information from captured images of the plurality of imaging units.   The apparatus according to claim 1, wherein the ranging system uses a stereo camera method, a TOF method, or a pattern projection method. An apparatus according to any one of claims 1 to 9,
A moving body apparatus comprising: a moving body to which a ranging system of the apparatus is attached via the gimbal. A distance measuring method using a distance measuring system attached to a moving body via a gimbal,
Measuring distance information to the object using the distance measuring system;
Detecting posture change information, which is information relating to a relative posture change between the distance measuring system and the moving body accompanying the movement of the moving body;
And a step of correcting the distance information using the posture change information.   12. The distance measuring method according to claim 11, wherein in the detecting step, tilt information of the swing axis of the gimbal accompanying the movement of the moving body is measured. The detecting step includes
A sub-step of detecting the posture of the moving body;
The distance measuring method according to claim 11, further comprising a sub-step of detecting an attitude of the distance measuring system. The ranging system includes a plurality of imaging units,
The distance measuring method according to claim 11, wherein in the measuring step, a parallax image is generated from the captured images of the plurality of imaging units, and distance information to an object shown in the captured image is calculated. In the plurality of imaging units, at least a part of the moving body falls within an angle of view,
15. The distance measuring method according to claim 14, wherein in the detecting step, the posture change information is obtained based on the parallax image. The distance measuring method according to claim 15, wherein at least one mark is attached to the at least part.