JP6801566B2 - Mobile robot - Google Patents

Description

The present invention relates to a mobile robot.

A distance image sensor is known that irradiates uniform light toward a target space, receives reflected light with an image sensor, and outputs the distance from the phase difference to the subject captured by each pixel (for example, Patent Document). See 1). There is also known a distance image sensor that irradiates a pattern light toward the target space, receives the reflected light with an image sensor, and outputs the distance to the subject captured by each pixel from the distortion and size of the pattern. ..

Japanese Unexamined Patent Publication No. 2006-153773

When the mobile robot tries to recognize the distance or direction of the transported object to be grasped from the output of the distance image sensor, the accuracy of the detection distance may be insufficient. One of the factors that reduce the accuracy of the detection distance is that the output for a certain distance includes a different offset for each pixel arranged two-dimensionally in the image sensor. If the image sensor is fixed to a stationary stage or the like so that it faces the reference plane, the distance image sensor can be calibrated with high accuracy. However, since it takes a considerable amount of work to attach and detach the distance image sensor mounted on the mobile robot, it is difficult to perform the calibration under the optimum conditions. Moreover, since the offset value changes over time, there is also a demand for calibration to be performed at a constant frequency.

The present invention has been made to solve such a problem, and an object of the present invention is to easily calibrate a distance image sensor mounted on a mobile robot.

The moving robot according to one aspect of the present invention includes a moving mechanism for moving, a distance image sensor that images the target space and outputs the distance to the subject for each pixel arranged in a two-dimensional shape, and the target space. A plane detection sensor that detects the orientation of the plane and the distance to at least one point on the plane, the output of the distance image sensor capturing the plane, the orientation and distance of the plane detected by the plane detection sensor, and the distance image sensor and the plane detection sensor. It is provided with a calibration unit that calibrates the distance image sensor using the relative positional relationship of the above, and the calibration unit executes the calibration a plurality of times by changing the distance to the plane by the moving mechanism.

In this way, the sensor mounted for use in a different application from the distance image sensor such as obstacle detection is used as a plane detection sensor, and the characteristic that the moving robot moves is utilized to take advantage of the distance image. The sensor is now calibrated at various distances. Therefore, it is not necessary to remove the distance image sensor from the mobile robot at the time of calibration, and highly accurate calibration can be performed at an appropriate frequency. That is, according to the present invention, the distance and direction to the object can be constantly and accurately detected by using the output of the distance image sensor.

It is external perspective view of the mobile robot which concerns on this embodiment. It is a control block diagram of a mobile robot. It is a figure which shows the output characteristic of the distance image sensor with respect to the plane of a certain distance. It is a figure which shows the output characteristic of a distance image sensor with respect to an inclined plane. It is a figure which shows an example of the calibration data of a certain pixel. It is a figure which shows the state of the calibration of the distance image sensor. It is a flow figure explaining the processing flow which concerns on calibration.

Hereinafter, the present invention will be described through embodiments of the invention, but the invention according to the claims is not limited to the following embodiments. Moreover, not all of the configurations described in the embodiments are indispensable as means for solving the problem.

FIG. 1 is an external perspective view of the mobile robot 100 according to the present embodiment. The mobile robot 100 includes, as a main structure, a carriage base 110 to which a moving mechanism is attached, an upper body base 120 to which sensors are attached, and an arm portion 150 for gripping a transported object or the like.

The bogie base 110 includes two drive wheels 112 and one caster 113 as a moving mechanism. The two drive wheels 112 are arranged so that the rotation shaft cores coincide with each of the opposite sides of the bogie base 110. Each drive wheel 112 is independently rotationally driven by a motor (not shown). The caster 113 is a trailing wheel, and a turning shaft extending in the vertical direction from the bogie base 110 is provided so as to support the wheel away from the rotation axis of the wheel, and follows the moving direction of the bogie base 110. To do. For example, if the two drive wheels 112 are rotated in the same direction at the same rotation speed, the mobile robot 100 goes straight, and if they are rotated in the opposite direction at the same rotation speed, the mobile robot 100 turns around a vertical axis passing through the center of gravity.

A control unit 190 is provided on the carriage base 110. The control unit 190 includes a control unit and a memory described later.

The upper body base 120 is supported by the carriage base 110 so that it can turn around a vertical axis with respect to the carriage base 110. The upper body base 120 is turned by a motor (not shown), and can face a predetermined direction with respect to the traveling direction of the carriage base 110.

The upper body base 120 is provided with various sensors for detecting obstacles and recognizing the surrounding environment. The 3D rider 130 is one of the sensors and is arranged in front of the upper body base 120. The 3D rider 130 is an obstacle detection sensor that measures the distance to an object existing in the vicinity. Specifically, the 3D lidar 130 drives, for example, a micromirror to scan the laser beam in a range of 120 degrees horizontally and 60 degrees vertically in the front direction of the upper body base 120. Then, the distance to the reflection point is calculated by detecting the reflected light with respect to the scanned projected light.

The distance image sensor 140 is also arranged in front of the upper body base 120. The distance image sensor 140 and the 3D rider 130 may be housed in one housing, and the housing may be arranged in front of the upper body base 120. The distance image sensor 140 is used to recognize the distance, shape, direction, etc. of the transported object, for example, when the hand of the arm portion 150 grips the transported object. The distance image sensor 140 includes an image sensor in which pixels that photoelectrically convert an optical image incident from the target space are arranged in a two-dimensional manner. The distance image sensor 140 outputs the distance to the subject for each pixel. Specifically, the distance image sensor 140 includes an irradiation unit that irradiates the target space with pattern light, receives the reflected light with an image sensor, and captures a subject captured by each pixel based on the distortion and size of the pattern in the image. Outputs the distance to.

A distance sensor that captures an image of the target space and outputs the distance to the subject for each pixel can be adopted as the distance image sensor 140. For example, it may be a distance sensor that irradiates uniform light toward the target space, receives the reflected light by the image sensor, and outputs the distance from the phase difference to the subject captured by each pixel. However, the distance image sensor targeted in the present embodiment can acquire highly accurate distance information by performing calibration described later.

The arm portion 150 is mainly composed of a plurality of arms and a hand. One end of the arm is pivotally supported by the upper body base 120. The other end of the arm pivotally supports the hand. When the arm unit 150 is driven by an actuator (not shown), it executes a gripping operation according to a given task, such as gripping a transported object.

FIG. 2 is a control block diagram of the mobile robot 100. The control unit 200 is, for example, a CPU, and exchanges information such as commands and sampling data with the drive wheel unit 210, the arm unit 220, the swivel unit 230, the memory 240, the 3D rider 130, the distance image sensor 140, and the like. Therefore, various calculations related to the control of the mobile robot 100 are executed.

The drive wheel unit 210 is provided on the bogie base 110, and includes a drive circuit for driving the drive wheels 112, a motor, an encoder for detecting the amount of rotation of the motor, and the like. The drive wheel unit 210 functions as a moving mechanism for autonomous movement. The control unit 200 executes rotation control of the motor by sending a drive signal to the drive wheel unit 210. In addition, by receiving the detection signal of the encoder, the moving speed, moving distance, turning angle, etc. of the moving robot 100 are calculated.

The arm unit 220 is provided in the arm unit 150, and includes a drive circuit and an actuator for driving the arm and the hand, an encoder for detecting the amount of movement of the actuator, and the like. The control unit 200 operates the actuator by sending a drive signal to the arm unit 220, and executes attitude control and grip control of the arm unit 150. Further, by receiving the detection signal of the encoder, the operating speed, operating distance, posture, etc. of the arm unit 150 are calculated.

The swivel unit 230 is provided so as to straddle the bogie base 110 and the upper body base 120, and includes a drive circuit for swiveling the upper body base 120, a motor, an encoder for detecting the amount of rotation of the motor, and the like. By sending a drive signal to the swivel unit 230, the control unit 200 can operate the motor and, for example, direct the 3D lidar 130 and the distance image sensor 140 in a specific direction.

The memory 240 is a non-volatile storage medium, and for example, a solid state drive is used. The memory 240 stores a control program for controlling the mobile robot 100, various parameter values used for control, a function, a look-up table, and the like. In particular, the calibration data 241 described later is stored in the distance image sensor 140.

Upon receiving the control signal from the control unit 200, the 3D lidar 130 drives the micromirror to scan the laser beam, continuously detects the reflected light, calculates the distance to each reflection point, and calculates the distance to each reflection point. Return to. When the distance image sensor 140 receives the control signal from the control unit 200, the distance image sensor 140 irradiates the pattern light from the irradiation unit, drives the image sensor to acquire the image data, and the distortion and size of the pattern from the image of the image data. Is analyzed and the distance information corresponding to each pixel is returned to the control unit.

The control unit 200 also plays a role as a function execution unit that executes various calculations and controls related to control. As will be described in detail later, the calibration unit 201 uses the 3D lidar 130 to calibrate the distance image sensor 140 and executes a control program for creating calibration data 241. The movement planning unit 202 plans the route to which the mobile robot 100 moves by using the map information stored in the memory 240, the obstacle information obtained from the sensor output, and the like.

FIG. 3 is a diagram showing the output characteristics of the distance image sensor 140 with respect to a plane of a certain distance. The square wave on the left side indicates the effective area of the image sensor included in the distance sensor 115. In the effective region, as described above, the pixels as the photoelectric conversion unit are arranged in a matrix. Of these, a horizontal line located above is referred to as an a line, a horizontal line located in the center is referred to as a b line, and a horizontal line located below is referred to as a c line, and will be described as representative lines of the effective area. .. Further, in each line, the representative positions of the left side, the center, and the right side are L, C, and R, respectively.

The table on the right side of FIG. 3 shows the case where the image sensor faces the target plane at a distance of 0.5 m, the case where the image sensor faces the target plane at a distance of 1.0 m, and the case where the image sensor faces the target plane at a distance of 1.5 m. The outputs of the a line, the b line, and the c line when facing each other at a distance of 0 m are shown. Here, when the image sensor faces a plane, it means that the plane is captured by the entire effective region of the image sensor, and the plane is parallel to the light receiving surface of the image sensor.

In each graph in the table on the right side of FIG. 3, the horizontal axis represents the horizontal relative position of the pixel in each line, and the vertical axis represents the output distance (m) output from each pixel. For example, looking at the output of the a-line when the image sensor faces the target plane at a distance of 0.5 m, although the output is almost correct 0.5 m at position L and position R, At position C, a distance smaller than 0.5 is output. Looking at the output of the entire a-line, it shows a downwardly convex curved line (thick solid line) with the position C at the center as the base, with respect to the 0.5 m line shown by the dotted line. Also, looking at the output of the b line when the image sensor faces the target plane at a distance of 2.0 m, it exists near the middle of position L and position C and near the middle of position C and position R. Although each pixel outputs an almost correct 2.0 m, the other pixels output a distance larger than 2.0 m as a whole.

That is, the output of the distance image sensor 140 includes different offset values for each pixel and for each distance to the detection target even if the pixels are the same. Therefore, in order to obtain more accurate distance information from the output of the distance image sensor 140, it is necessary to calibrate the distance image sensor 140. Specifically, the calibration unit 201, which is the function execution unit of the control unit 200, creates calibration data 241 for calibrating the output value for each pixel.

As shown in the table of FIG. 3, if the distance to the target plane is known in advance, a conversion formula or conversion table for converting the distance output by each pixel into the correct distance may be created. .. For example, if the pixel at the a-line position C outputs 0.4 m when the image sensor faces a plane of 0.5 m, if 0.4 m is output as calibration data for this pixel, it is 5/4. You can create a conversion formula that doubles.

In the example of FIG. 3, three lines are shown as representatives, but calibration data may be created for all lines in the effective area, or calibration data may be created for each line at regular intervals, and the thinned lines may be used. The created calibration data may be interpolated and used. Further, in each line for creating calibration data, calibration data may be created for all pixels, calibration data is created for each pixel at regular intervals, and the created calibration data is interpolated for the thinned pixels. You may use it.

In the example of FIG. 3, an example in which a plane of 0.5 m, 1.0 m, 1.5 m, and 2.0 m is imaged as a target, respectively, is shown, but of course, the range and the interval are not limited. The range may be changed according to the purpose of use of the distance image sensor 140, or may be a fine interval such as 10 mm depending on the required accuracy.

Further, the plane to be imaged at the time of calibration does not have to be the plane facing the image sensor. FIG. 4 is an example of the output characteristics of the distance image sensor 140 when an image is taken on a plane tilted with respect to the image sensor. Specifically, the distance to the plane on the line where the image sensor is located is 0.4 m from the pixel at position L, 0.6 m from the pixel at position C, and 0.8 m from the pixel at position R. know. A conversion formula or conversion table may be created in which the output (thick solid line) obtained by imaging such a plane is matched with the correct distance (dotted line) for each pixel. That is, if the orientation of the target plane to be imaged by the image sensor and the distance to at least one point included in the target plane are correctly acquired in advance, the distance from each pixel to the target plane can be calculated. Calibration data for each pixel can be created.

FIG. 5 is a diagram showing an example of calibration data of a specific pixel. The horizontal axis represents the output distance of the target pixel, and the vertical axis represents the calibration distance.

In the target pixel, the output distances d ₁ m, d ₂ m, d ₃ m, d with respect to a plane having correct distances of 0.5 m, 1.0 m, 1.5 m, and 2.0 m, respectively, as in the example of FIG. _{When 4} m is obtained, plot data represented by black dots is obtained. When these plot data are smoothly connected using a polynomial function, a curve as shown by a thick line is obtained. That is, the polynomial function representing the curve becomes the conversion formula as the calibration data. The fitting function may be another function instead of the polynomial function. Also, each plot data does not necessarily have to be a value on the fitting function. When the calibration data is created as a conversion table instead of a conversion formula, for example, a lookup table that numerically represents the set of the corresponding output distance and the calibration distance can be adopted.

Next, the procedure for calibrating the distance image sensor 140 in the present embodiment will be described. FIG. 6 is a diagram showing a state of calibration of the distance image sensor 140. Here, it is assumed that the wall surface W exists in the environment space in which the mobile robot 100 autonomously moves.

The control unit 200 recognizes that the wall surface W is a flat surface, its size, and its orientation from the measurement results obtained by scanning the wall surface W by the 3D rider 130. Then, when the wall surface W is imaged by the image pickup element of the distance image sensor 140, it is determined whether or not the entire effective region of the image pickup element can capture the wall surface W. That is, it is determined whether or not an object other than the wall surface W is reflected in the effective region of the image sensor. If it is determined that the image is not reflected, the calibration work using the wall surface W is started.

The 3D rider 130 and the distance image sensor 140 are arranged on the upper body base 120 with their positions and postures strictly determined relative to each other. Therefore, the control unit 200 can convert the distance obtained by measuring the object with the 3D rider 130 into the distance from the distance image sensor 140 to the object by using the geometric condition according to the arrangement. That is, if the distance to the wall surface W is measured by the 3D rider 130, the distance that should be obtained as the measurement result of the distance image sensor 140, in other words, the calibrated distance can be calculated.

As described above, the calibration work can be performed even if the wall surface W and the light receiving surface of the image sensor are not parallel to each other, but in order to speed up the calculation, here, the orientation of the wall surface W obtained from the output of the 3D lidar 130. The upper body base 120 is swiveled with reference to, so that the wall surface W and the light receiving surface of the image sensor face each other. Then, while driving the drive wheels 112 to approach the wall surface perpendicularly, the accurate distance between the distance image sensor 140 and the wall surface W is calculated using the output of the 3D rider 130, and imaging is performed at regular distance intervals. To do.

By changing the distance to the wall surface W in this way and repeatedly acquiring the exact distance and the output distance of each pixel at that time, the calibration data 241 as described with reference to FIG. 5 for each target pixel. Can be created. The completed calibration data 241 is stored in the memory 240. The calibration data 241 stored in the memory 240 is referred to when the distance image sensor 140 is used, and the distance output from the distance image sensor 140 is converted into a more accurate value.

As described above, if the distance image sensor 140 is calibrated using the movement mechanism of the mobile robot 100, the trouble of removing the distance image sensor 140 from the mobile robot 100 can be saved. Further, it is easy to repeat the calibration work at an appropriate frequency, and the calibration data can be appropriately updated in response to the change over time of the offset. That is, according to the present embodiment, the mobile robot 100 can constantly and accurately detect the distance and direction to the object by using the output of the distance image sensor 140.

In the above, an example in which the mobile robot 100 performs only the calibration work without performing other tasks has been described, but even if the calibration work is performed intermittently during the execution of the other tasks to complete the calibration data. good. FIG. 7 is a flow chart illustrating a processing flow when a process related to calibration is executed while another task is being executed. The flow in the figure describes the main processes related to the calibration work that the mobile robot 100 executes in parallel during the execution of other tasks.

In step S101, the control unit 200 sends a control signal to the drive wheel unit 210 so that the movement planning unit 202 moves along the planned route. The control unit 200 acquires the output of the 3D rider 130 that continuously executes the distance measurement while scanning the surroundings while moving (step S102).

The control unit 200 confirms from the output of the 3D rider 130 whether or not there is a flat surface having a predetermined area around it (step S103). Here, the predetermined area is an area in which an object other than the plane is not reflected in the effective area of the image sensor of the distance image sensor 140. Therefore, the width of the plane also depends on the distance from the image sensor to the plane, the angle of view of the imaging optical system, and the like. If the control unit 200 cannot find such a plane, the control unit 200 returns to step S101. If it can be found, the process proceeds to step S104.

In step S104, the calibration unit 201 of the control unit 200 confirms whether or not the calibration data corresponding to the distance from the distance image sensor 140 to the plane has been acquired. If it has already been acquired, the process returns to step S101. If it has not been acquired yet, the process proceeds to step S105.

In step S105, the calibration unit 201 causes the distance image sensor 140 to acquire image data of the plane. Then, in step S106, a calibration value for converting the distance output from each pixel of the image sensor into a correct distance is calculated. Subsequently, the process proceeds to step S107, and it is determined whether or not the acquisition of the calibration value is completed within a predetermined distance range and interval. That is, it is determined whether or not the calibration data is completed. If it is not completed, the process returns to step S101, and if it is completed, a series of calibration operations are completed to concentrate on the execution of the task. The calculation of the calibration value in step S106 may be collectively executed after acquiring the output value of each pixel in all the predetermined distance ranges and intervals.

In the present embodiment described above, the 3D lidar 130 is adopted as the plane detection sensor capable of detecting the orientation of the plane included in the target space and the distance to at least one point of the plane, but another sensor is adopted as the plane detection sensor. You may. For example, it may be a stereo camera unit provided with two image pickup elements for capturing a color image. Also, if a specific two-dimensional code is attached to the plane side, even if it is a code reader that can capture the two-dimensional code and detect the distance and direction from the size and inclination to the plane. good.

100 mobile robot, 110 trolley base, 112 drive wheels, 113 casters, 120 upper body base, 130 3D rider, 140 distance image sensor, 150 arm part, 190 control unit, 200 control part, 201 calibration part, 202 movement planning part, 210 drive wheel unit, 220 arm unit, 230 swivel unit, 240 memory, 241 calibration data

Claims (1)

A movement mechanism for moving and
A distance image sensor that captures the target space and outputs the distance to the subject for each pixel arranged in two dimensions,
A plane detection sensor that detects the orientation of a plane included in the target space and the distance to at least one point on the plane.
Calibration of the distance image sensor is performed using the output of the distance image sensor imaging the plane, the orientation and distance of the plane detected by the plane detection sensor, and the relative positional relationship between the distance image sensor and the plane detection sensor. Equipped with a calibration unit to perform
The calibration unit is a mobile robot that executes the calibration a plurality of times by changing the distance to the plane by the moving mechanism.

Images