WO2021241482A1

WO2021241482A1 - Route detection device

Info

Publication number: WO2021241482A1
Application number: PCT/JP2021/019562
Authority: WO
Inventors: 祐一小林
Original assignee: 国立大学法人静岡大学
Priority date: 2020-05-25
Filing date: 2021-05-24
Publication date: 2021-12-02
Also published as: JPWO2021241482A1

Abstract

The purpose of an embodiment of the present invention is to highly accurately detect routes in various environments. A route detection system 1 is provided with: a camera 2 that acquires a two-dimensional color picture reflecting a two-dimensional image in a visual field and acquires depth information of an image corresponding to each pixel in the two-dimensional color picture; and a computer 3 that processes the two-dimensional color picture and the depth information. The computer 3 detects the positions of pixels of a line segment image which is an image approximated by line segments on the two-dimensional color picture, identifies the depth corresponding to the positions of the pixels of the line segment image, calculates the height and the slope of the line segment image in a three-dimensional space on the basis of the positions of the pixels of the line segment image and the depth corresponding to the positions, and detects a route candidate in the three-dimensional space on the basis of the height and the slope.

Description

Route detector

The embodiment relates to a route detection device that detects a route.

Conventionally, a device for detecting a feature point based on a detection result of a sensor such as a camera has been known for the purpose of robot control or the like. For example, a device that detects a plane on three-dimensional coordinates based on a distance image detected by a distance image sensor (see Patent Document 1 below), or a corner of a contour line that processes an image taken by a television camera. A device for detecting a feature point of the above (see Patent Document 2 below) is known.

Japanese Unexamined Patent Publication No. 2014-85940 Japanese Unexamined Patent Publication No. 6-139357

In the conventional detection method in the devices and the like described in Non-Patent Document 1 and Non-Patent Document 2 described above, it is desired to accurately detect a route in various environments regardless of whether it is outdoors or indoors.

The present embodiment has been made in view of the above problems, and an object thereof is to provide a route detection device capable of detecting a route with high accuracy in various environments.

In order to solve the above problems, the path detection device according to one embodiment of the present disclosure acquires a two-dimensional image reflecting a two-dimensional image in the visual field and depth information of the image corresponding to each pixel of the two-dimensional image. The acquisition device is provided with at least one processor for processing the two-dimensional image and the depth information, and the at least one processor detects the position of the line image, which is an image similar to the line on the two-dimensional image. , The depth corresponding to the position of the line image is specified from the depth information, and the height and inclination of the line image in the three-dimensional space are determined based on the position and the depth corresponding to the position of the line image. Calculate and detect path candidates in 3D space based on height and tilt. The "line segment" of the "line segment image" here is not limited to a straight line, but is a concept that includes a wide range of curves, polygonal lines, and the like.

According to the above aspect, the position of the line segment image is detected on the two-dimensional image acquired by the acquisition device, and the depth corresponding to the position of the line segment image is specified based on the depth information acquired by the acquisition device. , The height and inclination of the line segment image in the three-dimensional space are calculated from the position of the line segment image and the corresponding depth, and the candidate path in the three-dimensional space is specified based on them. At this time, by determining the height and inclination of the line segment image according to the environment in which the subject exists, it is possible to detect the route candidate according to the environment with high accuracy.

According to the embodiment, the route can be detected with high accuracy under various environments.

It is a figure which shows the schematic structure of the route detection system which concerns on one preferred embodiment of this invention. It is a block diagram which shows the hardware configuration of the computer of FIG. It is a flowchart which shows the procedure of the route detection method which concerns on one preferred embodiment of this invention. It is a figure which shows the position of the 2D color image acquired by the camera 2 and the line segment image detected by the computer 3 on the image. It is a graph which shows the distribution of the height and the inclination of a plurality of line segment images calculated by a computer 3. It is a figure which shows the output example of the route candidate by a computer 3. It is an external view which shows the schematic structure of the robot 4 which shows the application example of this embodiment. It is a figure which shows the 2D color image acquired by a camera 2, and the position of a route candidate detected by a computer 3 on an image. It is a figure which shows the 2D color image acquired by a camera 2, and the position of a route candidate detected by a computer 3 on an image.

Hereinafter, a preferred embodiment of the route detection device according to the present invention will be described in detail with reference to the drawings. In the description of the drawings, the same or corresponding parts are designated by the same reference numerals, and duplicate description will be omitted.

The route detection system 1, which is a preferred embodiment of the present invention shown in FIG. 1, is a computer system that detects movement routes such as roads, sidewalks, passages, and stairs in the surrounding environment for the purpose of robot control and the like. .. The route detection system 1 includes a camera 2 which is an acquisition device for acquiring an image of an external environment in a visual field, and a computer 3 for processing the image.

The camera 2 has a built-in ranging camera 2a and a color camera 2b. The camera 2 is controlled by the computer 3 to operate the color camera 2b to capture a two-dimensional image in the field of view and acquire a two-dimensional color image reflecting the image, and the camera 2 is controlled by the computer 3 for measurement. It has a function of operating the distance camera 2a to measure the distance from the camera 2 corresponding to the position of the two-dimensional image in the field of view and acquiring the depth information for each pixel of the image. For example, the camera 2 acquires data including brightness information of each pixel for each color component of RGB as a two-dimensional color image, and at the same time acquires a depth image showing depth information for each pixel. The distance measurement function for obtaining a depth image is, for example, using the principle of triangulation using infrared rays with a stereo camera, irradiating an infrared pattern and measuring the distance from the change in the pattern, and TOF using infrared rays. (Time Of Flight) It can be realized by using a camera or by measuring the distance using a radar.

The computer 3 is a data processing device that detects the position in the three-dimensional space of a candidate path in the field of view of the camera 2 by processing the two-dimensional color image and the depth image acquired by controlling the camera 2. .. Here, in the present embodiment, the computer 3 is composed of one device, but may be composed of a plurality of devices, and these plurality of devices are connected to each other via a wired or wireless network. It may be configured to be connected to enable data communication.

FIG. 2 shows the hardware configuration of the computer 3. As shown in FIG. 2, the computer 3 is physically a CPU (Central Processing Unit) 101 which is a processor, a RAM (Random Access Memory) 102 or a ROM (Read Only Memory) 103 which is a recording medium, and a communication module 104. , And a computer or the like including the input / output module 106 and the like, each of which is electrically connected. The computer 3 may include a display, a keyboard, a mouse, a touch panel display, and the like as the input / output module 106, and may include a data recording device such as a hard disk drive and a semiconductor memory. Further, the computer 3 may be composed of a plurality of computers.

Returning to FIG. 1, the functional configuration of the computer 3 will be described. The computer 3 includes an image conversion unit 31, an edge detection unit 32, a Hough conversion unit 33, a two-dimensional position calculation unit 34, a mapping unit 35, a three-dimensional position calculation unit 36, and a route evaluation unit 37. Each functional unit of the computer 3 shown in FIG. 1 operates the communication module 104, the input / output module 106, etc. under the control of the CPU 101 by loading the program on the hardware such as the CPU 101 and the RAM 102. , Is realized by reading and writing data in the RAM 102. By executing this computer program, the CPU 101 of the computer 3 causes the computer 3 to function as each functional unit in FIG. 1, and sequentially executes processes corresponding to the route detection method described later. Various data necessary for executing this computer program and various data generated by executing this computer program are all stored in an internal memory such as ROM 103 and RAM 102, or a storage medium such as a hard disk drive.

The image conversion unit 31 acquires a two-dimensional color image and a depth image by controlling the operation of the camera 2, and converts the resolution as a parameter when acquiring the two-dimensional color image into a plurality of types to convert the resolution. Generate a 2D color image. For example, the image conversion unit 31 converts the resolution into a resolution of 640 pixels × 480 pixels and a resolution of 128 pixels × 96 pixels to generate a two-dimensional color image having a plurality of resolutions. However, the type and number of resolutions converted by the image conversion unit 31 can be arbitrarily set. Further, the image conversion unit 31 grayscales each of the two-dimensional color images having a plurality of resolutions to generate a two-dimensional grayscale image having a plurality of resolutions. As a method of grayscale conversion, a method of extracting any one of the RGB color components of a two-dimensional color image, a method of weighting and averaging the brightness of the RGB color components by a predetermined weight, and the like are adopted.

The edge detection unit 32 performs edge detection on the two-dimensional grayscale image generated by the image conversion unit 31, and outputs the two-dimensional coordinates of the points detected as edges on the two-dimensional grayscale image. The edge detection unit 32 uses, for example, the Canny method as an edge detection method.

The Hough transform unit 33 detects a plurality of positions of a line segment image that is approximated to a line segment on a two-dimensional grayscale image, targeting the two-dimensional coordinates of the edge output from the edge detection unit 32. The Hough transform unit 33 detects the position of the line segment image by using, for example, a stochastic Hough transform method, and outputs the two-dimensional coordinates of both ends of the detected line segment image.

The two-dimensional position calculation unit 34 obtains a mathematical formula representing the position of the line segment image based on the two-dimensional coordinates at both ends of the line segment image output from the Hough transform unit 33. For example, in the two-dimensional position calculation unit 34, the two-dimensional coordinates at both ends of the line segment image are (x ₁ , y ₁ ), (x ₂ , y ₂ ) in the xy coordinate system with respect to the image plane of the camera 2. If it is detected, the formula that expresses the position of the line segment image is the following formula;
y = (y _2- y ₁ ) {(x-x ₁ ) / (x _2- x ₁ )} + y ₁
Expressed as. _{Then, in the case of x 1} ≠ x ₂ , the two-dimensional position calculation unit 34 changes x from x ₁ _{to x 2} in the above equation to change x of all the pixels of the line segment image on the xy coordinates. Find the coordinates and y-coordinates. On the other hand, in _{the case of x 1} = x ₂ , the two-dimensional position calculation unit 34 sets x = x ₁ and changes y from y ₁ _{to y 2} , so that all the line segment images on the xy coordinates are displayed. The x-coordinate and y-coordinate of the pixel are obtained.

The mapping unit 35 corresponds to the positions (x-coordinates, y-coordinates) of the pixels of the line segment image on the two-dimensional coordinates obtained by the two-dimensional position calculation unit 34 from the depth image. Extract the depth information of the pixel position. For example, in the mapping unit 35, the depth information included in the depth image is represented by the value of the z coordinate of the xyz three-dimensional coordinate system, which is the distance from the xy plane which is the image plane of the two-dimensional color image. In this case, the value of the z coordinate is extracted and specified as a value indicating the depth of the position of the pixel of the line segment image. Then, the mapping unit 35 obtains a combination of the pixel position (x coordinate, y coordinate) of the line segment image on the two-dimensional coordinate and the depth information (z coordinate) of the pixel by the two-dimensional position calculation unit 34. It is generated and output for all the generated pixels.

The three-dimensional position calculation unit 36 is a predetermined three-dimensional space based on the positions (x-coordinates, y-coordinates) of the pixels of the line segment image output from the mapping unit 35 and the depth (z-coordinates) corresponding to the pixels. Calculate the position of the line image within. For example, the three-dimensional position calculation unit 36 uses a predetermined three-dimensional coordinate system (a three-dimensional coordinate system with a horizontal plane as an xy plane and an xy plane with the ground) based on the positions of the pixels of the line segment image and the corresponding depth. Calculate the coordinates of the line image in the 3D space of the 3D coordinate system, etc.). Then, the three-dimensional position calculation unit 36 outputs information on the position of the line segment image in the three-dimensional space. The information of the output position may be the coordinate information of the start point and the end point of the line segment image in the three-dimensional space, or an approximate expression of a straight line, a curve, or a broken line that approximates the line segment image in the three-dimensional space. It may be the information of the above, or it may be the coordinate information of each pixel of the line segment image in the three-dimensional space.

The route evaluation unit 37 obtains information on the positions of the plurality of line image calculated by the three-dimensional position calculation unit 36 in the three-dimensional space from the plurality of line images detected by the Huff conversion unit 33. Based on this, the candidate route in the three-dimensional space is detected. Specifically, the path evaluation unit 37 calculates the height H [m] and the slope θ [degree] from a predetermined plane (horizontal plane, ground, etc.) of a plurality of line segment images in a three-dimensional space. The height H may be calculated as the height of the center of gravity of the line segment image, or may be calculated as the height of the start point or the end point of the line segment image. Further, the slope θ may be calculated as the slope of the tangent line at the center of gravity, the start point, or the end point of the line segment image. Then, the path evaluation unit 37 compares the height H and the slope θ of the plurality of line segment images with the first threshold value and the second threshold value, respectively, and based on the result, the path evaluation unit 37 selects the third order from the plurality of line segment images. Extract route candidates in the original space. For example, 0 m and 0.6 m are preset as the first threshold value, and 0 degree and 10 degree are preset as the second threshold value, and the route evaluation unit 37 uses the following equation;
0 degree ≤ θ ≤ 10 degrees and 0 m ≤ H ≤ 0.6 m
The position of the line segment image that satisfies the condition is extracted as the position of the candidate route.

Further, the route evaluation unit 37 outputs information on the positions of the extracted route candidates. As the output mode, the position of the route candidate is highlighted on the two-dimensional color image acquired by the camera 2 and displayed on a display or the like, and the route candidate information is added to the map data or the like. Examples thereof include a mode of outputting to an internal or external recording medium, and a mode of outputting information on the three-dimensional position of a route candidate to an internal or external control unit that performs robot control or the like.

Next, the procedure of the route detection method executed by the route detection system 1 described above will be described. FIG. 3 is a flowchart showing a procedure for processing the route detection method. The route detection process shown in FIG. 3 is started by the operation of the user of the route detection system 1, and is repeatedly executed at a periodic timing, a timing when a predetermined state is detected by various sensors, or the like.

First, referring to FIG. 3, when the route detection process is activated, the camera 2 is controlled by the computer 3 to acquire a two-dimensional color image in the field of view and a depth image in the field of view (step S1). Next, the image conversion unit 31 of the computer 3 converts the acquired two-dimensional color image into a two-dimensional grayscale image after being converted to a predetermined resolution (step S2).

After that, the edge detection unit 32 of the computer 3 performs edge detection on the two-dimensional grayscale image, and outputs the two-dimensional coordinates of the points detected as edges (step S3). Then, the Hough transform unit 33 of the computer 3 executes a probabilistic Hough transform on the two-dimensional coordinates of the points detected as edges, thereby detecting a plurality of positions of the line segment image (step S4). After that, the two-dimensional position calculation unit 34 of the computer 3 calculates the two-dimensional coordinates of all the pixels of the plurality of line segment images based on the positions of the plurality of line segment images (step S5).

Next, the mapping unit 35 of the computer 3 outputs the depth information in association with the two-dimensional coordinates of the pixels of the plurality of line segment images based on the depth image (step S6). Further, the three-dimensional position calculation unit 36 of the computer 3 calculates the positions (three-dimensional coordinates) of the plurality of line segment images in a predetermined three-dimensional space (step S7). After that, the path evaluation unit 37 of the computer 3 calculates the height and slope of the line segment image from the predetermined plane in the three-dimensional space, and by comparing the calculation result with the predetermined threshold value, a plurality of lines are obtained. The position of the candidate route is extracted from the position of the line segment, and the position of the extracted candidate route is output in a predetermined output mode (step S8). Further, the computer 3 determines whether or not to convert the two-dimensional color image to the next resolution (step S9), and when converting to the next resolution (step S9; Yes), the two-dimensional color image is converted to the next resolution. When the processes of steps S2 to S8 are repeated for the two-dimensional color image and the conversion to all resolutions is completed (step S9; No), the route detection process is terminated.

Next, with reference to FIGS. 4 to 6, an example of the result of the route detection process according to the above-described embodiment is shown. In FIG. 4, the part (a) shows an example of a two-dimensional color image acquired by the camera 2, and the part (b) shows an example of the position of the line segment image detected by the computer 3. FIG. 5 is a graph showing the distribution of heights and inclinations of a plurality of line segment images calculated by the computer 3, and FIG. 6 is a diagram showing an output example of a route candidate by the computer 3.

As shown in these figures, according to the path detection process of the present embodiment, after the two-dimensional positions of a plurality of line segment images on the image are detected for the two-dimensional color image acquired by the camera 2. , The height and inclination of the plurality of line segment images with respect to a predetermined surface in the three-dimensional space are calculated (FIGS. 4 and 5). Here, in the part (b) of FIG. 4, the two-dimensional positions of the detected line segment images are shown by a large number of white lines, but the white lines have a predetermined length or more and are considered as route candidates. The line segment image is represented by a thick white line. At this stage, some line segment images that cannot be paths (for example, vertical line segment images) are also displayed as thick white lines. Then, a route candidate is extracted from the plurality of line segment images based on the calculated height and inclination of the line segment image, and the position is output. For example, from the distribution of the height and slope of the line segment image shown in FIG. 5, the height H [m] and the slope θ [degree] of the line segment image in the three-dimensional space are, for example, 0 degrees ≤ θ. A line segment image within a range (a range surrounded by a circle) satisfying ≦ 10 degrees and 0 m ≦ H ≦ 0.6 m is extracted as a route candidate. The result is reflected in the three-dimensional space, and as shown in FIG. 6, two line segment images CR on both sides of the passage (colors different from white on the actual display, etc., and other line segment images as route candidates). (Displayed so that it can be distinguished from) is extracted and output as a route candidate.

FIG. 7 shows a schematic configuration of a robot 4 showing an application example of this embodiment. In addition to the camera 2 and the computer 3 as the route detection system 1, the robot 4 further includes a drive mechanism 5 that enables the autonomous movement of the entire robot 4 under the control of the computer 3. The computer 3 controls autonomous movement by the drive mechanism 5 based on the positions of the route candidates output by the route evaluation unit 37. The drive mechanism 5 includes, for example, a wheel, a steering unit, a drive unit for the wheel or the steering unit, a drive power source such as a battery, and the like.

The operation and effect of the route detection system 1 of the above-described embodiment will be described.

According to the present embodiment, the position of the line image is detected on the two-dimensional color image acquired by the camera 2, and the depth corresponding to the position of the pixel of the line image is based on the depth information acquired by the camera 2. Is specified, and the height and inclination of the line image with respect to a predetermined surface in the three-dimensional space are calculated from the positions of the pixels of the line image and the corresponding depth, and based on these, the height and inclination in the three-dimensional space are calculated. Candidates for the route in are identified. At this time, by determining the height and inclination of the line segment image in the three-dimensional space according to the environment in which the subject exists, it is possible to detect the route candidate according to the environment with high accuracy. For example, when you want to control a robot that moves along a passage in a building or an outdoor sidewalk, you can detect a route suitable for the robot to move, and you want to control to move the wall surface or pillar of an outdoor building. A suitable route can be detected.

Here, the computer 3 repeatedly detects route candidates for the two-dimensional image whose resolution has been changed while changing the resolution of the two-dimensional color image to a plurality of. In this case, by using a two-dimensional image whose resolution is changed to be suitable for an environment such as outdoor or indoor, it is possible to detect a route candidate according to the environment with higher accuracy. For example, when the change in shade of an image such as indoors tends to be linear, the path can be detected with high accuracy using an image with higher resolution, and the change in shade in an image such as outdoors tends to be random. Can detect the path with high accuracy using an image with reduced resolution.

For example, FIGS. 8 and 9 show examples of detection results based on two-dimensional color images acquired at a plurality of resolutions in an outdoor environment. Part (a) of FIG. 8 shows a two-dimensional color image acquired at a resolution of 640 pixels × 480 pixels and a frame rate of about 5 fps, and part (b) of FIG. 8 shows this two-dimensional color image. The detection result of the candidate of the route used is shown. Further, the part (a) of FIG. 9 shows a two-dimensional color image acquired at a resolution of 128 pixels × 96 pixels and a frame rate of about 15 fps, and the part (b) of FIG. 9 shows the two-dimensional color. The detection result of the route candidate using an image is shown. In the part (b) of FIG. 8 and the part (b) of FIG. 9, the positions extracted as route candidates from the line segment images shown by the thick white lines are indicated by the reference numerals CR. As this result shows, in an environment where straight lines are difficult to see in an image, such as outdoors, only one route candidate is detected in a high-resolution image, whereas only one clear boundary part of a paved road is detected. In the low-resolution image, three route candidates are detected, including the vague boundary of the paved road. In this way, highly accurate route detection is realized according to the appearance of the route in the image depending on the environment.

Further, the computer 3 detects the position of the line segment image by grayscale-converting the two-dimensional color image to generate a two-dimensional grayscale image and detecting the edge of the two-dimensional grayscale image. By doing so, the processing load of the edge detection processing from the two-dimensional image is reduced, and as a result, the processing load of the entire route detection can be reduced.

Further, the computer 3 detects the position of the line segment image by performing a probabilistic Hough transform after performing edge detection of the two-dimensional grayscale image. In this case, the line segment image can be detected without omission regardless of the resolution of the two-dimensional color image, and as a result, the path can be detected with high accuracy.

Further, the computer 3 is a candidate for a route in the three-dimensional space based on the comparison result between the height of the line segment image and the first threshold value and the comparison result between the inclination of the line segment image and the second threshold value. Is being detected. In this case, by setting the threshold value according to the environment such as outdoor or indoor, it is possible to detect the route candidate according to the environment with higher accuracy.

Further, according to the configuration of the robot 4 provided with the above-mentioned route detection system 1, highly accurate autonomous movement control is realized by using the route detection result.

The present invention is not limited to the above-described embodiment. The configuration of the above embodiment can be changed in various ways.

For example, the computer 3 of the above embodiment has detected a candidate for a repetitive route in a two-dimensional color image converted into a plurality of resolutions, but the processing is not limited to such a process. The computer 3 uses some of the color components of the RGB color components of the two-dimensional color image, grayscale-converts the color components to generate a two-dimensional grayscale image, and extracts the two-dimensional color image from the two-dimensional color image. While changing the color component, the route candidate may be repeatedly detected for the image in which the brightness of each pixel of the extracted color component is converted into a grayscale image. This color component is a parameter for acquiring a two-dimensional grayscale image to be processed. In this case, by using the image data of the color component suitable for the environment such as outdoor or indoor, it is possible to detect the route candidate according to the environment with higher accuracy.

Further, the first and second threshold values used in the computer 3 for detecting the route candidate may be changed to arbitrary values. For example, when detecting a path in a horizontal plane, each threshold value and judgment criteria can be set on the condition that the height of the line segment image is close to the wheel position of the robot and the inclination of the line segment image is close to the horizontal direction. Is set. In addition, when detecting the path of the elevating robot, each threshold and judgment standard are set so that the inclination of the line segment image is close to the vertical direction and the lower end of the line segment image is close to the wheel position of the robot. Set.

Further, in order to detect an ambiguous line segment as a line segment image, the computer 3 adds a process of clustering short line segments to combine a direction vector or a group of line segments having close positions and detect it as a line segment image. You may.

When a shadow is detected outdoors or the like in the two-dimensional color image acquired by the camera 2, the computer 3 may add the following processing in order to prevent erroneous detection of the route.
-Comparison between candidates that integrate other image features (color area clustering processing, etc.) (voting between candidate paths).
-Comparison with the processing result by 3D point cloud processing (surface detection, etc.) and the path detected using LiDAR (Light Detection and Ranging).
-Use of an image classifier (pattern recognition technology such as deep learning) that distinguishes between shadows and non-shadows.

When the computer 3 detects the position of the curved line segment image, it may be represented by a large number of short straight lines obtained by processing such as LSD (Line Segment Detector), or the Hough transform for detecting the curve is used. You may.

Further, the computer 3 may improve the diversity and accuracy of path feature detection by using deep learning in the process of extracting a linear or curved line segment image as a local feature.

Further, the computer 3 may automatically select features such as data and parameters by using clustering by combining detection data from a plurality of sensors other than the camera 2 or a plurality of feature extraction methods. .. For example, as a sensor to acquire depth information, multiple sensors using infrared rays or lasers (LiDAR, TOF camera, etc.) are used at the same time, and detection data is automatically selected to improve accuracy (either one depending on the environment). You may select a sensor or use both sensor outputs). By doing so, when the environment is different, the accuracy can be improved by compensating for each other's weak points.

In addition, in various processes in the route detection system 1, processing that is combined with methods such as artificial intelligence (AI), machine learning, and neural network may be executed.

The two-dimensional image acquired and processed by the computer 3 is not limited to a color image, but may be a black-and-white image or an infrared image. By acquiring an infrared image, it is possible to detect a route even in an environment such as a dark room at night.

Here, in the above embodiment, here, at least one processor selects from a plurality of data or parameters when acquiring a two-dimensional image or depth information, and the two-dimensional image or the result obtained as a result of the selection. It is preferable to detect route candidates for depth information. In this case, data or parameters suitable for the environment such as outdoors and indoors are selected, and by using the resulting two-dimensional image or depth information, route candidates according to the environment can be selected with higher accuracy. Can be detected.

Further, it is preferable that at least one processor changes the resolution of the two-dimensional image and detects the route candidate for the two-dimensional image whose resolution has been changed. In this case, by using a two-dimensional image whose resolution is changed to be suitable for an environment such as outdoor or indoor, it is possible to detect a route candidate according to the environment with higher accuracy.

Further, the acquisition device acquires a two-dimensional image as a color image including the brightness of each of a plurality of colors, and at least one processor changes the color extracted from the two-dimensional image and of each pixel of the extracted color. It is also preferable to detect route candidates for brightness. In this case, by using the image data of the color component suitable for the environment such as outdoor or indoor, it is possible to detect the route candidate according to the environment with higher accuracy.

It is also preferable that at least one processor detects the position of the line segment image by grayscale-converting the two-dimensional image to generate a grayscale image and performing edge detection of the grayscale image. By doing so, the processing load of the edge detection processing from the two-dimensional image is reduced, and as a result, the processing load of the entire route detection can be reduced.

It is also preferable that at least one processor detects the position of the line segment image by performing Hough transform after performing edge detection of the grayscale image. In this case, the line segment image can be detected without omission regardless of the resolution of the two-dimensional image, and as a result, the route can be detected with high accuracy.

It is also preferable that at least one processor detects route candidates in the three-dimensional space based on the comparison result between the height and the first threshold value and the comparison result between the slope and the second threshold value. .. In this case, by setting the threshold value according to the environment such as outdoor or indoor, it is possible to detect the route candidate according to the environment with higher accuracy.

Further, it is preferable that a drive mechanism that enables autonomous movement is further provided, and at least one processor controls autonomous movement by the drive mechanism based on the detected route candidate. According to such a configuration, highly accurate autonomous movement control is realized by using the detection result of the route.

One aspect of the present disclosure is that a route detection device for detecting a route is used and can detect a route with high accuracy in various environments.

1 ... Route detection system (route detection device), 2 ... Camera (acquisition device), 3 ... Computer, 4 ... Robot, 5 ... Drive mechanism, 31 ... Image conversion unit, 32 ... Edge detection unit, 33 ... Hough conversion unit, 34 ... Two-dimensional position calculation unit, 35 ... Correspondence unit, 36 ... Three-dimensional position calculation unit, 37 ... Path evaluation unit.

Claims

An acquisition device that acquires a two-dimensional image that reflects a two-dimensional image in the field of view and depth information of the image corresponding to each pixel of the two-dimensional image.
It comprises at least one processor for processing the two-dimensional image and the depth information.
The at least one processor
The position of the line segment image, which is an image approximated to the line segment, is detected on the two-dimensional image.
From the depth information, specify the depth corresponding to the position of the line segment image, and
Based on the position of the line segment image and the depth corresponding to the position, the height and inclination of the line segment image in the three-dimensional space are calculated.
Detecting path candidates in three-dimensional space based on the height and inclination,
Route detector.
The at least one processor
When acquiring the two-dimensional image or the depth information, a candidate for the route is detected from the two-dimensional image or the depth information obtained as a result of selecting from a plurality of data or parameters. ,
The route detection device according to claim 1.
The at least one processor
The resolution of the two-dimensional image is changed, and the candidate of the route is detected in the two-dimensional image whose resolution has been changed.
The route detection device according to claim 2.
The acquisition device acquires the two-dimensional image as a color image including the luminance for each of a plurality of colors.
The at least one processor
The color extracted from the two-dimensional image is changed, and the candidate of the route is detected for the brightness of each pixel of the extracted color.
The route detection device according to claim 2.
The at least one processor
The position of the line segment image is detected by performing grayscale conversion of the two-dimensional image to generate a grayscale image and performing edge detection of the grayscale image.
The route detection device according to any one of claims 1 to 4.
The at least one processor
The position of the line segment image is detected by performing the Hough transform after performing the edge detection of the grayscale image.
The route detection device according to claim 5.
The at least one processor
Based on the comparison result between the height and the first threshold value and the comparison result between the slope and the second threshold value, a route candidate in the three-dimensional space is detected.
The route detection device according to any one of claims 1 to 6.
Route detector.
Further equipped with a drive mechanism that enables autonomous movement,
The at least one processor
Based on the detected path candidates, the autonomous movement by the drive mechanism is controlled.
The route detection device according to any one of claims 1 to 7.