JP6871782B2 - Road marking detector, road marking detection method, program, and road surface detector - Google Patents

Description

The present invention relates to a technique for detecting a road marking or a road surface based on measurement data of a reflection point cloud acquired by a laser scanner.

As a technique for measuring the shape of a feature, there is a laser measurement technique for acquiring three-dimensional point cloud data representing the shape of the feature using a laser scanner. For example, in the Mobile Mapping System (MMS), it is possible to acquire 3D point cloud data and visible images while driving on the road by using a laser scanner and a camera mounted on the vehicle, for map creation and navigation. It is used for spatial data collection.

Conventionally, it has been studied to detect a road marking drawn on a road surface by using a visible image obtained by photographing the road surface (Patent Document 1). However, visible images are susceptible to lighting conditions that change with the weather and shooting time.

On the other hand, it is also studied to detect road markings using the measurement data of the laser reflection point cloud acquired by MMS. Specifically, the road markings are detected by utilizing the difference in laser reflection intensity due to the difference in color and material between the road markings on the road surface and other parts (asphalt, concrete, etc.). Unlike the visible image, the measurement data of the laser reflection point cloud is not easily affected by the lighting conditions.

In addition, research and development of a technology for automatically generating a three-dimensional model of an urban space including buildings and roads from a group of laser reflection points by MMS is also being conducted.

Japanese Unexamined Patent Publication No. 2013-186655

The reflection intensity of the reflection point cloud by the laser scanner changes depending on the distance until the laser reaches the object, the angle of incidence, the intensity of the laser, the surface condition of the object, and the like. That is, the reflection intensity can be different even if the reflection is from an object of the same color and material. Therefore, in the laser measurement data acquired from a relatively wide space, the threshold value can be set appropriately in the method of uniformly binarizing the reflection intensity with one threshold value and separating the road marking and the other parts. There was the problem that it was not easy or could be difficult.

For example, the reflection intensity of a group of reflection points on a road surface that is far from the laser scanner and has a large incident angle compared to a road surface that is close to the laser scanner and has a small incident angle is a part of the road marking. Regardless, it can be low as a whole. In such a case, for example, even if an attempt is made to distinguish between the two by utilizing the fact that the white road marking gives basically a larger reflection intensity than the "ground" part of the darker road, it is far away. In road markings, the reflection intensity is reduced, and the difference from the reflection intensity of the ground part of the nearby road can be reduced or reversed. That is, with simple binarization, it may be difficult to adequately distinguish between road markings and other parts on both the near road surface and the distant road surface.

Furthermore, in laser measurement in a relatively large space, the point cloud density of the reflection point cloud is also not constant. That is, the point cloud density changes depending on the distance from the sensor, which makes it even more difficult to automatically extract and recognize road markings.

Further, in order to reduce erroneous detection of road markings, it is preferable to limit the spatial range to be detected to the road surface. However, there are various structures and situations in the vicinity of the road, and it may not be easy to detect the road surface with high accuracy from the laser reflection point cloud.

Therefore, there is a problem that the detection, recognition, and mapping of road markings are basically performed manually or semi-automatically, which requires time and cost.

The present invention has been made to solve the above problems, and provides a technique for detecting road markings and road surfaces with appropriate accuracy based on measurement data of a reflection point cloud acquired by a laser scanner. The purpose.

(1) The road marking detection device according to the present invention is a device that detects a road marking based on measurement data of a reflection point group acquired from a road surface by a laser scanner, and each reflection constituting the reflection point group. An index value indicating the degree of variation in the laser reflection intensity of the reflection point cloud existing in a predetermined vicinity region of the reflection point on the road surface is calculated in association with the points, and the road surface is based on the index value. Of these, a filtering means for determining a region where the laser reflection intensity is uniform and removing those in the region from the reflection point group, and a preset required resolution for detecting the road marking. Correspondingly, it has a clustering means for clustering the distribution of the reflection intensity undulating point cloud on the road surface and generating a point cloud cluster separated for each road sign.

(2) In the road marking detection device of the above (1), the reflection point group belonging to the point group cluster is further classified according to a threshold value determined for each point group cluster according to a predetermined standard with respect to the laser reflection intensity. The configuration may include a road marking candidate point group extracting means for extracting the reflection point group having the laser reflection intensity equal to or higher than the threshold value as one road marking candidate point group corresponding to the road marking.

(3) In the road marking detection device of the above (1) and (2), the predetermined neighborhood region is a circle whose diameter is larger than the minimum dimension of the construction portion of the road marking to be detected and equal to or less than the required resolution. Can be configured to be.

(4) In the road marking detection device of the above (2), an image generation means for generating an image of the road marking candidate figure based on the distribution area of the road marking candidate point group on the road surface, and the road marking candidate figure. The distance from the center of gravity to each contour pixel is discretely Fourier-transformed to calculate the frequency spectrum, and as the image features of the road marking candidate figure, the components and high frequencies in the predetermined frequency width at the end on the low frequency side in the frequency spectrum. Using an image feature extraction means for extracting components in a predetermined frequency width at the end of the side and a classifier that has learned in advance the image features of the road marking to be detected, the road marking candidate figure is the road marking. It can be configured to have a road marking determining means for determining whether or not the road marking is.

(5) In the road marking detection device of the above (4), the image feature extraction means may be configured to further calculate the moment of the road marking candidate figure as the image feature.

(6) The road surface detection device according to the present invention is a device that detects a road surface based on measurement data of a reflection point cloud acquired from the ground surface by a laser scanner, and projects the reflection point cloud onto a horizontal plane. Delaunay triangle division is performed based on the point cloud to generate a Delaunay triangle having the reflection point group as an apex, and the apex of the Delaunay triangle whose gradient exceeds a predetermined upper limit is removed from the reflection point group and allowed. The permissible slope point cloud extraction means for obtaining the slope point cloud and the clustering related to the distribution on the ground surface of the permissible slope point cloud are performed, and among the obtained clusters, those having the number of elements equal to or more than a predetermined threshold are the roads. It has a road surface candidate point cloud extraction means for extracting as a surface candidate point cloud.

(7) The road marking detection method according to the present invention is a method of detecting a road marking based on measurement data of a reflection point cloud acquired from a road surface by a laser scanner, and each reflection constituting the reflection point cloud. An index value indicating the degree of variation in the laser reflection intensity of the reflection point cloud existing in a predetermined vicinity region of the reflection point on the road surface is calculated in association with the points, and the road surface is based on the index value. Of these, a filtering step for determining a region where the laser reflection intensity is uniform and removing the reflection point group in the region, and a preset required resolution for detecting the road marking. Correspondingly, it has a clustering step of performing clustering on the distribution on the road surface with respect to the reflection intensity undulating point cloud and generating a point cloud cluster separated for each road sign.

(8) The program according to the present invention is a program for causing a computer to perform a process of detecting a road sign based on measurement data of a reflection point cloud acquired from a road surface by a laser scanner. An index value indicating the degree of variation in the laser reflection intensity of the reflection point group existing in a predetermined vicinity region of the reflection point on the road surface is calculated in association with each reflection point constituting the reflection point group, and the index value is calculated. A filtering means for determining a region of the road surface where the laser reflection intensity is uniform based on an index value, and obtaining a reflection intensity undulation point cloud obtained by removing the region of the reflection point group, and the road. A clustering means for clustering the distribution of the reflection intensity undulating point cloud on the road surface according to a preset required resolution for detecting the marking, and generating a point cloud cluster separated for each road marking. To function as ,.

According to the present invention, it becomes easy to automatically detect a road marking or a road surface with appropriate accuracy based on the measurement data of a reflection point cloud acquired by a laser scanner.

It is a block diagram which shows the schematic structure of the road marking detection system which concerns on embodiment of this invention. It is a schematic flow chart of the process which the road marking detection system which concerns on embodiment of this invention detects and recognizes a road marking from a laser measurement data. It is a schematic flow chart of the process of the permissible gradient point cloud extraction means mainly in the road marking detection system which concerns on embodiment of this invention. It is a schematic diagram explaining the processing of the permissible gradient point cloud extraction means. It is a schematic flow chart of the process of the road surface candidate point cloud extraction means mainly in the road marking detection system which concerns on embodiment of this invention. It is a schematic flow chart of the process of extracting the road marking object from the road surface in the road marking detection system which concerns on embodiment of this invention. It is a schematic diagram explaining the process of calculating the index value of the variation of the laser reflection intensity of the reflection point group in the region near each reflection point by the point cloud filtering means. It is a schematic diagram explaining the process of extracting the road marking object from the road surface in the road marking detection system which concerns on embodiment of this invention. It is a schematic flow chart of the road marking object discrimination processing in the road marking detection system which concerns on embodiment of this invention.

Hereinafter, the road marking detection system 2 which is an embodiment of the present invention (hereinafter referred to as an embodiment) will be described with reference to the drawings. This system is a system that detects a road marking based on measurement data of a group of reflection points acquired by a laser scanner from a target space for detecting a road marking, and detects a road surface from the ground surface according to the present invention. It includes a surface detecting device and a road marking detecting device according to the present invention for detecting a road marking from a road surface.

The measurement data of the reflection point group is acquired by, for example, MMS. In MMS, the laser scanner mounted on the automobile irradiates the laser diagonally downward or diagonally upward from the upper part of the vehicle body, and detects the laser reflected light from the object. The optical axis of the laser is scanned laterally, and laser pulses are emitted at every minute angle within the scanning angle range. The distance is measured based on the time from the laser emission to the reception of the reflected light, and at that time, the laser emission direction, time, and the position / posture of the vehicle body are measured. From these measurement data, point cloud data representing the three-dimensional coordinates of the point where the laser pulse is reflected in the target space can be obtained. The laser scanner also measures the intensity of reflected light (reflection intensity).

Road markings are markings drawn on the road surface, and are legally markings that display regulations or instructions regarding road traffic, but the road markings to be detected by the road marking detection system 2 are not limited to legal markings. Further, since the road marking is basically provided on the ground surface paved with concrete, asphalt, etc., the road surface in the present embodiment is basically a paved surface made of such a material. On the other hand, the road marking detection system 2 can detect a marking drawn on a pavement surface with paint or the like as a road marking, and can be used regardless of whether or not the pavement surface is a road. Therefore, the road surface in the present embodiment may include a paved surface such as a parking lot in addition to a road including a roadway and a sidewalk.

FIG. 1 is a block diagram showing a schematic configuration of the road marking detection system 2. This system includes an arithmetic processing unit 4, a storage device 6, an input device 8, and an output device 10. As the arithmetic processing unit 4, it is possible to create dedicated hardware for performing various arithmetic processing of this system, but in the present embodiment, the arithmetic processing unit 4 uses a computer and a program executed on the computer. Is built.

The CPU (Central Processing Unit) of the computer constitutes the arithmetic processing unit 4, and the allowable gradient point cloud extraction means 20, the road surface candidate point cloud extraction means 22, the point cloud filtering means 24, the point cloud clustering means 26, and the road, which will be described later. It functions as a marking candidate point cloud extracting means 28, an image generating means 30, an image feature extracting means 32, and a road marking discriminating means 34.

The storage device 6 is composed of a hard disk or the like built in the computer. The storage device 6 uses the arithmetic processing device 4 as an allowable gradient point cloud extraction means 20, a road surface candidate point cloud extraction means 22, a point cloud filtering means 24, a point cloud clustering means 26, a road marking candidate point cloud extraction means 28, and an image generation means. 30. Stores a program and other programs for functioning as the image feature extracting means 32 and the road marking discriminating means 34, and various data necessary for processing of this system. For example, the storage device 6 stores the laser measurement data 40 as the data to be processed. Further, the storage device 6 stores the classifier 42 generated by learning in advance for each road marking for which the image feature of the road marking is to be detected.

The input device 8 is a keyboard, a mouse, or the like, and is used by the user to operate the system. Further, the input device 8 may include an interface device for inputting data from another system.

The output device 10 is a display, a printer, or the like, and is used for showing the road surface or road markings obtained by this system to the user by screen display, printing, or the like.

The permissible gradient point cloud extraction means 20 performs Delaunay triangle division based on the point cloud obtained by projecting the reflection point cloud acquired from the ground surface onto the horizontal plane to generate a Delaunay triangle having the reflection point cloud as the apex. That is, the division is done in the horizontal plane, but the Delaunay triangle is defined in three-dimensional space. Then, the permissible gradient point cloud extraction means 20 removes the reflection points forming the vertices of the Delaunay triangle whose gradient exceeds a predetermined upper limit from the reflection point cloud, and outputs the remaining reflection point cloud as the permissible gradient point cloud. ..

The road surface candidate point cloud extraction means 22 removes the noise point cloud from the allowable gradient point cloud and extracts the road surface candidate point cloud. For example, the road surface candidate point cloud extraction means 22 clusters the allowable gradient point cloud regarding the distribution on the ground surface, and removes the obtained clusters having less than a predetermined threshold value as noise. Those above the threshold are extracted as a group of candidate points on the road surface. Specifically, the road surface candidate point cloud extraction means 22 performs clustering based on the Euclidean distance in the allowable gradient point cloud as the clustering process.

The point cloud filtering means 24 assigns an index value indicating the degree of variation in the laser reflection intensity of the reflection point cloud existing in a predetermined vicinity region of the reflection point on the road surface in association with each reflection point constituting the reflection point group. calculate. Then, a region on the road surface where the laser reflection intensity is uniform is determined based on the index value, and a reflection intensity undulation point group obtained by removing those in the region from the reflection point group is obtained.

The point cloud clustering means 26 clusters the reflection intensity undulating point group regarding the distribution on the road surface according to the preset required resolution for the detection of the road marking, and the point cloud separated for each road marking. Create a cluster. Specifically, the point cloud clustering means 26 performs clustering based on the Euclidean distance in the reflection intensity undulating point cloud as the clustering process.

The road marking candidate point cloud extraction means 28 classifies the reflection point cloud belonging to the point cloud cluster according to the threshold value determined for each point cloud cluster according to a predetermined standard with respect to the laser reflection intensity, and the reflection having the laser reflection intensity equal to or higher than the threshold value. The point cloud is extracted as a road marking candidate point cloud corresponding to one road marking.

The image generation means 30 generates an image of the road marking candidate figure based on the distribution area of the road marking candidate point group on the road surface.

The image feature extracting means 32 calculates the frequency spectrum by discrete Fourier transforming the distance from the center of gravity of the road marking candidate figure to each contour pixel, and uses the image feature of the road marking candidate figure on the low frequency side in the frequency spectrum. The component in the predetermined frequency width of the end portion and the component in the predetermined frequency width of the end portion on the high frequency side are extracted. In the present embodiment, the image feature extraction means 32 further calculates the moment of the road marking candidate figure as an image feature.

The road marking discriminating means 34 determines whether the road marking candidate figure is a road marking by using the classifier 42 stored in the storage device 6 for the image feature of the road marking to be detected.

The laser measurement data 40 is measurement data of a reflection point group acquired by an MMS traveling on a road, and includes information on three-dimensional coordinates and reflection intensity of each reflection point. The Cartesian coordinate system XYZ is defined as the three-dimensional coordinate system in the target space, the XY plane is defined as the horizontal plane, and the Z axis is defined as the vertical axis whose positive direction is upward.

The classifier 42 is generated by machine learning using learning data on road markings. Learning is performed by a method such as a support vector machine. The classifier 42 is a function that inputs the image feature of the road marking candidate figure and calculates the likelihood of the road marking. The classifier 42 is prepared for each road marking to be detected, for example.

FIG. 2 is a schematic flow chart of a process in which the road marking detection system 2 detects and recognizes the road marking from the laser measurement data 40. The road marking detection system 2 extracts the road surface from the laser measurement data 40 (step S1), extracts the road marking object from the road surface (step S2), and uses the classifier 42 to mark the road marking object. It is determined whether or not the road marking is, and which road marking is used (step S3).

Specifically, step S1 is performed by the allowable gradient point cloud extraction means 20 and the road surface candidate point cloud extraction means 22, and the road surface candidate point cloud is extracted from the laser measurement data 40. Step S2 is performed by the point cloud filtering means 24, the point cloud clustering means 26, and the road marking candidate point group extracting means 28, and the road marking candidate point group is extracted from the road surface candidate point group. Here, there may be a plurality of road markings on the road surface of the target space. Road marking candidate points are defined for each of these multiple road markings. That is, each road marking candidate point group is extracted as one road marking candidate object (road marking object). Step S3 is performed by the image generation means 30, the image feature extraction means 32, and the road marking determination means 34, converts the representation format of the road marking object from the road marking candidate point group into an image, and recognizes the road marking based on the image feature. Processing is done.

FIG. 3 is a schematic flow chart for explaining a part of the processing of step S1 of FIG. 2, and mainly shows a part of step S1 related to the processing of the permissible gradient point cloud extraction means 20.

The permissible gradient point cloud extraction means 20 reads out the laser measurement data 40 in the target space from the storage device 6. As already mentioned, the MMS laser scanner irradiates the laser diagonally downward and diagonally upward from the vehicle, so the laser measurement data 40 includes not only the reflection point cloud on the ground surface, but also buildings along the road and street trees. A group of reflection points such as is also included. Therefore, the allowable gradient point cloud extraction means 20 reflects the reflection point cloud (or the reflection corresponding to the laser pulse emitted upward from the laser scanner) having a Z coordinate higher than that of the laser scanner among the reflection point cloud included in the laser measurement data 40. A point) is set as a reflection point cloud that does not form a road surface, and the laser measurement data 40 of the reflection point is separated into the off-road measurement data E1 and removed (in the case of “Yes” in step S10).

The permissible gradient point cloud extraction means 20 performs Delaunay triangulation on the point cloud obtained by projecting the reflection point cloud constituting the remaining data excluding the off-road measurement data E1 from the laser measurement data 40 on the horizontal plane (step S12). The permissible gradient point cloud extraction means 20 defines a Delaunay triangle in a three-dimensional space with a reflection point corresponding to the apex of the Delaunay triangle defined on the horizontal plane as the apex. As a result, in step S12, a TIN (Triangulated Irregular Network) model composed of the Delaunay triangle is generated as a polygon model on the ground surface.

The permissible gradient point cloud extraction means 20 calculates the gradient of each triangle constituting the TIN (step S14), and sets the reflection point at the place where the slope is unexpectedly steep on the road surface as the reflection point that does not constitute the road surface. As a group, the laser measurement data 40 of the reflection point is separated into the off-road measurement data E2 and removed (in the case of "Yes" in step S16). Specifically, in step S16, when the gradient of the triangle exceeds a predetermined upper limit Th1, the reflection points forming the vertices of the triangle are classified into the off-road measurement data E2. For example, the upper limit Th1 of the gradient can be 30%. By the way, since the maximum longitudinal gradient of a road stipulated by law is 12%, if the upper limit gradient is about 30%, it is unlikely that the reflection points on the road surface will be accidentally removed. Reflection points on steep slopes that may occur on other than the road surface, such as steps and slopes at the boundary with the sidewalk, can be preferably removed.

FIG. 4 is a schematic diagram illustrating the processing of the permissible gradient point cloud extraction means 20. FIG. 4 shows, for example, an example of a reflection point group at the boundary between a roadway and a sidewalk. In FIG. 4, the sidewalk surface 52 is higher than the road surface 50, and a stepped surface 54 exists at the boundary. In order to make it easier to understand the difference in the Z coordinates of the reflection points, in FIG. 4, the reflection points on the road surface 50 are marked with “●”, the reflection points on the sidewalk surface 52 are marked with “○”, and the reflection points on the stepped surface are marked with “×”. It is represented by a "mark." FIG. 4A is a perspective view showing an example of a reflection point group after removing the off-road measurement data E1 from the laser measurement data 40, and FIG. 4B projects the reflection point group of FIG. 4A. It is a top view which shows the horizontal plane. In FIGS. 4 (a) and 4 (b), examples of triangles generated by the Delaunay triangle division with respect to the point cloud projected on the horizontal plane of FIG. 4 (b) are shown by dotted lines. Of these triangles, the gradient values of the triangles in the road surface 50 (all vertices are "●") and the triangles in the sidewalk surface 52 (all vertices are "○") are parables. Even if the road is provided on a slope, it basically does not exceed the upper limit slope. On the other hand, a triangle having vertices on a plurality of surfaces of the road surface 50, the sidewalk surface 52, and the step surface 54 is formed at the stepped portion at the boundary between the roadway and the sidewalk. Such a triangle stands at a relatively steep angle with respect to the road surface 50, and as a result of the gradient exceeding the upper limit, the reflection point forming the apex of the triangle is removed as off-road measurement data E2 in step S16.

FIG. 4C is a perspective view showing an example of the reflection point group after removing the off-road measurement data E2 from the reflection point group of FIG. 4A. In this example, steps S12 to S16 remove the reflection points on the stepped surface 54, which are reflection points that do not belong to the road surface 50 and the sidewalk surface 52. Further, among the reflection points on the road surface 50 and the sidewalk surface 52, points near the boundary can also be removed, and as a result, the distance between the reflection point group on the road surface 50 and the reflection point group on the sidewalk surface 52 can be increased.

In the Delaunay triangle division of the point cloud projected on the horizontal plane, the adjacency of the reflection points is determined by ignoring the difference in the Z coordinates between the reflection points and focusing only on the horizontal coordinates. Therefore, for example, Z with the road surface 50. Even if the reflection point has a large difference in coordinates, it is easy to form a Delaunay triangle with the reflection point on the road surface 50. That is, not only the reflection point on the step surface 54 where the height difference between the roadway and the sidewalk shown in FIG. 4 is small, but also the reflection point greatly separated from the road surface in the Z-axis direction in steps S12 to S16. Can be removed by processing.

The remaining laser measurement data 40 from which the off-road measurement data E2 has been removed is used as the allowable gradient point cloud data C1. The permissible slope point cloud extraction means 20 generates the permissible slope point cloud data C1, and then the road surface candidate point cloud extraction means 22 further performs a process of removing the noise point cloud from the permissible slope point cloud data C1, and the road surface candidate. Find the point cloud.

FIG. 5 is a schematic flow chart for explaining the portion of the process of step S1 of FIG. 2 following FIG. 3, and mainly shows the portion of step S1 related to the process of the road surface candidate point cloud extraction means 22. ..

The road surface candidate point cloud extraction means 22 performs outlier analysis on the allowable gradient point cloud measurement data C1 generated by the allowable gradient point cloud extraction means 20 (step S20). Specifically, the road surface candidate point cloud extraction means 22 extracts a local point cloud consisting of a predetermined number of reflection points in the vicinity of each reflection point included in the allowable gradient point cloud measurement data C1. Then, the Euclidean distance between each reflection point and another reflection point in the vicinity thereof is obtained in the local point cloud, and the reflection point whose difference from the average distance in the distribution of the distance is a predetermined magnitude Th2 or more is obtained. It is set as an outlier, separated into the noise point group E3, and removed (in the case of "Yes" in step S22). For example, the number of reflection points constituting the local point cloud can be 30 points. Further, when the distance distribution standard deviation of the represented by sigma _D, for example, the threshold value Th2 may be a 1 [sigma _D.

The road surface candidate point cloud extraction means 22 performs principal component analysis on the spatial distribution of the reflection point cloud remaining after removing the noise point cloud E3 in step S22 (step S24). The first principal component from the direction distribution of the points is greater by the principal component analysis in order, the second principal component, Motomari third principal component, the first to third eigenvalue lambda ₁ to [lambda] ₃ in response to each of these main components Is calculated. The eigenvalue is the variance value of the point cloud in the direction of the corresponding principal component, and is λ ₁ > λ ₂ > λ ₃ . Here, as λ ₁ / λ ₂ becomes larger, the point cloud is elongated and distributed along the direction of the first principal component. Also, when λ ₁ and λ ₂ are about the same, if λ ₃ is _{sufficiently smaller than λ 1} and λ ₂ , the point cloud will be distributed in a plane, and as λ ₃ becomes larger, the distribution shape will be a curved surface. It becomes.

Therefore, the road surface candidate point cloud extraction means 22 extracts the local point cloud and performs principal component analysis in the same manner as in step S20 (step S24), and when the degree of curvature of the distributed shape is equal to or greater than a predetermined threshold value Th3 (step S24). (In the case of "Yes" in step S26), the local point cloud is separated into point cloud data E4 other than the road and removed. For example, the number of reflection points forming the local point cloud in step S24 can be 30 points. Further, the threshold value Th ₃ can be set to 1% as the value of λ 3 in the normalized eigenvalue.

The road surface candidate point cloud extraction means 22 clusters the Euclidean distance with respect to the reflection point cloud remaining after the removal of the point cloud data E4 in step S26 (step S28). Since the road surface is relatively wide and continuous, the road surface candidate point group extraction means 22 extracts the generated clusters having more reflection points than a predetermined number Th4 as the road surface candidate point group C2 (step S30). (In the case of "Yes"), the other clusters are classified into the noise point group E5 (in the case of "No" in step S30). The threshold Th4 can be, for example, 1000 points.

Incidentally, as described above in the example of the roadway and the sidewalk in FIG. 4, the reflection point group near the step is removed by the processing of steps S12 to S16, and as a result, the reflection point group on the roadway surface 50 and the reflection on the sidewalk surface 52. The distance to the point cloud can be increased. Therefore, it can be expected that the reflection point group of the road surface 50 and the reflection point group of the sidewalk surface 52 form separate clusters. From this, in the clustering in step S28, basically, a cluster of one flat surface is composed of a group of reflection points of the flat surface, and a step existing outside the flat surface, another flat surface, a building, etc. It can be understood that the configuration does not include the reflection point cloud.

The process of detecting the road surface based on the measurement data of the reflection point cloud acquired by the laser scanner, which is step S1 in FIG. 2, has been described above. Next, a process of extracting a road marking object from the measurement data of the reflection point cloud (road surface point cloud measurement data C3) acquired from the road surface by the laser scanner, which is step S2 of FIG. 2, will be described.

FIG. 6 is a schematic flow chart illustrating the process of step S2 of FIG. In the present embodiment, the road marking detection system 2 uses the laser measurement data 40 for the road surface candidate point cloud C2 extracted in step S1 as the road surface point cloud measurement data C3. As the road surface point cloud measurement data C3, it is also possible to use data separately acquired without using the road marking detection system 2.

The point cloud filtering means 24 calculates an index value indicating the degree of variation in the laser reflection intensity of the reflection point cloud existing in a predetermined vicinity region of the reflection point for each reflection point included in the road surface point cloud measurement data C3 (step S40). ). Here, the vicinity area is referred to as a window area.

For example, the window area can be circular. Further, it is preferable that the diameter of the circular shape is larger than the minimum dimension of the construction portion (hereinafter, referred to as a paint portion) of the road marking to be detected. However, the diameter of the circle is set within a limit that does not deviate from the purpose of defining the neighboring region, and is set smaller than the assumed minimum distance between two road markings that should be recognized separately, for example. In the present embodiment, since the line width (thickness) constituting the road marking is often about 10 to 50 cm [cm], the window area is a circle with a diameter of 60 cm.

_{Further, in the present embodiment, the standard deviation σ I} of the laser reflection intensity in the reflection point group is used as an index value indicating the degree of variation in the laser reflection intensity of the reflection point group in the window region.

FIG. 7 is a schematic diagram illustrating the process of step S40. FIG. 7A is a plan view of the road surface, and two white lines 60 and 62 exist on the road surface as an example of road markings. The white line 60 and the white line 62 constitute separate road markings, and there is a distance of, for example, several meters [m] between them. Further, FIG. 7A illustrates window regions 64 at several positions.

FIG. 7B is a graph showing an example of a change in the laser reflection intensity of the road surface along the straight line 66 shown in FIG. 7A. The horizontal axis of FIG. 7B is the position on the road surface along the straight line 66, the vertical axis is the laser reflection intensity, and the upward direction is the direction in which the reflection intensity increases. In the example shown in FIG. 7B, the laser reflection intensity decreases from the left side to the right side macroscopically. Therefore, as described above as a problem to be solved by the present invention, for example, when binarized with the threshold value indicated by the straight line 70, the reflection point on the white line 60 exceeds the threshold value, but the reflection point on the white line 62 does not exceed the threshold value. There was a problem.

FIG. 7C is a graph showing the change 72 _{of the standard deviation σ I} when the window region is moved along the straight line 66. Similar to the horizontal axis of FIG. 7 (b), the horizontal axis of FIG. 7 (c) is the position on the road surface along the straight line 66, the vertical axis is the standard deviation σ _I , and the upward direction is σ _I. The direction is to increase. _{The standard deviation σ I} calculated in the window region set around a certain reflection point corresponds to the reflection point at the center of the window region.

The standard deviation σ _I increases when there are undulations in the laser reflection intensity in the window region, and decreases when the uniformity of the laser reflection intensity is high in the window region. Specifically, a position where the window area straddles the road marking portion and the "ground" portion of the road, such as the window areas 64a, 64b, 64f to 64h, that is, a position including the contour (edge) of the road marking. _{The standard deviation σ I} becomes large when there is, and conversely, the standard deviation when the window area includes only the “ground” part of the road, that is, when it does not include the edge of the road marking, such as the window areas 64c to 64e. σ _I becomes smaller. Here, even when the window area includes only the portion of the road marking, the window area does not include the edge of the road marking, and the standard deviation σ _I can be small. However, in the present embodiment, as described above, the diameter of the window area is set to be larger than the minimum dimension of the paint portion of the road marking to be detected. Therefore, when the window area includes only the road marking portion, it is basically It will not be considered. Therefore, the region of the road surface where the laser reflection intensity is determined to be uniform based on the _{standard deviation σ I is composed of the “ground” portion of the road in the present embodiment.}

The standard deviation σ _I is a value that changes according to the variation in the laser reflection intensity in the window region, and the laser reflection intensity itself does not directly affect the _{standard deviation σ I.} Therefore, for example, there is a difference in laser reflection intensity between the window regions 64c to 64e, but there is no difference in the standard deviation σ _I. Further, a gradual change in laser reflection intensity as shown as a macroscopic change in FIG. 7B does not increase the _{standard deviation σ I as much as a change in laser reflection intensity at the edge of the road marking.} Therefore, for example, the standard deviation sigma _I in the window region 64c~64e window regions 64a, 64b, smaller than the standard deviation sigma _I in 64F～64h.

The nature of these standard deviations sigma _I, in the distribution of the standard deviation sigma _I calculated for each reflection point of the road surface, the standard deviation sigma _I in window region not including the edges of the road sign, including the edges of the road sign It is generally smaller than the standard deviation σ _I in the window region and is preferably separated from _{the standard deviation σ I} in the window region including the edges.

Therefore, the point cloud filtering means 24 classifies a plurality of reflection points on the road surface into a plurality of classes based on the _{standard deviation σ I} calculated for each reflection point using the natural class classification method of Jenks (). Step S42). The lower the standard deviation σ _I , the smaller the undulations of the laser reflection intensity in the vicinity of the reflection point. That is, such a reflection point may be a reflection point on the "ground" of the road surface that is located in the center of the window area that does not include the edge of the road marking, and is therefore more than the radius of the window area from the edge of the road marking. Is high. Therefore, it is presumed that the lowest class reflection point group with the smallest standard deviation σ _I is the reflection point group of the “ground” on the road surface where the laser reflection intensity is uniform, and the point group E6 other than the road marking is estimated. (In the case of "Yes" in step S44), and the rest is extracted as the reflection intensity undulation point group C4 (in the case of "No" in step S44).

The smaller the number of classes classified here, the easier it is for reflection points of road markings to be mixed in the lowest class, and conversely, the larger the number of classes, the more reflection points other than road markings are included in the classes above the lowest class. It becomes easy to remove the reflection points other than the road markings. Therefore, the number of classes is set in consideration of this point. In the present embodiment, the number of classes is 4, the point cloud filtering means 24 removes the lowest class, and the upper 3 classes are passed to the point cloud clustering means 26 as the reflection intensity undulation point group C4.

The point cloud clustering means 26 clusters the reflection intensity undulating point cloud C4 based on the Euclidean distance according to the preset required resolution for detecting the road marking, and generates a point cloud cluster separated for each road marking. (Step S46).

The required resolution for detecting a road marking is the distance between the two painted parts of the road marking that are recognized as different road markings. In the present invention, the resolution is required to be set low enough that a plurality of paint portions constituting one road marking are recognized as one road marking, and the required resolution is basically a circle of a window area. The upper limit of the diameter of the road markings is set to the assumed minimum distance between the two road markings to be recognized separately as described above.

For example, as a road marking for a pedestrian crossing, there is a zebra pattern in which a plurality of white lines separated from each other are arranged in parallel at an interval of 45 to 50 cm. In the example of extracting the pedestrian crossing marking as one group instead of as individual white lines, it is necessary to set a dimension equal to or larger than the white line spacing as the required resolution.

The road marking candidate point cloud extraction means 28 performs two classes of clustering regarding the laser reflection intensity for each point cloud cluster generated by the point cloud clustering means 26 (step S48), and the reflection points having a laser reflection intensity equal to or higher than the threshold value. The group is extracted as a road marking candidate point cloud C5 corresponding to one road marking (in the case of "Yes" in step S50), while the reflection point cloud below the threshold is separated and removed as a point cloud E7 other than the road marking. (In the case of "No" in step S50). Specifically, in the reflection intensity undulation point group C4, in addition to the reflection point group of the paint portion of the road marking, the "ground" portion of the road from the outline of the paint portion to the distance of about the radius of the circle of the window area. The reflection point group of the above is included, and the road marking candidate point group extraction means 28 attempts to extract the point group of the paint portion.

Clustering in step S48 is performed, for example, using Jenks' natural classification method. Here, the clustering is performed for each point cloud cluster divided for each road sign, and by using Jenks' natural class classification method, the threshold value for classifying into two classes is set for each point cloud cluster. Keep in mind. That is, unlike the method of binarizing the white lines 60 and 62 constituting the separate road markings with a common threshold value (straight line 70) described in the explanation of FIG. 7B, within the target space of the road markings. It is less susceptible to fluctuations in laser reflection intensity due to differences in the position of road markings, and the detection accuracy of road markings can be improved. The clustering in step S48 may be another method of setting a threshold value for each point cloud cluster for each road marking without using the natural classification method.

FIG. 8 is a schematic diagram illustrating the above-mentioned process relating to the extraction of the road marking object in step S2 of FIG. FIG. 8 is a plan view of an example of a road surface, and FIGS. 8A to 8C show a common road surface 74. There are pedestrian crossings 76 and 78 at two locations on the road surface as examples of road markings. The pedestrian crossing 76 has a plurality of white lines 76p as the paint portion, and similarly, the pedestrian crossing 78 has a plurality of white lines 78p. In FIGS. 8A to 8C, the shaded area represents the region where the reflection point cloud exists.

Specifically, FIG. 8A shows the existence range of the reflection point cloud in the road surface point cloud measurement data C3, and the reflection point cloud exists on the entire road surface.

FIG. 8B shows the processing results by the point cloud filtering means 24 and the point cloud clustering means 26. The point cloud filtering means 24 removes reflection points in the window region where the laser reflection intensity is uniform, and the reflection point cloud group 80-1 existing in the "ground" portion of the painted portions 76p and 78p and the surrounding road. ~ 80-8 is extracted as the reflection intensity undulation point cloud C4. Further, the point cloud clustering means 26 sets the reflection point groups 80-1 to 80-4, which are close to each other, as one point cloud cluster 82 of the road sign, and the reflection point groups 80-5 to 80-8, which are close to each other, are designated as one point cloud cluster 82. Since the distance from the reflection point groups 80-1 to 80-4 is large, the point cloud cluster 84 is a road sign different from the point cloud cluster 82.

FIG. 8C shows the processing result by the road marking candidate point cloud extraction means 28. The road marking candidate point cloud extracting means 28 performs two classes of laser reflection intensity clustering on the point cloud cluster 82, removes the class having a small reflection intensity, and extracts the class having a large reflection intensity as the road marking candidate point cloud group C5. As a result, the reflection point group of the "ground" portion of the road surrounding the paint portion 76 is removed, and the reflection point group existing in the plurality of paint portions 76p is extracted as one road marking object. Similarly, the reflection point cloud existing in the plurality of paint portions 78p is extracted from the point cloud cluster 84 as another road marking object.

The process of extracting the road marking object, which is step S2 in FIG. 2, has been described above. Next, the identification process of the road marking object, which is step S3 of FIG. 2, will be described.

FIG. 9 is a schematic flow chart illustrating the process of step S3 of FIG. In the process of step S2 described above, the road marking candidate point group C5 is obtained as the road marking object, and this road marking object is passed as the processing target data to step S3. Specifically, a point cloud cluster corresponding to a road marking object is defined in the road marking candidate point group C5.

The image generation means 30 generates an image of the road marking candidate figure based on the distribution area on the road surface of the point group cluster. For example, a raster image is generated as the image. The raster image can be a binary image. For example, the pixels constituting the road marking candidate figure are represented by the pixel value "1" or white, and the pixels in the area outside the figure are represented by the pixel value "0" or black. To.

Specifically, the image generation means 30 first projects each reflection point constituting the point cloud cluster on the horizontal plane (or road surface) for each road marking object, and calculates the pixel value at the position where the reflection point is projected. An initial image is generated in which the pixel values of the pixels other than "1" are defined as "0" (step S60). Then, by the morphology processing, the gap between the point clouds in the initial image is filled, and an image in which the pixel value of the region corresponding to the paint portion of the road marking is “1” is generated (step S62).

The image feature extraction means 32 extracts an image feature from the road marking candidate figure generated as an area corresponding to the paint portion in step S62. First, the image feature extraction means 32 calculates the center of gravity of the road marking candidate figure (step S64), and extracts the outline of the road marking candidate figure (step S66). Then, the image feature extraction means 32 calculates the distance from the center of gravity (center of gravity distance) for each pixel (contour pixel) constituting the contour (step S68). The number of contour pixels is N, and the index given to each contour pixel in the image is n (n is an integer of 0 to N-1), and N center-of-gravity distances u (n) are obtained. The index is assigned to the image generated by the image generation means 30 according to a common rule. For example, the index is assigned in ascending order to the contour pixels appearing in the raster scan of the image.

The image feature extraction means 32 performs a discrete Fourier transform on this center of gravity distance u (n) to calculate the frequency spectrum given by the following equation (step S70).

This frequency spectrum is used as an image feature of the road marking candidate figure. Here, if all the components of the frequency spectrum are used for discriminating road markings using machine learning in the road marking discriminating means 34, the problem of overfitting (overfitting) is likely to occur. _{Therefore, in the frequency spectrum, the component at the predetermined frequency width δ L} at the end on the low frequency side and the component at the predetermined frequency width δ _H at the end on the high frequency side are extracted as image features. That is, U (0), U (1), ..., U (δ _L -1) are extracted from the low frequency side, and U (N-1), U (N-2), ..., U from the high frequency side. Extract (N-δ _H ). For example, δ _L and δ _H can be defined as a fixed ratio of the number of components to N components. Further, δ _L and δ _H can be set to predetermined constant values regardless of the total number of components N.

Further, the image feature extraction means 32 further calculates the moment of the road marking candidate figure as an image feature. For example, to calculate the zero-order moment _{m 0,0} defined by the following equation, first-order moment _m 1, _0, a _{m 0, 1} as an image feature. In the following equation, b (x, y) is a pixel value in pixels (x, y) of an image representing a road marking candidate figure, and as described above, it is 1 in the paint section and 0 in other cases. Further, the summation symbol Σ means to take the sum of all the pixels of the image.
m _0,0 = Σb (x, y)
m _1,0 = Σxb (x, y)
m _0,1 = Σyb (x, y)

By using these moments, for example, information on the size and orientation (rotation angle in the image plane) of the road marking candidate figure, which is lost in the frequency spectrum, is supplemented.

The road marking discriminating means 34 uses the image feature extracted by the image feature extracting means 32 as an input to the classifier 42 generated by machine learning, and the road marking candidate figure is a road marking based on the likelihood output by the classifier. (Step S72). For example, the image features of the road marking object (road marking candidate figure) to be discriminated are input to each of the classifiers 42 prepared for each of a plurality of types of road markings to be detected, and an output value equal to or higher than a predetermined value is obtained. The road marking corresponding to the classifier from which the maximum output value is obtained among the classifiers 42 is used as the determination result of the road marking object.

Incidentally, as already described, the classifier 42 is generated by machine learning using learning data for each type of road marking. That is, the learning of a certain type of road marking involves a plurality of data on a group of reflection points known to be the type of road marking object and a group of reflection points known to be not the type of road marking object. A plurality of data are used as training data. Extraction of image features during learning is performed in the same manner as the above-described processing with respect to step S3 of FIG.

In the above-described embodiment, the diameter of the window area is set to be larger than the minimum dimension of the paint portion of the road marking to be detected, but the size of the window region can be set to be smaller than the minimum dimension of the paint portion. In this case, the point cloud filtering means 24 creates a region in the painted portion where the laser reflection intensity is determined to be uniform. That is, the road marking object extracted by the road marking candidate point group extraction means 28 in step S2 of FIG. 2 is only the reflection point group in the edge region of the paint portion, and the reflection point group is removed inside the paint portion. Regions can arise. In this case, in step S3, for example, the image generation means 30 fills the gap between the point groups at the edge of the paint portion having a high density of the reflection point group, and fills the gap between the point groups to form a hollow road marking candidate figure along the contour of the paint portion. The image feature extraction means 32 can obtain the center of gravity and the contour from the hollow road marking candidate figure, and further extract the image feature. Further, the image generating means 30 may be configured to generate a road marking candidate figure in which the hollow portion is filled.

The road markings detected from the laser measurement data 40 by the road marking detection system 2 can be used for intelligent transport systems (ITS), data archives for automatic driving, and road map creation.

2 Road marking detection system, 4 Arithmetic processing device, 6 Storage device, 8 Input device, 10 Output device, 20 Allowable slope point cloud extraction means, 22 Road surface candidate point cloud extraction means, 24 point cloud filtering means, 26 point cloud clustering Means, 28 road marking candidate point cloud extracting means, 30 image generating means, 32 image feature extracting means, 34 road marking discriminating means, 40 laser measurement data, 42 classifier.

Claims (8)

It is a road marking detection device that detects road markings based on the measurement data of the reflection point cloud acquired from the road surface by a laser scanner.
An index value indicating the degree of variation in the laser reflection intensity of the reflection point group existing in a predetermined vicinity region of the reflection point on the road surface is calculated in association with each reflection point constituting the reflection point group. A filtering means for determining a region of the road surface where the laser reflection intensity is uniform based on the index value and obtaining a reflection intensity undulation point group obtained by removing the region of the reflection point group.
Clustering of the distribution on the road surface is performed on the reflection intensity undulation point cloud according to the preset required resolution for the detection of the road marking, and a point cloud cluster separated for each road marking is generated. Clustering means and
A road marking detection device characterized by having. In the road marking detection device according to claim 1, further
Regarding the laser reflection intensity, the reflection point group belonging to the point group cluster is classified according to a threshold value determined for each point group cluster based on a predetermined standard, and the reflection point group having the laser reflection intensity equal to or higher than the threshold value is defined as 1. Having a road marking candidate point cloud extraction means for extracting as a road marking candidate point cloud corresponding to the said road marking.
A road marking detection device characterized by. In the road marking detection device according to claim 1 or 2.
A road marking detection device characterized in that the predetermined neighborhood region is a circle whose diameter is larger than the minimum dimension of the construction portion of the road marking to be detected and equal to or less than the required resolution. In the road marking detection device according to claim 2.
An image generation means for generating an image of a road marking candidate figure based on a distribution area of the road marking candidate point group on the road surface, and an image generating means.
The frequency spectrum is calculated by discrete Fourier transforming the distance from the center of gravity of the road marking candidate figure to each contour pixel, and as an image feature of the road marking candidate figure, a predetermined frequency width at the end on the low frequency side in the frequency spectrum. Image feature extraction means for extracting the component in the above and the component in the predetermined frequency width at the end on the high frequency side, and
Using a classifier that has learned the image features of the road marking to be detected in advance, a road marking determining means for determining whether the road marking candidate figure is the road marking, and a road marking determining means.
A road marking detection device characterized by having. In the road marking detection device according to claim 4,
The road marking detection device is characterized in that the image feature extracting means further calculates the moment of the road marking candidate figure as the image feature. It is a road surface detection device that detects the road surface based on the measurement data of the reflection point cloud acquired from the ground surface by a laser scanner.
Delaunay triangulation is performed based on the point cloud obtained by projecting the reflection point cloud onto the horizontal plane to generate a Delaunay triangle having the reflection point group as the apex, and the gradient of the Delaunay triangle from the reflection point group has a predetermined upper limit. An allowable gradient point cloud extraction means for obtaining an allowable gradient point cloud by removing the apex of the excess,
A road surface candidate point group in which the permissible slope point group is clustered with respect to the distribution on the ground surface, and the obtained clusters having an element number of a predetermined threshold value or more are extracted as the road surface candidate point group. Extraction means and
A road marking detection device characterized by having. It is a road marking detection method that detects road markings based on the measurement data of the reflection point cloud acquired from the road surface by a laser scanner.
An index value indicating the degree of variation in the laser reflection intensity of the reflection point group existing in a predetermined vicinity region of the reflection point on the road surface is calculated in association with each reflection point constituting the reflection point group. A filtering step in which a region of the road surface where the laser reflection intensity is uniform is determined based on the index value, and a reflection intensity undulation point group obtained by removing the region of the reflection point group is obtained.
Clustering of the distribution on the road surface is performed on the reflection intensity undulation point cloud according to the preset required resolution for the detection of the road marking, and a point cloud cluster separated for each road marking is generated. Clustering steps and
A road marking detection method characterized by having. A program for causing a computer to perform a process of detecting a road sign based on measurement data of a reflection point cloud acquired from a road surface by a laser scanner.
An index value indicating the degree of variation in the laser reflection intensity of the reflection point group existing in a predetermined vicinity region of the reflection point on the road surface is calculated in association with each reflection point constituting the reflection point group. A filtering means for determining a region of the road surface where the laser reflection intensity is uniform based on the index value, and obtaining a reflection intensity undulation point group obtained by removing the region of the reflection point group, and
Clustering of the distribution on the road surface is performed on the reflection intensity undulation point cloud according to the preset required resolution for the detection of the road marking, and a point cloud cluster separated for each road marking is generated. A program characterized by functioning as a clustering means.

Images