JP5430627B2 - Road accessory detection device, road accessory detection method, and program - Google Patents

Description

  The present invention relates to a road accessory detection apparatus, a road accessory detection method, and a program for detecting a road accessory based on an image and point cloud data representing a three-dimensional shape of a feature surface.

  Various accessories are installed on the road for the purpose of ensuring safe and smooth traffic. Examples of road accessories include traffic lights, road signs, road markings, guardrails, traffic information boards, pedestrian bridges, sidewalks, separation zones, road lights, and road reflectors. In addition, regarding road accessories, details are stipulated in Article 2 of the Road Law. Some road accessories are called traffic safety facilities. Of the road accessories, there is already a road map describing the position of traffic lights. If the position of other road accessories is also acquired, it can be used as map information, for example, and a road map with higher accuracy can be created.

  Conventionally, as a technique for acquiring position information of road accessories, the following patent document discloses a technique that uses a two-dimensional image acquired by an on-vehicle monocular camera and performs analysis based on the luminance information to detect a guardrail and a white line. 1 is disclosed.

  Patent Document 2 below discloses a technique for detecting side walls such as card rails by calculating three-dimensional position data from a stereo image acquired by a stereo camera.

JP-A-1-242916 JP-A-6-266828

  The technology for detecting road accessories using images acquired with a monocular camera performs analysis based only on the spectral information of the images, so it accurately recognizes information about road accessories in a complex road environment. It is difficult to extract and position information cannot be acquired. For example, it is difficult to discriminate between guardrails and white lines on the road surface.

  In the technique using the stereo camera, the analysis for obtaining the three-dimensional data of the feature from the stereo image is complicated, and the density of the points where the three-dimensional data is obtained by stereo matching is relatively coarse, so that high-precision analysis is difficult. In addition, since the surrounding environment of the road is complicated, it is not easy to stably and automatically detect road accessories such as road signs and guardrails.

  Here, there is a technique for acquiring high-density point cloud data related to a road surface, a feature, or the like using an in-vehicle laser. In this technology, the feature is estimated from a shape represented by the point cloud data. Therefore, it is difficult to recognize the display content of road signs and the like drawn on a flat plate using a plurality of colors among road accessories. Conventionally, it has been necessary to manually read a feature based on point cloud data, and a feature extraction operation has been performed manually using an editing tool or the like in 3D CAD. .

  The present invention has been made to solve the above problems, and includes a road accessory detection device, a road accessory detection method, and a program that can automatically detect a road accessory with high accuracy. The purpose is to provide.

  The road accessory detection device according to the present invention detects a road accessory in the target space based on an image of the target space and point cloud data representing the three-dimensional shape of the feature surface in the target space. A storage means for preliminarily storing a form feature of the detection target including information about at least one of an image feature of the road appendage to be detected and a shape, size, and arrangement in the target space. And an image dividing means for dividing the image into a plurality of partial image areas each having a pixel value having a predetermined similarity, and the partial image area based on the image features stored in the storage means. Among them, candidate image region extracting means for extracting a candidate image region corresponding to the detection target, wall surface detecting means for detecting the wall surface of the feature based on the point cloud data, Candidate wall surface extracting means for extracting a candidate wall surface corresponding to the detection target among the detected wall surfaces based on the morphological features stored in the storage means, the candidate image region, and the image shooting position. Collation means for collating the projected image of the candidate wall surface and detecting the detection object based on the collation result.

  In the road accessory detection device according to the present invention, the collating means detects the guard rail that is the detection target in a region excluding a portion outside the road and a roadway portion of the road in the target space. Can do.

  In the road accessory detection apparatus according to the present invention, the verification means may detect the road sign that is the detection target in a region excluding a portion outside the road in the target space. Further, the road accessory detection device according to the present invention is a road sign identifying means for identifying a type or display content of the road sign by template matching in an image area corresponding to the road sign for the detected road sign. It can be set as the structure which has these.

  The road accessory detection device according to another aspect of the present invention further includes road marking detection means for detecting the road marking as the detection target, and a road surface image corresponding to the road surface in the image based on the point cloud data. Road image area extraction means for extracting an area, the candidate image area extraction means extracts a road marking candidate that is the candidate image area for the road marking, and the road marking detection means includes the road surface image The road marking candidate in the area is determined as the road marking. Further, in the road accessory detection device according to the present invention, the road marking detection unit can identify the content of the road marking by template matching for the road marking candidate determined to be the road marking. .

  The road accessory detection method according to the present invention is a method for detecting a road accessory in the target space based on an image of the target space and point cloud data representing a three-dimensional shape of the feature surface in the target space. An image dividing step for dividing the image into a plurality of partial image regions each having a pixel value having a predetermined similarity, and an image feature given in advance of the road accessory to be detected A candidate image region extraction step for extracting a candidate image region corresponding to the detection target from the partial image region, a wall surface detection step for detecting a wall surface of the feature based on the point cloud data, The wall surface detected based on a pre-given form feature including information on at least one of shape, size, and arrangement of the detection target in the target space A candidate wall surface extraction step for extracting a candidate wall surface corresponding to the detection target, the candidate image region, and a projection image of the candidate wall surface viewed from the image capturing position of the image are collated, and the detection is performed based on a collation result. And a collating step for detecting an object.

  The program according to the present invention performs processing for detecting a road accessory in the target space based on an image obtained by photographing the target space and point cloud data representing the three-dimensional shape of the feature surface in the target space. An image dividing means for dividing the image into a plurality of partial image areas each having a pixel value having a predetermined similarity, and a detection target among the road accessories. A candidate image region extracting unit that extracts a candidate image region corresponding to the detection target from the partial image region based on an image feature given in advance, and detecting a wall surface of the feature based on the point cloud data Wall surface detection means, a pre-given shape including information on at least one of shape, size and arrangement of the detection target in the target space Candidate wall surface extraction means for extracting a candidate wall surface corresponding to the detection target among the detected wall surfaces based on characteristics, the candidate image region, and a projected image of the candidate wall surface viewed from the image capturing position And collating means for detecting the detection object based on the collation result.

  According to the present invention, road accessories can be automatically detected with high accuracy.

1 is a block diagram showing a schematic configuration of a road accessory detection system according to an embodiment of the present invention. It is a schematic diagram which shows the example of a road sign. It is a general | schematic flowchart of the road appendage detection process of this invention which made the road sign and the guardrail the detection object. It is a flowchart explaining the collation process in case a detection target is a road sign. It is a flowchart explaining the collation process in case a detection target is a guardrail. It is a general | schematic flowchart of the road accessory detection process of this invention which made the road marking a detection target. It is a schematic diagram of the image image | photographed with the vehicle-mounted camera. It is a schematic diagram which shows the result of having processed the image of FIG. 7 by the image division means. It is a block diagram which shows the schematic structure of a wall surface detection means. It is a general | schematic flowchart of the wall surface extraction process by the wall surface extraction system which concerns on embodiment of this invention. It is a general | schematic flowchart of the process by a partial space setting means and a partial space selection means. It is a general | schematic flowchart of the process by a block space setting means and a horizontal plane search means. It is a schematic diagram explaining the search process of the edge in a block space. It is a general | schematic flowchart of the process by a vertical position determination means and a wall surface shape determination means. It is explanatory drawing for demonstrating grouping of the point group in the vertical search space by a vertical position determination means. It is a schematic diagram explaining the process which calculates | requires the shape of a wall surface element. It is a general | schematic flowchart of the process by a tracking means. It is a general | schematic flowchart of a continuous wall surface tracking process. It is a general | schematic flowchart of a continuous wall surface tracking process. It is a schematic flowchart of edge fine adjustment processing. It is a schematic flowchart of edge fine adjustment processing. It is a general | schematic flowchart of the process by a wall surface element registration means. It is a processing flowchart after finishing the process about the main edge in a certain hierarchy. It is a schematic flowchart of multiple edge processing. It is a schematic diagram explaining a multiple edge process.

  Hereinafter, a road accessory detection system 2 according to an embodiment of the present invention (hereinafter referred to as an embodiment) will be described with reference to the drawings. This system is a road accessory detection device that detects a road accessory in the target space based on an image of the target space and point cloud data representing the three-dimensional shape of the feature surface in the target space. For example, the image and the point cloud data can be acquired while traveling on a road using a mobile mapping system in which a camera and a laser scanner are mounted on an automobile.

  In the mobile mapping system, the laser scanner is mounted on the top of the automobile and irradiates the surrounding area with a laser. The optical axis of the laser is scanned in the vertical and horizontal directions by changing the elevation angle and azimuth angle, and laser pulses are emitted every minute angle within the scanning range. The distance is measured based on the time from the laser emission to the reception of the reflected light, and at that time, the laser emission direction, time, and the position / posture of the vehicle body are measured. From these measurement data, point group data representing the three-dimensional coordinates of the point reflecting the laser pulse is obtained. Simultaneously with the acquisition of point cloud data, a color image is taken using a camera. The measurement of the position and orientation of the vehicle body is performed using, for example, a GPS / IMU (Global Positioning System: Inertial Measurement Unit). Measurement may be performed by installing a laser scanner or a camera on the ground.

  In order for the point cloud data to represent the three-dimensional shape of the feature surface, it is necessary to perform laser scanning at a point density corresponding to the scale of unevenness on the feature surface. It needs to be shot at resolution. In this regard, laser scanning and image shooting performed from the ground such as roads using vehicles and tripods are, for example, road signs installed along roads, road markings, road accessories such as guardrails, and buildings built in the vicinity of roads In addition, it is possible to realize a scanning density and image resolution that can capture the shape of the vertical surface (wall surface) of the bag and the signboard and the pattern of the surface.

  FIG. 1 is a block diagram showing a schematic configuration of a road appendage detection system 2. The system includes an arithmetic processing device 4, a storage device 6, an input device 8, and an output device 10. As the arithmetic processing device 4, it is possible to make dedicated hardware for performing various arithmetic processing of this system, but in this embodiment, the arithmetic processing device 4 uses a computer and a program executed on the computer. Built.

  A CPU (Central Processing Unit) of the computer constitutes an arithmetic processing unit 4, which will be described later, image segmentation means 20, candidate image region extraction means 22, wall surface detection means 24, candidate wall surface extraction means 26, collation means 28, road sign identification It functions as means 30, road surface image area extraction means 32, and road marking detection means 34.

  The storage device 6 is composed of a hard disk or the like built in the computer. The storage device 6 converts the arithmetic processing unit 4 into an image dividing means 20, a candidate image area extracting means 22, a wall surface detecting means 24, a candidate wall surface extracting means 26, a collating means 28, a road sign identifying means 30, a road surface image area extracting means 32 and a road. A program for functioning as the sign detection means 34, other programs, and various data necessary for processing of the system are stored. For example, the storage device 6 stores point cloud data of an analysis target space and an image of the target space as processing target data. For example, a road and its vicinity area are set as the analysis target space. In-vehicle GPS / IMU measurement data obtained by measuring the position and orientation of a vehicle equipped with a camera and a laser scanner are also stored in the storage device 6.

  The storage device 6 also includes image feature information 36 of a road accessory to be detected and information on the configuration of the detection target, that is, the shape, size, and arrangement (position, orientation) in the target space. The detection target morphological feature information 38 including information on at least one of them is stored in advance.

  The input device 8 is a keyboard, a mouse, or the like, and is used for a user to operate the system.

  The output device 10 is a display, a printer, or the like, and is used to indicate the position of the road accessory in the target space obtained by the present system to the user by screen display, printing, or the like.

  For example, the present system uses road signs, road markings, and guardrails as detection targets. The road signs are classified into four types: guide signs, warning signs, regulatory signs, and instruction signs, and auxiliary signs for supplementing the meaning of the signs.

  FIG. 2 is a schematic diagram showing an example of a road sign. A guide sign is a sign indicating information such as a place name or a destination at an intersection. Basically, information signs related to general roads are written in white on white, and information signs related to automobile roads are written on white in white. For example, the guide sign 60 shown in FIG. 2 (a) indicates the direction, direction and distance on a general road, and characters and symbols are drawn in white on a blue rectangular plate. Become. Some general road guide signs have a white frame and a blue frame and display content. In addition, there is a guide sign having a shape other than a rectangle. For example, the guide sign 62 of the national road number shown in FIG. 2B is a rounded inverted triangle, and the prefectural road number shown in FIG. The guide sign 64 is hexagonal. In addition, there are road signs for general roads in which letters and the like are written in blue on a white framed blue frame. For example, the prefectural guidance sign 66 shown in FIG. 2D is drawn with a blue frame 66a and a prefectural sign 66b on a white rectangular plate.

  The warning sign is a sign indicating what should be warned. A warning sign 70 shown in FIG. 2 (e) has a yellow frame with a black frame 70a and a basically black pattern 70b on a rhombus (a square with diagonal lines oriented vertically and horizontally).

  Restriction signs are signs that prohibit or restrict any action, and many are round, and prohibition and thorough matters are drawn with red borders and blue letters and pictures inside, or red diagonal lines. For example, a red-slashed line 72b and a blue character 72c are drawn on the inside of the red border 72a on the traffic-blocking restriction sign 72 shown in FIG. There is also a regulation sign that shows a white pattern on a round or rectangular blue plate.

  The instruction sign is a sign indicating a facility on the road such as some permission or instruction, a pedestrian crossing, and in many cases, like the instruction sign 74 shown in FIG. It is.

  The road sign is installed based on the road traffic law and the road law, and as described above, the shape and color of the road sign, the contents of characters / graphics, or the display method thereof are defined in advance. Features based on the rules are stored in advance in the storage device 6 as image feature information 36.

  There are also regulations regarding the size and installation position (height) of road signs. The form feature information 38 includes information on the shape, size, or arrangement of the road sign. The morphological feature information 38 is extracted on the basis of a preliminarily provided rule for them and stored in the storage device 6 in advance.

  The road marking is a line, a symbol or a character, which is drawn on the road surface of the road based on the road traffic law and the road law, for example, a white line indicating a roadside belt or a lane boundary line. As the image characteristic information 36 of the road marking, a predetermined marking pattern and color are stored in the storage device 6. For example, white, orange, etc. are defined as colors. As the road sign form feature information 38, the storage device 6 stores in advance that the installation position is a road surface.

  The guardrail is a protective fence provided at a boundary with a road shoulder or a sidewalk or a median strip, and a steel plate or the like extends horizontally along the road to a height of about 80 cm. The standard color is white, but colors other than white, such as yellow, are also used in consideration of the landscape, and there are gray and silver colors on expressways and national roads. As the image feature information 36 of the guardrail, a shape and color elongated in the horizontal direction are stored in the storage device 6. Further, as the shape feature information 38, the shape, the range of the vertical dimension, the installation height, and the like are stored in the storage device 6 in advance.

  FIG. 3 is a schematic flowchart of the road appendage detection process for detecting road signs and guardrails by the road appendage detection system 2. Each means related to the processing of the arithmetic processing unit 4 will be described with reference to FIG.

  The image dividing means 20 extracts a plurality of fragments from the image by grouping adjacent pixels having pixel values similar to each other in the color image taken by the in-vehicle camera (S1). As a result, the image is divided into a plurality of image fragments (segments) including pixels having pixel values having a predetermined similarity. Here, each image fragment is referred to as a partial image region. The process of dividing into partial image areas is generally realized by a technique called segmentation. For example, k-average method and average shift method are known as typical segmentation methods.

  The candidate image area extraction unit 22 refers to the image feature information 36 and extracts a partial image area having an image feature (color, shape) to be detected as a candidate image area (S2).

  The wall surface detecting means 24 detects the wall surface of the feature based on the point cloud data acquired by the in-vehicle laser scanner (S3). The wall surface detection process S3 will be further described later.

  The candidate wall surface extraction means 26 refers to the form feature information 38 and extracts a detected wall surface having a form feature (shape, size, arrangement) to be detected as a candidate wall surface (S4).

  The collating unit 28 collates the candidate image area with the projected image of the candidate wall surface viewed from the image capturing position, and detects the detection target based on the collation result (S5). Specifically, the collation unit 28 sets a projection plane based on the viewpoint / line-of-sight direction of the in-vehicle camera, and obtains a projected image of the candidate wall surface on the projection plane. Then, the position / shape / size in the image of the candidate image region is compared with the position / shape / size of the projection image of the candidate wall surface in the projection image, and the position / shape / size is the same. A pair of a candidate image region and a candidate wall surface that are determined to be in correspondence with each other within a predetermined range that can be considered is obtained.

  FIG. 4 is a flowchart for explaining the collation processing S5 when the detection target is a road sign. The position of the road sign in the horizontal plane is a road (a roadway part, a sidewalk part, and a shoulder), and no road sign is arranged outside the road. Therefore, the collation means 28 detects a road sign in an area excluding a portion outside the road in the target space. That is, when there is a candidate wall surface corresponding to the candidate image area of the road sign, the position in the horizontal plane is checked, and if it is outside the road, it is not detected as a road sign (S10). Here, the road area in the horizontal plane can be determined using road data obtained in advance. In addition, by analyzing the point cloud data acquired by the in-vehicle laser scanner, the road surface, the wall surface position of the building along the road, the step position of the road shoulder and the boundary of the sidewalk can be extracted. It is also possible to determine road areas that can be installed.

  The candidate wall surface extraction unit 26 is configured to extract the candidate wall surface of the road sign in an area excluding the portion outside the road, instead of the narrowing down of the road sign candidates based on the position in the horizontal plane described above by the matching unit 28. May be.

  In addition to narrowing down candidates based on the above-described position in the horizontal plane for the road sign, the matching means 28 further narrows down candidates based on height (S11). Regarding the installation height of road signs, there is an installation standard for each installation method such as roadside, cantilever, gate-type, and attachment-type. The matching means 28 uses the height of the candidate wall surface based on the laser point cloud data. If the height does not conform to the road sign setting standard, the road sign is present even if a candidate image region corresponding to the candidate wall surface exists. Does not detect as. Here, the height of the roadside type is set to a relatively low range, while the cantilever type, the gate type, and the attachment type in which the vehicle passes under the sign defines a relatively high range. Therefore, if the position of the candidate wall surface in the horizontal plane is the edge of the shoulder or sidewalk, the candidate is narrowed down to the height range according to the roadside installation standards, while the position of the candidate wall surface in the horizontal plane is within the roadway In this case, it is possible to adopt a configuration that narrows down to height range candidates according to cantilever type, gate type, and attachment type installation standards.

  Also, a standard is set for the size of the road sign, and the matching means 28 uses the size of the candidate wall surface based on the laser point cloud data, and if the size does not conform to the standard of the road sign, Even if a candidate image region corresponding to the candidate wall surface exists, it is not detected as a road sign (S12).

  Of the set of candidate image areas and candidate wall surfaces corresponding to each other, the remaining one after the narrowing down by the above-described processing S10 to S12 is determined to be a road sign.

  Here, in step S2, the candidate image region is not extracted in a shape corresponding to the road sign because the color on the image becomes unclear due to the adjustment of light rays, or a part of the road sign is hidden by a tree or the like. As a result, collation may not be established and extraction may occur. The verification means 28 can be configured to correct the extraction omission (S13). For example, image information of an area corresponding to the projected image of the candidate wall surface of a candidate wall surface of a position and size that satisfies a road sign installation standard that does not correspond to the candidate image area of the road sign is found. It is determined whether it is a road sign based on. For example, a color histogram of pixel values in a region corresponding to the projection image is obtained, and the road sign is obtained from the intensity of each color (blue, green, red, yellow, white, black) used for the road sign and the balance of the color intensity. It is possible to determine whether or not, and what kind of sign is likely to be. In the determination, the position / shape / size of the candidate wall surface may be taken into consideration. For example, when there are a lot of yellow and black, if the shape is a diamond, it is determined to be a road sign or a warning sign, but if it is another shape such as a round shape, it is determined not to be a road sign can do.

  The collating means 28 performs a process similar to that for the road sign described above for the guardrail. FIG. 5 is a flowchart for explaining the collation processing S5 when the detection target is a guardrail. The position of the guardrail in the horizontal plane is basically the road shoulder, the boundary between the roadway and the sidewalk, and the median strip, the outer part of the road (shoulder), and the roadway (the area where the vehicle travels) No guardrails are placed on the median strip). Therefore, the collating means 28 detects the guardrail in an area excluding the outer portion and the roadway portion from the road in the target space. That is, when there is a candidate wall surface corresponding to the candidate image area of the guardrail, the position in the horizontal plane is checked, and if it is outside the road or inside the roadway, it is not detected as a guardrail (S15).

  The candidate wall surface extracting unit 26 extracts the guard rail candidate wall surface in the region excluding the outer portion and the roadway portion from the road instead of performing the above-described narrowing down of the guard rail candidates based on the position in the horizontal plane. You may comprise.

  The collating means 28 further narrows down the candidates based on the height in addition to the narrowing down of candidates based on the above-described position in the horizontal plane for the guardrail (S16). Further, the collating means 28 calculates the size and direction of the candidate wall surface based on the laser point cloud data, and if they do not match the guard rail, it is used as a guard rail even if a candidate image region corresponding to the candidate wall surface exists. It is not detected (S17).

  Of the set of candidate image regions and candidate wall surfaces corresponding to each other, the remaining one after the narrowing down by the above-described processing S15 to S17 is determined to be a guardrail.

  Here, the collating means 28 can be configured to correct the extraction omission also for the guardrail (S18). For example, when a candidate image area corresponding to a candidate wall surface of the guard rail is not found, it is determined whether or not it is a guard rail based on image information of an area corresponding to the projected image of the candidate wall surface. For example, a color histogram of pixel values in a region corresponding to the projected image is obtained, and if there is much white, it can be determined as a guardrail.

  The collation process S5 in FIG. 3 has been described with reference to FIGS. Subsequent to the collating process S5, the arithmetic processing unit 4 performs a road sign identifying process S6 and a detected road sign and guardrail position determining process S7 as shown in FIG.

  The road sign identifying means 30 performs road sign identification processing, and identifies the type of sign (whether it is a guide sign, a warning sign, a regulation sign, an instruction sign, or an auxiliary sign) or display contents (S6). The road sign identifying means 30 identifies the type or display content of the road sign by template matching in the image area corresponding to the road sign. The type of road sign and the template of display contents are stored in advance in the storage device 6 as, for example, image feature information 36. Alternatively, when a road sign database is obtained in advance, it may be stored in the storage device 6 and used instead of the image feature information 36. Some road signs have predetermined display contents, and some display contents are changed according to place names and distances. Of these, those whose display contents are fixed in advance are stored in the storage device 6 as templates, and this is matched with candidate image areas determined to be road signs, so that the sign type and display contents are Identified. For a display content including a part to be changed such as a place name or a distance, a template excluding the part to be changed is stored in the storage device 6 in advance and used for matching processing, and the type of sign can be identified. In this case, it is also possible to separately recognize the image contents such as the place name and the distance by character recognition. In template matching, a template is stored as an image viewed from a front position that is a predetermined distance away from a candidate wall surface, and for example, an image of a candidate image region is viewed from the front of the candidate wall surface using the position information of the candidate wall surface. It is preferable to perform the conversion after converting the image into a new image, or conversely performing coordinate conversion in the three-dimensional space according to the candidate image region.

  The arithmetic processing unit 4 obtains the position of the road sign and guardrail determined as described above using the measurement data of the in-vehicle GPS / IMU (S7). For example, for the confirmed road sign and guardrail, the arithmetic processing unit 4 detects a candidate wall surface, a candidate image region (or an image of an area corresponding to a projected image of the candidate wall surface), the type / display content and position of the road sign. As a result, it is stored in the storage device 6 or output to the output device 10.

  FIG. 6 is a schematic flowchart of a road accessory detection process for detecting road markings by the road accessory detection system 2. The candidate image area extracting unit 22 extracts candidate image areas (road marking candidates) for road marking from the partial image areas obtained by the image dividing unit 20 (S20). The storage device 6 stores image characteristic information 36 for road markings in advance. Alternatively, when a road marking database is obtained in advance, it may be stored in the storage device 6 and used instead of the image feature information 36. Candidate image region extraction means 22 uses these to extract road marking candidates by comparing feature information and images.

  The position of the road marking is the road surface. Therefore, the road surface image area extraction unit 32 extracts a road surface image area corresponding to the road surface from the image based on the laser point cloud data. Then, the road marking detection means 34 checks whether or not the position of the road marking candidate image is within the road surface image area, and if it is within the road surface image area, the road marking candidate is determined as a road marking (S21). Furthermore, the road marking detection means 34 may identify the display content (a marking pattern or color) of the road marking. In this case, the display content of the road sign is stored in the storage device 6 as image feature information 36 or a road sign database, and the road sign detection means 34 performs a matching process with a road sign candidate determined as a road sign, and displays the display content. Identify. In template matching, a template is stored as an image viewed from a position that is a predetermined distance away from the road marking candidate in the normal direction of the road surface, and the position information represented by the laser point cloud data is used, for example. It is preferable to convert the above image into an image viewed from the normal direction of the road surface, or conversely, after converting the coordinates in the three-dimensional space in accordance with the road marking candidate.

  FIG. 7 is a schematic diagram of an image photographed by an in-vehicle camera. A roadside road sign 80, a cantilever road sign 82, a guardrail 84, and a road sign 86 are photographed. FIG. 8 is a schematic diagram showing the result of processing the image of FIG. 7 by the image dividing means 20, and the image is divided into a plurality of partial image areas 90.

  Next, the wall surface detection process S3 will be described. FIG. 9 is a block diagram showing a schematic configuration of the wall surface detecting means 24. The wall surface detection unit 24 includes a partial space setting unit 40, a partial space selection unit 42, a block space setting unit 44, a horizontal plane search unit 46, a vertical position determination unit 48, a wall surface shape determination unit 50, a tracking unit 52, and a wall surface element registration. Means 54 are provided.

  FIG. 10 is a schematic flowchart of wall surface extraction processing by the wall surface detection means 24. Each means which comprises the wall surface detection means 24 is demonstrated referring this FIG.

  The partial space setting means 40 sets a plurality of columnar partial spaces obtained by dividing the target space by vertical planes as initial unit spaces for wall surface extraction processing (S40). In this embodiment, the partial space setting means 40 divides the target space into a two-dimensional orthogonal lattice (mesh) in a horizontal plane (XY plane in the XYZ orthogonal coordinate system) and forms a quadrangular prism-shaped partial space. Is generated. For example, the horizontal cross section of the partial space can be a square having a width W and a depth D of 50 cm. The height H of the partial space can coincide with the height of the target space (dimension in the Z-axis direction), and the height is the height of the wall surface to be detected, specifically, road signs and guardrails are arranged. It is set larger than the height. In the program constituting the partial space setting unit 40, the width W, the depth D, and the height H are parameterized and can be changed by the user using the input device 8, for example.

  The partial space selection means 42 selects a partial space including a point group having a height difference equal to or higher than a preset threshold value as a target partial space. Specifically, the subspace selection unit 42 selects a plurality of set subspaces in a predetermined order, and pays attention to the subspace when a point difference in the subspace has a height difference equal to or greater than a threshold value. Let it be a subspace. As described above, the point cloud data is acquired by laser irradiation from the laser scanner mounted on the vehicle on the road, and the wall surface visible from the laser scanner is captured on the three-dimensional shape by the point cloud data. Therefore, for example, the subspace selection means 42 determines a reference plane based on the portion of the plane corresponding to the road or the like in the three-dimensional shape, and horizontally in the width of the road along the road. A line is set (S42). Then, the starting point of selection of the partial space is set in order along the center line, the partial space is selected in the order from the starting point in both directions along the line orthogonal to the center line (S44), and the selected portion If the difference in level of the point group in the space is equal to or greater than the threshold, the subspace is determined as the target subspace (S46).

The block space setting unit 44 divides the target partial space by a horizontal plane and sets a plurality of block spaces that are stacked vertically (S48). For example, it is possible to set the width W, comparable to the depth D of the height H _B subspace of the block space.

  For each block space, the horizontal plane search means 46 searches for a line segment where the point group projected onto the horizontal plane in the horizontal plane gathers in the vicinity of a predetermined reference or more, and finds the line segment on the wall surface in the horizontal plane. It is determined as a position (horizontal wall surface position) (S50). By searching for the position of the wall surface in the horizontal plane for each block space, it is possible to discriminate between a plurality of wall surfaces with different positions and orientations that may exist at different heights in the subspace of interest in the vertical direction.

  The vertical position determining means 48 obtains the vertical position of the wall surface existing at the horizontal wall surface position in the target space (S52). Specifically, the vertical position determining means 48 presets, for each horizontal inner wall surface position detected in the target partial space, a projection position on the horizontal surface among the point group of the target partial space from the horizontal inner wall surface position. What is within the distance is extracted as a point of interest group. Then, for the attention point group extracted for each horizontal plane inner wall surface position, each range in which the attention point group is equal to or greater than a preset threshold with respect to the vertical position is defined as the vertical wall surface position at the horizontal plane inner wall position. Thereby, based on the horizontal plane inner wall surface position detected in a certain block space, the wall surface which exists in the said horizontal plane inner wall surface position in the height range other than the said block space is also detected. For example, one vertical wall surface position is determined for a wall surface extending over a plurality of continuous block spaces. In addition, when a plurality of wall surfaces separated in the height direction exist at the same horizontal plane wall surface position, that is, when the wall surfaces form a hierarchy in the vertical direction, the vertical wall surface position is obtained for each of the wall surfaces forming the hierarchy.

  A set of a horizontal plane inner wall surface position and a vertical wall surface position with respect to the horizontal plane inner wall surface position represents an arrangement of one wall surface in the target partial space. Since the wall surface may be a part of a larger wall surface that extends outside the target partial space, that is, in the lateral direction, it is referred to as a wall surface element here.

  The wall surface shape determining means 50 obtains the shape of the wall surface element existing at each vertical wall surface position at the horizontal wall surface position based on the distribution range of the point of interest group at the vertical wall surface position (S54). Specifically, the wall surface shape determining means 50 projects the point of interest on a vertical plane passing through the horizontal inner wall surface position, and obtains the shape of the wall surface element based on the distribution range of the projected point group. The shape is a trapezoid that approximates the distribution range of the point cloud projected on the vertical plane with the upper and lower bases, which are two parallel sides, set in the vertical direction.

  The tracking means 52 obtains a new horizontal plane inner wall surface position connected to the end of the horizontal plane inner wall surface position for each vertical wall surface position in the horizontal plane inner wall surface position that has already been detected (S56). The tracking means 52 sets a rectangular parallelepiped space in the lateral direction of the wall surface element, projects a point cloud in the adjacent space onto a horizontal plane, and creates a line segment where the projected point group gathers in the vicinity of a preset reference or more. A new horizontal plane inner wall surface position is determined by searching. By sequentially repeating the process, the horizontal wall surface position of the wall surface that continues in the horizontal direction from the wall surface element specified in the target partial space is traced.

  Specifically, the adjacent space is set to a rectangular parallelepiped whose planar shape is a rectangle in which the midpoint of the short side is connected to the end of the detected horizontal plane inner wall surface position. Here, the center line along the long side of the rectangle is the direction of the adjacent space in the horizontal plane. Further, the vertical range defined by the height of the adjacent space, that is, the upper surface and the bottom surface of the rectangular parallelepiped is arranged at a position corresponding to the vertical wall surface position.

  First, the tracking unit 52 sets the adjacent space in a direction in which the position of the detected inner surface of the horizontal plane is straightly extended. That is, the adjacent space is arranged so that the rectangular center line of the planar shape coincides with the extension line of the detected inner surface of the horizontal plane. And when a point group gathers in the said adjacent space more than the reference | standard set beforehand, the line segment along the distribution in the horizontal surface of the point group in the said adjacent space is made into a new horizontal plane inner wall surface position. On the other hand, when a new horizontal plane inner wall surface position is not detected in the adjacent space set by matching the direction with the extension line of the already detected horizontal plane inner wall surface position, the tracking means 52 changes the direction of the adjacent space. Find the position of the inner wall surface. In other words, among the adjacent spaces whose orientations are changed, a search is made for the one in which the point cloud is most concentrated inside, and the obtained line segment along the distribution in the horizontal plane of the point cloud in the adjacent space is determined as a new horizontal plane inner wall surface. Position.

  Information regarding the wall surface element detected in the target space as described above is registered in the storage device 6 by the wall surface element registration means 54. The registered wall surface shape information and connection relationship information can be used for recognizing the road surface shapes of road signs and guardrails, and can be used for grasping road edge steps and roadside building shapes.

  Hereinafter, a processing example of the wall surface detection unit 24 will be described in more detail. 11, 12, 14, and 17 to 24 are process flow diagrams showing a series of processes executed by the arithmetic processing unit 4 in a divided manner. The processing shown in FIG. 11 corresponds to the processing by the partial space setting means 40 and the partial space selection means 42 described above. When the wall surface detection unit 24 starts the wall surface extraction process (S60), the partial space setting unit 40 sequentially creates partial spaces in the target space. For example, when dividing the target space into a mesh shape along the X-axis direction and the Y-axis direction, if the width W and the depth D of the partial space are set as parameters, the intervals between the meshes (the size of the partial space) Thus, the number of partial spaces arranged in the X and Y directions is determined according to the size of the target space. The range of the index indicating the position of the partial space in each of the X and Y directions is determined based on the number of arrays. The partial space setting means 40 changes an index representing a two-dimensional mesh-like arrangement of the partial space in a predetermined order, and is defined by a position (coordinate range in each direction of X and Y) corresponding to the set index. A space is set (S62).

  Here, when the point cloud data is acquired by performing laser irradiation from the vehicle traveling on the road, as described above, the reference plane is based on the portion of the plane corresponding to the road or the like in the three-dimensional shape represented by the point cloud data. Is set, and the center line of the three-dimensional shape is set along the road. In order to simplify the explanation, the center line is assumed to be the X axis (ie, Y = 0), the index in the X direction increases in the direction in which the vehicle travels, and the Y direction corresponding to the road width direction leaves the center line. It is assumed that the absolute value of the index increases according to For example, the index in the Y direction is set to 0 at the position of the center line, set to 1, 2, 3,... To the left with respect to the traveling direction, and set to -1, -2, -3,. be able to. In this case, for example, the subspace setting means 40 sequentially increments the index in the X direction, and changes the index in the Y direction in both the positive and negative directions of the Y axis at each value of the X direction index. . That is, the subspace setting means 40 sets the subspace by increasing the index in the Y direction sequentially from 0 at a certain index value in the X direction, and sets the subspace by decreasing in order from -1. To do.

  If the index-designated mesh subspace is unprocessed (in the case of “No” in S64), the subspace selecting means 42 extracts the point group in the subspace from the storage device 6 such as a hard disk, for example. The data is taken into a work area such as a RAM (Random Access Memory) (S66). If there is an elevation difference of a predetermined threshold value Th1 or more in the point cloud captured in the partial space (in the case of “Yes” in S68), the partial space selection means 42 determines that the partial space is the target partial space, Processing such as searching for the horizontal wall surface position and the vertical wall surface position in the target partial space is started (processing proceeds to node B in FIG. 12). For example, the threshold value Th1 can be set small enough to eliminate a partial space including only a substantially flat portion of the ground surface such as a road. Further, the threshold value Th1 may be set according to the height of the feature for which the wall surface is to be detected, so that the wall surface of the low feature that is not of interest is not detected. For example, when a road sign is a detection target and a guardrail is not a detection target, the threshold Th1 can be set to about 1 m.

  When the difference in level of the point cloud captured from the set partial space is less than the threshold Th1 (in the case of “No” in S68), the wall surface extraction process in the partial space is not performed. In this case, the partial space setting means 40 sets the next partial space (the process returns to node A).

  When the processing has been completed for all the meshes (in the case of “Yes” in S64), the arithmetic processing device 4 finishes the wall surface extraction processing (S70).

  FIG. 12 is a schematic process flow diagram of a part performed by the block space setting unit 44 and the horizontal plane search unit 46 described above in a series of processes executed by the arithmetic processing unit 4.

Block space setting unit 44, attention subspace is divided by a horizontal plane at constant height H _B, sets a plurality of blocks spaces stacked vertically, the storage device data point group located in each block in the space 6 is taken into a work area such as a RAM (S80). For example, when the width W and the depth D of the partial space are 50 cm, the height H _B of the block space can be 20 to 50 cm. The number of stages of the block space set by the division is n, and the index I is used to represent the 1st floor (I = 1, 2,..., N) in order from the bottom.

  The horizontal plane search means 46 sequentially selects the n-stage block space and checks whether an edge (horizontal wall surface position) exists in the block space. For example, the horizontal plane search means 46 increases the index I by 1 and selects a block space for examining the presence of an edge (S82, S84). The horizontal plane search means 46 checks whether an edge exists in the selected block space on the first floor (S86), and if it does not exist (if “No” in S88), selects the next block space ( S84). If I exceeds n, the process returns to node A in FIG. 11 to search for the next target subspace (S90).

  FIG. 13 is a schematic diagram for explaining the edge search processing S86 in the block space. FIG. 13 is a top view of the block space, showing an example of the arrangement of the rectangle 100 defined by the width W and the depth D of the block space and the point group in the XY plane in the block space. Has been. The horizontal plane search means 46 selects two arbitrary data points PA and PB from the point group in the block space, and uses the line segment L0 in the XY plane having these two points as both ends as an edge candidate line. In addition to the setting, a band-like area EA having a width w (total width 2w) is set on both sides of the line segment L0. The horizontal plane searching means 46 counts the number of data points whose coordinates on the XY plane are located in the area EA. Whether or not the data point is in the area EA can be determined by whether or not the length of the perpendicular line from the data point to the line segment L0 on the XY plane is w or less, for example. In addition, the edge condition imposes that the three-dimensional shape represented by the point group in the area EA is not flat. Specifically, the horizontal plane search means 46 is not flat if the height difference (difference between the maximum elevation and the minimum elevation) ΔZ of the data points included in the area EA is equal to or greater than a preset step threshold γ. judge.

  Here, the width w is a parameter and absorbs the variation in the position in the direction orthogonal to the edge of the data point corresponding to the wall surface. For example, w can be about 3 cm. The height difference threshold γ can be set to about 2 cm, for example.

  The horizontal plane search means 46 sets the candidate lines for all combinations of two data points in the block space and performs the above-described determination, and the most data points gather in the area EA, and ΔZ is greater than or equal to the threshold value γ. A line segment L0 is an edge found in the block space. If an edge exists (“Yes” in S88), it is determined whether the edge is a new one that has not been subjected to processing for determining a wall surface element (S92). The determination is performed based on whether or not the same edge as the edge found in the process S86 exists by searching a list in which the edges to be processed for determining the wall surface element are registered. If the same thing does not exist in the list, it is determined as a new edge (in the case of “No” in S92), and the coordinates (ax, ay) and (bx, by) of both ends PA and PB of the edge are registered in the list. (S94). Thus, when a new edge is detected in a certain block space, the vertical wall surface position search in the target partial space and the shape of the wall surface element are determined for the edge (node D in FIG. 14). To proceed). On the other hand, the same edge as the edge found in the process S86 is already registered in the list in the process for the other block space, and the process such as the search for the vertical wall surface position may be performed. In this case (“Yes” in S92), processing such as registration of the edge in the list and search for the vertical wall surface position is omitted, and the horizontal plane search means 46 performs processing for the next block space ( Processing returns to node C).

  FIG. 14 is a schematic process flow diagram of the vertical position determining means 48 and the wall surface shape determining means 50. The vertical position determination means 48 sets a vertical search space as a space for searching for the position of the wall surface in the vertical direction at the edge L0 extracted in step S86, and data of point groups located in the vertical search space Is taken from the storage device 6 into a work area such as a RAM (S110). The vertical search space is a rectangular parallelepiped space whose planar shape is a rectangle having the edge L0 as the central axis in the longitudinal direction, and the bottom surface and the top surface thereof are positioned at the same height as the bottom surface and the top surface of the partial space. The dimension ξ of the rectangular short side of the planar shape can be set, for example, to be the same as or slightly larger than the width 2w of the strip-shaped area EA from which the edge is extracted. For example, ξ can be set to 10 cm.

  The vertical position determination means 48 groups the point group in the vertical search space based on the distribution in the Z direction (S112). Thereby, the hierarchical structure of the wall surface of the vertical direction in the wall surface position in a certain horizontal surface is caught.

In the grouping, the vertical position determining means 48 sets, for example, a plurality of sections in the Z direction with a constant step size h from the minimum elevation of the point group taken into the vertical search space or upward from the maximum elevation. Then, the point cloud in the vertical search space is divided into a plurality of hierarchies (ranks) based on the section, and the number of point clouds (score) is totaled for each rank. Here, in accordance with the step size h, the resolution at which two wall surfaces adjacent in the vertical direction can be detected separately is determined, and basically h is set to H _B or less. Here, as h becomes smaller, the resolution becomes higher. However, if h is too small, the score in the section of the height range where the wall surface exists becomes small, and the score in the height range where the wall surface does not exist (basically 0). Is expected to be difficult). h is set in consideration of these factors. The vertical position determining means 48 groups the point clouds in the vertical search space based on the point cloud distribution obtained by counting the number of point clouds for each rank. Specifically, each rank range in which the point cloud is gathered is defined as one group of point clouds, and the rank range is defined as one vertical wall surface position.

  FIG. 15 is an explanatory diagram for explaining grouping, and shows an example of a point cloud distribution table as a result of totaling the number of point clouds for each rank with h being 0.2 m. For example, the vertical position determining means 48 checks the rank in order from the bottom (minimum elevation) to the top, and assigns a group number in order to each range where ranks having scores greater than 0 are consecutive. In the example shown in FIG. 15, there are four rank ranges with a score greater than 0: ranks 1 to 4, ranks 9 to 12, ranks 18 to 22, and ranks 25 to 26. ~ 4 is given. If a rank with a score greater than 0 appears alone, that is, if the score is 0 before and after the rank, the vertical position determination means 48 assigns a group number with the one rank as one rank range. .

  The number of hierarchies determined by grouping is represented as m. In the example shown in FIG. The wall surface shape determining means 50 sequentially selects rank ranges (vertical wall surface positions) set in the m level. For example, the wall surface shape determining means 50 increases the group number J by 1 and selects the rank range for determining the shape of the wall surface element (S114, S116). Then, the data of the point group existing in the selected rank range among the point groups existing in the vertical search space is taken into the work area such as the RAM from the storage device 6 (S120), and the trapezoid as the wall surface element is determined from the point group data. Processing is performed (S122). When the shape of the wall surface element can be determined (in the case of “Yes” in S124), the tracking process for sequentially obtaining the horizontal wall surface position of the wall surface element connected in the lateral direction of the wall surface element is started (FIG. 17). Processing proceeds to node F).

  When wall surface element determination processing is completed for all m layers (in the case of “Yes” in S118), the process returns to node C in FIG. 12, the next edge is searched, and the same processing is repeated for the edge. Further, when the shape of the wall surface element cannot be determined in a certain hierarchy (“No” in S124), the process returns to the process S116, and the process for the next hierarchy is performed.

  FIG. 16 is a schematic diagram for explaining the processing S122 for obtaining the shape of the wall surface element, and shows an example of a state in which a point group existing in the vertical search space is projected onto a vertical plane passing through an edge. ing. In FIG. 16, black points 132 distributed on the vertical plane 130 indicated by rectangles are points projected from the point group. In the rectangular vertical surface 130, the lateral direction is along the edge, the left side corresponds to the one end PA side of the edge, and the right side corresponds to the other end PB side. The vertical direction of the vertical plane 130 is the vertical direction, and the upper end and the lower end correspond to the upper end and the lower end of the rank range, respectively. The wall surface shape determining means 50 divides the vertical surface 130 into a grid and defines a plurality of small rectangles 134 arranged in rows and columns. For example, the small rectangle 134 can have a width of about 10 cm and a height of about 5 cm.

The wall surface shape determining means 50 selects columns of small rectangles 134 arranged vertically, for example, in order from the left side, and searches for columns in which there are two or more small rectangles 134 onto which the point cloud is projected. When a range in which such a column appears, that is, a column range from the first position (the η _S column) to the last position (the η _E column) is obtained, a process for specifying the shape of the wall surface element is further performed. Done. On the other hand, when such a column range does not exist, it is determined that no wall surface element exists in the hierarchy.

The wall surface shape determining means 50 is one of two parallel sides of the trapezoid based on the point group (black dot 132) projected on the vertical plane 130, which is located at the left end portion (here, this is defined as the upper base of the trapezoid). The X and Y coordinates of the vertical line to be determined are determined, and the Z coordinates of both ends of the side (upper base) are determined based on the altitude of the point group distributed in the left end portion. In addition, the X and Y coordinates of the vertical line that is the other of the two parallel sides of the trapezoid (here, this is the lower base of the trapezoid) are determined based on what is located at the right end portion, and distributed to the right end portion. Based on the elevation of the point group, the Z coordinates of both ends of the side (lower base) are determined. For example, the wall surface shape determining means 50 determines the X, Y coordinates of the upper base of the trapezoid and the Z coordinates of both ends thereof based on the distribution of point groups in the two columns of the η _S column and the (η _S +1) column. The X and Y coordinates of the lower base of the trapezoid and the Z coordinates of both ends thereof are determined based on the distribution of the point group in the two columns of the η _E column and the (η _E −1) column. In this case, specifically, the maximum value (corresponding to the point 136) and the minimum value (corresponding to the point 138) among the Z coordinates of the point group in the two columns of the η _S column and the (η _S +1) column. Z coordinates of both ends of the upper base can be used, and the maximum value (corresponding to the point 140) and the minimum value (corresponding to the point 140) among the Z coordinates of the point group in the two columns of the η _E column and the (η _E −1) column ( (Corresponding to point 142) can be the Z coordinates of the lower bottom ends. The X and Y coordinates of the upper base are the X and Y coordinates at the leftmost point 144 of the point group and the boundary line 146 between the η _S column and the (η _S +1) column. The X and Y coordinates can be defined by the X and Y coordinates at the rightmost point 148 in the point group or the boundary 150 between the η _E column and the (η _E −1) column.

  FIG. 17 is a schematic process flow diagram of the tracking means 52. The tracking means 52 sets the edge detected in the target subspace as the base point of the continuous tracking process (S160), and performs an initialization process for emptying the edge lists A and B (S162). It is checked whether the wall surface element acquired for the base edge has already been registered in the graphic database (S166). If not registered, that is, if it is new (in the case of “No” in S166), the wall surface element is selected. Add to the edge list A (S168).

  Further, the tracking unit 52 tracks edges that are continuous with both ends of the base edge. Specifically, for each vertical wall surface position, a continuous wall surface tracking process is performed in which edges that are continuous in the direction extending from one end PA of the base edge are detected, and a wall surface element corresponding to the detected edge is obtained. Are additionally registered in the edge list A (S170). Similarly, the continuous wall surface tracking process is also performed in the direction extending from the other end PB of the base edge, and the acquired wall surface element is additionally registered in the edge list B (S172). When the process by the tracking unit 52 is finished for a certain base point edge, the process by the wall surface element registration unit 54 is performed (the process proceeds to the node G in FIG. 22).

  On the other hand, when the wall surface element at the base edge is already registered in the graphic database in the process S166 (in the case of “Yes” in S166), the arithmetic processing unit 4 performs the tracking process for the wall surface element. The registration process is not performed, and the process proceeds to node H shown in FIG. Processing in this case will be described later.

  18 and 19 are schematic flowcharts of the continuous wall surface tracking process performed in processes S170 and S172. The tracking means 52 sets a rectangular parallelepiped search space as a space for searching for an edge that is continuous with an already detected edge. The search space is a space adjacent to the search space of the detected edge, and the midpoint of the short side of the rectangle that is the planar shape of the search space is arranged at the end of the detected edge. Further, the range of the Z value (height of the top surface and the bottom surface) of the search space is set according to the vertical position of the wall surface element at the detected edge. For example, considering that the upper side and the lower side of the trapezoid, which is the shape of the detected wall surface element, may have a gradient, the height of the upper surface of the search space is set to the vertical 2 of the detected wall surface element. It is possible to set the height of the side higher than the upper end of the side to which the search space is connected, while setting the height of the bottom surface of the search space lower than the lower end of the side. The length of the long side of the planar shape of the search space can be set to be approximately the same as the width W and the depth D of the partial space, and can be set to 50 cm, for example.

  First, the tracking means 52 extends to the side where the line segment that is a sharpened edge is to be tracked, and arranges a search space in the direction of the extended line (S180). That is, the search space is arranged so that the center line along the long side of the rectangle which is the planar shape coincides with the extension line. Then, the data of the point group located in the search space is fetched from the storage device 6 into a work area such as a RAM (S180), and an edge is searched for in the search space by the same method as the processing S86 (S182). When an edge is present in the search space (“Yes” in S184), the tracking unit 52 further performs a process of determining a trapezoid that becomes a wall surface element from the point cloud data in the search space in the same manner as the process S122. Perform (S186). When the shape of the wall surface element can be acquired (in the case of “Yes” in S188), an edge fine adjustment process is performed (S190). Then, the process returns to step S180, and the continuous wall surface tracking process is continued with the edge obtained now as the detected edge.

  When the edge or wall surface element cannot be detected in the search space on the extension line of the detected edge (in the case of “No” in the processing S184 and “No” in the S188), the process proceeds to the node CB in FIG. Search processing in a direction deviated from the extension line of the detected edge is performed. In the search process, the search space is set by changing the central axis in the horizontal plane from the position of the extension line to the clockwise direction and the counterclockwise direction. For example, a change range (± θ) of the direction of the central axis from the direction of the extension line and its division number k are set, and a plurality of search spaces having different angles in the horizontal plane by θ / k are set. The edge is searched for, and the edge having the most point cloud is selected from the edges obtained in the plurality of search spaces set by changing the angle (S200).

  When the edge where the first point cloud gathers can be identified (in the case of “Yes” in S202), the tracking unit 52 further determines the trapezoid as the wall surface element from the point cloud data in the search space in the same manner as the processing S122. Processing is performed (S204). When the shape of the wall surface element can be acquired (in the case of “Yes” in S206), a fine adjustment process described later is performed (S208). Then, the process returns to the node CA in FIG. 18, and the continuous wall surface tracking process is continued with the obtained edge as the detected edge.

  When the edge or wall surface element cannot be detected in the search space with the angle changed (“No” in the process S202 and “No” in the S206), the continuous wall surface tracking process ends, and the process is performed. The process returns to the process S170 or S172 called as a subroutine.

20 and 21 are schematic flowcharts of the edge fine adjustment processing S190 and S208. When a line segment indicating an edge is obtained, the arithmetic processing unit 4 divides the line segment into sections of a certain length (S220), searches for a point (representative point) through which the edge passes in each section, and determines the edge as X Instead of a line segment connecting PA and PB in the -Y plane, a polygonal line connecting representative points obtained in each section is used. Here, the number of sections is N. Processing device 4, the representative point P _K is obtained from the section K selected minimum coordinates (X _{K, Y K)} in the horizontal plane of the representative points P _K and altitude values of wall elements in the section The value ZL _K and the maximum value ZH _K are obtained as representative point information. For example, ZL _K and ZH _K can be obtained by interpolating values at both ends PA and PB of the edge.

  First, the arithmetic processing unit 4 uses the one end PA of the edge as a representative point, and stores representative point information about the PA in the edge representative point list (S222). Subsequently, the section number K is incremented by 1 to select a section (S224, S226), and a representative point is searched for in each section (S230). In the processing S230, the horizontal section is a region near the selected section, a space having a height corresponding to the wall surface element is set in the vertical direction, the point cloud data in the section is taken in, and the horizontal section of the space is set. Find the position where the point cloud is most dense. If the number of points crowded at the position is equal to or larger than the predetermined number (in the case of “Yes” in S232), the position is added to the edge representative point list as a representative point (S234). On the other hand, if the number of points crowded at the position is less than a predetermined number (in the case of “No” in S232), the next representative point is not added to the edge representative point list from the relevant section. Proceed to section processing. When the processing is completed for all the sections (“Yes” in S228), the other end PB of the edge is added to the edge representative point list (S236), and the process proceeds to node CV-R in FIG.

The arithmetic processing unit 4 counts the number m of representative points stored in the edge representative point list (S240). Then, select the pair of the representative point _{P J} and _{P J + 1} adjacent increasing by one the J from 1 to (m-1) (S242~S246) , the representative point information and the representative point information _{P J + 1} in _{P J} It is checked whether the wall surface element represented by is already registered in the graphic database (S248). If not registered (in the case of “No” in S250), the wall surface element is added to the edge list A or B (S252), and the process returns to S244. On the other hand, if the graphic has already been registered in the graphic database (“Yes” in S250), the edge fine adjustment processes S190 and S208 are terminated.

  Now, as described in the explanation of FIG. 17, when the processing by the tracking unit 52 is finished for a certain base point edge, the processing by the wall surface element registration unit 54 is performed. FIG. 22 is a schematic process flow diagram of the wall surface element registration means 54. If the edge lists A and B generated by the tracking unit 52 are not empty (in the case of “No” in S260 and S264), the wall surface element registration unit 54 stores the wall surface elements stored in these edge lists in the graphic database. (S262, S266), and the process proceeds to node H in FIG.

  The base edge that has been processed so far is the edge detected in step S88 from the block space on the I-th floor specified in step S84. When the arithmetic processing unit 4 finishes the processing at the J-th layer specified in step S116 for the base point edge, it searches for other edges in the height range corresponding to the layer in the target subspace and searches for the wall surface element. The process of determining is performed. Thereby, for example, multiple edges in the same height range can be detected, and wall elements corresponding to each can be determined. FIG. 23 is a process flow diagram after finishing the process for the first edge (referred to as the main edge) in the J-th layer. After finishing the processing of the main edge in the J-th layer, the arithmetic processing unit 4 searches for another edge (referred to as a multiple edge) existing in the layer and determines a wall element (multiple edge processing) S270. . When the multiple edge process S270 is completed, the process returns to the node E in FIG. 14, and the process of the main edge in the next hierarchy is started.

  FIG. 24 is a schematic process flow diagram of multiple edge processing. In the multiple edge processing, a search is made for other edges on both sides of the main edge. FIG. 25 is a schematic diagram for explaining multiple edge processing, and shows a planar arrangement of a partial space. In the example shown in FIG. 25A, the target partial space 280 includes an edge 282 where the point cloud is most concentrated and an edge 284 whose point cloud is less collected than the edge 282. The edge 282 as the main edge is detected by the processing by the horizontal plane searching means 46, the vertical position determining means 48, the wall surface shape determining means 50, the tracking means 52, and the wall surface element registering means 54, which have already been described. Graphic registration processing is performed. In the multiple edge processing, multiple edge search spaces 286 and 288 are set on both sides so as not to include the edge 282 (FIG. 25B). The multi-edge search space is set to a rectangular parallelepiped space whose planar shape is a rectangle having the same size and shape as the partial space and whose Z value range is the same as that of the J-th layer. For example, a multiple edge search space is created in a region on the left side of the main edge (edge 282 in FIG. 25) in the direction in which the main edge extends (S300), and a point cloud in the space is captured (S300). Similarly, the edge search is performed in the multiple edge search space, and it is determined whether or not an edge exists (S302). If there is an edge (“Yes” in S302), a space having the same planar shape as the vertical search space in the processing S110 of FIG. 14 is set around the edge in the multiple edge search space. A process of determining the shape of the wall surface element from the point group in the same manner as the process S122 is performed (S304). When the shape of the wall surface element can be acquired (in the case of “Yes” in S306), the continuous wall surface tracking processing S160 to S172 by the tracking unit 52 described in FIG. 17 and the wall surface element registration unit 54 described in FIG. The same processing S308 as the registration processing S260 to S266 is performed. Similarly, a multiple edge search space is created in the area on the right side of the main edge, and processes S310 to S318 similar to the processes S300 to S308 in the case of the left side described above are performed.

  In the above embodiment, the program that causes the computer to operate as each means of the road accessory detection system 2 is stored in the storage device 6 and the computer reads and executes the program. However, in other configurations, the program is a network. In this case, the road accessory detection system 2 includes a communication device, the communication device acquires a program from a network or the like, and provides it to the arithmetic processing device 4 The data is stored in the storage device 6. Further, the program can be provided by being stored in a recording medium such as a CD-ROM (Compact Disc Read Only Memory).

  In the above-described embodiment, road signs, guardrails, and road markings among the road accessories are detection targets. However, this is an example, and the image feature information 36 and the form feature information are also stored in the storage device 6 for other road accessories. 38 can be stored and their detection performed.

  2 Road appendage detection system, 4 arithmetic processing unit, 6 storage device, 8 input device, 10 output device, 20 image segmentation means, 22 candidate image region extraction means, 24 wall surface detection means, 26 candidate wall surface extraction means, 28 collation means , 30 road sign identifying means, 32 road surface image area extracting means, 34 road marking detecting means, 36 image feature information, 38 form feature information, 40 partial space setting means, 42 partial space selecting means, 44 block space setting means, 46 horizontal plane Internal search means, 48 vertical position determination means, 50 wall surface shape determination means, 52 tracking means, 54 wall surface element registration means, 80, 82 road sign, 84 guardrail, 86 road marking.

Claims (8)

A road accessory that detects a road accessory in the target space based on an image obtained by photographing the target space and point cloud data representing a three-dimensional shape of the surface of the feature in the target space , acquired by means independent from each other . A detection device,
Storage means for preliminarily storing a form feature of the detection target including information on at least one of an image feature of the road appendage to be detected and a shape, size, and arrangement in the target space;
Image dividing means for dividing the image into a plurality of partial image areas each having a pixel value having a predetermined similarity;
Candidate image area extraction means for extracting a candidate image area corresponding to the detection target from the partial image area based on the image feature stored in the storage means;
Wall surface detecting means for detecting the wall surface of the feature based on the point cloud data;
Candidate wall surface extracting means for extracting a candidate wall surface corresponding to the detection target among the detected wall surfaces based on the form feature stored in the storage means;
Collating means for collating the candidate image area and the projected image of the candidate wall surface viewed from the image capturing position, and detecting the detection target based on a collation result;
A road appendage detection device characterized by comprising: In the road accessory detection device according to claim 1,
The said matching means detects the guardrail which is the said detection object in the area | region except the part outside a road and the roadway part of the said road among the said object spaces, The road accessory detection apparatus characterized by the above-mentioned. In the road appendage detecting device according to claim 1 or 2,
The said matching means detects the road sign which is the said detection object in the area | region except the outside part from the road among the said object space, The road accessory detection apparatus characterized by the above-mentioned. In the road accessory detection apparatus according to claim 3,
A road accessory detection apparatus comprising: road sign identifying means for identifying the type or display content of the road sign by template matching in an image area corresponding to the road sign for the detected road sign. In the road accessory detection apparatus according to any one of claims 1 to 4,
Road marking detection means for detecting the road marking as the detection target;
Road surface image area extracting means for extracting a road surface image area corresponding to a road surface in the image based on the point cloud data;
Have
The candidate image area extracting means extracts a road marking candidate that is the candidate image area for the road marking,
The road marking detection means determines the road marking candidate in the road surface image area as the road marking;
Road appendage detection device characterized by. In the road accessory detection device according to claim 5,
The road marking detection device is characterized in that, for the road marking candidate determined to be the road marking, the content of the road marking is identified by template matching. A road accessory that detects a road accessory in the target space based on an image obtained by photographing the target space and point cloud data representing a three-dimensional shape of the surface of the feature in the target space , acquired by means independent from each other . A detection method,
An image dividing step of dividing the image into a plurality of partial image regions each having a pixel value having a predetermined similarity;
A candidate image region extracting step of extracting a candidate image region corresponding to the detection target from the partial image region based on an image feature given in advance of the road appendage as a detection target;
A wall surface detecting step for detecting a wall surface of the feature based on the point cloud data;
A candidate wall surface corresponding to the detection target is extracted from the detected wall surfaces based on a pre-given form feature including information on at least one of shape, size, and arrangement of the detection target in the target space. A candidate wall surface extraction step,
Collating the candidate image region with a projected image of the candidate wall surface viewed from the shooting position of the image, and detecting the detection target based on a matching result;
A road appendage detection method characterized by comprising: A road accessory is detected in the target space on the basis of an image obtained by photographing the target space and point cloud data representing a three-dimensional shape of the feature surface in the target space , acquired by means independent from each other on a computer. A program for processing, the computer
Image dividing means for dividing the image into a plurality of partial image areas each having a pixel value having a predetermined similarity;
Candidate image region extraction means for extracting a candidate image region corresponding to the detection target from the partial image region based on an image feature given in advance of the road accessory to be detected;
Wall surface detecting means for detecting the wall surface of the feature based on the point cloud data;
A candidate wall surface corresponding to the detection target is extracted from the detected wall surfaces based on a pre-given form feature including information on at least one of shape, size, and arrangement of the detection target in the target space. A candidate wall surface extracting unit that performs the matching, and the candidate image region and the projected image of the candidate wall surface viewed from the photographing position of the image are collated, and the collating unit detects the detection target based on the collation result. A program characterized by

Images