JP2009052907A - Foreign matter detecting system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a foreign matter detection system capable of heightening working efficiency and accuracy in foreign matter detection. <P>SOLUTION: The foreign matter detection system 100 specifies the presence or absence of foreign matter on a road surface and the position of the foreign matter on a map. The foreign matter detection system 100 is approximately constituted of an inspection data acquisition part 7 mounted to a vehicle (moving means) 15; a degradation determination part 9; and a diagnostic result managing part 16. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a foreign object detection system, and more particularly, to a foreign object detection system that detects the presence or absence of foreign objects (such as depressions and cracks) on a road surface, and the position and size of foreign objects.

Traditionally, trunk roads such as national roads and expressways have been regularly monitored and repaired by the competent authorities to maintain the road surface so as to meet certain standards. Periodic monitoring is performed at night when traffic is low, or in a state where the traffic of the vehicle is blocked, and a road surface scanner using a laser sensor, for example, is used as the monitoring means. On the other hand, prefectural roads and city roads other than national roads and expressways are difficult to implement by introducing large and expensive equipment such as road surface scanners due to traffic congestion or budget. The road surface situation was photographed with a camera or the like, and the road surface state was confirmed based on the image to determine the necessity of repair.
As a conventional technique, Patent Document 1 describes a foreign object detection sensor that outputs detection information of a foreign object on a road surface, a GPS receiver that receives a positioning signal, a wireless receiver that receives GPS correction data, and a corrected pseudo distance. Positioning computing unit that measures the position of the moving vehicle, traveling controller that controls the traveling position of the moving vehicle, and foreign object position calculation that calculates the position of the foreign object based on the traveling position of the moving vehicle and the foreign object detection information of the foreign object detection sensor Road surface maintenance management that detects and removes foreign objects on the runway, equipped with a detector, foreign object detection information output from the foreign object detection sensor, travel position information of the moving vehicle, and a wireless transmitter that transmits the position information of the foreign object A system is disclosed.
JP 2005-275723 A

However, in conventional monitoring and repair work for prefectural roads, city roads, etc., there are many factors that depend on the visual observation of the person in charge, and it has not always been possible to accurately grasp the road surface condition. In addition, since the road surface was confirmed based on the captured image, the person in charge of confirmation confirmed the criteria for determining the foreign matter, and not only the foreign matter could not be detected accurately, but also the foreign matter was detected from a large amount of image information. There is a problem that it takes a lot of time and labor to detect.
In addition, the conventional technique disclosed in Patent Document 1 scans a road surface with laser light, captures reflected light with a CCD camera, transmits measured data to a data center wirelessly, and detects foreign matter by image processing. Since the target road surface is limited to a runway or the like, there is a problem that the system configuration is fixed and the system cost is increased.
The present invention has been made in view of such problems, and captures the situation of the road to be inspected with a camera, acquires positional information of the photographing location with a GPS, associates it with the photographed image, and based on these information on the road An object of the present invention is to provide a foreign object detection system capable of detecting foreign objects and displaying them on a map and improving the efficiency of foreign objects while improving work efficiency by preparing a road based on the result. .
Another object is to reduce system cost by using a general-purpose device.

In order to solve such a problem, the present invention provides a foreign object detection system that identifies the presence or absence of a foreign object on a road surface and the position of the foreign object on a map. Imaging means to obtain, moving means for moving the imaging means along the road surface, position information acquisition means for acquiring position information of the moving means, and foreign matter from the image data relating to the road surface obtained by the imaging means Foreign matter detection means for detecting, and display means for displaying the position of the foreign matter detected by the foreign matter detection means on a map based on the position information acquired by the position information acquisition means, are provided.
The present invention provides imaging means (analog camera, network camera, etc.) for photographing the road surface to be inspected, moving means (vehicle, etc.) moving on the road while photographing, and position information (latitude) of the vehicle (imaging means) that changes every moment. , Longitude) from the GPS satellite and stored in association with the image, foreign object detection means for determining the foreign object from the road surface image and position information stored by the positional information acquisition means, and foreign object detection means Display means (GIS) for displaying the position of the foreign matter detected by the above on a map. The greatest feature of the present invention resides in a determination method for determining a normal road surface and a foreign object from an image obtained by photographing the road surface. Further, by taking a picture while traveling on the road to be inspected at a normal speed, it is possible to recognize the size of the foreign matter and the position on the map by performing image analysis from the image by batch processing. Thereby, since an operator can maintain a road based on the result, work efficiency can be improved and the accuracy of foreign object detection can be improved.

According to a second aspect of the present invention, the foreign object detection means detects a moving image as a moving object in the image data as compared with the image data of at least one frame before the image data. A first pothole determining means for determining a pothole that is included in a predetermined reference size range from the moving objects detected by the moving object detecting means, and a counter extending in the width direction provided on the screen It is characterized by comprising recording means for recording information relating to the pothole that has passed on the line, and second pothole determination means for determining the type of the foreign matter based on the shape of the pothole.
The image obtained by photographing the road surface moves relatively in the direction opposite to the moving direction of the moving means. If there is something on the road, it will be photographed as if it were moving as a moving object. The moving object detection means detects this moving object. However, since a manhole or a vehicle parked on the road is also photographed as a moving object, it is necessary to exclude moving objects other than the standard size defined in advance (first filtering). Further, there is a counter line having a predetermined length in the width direction on the screen, and information relating to the pothole that has passed through the counter line is recorded. Further, there is a second filtering for dividing the recorded information into foreign material types (holes, cracks, etc.). As a result, the foreign object detection accuracy can be improved and the size of the foreign object and the type of the foreign object can be adjusted depending on the setting of filtering.

According to a third aspect of the present invention, the moving object detecting means includes a cluster detecting means for detecting a cluster (defined) of pixels constituting image data relating to the moving object, and a contour line around the cluster detected by the cluster detecting means. A contour line generating means for generating a current contour line by comparing a previous contour line generated in a previous frame to track the contour line with a current contour line generated by the current frame; Determining means for determining whether the outline is continuous with the previous outline or a new outline, and ID providing means for assigning an ID to the outline determined by the determination means It is characterized by.
The cluster detection means first creates an image that is assumed to be the background of the road surface, and determines the color of the background. For example, if the road surface is wet with rain, the background color becomes dark, and if it is dry, it becomes whitish. Next, the background color and the current frame are compared in units of pixels (pixels), and if there is a moving object, a cluster (an outline gathering around the moving object) appears. The outline line generation means generates an outline line from the cluster and creates a closed line surrounding the cluster. Merge if multiple clusters are close together. The determination means determines whether the outline line is a continuous outline line or a newly appeared outline line by tracing the outline line. And ID for identification is given to each outline. Thereby, an outline line corresponding to the road surface condition is generated, and image processing can be easily performed.

According to a fourth aspect of the present invention, when the moving object detected by the moving object detection means is determined not to be within the reference size range by the first pothole determining means, the foreign object detection means detects the moving object as a foreign object. It is characterized by not detecting as.
A manhole, a parked vehicle, or a road shoulder may be photographed on the road surface. If these are detected as the original foreign matter, the detection accuracy of the foreign matter is lowered. Therefore, in the present invention, the size of the moving object is regulated within the reference size range in order to exclude the moving object that should not be detected as a foreign object. Thereby, the thing which is not a foreign material can be excluded from a moving body, and the precision of a moving body can be improved.

According to a fifth aspect of the present invention, the counter line has a predetermined length in the width direction of the screen, and the foreign object detection unit does not determine that the moving object outside the counter line is a foreign object.
In many cases, parked vehicles and shoulders are located around the screen. Therefore, a counter line having a predetermined length in the width direction is provided, and an image photographed outside the counter line in the width direction is not a moving object. Thereby, images, such as a vehicle and a road shoulder, can be excluded easily.

According to a sixth aspect of the present invention, the recording unit includes at least a still image of the foreign matter determined to be the pothole by the first pothole determining unit, a time when the foreign matter passed through the counter line, and position information regarding the foreign matter. It is characterized by recording.
If the moving object passes through the counter line, it is determined as a pothole by the first pothole determining means and is recorded. The recorded contents are a still image of the moving object, a time when the moving object passes the counter line, and position information at that time. This makes it easy to display on a GIS map.

According to a seventh aspect of the present invention, when the foreign matter detection mark on the map displayed on the display screen is clicked, the display means displays a time when the foreign matter is detected and an image of the foreign matter. To do.
After the photographing is finished and the foreign object detection is finished, these data are recorded in the recording device. Thereafter, when the worker repairs the road, the map is displayed by displaying the GIS screen on the PC based on the data, and the location and position information can be acquired. That is, a mark is displayed on the map where a foreign object exists on the road. Also, by clicking on the mark, the image of the foreign object and the time are displayed. As a result, the location of the foreign matter and the situation of the foreign matter can be grasped in advance, so that the preparation for the work can be completed.

According to an eighth aspect of the present invention, the second pothole determining means holds in advance shape information relating to the contour line created from the actual foreign matter shape, and determines the type of the foreign matter by pattern matching with the shape information relating to the contour line. It is characterized by that.
The data recorded in the recording device includes all of the objects determined to be foreign objects, but the type of the foreign object (hole, crack, etc.) cannot be grasped. Therefore, in order to determine the type of foreign matter from these data, various outline shapes are recorded in advance, and the type of foreign matter is determined by performing pattern matching with the outline shape. Thereby, since the kind of foreign material can be recognized beforehand, the tool for repair can be prepared beforehand.

A ninth aspect of the present invention is characterized in that the imaging means is configured to take an image in front of the moving direction of the moving means.
The foreign object detection of the present invention does not compare with a normal screen prepared in advance, but detects whether or not a moving object is present by comparison with the immediately preceding background color. Therefore, it is possible to continuously detect a change from the background color by photographing an image ahead of the moving direction.

According to the present invention, by taking a picture while traveling on the road to be inspected at a normal speed, it is possible to recognize the size of the foreign object and the position on the map by performing image analysis from the image by batch processing. The worker can maintain the road based on the result, and the work efficiency can be improved and the foreign object detection accuracy can be improved.
In addition, the presence of the second filtering makes it possible to improve the accuracy of foreign object detection and to adjust the size of the foreign object and the type of the foreign object depending on the setting of filtering.
In addition, the moving object detection unit can easily perform image processing because the contour line corresponding to the road surface condition is generated.
In addition, in order to exclude moving objects that should not be detected as foreign objects, by restricting the size of moving objects to be within the standard size range, excluding non-foreign objects in moving objects, the accuracy of moving objects can be improved. Can be increased.
In addition, by not using an image taken outside the counter line as a moving object, images such as vehicles and road shoulders can be easily excluded.

Further, since the still image of the moving object, the time when the moving object passes the counter line, and the position information at that time are recorded, it can be easily displayed on the GIS map.
Also, by clicking the mark on the display screen, the image and time of the foreign object are displayed, so the location of the foreign object and the state of the foreign object can be grasped in advance, and the work preparation can be completed. .
In addition, in order to determine the type of foreign matter, various outline shapes are recorded in advance, and the type of foreign matter is determined by performing pattern matching with the outline shape, so the type of foreign matter can be recognized in advance and repairs can be performed. Can be prepared in advance.
Further, by taking a moving forward image, it is possible to continuously detect a change from the background color.

Hereinafter, the present invention will be described in detail with reference to embodiments shown in the drawings. However, the components, types, combinations, shapes, relative arrangements, and the like described in this embodiment are merely illustrative examples and not intended to limit the scope of the present invention only unless otherwise specified. .
FIG. 1 is a block diagram showing a configuration of a foreign object detection system 100 according to an embodiment of the present invention. This foreign matter detection system 100 is a foreign matter detection system that specifies the presence or absence of foreign matter on the road surface and the position of this foreign matter on a map, and is broadly divided into inspection data acquisition units mounted on a vehicle (moving means) 15. 7, a deterioration determination unit 9, and a diagnosis result management unit 16. In addition, the range mounted in the vehicle 15 is not limited to this embodiment, You may mount all. Alternatively, the data acquired by the inspection data acquisition unit 7 may be transmitted wirelessly and recorded on the center side.
The inspection data acquisition unit 7 includes a camera (imaging unit) 1 that images a road surface, an image acquisition unit 4 that processes an image captured by the camera 1, and a GPS data acquisition unit (position information) that acquires position information of the vehicle 15. Acquisition means) 5, the synchronization control unit 6 that controls the synchronization of the data of the image acquisition unit 4 and the GPS data acquisition unit 6, and synchronization is established by the synchronization control unit 6, and the GPS data 8a and the shooting data 8b are stored in association with each other. HDD 8 to be configured. The GPS data acquisition unit 5 receives the GPS satellite 2 data from the antenna 3 and calculates.

In addition, the deterioration determination unit 9 includes an image reproduction unit 10 that reproduces GPS data 8a and shooting data 8b stored in the HDD 8, a foreign object detection unit (foreign object detection unit) 11 that detects foreign objects from the shooting data 8b, and foreign object detection. And an information recording unit 12 that records the foreign object image detected by the unit 11 and the GPS data 8a in association with each other.
The diagnosis result management unit 16 takes out information recorded in the information recording unit 12 as deterioration information data 13 and displays the position of the foreign matter on the map based on the position information acquired by the GPS data acquisition unit 5. (Display means) 14 is provided.
That is, the foreign object detection system 100 of the present embodiment includes a camera (analog camera, network camera, etc.) 1 that captures the road surface to be inspected, a vehicle 15 that moves on the road while capturing, and a vehicle 15 (camera 1) that changes every moment. The position information (latitude and longitude) of the vehicle is acquired from the GPS satellite 2 and stored in association with the image, and the foreign object detection unit 11 determines the foreign object from the road surface image and the position information stored by the inspection data acquisition unit 7. And a display unit 14 (GIS) that displays the position of the foreign matter detected by the foreign matter detection unit 11 on a map. The greatest feature of the present invention resides in a determination method for determining a normal road surface and a foreign object from an image obtained by photographing the road surface. Further, by taking a picture while traveling on the road to be inspected at a normal speed, it is possible to recognize the size of the foreign matter and the position on the map by performing image analysis from the image by batch processing. Thereby, since the worker can repair and maintain the road based on the result, the work efficiency can be improved and the foreign object detection accuracy can be improved.

FIG. 2A is a diagram illustrating an example displayed on the display unit 14 by, for example, a geographic information system (GIS). FIG. 2B is a diagram illustrating a state in which a foreign object is detected by the foreign object detection unit 11 while the image data 8 b is reproduced by the image reproduction unit 10. As shown in FIG. 2A, the display unit 14 displays a map 20 indicating the location where the foreign object is detected, and marks 21a and 21b are displayed at locations where the foreign object exists on the road. For example, when the mark 21a is clicked, an image 22 of the foreign object at that location is displayed (in the figure, only the image of the foreign object is displayed, but the time when the foreign object is detected can also be displayed).
After the photographing is finished and the foreign object detection is finished, these data are recorded in the information recording unit 12. Thereafter, when the worker repairs the road, the map 20 is displayed by bringing the data and displaying the GIS screen on the PC, and the location and position information can be acquired. That is, the marks 21a and 21b are displayed on the map at locations where foreign objects exist on the road. In addition, by clicking the marks 21a and 21b, a foreign object image 22 is displayed. Thereby, since the location of the foreign material and the situation of the foreign material can be grasped in advance, work preparation can be made thoroughly.

Further, as shown in FIG. 2B, a foreign object (in this case, a depressed hole) 25 existing on the road surface 28 is detected on the screen 23. When the relatively moving foreign object 25 passes through the counter line 24 provided on the screen, it is recorded as a still image in the information recording unit 12 together with the position information. On the road shoulder, the parked vehicle 26 and the road shoulder 27 are also detected as foreign objects, but are excluded if they are outside the range of the counter line 24 (details will be described later).
That is, the parked vehicle 26 and the road shoulder 27 are often located around the screen 23. Thus, by providing a counter line 24 having a predetermined length in the width direction and not taking an image taken outside the counter line 24 as a moving object, images such as the vehicle 26 and the road shoulder 27 can be easily excluded. it can.

FIG. 3 is a flowchart for explaining the operation of the foreign matter detection means in the foreign matter detection system of the present invention. First, the image data recorded in the HDD 8 is reproduced and input by the image reproducing unit 10 (S1). Next, in order to improve detection accuracy, various preprocessing is performed on the captured video (S2). For example, a median filter, a Gaussian filter, or a low-pass filter is used for noise removal. Also, shadow erasure processing is performed. Next, in order to detect a moving object, a cluster of pixels constituting the moving object is detected, the cluster is surrounded by a contour line, and the contour line is traced (details will be described with reference to FIG. 4) ( S3). Next, as pothole determination 1 (first pothole determination means), the size of the moving object obtained in step S3 is determined (S4), and those other than the standard size defined in advance are excluded (S5). In order to determine the size of the pothole, for example, those larger than a predetermined size or smaller than a predetermined size are excluded. Next, it is determined whether or not the determined pothole has passed through the counter line 24 (S6). If it has not passed (NO in S6), the process returns to step S1 and repeats to pass through the counter line 24 (S6). YES) The moving object information is recorded in the information recording unit (recording means) 12 (S7). At that time, the size, position, time, instantaneous image, and outline of the moving object are recorded. Next, as pothole determination 2 (second pothole determination means), the type of foreign matter (hole, crack, etc.) is determined by graphic determination of the outline (S8). That is, as a method for identifying the type of foreign matter, for example, an outline shape created from an actual foreign matter shape is held in the system, and a matching type with the shape is set as the foreign matter type. The above operation is repeated until the determination is completed (S9).
That is, the foreign object detection means of the present invention includes an animal body detection means for detecting an image moving relatively in the image data as an animal body as compared with image data at least one frame before the image data, and the animal First pothole determination means for determining a pothole that is included in a predetermined reference size range from the moving object detected by the body detection means, and a counter line extending in the width direction provided on the screen Recording means for recording information relating to the pothole that has passed through, and second pothole determination means for determining the type of foreign matter based on the shape of the pothole.

  The image obtained by photographing the road surface moves relatively in the direction opposite to the moving direction of the vehicle 15. If there is something on the road, it will be photographed as if it were moving as a moving object. The moving object detection means detects this moving object. However, since a manhole or a vehicle parked on the road is also photographed as a moving object, it is necessary to exclude moving objects other than the standard size defined in advance (first filtering). On the screen, there is a counter line 24 having a predetermined length in the width direction, and information regarding the pothole that has passed through the counter line 24 is recorded. Further, there is a second filtering for dividing the recorded information into foreign material types (holes, cracks, etc.). As a result, the foreign object detection accuracy can be improved and the size of the foreign object and the type of the foreign object can be adjusted depending on the setting of filtering.

FIG. 4 is a flowchart for explaining in detail the operation of the moving object detection means. First, in order to detect a cluster of pixels to which a moving object belongs (reference numeral 30), an image assumed to be a background is created from an image several tens of frames before (S21). Next, a histogram of the color change (gray scale) of the frame to be evaluated is created for each pixel (S22). Next, the color assumed to be the background is determined from the histogram (S23). Next, the background created in step S21 and the current frame are compared in pixel units (S24). If the object is an animal, a cluster (contour around the animal body, foreground pixels) appears (YES in S25). ) Detect as an animal body (S26). Next, an outline is generated from the cluster (S27) (reference numeral 31). That is, a closed line surrounding the moving object is created, and when several clusters are close to each other, they are integrated into one closed line. Next, in order to perform tracking of the contour line (reference numeral 32), the contour line newly generated from the contour line generated in the previous frame and the contour line generated in the current frame is newly generated. It is determined whether it is a line (S28). And ID is provided to each outline (S29).
That is, the moving object detecting means of the present invention includes a cluster detecting means for detecting a cluster of pixels constituting image data relating to the moving object, and an outer line generating means for generating an outer line around the cluster detected by the cluster detecting means. And the previous outline generated in the previous frame to track the outline and the current outline generated by the current frame, and the current outline is continuous with the previous outline A determination unit that determines whether the outline is a new outline or a new outline is provided, and an ID provision unit that assigns an ID to the outline determined by the determination unit.

  The cluster detection means of the present invention first creates an image assumed to be the background of the road surface 28 and determines the background color. For example, if the road surface 28 is wet with rain, the background color becomes dark, and if the road surface 28 is dry, it becomes whitish. Next, the background color and the current frame are compared in units of pixels, and if there is a moving object, a cluster appears. Also, a contour line is generated from the cluster, and a closed line surrounding the cluster is created. Merge if multiple clusters are close together. Further, by tracing the outline, it is determined whether the outline is a continuous outline or a newly appeared outline. And ID for identification is given to each outline. Thereby, an outline line corresponding to the road surface condition is generated, and image processing can be easily performed.

It is a block diagram which shows the structure of the foreign material detection system 100 which concerns on one Embodiment of this invention. (A) is a figure which shows an example displayed on the display part, for example by geographic information system (GIS), (b) is a figure which shows a mode that the foreign material detection part detected the foreign material detection part, reproducing the imaging | photography data 8b with an image reproduction part. It is. It is a flowchart explaining operation | movement of the foreign material detection means in the foreign material detection system of this invention. (A) (b) is a flowchart for demonstrating the operation | movement of a moving body detection means in detail.

Explanation of symbols

  1 camera, 2 GPS satellites, 3 antennas, 4 image acquisition unit, 5 GPS data acquisition unit, 6 synchronization control unit, 7 inspection data acquisition unit, 8 HDD, 9 deterioration determination unit, 10 image reproduction unit, 11 foreign matter detection unit, DESCRIPTION OF SYMBOLS 12 Information recording part, 13 Deterioration information data, 14 Display part, 15 Vehicle, 16 Diagnosis result management part, 100 Foreign object detection system

Claims (9)

A foreign matter detection system for identifying the presence or absence of foreign matter on a road surface and the position of the foreign matter on a map,
Obtained by an imaging unit that captures the road surface and obtains image data, a moving unit that moves the imaging unit along the road surface, a position information acquisition unit that acquires position information of the moving unit, and the imaging unit Foreign matter detection means for detecting foreign matter from the image data relating to the road surface; and display means for displaying the position of the foreign matter detected by the foreign matter detection means on a map based on the positional information acquired by the positional information acquisition means; A foreign object detection system comprising:   The foreign object detection means includes a moving object detection means for detecting, as an moving object, an image that is moving relatively in the image data as compared with image data at least one frame before the image data, and the moving object detection means. The first pothole determining means for determining a pothole that is included in the reference size range defined in advance from the moving object detected by the above and a counter line extending in the width direction provided on the screen. 2. The recording apparatus according to claim 1, further comprising: a recording unit that records information about the pothole; and a second pothole determination unit that determines a type of the foreign matter based on a shape of the pothole. Foreign object detection system.   The moving object detecting means includes a cluster detecting means for detecting a cluster of pixels constituting image data relating to the moving object, an outer line generating means for generating an outer line around the cluster detected by the cluster detecting means, Comparing the previous outline generated in the previous frame to track the outline with the current outline generated by the current frame, and the current outline is continuous with the previous outline 2. A determination unit that determines whether the line is a new outline or an ID line and an ID provision unit that assigns an ID to the outline determined by the determination unit. Or the foreign material detection system of 2.   If the moving object detected by the moving object detecting means is determined not to be within the reference size range by the first pothole determining means, the foreign object detecting means does not detect the moving object as a foreign object. The foreign matter detection system according to claim 1 or 2, characterized in that   5. The counter line according to claim 1, wherein the counter line has a predetermined length in the width direction of the screen, and the foreign matter detection means does not detect the moving object outside the counter line as a foreign matter. The foreign matter detection system according to claim 1.   The recording means records at least a still image of the foreign matter determined to be the pothole by the first pothole determining means, a time when the foreign matter passed through the counter line, and position information regarding the foreign matter. The foreign object detection system according to any one of claims 1 to 5.   2. The display unit according to claim 1, wherein when the foreign object detection mark on the map displayed on the display screen is clicked, the time when the foreign object is detected and an image of the foreign object are displayed. The foreign matter detection system described.   The second pothole determining means holds in advance shape information relating to the outline drawn from the actual foreign substance shape, and determines the type of the foreign substance by pattern matching with the shape information relating to the outline. The foreign matter detection system according to claim 2.   The foreign object detection system according to any one of claims 1 to 8, wherein the imaging unit is configured to capture an image in front of a moving direction of the moving unit.

Images