JP2023112536A - Method for calculating road surface damage position - Google Patents

Abstract

To provide a method for calculating a road surface damage position capable of inexpensively measuring damage on a road surface of a road with excellent precision.SOLUTION: A method for calculating a road surface damage position includes: a step S1 of selecting a road sign; a step S2 of photographing the road sign by an imaging device of a vehicle; a step S3 of separating an image ruler from a photographed image so as to store the ruler in a ruler layer; a step S4 of photographing a road surface by the imaging device while the vehicle travels on a road so as to obtain a moving image and also recording a vehicle speed corresponding to a frame of the moving image; a step S5 of identifying the plurality of frames where damage of the road surface exists; a step S6 of overlapping the extracted frames with the image ruler; a step S7 of extracting a damage frame where the damage closest to the image ruler exists; a step S8 of extracting the plurality of frames where a position marker exists; a step S9 of overlapping the extracted frames with the image ruler; a step S10 of extracting a marker frame where the position marker closest to the image ruler exists; and a step S11 of calculating a distance between a photographing position of the damage frame and a photographing position of the marker frame.SELECTED DRAWING: Figure 1

Description

TECHNICAL FIELD The present invention relates to a method for calculating the position of road damage when inspecting a road based on images taken from a vehicle.

On expressways, as part of road maintenance and management, road inspections are carried out in which roads are inspected while an inspection vehicle is traveling. In road inspections, an inspector riding in an inspection vehicle checks whether there is damage to the road surface based on visual observation and vibration of the vehicle. Road surface damage includes potholes and bidirectional cracks that precede potholes, and early detection of such damage is important to ensure the safety of vehicles traveling on roads. is.

When the inspector discovers damage to the road surface during the road inspection, the inspector identifies the size and position of the damage, and notifies the repairman in charge of the repair of the damage information regarding the identified size and position of the damage. Based on the damage information, the repairman goes to the location of the damage with equipment and materials necessary for repair and repairs the damage. Accurate damage information must be collected during road inspections for prompt execution of repairs.

Conventionally, as a road inspection auxiliary system used for road inspection, a camera that photographs the front of the vehicle in the traveling direction, a GPS (Global Positioning System) receiver, a push button operated by an inspector, the camera and GPS A proposal has been made that includes a personal computer connected to a receiver and push buttons (see Patent Document 1). In a road inspection performed using this road inspection assistance system, when an inspector confirms damage to the road surface, the inspector presses a push button to generate index information. This index information is associated with the image information captured by the camera when the push button is pressed, and is associated with the position information acquired by the GPS receiver, and these information are stored in the storage device of the personal computer. . After the road inspection is completed, the information in the storage device is analyzed, and the size and position of the damage are identified based on the image information and position information associated with the index information.

Utility Model Registration No. 3195716

However, it is difficult for a GPS receiver to receive GPS satellite signals in tunnels, under elevated tracks, between high-rise buildings, etc., and there is a risk that position information cannot be obtained. For this reason, there is a possibility that the position of the damage on the road surface cannot be identified.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a method for reliably identifying the location of road damage during on-road inspection.

In order to solve the above-mentioned problems, the road surface damage position calculation method of the present invention is based on reference photographing in which a reference linear body installed on the road and extending in the width direction of the road is photographed by an imaging device installed in the vehicle. process and
an image ruler delivery step of separating an image ruler indicating the reference linear body from the image taken in the reference photography step and storing the image ruler in a ruler layer;
A moving image acquisition step of capturing a moving image by capturing a road surface with the imaging device while traveling on the road to be inspected, and recording the speed of the vehicle corresponding to each frame of the moving image;
a damage candidate frame identification step of identifying a frame containing damage to the road surface from the moving image captured in the moving image acquisition step;
a damaged ruler superimposing step of superimposing an image ruler of the ruler layer on the frame identified in the damaged candidate frame identifying step;
a damaged frame identifying step of identifying a damaged frame, which is a frame in which the image ruler overlaps or is closest to the damage, among the frames on which the image ruler is superimposed in the damage ruler superimposing step;
a sign candidate frame specifying step of specifying a frame containing a position sign specifying a position on the road from the moving image captured in the moving image acquiring step;
a sign ruler superimposing step of superimposing an image ruler of the ruler layer on the frame identified in the sign candidate frame identification step;
a marker frame identifying step of identifying a marker frame, which is a frame in which the image ruler overlaps or is closest to the position marker, among the frames on which the image ruler is superimposed in the marker ruler superimposing step;
The distance between the photographing position of the damaged frame identified in the damaged frame identifying step and the photographing position of the sign frame identified in the sign frame identifying step is the speed and frame of the vehicle recorded in the moving image obtaining step. and a distance calculation step of calculating based on the rate.

According to the above configuration, in the reference photographing step, the imaging device installed in the vehicle photographs the reference linear body on the road. The reference linear body is installed on the road and extends in the width direction of the road, and road markings such as stop lines can be used. Also, a ruler or the like installed on the road may be used for photographing. In the image standard storing step, the image ruler indicating the reference linear body is separated from the image photographed in the reference photographing step and stored in the ruler layer. In the moving image acquisition step, while traveling on the road to be inspected, the road surface is photographed by the imaging device to obtain a moving image, and the speed of the vehicle corresponding to each frame of the moving image is recorded. The speed of the vehicle is preferably measured by a speedometer mounted on the vehicle. In the damage candidate frame identification step, frames including damage to the road surface are identified from the moving image captured in the moving image acquisition step. In the damaged ruler superimposing step, the image ruler of the ruler layer is superimposed on the frame identified in the damaged candidate frame identifying step. In the damaged frame identifying step, among the frames on which the image ruler is superimposed in the damaged ruler superimposing step, a damaged frame that is a frame in which the image ruler overlaps or is closest to the damage is identified. In the marker candidate frame identifying step, a frame including a position marker identifying a position on the road is identified from the moving image captured in the moving image acquiring step. In the sign ruler superimposition step, the image ruler of the ruler layer is superimposed on the frame identified in the sign candidate frame identification step. In the marker frame specifying step, a marker frame, which is a frame in which the image ruler overlaps or is closest to the position marker, is specified among the frames on which the image ruler is superimposed in the marker ruler superimposing step. In the distance calculating step, the distance between the photographing position of the damaged frame identified in the damaged frame identifying step and the photographing position of the marked frame identified in the marked frame identifying step is recorded in the moving image obtaining step. Calculated based on vehicle speed and frame rate. In this way, the position of the road damage can be specified by the distance from the position marker based on the information captured by the imaging device installed in the vehicle and the speed of the vehicle. Therefore, unlike conventional road inspection assistance systems that specify damage based on positional information acquired by a GPS receiver, the system is not affected by the reception conditions of the GPS receiver. As a result, the location of the damage on the road surface can be stably and accurately specified, the location of the damage can be accurately communicated to the road repairman, and the damage on the road surface can be quickly repaired. In addition, by adopting kiloposts, which are commonly used for identifying locations on roads in road management, as position markers, repair workers can quickly and accurately identify the locations of damage. Damage can be repaired quickly. In addition, since the image acquired in the reference photographing step and the moving image acquired in the moving image acquiring step are both captured by an imaging device installed in the vehicle, the damage ruler superimposing step and the marked ruler By superimposing the image ruler on the moving image in the superimposing process, the relationship between the road surface damage and the position marker can be accurately grasped by eliminating the effects of lens distortion and the like.

In one embodiment of the road surface damage position calculation method, the marker candidate frame identifying step identifies a frame captured at a position closest to the damaged frame identified in the damaged frame identifying step and containing the position marker.

According to the above-described embodiment, in the marker candidate frame identification step, frames containing markers can be relatively easily identified by checking moving images while advancing or rewinding from damaged frames.

In one embodiment of the road surface damage position calculation method, the marker candidate frame identification step identifies a frame including the position marker, separated by the fewest number of frames from the damaged frame identified in the damaged frame identification step.

According to the above-described embodiment, in the marker candidate frame identification step, frames containing markers can be relatively easily identified by checking moving images while advancing or rewinding from damaged frames.

In one embodiment of the road surface damage position calculation method, the reference photographing step photographs a road marking drawn on the road as the reference linear object.

According to the above embodiment, an image ruler can be obtained with little effort by photographing a road marking drawn on a road as a reference linear body. Here, as the road marking, figures drawn on the road surface such as a stop line are widely applicable.

In one embodiment of the road surface damage position calculation method, the reference photographing step photographs a ruler installed on the road as the reference linear body.

According to the above embodiment, an image ruler can be quickly obtained by setting a prepared ruler on the road and photographing the ruler as a reference linear body.

In one embodiment of the road surface damage position calculation method, the ruler includes a plurality of parallel lines provided at equal intervals in plan view.

According to the above-described embodiment, a ruler including a plurality of parallel lines at regular intervals is placed on the road, and this ruler is photographed as a reference linear body in the reference photographing step, thereby photographing the reference linear body. With the image ruler obtained by the method, the distance in the moving direction of the vehicle on the road surface can be easily and accurately measured.

In one embodiment of the road surface damage position calculation method, the damage candidate frame identifying step identifies a frame including the road surface damage by image recognition.

According to the above-described embodiment, by performing image recognition on moving images, it is possible to easily identify frames that include damage to the road surface.

In one embodiment of the road surface damage position calculation method, the marker candidate frame identifying step identifies a frame including the position marker by image recognition.

According to the above embodiment, it is possible to easily identify a frame containing a position marker by performing image recognition on a moving image.

In one embodiment of the road surface damage position calculation method, the moving image acquisition step associates damage index information with frames of the moving image according to input by a staff member who has perceived damage to the road surface,
In the damage candidate frame identification step, a frame surrounding the frame associated with the damage index information of the moving image is searched to identify a frame including the damage to the road surface.

According to the above-described embodiment, in the moving image acquisition process, the staff perceives the damage to the road surface and inputs it, and the damage index information is associated with the frame of the moving image according to this input. After that, in the damage candidate frame identification step, by searching for the frames surrounding the frame associated with the damage index information, the frames containing road surface damage can be easily and quickly identified. can be specified.

In one embodiment of the road surface damage position calculation method, in the moving image acquisition step, the marker index information is associated with the frame of the moving image according to the input by the staff who perceives the position marker,
In the marker candidate frame identifying step, frames surrounding the frame associated with the marker index information of the moving image are searched to identify a frame containing the position marker.

According to the above-described embodiment, in the moving image acquisition process, the staff perceives the position marker and inputs it, and the marker index information is associated with the frame of the moving image according to this input. After that, in the marker candidate frame identification step, frames surrounding the frame associated with the marker index information are searched for in the moving image, thereby easily and quickly identifying the frame containing the position marker. can do.

The method for calculating the road surface damage position of one embodiment associates the traffic sign index information with the frames surrounding the frame associated with the damage index information among the moving image and the speed information obtained in the moving image obtaining step. a transmitting step of transmitting frame and rate information between adjacent frames of the received frame;
a receiving step of receiving the frame and speed information transmitted in the transmitting step;
Based on the frames and speed information received in the receiving step, the damaged candidate frame identifying step, the damaged ruler superimposing step, the damaged frame identifying step, the marker candidate frame identifying step, the marker ruler superimposing step, and the marker frame identifying step. and the above distance calculation step.

According to the above embodiment, in the transmission step, among the moving image and the speed information acquired in the moving image acquisition step, the frames surrounding the frame associated with the damage index information and the frame associated with the sign index information are transmitted. Frames and velocity information between surrounding frames are transmitted. The frames and speed information transmitted in this transmission step are received in the reception step. Preferably, the transmitting and receiving steps are performed by wireless communication. Based on the frames and speed information received in the receiving step, the damaged candidate frame identifying step, the damaged ruler superimposing step, the damaged frame identifying step, the marker candidate frame identifying step, the marked ruler superimposing step, the marked frame identifying step, and the distance calculating step. By doing so, it is possible to specify the damaged position of the road surface at the receiving position distant from the transmitting position.

In one embodiment of the road surface damage position calculation method, the imaging device is a drive recorder.

According to the above-described embodiment, by using the drive recorder, it is possible to accurately identify the damaged position of the road surface with inexpensive equipment. Therefore, it is possible to identify damage to the road surface with good accuracy through daily inspection of the road, and to quickly repair the damage.

FIG. 3 is a flow diagram showing a method for calculating road surface damage positions according to an embodiment of the present invention; FIG. 4 is a schematic diagram showing how an imaging device mounted on a vehicle captures an image of a reference linear body; It is a block diagram which shows an imaging device. FIG. 4 is a schematic diagram showing an image of a reference linear body; FIG. 4 is a schematic diagram showing a state in which an image ruler is superimposed on a road surface damage position in a moving image; FIG. 4 is a schematic diagram showing a state in which an image ruler is superimposed on a position marker in a moving image; FIG. 4 is a schematic diagram showing the relationship between the speed at which frames of a moving image are acquired and the shooting positions of the frames; FIG. 4 is a schematic diagram showing the relationship between position markers and image rulers in a plurality of frames of a moving image; 4 is a plan view showing a ruler as a reference linear body; FIG. FIG. 11 is a plan view showing a ruler of another embodiment;

The present invention will be described in detail below with reference to the illustrated embodiments.

FIG. 1 is a flow diagram showing a road surface damage position calculation method according to an embodiment of the present invention. The road surface damage position calculation method of the present embodiment aims to detect damage such as cracks, potholes, delamination, and rutting on the pavement surface of the road and to identify the positions of these damages. In the method of calculating the road surface damage position of the present embodiment, a vehicle is equipped with an imaging device, the vehicle travels on the road to be inspected, and the road surface is photographed with the imaging device. This is to detect the damage of the hull and to calculate the position of the damage.

In the road surface damage position calculation method of the present embodiment, first, a road marking to be used as a reference linear body is selected (step S1). Road markings are elements that extend in the direction perpendicular to the direction of travel when a vehicle equipped with an imaging device is traveling straight, that is, in the width direction of the vehicle, such as a stop line drawn on the road surface. Select the one that contains Moreover, roads on which road markings are installed include a wide range of places where the reference linear body can be photographed, such as parking lots and workshops. Moreover, the reference linear object is not limited to road markings, and a ruler installed on the road surface may be used. It may be a ruler installation step in which it is installed perpendicular to the running direction of the.

Next, a reference photographing step is performed. That is, the road marking as the reference linear body selected in step S1 is photographed by an imaging device installed in the vehicle while positioned in front of the vehicle in the running direction (step S2). The imaging device generates and outputs a visible light image input through a lens, and is preferably capable of outputting both a still image and a moving image. Further, when a ruler is placed on the road surface as the reference linear body in step S1, the image is taken with the ruler positioned in front of the traveling direction of the vehicle.

Next, an image ruler formed of pixels representing road markings is separated from the image obtained in the reference photographing step, and stored in the ruler layer (step S3). The image specification delivery process is executed by an image processing unit provided in the imaging device. Also, an information processing device such as a personal computer or a smartphone is connected to the imaging device, an image including the reference linear body is transferred from the imaging device to the information processing device, and an image processing unit provided in the information processing device executes the image. may Here, in the image stored in the ruler layer, pixels other than pixels indicating road markings may be set to be non-display or may be deleted. Further, the photographed image of the road marking has a single layer, and this layer may be used as a ruler layer to store only the image ruler. Also, when the ruler installed on the surface of the road is photographed in step S2, the image ruler formed by the pixels indicating the ruler portion is separated from the photographed image.

Subsequently, while the vehicle is traveling on the road to be inspected, the imaging device captures the scenery in front including the road surface to obtain a moving image, and the speed of the vehicle corresponding to each frame constituting the moving image is recorded. A moving image acquisition step is performed (step S4). When this moving image acquisition step is performed, the position and angle of the imaging device used when the reference linear object was imaged in the reference imaging step of step S2 are left unchanged. The speed of the vehicle corresponding to each frame of the moving image is specified based on the signal output from the speed measuring device installed in the vehicle. For example, a vehicle speed pulse output from a vehicle transmission or a speedometer is input to an imaging device, the speed at which a video frame is generated is specified from the vehicle speed pulse, and speed information indicating the specified speed is stored in the frame. Store in the storage device in association with each other.

Either the reference photographing step of step S2 or the moving image acquisition step of step S4 may be performed first. Further, when the reference linear object is photographed in the reference photographing step and when the moving image is acquired in the moving image acquisition, the loading capacity of the vehicle must be the same and the photographing conditions of the photographing device must be the same. In order to make the load capacity of the vehicle the same, it is necessary to have the same person on board in the reference photographing process and the moving image acquisition process, and to use the same load in the vehicle.

Further, the step S3 of providing the image specification standard may be performed after the step S4 of acquiring the moving image.

After the moving image acquisition step of step S4, a damage candidate frame identification step of identifying a plurality of frames showing damage to the road surface from the moving image obtained in the moving image acquisition step is performed (step S5). A frame means a still image that constitutes a moving image. The damage candidate frame identifying step can be performed by reproducing a moving image on a display device of the imaging device and visually recognizing the reproduced moving image by the operator. Alternatively, an information processing device such as a personal computer or a smartphone is connected to the imaging device, a video is transferred from the imaging device to the information processing device, and the video is played back on a display device of the information processing device to display damaged frames. may be specified. Frames containing damage in a moving image may be extracted from the moving image and identified by image recognition techniques.

Subsequently, a damage ruler superimposing step is performed to create a plurality of damage composite frames by superimposing the image ruler of the ruler layer on the plurality of frames including the damage identified in step S5 (step S6). This damage composite frame can be formed by superimposing a moving image and an image ruler by known image processing techniques.

Then, among the plurality of damaged composite frames formed in the damaged ruler superimposing step, a damaged frame that is a frame in which the image ruler overlaps with the damage or a frame in which the image ruler is closest to the damage is performed (step S7).

Further, a marker candidate frame specifying step of specifying a frame containing a position marker specifying a position on the road from the moving image captured in the moving image acquisition step of step S4 is performed (step S8). Here, the position marker means a marker or the like, such as a kilometer post, which is installed around the road and whose installation position is known. Position signs include not only kiloposts, but also pier numbers of bridges supporting roads, various road signs, and the like. As for the frame containing the position marker, it is preferable to select a frame showing the position marker installed at the closest position from the damage transferred to the frame identified in the damaged frame identification step of step S7. Also, from the frames including damage identified in the damaged frame identifying step of step S7, a frame in which the position marker is shown and separated by the smallest number of frames may be selected. Also, the frame containing the position marker is extracted from the frame captured before the frame containing the damage identified in the damaged frame identifying step of step S7. That is, the frame in which the position marker located behind the damaged frame identified in step S7 in the traveling direction of the vehicle is identified. Here, the frame including the position marker may be extracted from the frame after the frame including the damage identified in step S7 above, that is, the frame in front of the vehicle beyond the damage in the frame identified in step S7. A frame in which the location marker is located may be identified. This marker candidate frame identifying step can be performed by reproducing a moving image on the display device of the imaging device and visually recognizing the reproduced moving image by the operator. In addition, an information processing device such as a personal computer or a smartphone is connected to the imaging device, a moving image is transferred from the imaging device to the information processing device, and the moving image is reproduced on a display device of the information processing device to display frames including position markers. may be specified. Also, the frame containing the position marker in the moving image may be extracted from the moving image and identified by image recognition.

Subsequently, a marker ruler superimposing step is performed to create a plurality of marker composite frames by superimposing the image ruler of the ruler layer on the plurality of frames including the position markers identified in step S8 (step S9). This sign composite frame can be formed by superimposing a moving image and an image ruler by known image processing techniques.

After that, among the plurality of marker composite frames formed in the marker ruler superimposing step, a frame in which the image ruler overlaps the position marker, or a frame in which the image ruler is closest to the position marker. A marker frame specifying step for specifying a marker frame is performed (step S10).

Then, a distance calculation step of calculating the distance between the photographing position of the damaged frame identified in the damaged frame identifying step and the photographing position of the marked frame identified in the marked frame identifying step is performed (step S11). In this distance calculation step, the speed stored in association with the moving image frame in the moving image acquisition step is used. That is, for all frames from the labeled frame to the damaged frame, the velocity associated with each frame is read, and each velocity value is multiplied by the imaging time interval of the frames. The frame shooting time interval is 1/r(s) when the frame rate is r(fps). The total distance from the marked frame to the damaged frame is obtained for the distance traveled by the vehicle between each frame thus obtained, and the distance from the marked frame to the damaged frame is calculated. Thus, from the position marker, the distance to the road damage is calculated. By combining the content of the location marker and the distance from the location marker to the damage, the location of the damage can be represented with high accuracy. For example, by notifying the repairer of the position of the damage specified by the road surface damage position calculation method of the present embodiment by the position of the position marker and the distance from this position marker to the damage, the repairer can quickly and accurately It is possible to grasp the position of damage on the road surface. Therefore, it is possible to have the repairman quickly reach the damaged position and repair the damage quickly.

As described above, according to the road surface damage position calculation method of the present embodiment, the same imaging device installed in the same vehicle is used to perform the reference photographing process and the moving image acquisition process. Therefore, when the image ruler formed from the image obtained in the reference photographing step is superimposed on the moving image frame obtained in the moving image acquisition step, there is no need to correct or convert the image. For example, there is no need to correct distortion caused by the lens, or to correct perspective distortion according to the shooting angle. Therefore, the damaged ruler superimposing process and the marked ruler superimposing process can be easily performed. Further, with respect to the damaged frame identified in the damaged frame identifying step and the marked frame identified in the marked frame identifying step, no error due to image correction or conversion occurs in the position of the image ruler with respect to the frame. Therefore, the distance between the damage in the damage frame and the position marker in the marker frame can be determined with little error. Further, in the road surface damage position calculation method of the present embodiment, the reference photographing step and the moving image acquisition step can be executed by an imaging device installed to photograph through the front window of the vehicle, for example. Therefore, for example, without using a special imaging device such as a line scan camera, and without using a vehicle with a special structure in which the line scan camera is installed in the vertical direction of the road surface, the vehicle can be simply equipped. can. Therefore, the road surface damage position calculation method of the present embodiment can be implemented by installing relatively inexpensive equipment in many vehicles. As a result, road surface damage inspection can be performed at relatively low cost as a relatively frequent daily inspection.

In addition, since the road surface damage position calculation method of the present embodiment can specify the road surface damage position without using a GPS receiver, it is possible to prevent the inconvenience that the accuracy of position specification is lowered due to the reception condition of the GPS signal. . In addition, in the method of calculating the road surface damage position of the present embodiment, a GPS signal may be used as an auxiliary.

FIG. 2 is a schematic diagram showing how the reference linear body is photographed by the imaging device mounted on the vehicle in the reference photographing step of step S2. In FIG. 2, 1 is a road surface which is the surface of a road, 2 is a vehicle, 3 is an imaging device, and 16 is a road marking as a reference linear body. As shown in FIG. 2, in the reference photographing step, the vehicle 2 is positioned on the road surface 1 on which the vehicle 2 is traveling so that a linear road marking 16 extends in the width direction of the vehicle in front of the vehicle 2 in the traveling direction. 2 is arranged, the imaging device 3 photographs the road marking 16. - 特許庁

FIG. 3 is a block diagram showing the imaging device 3. As shown in FIG. As shown in FIG. 3, the imaging device 3 controls the operations of the lens 6, the imaging unit 7 that includes an imaging element that converts the light from the lens 6 into an electrical signal, and outputs image data, and the imaging device 3. It has a control unit 8 and a storage unit 9 as a storage device for storing image data created by the imaging unit 7 . The imaging device 3 is supplied with electric power by a storage battery (not shown) or a power line connected to the power supply device of the vehicle. The imaging device 3 also has an input unit 11 for inputting operation commands and various setting inputs, and a display unit 12 for displaying captured images and information on settings and operations. This imaging device 3 can be configured by a digital camera, a drive recorder, or the like. The imaging device 3, which is composed of a digital camera, a drive recorder, etc., can be installed in the interior of the vehicle 2 so as to photograph the road surface in front of the vehicle in the direction of travel and the surroundings of the road. The imaging device 3 can be arranged, for example, on the ceiling surface of the vehicle 2, a rearview mirror for checking the rear, the upper part of the front window, the dashboard, or the like. Therefore, unlike conventional inspection vehicles, there is no need to fix a special member to the outside of the vehicle to support the camera in the vertical direction of the road surface.

FIG. 4 is a schematic diagram showing an image 15 obtained by photographing the road marking 16 as the reference linear body with the imaging device 3 in the reference photographing step of step S2. As shown in FIG. 4, an image 15 is acquired showing a road marking 16 positioned in front of the vehicle 2 in the direction of travel and extending perpendicularly to the direction of travel. A stop line, for example, is preferably used as the road marking 16 . From this image 15, an image ruler 18 is created by cutting out a portion extending in the width direction of the road marking 16 and stored in the ruler layer in the image definition delivery step of step S3.

FIG. 5 is a schematic diagram showing a damaged frame 17 in which the image ruler 18 overlaps with the damaged road surface 19 in the damaged frame identifying process of step S7. This damage frame 17 is identified from a plurality of damage composite frames created by superimposing an image ruler 18 of a ruler layer on a plurality of frames including damage 19 in the damage ruler superimposing step of step S6.

FIG. 6 is a schematic diagram showing the marker frame 21 in which the image ruler 18 overlaps the position marker 20 in the marker frame identifying process of step S10. This marker frame 21 is specified from a plurality of marker composite frames created by superimposing the image ruler 18 of the ruler layer on a plurality of frames including the position marker 20 in the marker ruler superimposing step of step S9. .

FIG. 7 is a diagram schematically showing the processing performed in the distance calculation step of step S11, and shows the relationship between the speed at the time of acquiring the moving image frame and the shooting position of the frame. As shown in FIG. 7, it is assumed that the damage frame 17 is the n-th frame from the marker frame 21 of the moving image, and the velocity of the vehicle 2 is _V1 to _Vn in each frame from the marker frame 21 to the damage frame 17. are sequentially associated up to . It is also assumed that the shooting time interval of each frame is Δt. In this case, for example, the distance l ₁ between the marker frame 21 and the frame following this marker frame 21 is expressed as (V ₁ +V ₂ )Δt/2. Also, the distance l _n−1 between the frame before the damaged frame 17 and the damaged frame 17 is expressed as (V _n−1 +V _n )Δt/2. That is, the distance l _i between the i-th frame from the marker frame 21 and the i+1-th frame is expressed as (V _i +V _i+1 )Δt/2. That is, the distance l _i between the frames is obtained by multiplying the average of the velocities V _i and V _i+1 when the adjacent frames were shot by the frame shooting time interval Δt. Here, the frame shooting time interval Δt is the reciprocal of the frame rate r. The distance _L from the marked frame 21 to the damaged frame 17 can be obtained by calculating the sum of the distances li between the frames from 1 to n. Thus, in the distance calculation process, the distance from the imaging position of the marker frame 21 to the imaging position of the damaged frame 17 can be calculated based on the moving image and speed information obtained in the moving image acquisition process.

表１から分かるように、車両２が時速１００ｋｍで走行していた場合であっても、フレーム間の距離は９３ｃｍであるため、図８の位置標識２０が位置Ｐ_２又はＰ_３，のいずれに位置するフレームを選択しても、画像定規１８からのずれは、５０ｃｍ以下である。すなわち、位置標識２０の実際の位置に対する誤差は、９３ｃｍの半分である４６．５ｃｍに抑えられる。したがって、位置標識２０が画像定規１８と一致しなくても、実用的に許容可能な誤差の範囲で、路面の損傷位置を特定できる。なお、ステップＳ７の損傷フレーム特定工程において、画像定規１８が損傷１９に最も近接しているフレームである損傷フレームを特定する場合についても同様である。すなわち、損傷１９を含む複数の損傷複合フレームのうち、画像定規１８が損傷１９に最も近いフレームを選択することにより、フレームの撮影位置間の距離の半分の誤差で、損傷１９の位置を特定できる。 FIG. 8 is a schematic diagram showing, in one frame 22, the relationship between the image ruler 18 and the position marker 20 in a plurality of continuous frames in the marker frame identifying step of step S10. FIG. 8 shows the positions P ₂ , P ₃ and P ₄ of the position marker 20 in the three frames following the frame 22 in which the position marker 20 exists at the position P ₁ . It is assumed that all frames are shot at the same speed. Frames 22 in which position markers 20 are present at positions P ₁ , P ₂ , P ₃ , and P ₄ are all marker composite frames, from which marker frames are identified. Of the frames 22 in which the position marker 20 exists at positions P ₁ , P ₂ , P ₃ , and P ₄ , the frame in which the image ruler 18 is closest to the position marker 20 has the position marker 20 at position P ₂ or P ₃ . is a frame in By selecting one of these frames as the marker frame, the positional error of the marker 20 can be reduced to less than 1/2. Here, when the frame rate of the moving image captured by the imaging device 3 is 30 (fps), the mutual distance l between the shooting positions of adjacent frames changes as shown in Table 1 according to the speed of the vehicle.

As can be seen from Table 1, even when the vehicle ₂ is traveling at a speed of 100 km/h, the distance between the frames is ₉₃ cm. Even if the selected frame is located, the deviation from the image ruler 18 is no more than 50 cm. That is, the error with respect to the actual position of the position marker 20 is reduced to 46.5 cm, which is half of 93 cm. Therefore, even if the position marker 20 does not match the image ruler 18, the damaged position of the road surface can be identified within a practically allowable error range. The same applies to the case where the image ruler 18 identifies the damaged frame, which is the frame closest to the damage 19, in the damaged frame identifying step of step S7. That is, by selecting the frame closest to the damage 19 with the image ruler 18 from among multiple damage composite frames including the damage 19, the position of the damage 19 can be specified with an error of half the distance between the imaging positions of the frames. .

In FIG. 8, if there is no change in velocity between the times when the frames were captured when the position marker 20 was at positions P ₁ , P ₂ , P ₃ and P ₄ , the positions P ₂ , P ₃ and P ₄ of the position marker 20 are separated from each other by the same distance l. However, because the imaging device 3 is tilted with respect to the road surface and performs imaging, the mutual distances of the positions P ₁ , P ₂ , P ₃ , and P ₄ of the position marker 20 are different on the frame. appear. That is, the distance in the running direction of the vehicle 2 in the frame is reduced and displayed toward the top of the image. Therefore, by using a ruler with reference lines and auxiliary lines as a reference linear body for forming the image ruler, the positions of the marker 20 and damage 19 on the marker frame and damage frame can be specified with high accuracy. can be done.

FIG. 9 is a schematic plan view showing a ruler 23 as a reference linear body. The ruler 23 has a base line 24 formed in a straight line on a rectangular plate-like body, and three auxiliary lines 25, 25 arranged at equal intervals in the vertical direction of the base line 24 in FIG. , 25 are described. The reference line 24 and the auxiliary line 25 have the same width and are arranged at the same interval as the width. Also, the reference line 24 and the auxiliary line 25 are colored in different colors. Note that the number of auxiliary lines 25 is not limited to three.

This ruler 23 is arranged on the road surface and photographed by the imaging device 3 in the reference photographing step of step S2. Thereafter, in step S3, the reference line 24 and the auxiliary line 25 are separated from the photographed image and used as an image ruler. The image ruler having the reference line 24 and the auxiliary line 25 is superimposed on a plurality of frames including the damage 19 in the damage ruler superimposing step of step S6 to create a composite damage frame. Further, the image ruler having the reference line 24 and the auxiliary line 25 is superimposed on a plurality of frames including the position marker 20 in the marker rule superimposing step of step S9 to create a marker composite frame.

For example, when the reference line 24 and the auxiliary line 25 of the ruler 23 are described with a width of 5 cm and an arrangement interval, the marker composite frame and the damage composite frame of the moving image captured when the speed of the vehicle 2 is 100 km/h In any of the frames, the lesion 19 and the position marker 20 can be located based on the reference line 24 and the auxiliary line 25 . For example, if an image ruler formed using ruler 23 _above is used to create a marker composite frame similar to frame ₂₂ of FIG. The position marker 20 fits within the description range of the auxiliary lines 25 , 25 , 25 . Therefore, the distance of the position marker 20 from the reference line 24 can be specified with an error of 5 cm or less. By using the ruler 23 including the reference line 24 and the auxiliary line 25 as the reference linear body in this way, the positions of the damage 19 and the position marker 20 in the moving image can be accurately specified.

FIG. 10 is a schematic plan view showing a ruler 26 as a reference linear body of another embodiment. This ruler 26 has a reference line 24 and auxiliary lines 25, 25, 25 similar to the ruler 23 of FIG. , 27. A marker composite frame and a damage composite frame including an image ruler formed by the ruler 26 of this embodiment can readily identify auxiliary lines 25 overlapping or adjacent to the damage 19 or location marker 20 based on the numerals 27. can. Therefore, the position of the damage 19 or the position marker 20 can be easily identified with high accuracy.

In the above-described embodiment, the imaging device 3 has a push button connected as an input unit for receiving input from a staff member, an index information generation unit that generates index information when the push button is pressed, and a an index information recording unit that records in a storage unit 9 in association with frames of a moving image acquired when is pressed; You may comprise a part. In this case, in the moving image acquisition process, when the person in charge of getting on the vehicle 2 perceives the damage 19 by visual recognition of the road surface, vibration of the vehicle 2, or the like, the push button is pressed. Also, when the staff sees the position marker 20 around the road, the push button is pressed. As a result, the index information generating section generates index information, and the index information recording section records the index information in the storage section 9 in association with the frame of the moving image captured when the push button is pressed. As a result, the moving image is stored in the storage unit 9 in association with the index information. After that, in the damage candidate frame identifying step, the frame search unit easily and quickly extracts frames showing damage to the road surface from the moving images stored in the storage unit 9 of the imaging device 3 based on the index information. can be identified by Further, in the marker candidate frame identifying step, the frame search unit easily and quickly extracts the frame in which the position marker 20 is shown from the moving image stored in the storage unit 9 of the imaging device 3 based on the index information. can be specified. Here, the push button may be configured to accept different buttons or push patterns depending on whether the road surface damage 19 is found or the position marker 20 is found. Correspondingly, the index information generator may be configured to generate damage index information corresponding to the damage 19 and marker index information corresponding to the location marker 20 . By associating the damage index information and the marker index information with the frame of the moving image, the damage candidate frame and the marker candidate frame can be identified more quickly.

Further, in the above-described embodiment, the imaging device 3 may include a transmission unit that transmits captured moving images and still images, speed information, and index information by wireless communication. In this case, an information processing device having a receiving section for receiving information transmitted by the transmitting section of the imaging device 3 is arranged at a location different from that of the vehicle 2 . With this information processing device, an image standard delivery process, a damaged candidate frame identifying process, a damaged ruler superimposing process, a damaged frame identifying process, a marker candidate frame identifying process, a marker ruler superimposing process, a marker frame identifying process, and a distance Execute the calculation step. As a result, damage to the road surface can be detected at a position away from the vehicle 2 substantially in real time as the image of the road is captured by the imaging device 3 of the vehicle 2, and the position of the damage can be specified. Therefore, the road surface can be repaired in a short period of time after the image pickup device 3 of the vehicle 2 takes an image of the road.

Further, when the imaging device 3 and an information processing device arranged at a location different from the vehicle 2 are connected by wireless communication, a push button as the input unit is connected to the imaging device 3, and the index information generating unit and the push button are connected to the imaging device 3. and the index information recording section, and the information processing apparatus preferably includes the frame searching section. In this case, when a person in charge of getting on the vehicle 2 perceives the damage 19 on the road surface and presses the push button, the index information generation unit generates index information in response to this and associates it with the frame of the moving image. As surrounding frames, the transmission unit transmits frames acquired after several frames before the frame associated with the index information. This frame transmission continues until the staff member visually recognizes the position marker 20 and presses the push button, and in response, the index information generation unit generates index information and associates it with the frame of the moving image. It is transmitted up to several frames after the attached frame. In this way, by transmitting the frames around the frame associated with the index information indicating the damage 19 to the frames around the frame associated with the index information indicating the position indicator 20, these frames are transmitted. With the received information processing device, the damage candidate frame showing the damage 19 and the marker candidate frame showing the position marker 20 can be reliably identified, and the damaged frame and the marker frame can be accurately identified.

The present invention is not limited to the embodiments described above, and many modifications are possible within the technical concept of the present invention by those skilled in the art.

1 road surface 2 vehicle 3 imaging device 6 lens 7 imaging unit 8 control unit 9 storage unit 11 input unit 12 display unit 16 road marking 17 damage frame 18 image ruler 19 damage 20 position mark 21 mark frame 22 frame 23, 26 ruler 24 reference line 25 auxiliary line 27 number

Claims (12)

a reference photographing step of photographing a reference linear body installed on a road and extending in the width direction of the road with an imaging device installed in a vehicle;
an image ruler delivery step of separating an image ruler indicating the reference linear body from the image taken in the reference photography step and storing the image ruler in a ruler layer;
A moving image acquisition step of capturing a moving image by capturing a road surface with the imaging device while traveling on the road to be inspected, and recording the speed of the vehicle corresponding to each frame of the moving image;
a damage candidate frame identification step of identifying a frame containing damage to the road surface from the moving image captured in the moving image acquisition step;
a damaged ruler superimposing step of superimposing an image ruler of the ruler layer on the frame identified in the damaged candidate frame identifying step;
a damaged frame identifying step of identifying a damaged frame, which is a frame in which the image ruler overlaps or is closest to the damage, among the frames on which the image ruler is superimposed in the damage ruler superimposing step;
a sign candidate frame specifying step of specifying a frame containing a position sign specifying a position on the road from the moving image captured in the moving image acquisition step;
a sign ruler superimposing step of superimposing an image ruler of the ruler layer on the frame identified in the sign candidate frame identification step;
a marker frame identifying step of identifying a marker frame, which is a frame in which the image ruler overlaps or is closest to the position marker, among the frames on which the image ruler is superimposed in the marker ruler superimposing step;
The distance between the photographing position of the damaged frame identified in the damaged frame identifying step and the photographing position of the sign frame identified in the sign frame identifying step is the speed and frame of the vehicle recorded in the moving image obtaining step. and a distance calculation step of calculating based on the rate. In the method for calculating the road surface damage position according to claim 1,
The road surface damage position calculation method, wherein the marker candidate frame identifying step identifies a frame photographed at a position closest to the damaged frame identified in the damaged frame identifying step and including the position marker. In the method for calculating the road surface damage position according to claim 1,
The road surface damage position calculation method, wherein the marker candidate frame identification step identifies a frame including the position marker separated by the fewest number of frames from the damaged frame identified in the damaged frame identification step. In the method for calculating the road surface damage position according to claim 1,
A road surface damage position calculation method, wherein the reference photographing step photographs a road marking drawn on the road as the reference linear body. In the method for calculating the road surface damage position according to claim 1,
The method for calculating a road surface damage position, wherein the reference photographing step photographs a ruler installed on the road as the reference linear body. In the method for calculating the road surface damage position according to claim 5,
A road surface damage position calculation method, wherein the ruler includes a plurality of parallel lines provided at equal intervals in plan view. In the method for calculating the road surface damage position according to claim 1,
The road surface damage position calculation method, wherein the damage candidate frame identifying step identifies a frame including the road surface damage by image recognition. In the method for calculating the road surface damage position according to claim 1,
The road surface damage position calculation method, wherein the marker candidate frame specifying step specifies a frame including the position marker by image recognition. In the method for calculating the road surface damage position according to claim 1,
In the moving image acquisition step, the damage index information is associated with the frame of the moving image according to the input by the staff member who perceives the damage to the road surface,
In the damage candidate frame identifying step, a frame including the damage to the road surface is identified by searching frames surrounding the frame associated with the damage index information of the moving image. How the position is calculated. In the method for calculating the road surface damage position according to claim 9,
In the moving image acquisition step, the marker index information is associated with the frame of the moving image in response to an input by an attendant who perceives the position marker;
The road surface damage position is characterized in that in the sign candidate frame identifying step, a frame including the position sign is identified by searching frames surrounding the frame associated with the sign index information of the moving image. calculation method. In the method for calculating the road surface damage position according to claim 10,
Frames between the frames surrounding the frame associated with the damage index information and the frames surrounding the frame associated with the sign index information among the moving image and the speed information obtained in the moving image obtaining step and a transmission step of transmitting speed information;
a receiving step of receiving the frame and speed information transmitted in the transmitting step;
Based on the frames and speed information received in the receiving step, the damaged candidate frame identifying step, the damaged ruler superimposing step, the damaged frame identifying step, the marker candidate frame identifying step, the marker ruler superimposing step, and the marker frame identifying step. A method for calculating a road surface damage position, characterized by performing the step and the distance calculating step. In the method for calculating the road surface damage position according to claim 1,
A road damage position calculation method, wherein the imaging device is a drive recorder.

Images