JP6965536B2 - Information processing system, evaluation system, information processing method and program - Google Patents

Description

The present invention relates to an information processing system for estimating an evaluation interval to be evaluated, an evaluation system, an information processing method, and a program for causing a computer to execute a process for estimating the evaluation interval.

Roads, railroad tracks, tunnels, etc. are damaged by vehicle traffic, climate, external pressure, etc., and if the damage spreads, it will lead to serious accidents, etc., so regular inspections will be conducted to evaluate it. Is needed. As for the road surface, typical damage inspection items include cracks, rutting (unevenness in the width direction of the road), flatness (unevenness in the vehicle traveling direction), and the like.

When inspecting these items, it is necessary to acquire captured images of the road surface for cracks, rutting, and three-dimensional shape measurement data for flatness. For this reason, camera equipment for crack photography, optical cutting measurement equipment that measures the three-dimensional shape by irradiating the target with a laser and taking a camera image, and road surface height sensors installed at two or more locations in front of and behind the vehicle are used. The equipped system is generally used.

Road inspections are conducted once a year for specific routes, and a preliminary survey is conducted prior to photography and measurement using the above system. In the preliminary survey, the start position and end position of the evaluation section (inspection section) of the evaluation target (road surface), the boundary of the municipality, the lane composition, the position of the road structure (bridge, tunnel, viaduct), etc., and the route are confirmed. It is necessary to mark the road surface at the start position and the end position of the inspection section so that they can be confirmed from the photographed image. This road surface marking increases the inspection cost and the inspection man-hours.

Therefore, a technique has been proposed in which a road surface index indicating road surface properties is calculated based on the acquired position data and acceleration data, and a lane corresponding to a pre-measured road surface index similar to the road surface index is specified as the lane in which the vehicle has traveled. (See Patent Document 1).

However, the above technique has a problem that it may not be determined as a similar pre-measured road surface index even though it is a corresponding section, depending on the condition of the road surface, changes over time, and the like.

The present invention has been made in view of the above problems, and is a system capable of estimating an evaluation interval with high accuracy even when the state of the evaluation target is different or the evaluation target changes with time. The purpose is to provide methods and programs.

In order to solve the above-mentioned problems, in one embodiment of the invention, it is an information processing system that estimates an evaluation interval to be evaluated.
An image acquisition means for acquiring an image including an evaluation target imaged by an image pickup device mounted on a vehicle traveling on the evaluation target, and an image acquisition means.
A detection means that refers to an image acquired by an image acquisition means and detects a reference object having a specified feature, and a detection means.
In response to the detection of the reference object by the detection means, the instruction means for instructing the distance measuring device mounted on the vehicle to measure the mileage of the vehicle, and
A distance acquisition means for acquiring the mileage measured by the distance measuring device, and
Includes a position estimation means that estimates the positions of one end and the other end of the evaluation section based on the position where each mileage acquired by the distance acquisition means has reached the specified distance for one end and the other end of the evaluation section. , Provides an information processing system.

According to the present invention, it is possible to estimate the evaluation interval with high accuracy even when the state of the evaluation target is different or the evaluation target changes with time.

The figure which showed the configuration example of the evaluation system. The figure which exemplifies the place where the road surface as an evaluation target is inspected by an evaluation system. The figure which showed an example of the hardware composition of the road surface camera which image | images the road surface provided in the evaluation system. The figure which showed an example of the hardware composition of the information processing apparatus provided in the evaluation system. The block diagram which showed an example of the functional structure of an information processing apparatus. A flowchart showing an example of a road surface inspection process executed by the evaluation system. A flowchart showing an example of the process of setting the start point of the inspection section executed by the evaluation system. The figure which illustrated the position of the vehicle on the road surface at the time of specifying the start point of an inspection section, and the image taken by the peripheral camera. A flowchart showing an example of the process of setting the end point of the inspection section executed by the evaluation system. The figure which illustrated the position of the vehicle on the road surface at the time of specifying the end point of an inspection section, and the image taken by a peripheral camera. The figure explaining the summary information. The figure which showed an example of a reference subject. The figure which showed another example of a reference subject. The figure explaining the process of estimating the start point or the end point away from a reference subject. The flowchart which showed an example of the process of estimating the section length. The flowchart which showed an example of the quality judgment process of the position estimation data and the road surface property measurement data.

FIG. 1 is a diagram showing a configuration example of an evaluation system. The evaluation system is a system that estimates the evaluation section to be evaluated and evaluates the surface texture of the estimated evaluation section. The evaluation target may be any structure as long as it extends long to one of the road surface, the railroad track, the inner surface of the tunnel, and the like. This evaluation system is used to evaluate the surface properties of such a structure that extends long on one side by dividing it into certain sections. Hereinafter, the evaluation target will be the road surface, the evaluation section will be the road surface inspection section, and the evaluation system will be described as a road surface inspection system that inspects the road surface and evaluates the presence or absence of damage and its degree.

The road surface inspection system stores various information such as a road surface camera 12 mounted on the vehicle 10 as an image pickup device for imaging the road surface 11, a peripheral camera 13 as an image pickup device for imaging the periphery of the road surface 11 including the road surface 11. It also includes an information processing device 14 that performs processing. Further, the road surface inspection system is mounted on the vehicle 10 and further includes a GPS (Global Positioning System) sensor 15 as a position measuring device for measuring the position of the vehicle 10 and a distance measuring device 16 for measuring the mileage of the vehicle 10. I have.

The flatness of the road surface 11 is impaired due to cracks and rutting caused by the passage of a large number of vehicles. These are included in the damage inspection items as damage to the road surface 11. Rutting is unevenness in the width direction of the road. The impaired flatness means that unevenness occurs in the traveling direction of the vehicle.

The road surface cameras 12 are installed on the rear side of the vehicle 10 as shown in FIGS. 1 and 2 (a), and are arranged in a row so that two or more cameras face the road surface 11 diagonally downward as shown in FIG. 2 (b). It is a stereo camera arranged side by side. As shown in FIGS. 2A and 2B, the road surface camera 12 captures a predetermined range 17 of the road surface 11 to be evaluated by each camera. The stereo camera is a camera for obtaining depth information such as unevenness formed on the road surface 11 to be evaluated by using, for example, the parallax information of two cameras arranged side by side.

Parallax means that the target point (target point) differs in the visible direction, and by using the parallax, the distance to the target point can be obtained by the principle of triangulation. If this distance is constant at any target point, the road surface 11 has no unevenness, and if it is not constant, it can be detected that the road surface 11 has unevenness. In addition, it is possible to detect the height of a mountain or a groove of a depth according to the distance.

In this road surface inspection system, as shown in FIG. 2, while the vehicle 10 is traveling on the road surface 11, the road surface camera 12 takes an image of a predetermined range 17 of the road surface 11 so that a part thereof overlaps in the traveling direction. .. Then, the surface texture such as the unevenness of the road surface 11 is measured from each of the captured images, and the presence or absence of damage to the road surface 11 and the degree of damage are evaluated based on the measured surface texture. The information processing apparatus 14 can perform a process of measuring the above surface texture from the captured image, detecting the presence or absence of damage based on the surface texture, and evaluating the degree of damage.

In the examples shown in FIGS. 1 and 2, the road surface camera 12 is used to acquire information for measuring the surface texture of the road surface 11, but the present invention is not limited to this. For example, a light cutting measuring device composed of a laser irradiation device and a camera, a monocular camera capable of recognizing a distance, or the like may be used.

The peripheral camera 13 is installed in the vehicle 10 and images a reference object having a predetermined feature in the road surface 11 in front of the vehicle 10 and its surroundings. The reference object may be any object as long as it is a reference object when estimating one end and the other end of the inspection section, respectively. Examples of the reference object include a traffic light, a sign, a curb, a stop line, a pedestrian crossing line, a manhole, an intersection center, a mailbox, and the like.

The information processing device 14 is a tablet terminal, a notebook PC, or the like, and estimates one end and the other end of the inspection section of the road surface 11 based on the above-mentioned measurement and evaluation of the surface texture and the image captured by the peripheral camera 13. Execute the process.

The GPS sensor 15 includes a GPS receiver, receives signals transmitted from a plurality of GPS satellites, and measures the position of the GPS receiver as latitude and longitude information from the time difference in arrival time of the signals. Therefore, the GPS sensor 15 can also measure the current position of the vehicle 10. However, since the GPS sensor 15 measures the position of the GPS receiver with an accuracy of several meters, it means that there is a measurement error of about several meters, and the accuracy is low.

The distance measuring device 16 is a mileage meter or the like, and calculates the mileage by, for example, holding information on the circumference of the wheels of the vehicle 10 and measuring the number of rotations. The mileage can be calculated by multiplying the circumference of the wheel by the measured number of revolutions. Since this is an example, the distance may be measured by other methods.

The road surface inspection system uses a peripheral camera 13, an information processing device 14, a GPS sensor 15, and a distance measuring device 16 to cover one end and the other end of the inspection section of the road surface 11 with higher accuracy than the position measurement accuracy of the GPS sensor 15. presume. One end is the start position (start point) of the inspection section, and the other end is the end position (end point) of the inspection section.

The vehicle 10 may be of a two-box type consisting of two boxes (boxes) consisting of a front engine portion and a passenger compartment portion continuous thereto, or a one-box type vehicle body type consisting of one box. Then, the road surface camera 12 and the peripheral camera 13 can be mounted on the roof of the vehicle 10, and can image the road surface 11 behind the vehicle 10 and the periphery in front of the vehicle 10. The peripheral camera 13 may be mounted near the windshield in the vehicle. The information processing device 14 may be installed inside the vehicle 10, and the GPS sensor 15 and the distance measuring device may be installed inside or outside the vehicle 10.

In the example shown in FIG. 1, the surface texture is measured, evaluated, and the inspection section is estimated by one device called the information processing device 14, but the present invention is not limited to this, and each process is performed by two or more devices. It may be a configuration to be executed. At that time, each device may be connected by a wire such as a cable, or may be connected wirelessly. In addition, each device may be directly connected to each other, or may be connected via a network such as LAN (Local Area Network) or WAN (Wide Area Network).

When connected via a network, the network may be one network or a network consisting of two or more connected by a relay device such as a router or a proxy server. Therefore, an information processing system including any of an information processing device 14 composed of one device and a system composed of two or more devices can be adopted as a system for executing the above processing. The information processing system is not limited to one composed of one or more devices, and may be composed of electronic components (modules) such as integrated circuits.

The road surface camera 12 is not limited to being installed on the rear side of the vehicle 10 in the direction opposite to the traveling direction, and may be installed on the front side of the vehicle 10 in the traveling direction. .. Like the road surface camera 12, the peripheral camera 13 is not limited to being installed on the front side of the vehicle 10 in the direction of travel, but is also installed on the rear side of the vehicle 10 in the direction opposite to the direction of travel. You may be.

The hardware configuration of the road surface camera 12 will be described with reference to FIG. The road surface camera 12 is composed of two or more imaging means. In the example shown in FIG. 3, the road surface camera 12 is composed of two left and right cameras 20 and 21, the camera 20 includes a lens 22, an image sensor 23, and a sensor controller 24, and the camera 21 also has a lens 25 and an image sensor 26. , The sensor controller 27 is provided. The lens 22 and the lens 25, the image sensor 23 and the image sensor 26, and the sensor controller 24 and the sensor controller 27 are the same.

The two left and right cameras 20 and 21 are configured to be assembled in parallel, and the road surface 11 is simultaneously imaged by the two cameras 20 and 21. Since the positions on the two images obtained by imaging are different from each other, the position and the three-dimensional shape of the object can be obtained from the difference in the positions (parallax). The method is well known and will not be described in detail here.

The image sensor 23 converts the light incident through the lens 22 into an electric signal and outputs it as image data. As the image sensor 23, for example, an image sensor composed of a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor) can be used.

The sensor controller 24 performs exposure control of the image sensor 23, image readout control, communication with an external device, image data transmission control, and the like. Exposure refers to exposing the image sensor 23 to incident light that has passed through the lens 22. The exposure control can be realized by controlling the exposure time by opening and closing the shutter, for example. The sensor controller 24 communicates with the information processing device 14 as an external device, and transmits image data of the captured image to the information processing device 14.

The peripheral camera 13 can adopt the same configuration as one of the two left and right cameras 20 and 21 constituting the road surface camera 12. Further, the peripheral camera 13 may be a stereo camera composed of two left and right cameras, like the road surface camera 12. The road surface camera 12 and the peripheral cameras 13 are not limited to one each, and may be a plurality of cameras.

The hardware configuration of the information processing apparatus 14 will be described with reference to FIG. The information processing device 14 has a CPU (Central Processing Unit) 30, a ROM (Read Only Memory) 31, a RAM (Random Access Memory) 32, an HDD (Hard Disk Drive) 33, a communication I / F 34, and an input / output I as hardware. It includes / F35, a display device 36, and an input device 37. These are connected to the bus 38 and exchange information with each other via the bus 38.

The CPU 30 controls the entire information processing device 14, and executes processing such as detecting the presence or absence of damage to the road surface 11 and evaluating the damage. Further, the CPU 30 executes a process of estimating one end and the other end of the inspection section of the road surface 11 based on the image captured by the peripheral camera 13. The ROM 31 stores a boot program for activating the information processing device 14, firmware for controlling the HDD 33, the communication I / F 34, and the like. The RAM 32 provides a work area for the CPU 30.

The HDD 33 stores an OS (Operating System), a received image, a program for executing the above processing, and the like. The communication I / F 34 connects to the road surface camera 12, the peripheral camera 13, the GPS sensor 15, and the distance measuring device 16, and communicates with these devices.

The input / output I / F 35 connects the display device 36 and the input device 37 to the bus 38, displays the information processed by the CPU 30 on the display device 36, and provides the information input from the input device 37 to the CPU 30. Controls the input and output of information. The display device 36 is a device for displaying information such as a liquid crystal display, and the input device 37 is a device for receiving input of information such as a mouse and a keyboard.

In addition to these, the information processing device 14 is connected to an external storage device such as an external HDD, and has an external storage I / F, a voice input device, and a voice output device for controlling storage and reading of information in the external storage device. , An image pickup device such as a camera may be provided.

The functional configuration of the information processing apparatus 14 will be described with reference to FIG. The information processing device 14 can generate one or more functional units by the CPU 30 executing a program stored in the HDD 33, and can include those functional units.

As a functional unit, the information processing device 14 stores a road surface image acquisition unit 40 that acquires two images simultaneously captured by the road surface camera 12 and two images acquired by the road surface image acquisition unit 40 as image data. A unit 41 is provided. Further, the information processing device 14 calculates the distance from the road surface camera 12 to each position of the road surface 11 by using the two images stored by the storage unit 41 to measure the surface texture, and measures the surface texture of the data. A measurement unit 42 that outputs data is provided. Further, the information processing device 14 detects the presence or absence of damage to the road surface 11 based on the surface property measured by the measuring unit 42, that is, the road surface property measurement data output by the measuring unit 42, and evaluates the degree of damage. It includes an evaluation unit 43 for output. In this example, the road surface property measurement data output by the measurement unit 42 is temporarily stored in the storage unit 41, and the evaluation unit 43 acquires the road surface property measurement data from the storage unit 41.

The information processing device 14 has a function of estimating an inspection section, and as functional units for that purpose, a peripheral image acquisition unit 44, an object detection unit 45, an instruction unit 46, a distance acquisition unit 47, and a position estimation unit 48. And. The peripheral image acquisition unit 44 acquires an image captured by the peripheral camera 13 from the peripheral camera 13.

The object detection unit 45 refers to the image acquired by the peripheral image acquisition unit 44, and detects a reference object having the specified characteristics, that is, a reference subject that serves as a reference when estimating the start point or end point of the inspection section. do. The object detection unit 45 sequentially refers to the images acquired by the peripheral image acquisition unit 44, and confirms whether or not a reference subject exists in the image.

In response to the detection of the reference subject by the object detection unit 45, the instruction unit 46 instructs the distance measuring device 16 mounted on the vehicle 10 to start measuring the mileage of the vehicle 10. In response to this instruction, the distance measuring device 16 starts measuring the mileage of the vehicle 10. The distance acquisition unit 47 receives a notification from the instruction unit 46 that the measurement of the mileage has started, and acquires the mileage measured from the distance measuring device 16. After receiving the above notification, the distance acquisition unit 47 can acquire the mileage at regular time intervals.

The position estimation unit 48 confirms whether or not the mileage acquired by the distance acquisition unit 47 has reached a predetermined distance measured in advance, and if the distance is reached, an inspection is performed based on the position where the distance is reached. Estimate the starting point of the interval. The position estimation unit 48 estimates the end point in the same manner.

The starting point is a position estimated by detecting the first reference subject by the object detection unit 45 and measuring the mileage. The end point is a position that has passed the start point, then the object detection unit 45 detects the reference subject, and the mileage is measured and estimated. The start point and end point may be estimated by estimating the position where the specified distance is reached as the start point and the end point, or the position separated from the position where the specified distance is reached by a predetermined distance. It may be estimated as a start point and an end point. Further, either one of the start point and the end point may be set as a position where the designated distance is reached, and the other may be set as a position separated by a predetermined distance from the position where the designated distance is reached.

The information processing device 14 can include a position information acquisition unit 49 that acquires the position information of the vehicle 10 from the GPS sensor 15. Further, the information processing device 14 is an intersection as a target object existing in front of or behind the start point or end point of the inspection section in the traveling direction of the vehicle 10 based on the position information acquired by the position information acquisition unit 49. The intersection detection unit 50 for detecting the above can be provided. The target object may be any object as long as it is a target object for specifying the reference subject, and examples thereof include an intersection and a T-junction. Hereinafter, the target object will be described as an intersection.

The intersection detection unit 50 detects an intersection when the distance between the GPS sensor 15 and the intersection is within a certain distance. The intersection detection unit 50 uses the position information of the intersection when detecting the intersection. The position information of the intersection can be set in advance, for example, as GPS position information of the center position of the intersection. The center position of the intersection is a position that is an intersection of the center line of the road surface extending in one direction (center line) and the center line of the road surface extending perpendicular to the center line.

Upon receiving the detection of the intersection by the intersection detection unit 50, the instruction unit 46 can instruct the peripheral camera 13 to start imaging. Then, the instruction unit 46 can notify the peripheral image acquisition unit 44 that the peripheral camera 13 has started imaging.

The road surface inspection process executed by the road surface inspection system will be described with reference to FIG. The inspection process starts from step 600, and in step 605, the outline information of the inspection section is set. The summary information is GPS position information regarding the start and end points of the inspection section. GPS position information is low-precision information with an accuracy of several meters to 10 m. The GPS position information is the position information of the intersection existing before the start point and the end point of the inspection section.

In step 610, the start point of the inspection section is set. The starting point is estimated based on the position where the distance measurement is started when the reference subject is detected after detecting the intersection near the reference subject and the distance is reached. The details will be described later.

In step 615, traveling photography is performed from the position where the start point of the inspection section is set, and recording of the captured road surface image is started. The road surface image is repeatedly imaged at a timing when a part of the image overlaps with respect to the traveling direction of the vehicle 10. Since a plurality of road surface cameras 12 are installed so as to be able to take an image of the entire width direction of the road surface 11, a plurality of road surface cameras 12 simultaneously take an image so that a part of the image overlaps with the width direction. The road surface image is captured in this way, and the end point of the inspection section is set in step 620. The end point is estimated by the same method as the start point. This will also be described in detail later.

In step 625, the mileage measured from the start point to the end point of the inspection section is set as the section length. In step 630, the set section length is divided into subdivided sections with a maximum length of 100 m, the crack rate, rutting amount, flatness, and MCI (Maintenance Control Index) are obtained, and the form with these is output as the section evaluation result. MCI is a pavement maintenance index that expresses the degree of pavement damage by combining three factors: crack rate, rutting amount, and flatness. When the section evaluation result is output for each subdivision section, the process ends in step 635.

In addition to the above, the section evaluation result can include the number of cracks, the area of the partially repaired part (patch), and the amount of dents. In this way, the section position can be set automatically, and even in the re-evaluation run after a certain period of time, it is possible to evaluate at almost the same position and monitor the road surface condition with the passage of time. In order to realize this monitoring, the information processing device 14 can be further provided with a display unit, and the display unit can display the evaluation result, the image captured by the road surface camera 12, and the like.

The process of setting the start point of the inspection section of step 610 shown in FIG. 6 will be described with reference to FIGS. 7 and 8. Starting from step 700 shown in FIG. 7, at this stage, the GPS sensor 15 has already started measuring the position of the vehicle 10, and the road surface camera 12, the peripheral cameras 13, and the distance measuring device 16 are waiting for operation (standby). State).

In step 705, the position information acquisition unit 49 acquires the position information of the vehicle 10 from the GPS sensor 15, and in step 710, it is confirmed whether or not the intersection detection unit 50 has detected the intersection based on the acquired position information. At this time, as shown in FIG. 8A, the vehicle 10 is traveling around the intersection 60 toward the intersection 60. The intersection detection unit 50 confirms whether or not the center position of the intersection 60 is within 10 m, for example, with the GPS sensor 15 as the center.

It is desirable that the detection range of the GPS sensor 15 is set to be longer than the sum of the position detection accuracy of the GPS sensor 15 and the distance from the installation position of the GPS sensor 15 to the center position of the image pickup range of the road surface camera 12. The central position of the imaging range of the road surface camera 12 will be described later.

If no intersection is detected, the process returns to step 705, and if an intersection is detected, the process proceeds to step 715. In step 715, the instruction unit 46 instructs the peripheral camera 13 to start imaging, and the peripheral camera 13 that has started the operation starts capturing an image including the road surface 11 and the periphery of the road surface 11. .. The image captured at this time is an image of the road surface 11 extending in one direction in the traveling direction of the vehicle 10, as shown in FIG. 8 (b).

In step 720, the peripheral image acquisition unit 44 acquires an image from the peripheral camera 13, and in step 725, the object detection unit 45 refers to the image and first confirms whether or not the intersection is included in the image. Therefore, the object detection unit 45 can also detect an intersection, which is a target object in the image, as an object. If it is not included, the process returns to step 720, and if it is included, it is assumed that the intersection 60 is detected, and the process proceeds to step 730.

The image when the intersection 60 is detected is an image as shown in FIG. 8 (c). In this image, the road surface 11 extending with almost the same vehicle width on one side approaches the intersection 60, so that the vehicle width is widened to the left and right only at the intersection 60, and it extends to the left and right edges of the image. It is extending. The roads that extend to the left and right are hidden by buildings, etc., and may not extend to the left and right edges of the image. Therefore, by detecting the part where the vehicle width extends to the left and right beyond a certain length, an intersection 60 can be detected. When the intersection 60 is detected, the instruction unit 46 gives an instruction to the road surface camera 12, and the road surface camera 12 that has started the operation starts imaging the road surface 11.

In step 730, the object detection unit 45 refers to the image and confirms whether the reference subject is included in the image. If it is not included, the process returns to step 720, and if it is included, it is assumed that the reference subject has been detected, and the process proceeds to step 735. The image when the reference subject is detected is an image as shown in FIG. 8D. The reference subject 61 can be detected when at least a part of the reference subject 61 is included in the image and can be recognized as the target reference subject 61. During this time, the image pickup by the road surface camera 12 is continued. However, the captured image is not recorded.

In step 735, the instruction unit 46 gives an instruction to the distance measuring device 16, and the distance measuring device 16 starts measuring the mileage of the vehicle 10. The situation at this time is the position of the road surface camera 12 mounted on the vehicle 10 as shown in FIG. 8 (e), specifically, the central position of the imaging range of the road surface camera 12 and the reference subject 61 facing the vehicle 10. The distance to the nearest end is the distance [D_1]. Therefore, the mileage measurement is started with the mileage value [D_C1] = 0, and the mileage is measured until the mileage [D_1] is reached.

In step 740, the distance acquisition unit 47 acquires the mileage measured from the distance measuring device 16. In step 745, the position estimation unit 48 confirms whether or not the distance has reached a predetermined distance based on the acquired mileage. That is, it is confirmed whether or not the distance [D_1] shown in FIG. 8 (e) has been reached. If it has not been reached, the process returns to step 740, and if it has been reached, the process proceeds to step 750. When the distance [D_1] is reached, the position of the road surface camera 12 mounted on the vehicle 10 as shown in FIG. 8 (f), specifically, the central position of the imaging range of the road surface camera 12, and the above-mentioned reference subject 61. The end is at a position that coincides with the direction of travel.

In step 750, the start point of the inspection section is estimated from the position where the designated distance is reached. Here, the position where the specified distance is reached is estimated as the start point of the inspection section. Then, in step 755, the distance value and the time of the mileage measured up to the position where the designated distance is reached are recorded, and the start point of the inspection section is set. Further, the captured image at that time is recorded as an imaging frame, and the recording of the image captured by the road surface camera 12 is started.

When the start point of the inspection section is set, the instruction unit 46 instructs the peripheral camera 13 to end the imaging, and the peripheral camera 13 returns to the standby state. When the state returns to the standby state, the process of setting the start point ends in step 760. At this time, the instruction unit 46 can instruct the distance measurement device 16 to end the measurement of the mileage.

In the inspection section, the process of step 615 of FIG. 6 is performed, and the road surface camera 12 simultaneously captures the road surface 11 and records the simultaneously captured images. Distance measurement is also carried out. The distance can be measured by a distance measuring device provided separately. The distance can be measured by the distance measuring device 16 following the setting of the start point of the inspection section.

A process of setting the end point of the inspection section of step 620 shown in FIG. 6 will be described with reference to FIGS. 9 and 10. Starting from step 900, the recording of the image captured by the road surface camera 12 is continued at this time. In addition, distance measurement is continuing. In step 905, the position information acquisition unit 49 acquires the position information of the vehicle 10 from the GPS sensor 15, and in step 910, it is confirmed whether or not the intersection detection unit 50 has detected the intersection based on the acquired position information. This intersection is an intersection near the end point of the inspection section, and is an intersection 62 different from the intersection 60 near the start point.

At this time, as shown in FIG. 10A, the vehicle 10 is traveling around the intersection 62 toward the intersection 62. The intersection detection unit 50 confirms whether or not the center position of the intersection 62 is within 10 m, for example, with the GPS sensor 15 as the center. The position information of this center position is also preset as GPS position information.

If the intersection is not detected in step 910, the process returns to step 905, and if the intersection is detected, the process proceeds to step 915. In step 915, the instruction unit 46 instructs the peripheral camera 13 to start imaging, and the peripheral camera 13 that has started the operation captures an image including the road surface 11 and the periphery of the road surface 11. The image captured at this time is an image of the road surface 11 extending in one direction in the traveling direction of the vehicle 10 as shown in FIG. 10 (b).

In step 920, the peripheral image acquisition unit 44 acquires an image from the peripheral camera 13, and in step 925, the object detection unit 45 refers to the image and first confirms whether or not the intersection is included in the image. If it is not included, the process returns to step 920, and if it is included, it is assumed that the intersection 62 is detected, and the process proceeds to step 930.

The image when the intersection 62 is detected is an image as shown in FIG. 10 (c). In this image, the road surface 11 extending with almost the same vehicle width on one side approaches the intersection 62, so that the vehicle width is widened to the left and right only at the intersection 62, which extends to the left and right edges of the image. It is extending. Note that the road extending to the left and right is hidden by a building or the like and may not extend to the left and right edges of the image. Therefore, by detecting the portion where the vehicle width extends to the left and right by a certain length, the intersection 62 Can be detected. When the intersection 62 is detected, the instruction unit 46 gives an instruction to the road surface camera 12, and the road surface camera 12 that has started the operation starts imaging the road surface 11.

In step 930, the object detection unit 45 refers to the image and confirms whether the reference subject is included in the image. If it is not included, the process returns to step 920, and if it is included, it is assumed that the reference subject has been detected, and the process proceeds to step 935. The image when the reference subject is detected is an image as shown in FIG. 10 (d). The reference subject 63 can be detected when at least a part of the reference subject 63 is included in the image and can be recognized as the target reference subject 63. During this period, the image pickup by the road surface camera 12 and the recording of the captured image are continued.

In step 935, the instruction unit 46 gives an instruction to the distance measuring device 16, and the distance measuring device 16 starts measuring the mileage of the vehicle 10. If the mileage of the inspection section has already been measured, continue to measure the distance and record the position of the mileage to start the measurement. In step 940, the distance acquisition unit 47 acquires the mileage measured from the distance measuring device 16. The situation when the reference subject 63 is detected is the position of the road surface camera 12 installed in the vehicle 10 as shown in FIG. 10 (e), specifically, the central position of the imaging range of the road surface camera 12 and the reference subject 63. , The distance to the nearest end facing the vehicle 10 is the distance [D_2]. Therefore, the current mileage value [D_C2] is set to 0, and the mileage from there until the distance [D_2] is reached is measured.

In step 945, the position estimation unit 48 confirms whether or not the distance has reached a predetermined distance based on the acquired mileage. That is, it is confirmed whether or not the distance [D_2] shown in FIG. 10 (e) has been reached. If it has not been reached, the process returns to step 940, and if it has been reached, the process proceeds to step 950. When the distance [D_2] is reached, the position of the road surface camera 12 installed on the vehicle 10 as shown in FIG. 10 (f), specifically the central position of the imaging range of the road surface camera 12, and the above-mentioned reference subject 63. The end is at a position that coincides with the direction of travel.

In step 950, the end point of the inspection section is estimated from the position where the designated distance is reached. Here, the position where the specified distance is reached is estimated as the end point of the inspection section. Then, in step 955, the distance value and the time of the mileage measured up to the position where the designated distance is reached are recorded, and the end point of the inspection section is set. In addition, the captured image at that time is recorded as the last imaging frame of this inspection section.

When the end point of the inspection section is detected, the instruction unit 46 instructs the peripheral camera 13 to end the imaging, and the peripheral camera 13 returns to the standby state. Further, the instruction unit 46 instructs the distance measuring device 16 to end the measurement of the mileage, and the distance measuring device 16 returns to the standby state. Further, the instruction unit 46 instructs the road surface camera 12 to end the imaging, and the road surface camera 12 returns to the standby state. When these have returned to the standby state, the process of estimating the end point ends in step 960.

Here, the outline information that should be measured and set in advance will be described. As the summary information, there is GPS position information of intersections 60 and 62 as target objects. GPS position information is position information with low accuracy because the latitude / longitude detection accuracy is as low as several m to 10 m. Further, as the summary information, there is also information on the distance between the start point and the end point, but since this information is also GPS position information, it is distance information with low accuracy.

As the summary information, there is a distance [D_CS] between the reference subject 61 shown in FIG. 11 and the peripheral camera 13. The distance [D_CS] is the above distance [D_1]. The same applies to the reference subject 63, in which case the distance [D_CS] is the above-mentioned distance [D_2]. The distance [D_CS] is between the position of the image sensor of the peripheral camera 13, specifically the peripheral camera 13, and the position of the reference subject 61 when even a part of the reference subject 61 is within the imaging range of the peripheral camera 13. It can be determined as a distance. The distance is not limited to this, and may be determined as the distance when the entire reference subject 61 enters.

Further, as the outline information, there is also a distance [D_CC] as a distance between the position of the image sensor of the peripheral camera 13 and the central position of the image pickup range of the road surface camera 12. The central position of the imaging range is a position on the road surface 11 which is the central position of the captured image. From the measurement of these distances, the distance between the position of the reference subject 61 and the center position of the imaging range of the road surface camera 12 can be added, and the distance can be obtained ^{as the distance [D_ *].} In the case of the reference subject 63, the distance [D_ ^* ] can be obtained in the same manner.

By setting these summary information in advance and executing the processes shown in FIGS. 7 and 9, the position accuracy of the start point and the end point of the inspection section can be adjusted from several meters to 0.05 m of GPS information. It can be improved to ~ 0.1 m.

Reference subjects 61 and 63 will be described with reference to FIG. Reference subjects 61 and 63 include a traffic light and a sign pole 70 in FIG. 12 (a), a curb 71 in FIG. 12 (b), a white line stop line 72 in FIG. 12 (c), and a pedestrian crossing line 73 in FIG. 12 (d). The manhole 74 of FIG. 12 (e), the intersection center of FIG. 12 (f), and the like can be adopted. Note that the road markings such as the intersection Chuose 75 in FIG. 12 (f) may disappear, so that they are unlikely to be damaged, moved, or disappear over time, as shown in FIGS. 12 (a) to 12 (e). Is desirable. In FIGS. 12 (a) to 12 (e), the order of priority that it is desirable to use as the reference subjects 61 and 63 is the traffic light, the sign pole 70, the manhole 74, and the curb in the order of low possibility of damage, movement, or disappearance. It can be 71, a white line stop line 72, and a pedestrian crossing line 73.

It is desirable to pay attention to the following points when measuring the mileage by adopting these reference subjects 61 and 63. That is, in order to ensure the measurement accuracy of the mileage, in the traffic light or the sign pole 70, the end of the curved surface facing the vehicle 10 on the side closest to the vehicle 10 or the end of the curved surface farthest from the vehicle 10 in the traveling direction. It is desirable to use the part as a reference. For the curb 71, the white line stop line 72, the pedestrian crossing line 73, and the manhole 74, it is desirable to refer to the end on the side closest to the vehicle 10 or the end on the side farthest from the vehicle 10 in the traveling direction.

Further, the distance measuring device 16 starts the measurement, and the position where the specified distance is reached is estimated as the start point or the end point of the inspection section, but the measurement is sufficiently close to the reference subjects 61 and 63. It is desirable to start from the state where it is. That is, it is desirable that the specified distance is short. This is because the longer the distance, the larger the measurement error by the distance measuring device 16. The distance [D_CS] shown in FIG. 12 is preferably less than, for example, 3 m.

Depending on the inspection section, there may be two or more subjects. In that case, the subject can be selected according to the above priority. The reference subjects 61 and 63 may be on the front side of the intersections 60 and 62, may be on the back side, and may be inside the intersections 60 and 62. In the example shown in FIG. 12, the white line stop line 72 of FIG. 12 (c) is on the front side, the intersection center of FIG. 12 (f) is inside the intersections 60 and 62, and the others are on the back side. ..

If the subject as shown in FIG. 12 does not exist, or if the subject is hidden by another vehicle and the detection accuracy of the subject is not sufficient, the intersection shown in FIGS. 13 (a) and 13 (b). The road surface edge 76 of 60 may be adopted as the reference subject 61. The road surface end 76 is not a road surface 11a extending in the traveling direction, but a road surface end portion on either the front side or the back side of the road surface 11b extending in a direction substantially perpendicular to the road surface 11a. In FIGS. 13A and 13B, the road surface end 76 on the back side is adopted. Incidentally, FIG. 13 (a) is an image taken, and FIG. 13 (b) is a view when the road surface 11 is viewed from the sky. Similarly, the road surface edge 76 can be adopted as the reference subject 63.

Until now, the start and end points of the inspection section were set to the positions of the reference subjects 61 and 63, specifically, the ends of the reference subjects 61 and 63 on the side closer to the vehicle 10, and were set to the same positions as the ends. The position is estimated. However, the start point and the end point may be different from the positions of the reference subjects 61 and 63. For example, a position separated from the positions of the reference subjects 61 and 63 by a predetermined distance is set as the start point and the end point. It is also possible.

This is because the distance from the road surface end 76 on the back side of the intersection 60 to the end on the side of the reference subject 61 closest to the vehicle 10 as shown in FIG. 14A exceeds 3 m, or the detection accuracy is high. It is effective when it is not sufficiently obtained (when the accuracy is larger than 0.1 m).

In this case, for example, when the intersection detection unit 50 detects the intersection 60, the instruction unit 46 instructs the distance measuring device 16 to start the distance measurement, and instructs the road surface camera 12 to take an image of the road surface 11. To start. After that, when the object detection unit 45 detects the reference subject 61, as shown in FIG. 14 (b), the distance [D_SS] set as the summary information is used, and the road surface edge 76 at the intersection 60 is arbitrarily less than 3 m. The position of is estimated as a virtual start point, and that position can be set as the start point. Similarly, the end point can be estimated using the distance [D_SE] set as the summary information, and the position can be set as the end point. As long as it is less than 3 m from the road surface edge 76, it may be any distance such as 1 m or 2 m from the road surface edge 76.

As described above, the start point and end point of the inspection section can be estimated, and the distance from the start point to the end point can be set as the section length. The section length is from the distance from the start point to the end point measured by the distance acquisition device 47 separately provided, or from the measurement start position to the start point and the end point measured by the distance measurement device 16. It can be obtained by calculating using the mileage information.

The section length of the inspection section changes depending on the width and shape of the road and the position where the vehicle 10 travels. Therefore, it is desirable to set the section length in consideration of these conditions.

FIG. 15 is a flowchart showing an example of a process of estimating the section length in consideration of the above conditions. After setting the start point of the inspection section by estimating and recording it by the above method, the process starts from step 1500. In step 1505, the peripheral image acquisition unit 44 acquires an image captured from the peripheral camera 13 so as to include one end to the other end in the width direction of the roadway. Here, the peripheral camera 13 is used as a width measuring device for measuring the width of the road surface 11, and the peripheral image acquisition unit 44 is used as a width information acquisition unit for acquiring the width information of the road surface 11, but the present invention is not limited to this. Alternatively, a width measuring device and a width information acquisition unit may be provided separately.

In step 1510, the separately provided interval estimation unit estimates the width of the road including the oncoming lane and the position of the center line (center position of the width of the road) based on the acquired image. In addition, the interval estimation unit estimates the interval between the traveling position and the center line based on the acquired image. For example, the width of the road, the position of the center line, and the interval can be calculated from the number of pixels constituting the image. The position estimation unit 48 may be used to execute these estimation processes.

The road surface inspection system further includes a radius of curvature measuring device for measuring the radius of curvature of the traveling path in the vehicle 10. In step 1515, the indicating unit 46 gives an instruction to the radius of curvature measuring device, and the radius of curvature measuring device measures the radius of curvature of the traveling path. As the radius of curvature measuring device, a measuring instrument such as a GPS sensor 15 can be used. Then, a separately provided radius of curvature acquisition unit acquires the measured radius of curvature information (value of the radius of curvature).

Here, the radius of curvature of the traveling path is described as being measured by the radius of curvature measuring device, but the present invention is not limited to this, and the road surface property measurement data obtained based on the image captured by the road surface camera 12 is used. You may ask. Specifically, it can be estimated from the curvature of the three-dimensional data synthetically connected in the traveling direction of the road surface 11 by using the road surface property measurement data.

In step 1520, the section length of the center line is estimated using the distance value between the traveling position and the center line estimated in step 1510 and the value of the radius of curvature measured in step 1515. This section length can be estimated by a separately provided section length estimation unit. When this calculation is completed, the process proceeds to step 1525 and the process of estimating the section length is completed. By such processing, it is possible to set the section length with high accuracy even for the road width and the curve condition.

After setting the section length, the crack rate etc. can be calculated and the form with them can be output, but if the section length cannot be calculated correctly or the image of the road surface 11 cannot be acquired correctly, the correct section The evaluation result cannot be output. Therefore, it is desirable to judge the quality of the estimation results of the start point and the end point used for calculating the section length after setting the section length and before outputting the section evaluation result. This pass / fail determination process will be described with reference to FIG.

After setting the section length, this pass / fail determination process is started from step 1600. In step 1605, it is determined whether or not the object detection unit 45 can detect the intersection in the image captured by the peripheral camera 13. This determination process can be performed by a determination unit provided separately. If the intersection can be detected, the process proceeds to step 1610, and if the intersection cannot be detected, the process proceeds to step 1630.

As an example where the intersection cannot be detected, there is an obstacle such as another vehicle on the roadside, and it is necessary to meander to avoid the obstacle. There is a case where the image is not completely captured. That is, there is a defect in the image portion necessary for determining the intersection.

From this, whether or not the intersection can be detected can be determined by whether or not the defect ratio is equal to or less than the threshold value. The defect ratio is determined by measuring in advance the area of the road surface portion extending to the left and right of the intersection when there is no obstacle, and hiding the area with an obstacle in the road surface portion in the image captured by the peripheral camera 13. It can be calculated as the ratio of the area of the divided parts.

In step 1610, it is determined whether or not the object detection unit 45 can detect the reference subject in the image captured by the peripheral camera 13. Reference subjects are traffic lights, manholes, curbs, pedestrian crossings, white line stop lines, and the like. If the reference subject can be detected, the process proceeds to step 1615, and if the reference subject cannot be detected, the process proceeds to step 1630.

Examples of cases where the reference subject cannot be detected include the case where another vehicle is parked on the roadside or the case where a foreign object is present on the road and covers a part of the reference subject, as in the case of an intersection. Can be mentioned. Similar to the intersection, whether or not the reference subject can be detected can be determined by whether or not the defect ratio is equal to or less than the threshold value.

In addition to detecting intersections and reference subjects, the object detection unit 45 can detect obstacles such as other vehicles parked on the roadside, foreign substances on the road, puddles, and snow cover. Examples of foreign substances on the road include fallen leaves, earth and sand, and falling objects.

In step 1615, it is determined whether the object detection unit 45 has detected a foreign object, an obstacle, or the like on the road in the image captured by the road surface camera 12. When detected, the ratio of the area of foreign matter or obstacles covering the road surface 11 (the area of the area where the evaluation is invalid (invalid area)) to the area of the road surface 11 of the section length set in step 1620 is the threshold value. Determine if it is less than or equal to: If it is determined that the value is equal to or less than the threshold value, the process proceeds to step 1625, the determination is made as good, and the good / bad determination process ends in step 1635. Similarly, when no foreign matter or the like is detected in step 1615, the process proceeds to step 1625 and the determination is good. If it is determined to be good, the crack rate and the like can be obtained from the set section length, and the section evaluation result can be output.

On the other hand, if it is determined that the detection has not been performed in steps 1605 and 1610, and if it is determined that the threshold value is exceeded in step 1620, it is determined in step 1630 that the quality is not detected, and the quality determination process is terminated in step 1635. If it is determined to be negative, the section evaluation will not be performed.

Each of the above threshold values may have a different value or may have the same value. Each threshold value can be set to, for example, the same value as the allowable threshold value, and can be set to 3/100 (3%). This enables automatic judgment without human intervention.

By having a function to perform such a pass / fail judgment process, it is possible to minimize rework in the evaluation work, such as making a judgment at the same time as imaging, or immediately after a few seconds or minutes, and re-running. ..

So far, the present invention has been described with the above-described embodiments as an information processing system, an evaluation system, an information processing method, and a program. However, the present invention is not limited to the above-described embodiment, and can be modified within the range conceivable by those skilled in the art, such as other embodiments, additions, changes, and deletions. .. Further, any aspect is included in the scope of the present invention as long as the action / effect of the present invention is exhibited.

Therefore, a recording medium on which the program is recorded, an external device that provides the program, and the like can also be provided.

10 ... Vehicle, 11, 11a, 11b ... Road surface, 12 ... Road surface camera, 13 ... Peripheral camera, 14 ... Information processing device, 15 ... GPS sensor, 16 ... Distance measuring device, 20, 21 ... Camera, 22, 25 ... Lens , 23, 26 ... Image sensor, 24, 27 ... Sensor controller, 30 ... CPU, 31 ... ROM, 32 ... RAM, 33 ... HDD, 34 ... Communication I / F, 35 ... Input / output I / F, 36 ... Display device , 37 ... Input device, 38 ... Bus, 40 ... Road surface image acquisition unit, 41 ... Storage unit, 42 ... Measurement unit, 43 ... Evaluation unit, 44 ... Peripheral image acquisition unit, 45 ... Object detection unit, 46 ... Indicator unit, 47 ... Distance acquisition unit, 48 ... Position estimation unit, 49 ... Position information acquisition unit, 50 ... Intersection detection unit, 60, 62 ... Intersection, 61, 63 ... Reference subject, 70 ... Signals and sign poles, 71 ... Edge stone, 72 ... white line stop line, 73 ... crossing pedestrian line, 74 ... manhole, 75 ... intersection Chuose, 76 ... roadside

Japanese Unexamined Patent Publication No. 2016-045063

Claims (10)

An information processing system that estimates the evaluation section to be evaluated.
An image acquisition means for acquiring an image including the evaluation target captured by an image pickup device mounted on a vehicle traveling on the evaluation target, and an image acquisition means.
A detection means for detecting a reference object having a specified feature by referring to the image acquired by the image acquisition means, and
Upon receiving the detection of the reference object by the detection means, the instruction means for instructing the distance measuring device mounted on the vehicle to measure the mileage of the vehicle, and
A distance acquisition means for acquiring the mileage measured by the distance measuring device, and
With respect to one end and the other end of the evaluation section , based on the position where each mileage after receiving an instruction from the instruction means acquired by the distance acquisition means reaches a designated distance, the one end of the evaluation section and the other end. An information processing system including a position estimating means for estimating the position of the other end. A position information acquisition means for acquiring the position information of the vehicle measured by the position measurement device mounted on the vehicle, and
Including a second detecting means for detecting a designated target object based on the position information acquired by the position information acquiring means.
The information processing system according to claim 1, wherein the instruction means instructs the image pickup apparatus to start imaging in response to the detection of the target object by the second detection means. The position estimating means sets the position reached by each mileage acquired by the distance acquiring means as the position of one end and the other end of the evaluation section, or the mileage acquired by the distance acquiring means. The reached position is one of the position of one end and the position of the other end of the evaluation section, and the position separated by a certain distance from the position where the other mileage is reached is the position of one end and the position of the other end of the evaluation section. The information processing system according to claim 2, wherein the position separated by a certain distance from the position reached by each mileage acquired by the distance acquisition means is estimated as the position of one end and the position of the other end. Width information acquisition means for acquiring width information measured by the width measuring device for measuring the width of the evaluation target, and
A radius of curvature acquisition means for acquiring the radius of curvature measured by the radius of curvature measuring device for measuring the radius of curvature of the evaluation target, and
Using the width information measured by the width information acquisition means and the radius of curvature measured by the radius of curvature acquisition means, the position of one end and the position of the other end of the evaluation section estimated by the position estimation means. The information processing system according to any one of claims 1 to 3, further comprising a section length estimating means for estimating the section length determined by. The vehicle is equipped with a second imaging device that images the evaluation target.
The information processing system
A second image acquisition means for acquiring an image captured by the second image pickup device, and
Using the image acquired by the second image acquisition means, a measuring means for measuring the surface texture of the evaluation target in the evaluation section, and a measuring means.
The information processing system according to any one of claims 1 to 4, which includes an evaluation means for evaluating the evaluation target in the evaluation section based on the surface texture measured by the measuring means. The invention according to any one of claims 1 to 5, further comprising a determination means for determining whether or not the detection means can detect the reference object based on the image acquired by the image acquisition means. Information processing system. The information processing system according to claim 2, further comprising a determination means for determining whether or not the detection means can detect the reference object and the target object based on the image acquired by the image acquisition means. .. A first imaging device mounted on a vehicle traveling on an evaluation target, a distance measuring device mounted on the vehicle and measuring the mileage of the vehicle, and an information processing system for estimating an evaluation section of the evaluation target. An evaluation system including a second imaging device that images the evaluation target.
The information processing system
A first image acquisition means for acquiring an image including the evaluation target captured by the first image pickup apparatus, and
A detection means for detecting a reference object having a specified feature by referring to the image acquired by the first image acquisition means, and
Upon receiving the detection of the reference object by the detection means, the instruction means for instructing the distance measuring device to measure the mileage of the vehicle, and
A distance acquisition means for acquiring the mileage measured by the distance measuring device, and
With respect to one end and the other end of the evaluation section , based on the position where each mileage after receiving an instruction from the instruction means acquired by the distance acquisition means reaches a designated distance, the one end of the evaluation section and the other end. A position estimating means for estimating the position of the other end,
A second image acquisition means for acquiring an image captured by the second image pickup device, and
Using the image acquired by the second image acquisition means, a measuring means for measuring the surface texture of the evaluation target in the evaluation section, and a measuring means.
An evaluation system including an evaluation means for evaluating the evaluation target in the evaluation section based on the surface texture measured by the measuring means. An information processing method that estimates the evaluation section to be evaluated.
A step of acquiring an image of the evaluation target captured by an imaging device mounted on a vehicle traveling on the evaluation target, and a step of acquiring the image of the evaluation target.
A step of detecting a reference object having a specified feature by referring to the acquired image, and
In response to the detection of the reference object, a step of instructing the distance measuring device mounted on the vehicle to measure the mileage of the vehicle, and
A step of acquiring the mileage measured by the distance measuring device, and
The positions of one end and the other end of the evaluation section are estimated based on the positions where each mileage after receiving the measurement instruction acquired for one end and the other end of the evaluation section reaches the specified distance. Information processing methods, including steps. A program for causing a computer to execute a process of estimating an evaluation interval to be evaluated.
A step of acquiring an image including the evaluation target captured by an imaging device mounted on a vehicle traveling on the evaluation target, and a step of acquiring the image including the evaluation target.
A step of detecting a reference object having a specified feature by referring to the acquired image, and
In response to the detection of the reference object, a step of instructing the distance measuring device mounted on the vehicle to measure the mileage of the vehicle, and
A step of acquiring the mileage measured by the distance measuring device, and
The positions of one end and the other end of the evaluation section are estimated based on the positions where each mileage after receiving the measurement instruction acquired for one end and the other end of the evaluation section reaches the specified distance. A program that executes steps and.

Images