JP7160372B2 - Infrared automatic thermal analysis and diagnosis method, information processing device, infrared automatic thermal analysis and diagnosis system, and program - Google Patents

Description

TECHNICAL FIELD The present invention relates to an infrared automatic thermal analysis and diagnosis method, an information processing apparatus, an infrared automatic thermal analysis and diagnosis system, and a program, and more particularly to diagnosis of soundness of paved roads using infrared thermal images.

As one of the methods for diagnosing the soundness of paved roads such as expressways, an infrared thermal image of the paved road is taken with an infrared thermal camera installed on a traveling vehicle, and the infrared thermal image is analyzed to detect damage to the paved road. There is a method for diagnosing the presence or absence of For example, Patent Literature 1 discloses a road surface defect detection system and method for identifying types of defects on a detected road surface by analyzing an infrared thermal image of the detected road surface.

In this type of diagnostic method, the presence, type, degree (damage level), etc. of damage are diagnosed using the temperature difference that occurs between a healthy portion and a damaged portion of a paved road. Therefore, it is important to derive the temperature of the healthy portion of the paved road in the infrared thermal image with high accuracy.

JP 2012-177675 A

However, since the temperature of the paved road changes depending on the weather, the presence or absence of surrounding structures, etc., it may be difficult to derive the temperature of the healthy portion of the paved road in the infrared thermal image with high accuracy. In addition, when an analyst observes an infrared thermal image and makes a diagnosis, there are individual differences in the criteria for estimating the temperature of a healthy portion and determining the presence or absence of damage. For this reason, it is difficult to improve the objectivity, quantification, and consistency of results in the conventional method of diagnosing the soundness of paved roads using infrared thermal images.

In one aspect, the present invention aims to provide a technology capable of improving the objectivity, quantification, and result consistency of paved road health diagnosis using infrared thermal images.

In the automatic infrared thermal analysis and diagnosis method according to one aspect, a computer analyzes a plurality of infrared thermal images continuously photographed from different areas of a paved road on which the vehicle is traveling by an infrared thermocamera attached to the vehicle. In the infrared automatic thermal analysis diagnosis method for diagnosing the soundness of the paved road, the computer sets an analysis target area in the infrared thermal image for each infrared thermal image, and a plurality of analysis target areas are set in the infrared thermal image. blocks, and for each block, the temperature of the healthy portion of the paved road is estimated based on the temperature distribution in the block to derive the estimated healthy temperature in the block, and the plurality of blocks derived for each block An in-image estimated healthy temperature is derived by estimating the temperature of a healthy portion in an area corresponding to the analysis target area of the paved road on which the vehicle travels, based on the in-image estimated healthy temperature.

An information processing device according to one aspect analyzes a plurality of infrared thermal images continuously photographed from different areas of a paved road on which the vehicle is traveling by an infrared thermocamera attached to the traveling vehicle, and analyzes the paved road. An information processing apparatus for diagnosing soundness, for each infrared thermal image, an analysis target area is set in the infrared thermal image, the analysis target area is divided into a plurality of blocks, and each block is divided into a plurality of blocks. an intra-block information deriving unit that estimates the temperature of a healthy portion of the paved road based on the temperature distribution of the block and derives an estimated healthy temperature in the block; a healthy temperature estimating unit for estimating the temperature of a healthy portion in an area corresponding to the analysis target area of the paved road on which the vehicle travels, based on the estimated healthy temperature, and deriving an in-image estimated healthy temperature; Prepare.

An infrared automatic thermal analysis and diagnosis system according to one aspect comprises an infrared thermo camera and a plurality of infrared thermographs continuously photographing different areas of a paved road on which the vehicle is traveling by means of the infrared thermo camera attached to the traveling vehicle. and an information processing device that analyzes an image and diagnoses the soundness of the paved road, wherein the information processing device sets an analysis target area in the infrared thermal image for each infrared thermal image, and determines the analysis target area. an in-block information derivation unit that divides the inside into a plurality of blocks, estimates the temperature of a healthy portion of the paved road for each block based on the temperature distribution in the block, and derives an estimated healthy temperature in the block; For each infrared thermal image, the temperature of a healthy portion in the area corresponding to the analysis target area of the paved road on which the vehicle travels is calculated based on the plurality of estimated healthy temperatures in the block derived for each block. a healthy temperature estimator for estimating and deriving an in-image estimated healthy temperature.

A program according to one aspect, in a computer, analyzes a plurality of infrared thermal images continuously photographed by an infrared thermocamera attached to a traveling vehicle and different areas of a paved road on which the vehicle travels, and analyzes the paved road. A program for executing a process of diagnosing the soundness of an infrared thermal image, for each infrared thermal image, setting an analysis target area in the infrared thermal image, dividing the analysis target area into a plurality of blocks, and for each block, Based on the temperature distribution in the block, the temperature of the healthy portion of the paved road is estimated to derive an estimated healthy temperature in the block, and based on the plurality of estimated healthy temperatures in the block derived for each block, the vehicle estimating the temperature of a healthy part in an area corresponding to the analysis target area of the paved road traveled by, and deriving an in-image estimated healthy temperature.

According to the above-described aspect, it is possible to improve the objectivity, quantification, and result consistency of pavement soundness diagnosis using infrared thermal images.

BRIEF DESCRIPTION OF THE DRAWINGS It is a figure explaining the structural example of the infrared automatic thermal-analysis diagnostic system which concerns on one Embodiment. 1 is a block diagram showing the functional configuration of an information processing device according to an embodiment; FIG. 4 is a flow chart illustrating an infrared automatic thermal analysis diagnostic method according to one embodiment. FIG. 11 is a flowchart for explaining an example of intra-block information derivation processing; FIG. 7 is a flowchart illustrating an example of an in-image estimated healthy temperature derivation process; 4 is a flowchart illustrating an example of damage level evaluation processing; 6 is a flowchart illustrating an example of point cloud coordinate derivation processing; It is a figure containing the photograph for demonstrating an example of an infrared thermal image. FIG. 4 is a diagram including a photograph for explaining an example of a plurality of blocks set within an infrared thermal image; It is a figure explaining an example of the information in a block. It is a figure explaining an example of the calculation method of the in-screen estimated healthy temperature. It is a figure explaining an example of the display method of the evaluation result of a damage level. FIG. 3 is a diagram for explaining the correspondence relationship between an infrared thermal image capturing area on an actual paved road and a position in the infrared thermal image; It is a figure explaining an example of the relationship between the moving direction of a vehicle, and a reference direction. It is a figure explaining an example of the derivation|leading-out method of a point group coordinate. It is a figure which shows an example of point group coordinates. It is a figure explaining the hardware configuration example of a computer.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the following description, detailed description of well-known image processing that can be applied to analysis and diagnosis of infrared thermal images will be omitted.

FIG. 1 is a diagram illustrating a configuration example of an infrared automatic thermal analysis and diagnosis system according to one embodiment. FIG. 1 shows an example of an infrared automatic thermal analysis/diagnosis system 1 for diagnosing the soundness of a paved road 9 on which a vehicle 5 can run, using an infrared thermal image.

An infrared automatic thermal analysis and diagnosis system 1 illustrated in FIG. 1 includes an infrared thermo camera 2, a GPS (Global Positioning System) receiver 3, and an information processing device 4. The infrared thermocamera 2, the GPS receiver 3, and the information processing device 4 are installed in a vehicle 5 used for diagnosis during diagnosis. In some cases, the information processing device 4 is brought into the room from the vehicle 5 and used for diagnosis.

The infrared thermo camera 2 is a camera that detects infrared rays radiated from a subject and captures an infrared thermal image. The infrared thermocamera 2 is installed above the vehicle 5 with a mounting bracket 6 so as to capture an infrared thermal image within a predetermined imaging region R ahead of the vehicle 5 on the paved road (road surface) 9 to be diagnosed. be done. When the extension direction of the imaging center VC of the infrared thermocamera 2 has a significant inclination with respect to the normal direction of the paved road 9, the imaging region R extends from the end RN closest to the vehicle 5 toward the farthest end RF. It becomes a substantially trapezoidal (more specifically, substantially isosceles trapezoidal) region in which the dimension in the width direction (not shown) monotonically increases. The angle of view φ of the infrared thermocamera 2, the distance LG from the vehicle 5 to the imaging area R on the paved road 9, the area of the imaging area R (in other words, the widthwise dimension of the edge RN closest to the vehicle 5, the road Dimension LR in the stretching direction) and the like are arbitrary.

The GPS receiver 3 is a position detection device that receives radio waves emitted by GPS satellites and derives the position (latitude and longitude) of the GPS receiver 3 on the earth. The GPS receiver 3 may be built in the infrared thermocamera 2, for example. Further, as the GPS receiver 3 of the infrared automatic thermal analysis and diagnosis system 1, for example, a GPS receiver included in a car navigation system installed in the passenger compartment of the vehicle 5 may be used. The GPS receiver 3 acquires position information of the vehicle 5 (infrared thermocamera 2) when an infrared thermal image of the vehicle 5 traveling on the paved road 9 to be diagnosed is captured by the infrared thermocamera 2. 1 is an example of an acquisition device;

The information processing device 4 acquires and analyzes an infrared thermal image captured by the infrared thermocamera 2, and automatically performs a process of diagnosing the soundness of the paved road 9 to be diagnosed. In addition, the information processing device 4 acquires position information (latitude and longitude) derived by the GPS receiver 3, and automatically performs processing for calculating the latitude and longitude of each position on the paved road 9 in the infrared thermal image for each pixel. do in The infrared thermocamera 2 and the GPS receiver 3 are communicably connected to the information processing device 4 by wireless communication or a transmission cable such as a USB (Universal Serial Bus) cable. The information processing device 4 may be, for example, a portable device that can store infrared thermal images and position information and then be taken out of the vehicle 5 to perform the above-described analysis, diagnosis, and other processes.

When diagnosing the soundness of the paved road 9 by the infrared automatic thermal analysis diagnosis system 1 installed in the vehicle 5, the infrared heat captured by the infrared thermo camera 2 while the vehicle 5 is traveling on the paved road 9 to be diagnosed. The image and the positional information of the vehicle 5 at the time of photographing are periodically transferred to the information processing device 4 . At this time, the vehicle 5 is configured so that the entire paved road 9 can be diagnosed by connecting the partial areas of the paved road 9 that are the targets of diagnosis in each of the plurality of infrared thermal images transferred to the information processing device 4 . and the imaging interval of the infrared thermo camera 2 are set. The information processing device 4 analyzes the acquired infrared thermal image and diagnoses the soundness of the paved road 9 . The information processing device 4 also calculates the latitude and longitude of each position on the paved road 9 in the infrared thermal image for each pixel based on the acquired position information of the vehicle 5 .

FIG. 2 is a block diagram illustrating the functional configuration of the information processing apparatus according to one embodiment;

The information processing device 4 of the infrared automatic thermal analysis and diagnosis system 1 according to this embodiment includes, for example, a transmission/reception unit 400, a control unit 410, an input unit 420, a display unit 430, and a storage unit 440, as shown in FIG. .

The transmission/reception unit 400 transmits and receives various information between the information processing device 4 and an external device connected to the information processing device 4 . The transmitting/receiving unit 400 receives infrared thermal images transferred from the infrared thermocamera 2 to the information processing device 4 and position information transferred from the GPS receiver 3 to the information processing device 4 . Further, the transmitting/receiving unit 400 can transmit an infrared thermal image, position information, analysis diagnosis processing results, etc. to a server device (not shown) or the like connected to the information processing device 4 via a network, for example. can.

The control unit 410 performs various types of processing including processing for controlling the operation of the information processing device 4 . The control unit 410 analyzes the infrared thermal image and performs processing for diagnosing the soundness of the paved road 9 . Based on the position information of the vehicle 5, the control unit 410 also performs a process of calculating the position (latitude and longitude) on the earth of each position of the paved road 9 in the infrared thermal image for each pixel. A control unit 410 capable of performing these processes includes an intra-block information deriving unit 411 , a healthy temperature estimating unit 412 , a damage level evaluating unit 413 , and a point cloud coordinate deriving unit 414 .

The intra-block information derivation unit 411 sets a plurality of blocks in one infrared thermal image, and derives various information regarding the temperature of the paved road 9 for each block based on the temperature distribution within the block. The intra-block information deriving unit 411, for example, targets a predetermined partial area of the entire area of the paved road 9 in the infrared thermal image for analysis diagnosis, and divides the partial area into a plurality of blocks. The in-block information deriving unit 411, for example, calculates the temperature of the healthy portion of the paved road 9 estimated based on the temperature distribution in the block (hereinafter referred to as “estimated healthy temperature in the block”), the maximum temperature, the minimum temperature, the standard deviation, and derive the standard error. The in-block estimated healthy temperature is, for example, the average temperature of the paved road 9 calculated based on the temperature distribution in the block.

A healthy temperature estimating unit 412 estimates the temperature of a healthy portion of the paved road (road surface) 9 in the infrared thermal image based on the intra-block information derived from each of a plurality of blocks set in the infrared thermal image. A value (hereinafter referred to as “in-image estimated healthy temperature”) is calculated. The healthy temperature estimating unit 412 selects the intra-block information used for calculating the estimated intra-image healthy temperature based on the intra-block information of each block, for example.

The damage level evaluation unit 413 evaluates the damage level of the portion of the paved road 9 inside the block for each block based on the estimated healthy temperature in the image of the infrared thermal image and the block information of each block in the infrared thermal image. evaluate. The damage level evaluation unit 413 evaluates the damage level based on, for example, the temperature difference between the in-block estimated healthy temperature and the estimated in-image healthy temperature, and the degree of variation in the temperature distribution. For example, standard deviation and standard error are used as the degree of variation in temperature distribution. Further, the damage level evaluation unit 413 evaluates a more detailed damage level (detailed damage level) based on the temperature difference between the in-block estimated healthy temperature and the estimated in-image healthy temperature. For example, the detailed damage level evaluation may target only blocks evaluated as having damage in the damage level evaluation among all blocks.

The point cloud coordinate derivation unit 414 derives point cloud coordinates that associate the pixels of the infrared thermal image with the positions on the paved road 9 based on the position information of the vehicle 5 . The point cloud coordinates are the actual paved road 9 corresponding to each position (each pixel) of the paved road 9 in the infrared thermal image represented by a predetermined coordinate system set in the real space where the paved road 9 exists. is the position of When the point group coordinates are derived based on the position information from the GPS receiver 3 described above, the coordinate system set in the real space is, for example, a geographic coordinate system that represents the position on the paved road 9 with the latitude and longitude of the earth. do. The point cloud coordinate derivation unit 414 calculates, based on the position information of the vehicle 5, for example, the photographing direction of the infrared thermocamera 2 with respect to a predetermined reference direction such as due north at the time of photographing the infrared thermal image. In addition, the point cloud coordinate deriving unit 414 derives the point cloud coordinates based on the angle formed by the reference direction and the imaging direction, and the correspondence relationship between the positions within the imaging range of the infrared thermocamera 2 and the pixels in the infrared thermal image. calculate.

The input unit 420 receives input of various information related to the operation of the information processing device 4 . The display unit 430 visualizes and displays various types of information such as infrared thermal images, position information, analysis and diagnosis results, and the like. The storage unit 440 stores various types of information including infrared thermal images, position information, and analysis and diagnosis results.

The information processing device 4 according to the present embodiment may be a dedicated device combining dedicated hardware for implementing the functions of the respective units (functional blocks) of the control unit 410 illustrated in FIG. It may also be a computer that executes programs for analysis and diagnosis.

FIG. 3 is a flowchart illustrating an infrared automatic thermal analysis diagnostic method according to one embodiment.

When diagnosing the soundness of the paved road 9 using the infrared automatic thermal analysis/diagnosis system 1 of the present embodiment, the vehicle 5 equipped with the infrared automatic thermal analysis/diagnosis system 1 is placed on the paved road 9 as described above. let it run. When the infrared thermocamera 2, the GPS receiver 3, and the information processing device 4 are turned on, for example, the infrared thermal image transferred from the infrared thermocamera 2 and the GPS receiver 3 are displayed on the display unit 430 of the information processing device 4. The location information transferred from is displayed. Further, for example, when an input to start diagnosis is received by the input unit 420, the information processing device 4 starts accumulating infrared thermal images and position information (step S1), and analyzes and diagnoses each acquired infrared thermal image (step SL1) is performed. At this time, the vehicle 5 is driven at a substantially constant speed, and the information processing device 4 receives the infrared thermal image and the position data at time intervals of driving a predetermined distance (for example, 5 m) derived based on the speed of the vehicle 5 . Acquire and store information. In the accumulation of the infrared thermal image and the positional information, for example, before step S1, the conditions for ending the accumulation are set according to the section to be diagnosed or the number of photographed infrared thermal images (not shown in FIG. 3). Terminate when the set conditions are met. Further, the accumulation of the infrared thermal image and the position information may be terminated, for example, when the input unit 420 receives an accumulation end input while the analysis diagnosis processing (step SL1) is being performed.

The analysis/diagnosis process (step SL1) is a loop process that includes steps S2 to S6 and performs the steps S2 to S6 for each infrared thermal image.

In the analysis diagnosis process, the information processing device 4 first performs an intra-block information derivation process (step S2). The intra-block information deriving process is performed by the intra-block information deriving section 411 of the control section 410 . The intra-block information derivation unit 411 divides the area to be diagnosed in the n-th infrared thermal image, which is the target of analytical diagnosis, into a plurality of blocks, and for each block, various information about the temperature distribution in the block. (intra-block information) is derived. The information derived by the intra-block information derivation unit 411 includes an intra-block estimated healthy temperature calculated based on the intra-block temperature distribution and information indicating the degree of temperature variation within the block. The intra-block information deriving unit 411 calculates, for example, the average temperature in the block as the estimated healthy temperature in the block, and calculates the maximum temperature, the minimum temperature, the standard deviation, and the standard error as information indicating the degree of temperature variation. A specific example of the intra-block information derivation process will be described later with reference to FIG.

Next, the information processing device 4 performs an in-image estimated healthy temperature derivation process (step S3). The in-image estimated healthy temperature derivation process is performed by the healthy temperature estimation unit 412 of the control unit 410 . The healthy temperature estimator 412 derives the estimated healthy temperature in the image based on the estimated healthy temperature in each block and the information indicating the degree of temperature variation. A specific example of the in-image estimated healthy temperature derivation process will be described later with reference to FIG.

Next, the information processing device 4 performs damage level evaluation processing (step S4). The damage level evaluation process is performed by the damage level evaluation section 413 of the control section 410 . The damage level evaluation unit 413 determines the portion of the paved road 9 within the block based on the information indicating the temperature difference between the estimated healthy temperature in the block and the estimated healthy temperature in the image and the degree of temperature variation for each block. assess the damage level of The damage level evaluation unit 413 synthesizes the evaluation result of the damage level with the infrared thermal image and displays it. A specific example of the damage level evaluation process will be described later with reference to FIG.

Next, the information processing device 4 performs point group coordinate derivation processing (step S5). The point cloud coordinate derivation process is performed by the point cloud coordinate derivation unit 414 of the control unit 410 . The point cloud coordinate deriving unit 414 calculates the surface (road surface ), the angle of the photographing direction of the infrared thermocamera 2 at the time of photographing the n-th infrared thermal image with respect to a predetermined reference direction is calculated. In addition, based on the calculated angle, the point cloud coordinate derivation unit 414 determines the position (latitude and longitude ) is calculated for each pixel. A specific example of the point group coordinate derivation process will be described later with reference to FIG.

After completing the processing of steps S2 to S5, the information processing device 4 associates the n-th infrared thermal image and position information with the processing result and stores them in the storage unit 440 (step S6).

When the information processing device 4 performs the processing of steps S2 to S6 for all sets of infrared thermal images and position information used for diagnosis, the analysis diagnosis processing (step SL1) is completed, and the soundness of the paved road 9 is evaluated. End diagnostics. The information processing device 4 may pipeline the processes of steps S2 to S6.

FIG. 4 is a flowchart for explaining an example of an intra-block information derivation process. FIG. 4 shows an example of the contents of processing performed by the control unit 410 (block information derivation unit 411) of the information processing device 4 as the block information derivation processing (step S2) illustrated in FIG.

In the intra-block information derivation process, the intra-block information deriving unit 411 sets a plurality of blocks in the n-th infrared thermal image to be diagnosed (step S21), and for each block, An information derivation process (step SL2) for deriving various kinds of information is performed.

In step S21, the intra-block information deriving unit 411 divides, for example, the diagnosis target area of the paved road 9 in the infrared thermal image into a plurality of blocks. The area to be diagnosed is, for example, a rectangular area of a predetermined size on the actual paved road 9 including part of the lower side in the infrared thermal image (for example, as will be described later with reference to FIGS. 9 and 13). This area corresponds to a rectangular area having dimensions of 5 m and 3 m in the extension direction and width direction of the paved road 9 . The intra-block information deriving unit 411 divides the block into a plurality of blocks so that each block corresponds to a rectangular area of a predetermined size on the actual paved road 9, for example. A rectangular area of the paved road 9 corresponding to one block is, for example, a square area with a side of 0.25 m or 0.5 m.

The information deriving process (step SL2) is a loop process that includes steps S22 to S24 and performs the steps S22 to S24 for each block.

In the information deriving process, the intra-block information derivation unit 411 first derives an in-block estimated healthy temperature based on the temperature distribution in the block to be processed (step S22). The in-block estimated healthy temperature is the temperature of the healthy portion of the paved road 9 estimated based on the temperature distribution in the block, as described above. The in-block information deriving unit 411 derives (calculates), for example, the average temperature in the block as the in-block estimated healthy temperature.

Next, the intra-block information derivation unit 411 derives the maximum temperature, minimum temperature, standard deviation, and standard error within the block (step S23). The intra-block information deriving unit 411 derives (calculates) standard deviation and standard error using a well-known function (computational formula).

After completing the processing of steps S22 and S23, the intra-block information derivation unit 411 stores the derived intra-block information in association with the identification information of the block to be processed (step S24). Block identification information and intra-block information are stored and held in the storage unit 440, for example.

When the intra-block information derivation unit 411 performs the processing of steps S22 to S24 for all the blocks set in the infrared thermal image, it ends the information derivation process (step SL2), and ends the intra-block information derivation process. .

As described above, when the information processing apparatus 4 of the present embodiment analyzes the infrared thermal image and diagnoses the soundness of the paved road 9, first, the diagnostic target area of the paved road 9 in the infrared thermal image is divided into a plurality of blocks, and various information regarding the temperature distribution within the block is derived for each block. The information to be derived includes an estimated value of the temperature of the healthy portion of the paved road 9 (estimated healthy temperature within the block), maximum temperature, minimum temperature, standard deviation, and standard error. The information processing device 4 utilizes the intra-block information derived for each block to obtain an estimated temperature value (image internal estimated healthy temperature).

FIG. 5 is a flowchart for explaining an example of an in-image estimated healthy temperature derivation process. FIG. 5 shows an example of the contents of processing performed by the control unit 410 (healthy temperature estimating unit 412) of the information processing device 4 as the in-image estimated healthy temperature deriving process (step S3) illustrated in FIG.

In the intra-image estimated healthy temperature derivation process, the healthy temperature estimating unit 412 first determines, based on the intra-block information of each block derived by the intra-block information derivation process, the paved road 9 in the n-th infrared thermal image. The temperature of a sound portion (healthy temperature) in the region to be diagnosed is estimated (step S31). The sound temperature estimating unit 412 extracts blocks with high stability (uniformity) with small variations in temperature distribution within a block below a threshold based on, for example, the standard deviation and standard error included in the block information. In the case of , the average value of the estimated healthy temperature inside the block of those blocks, and in the case of one, the estimated healthy temperature inside the block of the block is the healthy temperature of the paved road 9 in the n-th infrared thermal image. Estimated value.

For estimating the healthy temperature, the block information of all the blocks in the infrared thermal image may be used, or only the block information of the blocks satisfying a predetermined condition among the block information of all the blocks may be used. good too. For example, in estimating the healthy temperature, only the intra-block information of blocks whose estimated intra-block healthy temperature (for example, the average temperature in the block) is within a predetermined temperature range may be used. The predetermined temperature range is, for example, the lower limit temperature TL obtained by adding the error temperature ΔT1 (>0) to the lowest average temperature TAmin of the average temperatures of each block, and the error from the highest average temperature TAmax of the average temperatures of each block. It can be between the upper limit temperature TH obtained by subtracting the temperature ΔT2 (>0). The error temperatures ΔT1 and ΔT2 are, for example, ΔT1=ΔT2=0.3°C.

Next, the healthy temperature estimator 412 determines whether or not the in-image estimated healthy temperature of the (n−1)th infrared thermal image is a valid temperature other than indefinite (step S32). In the case of n=1, in other words, the infrared thermal image that is the object of the current in-image estimated healthy temperature derivation process is the first to be processed in the analysis and diagnosis process (step SL1) illustrated in FIG. If it is the selected infrared thermal image, the healthy temperature estimator 412 determines NO in step S32.

If the in-image estimated healthy temperature of the n−1th infrared thermal image is indefinite (step S32; NO), the healthy temperature estimator 412 next calculates the temperature distribution of the blocks associated with the healthy temperature estimated in step S31. is greater than or equal to a threshold value (step S33). The degree of variation is, for example, one or both of standard deviation and standard error. Also, the threshold is arbitrary, and is set based on, for example, the size of the area of the paved road 9 corresponding to one block, the standard deviation and standard error of the temperature on the paved road 9 that is known to be sound, and the like. . If the degree of variation is equal to or greater than the threshold (step S33; YES), the healthy temperature estimator 412 indefinitely determines and holds the in-image estimated healthy temperature of the n-th infrared thermal image (step S34). If there is an in-image estimated healthy temperature of the most recent infrared thermal image before the n-2th image, the in-image estimated healthy temperature is determined and held (step S35), and the in-image estimated End the normal temperature derivation process. If the degree of variation is less than the threshold (step S33; NO), the healthy temperature estimating unit 412 determines and holds the estimated healthy temperature as the in-image estimated healthy temperature of the n-th infrared thermal image (step S36). ), the in-image estimated healthy temperature derivation process ends. By the processing of steps S33 to S36, it is possible to prevent deterioration of the estimation accuracy (reliability) of the in-image estimated healthy temperature when, for example, n=1.

On the other hand, if the in-image estimated healthy temperature of the (n−1)th infrared thermal image is a valid temperature other than indefinite (step S32; YES), the healthy temperature estimator 412 next determines the estimated healthy temperature and the in-image estimated healthy temperature of the (n-1) infrared thermal image is greater than or equal to a threshold value, or whether the degree of variation in temperature distribution is greater than or equal to a threshold value (step S37). The healthy temperature difference threshold is, for example, the temperature difference (eg, 1.5° C.) between the healthy portion and the damaged portion observed when damage of a predetermined damage level occurs. The degree of variation is, for example, one or both of standard deviation and standard error. In addition, the threshold of the degree of variation is arbitrary. set based on If the healthy temperature difference is less than the threshold and the degree of variation in the temperature distribution is less than the threshold (step S37; NO), the healthy temperature estimator 412 calculates the estimated healthy temperature in the n-th infrared thermal image. The in-image estimated healthy temperature is determined and held (step S36), and the in-image estimated healthy temperature derivation process ends. If the difference in healthy temperature is equal to or greater than the threshold or the degree of variation in temperature distribution is equal to or greater than the threshold (step S37; YES), the healthy temperature estimator 412 calculates the estimated healthy temperature in the n−1th infrared thermal image. is determined as the in-image estimated healthy temperature of the n-th infrared thermal image and held (step S38), and the in-image estimated healthy temperature derivation process ends.

In this way, the information processing apparatus 4 of the present embodiment can determine the degree of variation in the temperature distribution of the block associated with the healthy temperature estimated based on the information in the block of the n-th infrared thermal image, and the past temperature distribution in the same diagnostic time. Based on the estimated healthy temperature in the image of the (n-1th or earlier) infrared thermal image, a process of determining the estimated healthy temperature in the image of the n-th infrared thermal image is performed. In this process, if the estimation accuracy (reliability) of the healthy temperature estimated based on the information in the block of the n-th infrared thermal image is low, Let the estimated healthy temperature be the in-image estimated healthy temperature of the n-th infrared thermal image. For this reason, in-image estimation with high estimation accuracy reflects the continuity (uniformity) of the temperature of the healthy portion in the area of the actual paved road 9 corresponding to the paved road 9 in a plurality of continuous infrared thermal images. A healthy temperature can be derived. Also, in the actual paved road 9, the temperature of a portion with one type of damage is higher than that of a healthy portion, and the temperature of a portion with another type of damage is lower than that of a healthy portion. It is known to become For this reason, as described above, by excluding blocks with high or low average temperatures among a plurality of blocks in one infrared thermal image , the temperature information of the damaged portion is excluded and the temperature information in the image is removed. An estimated healthy temperature can be derived. Therefore, it is possible to prevent the estimation accuracy of the in-image estimated healthy temperature from being lowered by using the intra-block information of the block corresponding to the damaged portion.

After deriving the in-image estimated healthy temperature, the information processing device 4 of the present embodiment evaluates the damage level of the paved road 9 for each block based on the in-block information and the in-image estimated healthy temperature.

FIG. 6 is a flowchart illustrating an example of damage level evaluation processing. FIG. 6 shows an example of the contents of processing performed by the control unit 410 (damage level evaluation unit 413) of the information processing device 4 as the damage level evaluation processing (step S4) illustrated in FIG.

In the damage level evaluation process, the damage level evaluation unit 413 performs a block evaluation process (step SL4) for evaluating the damage level of the paved road 9 for each block, and then synthesizes and displays the evaluation result with an infrared thermal image ( step S51). The block evaluation process (step SL4) is a loop process including steps S41 to S50.

In the block evaluation process, the damage level evaluation unit 413 first determines the temperature difference d is calculated (step S41).

Next, based on the calculated temperature difference d and the degree of variation in the temperature distribution within the block, the damage level evaluation unit 413 determines whether the damage level of the portion of the paved road 9 within the block is intact, mild, moderate, or Evaluate whether it is serious (step S42). In step S42, the damage level evaluation unit 413 calculates, for example, a score corresponding to the magnitude of the value based on a predetermined calculation rule for each of the absolute value, standard deviation, and standard error of the temperature difference d. Intact, mild, moderate, and severe are rated based on the total score.

Next, the damage level evaluation unit 413 determines whether the temperature difference d satisfies d≦−b or b<d (step S43). The determination threshold value b (>0) in step S43 is a value that has a magnitude relationship of b>s with the determination threshold value s (>0) in step S45, which will be described later. For example, b=2 to 3 (° C.). . If d≤-b or b<d (step S43; YES), the damage level evaluation unit 413 evaluates the detailed damage level of the paved road 9 portion in the block as severe (step S44).

If -b<d≤b (step S43; NO), the damage level evaluation unit 413 determines whether -b<d≤-s or s<d≤b (step S45). The determination threshold value s (>0) in step S45 is set to 1.0 (° C.), for example. If −b<d≦−s or s<d≦b (step S45; YES), the damage level evaluation unit 413 evaluates the detailed damage level of the portion of the paved road 9 in the block as moderate. (Step S46).

If −s<d≦s (step S45; NO), the damage level evaluation unit 413 determines whether or not the evaluation result of the damage level in step S42 indicates no damage (step S47). If the damage level is not intact (step S47; NO), the damage level evaluation unit 413 evaluates the detailed damage level of the portion of the paved road 9 in the block as light (step S48). If the damage level is intact (step S47; YES), the damage level evaluation unit 413 evaluates the detailed damage level of the portion of the paved road 9 in the block as intact (step S49).

After evaluating the detailed damage level in steps S44, S46, S48, or S49, the damage level evaluation unit 413 stores the evaluation results (damage level and detailed damage level) in association with the block identification information (step S50 ). Block identification information and evaluation results are stored and held in the storage unit 440, for example.

When the damage level evaluation unit 413 has performed the processing of steps S41 to S50 for all blocks set in the infrared thermal image, the block evaluation processing (step SL4) ends. After completing the block evaluation process, the damage level evaluation unit 413 synthesizes the evaluation result with the infrared thermal image and displays it (step S51), and ends the damage level evaluation process. The damage level evaluation unit 413, for example, synthesizes with the infrared thermal image a frame that color-codes a plurality of blocks in the infrared thermal image according to the evaluation result of the damage level in step S42, and the detailed damage level by the processing of steps S43 to S49. Character information indicating the evaluation result of is synthesized in each block and displayed.

In this way, the information processing apparatus 4 of the present embodiment provides, for each block, the temperature difference d between the estimated healthy temperature in the block and the estimated healthy temperature in the image, and the degree of variation in the temperature distribution in the block (standard deviation or standard error), evaluate the damage level of the portion of the paved road 9 within the block. The information processing device 4 also evaluates the detailed damage level of the portion of the paved road 9 in the block based on the calculated absolute value of the temperature difference d of the normal temperature. Furthermore, the information in the block used for the evaluation of the damage level (the estimated healthy temperature in the block, the standard deviation, the standard error, etc.) and the estimated healthy temperature in the image are included in the infrared thermal image by the information processing device 4 of the present embodiment. It is a value derived (calculated) using temperature information. Moreover, the intra-image estimated healthy temperature can be derived (calculated) by excluding the intra-block information of the block including the damaged portion of the paved road 9 . For this reason, the evaluation result of the damage level by the information processing device 4 is more objective, quantitative, and identical in comparison with the evaluation result in the conventional analytical diagnosis method performed by the analyst observing the infrared thermal image, for example. is high. Note that the block evaluation process (step SL4) is not limited to the process illustrated in FIG. There may be. In addition, the damage level and the detailed damage level are not limited to the above-described four stages (intact, longitude, moderate, and severe), and can be changed as appropriate. may be different.

Furthermore, the information processing apparatus 4 according to the present embodiment performs point cloud coordinate derivation processing (step S5) after evaluating the damage level, as described with reference to FIG. As described above, the point cloud coordinates are represented by a predetermined coordinate system set on the road surface (surface) of the paved road 9 existing in the real space, and each position (each pixel ) and the corresponding actual position on the paved road 9 .

FIG. 7 is a flowchart illustrating an example of point cloud coordinate derivation processing. FIG. 7 shows an example of the contents of processing performed by the control unit 410 (point cloud coordinate derivation unit 414) of the information processing device 4 as the point cloud coordinate derivation processing (step S5) illustrated in FIG.

In the point cloud coordinate derivation process, the point cloud coordinate derivation unit 414 first obtains the position information G(n) when the nth infrared thermal image was captured and the position when the n+1th infrared thermal image was captured. Information G(n+1) is read (step S51). The position information G(n) and G(n+1) are position information acquired from the GPS receiver 3 and stored in the storage unit 440, for example.

Next, the point cloud coordinate derivation unit 414 calculates the angle between the movement direction of the infrared thermal image capturing position (vehicle 5) based on the position information G(n) and G(n+1) and the reference direction (step S52 ). The reference direction is a predetermined direction in a coordinate system set on the road surface of the paved road 9 existing in real space. As described above, when the point cloud coordinates are derived based on the position information from the GPS receiver 3, the coordinate system set on the road surface is, for example, geographic coordinates representing the position on the paved road 9 with the latitude and longitude of the earth. system. In this case, the reference direction is, for example, the true north direction.

For example, when the infrared thermocamera 2 installed on a vehicle 5 traveling at 80 km/h captures an infrared thermal image each time the vehicle 5 moves 5 m, the infrared thermal image capturing interval is 0.225 seconds. When infrared thermal images are continuously photographed at such moving distances and time intervals, the extending direction of the photographing center VC (see FIG. 1) of the infrared thermocamera 2 projected onto the road surface of the paved road 9 is the front of the vehicle 5. , the extending direction of the projected imaging center VC can be approximated to the movement direction of the vehicle 5 based on the position information G(n) and G(n+1).

Next, the point cloud coordinate deriving unit 414 calculates the point cloud coordinates of the infrared thermal image captured by the infrared thermocamera 2 when the vehicle 5 is moving in the reference direction (hereinafter referred to as “reference point cloud coordinates”), and the calculated The point group coordinates of the n-th infrared thermal image are derived based on the obtained angle (step S53). When the vehicle 5 is moving in the reference direction, the extending direction of the imaging center VC of the infrared thermocamera 2 projected onto the paved road 9 is the true north direction. Therefore, the directions of the latitude and longitude lines in the infrared thermal image can be easily derived based on the shape and dimensions of the photographing area R of the paved road 9 photographed by the infrared thermocamera 2 . Therefore, the reference point group coordinates are, for example, the directions of the derived latitude and longitude lines, and the distance LG from the position of the vehicle 5 acquired by the GPS receiver 3 to the edge RN closest to the vehicle 5 in the photographing area R. , and the dimensions of the photographing area R of the infrared thermocamera 2 can be easily derived. Also, if the angle between the reference direction and the movement direction of the vehicle 5 when the n-th infrared thermal image was taken and the position information G(n) are known, the reference point group coordinates can be converted to n It can be converted into point cloud coordinates in the first infrared thermal image.

After finishing the process of step S53, the point cloud coordinate derivation unit 414 ends the point cloud coordinate derivation process for the n-th infrared thermal image. When the point cloud coordinate derivation process is completed, the information processing device 4 obtains the n-th infrared thermal image and the position information G(n) to be processed, and the n-th image as described with reference to FIG. are stored in association with the results of analysis and diagnosis processing for the infrared thermal image (step S6). In step S6, for example, the information in each block, the estimated healthy temperature in the image, the evaluation result of the damage level of each block, and the point cloud coordinates are associated with the infrared thermal image and the position information G(n), and the storage unit 440 memorize to

As described above, the information processing apparatus 4 of the present embodiment performs the analysis and diagnosis processing on the infrared thermal image by performing absolute detection on the actual paved road 9 corresponding to each position (each pixel) of the paved road 9 in the infrared thermal image. A point cloud coordinate indicating a specific position is derived, and the point cloud coordinate is stored and held in association with the infrared thermal image. Therefore, it is possible to easily identify the actual position on the paved road 9 corresponding to the portion evaluated as damaged in the infrared thermal image, and the infrared automatic thermal analysis and diagnosis system 1 of the present embodiment can be used. It is possible to efficiently confirm the actual damage type and damage level of the paved road 9 based on the analysis diagnosis.

Hereinafter, specific examples of analysis and diagnosis using the infrared automatic thermal analysis and diagnosis system 1 of the present embodiment will be described with reference to FIGS. 8 to 16. FIG.

FIG. 8 is a diagram including a photograph for explaining an example of an infrared thermal image. FIG. 9 is a diagram including photographs for explaining an example of a plurality of blocks set in an infrared thermal image. FIG. 10 is a diagram illustrating an example of intra-block information. FIG. 11 is a diagram illustrating an example of a method for calculating an in-screen estimated healthy temperature. FIG. 12 is a diagram illustrating an example of a method of displaying evaluation results of damage levels. 13A and 13B are diagrams for explaining a correspondence relationship between an infrared thermal image capturing area on an actual paved road and a position in the infrared thermal image. FIG. 14 is a diagram illustrating an example of the relationship between the moving direction of the vehicle and the reference direction. FIG. 15 is a diagram illustrating an example of a method of deriving point cloud coordinates. FIG. 16 is a diagram showing an example of point cloud coordinates.

When diagnosing the soundness of the paved road 9 by the infrared automatic thermal analysis diagnostic system 1 of the present embodiment, as described above, the infrared thermocamera 2 installed on the traveling vehicle 5 is used to acquires an infrared thermal image of a predetermined area (capturing area R). An infrared thermal image captured by the infrared thermocamera 2 is transferred to the information processing device 4 . At this time, for example, a screen 11 as illustrated in FIG. 8 is displayed on the display unit 430 of the information processing device 4 . The screen 11 includes an infrared thermal image 12 of U×V pixels captured by the infrared thermocamera 2 and a temperature scale 13 indicating the relationship between color (gradation) and temperature in the infrared thermal image 12 . A region PR1 on the paved road 9 in the infrared thermal image 12 of the screen 11 illustrated in FIG. 8 includes a partial region having a higher temperature than other regions. Further, near the center of the area PR2 on the paved road 9 in the infrared thermal image 12 of the screen 11 illustrated in FIG. 8, a partial area whose temperature is slightly lower than the surrounding area is included.

In a conventional analysis and diagnosis method using an infrared thermal image, for example, an analyst determines the temperature of a partial area with a high temperature in the area PR1 in the infrared thermal image 12 displayed on the display unit 430 and the ambient temperature of that partial area. to evaluate the damage level in the region PR1. Further, in the conventional analysis and diagnosis method, for example, if there is a partial area, such as the area PR2, where the temperature is slightly lower than the surrounding area, but the temperature distribution in the area is uniform, the analyst The inside may be evaluated as sound (no damage). However, damage to the paved road 9 includes not only damage that causes the temperature of the paved road 9 to become higher than the temperature of the sound portion, but also damage that causes the temperature of the paved road 9 to become lower than the temperature of the sound portion. For this reason, in the conventional analytical diagnosis method performed by an analyst observing the infrared thermal image 12, the objectivity, quantitativeness, and consistency of results may deteriorate due to individual differences. Furthermore, as the distance of the paved road 9 to be diagnosed increases, the number of infrared thermal images to be analyzed also increases. There is the problem of low

On the other hand, in the infrared automatic thermal analysis and diagnosis system 1 according to the present embodiment, as described with reference to FIGS. Automatically estimate the temperature and automatically evaluate the damage level.

The information processing apparatus 4 of the present embodiment first performs block information derivation processing (step S2) to divide the area of the paved road 9 in the infrared thermal image to be diagnosed into a plurality of blocks. Intra-block information about the temperature distribution in the block is derived for each block. The infrared thermal image 12 is, for example, an image of U×V pixels as illustrated in FIG. Therefore, the distance from the lower end to the upper end of the paved road 9 in the infrared thermal image 12 corresponds to the dimension LR (see FIG. 1) of the photographing area R of the actual paved road 9 in the road extending direction.

In the intra-block information derivation process, for example, as illustrated in FIG. 9, a substantially trapezoidal area 13 in the infrared thermal image 12 is targeted for analysis and diagnosis, and the area 13 is divided into a plurality of blocks. The area 13 corresponds to, for example, a rectangular area of a predetermined size including the edge RN closest to the vehicle 5 in the photographing area R of the infrared thermocamera 2 on the actual paved road 9 . The dimensions of the rectangular area on the actual paved road 9 can be set to any value according to the photographing area R of the infrared thermocamera 2 and the like. For example, the dimensions of the rectangular area on the actual paved road 9 can be 3 m in the horizontal direction (width direction) and 5 m in the vertical direction (traveling direction of the vehicle 5). In this case, in the substantially trapezoidal region 13 in the infrared thermal image 12, the length WF of the upper base and the length WN of the lower base are the number of pixels corresponding to the length of 3 m on the actual paved road 9, and the height L0 is The number of pixels corresponds to the length of 5 m on the actual paved road 9 . The length WF of the upper base, the length WN of the lower base, and the height L0 are the dimensions of the imaging area R and the rectangular area of the infrared thermocamera 2 on the actual paved road 9, and the reference direction in the road surface of the paved road 9. and the extending direction of the imaging center VC of the infrared thermocamera 2 projected on the road surface of the paved road 9, can be calculated by arithmetic processing based on a known projection method.

Further, when dividing the area 13 to be analyzed and diagnosed into a plurality of blocks, the division is made so that the shape of the actual paved road 9 corresponding to the portion of the paved road 9 in each block is the same. . For example, of the 60 blocks in the area 13 illustrated in FIG. 9, the blocks B1 and B24 differ in shape and area in the infrared thermal image 12 . However, the portion within the block B1 and the portion within the block B24 of the actual paved road 9 have substantially the same shape and area. Also, with regard to the other blocks within the area 13 illustrated in FIG. 9, the portion within each block on the actual paved road 9 has substantially the same shape and area as the portion within the block B1.

When the predetermined width (horizontal dimension) in the infrared thermal image 12 monotonically decreases from the bottom end to the top end, the portion near the top end of the image 12 is excluded from the analysis and diagnosis targets as illustrated in FIG. do. As a result, for example, it is possible to prevent deterioration in the reliability of diagnostic results due to the use of in-block information with a small number of pixels included in a block and low significance for analysis and diagnosis.

The information processing device 4 derives, for example, intra-block information as illustrated in FIG. In the table 14 of FIG. 10, as examples of the block information to be derived, the estimated block healthy temperature TA (n, i), the highest temperature TH (n, i), the lowest temperature TL (n, i), the standard deviation SD (n, i) and standard error SE (n, i) (n is information identifying an infrared thermal image, i is a block number). The in-block estimated healthy temperature TA(n, i) is, for example, the average temperature in the block. Note that the estimated healthy temperature TA(n, i) in the block is not limited to the average temperature in the block. , may be a value derived (calculated) by another well-known analysis method. Also, the maximum temperature TH (n, i), the minimum temperature TL (n, i), the standard deviation SD (n, i), and the standard error SE (n, i) are, for example, the variations in the temperature distribution within the block. It is an example of the information which shows a degree. The information indicating the degree of variation in the temperature distribution within the block is not limited to the combination of the four types of information illustrated, but may be one, two, or three types of information selected from these four types of information. may contain other information.

After finishing the in-block information deriving process, the information processing device 4 derives the in-image estimated healthy temperature based on the derived in-block information. The information processing device 4 selects blocks used for deriving the in-image estimated healthy temperature from the in-block information of all the blocks in the area 13, for example, based on the in-block estimated healthy temperature TA(n, i) of each block. Extract internal information. For example, as shown in FIG. 11, the information processing device 4 extracts the lowest average temperature TAmin and the highest average temperature TAmax from the average temperature calculated as the in-block estimated healthy temperature TA(n, i), and extracts the lower limit temperature Calculate TL=TAmin+ΔT1 and the upper temperature limit TH=TAmax−ΔT2. Then, the information processing device 4 extracts a block whose in-block estimated healthy temperature TA(n, i) is within the temperature range from the lower limit temperature TL to the upper limit temperature TH, and uses the block in-block information of the extracted block to generate an image. Derive the inner estimated healthy temperature. ΔT1 and ΔT2 are error temperatures and can be set to arbitrary values. The first error temperature ΔT1 and the second error temperature ΔT2 may be the same value or different values.

In this way, by deriving the in-image estimated healthy temperature by excluding the block-internal information of the block whose in-block estimated healthy temperature TA(n, i) is outside the temperature range from the lower limit temperature TL to the upper limit temperature TH, For example, it is possible to exclude the intra-block information of blocks with a high percentage of damaged portions (in other words, blocks with a low percentage of healthy portions) out of the entire portion of the paved road 9 in the block. Therefore, it is possible to prevent the estimation accuracy of the in-image estimated healthy temperature from deteriorating due to the use of the intra-block information of a block having a low ratio of healthy portions to the entire paved road 9 in the block.

Further, the information processing device 4 uses not only the estimated healthy temperature in the block (for example, the average temperature in the block) but also the degree of variation in the temperature distribution in the block as described with reference to FIG. , to derive the in-image estimated healthy temperature. For this reason, for example, the average temperature in the block is within the temperature range from the above-described lower limit temperature TL to the upper limit temperature TH, such that a portion higher than the temperature of the healthy portion and a portion lower than the temperature of the healthy portion are mixed in the block. However, it is possible to prevent the in-block estimated healthy temperature of a block with a large degree of variation in temperature distribution (for example, standard deviation or standard error) from being used as the in-image estimated healthy temperature. The degree of variation in the temperature distribution is not limited to the standard deviation and standard error as described above. Using the difference between the estimated healthy temperature TA (n, i) in the block and the lowest temperature TL (n, i) in the block, the difference between the highest temperature TH (n, i) and the lowest temperature TL (n, i), etc. may

After deriving the in-block information and the in-image estimated healthy temperature, the information processing device 4 performs, for example, the damage level evaluation process described with reference to FIG. The degree of variation in temperature distribution used for the evaluation in step S42 of the damage level evaluation process is not limited to the standard deviation and standard error described above, and may include values from other viewpoints. After completing the block evaluation process (step SL4) of the damage level evaluation process, the information processing device 4 synthesizes the evaluation result with the infrared thermal image and displays it (step S51). In step S51, the information processing device 4 synthesizes the evaluation information 15 as shown in FIG. 12 with the infrared thermal image and displays the result.

The evaluation information 15 includes a diagnostic area presentation line 16, damaged block presentation lines 17, 18 and 19, and detailed damage level information (not shown). The diagnosis area presentation line 16 is a frame indicating an evaluation target area (that is, an area 13 in which a plurality of blocks are set in the infrared thermal image) of the paved road 9 portion in the infrared thermal image, and its and the borders of blocks set within the region. Damaged block presentation lines 17, 18, and 19 are frames surrounding blocks whose damage levels were evaluated as light (Lv1), moderate (Lv2), and severe (Lv3), respectively, in step S42. In the evaluation information 15 exemplified in FIG. 12, the damaged block presentation lines 17, 18, and 19 are grouped by a plurality of continuous blocks with the same damage level evaluation, and are divided into one line along the outer circumference of the group. It has a circular line. Further, in FIG. 12, the damaged block presentation lines 17 and 18 are indicated by dashed lines with different line segment lengths, and the damaged block presentation line 19 is indicated by a dotted line. Other methods may also be used. For example, damaged block presentation lines 17, 18, and 19 may be differentiated by color.

Further, in the evaluation information 15 illustrated in FIG. 12, information (block number) for identifying the block and character information indicating the detailed damage level are arranged in each block presented by the diagnostic region presentation line 16 . The character information indicating the detailed damage level is, for example, AA for severe damage, A for moderate damage, B for light damage, and no display for no damage. In the evaluation information 15 illustrated in FIG. 12, for example, six blocks with block numbers 25, 31, 37, 43, 49, and 55 included in one damaged block presentation line 18 are damaged in step S42. The evaluation result of the level is moderate (Lv2). However, among these six blocks, four blocks with block numbers 25, 31, 37, and 43 have a detailed damage level of A (moderate), and two blocks with block numbers 49 and 55 has a detailed damage level of B (slight). In this way, in the information processing apparatus 4 of the present embodiment, detailed damage level evaluation results are displayed for each of a plurality of consecutive blocks that have the same damage level in the general evaluation. Therefore, it is possible to easily grasp the degree of progress of the damage in the damaged portion of the paved road 9 corresponding to a plurality of continuous blocks from the character information indicating the detailed damage level.

Furthermore, the information processing apparatus 4 of the present embodiment performs the point group coordinate derivation process (step S5) as described above, and determines the latitude and longitude of the earth corresponding to each position of the paved road 9 in the infrared thermal image. It is derived (calculated) for each pixel. For example, as shown in FIG. 13 , the infrared thermal image 12 of U×V pixels is an image obtained by photographing a substantially trapezoidal photographing region R(n) on the actual paved road 9 . At this time, the pixels (0, 0), (U-1, 0), (0, V-1), and (U-1, V-1) at the four corners of the infrared thermal image 12 are trapezoidal images. Corresponding to vertices RF1(n), RF2(n), RN1(n), and RN2(n) in region R(n). Any pixel (u, v) excluding the four corner pixels in the infrared thermal image 12 corresponds to any point Q within the imaging region R(n). Therefore, if absolute position information of each position on the paved road 9 such as latitude and longitude can be associated with each pixel of the infrared thermal image 12, the same position on the paved road 9 in a plurality of infrared thermal images 12 can be obtained. Position can be easily identified.

Each piece of positional information acquired by the information processing device 4 from the GPS receiver 3 is information indicating the position of the vehicle 5 (infrared thermocamera 2) on the earth, from the position specified by the positional information G(n). It does not contain information indicating which direction the image was taken. Therefore, the latitude and longitude of each point in the imaging area R(n) of the paved road 9 cannot be derived from only the position information G(n) when the n-th infrared thermal image 12 is captured. Therefore, the information processing apparatus 4 of this embodiment performs the point group coordinate derivation process as described with reference to FIG.

For example, as shown in FIG. 14, in the infrared automatic thermal analysis and diagnosis system 1 of the present embodiment, every time the vehicle 5 moves by a predetermined distance Lcap (for example, about 5 m), an infrared thermal image 12 and position information Get G(n−1), G(n), G(n+1). At this time, since the imaging interval of the infrared thermal image 12 is, for example, about 0.2 seconds to 0.3 seconds, as shown in FIG. The direction DVC can be approximated to the movement direction of the vehicle 5 derived based on the position information G(n−1), G(n), G(n+1) when the infrared thermal image 12 was captured. For this reason, the information processing apparatus 4 of the present embodiment uses the angles θ1 and θ2 of the movement direction of the vehicle 5 with respect to the reference direction in the coordinate system used to specify the position of each point on the paved road 9, for example, The position of each point within the imaging region R(n) is derived. If the position information G(n) acquired by the GPS receiver 3 is information represented by latitude and longitude, the reference direction is the true north direction.

In order to derive the position of each point in the photographing area R(n) using the angle of the movement direction of the vehicle 5 with respect to the reference direction (due north direction), the information processing device 4, for example, is illustrated in FIG. As described above, the position (latitude and longitude) of each point in the photographing area (hereinafter referred to as "reference photographing area") Rref when the direction DVC of the photographing center of the infrared thermocamera 2 is the reference direction is derived. In FIG. 15, the reference shooting area Rref with the shooting center direction DVC as the reference direction is indicated by a dotted trapezoid in a two-dimensional coordinate system with latitude and longitude as the Y axis and the X axis, respectively. At this time, the position (latitude and longitude) of each point in the reference photographing region Rref is obtained from position information G(n) indicating the position of the vehicle 5 (infrared thermocamera 2) at the time of photographing, and from the position of the vehicle 5 to each point. It can be calculated based on the direction and distance to.

After calculating the position of each point in the reference imaging region Rref, the information processing device 4 actually captured the n-th infrared thermal image in the reference direction (+Y-axis direction) as shown in FIG. Coordinate transformation is performed to rotate the position of each point in the reference imaging region Rref on a two-dimensional plane using the angle θ2 between the imaging center and the extension direction DVC (+Y′ direction). Coordinate transformation can be performed using, for example, a well-known transformation matrix. By this coordinate transformation, for example, the vertices RRF1, RRF2, RRN1, and RRN2 of the reference imaging region Rref are rotated by an angle θ2 around the point specified by the position information G(n), respectively, to positions RF1(n ), RF2(n), RN1(n), and RN2(n). Also, for example, a point QR within the reference imaging region Rref is converted to a position Q. As shown in FIG. As a result, the latitude and longitude of each point within the photographing area R(n) when the n-th infrared thermal image is photographed can be obtained. Therefore, as described with reference to FIG. 13, the latitude and longitude of each point in the photographing region R(n) corresponding to each position of the paved road 9 in the n-th infrared thermal image 12 are calculated for each pixel. can be derived to Latitude and longitude information corresponding to each position (each pixel) of the paved road 9 in the derived infrared thermal image 12 is stored as point group coordinates in a format such as the table 20 illustrated in FIG. store in 440;

In the conventional analytical diagnosis method using an infrared thermal image, as described above, since the analyst observes the infrared thermal image and diagnoses the presence or absence of damage, each position of the paved road 9 in the infrared thermal image (each pixel) and the corresponding latitude and longitude were difficult to derive. For this reason, when identifying the actual damaged portion of the paved road 9 based on the result of analysis diagnosis, for example, an infrared thermal image or an image taken with a visible light camera when taking an infrared thermal image is used. Therefore, it was difficult to efficiently identify the position and range of the damaged portion.

On the other hand, the information processing apparatus 4 of the present embodiment performs the point cloud coordinate derivation process described above, and derives point cloud coordinates indicating the latitude and longitude corresponding to each position (each pixel) in the infrared thermal image 12. , is stored in the storage unit 440 in a format such as the table 20 . For this reason, after the analysis diagnosis is completed, the latitude and longitude of the damaged portion of the paved road 9 in the infrared thermal image 12 can be determined by referring to the infrared thermal image and the point cloud coordinates in the form of the table 20. It can be easily identified from the group coordinates, and the position and range of the actual damaged portion of the paved road 9 can be efficiently identified.

The above-described embodiments are specific examples for easy understanding of the invention, and the present invention is not limited to the above-described embodiments. The information processing device 4, the infrared automatic thermal analysis/diagnosis system 1, the infrared automatic thermal analysis/diagnosis method, and the program according to the present invention can be variously modified and changed without departing from the scope of the claims.

For example, the information processing device 4 is not limited to a dedicated device combining dedicated hardware for implementing the functions of the respective units (functional blocks) of the control unit 410 as illustrated in FIG. It may be a computer that executes the program of

FIG. 17 is a diagram illustrating a hardware configuration example of a computer.

A computer 30 illustrated in FIG. 17 includes a processor 31 , a main memory device 32 , an auxiliary memory device 33 , an input device 34 , a display device 35 , an input/output interface 36 , a communication device 37 and a media drive device 38 . These pieces of hardware of computer 30 are interconnected by bus 39 .

The processor 31 is, for example, a CPU (Central Processing Unit). The main storage device 32 includes ROM (Read Only Memory) and RAM (Random Access Memory). The processor 31 controls the overall operation of the computer 30 by executing programs loaded in the main storage device 32, for example. The processor 31 can execute a program included in the OS (Operating System) and a program including part or all of the analysis diagnosis processing described with reference to FIGS. The main storage device 32 stores a program including processing related to analysis and diagnosis described with reference to FIGS. can be done.

The auxiliary storage device 33 is a storage device having a larger capacity than the main storage device 32, and is, for example, a HDD (Hard Disk Drive) or an SSD (Solid State Drive). Auxiliary storage device 33 stores programs including processing related to analysis and diagnosis described with reference to FIGS. can be done.

The input device 34 is, for example, a device such as a keyboard and a mouse that performs input operations to the computer 30. The display device 35 is, for example, a device that visualizes and displays various data such as a liquid crystal display. If the computer 30 is equipped with a touch panel display such as a tablet computer or a smartphone, the input device 34 may include, for example, a digitizer (position detection device) placed over the display surface of the display device 35. .

The input/output interface 36 is a hardware interface for connecting the computer 30 and peripheral devices. The infrared thermocamera 2 and GPS receiver 3 are connected to the computer 30 via an input/output interface 36 .

The communication device 37 connects the computer 30 to a communication network 41 such as the Internet, and communicates with an external device such as a server device 42 via the communication network 41 . The communication device 37 communicates with the server device 42 or the like via the communication network 41 by wireless communication or by wired communication using a communication cable such as a LAN (Local Area Network) cable. The server device 42 can be used, for example, to aggregate the results of analysis and diagnosis processing performed by the computer 30 .

The media drive device 38 accesses the portable recording medium 40 to read information recorded on the portable recording medium 40 and write information to the portable recording medium 40 . The portable recording medium 40 includes, for example, optical discs, magnetic discs, magneto-optical discs, and card-type semiconductor memory devices.

Note that the hardware configuration of the computer 30 that can be used as the information processing device 4 is not limited to the configuration illustrated in FIG. 17, and can be changed as appropriate. Computer 30 may not include one or more of the hardware configurations illustrated in FIG. 17 (eg, media drive 38). Also, the computer 30 may include hardware different from the hardware configuration illustrated in FIG.

The analysis and diagnosis processing performed by the information processing device 4 (computer 30) is not limited to the content and order of the processing described with reference to FIGS. 3 to 7, and can be changed as appropriate. For example, the analysis and diagnosis processing (step SL1 in FIG. 3) performed by the information processing device 4 (computer 30) may be performed separately from the processing of accumulating the infrared thermal image and position information used for diagnosis. In this case, the infrared thermal image and the positional information used for diagnosis may be transferred to the information processing device 4 (computer 30) via the portable recording medium 40 described above and stored therein. Further, for example, the point cloud coordinate derivation process (step S5) is not limited to the process performed as part of the analysis diagnosis process (step SL1 in FIG. It may be a separate process from the series of analysis and diagnosis processes including the derivation process (step S3) and the damage level evaluation process (step S4).

Furthermore, the system configuration of the infrared automatic thermal analysis and diagnosis system 1 according to the present invention is not limited to the configuration described above, and can be changed as appropriate. For example, the infrared automatic thermal analysis and diagnosis system 1 may include a visible light camera that photographs the paved road 9 including the imaging region R of the infrared thermocamera 2 .

1 Infrared automatic thermal analysis diagnostic system 2 Infrared thermo camera 3 GPS receiver 4 Information processing device 400 Transmission/reception unit 410 Control unit 411 In-block information derivation unit 412 Healthy temperature estimation unit 413 Damage level evaluation unit 414 Point group coordinate derivation unit 420 Input unit 430 Display unit 440 Storage unit 5 Vehicle 6 Metal fitting 9 Paved road 12 Infrared thermal image 14 In-block information 15 Evaluation information 20 Table (of point group coordinates) 30 Computer 31 Processor 32 Main storage device 33 Auxiliary storage device 34 Input device 35 Display device 36 input/output interface 37 communication device 38 media drive device 39 bus 40 portable recording medium 41 communication network 42 server device

Claims (12)

A computer analyzes a plurality of infrared thermal images continuously photographed from different areas of a paved road on which the vehicle is traveling by an infrared thermocamera attached to the traveling vehicle, and diagnoses the soundness of the paved road. A thermal analysis diagnostic method,
The computer, for each infrared thermal image,
setting an analysis target area in an infrared thermal image and dividing the analysis target area into a plurality of blocks;
estimating the temperature of a healthy portion of the paved road for each block based on the temperature distribution in the block to derive an estimated healthy temperature in the block;
In-image estimation by estimating the temperature of a healthy part in an area corresponding to the analysis target area of the paved road on which the vehicle travels, based on the plurality of the estimated healthy temperatures in the block derived for each block. An infrared automatic thermal analysis and diagnosis method characterized by deriving a healthy temperature. In the infrared automatic thermal analysis diagnosis method according to claim 1,
The infrared automatic thermal analysis and diagnosis method, wherein the computer derives an average temperature within the block as the estimated healthy temperature within the block. In the infrared automatic thermal analysis diagnosis method according to claim 1 or 2,
The processing performed by the computer for each infrared thermal image is
deriving, for each block, one or more of a maximum temperature, a minimum temperature, a standard deviation of temperature distribution, and a standard error of temperature distribution in the block;
setting a temperature range based on a maximum average temperature and a minimum average temperature of the estimated healthy temperatures in the plurality of blocks derived from each of the plurality of blocks in the analysis target region;
the estimated healthy temperature in the block derived from the block whose estimated healthy temperature in the block is within the set temperature range, the maximum temperature and the minimum temperature derived, deriving the in-image estimated healthy temperature based on one or more of the standard deviation of the temperature distribution and the standard error of the temperature distribution;
An infrared automatic thermal analysis diagnosis method characterized by: In the infrared automatic thermal analysis diagnosis method according to any one of claims 1 to 3,
The computer is
A first in-image estimated healthy temperature derived from a first infrared thermal image of the plurality of infrared thermal images, and a second infrared thermal image taken prior to the first infrared thermal image. setting the in-image estimated healthy temperature for the first infrared thermal image to the second in-image estimated healthy temperature when the temperature difference from the second estimated healthy temperature in the image exceeds a predetermined range. An infrared automatic thermal analysis diagnostic method characterized by: In the infrared automatic thermal analysis diagnosis method according to any one of claims 1 to 3,
The computer is
When the variation in the temperature distribution in the analysis target region of the first infrared thermal image among the plurality of infrared thermal images exceeds a predetermined range, the first infrared thermal image is captured before the first infrared thermal image. and setting an in-image estimated healthy temperature derived from a second infrared thermal image in which variation in temperature distribution in the analysis target area is within a predetermined range as the in-image estimated healthy temperature for the first infrared thermal image. An infrared automatic thermal analysis diagnostic method characterized by: In the infrared automatic thermal analysis diagnosis method according to any one of claims 1 to 5,
The computer is
Information about the temperature distribution derived based on the temperature distribution in the block including the in-block estimated healthy temperature and the in-image estimated healthy temperature for each of the paved roads on which the vehicle travels A method for infrared automatic thermal analysis and diagnosis, characterized by evaluating the degree of damage within a region corresponding to said block of and outputting the result of said evaluation. In the infrared automatic thermal analysis diagnosis method according to claim 6,
The computer is
An infrared automatic thermal analysis and diagnosis method, wherein the degree of damage is evaluated based on a temperature difference between the estimated healthy temperature in the block and the estimated healthy temperature in the image and variations in temperature distribution in the block. In the infrared automatic thermal analysis diagnosis method according to any one of claims 1 to 7,
The computer further
Acquiring position information of the vehicle when each of the plurality of infrared thermal images is captured;
For each infrared thermal image ,
The infrared thermal image is captured based on the position information when the infrared thermal image was captured, the position information when the infrared thermal image following the infrared thermal image was captured, and a predetermined reference orientation. identifying the direction of travel of the vehicle when
Based on the specified moving direction of the vehicle and the imaging conditions of the infrared thermal image by the infrared thermocamera, each position of the paved road in the infrared thermal image and the coordinate system used for the positional information corresponding to the An infrared automatic thermal analysis and diagnosis method characterized by deriving the position of a paved road. In the infrared automatic thermal analysis diagnosis method according to claim 8,
The position information is information indicating the position of the running vehicle in terms of latitude and longitude,
The conditions for capturing the infrared thermal image by the infrared thermocamera include the positional relationship between the position of the vehicle specified by the positional information and the area captured by the infrared thermocamera on the road surface of the paved road. An infrared automatic thermal analysis diagnostic method characterized by: An information processing device for diagnosing the soundness of a paved road by analyzing a plurality of infrared thermal images continuously photographed from different areas of a paved road on which the vehicle is traveling using an infrared thermocamera attached to a traveling vehicle. hand,
For each infrared thermal image, an analysis target area is set in the infrared thermal image, the analysis target area is divided into a plurality of blocks, and the soundness of the paved road is determined for each block based on the temperature distribution in the block. an in-block information deriving unit for estimating the temperature of the part and deriving an estimated healthy temperature in the block;
For each infrared thermal image, the temperature of a healthy portion in the area corresponding to the analysis target area of the paved road on which the vehicle travels is calculated based on the plurality of estimated healthy temperatures in the block derived for each block. An information processing apparatus, comprising: a healthy temperature estimating unit for estimating and deriving an estimated healthy temperature in an image. infrared thermo camera,
An information processing device that analyzes a plurality of infrared thermal images continuously photographed by the infrared thermocamera attached to the traveling vehicle and different areas of the paved road on which the vehicle travels, and diagnoses the soundness of the paved road. , including
The information processing device is
For each infrared thermal image, an analysis target area is set in the infrared thermal image, the analysis target area is divided into a plurality of blocks, and the soundness of the paved road is determined for each block based on the temperature distribution in the block. an in-block information deriving unit for estimating the temperature of the part and deriving an estimated healthy temperature in the block;
For each infrared thermal image, the temperature of a healthy portion in the area corresponding to the analysis target area of the paved road on which the vehicle travels is calculated based on the plurality of estimated healthy temperatures in the block derived for each block. An infrared automatic thermal analysis and diagnosis system, comprising: a healthy temperature estimating unit for estimating and deriving an estimated healthy temperature in an image. The computer analyzes a plurality of infrared thermal images continuously photographed from different areas of the paved road on which the vehicle is traveling by an infrared thermocamera attached to the traveling vehicle, and diagnoses the soundness of the paved road. A program that executes
For each infrared thermal image,
setting an analysis target area in an infrared thermal image and dividing the analysis target area into a plurality of blocks;
estimating the temperature of a healthy portion of the paved road for each block based on the temperature distribution in the block to derive an estimated healthy temperature in the block;
In-image estimation by estimating the temperature of a healthy part in an area corresponding to the analysis target area of the paved road on which the vehicle travels, based on the plurality of the estimated healthy temperatures in the block derived for each block. A program characterized by including processing for deriving a healthy temperature.

Images