JP6516384B2 - INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING SYSTEM, INFORMATION PROCESSING METHOD, AND INFORMATION PROCESSING PROGRAM - Google Patents

Description

  The present invention relates to an information processing apparatus, an information processing system, an information processing method, and an information processing program.

  With the aging of roads, bridges, and facilities, efficient maintenance of buildings is required. To carry out the maintenance of a structure, it is necessary to detect the damage of the structure, but in the visual confirmation of the damage, it is difficult to quantitatively evaluate and comprehensively analyze. Therefore, automatic detection by a machine is required. In Patent Document 1, for example, an on-vehicle camera shoots a road surface image of a road surface on which the sun shines, and a road surface image of a road surface on which the shadow appears, so that partial regions in the front, rear, left and right images overlap. The road surface image is subjected to joint processing by performing trapezoidal correction, calculation of the degree of coincidence of overlapping regions, and the like.

JP, 2011-90367, A

  However, if the image subjected to the bonding process using the technique of Patent Document 1 is analyzed as it is, the load required for the analysis becomes large. In addition, it was not possible to automatically detect the damage of the building only by such a bonded image, and it was not possible to quantitatively evaluate or comprehensively analyze the damage of the building. As described above, there is a problem that the convenience at the time of automatic detection of damage to a building is not sufficient.

  One aspect of the present invention is made in view of the above-mentioned subject, and an information processing device, an information processing system, an information processing method, and an information processing method capable of improving convenience at the time of automatic detection of damage of a building. One purpose is to provide a program.

(1) The present invention has been made to solve the problems described above, and one aspect of the present invention provides an image acquisition unit that acquires an image of a region including a surface captured by an imaging device, and a plurality of images Is selected, the selected inter-image distance is compared with a predetermined set distance, and if the inter-image distance is larger than the set distance, the image being selected is set as an examination image candidate, and the inter-image distance is A road surface which is not set as an inspection image candidate when the distance is smaller than the set distance, and which is based on an image extracting unit and position information corresponding to the inspection image candidate, among images of a plurality of road surfaces The inspection image is extracted from the inspection image candidates extracted by the image extraction unit so that at least a part of the area of the road surface in the image of the road and the area of the road surface in the image of the other road overlap. An extraction unit and the overlapping area extraction unit For each of has been examined images, and the check state determining state determination unit of the surface of the image, for each of the test image, on the basis of the state of the surface in the inspection image, the surface in the inspection images A calculation unit for calculating the degree of damage , wherein the predetermined set distance is set larger as the road surface damage recognition accuracy in one inspection image is higher , thereby reducing duplication of the road surface shown in each inspection image; The information processing apparatus is configured to be set smaller as the recognition accuracy of the road surface damage in one inspection image is lower, thereby increasing the number of road surface duplications included in each inspection image .

( 2 ) Further, one aspect of the present invention is the information processing apparatus described above, wherein the classification determination unit classifies any one of a plurality of stages according to the degree of damage to the surface in the partial image. Further comprising

( 3 ) Further, one aspect of the present invention is the information processing apparatus described above, wherein the partial image generation unit divides and divides the image in the image acquired by the image acquisition unit. Generating a first partial image and a second partial image including vertexes or side portions of the first partial image.

( 4 ) Further, according to an aspect of the present invention, in the information processing apparatus described above, the partial image generation unit divides and divides the image between the images acquired by the image acquisition unit. Generating a first partial image and a second partial image including vertexes or side portions of the first partial image.

( 5 ) Further, according to an aspect of the present invention, in the information processing apparatus described above, the calculation unit is configured to calculate, for each of the plurality of continuous images acquired by the image acquiring unit, the respective images. Calculate the degree of damage to the surface of the

( 6 ) Further, one aspect of the present invention is the information processing apparatus described above, further comprising: a state determination unit that determines a state of a surface in each of the images, wherein the calculation unit Based on the state of the surface in the image, the degree of damage to the surface in the image is calculated.

( 7 ) Further, one aspect of the present invention is the information processing apparatus described above, wherein classification determination is performed to classify any one of a plurality of stages according to the degree of damage to the surface in each of the images. It further comprises a department.

( 8 ) Moreover, 1 aspect of this invention is an information processing apparatus as described above, Comprising: The said 1st divided | segmented said image among the continuous said several images which the said image acquisition part acquired, The divided 1st Generating a partial image and a second partial image including vertexes or sides of the first partial image;

( 9 ) Further, according to an aspect of the present invention, there is provided an image acquisition unit that acquires an image of a region including a surface captured by an imaging device;
A plurality of images are selected, the selected inter-image distance is compared with a predetermined set distance, and when the inter-image distance is larger than the set distance, the image being selected is set as an examination image candidate, When the inter-distance is smaller than the set distance, the selected image is not set as the inspection image candidate. Based on the image extraction unit and the position information corresponding to the inspection image candidate, among the images of the plurality of road surfaces The inspection image is extracted from among the inspection image candidates extracted by the image extraction unit so that at least a part of the area of the road surface in the image of a certain road surface and the area of the road surface in the image of another road surface overlap Overlap region extraction unit, a state determination unit that determines the state of the surface in the inspection image for each of the inspection images extracted by the overlap region extraction unit, and the inspection image for each of the inspection images based on the state of the surface , And a calculation unit for calculating a degree of damage of the surface in the inspection image, the predetermined setting distance, by recognition accuracy of the road surface damage in one test image is set higher large, each test image The present invention is an information processing system that reduces duplication of a road surface to be photographed and is set smaller as recognition accuracy of road surface damage in one inspection image is lower, thereby increasing duplication of a road surface to be photographed in each inspection image .

(12) Further, according to an aspect of the present invention, the computer of the information processing apparatus selects and selects a plurality of images, and an image acquisition process of acquiring an image of a region including a plane imaged by the imaging apparatus . The inter-image distance is compared with a predetermined set distance, and when the inter-image distance is large compared to the set distance, the image being selected is set as the inspection image candidate, and the inter-image distance is small relative to the set distance The image selection process does not set the image being selected as the inspection image candidate, and the area of the road surface in the image of the road surface among the images of the plurality of road surfaces based on the position information corresponding to the inspection image candidate An overlap area extraction process of extracting an inspection image from the inspection image candidates extracted in the image extraction process so that at least a part of the road surface area in the image of another road surface overlaps; Extracted in the extraction process For each test image, and a state determining state determination process of surface in the inspection image, for each of the test image, on the basis of the state of the surface in the inspection image, damage degree of the surface in the inspection images Calculation step of calculating the predetermined set distance is set larger as the recognition accuracy of the road surface damage in one inspection image is higher , thereby reducing the overlapping of the road surface shown in each inspection image, 1 This is an information processing method in which the overlapping of the road surface shown in each inspection image is increased by being set smaller as the recognition accuracy of the road surface damage in one inspection image is lower .

(13) Further, according to an aspect of the present invention, in the computer of the information processing apparatus, an image acquisition process of acquiring an image of an area including a surface imaged by the imaging apparatus and a plurality of images are selected and selected. The inter-image distance is compared with a predetermined set distance, and when the inter-image distance is large compared to the set distance, the image being selected is set as the inspection image candidate, and the inter-image distance is small relative to the set distance The image selection process does not set the image being selected as the inspection image candidate, and the area of the road surface in the image of the road surface among the images of the plurality of road surfaces based on the position information corresponding to the inspection image candidate An overlap area extraction process of extracting an inspection image from the inspection image candidates extracted in the image extraction process so that at least a part of the road surface area in the image of another road surface overlaps; Extracted in the extraction process For each test image, and a state determining state determination process of surface in the inspection image, for each of the test image, on the basis of the state of the surface in the inspection image, damage degree of the surface in the inspection images Calculation process for calculating the road surface and a program for executing the process , wherein the predetermined set distance is set larger as the recognition accuracy of the road surface damage in one inspection image is higher, thereby overlapping the road surface shown in each inspection image The program is configured to reduce the road surface damage in each inspection image by setting the smaller the lower the road surface damage recognition accuracy in one inspection image, the lower the road surface damage recognition accuracy .

  ADVANTAGE OF THE INVENTION According to this invention, the convenience in the case of the automatic detection of the damage of a building can be improved.

It is a schematic diagram showing an example of composition of an information processing system concerning a 1st embodiment of the present invention. It is an explanatory view showing an example of a road surface image concerning a 1st embodiment of the present invention. It is explanatory drawing which shows another example of the image of the road surface which concerns on the 1st Embodiment of this invention. It is explanatory drawing which shows another example of the image of the road surface which concerns on the 1st Embodiment of this invention. It is explanatory drawing which shows another example of the image of the road surface which concerns on the 1st Embodiment of this invention. It is explanatory drawing which shows another example of the image of the road surface which concerns on the 1st Embodiment of this invention. It is a schematic block diagram which shows an example of a structure of the information processing apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows an example of the calculation process of the damage degree which concerns on the 1st Embodiment of this invention. It is a schematic block diagram showing an example of composition of an information processor concerning a 2nd embodiment. It is a flowchart which shows an example of the calculation process of the damage degree which concerns on the 2nd Embodiment of this invention. It is a flowchart which shows an example of the calculation process of the damage degree which concerns on the 2nd Embodiment of this invention. It is a schematic block diagram showing an example of composition of an information processor concerning a 3rd embodiment of the present invention. It is a flow chart which shows an example of damage degree calculation processing concerning a 3rd embodiment of the present invention. It is a flow chart which shows an example of damage degree calculation processing concerning a 3rd embodiment of the present invention. It is a figure which shows an example of the damage classification of the road surface which concerns on the 4th Embodiment of this invention. It is a figure which shows another example of the damage classification of the road surface which concerns on the 4th Embodiment of this invention. It is a figure which shows another example of the damage classification of the road surface which concerns on the 4th Embodiment of this invention. It is a figure which shows another example of the damage classification of the road surface which concerns on the 4th Embodiment of this invention. It is a figure which shows another example of the damage classification of the road surface which concerns on the 4th Embodiment of this invention. It is a figure which shows another example of the damage classification of the road surface which concerns on the 4th Embodiment of this invention. It is a figure which shows another example of the damage classification of the road surface which concerns on the 4th Embodiment of this invention. It is a figure which shows another example of the damage classification of the road surface which concerns on the 4th Embodiment of this invention. It is a schematic block diagram which shows an example of a structure of the information processing apparatus which concerns on the 4th Embodiment of this invention. It is a flowchart which shows an example of the damage classification determination processing which concerns on the 4th Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

First Embodiment
FIG. 1 is a schematic view showing an example of the configuration of an information processing system sys according to the first embodiment of the present invention.
The information processing system sys is configured to include at least the information processing device 1. The information processing system sys is a system that analyzes the state of the surface of a building. Below, as an example, the case where the information processing system sys analyzes the damage of the surface of a building is explained.
The structures are, for example, roads, bridges, facilities, etc., and also include social infrastructure. Also, the surface is the outer surface or the inner surface of a building. That is, the surface of a building is, for example, the road surface or bridge of a road, the upper surface, the side surface, or the lower surface of a facility. The damage is physical unevenness, change in color, cracking or breakage. In the following description, although the case of evaluating and analyzing road surface damage is described as an example, the present invention is not limited to this, and is applicable to evaluation and analysis of damage of various buildings.

  Damage to the road surface includes, for example, pitting of the road surface, pot holes, cracks (also referred to as cracks), steps, construction seams and the like. Cracks include linear cracks and planar cracks. The linear cracks include longitudinal cracks, transverse cracks and the like. The planar cracks include turtle crack and the like.

  In the information processing system sys, the information processing device 1 acquires an image P of one or more road surfaces. The image P of the road surface may be captured and acquired by the imaging device when the information processing system sys includes an imaging device, or the road surface captured by a fixed point camera, a user's hand-held camera, an on-vehicle camera, etc. The image P of may be acquired via a network or an input interface.

  The information processing device 1 calculates the degree of damage to the road surface in the image based on the acquired image of the road surface.

FIG. 2 is an explanatory view showing an example of an image of a road surface according to the first embodiment of the present invention. FIG. 3 is an explanatory view showing another example of the image of the road surface according to the first embodiment of the present invention. FIG. 4 is an explanatory view showing another example of the image of the road surface according to the first embodiment of the present invention. FIG. 5 is an explanatory view showing another example of the image of the road surface according to the first embodiment of the present invention. FIG. 6 is an explanatory view showing another example of the image of the road surface according to the first embodiment of the present invention.
The image P1 of the road surface shown in FIG. 2 is an example of the ridge D1. The image P2 of the road surface shown in FIG. 3 is an example of the pothole D2. The image P3 of the road surface shown in FIG. 4 is an example of the crack D3. The image P4 of the road surface shown in FIG. 5 is an example of the step D4. The image P5 of the road surface shown in FIG. 6 is an example of the patching D5. The information processing apparatus 1 evaluates and analyzes the degree of damage to the various road surfaces described above, in addition to the examples illustrated in FIGS. 2 to 6.

  Next, an outline of analysis of road surface damage using the information processing system sys will be described.

  First, the user captures an image of the road surface. The image of the road surface is an image including the road surface, such as an image taken by a vehicle, an image taken by a moving object such as an image taken while moving the road surface and a road shoulder, and an image taken by a fixed point camera As long as it is, it may be photographed by any photographing method. The image is not limited to a still image, and may be a moving image. The moving object is, for example, a vehicle, a person, an aircraft, an unmanned photographing device, an animal or the like. For example, when photographing with a vehicle-mounted camera or when photographing while moving the road surface or the road shoulder, the user appropriately photographs the road surface according to the movement of the vehicle or the user.

In the following description, although an example in which position information and a time stamp are stored in a standard format (for example, EXIF, etc.) for an image of a road surface will be described, the format and storage method of the position information and time stamp And the extraction method is optional. The position information and the time stamp may be absolute values or relative values.
In addition, acquisition of position information and a time stamp may use an imaging device or an external device equipped with a GPS (Global Positioning System) sensor.

Next, the information processing apparatus 1 acquires an image of a road surface via an input interface such as a communication network or a memory card. For example, the information processing apparatus 1 analyzes the acquired image of the road surface and analyzes the damage of the road surface in the image. For example, the information processing apparatus 1 calculates the feature amount of the road surface in the image to analyze the state of the road surface, or analyzes the state of the road surface using artificial intelligence. Artificial intelligence comprises, for example, a multilayer perceptron (multilayer neural network). In artificial intelligence, deep learning (deep learning) is performed on the acquired image of the road surface based on teacher images (for example, images P1 to P5 and the like) obtained by imaging analysis targets such as road surface damage. For artificial intelligence, any technique, framework or the like used in conventional artificial intelligence may be applied.
The information processing apparatus 1 outputs the degree of damage to the road surface, the type of damage to the road surface, and the like as the analysis result of the image of the road surface. In this embodiment, an example in the case of analysis by deep learning will be described.

FIG. 7 is a schematic block diagram showing an example of the configuration of the information processing apparatus 1 according to the first embodiment of the present invention.
The information processing apparatus 1 includes an input unit 10, a communication unit 13, a storage unit 15, a control unit 17, and an output unit 19. The control unit 17 is configured to include an image acquisition unit 171, an image extraction unit 173, an area extraction unit 175, and a calculation unit 177. The calculation unit 177 includes a state determination unit 1771 and a degree calculation unit 1773.

  The input unit 10 receives input of various information. The input unit 10 is, for example, a pointing device such as a mouse or a touch pad, a keyboard, or an input interface for acquiring information from an external device.

  The communication unit 13 transmits and receives various information to and from other devices. The communication unit 13 includes, for example, a communication IC (Integrated Circuit), a communication port, and the like.

  The storage unit 15 stores various information. The storage unit 15 includes, for example, a read only memory (ROM), a random access memory (RAM), and the like. In addition, the storage unit 15 may include a hard disc drive (HDD), an electrically erasable programmable read only memory (EEPROM), a flash memory, and the like. The storage unit 15 stores various programs to be executed by a CPU (Central Processing Unit, not shown) and a GPU (Graphics Processing Unit, not shown) included in the information processing apparatus 1 and results of processing executed by the CPU and GPU. Do. In addition, the storage unit 15 stores an analysis result (learning result) by the information processing device 1. The analysis result is, for example, information indicating the state of the neural network. The information which shows the state of the network of a neural network is the information etc. which show the value of the parameter of the activation function concerning each node (unit) which constitutes a neural network, for example.

  The output unit 19 outputs various information. The output unit 19 is, for example, a display device such as a liquid crystal display, an organic EL display, or a plasma display, an output interface that outputs an analysis result, or the like.

The control unit 17 controls the information processing apparatus 1. Specifically, for example, a program stored in advance in the storage unit 15 is controlled by execution of a CPU or a GPU. Note that part or all of the control unit 17 may be realized as an integrated circuit of hardware such as LSI (Large Scale Integration) or ASIC (Application Specific Integrated Circuit).
The control unit 17 is configured to include an image acquisition unit 171, an image extraction unit 173, an area extraction unit 175, and a calculation unit 177. The calculation unit 177 includes a state determination unit 1771 and a degree calculation unit 1773.

  The image acquisition unit 171 acquires an image of one or more road surfaces via the input unit 10 or the communication unit 13. The image acquisition unit 171 outputs information representing the acquired image of the road surface to the image extraction unit 173.

  When the image acquisition unit 171 receives information representing an image of a road surface, the image extraction unit 173 performs various image processing on the image of the road surface. The image extraction unit 173 converts each frame into a still image, for example, when the image of the road surface is a moving image encoded. Further, the image extracting unit 173 refers to the metadata of the image of the road surface included in the information representing the image of the road surface, and extracts the time stamp, the position information, and the like. The image extraction unit 173 executes processing such as identification of an inspection image candidate and extraction of the inspection image candidate from the acquired series of images of the road surface. Note that the image extraction unit 173 may execute processing such as identification and extraction of an image that does not analyze the presence or absence of road surface damage from the acquired series of road surface images.

The area extraction unit 175 extracts the image based on the position information extracted from the metadata so that there is no area of road surface overlapping among the images of a plurality of road surfaces, and there is no area of road surface missing. An inspection image is extracted from the inspection image candidates. Here, the images of the plurality of road surfaces are the images of the plurality of frames. The area extraction unit 175 outputs the information indicating the extracted inspection image and the corresponding position information to the calculation unit 177.
In addition, the region extraction unit 175 does not use position information, and there is no region of road surface overlapping among images of a plurality of road surfaces, and there is no region of road surface that is missing, the inspection image extracted by the image extraction unit 173 An examination image may be extracted from the candidates. In this case, for example, corresponding point matching between a certain examination image candidate and the examination image candidate continuous with the examination image candidate may be used.

  The calculation unit 177 analyzes the damage of the road surface in the inspection image by deep learning. Specifically, the state determination unit 1771 determines whether or not the road surface in each inspection image is damaged by deep learning, and the inspection image in which the road surface damage is detected is “1”, and the road surface damage is detected. The inspection image which is not processed is binarized as "0" to calculate the damage presence / absence information. The degree calculator 1773 calculates the degree of damage based on the damage presence / absence information. Specifically, the degree calculation unit 1773 calculates the degree of damage by dividing the total sum of damage presence / absence information in a predetermined section, for example, a 20 m section, by the number of inspection images in the predetermined section. The calculation unit 177 causes the output unit 19 to output the calculated degree of damage and the position information.

The calculating unit 177 may output the damage presence / absence information as it is to the output unit 19 without calculating the degree of damage.
Note that the calculation unit 177 compares the calculated damage degree with a plurality of threshold values, determines which threshold value range the damage degree falls within, and divides the damage degree into a plurality of damage categories according to the determination result. Good. The plurality of damage sections may be, for example, a graded section such as section 1 having a small degree of damage, section 2 having a medium degree of damage, and section 3 having a high degree of damage. In this case, instead of or in addition to causing the output unit 19 to output the degree of damage, the calculating unit 177 may output the damage category.

FIG. 8 is a flowchart showing an example of the damage degree calculation process according to the first embodiment of the present invention.
In step S101, the image acquisition unit 171 acquires an image of one or more road surfaces via the input unit 10 or the communication unit 13.
In step S103, when the image acquisition unit 171 receives information representing an image of a road surface, the image extraction unit 173 performs various image processing on the image of the road surface. Further, the image extracting unit 173 refers to the metadata of the image of the road surface included in the information representing the image of the road surface, and extracts the time stamp, the position information, and the like. The image extraction unit 173 executes processing such as identification of an inspection image candidate and extraction of the inspection image candidate from the acquired series of images of the road surface.

In step S105, based on the position information extracted from the metadata, the area extraction unit 175 does not have an area of road surface overlapping among images of a plurality of road surfaces, and does not have an area of road surface that is missing. An inspection image is extracted from among the inspection image candidates extracted by 173.
In step S107, the state determination unit 1771 determines whether or not the road surface in each inspection image is damaged by deep learning, and the inspection image in which the road surface damage is detected is “1”, and the road surface damage is detected. The inspection image which is not detected is binarized as "0" to calculate the damage presence / absence information.

  In step S109, the degree calculator 1773 calculates the degree of damage based on the damage presence / absence information. Specifically, the degree calculation unit 1773 calculates the degree of damage by dividing the total sum of damage presence / absence information in a predetermined section, for example, a 20 m section, by the number of inspection images in the predetermined section.

As described above, the information processing apparatus 1 according to the first embodiment acquires a surface based on the image acquired by the image acquisition unit 171 acquiring the image of the area including the surface photographed by the imaging apparatus and the image acquired by the image acquisition unit 171. And a calculation unit 177 for calculating the degree of damage of the
As a result, surface damage can be automatically detected, and quantitative evaluation can be performed, whereby convenience in automatic detection of structure damage can be improved.

Second Embodiment
In the second embodiment, an example in which a part of the area is overlapped between images of a plurality of road surfaces to extract an inspection image will be described. In the second embodiment, parts different from the first embodiment will be mainly described.

FIG. 9 is a schematic block diagram showing an example of the configuration of the information processing apparatus 1A according to the second embodiment.
The information processing apparatus 1 </ b> A includes an input unit 10, a communication unit 13, a storage unit 15, a control unit 17, and an output unit 19. The control unit 17 includes an image acquisition unit 171, an image extraction unit 173, an overlapping area extraction unit 175A, and a calculation unit 177. The calculation unit 177 includes a state determination unit 1771 and a degree calculation unit 1773.

  The overlapping area extracting unit 175A determines, based on the position information extracted from the metadata, a road surface area in an image of a road surface and a road surface overlapping an area of a road surface in an image of another road surface among images of a plurality of road surfaces. An inspection image is extracted from among the inspection image candidates extracted by the image extraction unit 173 so that at least a part of the region and the region overlap. In other words, the overlapping area extraction unit 175A extracts the inspection image from the inspection image candidates so that the road surface areas overlap temporally between the images of the plurality of road surfaces. Further, in each inspection image extracted by the overlapping area extracting unit 175A, the vertex portion or the side portion of the road surface area of the image of a road surface is included in the road surface area of the other road surface image among the plurality of road surface images. . The overlapping area extraction unit 175A outputs the information representing the extracted inspection image and the corresponding position information to the calculation unit 177.

FIG. 10 is a flow chart showing an example of damage degree calculation processing according to the second embodiment of the present invention.
In step S201, the image acquisition unit 171 acquires an image of one or more road surfaces via the input unit 10 or the communication unit 13.
In step S203, the image extraction unit 173 executes an image extraction process described later.
In step S205, the overlapping area extracting unit 175A determines, based on the position information extracted from the metadata, an area of a road surface in an image of a road surface and an area of a road surface in an image of another road surface among images of a plurality of road surfaces. The inspection image is extracted from the inspection image candidates extracted by the image extraction unit 173 so that at least a part of the overlapping road surface area overlaps.

In step S207, the state determination unit 1771 determines whether or not the road surface in each inspection image is damaged by deep learning, and the inspection image in which the road surface damage is detected is “1”, and the road surface damage is detected. The inspection image which is not detected is binarized as "0" to calculate the damage presence / absence information.
In step S209, the degree calculator 1773 calculates the degree of damage based on the damage presence / absence information. Specifically, the degree calculation unit 1773 calculates the degree of damage by dividing the total sum of damage presence / absence information in a predetermined section, for example, a 20 m section, by the number of inspection images in the predetermined section.

FIG. 11 is a flowchart showing an example of the process of calculating the degree of damage according to the second embodiment of the present invention.
In step S301, the image extraction unit 173 selects an image of the road surface of the frame. Specifically, when the road surface image is not selected, the image extracting unit 173 selects the road surface image of the first frame. On the other hand, when the frame has already been selected, the image extraction unit 173 selects the image of the road surface of the frame next to the frame being selected.

In step S303, the image extracting unit 173 refers to the position information corresponding to the image of the frame being selected, and calculates the movement distance from the position information corresponding to the immediately preceding frame. Hereinafter, this movement distance is referred to as “distance between images”.
In step S305, the image extraction unit 173 adds the calculated movement distance to the total movement distance. The movement distance sum is initialized at the start of the image extraction process.

  In step S307, the image extracting unit 173 compares the total movement distance with the predetermined set distance. If the total movement distance is larger than the set distance (step S307; YES), the control unit 17 executes the process of step S309. On the other hand, when the total movement distance is smaller than the set distance (step S307; NO), the control unit 17 executes the process at step S315.

In step S309, the image extraction unit 173 sets the total movement distance to 0 and initializes it.
In step S311, the image extraction unit 173 sets the image being selected as an examination image candidate.

In step S313, the image extraction unit 173 confirms the presence or absence of the image of the road surface of the next frame. When an image exists (step S313; YES), the control unit 17 executes the process of step S301. If there is no image (step S313; NO), the control unit 17 executes the process at step S315.
In step S315, the image extracting unit 173 outputs the inspection image candidate to the overlapping area extracting unit 175A.

  The set distance may be set, for example, in accordance with the recognition accuracy of the road surface damage in one inspection image. Further, for example, in the case where it is possible to detect road surface damage that appears not only in the vicinity but also in a single inspection image, the set distance may be increased to reduce duplication of the road surface shown in each inspection image. In addition, in the case where it is not possible to detect a road surface damage that appears in a distance in one inspection image, the set distance may be reduced to increase the overlap of the road surface that is displayed in each inspection image. As a result, it is possible to maintain the road surface damage recognition accuracy while reducing the load required to analyze the road surface damage.

As described above, the information processing apparatus 1A according to the second embodiment acquires a plane based on the image acquired by the image acquiring unit 171 acquiring the image of the area including the plane photographed by the imaging apparatus and the image acquired by the image acquiring unit 171. And a calculation unit 177 for calculating the degree of damage of the
Further, the calculation unit 177 calculates the degree of damage to the surface in each of the plurality of continuous images acquired by the image acquisition unit 171. Further, the information processing apparatus 1A further includes a state determination unit 1771 that determines the state of the surface in each image, and the calculation unit 177 calculates the state in each image based on the state of the surface in each image. Calculate the degree of surface damage.
As a result, surface damage can be automatically detected, and quantitative evaluation can be performed, whereby convenience in automatic detection of structure damage can be improved.

Third Embodiment
In the third embodiment, an example will be described in which one road surface image is divided into a plurality of regions, and each of the plurality of regions overlaps with another region in the image. In the third embodiment, parts different from the first embodiment will be mainly described.

FIG. 12 is a schematic block diagram showing an example of the configuration of an information processing apparatus 1B according to the third embodiment of the present invention.
The information processing apparatus 1 </ b> B includes an input unit 10, a communication unit 13, a storage unit 15, a control unit 17, and an output unit 19.
The control unit 17 is configured to include an image acquisition unit 171, an image extraction unit 173, a partial image extraction unit 175B, and a calculation unit 177. The calculation unit 177 includes a state determination unit 1771 and a degree calculation unit 1773.

  The partial image extraction unit 175B extracts, from each of the inspection image candidates, areas of a plurality of road surfaces that partially overlap with each other, based on the position information extracted from the metadata. In other words, the partial image extraction unit 175B extracts a plurality of areas including the road surface in a certain inspection image candidate, and in each of the plurality of areas, the vertexes and sides of one area are included in another area . The partial image extraction unit 175B specifies the inspection image candidate from which the plurality of regions are extracted as an inspection image, and information representing the inspection image, region information representing the plurality of regions extracted from the inspection image candidates, and position information Are output to the calculation unit 177.

The calculation unit 177 analyzes the damage of the road surface in the inspection image by deep learning. Specifically, the state determination unit 1771 determines in each of the plurality of regions whether or not the road surface in each inspection image is damaged by deep learning, and calculates the value of the determination score (certainty factor).
The degree calculator 1773 calculates a degree of damage, for example, a crack rate, based on the damage presence / absence information. Specifically, the degree calculation unit 1773 calculates the degree of damage by dividing the sum of determination score values in a predetermined section, for example, a 20 m section, by the number of areas in the inspection image in the predetermined section. The calculation unit 177 causes the output unit 19 to output the calculated degree of damage and the position information. Here, when the movement distance does not reach the predetermined section, the calculation unit 177 calculates the degree of damage for each inspection image.

  In addition, the calculation unit 177 may compare the calculated determination score value with a predetermined threshold value. For example, when the determination score value is equal to or greater than the predetermined threshold value, “1”, the determination score value is less than the predetermined threshold value When it is, it may be binarized like "0". In addition, when the value of the determination score is equal to or less than the predetermined threshold, the value of the determination score may be set to “0”.

FIG. 13 is a flowchart showing an example of damage degree calculation processing according to the third embodiment of the present invention.
In step S401, the image acquisition unit 171 acquires an image of one or more road surfaces via the input unit 10 or the communication unit 13.
In step S403, the image extraction unit 173 executes an image extraction process described later.
In step S405, the partial image extraction unit 175B extracts, from each of the inspection image candidates, a plurality of road surface areas overlapping with each other on the basis of the positional information extracted from the metadata. The partial image extraction unit 175B specifies the inspection image candidate from which the plurality of regions are extracted as an inspection image, and information representing the inspection image, region information representing the plurality of regions extracted from the inspection image candidates, and position information Are output to the calculation unit 177.

In step S407, the state determination unit 1771 determines whether or not the road surface in each inspection image is damaged by deep learning in each of the plurality of areas, and calculates the value of the determination score (certainty factor).
In step S409, the degree of damage is calculated by dividing the sum of the determination score values in a predetermined section, for example, a 20 m section, by the number of areas in the inspection image in the predetermined section.

FIG. 14 is a flowchart showing an example of damage degree calculation processing according to the third embodiment of the present invention.
In step S501, the image extraction unit 173 selects an image of the road surface of the frame. Specifically, when the road surface image is not selected, the image extracting unit 173 selects the road surface image of the first frame. On the other hand, when the frame has already been selected, the image extraction unit 173 selects the image of the road surface of the frame next to the frame being selected.

In step S503, the image extracting unit 173 refers to the position information corresponding to the image of the frame being selected, and calculates the movement distance from the position information corresponding to the immediately preceding frame. Hereinafter, this movement distance is referred to as “distance between images”.
In step S505, the image extraction unit 173 adds the calculated movement distance to the total movement distance. The movement distance sum is initialized at the start of the image extraction process.

  In step S507, the image extracting unit 173 compares the total movement distance with the predetermined set distance. If the total movement distance is larger than the set distance (step S507; YES), the control unit 17 executes the process of step S509. On the other hand, if the total movement distance is smaller than the set distance (step S507; NO), the control unit 17 executes the process at step S515.

In step S509, the image extraction unit 173 sets the total movement distance to 0 and initializes it.
In step S511, the image extraction unit 173 sets the image being selected as an examination image candidate.

In step S513, the image extraction unit 173 confirms the presence or absence of the image of the road surface of the next frame. When an image exists (step S513; YES), the control unit 17 executes the process of step S501. If an image does not exist (step S513; NO), the control unit 17 executes the process at step S515.
In step S515, the image extracting unit 173 outputs the inspection image candidate and the area information to the partial image extracting unit 175B.

As described above, the information processing apparatus 1B according to the third embodiment acquires a surface based on the image acquired by the image acquiring unit 171 acquiring the image of the area including the surface photographed by the imaging apparatus and the image acquired by the image acquiring unit 171. And a calculation unit 177 for calculating the degree of damage of the
The information processing apparatus 1B further includes a partial image generation unit (partial image extraction unit 175B) that divides the image acquired by the image acquisition unit 171 and generates a plurality of partial images that are a part of the image. The degree of damage to the surface in the partial image is calculated for each of the partial images generated by the partial image generation unit (partial image extraction unit 175B).
The information processing apparatus 1B further includes a state determination unit 1771 that determines the state of the surface in the partial image for each of the partial images generated by the partial image generation unit (partial image extraction unit 175B), and the calculation unit 177 Based on the state of the surface in the partial image, the degree of damage to the surface in the partial image is calculated.
The partial image generation unit (partial image extraction unit 175B) divides the image in the image acquired by the image acquisition unit 171, and divides the divided first partial image and the vertex portion or the side portion of the first partial image. And generating a second partial image to be included.
As a result, surface damage can be automatically detected, and quantitative evaluation can be performed, whereby convenience in automatic detection of structure damage can be improved.

  The state determination unit 1771 may calculate the damage class according to the damage type. Specifically, for example, planar cracks such as turtle crack and plural linear cracks, single linear cracks such as damage class “1”, longitudinal cracks, and transverse cracks, damage class “0.6” ", Damage class without damage such as cracks, etc., damage class per area may be calculated by machine learning like damage class" 0 ". In this case, the damage degree may be calculated using the value of the damage class instead of the value of the determination score in the third embodiment.

  In addition, although the damage degree demonstrated an example of the crack rate, the damage degree may be a pitting amount. Moreover, the damage degree may be a damage category other than the crack rate and the amount of pitting.

Fourth Embodiment
In the first to third embodiments, an example of calculating the degree of damage has been described.
In the fourth embodiment, an example in which the type of damage is determined will be described. Here, in the fourth embodiment, parts different from the first embodiment, the second embodiment, the third embodiment, and each modification will be mainly described.
Here, when the road surface is repaired, the work process, the number of work days, the cost, and the like pertaining to the repair change according to the type of damage to the road surface. Therefore, if the type of damage on the road surface can be detected, that is, if the condition of the road surface can be evaluated, these estimates can be made, and the convenience of the user can be improved.

In the following description, the state determination unit 1771 uses the road surface by combining the first embodiment, the second embodiment, the third embodiment, the contents described in each modification, or a part or all of them. An example of the case where one of the value of the determination score for each area, the value of the damage class, and the damage presence / absence information is calculated as the damage score will be described.
Further, the damage degree calculated by the degree calculation unit 1773 by combining the contents described in the first embodiment, the second embodiment, the third embodiment, the respective modifications, or a part or all of them, It may be used as a damage score. In this case, the road surface of the predetermined section for which the damage degree is calculated may be treated as an area of one road surface, or the predetermined section may be treated as a short section, and the road surface of the short predetermined section may be treated as an area of one road surface.

  In the fourth embodiment, the type of road surface damage is determined based on the distribution of the plurality of road surface areas and the damage scores in the respective areas. The types of damage to the road surface are, for example, longitudinal cracks, transverse cracks, tortoise cracks, longitudinal cracks and pittings, construction seams, depressions, generally sparse damage, and the like. This will be described with reference to FIGS. 15 to 22.

FIG. 15 is a view showing an example of the type of road surface damage according to the fourth embodiment of the present invention. FIG. 16 is a view showing another example of the type of road surface damage according to the fourth embodiment of the present invention. FIG. 17 is a view showing another example of the road damage type according to the fourth embodiment of the present invention. FIG. 18 is a view showing another example of the type of road surface damage according to the fourth embodiment of the present invention. FIG. 19 is a view showing another example of the type of road surface damage according to the fourth embodiment of the present invention. FIG. 20 is a view showing another example of the type of road surface damage according to the fourth embodiment of the present invention. FIG. 21 is a view showing another example of the type of road surface damage according to the fourth embodiment of the present invention. FIG. 22 is a view showing another example of the type of road surface damage according to the fourth embodiment of the present invention.
Here, each drawing represents a part of the road surface, and the squares in each drawing, that is, the direction in which the number of road surface areas is large is the running direction, and the direction in which the number of road surface areas is small is the width direction.

  The example illustrated in FIG. 15 is a longitudinal crack. The solid line shown represents a crack, and the crack is traversed across a plurality of regions indicated by hatches. The example illustrated in FIG. 16 is a transverse crack. The solid line shown represents a crack, and the crack crosses over a plurality of regions indicated by hatches. The example illustrated in FIG. 17 is a turtle crack. The solid line shown in the figure represents a crack, and the crack is generated in a lattice pattern, that is, a turtle-like shape, over a plurality of regions indicated by hatches. The example illustrated in FIG. 18 is longitudinal cracking and digging. The solid line shown in the figure represents a crack, and a plurality of cracks longitudinally cross over a plurality of regions indicated by hatches, and a plurality of cracks longitudinally occur substantially in parallel. The example illustrated in FIG. 19 is a construction joint. The solid lines shown represent joints created by construction, and joints are generated over a plurality of areas indicated by hatches, for example, over more than longitudinal cracks.

  The example illustrated in FIG. 20 is a recess. The solid line shown represents a crack, and a crack occurs in a mouth shape over a plurality of regions indicated by hatches. The example illustrated in FIG. 21 is a sparsely distributed injury throughout. The solid lines shown in the figure represent damage such as cracks, and occur sparsely in a plurality of areas indicated by hatches and in predetermined sections. The example illustrated in FIG. 22 is a turtle crack. The solid line shown in the figure represents a crack, and the crack is generated in a lattice pattern, that is, a turtle-like shape, over a plurality of regions indicated by hatches. Here, two types of hatches shown in FIG. 22 are an area where the damage score is equal to or more than a first predetermined value and less than a second predetermined value, and an area which is equal to or more than a second predetermined value. The hatch central portion is an area where the damage score is high, that is, an area where the damage score is equal to or more than a second predetermined value. In the case of the example illustrated in FIG. 22, the value of the determination score or the value of the damage class may be used as the damage score.

  The information processing apparatus 1C classifies and determines the type of damage to the road surface as shown in FIGS. 18 to 22 according to the damage score for each area of the road surface and the distribution of the area where the damage score is equal to or more than a predetermined value.

FIG. 23 is a schematic block diagram showing an example of the configuration of an information processing apparatus 1C according to the fourth embodiment of the present invention.
The information processing apparatus 1 </ b> C includes an input unit 10, a communication unit 13, a storage unit 15, a control unit 17, and an output unit 19. The control unit 17 includes a damage score calculation unit 177C and a classification unit 179. The classification unit 179 includes an integration unit 1791 and a classification determination unit 1793.

The damage score calculation unit 177C has a function of part or all of the control unit 17 of the above-described information processing device 1, 1A, 1B, and the first embodiment, the second embodiment, the third embodiment, The damage score is calculated by combining the contents described in each modification, or some or all of them. The specific description uses the description of each embodiment and each modification described above, and the description is omitted.
The damage score calculation unit 177C outputs the calculated damage score and information on the area of the road surface corresponding to the damage score to the classification unit 179.

  The classification unit 179 classifies the type of damage on the road surface according to the distribution of the damage score of each mesh. Specifically, the integration unit 1791 calculates the damage score for each mesh.

  The classification determination unit 1793 determines the type of damage by comparing the distribution of damage scores for each region in a predetermined section with the distribution of damage for each known damage type. Specifically, the classification determination unit 1793 determines that a longitudinal crack is generated, for example, when regions of damage scores equal to or greater than a predetermined value are distributed continuously in a predetermined number, for example, three or more in the longitudinal direction. In addition, for example, in a case where areas of damage scores equal to or more than a predetermined value are distributed continuously in a predetermined number, for example, three or more, in a lateral direction, the classification determination unit 1793 determines as a transverse crack. In addition, the classification determination unit 1793 may, for example, have a predetermined number, for example, two or more, vertically continuous areas with damage score areas having a predetermined value or more, and a predetermined number, for example, two or more areas. If the distribution is continuous in the lateral direction, it is judged to be turtle shell cracking.

  In addition, for example, when a plurality of longitudinal cracks are distributed in parallel, the classification determination unit 1793 determines that the longitudinal cracks are cracked and dug. In addition, for example, in a case where areas of damage scores equal to or larger than a predetermined value are distributed in a predetermined number, for example, six or more in a longitudinal direction, the classification determination unit 1793 determines that the area is a construction joint. In addition, the classification determination unit 1793 may, for example, have a predetermined number, for example, two or more, vertically continuous areas with damage score areas having a predetermined value or more, and a predetermined number, for example, two or more areas. If the distribution is continuous in the lateral direction and is distributed in the shape of the mouth, it is judged as a depression. The classification determination unit 1793 determines that the damage is sparse as a whole, for example, in a case where areas of damage scores equal to or larger than a predetermined value are distributed in a predetermined number or sparsely.

  Thus, the classification determination unit 1793 determines the type of damage to the road surface according to the distribution of the damage score area in the predetermined section. The classification determination unit 1793 outputs the determination result to the output unit 19.

The predetermined values used by the classification determination unit 1793 to determine the type of damage may be the same value or different values.
Note that the integration unit 1791 may not associate the position of the road surface to be determined with the area according to the position information of the area included in the information of each area. That is, the integration unit 1791 may not use the position information. In this case, the processing of the integration unit 1791 and the classification determination unit 1793 may be performed as the classification unit 179. For example, the classification unit 179 integrates the information of each area included in the information of the area of the road surface and the damage score of each area, associates each area of the road surface with each damage score, and The process of 1793 may be performed.

FIG. 24 is a flowchart showing an example of damage type determination processing according to the fourth embodiment of the present invention.
In step S601, the damage score calculation unit 177C has a function of part or all of the control unit 17 of the above-described information processing device 1, 1A, 1B, and the first embodiment, the second embodiment, the third embodiment. The damage score is calculated by combining the embodiment described above, the contents described in each modification, or some or all of them.
In step S603, the classification unit 179 integrates the information of each area included in the information of the area of the road surface and the damage score of each area.
In step S605, the classification unit 179 determines which type of damage among the damage types of the plurality of road surfaces according to the distribution of damage scores for each area in a predetermined section. Further, the determination based on the distribution may be performed on a rule basis based on a known distribution as described above, or may be performed on deep learning based on teacher data of the distribution.

As described above, according to the fourth embodiment, the information processing apparatus 1C is based on the image acquired by the image acquisition unit 171 which acquires an image of the area including the surface imaged by the imaging apparatus, and the image acquired by the image acquisition unit 171. And a type determination unit (classification determination unit 1793) that determines the type of damage on the surface according to the degree of damage calculated by the calculation unit 177.
The information processing apparatus 1C further includes a partial image generation unit (partial image extraction unit 175B) that divides the image acquired by the image acquisition unit 171 and generates a plurality of partial images that are a part of the image. The (classification determination unit 1793) determines the damage type of the surface based on the damage degree of the surface in the partial image extracted by the partial image generation unit (partial image extraction unit 175B).

In addition, the type determination unit (classification determination unit 1793) determines the type of damage on the surface based on the distribution of partial images in which the degree of damage is equal to or greater than a predetermined value.
In addition, the type determination unit (classification determination unit 1793) integrates the damage degree of the surface in each of the plurality of partial images generated from one image, and determines the damage type of the surface.
In addition, the type determination unit (classification determination unit 1793) determines the type of surface damage based on the damage level of the surface in each of the plurality of continuous images acquired by the image acquisition unit 171.
In addition, the type determination unit (classification determination unit 1793) determines the type of damage on the surface based on the distribution of a plurality of images in which the degree of damage is equal to or greater than a predetermined value.
In addition, the type determination unit (classification determination unit 1793) integrates the degree of damage to the surface in each of the plurality of continuous images to determine the type of surface damage.
Also, the calculation unit 177 calculates the degree of damage by deep learning.

  As a result, since it is possible to classify which damage it is, it is possible to improve the convenience of the user.

  In each of the above embodiments and modifications, the position information and the damage type of the road surface or / and the damage degree of the road surface may be associated with each other and output, for example, on the map. The damage type or / and the road surface damage degree may be correlated and output as a list.

  In each embodiment and each modification, the value of the determination score may be multiplied by the value of the determination class as a weight to be used as a damage score or to calculate the degree of damage.

  In the above-described embodiments and modifications, in the case where the amount of burrow is used as the degree of damage, the depth of the burrow may be calculated by deep learning.

  Although each embodiment and each modification were explained, it is an example and it is not restricted to these, for example, either of each embodiment or each modification, a part of each embodiment, or each modification A part may be combined with one or more other embodiments or one or more other variations to realize one aspect of the present invention.

  Note that a program for executing each process of the information processing apparatus 1 in the present embodiment is recorded in a computer readable recording medium, and the computer system reads the program recorded in the recording medium and executes it. The above-described various processes related to the information processing apparatus 1 may be performed by the above.

  Note that the “computer system” referred to here may include an OS and hardware such as peripheral devices. The "computer system" also includes a homepage providing environment (or display environment) if the WWW system is used. In addition, “computer readable recording medium” means flexible disk, magneto-optical disk, ROM, writable nonvolatile memory such as flash memory, portable medium such as CD-ROM, hard disk incorporated in computer system, etc. Storage devices.

  Furthermore, “computer readable recording medium” refers to volatile memory (for example, DRAM (Dynamic Memory) inside a computer system serving as a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line). As Random Access Memory), it is assumed that the program which holds the program for a fixed time is included. The program may be transmitted from a computer system in which the program is stored in a storage device or the like to another computer system via a transmission medium or by transmission waves in the transmission medium.

  Here, the “transmission medium” for transmitting the program is a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. Further, the program may be for realizing a part of the functions described above. Furthermore, it may be a so-called difference file (difference program) that can realize the above-described functions in combination with a program already recorded in the computer system.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design and the like within the scope of the present invention.

sys information processing system 1, 1A, 1B, 1C information processing apparatus 10 input unit 13 communication unit 15 storage unit 17 control unit 171 image acquisition unit 173 image extraction unit 175 area extraction unit 175A overlapping area extraction unit 175B partial image extraction unit 177 calculation Unit 1771 State determination unit 1773 Degree calculation unit 177C Damage score calculation unit 179 Classification unit 1791 Integration unit 1793 Classification determination unit 19 Output unit

Claims (11)

An image acquisition unit that acquires an image of an area including a surface captured by an imaging device;
A plurality of images are selected, the selected inter-image distance is compared with a predetermined set distance, and when the inter-image distance is larger than the set distance, the image being selected is set as an examination image candidate, An image extraction unit which does not set an image being selected as an inspection image candidate when the inter-distance is smaller than a set distance;
Based on the position information corresponding to the inspection image candidate, at least a part of the area of the road surface in the image of one road surface and the area of the road surface in the image of another road overlap between the images of a plurality of road surfaces. An overlapping area extraction unit that extracts an inspection image from among the inspection image candidates extracted by the image extraction unit;
A state determination unit that determines the state of the surface in the inspection image for each of the inspection images extracted by the overlapping area extraction unit;
And a calculation unit that calculates the degree of damage to the surface in the inspection image based on the state of the surface in the inspection image for each of the inspection images .
The predetermined set distance is
By setting the larger the recognition accuracy of the road surface damage in one inspection image, the duplication of the road surface shown in each inspection image is reduced.
By setting the smaller the lower the road surface damage recognition accuracy in one inspection image, the road surface overlap shown in each inspection image is increased.
Information processing device. The image processing apparatus further includes a classification determination unit that classifies any one of a plurality of stages according to the degree of damage to the surface in the partial image.
An information processing apparatus according to claim 1 . In the image acquired by the image acquisition unit, a part that divides the image and generates a divided first partial image and a second partial image including a vertex part or a side part of the first partial image Including an image generation unit,
The information processing apparatus according to claim 1 . The partial image generation unit divides the image between the images acquired by the image acquisition unit, and divides the image into a first partial image and a second portion including a vertex portion or a side portion of the first partial image. Generate an image,
The information processing apparatus according to claim 3. The calculation unit calculates a degree of damage to a surface in each of the plurality of continuous images acquired by the image acquisition unit.
An information processing apparatus according to claim 1. A state determination unit that determines the state of the surface in each of the images;
The calculation unit calculates the degree of damage to the surface in each of the images based on the state of the surface in each of the images.
The information processing apparatus according to claim 5 . The image processing apparatus further includes a classification determination unit that classifies any one of a plurality of stages according to the damage degree of the surface in each of the images.
The information processing apparatus according to claim 5 or claim 6. The first partial image obtained by dividing the image and dividing the plurality of continuous images obtained by the image acquisition unit, and a second partial image including a vertex portion or a side portion of the first partial image; To generate
The information processing apparatus according to any one of claims 5 to 7 . An image acquisition unit that acquires an image of an area including a surface captured by an imaging device;
A plurality of images are selected, the selected inter-image distance is compared with a predetermined set distance, and when the inter-image distance is larger than the set distance, the image being selected is set as an examination image candidate, An image extraction unit which does not set an image being selected as an inspection image candidate when the inter-distance is smaller than a set distance;
At least a part of an area of a road surface in an image of a road surface and an area of a road surface in an image of another road surface overlap between images of a plurality of road surfaces based on position information corresponding to the inspection image candidate As described above, an overlapping area extraction unit that extracts an inspection image from among the inspection image candidates extracted by the image extraction unit;
A state determination unit that determines the state of the surface in the inspection image for each of the inspection images extracted by the overlapping area extraction unit;
And a calculation unit that calculates the degree of damage to the surface in the inspection image based on the state of the surface in the inspection image for each of the inspection images .
The predetermined set distance is
By setting the larger the recognition accuracy of the road surface damage in one inspection image, the duplication of the road surface shown in each inspection image is reduced.
By setting the smaller the lower the road surface damage recognition accuracy in one inspection image, the road surface overlap shown in each inspection image is increased.
Information processing system. The computer of the information processing apparatus is
An image acquisition process of acquiring an image of an area including a surface imaged by an imaging device;
A plurality of images are selected, the selected inter-image distance is compared with a predetermined set distance, and when the inter-image distance is larger than the set distance, the image being selected is set as an examination image candidate, An image extraction process in which the image being selected is not set as an inspection image candidate when the inter-distance is smaller than the set distance;
At least a part of an area of a road surface in an image of a road surface and an area of a road surface in an image of another road surface overlap between images of a plurality of road surfaces based on position information corresponding to the inspection image candidate Thus, an overlapping area extraction process of extracting an inspection image from among the inspection image candidates extracted in the image extraction process;
A state determination step of determining the state of the surface in the inspection image for each of the inspection images extracted in the overlapping region extraction step;
Calculating , for each of the inspection images, the degree of damage to the surface in the inspection image based on the state of the surface in the inspection image ;
The predetermined set distance is
By setting the larger the recognition accuracy of the road surface damage in one inspection image, the duplication of the road surface shown in each inspection image is reduced.
By setting the smaller the lower the road surface damage recognition accuracy in one inspection image, the road surface overlap shown in each inspection image is increased.
Information processing method. In the computer of the information processing apparatus
An image acquisition process of acquiring an image of an area including a surface imaged by an imaging device;
A plurality of images are selected, the selected inter-image distance is compared with a predetermined set distance, and when the inter-image distance is larger than the set distance, the image being selected is set as an examination image candidate, An image extraction process in which the image being selected is not set as an inspection image candidate when the inter-distance is smaller than the set distance;
At least a part of an area of a road surface in an image of a road surface and an area of a road surface in an image of another road surface overlap between images of a plurality of road surfaces based on position information corresponding to the inspection image candidate Thus, an overlapping area extraction process of extracting an inspection image from among the inspection image candidates extracted in the image extraction process;
A state determination step of determining the state of the surface in the inspection image for each of the inspection images extracted in the overlapping region extraction step;
A calculation process of calculating the degree of damage to the surface in the inspection image based on the state of the surface in the inspection image for each of the inspection images ;
The predetermined set distance is
By setting the larger the recognition accuracy of the road surface damage in one inspection image, the duplication of the road surface shown in each inspection image is reduced.
By setting the smaller the lower the road surface damage recognition accuracy in one inspection image, the road surface overlap shown in each inspection image is increased.
program.