JP6764842B2 - Information processing equipment, information processing system, information processing method, and information processing program - Google Patents

Description

The present invention relates to an information processing device, an information processing system, an information processing method, and an information processing program.

With the aging of roads, bridges, facilities, etc., it is required to efficiently maintain buildings. In order to carry out maintenance of a building, it is necessary to detect damage to the building, but it is difficult to quantitatively evaluate or comprehensively analyze the damage visually. Therefore, automatic detection by a machine is required. In Patent Document 1, for example, an in-vehicle camera captures a road surface image of a road surface with a sun and a road surface image of a road surface with a shade so that some areas in the front, rear, left, and right images overlap. The road surface image is joined by performing keystone correction, calculation of the degree of coincidence of overlapping areas, and the like.

Japanese Unexamined Patent Publication No. 2011-90367

However, if the image subjected to the joining process is analyzed as it is by using the technique of Patent Document 1, the load required for the analysis becomes large. In addition, it was not possible to automatically detect damage to buildings using only such images that had been joined, and it was not possible to quantitatively evaluate or comprehensively analyze damage to buildings. As described above, there is a problem that the convenience in the automatic detection of damage to the building is not sufficient.

One aspect of the present invention has been made in view of the above-mentioned problems, and is an information processing device, an information processing system, an information processing method, and an information processing method capable of improving convenience in automatic detection of damage to a building. One of the purposes is to provide a program.

(1) The present invention has been made in order to solve the above problems, one aspect of the present invention includes an image acquisition unit that acquires an image of an area including the road surface captured by the imaging device, a plurality of Select an image, compare the inter-image distance, which is the distance of the position information corresponding to each selected image, with the predetermined set distance, and if the inter-image distance is larger than the set distance, it is being selected. An image is set as an inspection image candidate, and when the distance between images is smaller than the set distance, the selected image is not set as an inspection image candidate. For the image extraction unit and the position information corresponding to the inspection image candidate. Based on this, the inspection image extracted by the image extraction unit so that at least a part of the road surface region in the image of one road surface and the road surface region in the image of another road surface overlap between the images of the plurality of road surfaces. and overlap region extraction unit that extracts an inspection image from the candidate, and the overlap for each of the test images extracted by the area extracting unit, the inspection of the state of the road surface is judged state determining section in the image, the inspection image for each of, based on the state of the road surface in the inspection image, a calculation unit that calculates damage degree of the road surface in the inspection image, damage of the road surface in accordance with the degree of damage to the calculating unit has calculated A type determination unit for determining the type is provided, and the predetermined set distance is set larger as the recognition accuracy of road surface damage in one inspection image is higher, thereby reducing duplication of the road surface reflected in each inspection image. The lower the recognition accuracy of road surface damage in one inspection image, the smaller the setting, so that the overlap of the road surface reflected in each inspection image is increased, and the type determination unit is calculated by the calculation unit in a plurality of road surface areas. The damage type is determined based on the number of consecutive regions in which the degree of damage is equal to or greater than a predetermined value, the directions between regions in which the degree of continuous damage is equal to or greater than a predetermined value, and the dispersion of the entire region of a plurality of road surfaces. It is an information processing device that makes a judgment.

( 2 ) Further, one aspect of the present invention is the information processing apparatus described above, and the calculation unit calculates the degree of damage by deep learning.

(3) Further, an embodiment of the present invention includes an imaging apparatus, an image acquiring unit that acquires an image of an area including the road surface captured by the imaging device, select a group of images, each being selected The distance between images, which is the distance of the position information corresponding to the image, is compared with the predetermined set distance, and if the distance between images is larger than the set distance, the selected image is set as an inspection image candidate, and the distance between images is set. When the distance is smaller than the set distance, the selected image is not set as the inspection image candidate.The image extraction unit and the image of a plurality of road surfaces based on the position information corresponding to the inspection image candidate, An overlapping area for extracting an inspection image from the inspection image candidates extracted by the image extraction unit so that at least a part of the road surface area in the image of a certain road surface and the road surface area in the image of another road surface overlap. an extraction unit, for each of the test images extracted by the overlapping area extracting unit, and the inspection of the condition of the road surface is judged state determining section in the image, for each of the test image, road surfaces in the inspection image based on the state, comprising: a calculation unit for calculating a degree of damage of the road surface in the inspection image, and a type determination unit determines the damage type of the road surface in accordance with the degree of damage to the calculating unit has calculated The predetermined distance is set larger as the recognition accuracy of road surface damage in one inspection image is higher, so that the overlap of the road surface reflected in each inspection image is reduced and the recognition accuracy of road surface damage in one inspection image is improved. The lower the setting, the smaller the overlap of the road surface reflected in each inspection image, and the type determination unit is a region of a plurality of road surfaces in which the degree of damage calculated by the calculation unit is equal to or greater than a predetermined value. It is an information processing system that determines the damage type based on the integration of the number of continuations, the directions of regions in which the degree of continuous damage is equal to or higher than a predetermined value, and the dispersion of the entire region of a plurality of road surfaces.

(4) Further, an embodiment of the present invention, a computer of the information processing apparatus, an image acquisition step of acquiring an image of an area including the road surface captured by the imaging device, select a group of images, select The distance between images, which is the distance of the position information corresponding to each image, is compared with the predetermined set distance, and if the distance between images is larger than the set distance, the selected image is set as an inspection image candidate. , When the distance between images is smaller than the set distance, the selected image is not set as an inspection image candidate. Based on the image extraction process and the position information corresponding to the inspection image candidate, images of a plurality of road surfaces An inspection image is extracted from the inspection image candidates extracted by the image extraction process so that at least a part of the road surface area in the image of a certain road surface and the road surface area in the image of another road surface overlap between the two. and overlap region extraction step of, for each of the test images extracted by the overlap region extraction process, a state determining state determination process of the road surface in the inspection image, for each of the test image, the test in the image based of the state of the road surface, the calculation step of calculating a degree of damage of the road surface in the inspection image, and determines the type determining process damage type of the road surface in accordance with the degree of damage was calculated by the calculating step , And the predetermined set distance is set larger as the recognition accuracy of the road surface damage in one inspection image is higher, thereby reducing the overlap of the road surface reflected in each inspection image and reducing the road surface damage in one inspection image. The lower the recognition accuracy is, the smaller the setting is made, so that the road surface overlaps in each inspection image is increased. In the type determination process, the degree of damage calculated by the calculation process is a predetermined value or more in a plurality of road surface areas. It is an information processing method that determines the damage type based on the number of consecutive regions, the directions of regions in which the degree of continuous damage is equal to or higher than a predetermined value, and the dispersion of the entire region of a plurality of road surfaces. ..

(5) Further, an embodiment of the present invention, a computer of the information processing apparatus, an image acquiring step of acquiring an image of an area including the road surface captured by the imaging device, select a group of images, select The distance between images, which is the distance of the position information corresponding to each image, is compared with the predetermined set distance, and if the distance between images is larger than the set distance, the selected image is set as an inspection image candidate. , When the distance between images is smaller than the set distance, the selected image is not set as the inspection image candidate. Based on the image extraction step and the position information corresponding to the inspection image candidate, images of a plurality of road surfaces. An inspection image is extracted from the inspection image candidates extracted by the image extraction step so that at least a part of the road surface area in the image of a certain road surface and the road surface area in the image of another road surface overlap between the two. and overlap region extraction step of, for each of the test images extracted by the overlap region extraction step, a state determining state determination step of the road surface in the inspection image, for each of the test image, the test in the image based on the state of the road surface of a calculation step of calculating a degree of damage of the road surface in the inspection image, and determines the type determination step damage type of the road surface in accordance with the degree of damage was calculated by the calculating step , And the predetermined set distance is set larger as the recognition accuracy of road surface damage in one inspection image is higher, thereby reducing the overlap of the road surface reflected in each inspection image and reducing the road surface damage in one inspection image. The lower the recognition accuracy is, the smaller the setting is made so that the road surface overlaps in each inspection image is increased. In the type determination step, the degree of damage calculated by the calculation step is equal to or more than a predetermined value in a plurality of road surface areas. It is a program that determines the damage type based on the number of consecutive regions, the directions of regions in which the degree of continuous damage is equal to or higher than a predetermined value, and the dispersion of the entire region of a plurality of road surfaces.

According to the present invention, convenience in automatic detection of damage to a building can be improved.

It is the schematic which shows an example of the structure of the information processing system which concerns on 1st Embodiment of this invention. It is explanatory drawing which shows an example of the image of the road surface which concerns on 1st Embodiment of this invention. It is explanatory drawing which shows another example of the image of the road surface which concerns on 1st Embodiment of this invention. It is explanatory drawing which shows another example of the image of the road surface which concerns on 1st Embodiment of this invention. It is explanatory drawing which shows another example of the image of the road surface which concerns on 1st Embodiment of this invention. It is explanatory drawing which shows another example of the image of the road surface which concerns on 1st Embodiment of this invention. It is a schematic block diagram which shows an example of the structure of the information processing apparatus which concerns on 1st Embodiment of this invention. It is a flowchart which shows an example of the calculation process of the degree of damage which concerns on 1st Embodiment of this invention. It is a schematic block diagram which shows an example of the structure of the information processing apparatus which concerns on 2nd Embodiment. It is a flowchart which shows an example of the calculation process of the degree of damage which concerns on 2nd Embodiment of this invention. It is a flowchart which shows an example of the calculation process of the degree of damage which concerns on 2nd Embodiment of this invention. It is a schematic block diagram which shows an example of the structure of the information processing apparatus which concerns on 3rd Embodiment of this invention. It is a flowchart which shows an example of the damage degree calculation process which concerns on 3rd Embodiment of this invention. It is a flowchart which shows an example of the damage degree calculation process which concerns on 3rd Embodiment of this invention. It is a figure which shows an example of the damage type of the road surface which concerns on 4th Embodiment of this invention. It is a figure which shows another example of the damage type of the road surface which concerns on 4th Embodiment of this invention. It is a figure which shows another example of the damage type of the road surface which concerns on 4th Embodiment of this invention. It is a figure which shows another example of the damage type of the road surface which concerns on 4th Embodiment of this invention. It is a figure which shows another example of the damage type of the road surface which concerns on 4th Embodiment of this invention. It is a figure which shows another example of the damage type of the road surface which concerns on 4th Embodiment of this invention. It is a figure which shows another example of the damage type of the road surface which concerns on 4th Embodiment of this invention. It is a figure which shows another example of the damage type of the road surface which concerns on 4th Embodiment of this invention. It is a schematic block diagram which shows an example of the structure of the information processing apparatus which concerns on 4th Embodiment of this invention. It is a flowchart which shows an example of the damage type determination process which concerns on 4th Embodiment of this invention.

Hereinafter, each embodiment of the present invention will be described with reference to the drawings.

<First Embodiment>
FIG. 1 is a schematic view showing an example of the configuration of the information processing system systems according to the first embodiment of the present invention.
The information processing system systems includes at least the information processing device 1. The information processing system systems is a system that analyzes the state of the surface of a building. In the following, as an example, a case where the information processing system systems analyzes damage on the surface of a building will be described.
Buildings are, for example, roads, bridges, facilities, etc., and also include social infrastructure. The surface is the outer surface or the inner surface of the building. That is, the surface of a building is, for example, a road surface or a bridge, an upper surface, a side surface, or a lower surface of a facility. Damage is physical unevenness, color change, cracks, breakage, and the like. In the following description, a case of evaluating and analyzing road surface damage as an example will be described, but the present invention is not limited to this, and can be applied to the evaluation and analysis of damage to various buildings.

Road surface damage includes, for example, road surface ruts, potholes, cracks (also referred to as cracks), steps, construction seams, and the like. The cracks include linear cracks and planar cracks. The linear cracks include longitudinal cracks, transverse cracks, and the like. Planar cracks include hexagonal cracks and the like.

In the information processing system systems, the information processing device 1 acquires images P of one or a plurality of road surfaces. When the information processing system systems includes an image pickup device, the road surface image P may be captured and acquired by the image pickup device, or the road surface image taken by a fixed point camera, a user's handheld camera, an in-vehicle camera, or the like. Image P may be acquired via a network or input interface.

The information processing device 1 calculates the degree of damage to the road surface in the image based on the acquired image of the road surface.

FIG. 2 is an explanatory view showing an example of an image of a road surface according to the first embodiment of the present invention. FIG. 3 is an explanatory view showing another example of the image of the road surface according to the first embodiment of the present invention. FIG. 4 is an explanatory view showing another example of the image of the road surface according to the first embodiment of the present invention. FIG. 5 is an explanatory view showing another example of an image of the road surface according to the first embodiment of the present invention. FIG. 6 is an explanatory view showing another example of the image of the road surface according to the first embodiment of the present invention.
The road surface image P1 shown in FIG. 2 is an example of rut D1. The image P2 of the road surface shown in FIG. 3 is an example of the pothole D2. The image P3 of the road surface shown in FIG. 4 is an example of the crack D3. The image P4 of the road surface shown in FIG. 5 is an example of the step D4. The road surface image P5 shown in FIG. 6 is an example of patching D5. In addition to the examples shown in FIGS. 2 to 6, the information processing device 1 evaluates and analyzes the degree of damage to various road surfaces described above.

Next, the outline of the analysis of road surface damage using the information processing system systems will be described.

First, the user takes an image of the road surface. The image of the road surface includes an image including the road surface, for example, an image taken by an in-vehicle camera, an image taken by a moving body such as an image taken while moving the road surface or the shoulder, or an image taken by a fixed point camera. If it is, it may be taken by any shooting method. The image is not limited to a still image and may be a moving image. The moving body is, for example, a vehicle, a person, a flying object, an unmanned photographing device, an animal, or the like. For example, when shooting with an in-vehicle camera or when the user shoots while moving on the road surface or shoulder, the road surface is appropriately shot according to the movement of the vehicle or the user.

In the following description, an example of the case where the position information and the time stamp are stored in the standard format (for example, EXIF) for the image of the road surface will be described, but the format and the storage method of the position information and the time stamp will be described. And the extraction method is arbitrary. Further, the position information and the time stamp may be absolute values or relative values.
The position information and the time stamp may be acquired by using a photographing device or an external device equipped with a GPS (Global Positioning System) sensor.

Next, the information processing device 1 acquires an image of the road surface via an input interface such as a communication network or a memory card. The information processing device 1 analyzes, for example, the acquired image of the road surface and analyzes the damage to the road surface in the image. The information processing device 1 calculates, for example, the feature amount of the road surface in the image and analyzes the state of the road surface, or analyzes the state of the road surface by using artificial intelligence. Artificial intelligence includes, for example, a multi-layer perceptron (multi-layer neural network). In artificial intelligence, deep learning is performed on the acquired road surface image based on a teacher image (for example, images P1 to P5, etc.) obtained by capturing an analysis target such as road surface damage. In addition, any technique, framework, or the like used in the conventional artificial intelligence may be applied to the artificial intelligence.
The information processing device 1 outputs the degree of damage to the road surface, the type of damage to the road surface, and the like as the analysis result of the image of the road surface. In this embodiment, an example of analysis by deep learning will be described.

FIG. 7 is a schematic block diagram showing an example of the configuration of the information processing apparatus 1 according to the first embodiment of the present invention.
The information processing device 1 includes an input unit 10, a communication unit 13, a storage unit 15, a control unit 17, and an output unit 19. The control unit 17 includes an image acquisition unit 171, an image extraction unit 173, a region extraction unit 175, and a calculation unit 177. The calculation unit 177 includes a state determination unit 1771 and a degree calculation unit 1773.

The input unit 10 accepts input of various information. The input unit 10 is, for example, a pointing device such as a mouse or a touch pad, an input interface for acquiring information from a keyboard, an external device, or the like.

The communication unit 13 transmits and receives various information to and from other devices. The communication unit 13 includes, for example, a communication IC (Integrated Circuit), a communication port, and the like.

The storage unit 15 stores various information. The storage unit 15 includes, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. In addition, the storage unit 15 may include an HDD (Hard Disk Drive), an EEPROM (Electrically Erasable Programmable Read Only Memory), a flash memory, and the like. The storage unit 15 stores various programs for execution by the CPU (Central Processing Unit, not shown) or GPU (Graphics Processing Unit, not shown) included in the information processing device 1, and the results of processing executed by the CPU or GPU. To do. Further, the storage unit 15 stores the analysis result (learning result) by the information processing device 1. The analysis result is, for example, information indicating the state of the network of the neural network. The information indicating the network state of the neural network is, for example, information indicating the value of the parameter of the activation function related to each node (unit) constituting the neural network.

The output unit 19 outputs various information. The output unit 19 is, for example, a display device such as a liquid crystal display, an organic EL display, or a plasma display, an output interface for outputting analysis results, and the like.

The control unit 17 controls the information processing device 1. Specifically, for example, a program stored in advance in the storage unit 15 is controlled by being executed by a CPU or GPU. In addition, a part or all of the control unit 17 may be realized as an integrated circuit of hardware such as LSI (Large Scale Integration) and ASIC (Application Specific Integrated Circuit).
The control unit 17 includes an image acquisition unit 171, an image extraction unit 173, a region extraction unit 175, and a calculation unit 177. The calculation unit 177 includes a state determination unit 1771 and a degree calculation unit 1773.

The image acquisition unit 171 acquires an image of one or more road surfaces via the input unit 10 and the communication unit 13. The image acquisition unit 171 outputs information representing the acquired image of the road surface to the image extraction unit 173.

When information representing the image of the road surface is input from the image acquisition unit 171, the image extraction unit 173 performs various image processing on the image of the road surface. The image extraction unit 173 converts each frame into a still image, for example, when the image of the road surface is an encoded moving image. Further, the image extraction unit 173 extracts time stamps, position information, and the like by referring to the metadata of the road surface image included in the information representing the road surface image. The image extraction unit 173 executes processing such as identification of inspection image candidates and extraction of the inspection image candidates from the acquired series of road surface images. The image extraction unit 173 may execute processing such as identification and extraction of an image that does not analyze the presence or absence of road surface damage from the acquired series of road surface images.

The area extraction unit 175 was extracted by the image extraction unit 173 based on the position information extracted from the metadata so that there is no overlapping road surface area between the images of the plurality of road surfaces and there is no missing road surface area. An inspection image is extracted from the inspection image candidates. Here, the space between the images of the plurality of road surfaces is the space between the images of the plurality of frames. The area extraction unit 175 outputs information representing the extracted inspection image and the corresponding position information to the calculation unit 177.
The area extraction unit 175 does not use the position information, and the inspection image extracted by the image extraction unit 173 so that there is no overlapping road surface area between the images of the plurality of road surfaces and there is no missing road surface area. An inspection image may be extracted from the candidates. In this case, for example, correspondence point matching between a certain inspection image candidate and a continuous inspection image candidate with the inspection image candidate may be used.

The calculation unit 177 analyzes the damage on the road surface in the inspection image by deep learning. Specifically, the state determination unit 1771 determines whether or not the road surface in each inspection image is damaged by deep learning, sets the inspection image in which the road surface damage is detected as "1", and detects the road surface damage. The inspection image that has not been performed is binarized as “0” to calculate the damage presence / absence information. The degree calculation unit 1773 calculates the degree of damage based on the damage presence / absence information. Specifically, the degree calculation unit 1773 calculates the degree of damage by dividing the sum of the damage presence / absence information in the predetermined section, for example, the 20 m section by the number of inspection images in the predetermined section. The calculation unit 177 causes the output unit 19 to output the calculated damage degree and the position information.

The calculation unit 177 may output the damage presence / absence information to the output unit 19 as it is without calculating the damage degree.
The calculation unit 177 may compare the calculated damage degree with a plurality of threshold values, determine which threshold value the damage degree is within, and classify the damage degree into a plurality of damage categories according to the determination result. Good. The plurality of damage categories may be, for example, gradual categories such as category 1 having a small degree of damage, category 2 having a moderate degree of damage, and category 3 having a large degree of damage. In this case, the calculation unit 177 may output the damage classification instead of or in addition to outputting the damage degree to the output unit 19.

FIG. 8 is a flowchart showing an example of the damage degree calculation process according to the first embodiment of the present invention.
In step S101, the image acquisition unit 171 acquires an image of one or more road surfaces via the input unit 10 and the communication unit 13.
In step S103, when the image extraction unit 173 inputs information representing the image of the road surface from the image acquisition unit 171, the image extraction unit 173 performs various image processing on the image of the road surface. Further, the image extraction unit 173 extracts time stamps, position information, and the like by referring to the metadata of the road surface image included in the information representing the road surface image. The image extraction unit 173 executes processing such as identification of inspection image candidates and extraction of the inspection image candidates from the acquired series of road surface images.

In step S105, the area extraction unit 175 is an image extraction unit so that there is no overlapping road surface area between the images of the plurality of road surfaces and there is no missing road surface area based on the position information extracted from the metadata. An inspection image is extracted from the inspection image candidates extracted by 173.
In step S107, the state determination unit 1771 determines whether or not the road surface in each inspection image is damaged by deep learning, sets the inspection image in which the road surface damage is detected as “1”, and the road surface damage is detected. The damage presence / absence information is calculated by binarizing the unchecked inspection image as “0”.

In step S109, the degree calculation unit 1773 calculates the degree of damage based on the damage presence / absence information. Specifically, the degree calculation unit 1773 calculates the degree of damage by dividing the sum of the damage presence / absence information in the predetermined section, for example, the 20 m section by the number of inspection images in the predetermined section.

As described above, the information processing apparatus 1 according to the first embodiment is based on the image acquisition unit 171 that acquires an image of the region including the surface captured by the image pickup apparatus and the image acquired by the image acquisition unit 171. It is provided with a calculation unit 177 for calculating the degree of damage of the above.
As a result, damage to the surface can be automatically detected and quantitative evaluation can be performed, so that convenience in automatic detection of damage to the building can be improved.

<Second embodiment>
In the second embodiment, an example will be described in which an inspection image is extracted by overlapping a part of a region between images of a plurality of road surfaces. In the second embodiment, a part different from the first embodiment will be mainly described.

FIG. 9 is a schematic block diagram showing an example of the configuration of the information processing apparatus 1A according to the second embodiment.
The information processing device 1A includes an input unit 10, a communication unit 13, a storage unit 15, a control unit 17, and an output unit 19. The control unit 17 includes an image acquisition unit 171, an image extraction unit 173, an overlapping region extraction unit 175A, and a calculation unit 177. The calculation unit 177 includes a state determination unit 1771 and a degree calculation unit 1773.

The overlapping area extraction unit 175A is based on the position information extracted from the metadata, and is a road surface that overlaps the road surface area in the image of a certain road surface and the road surface area in the image of another road surface between the images of the plurality of road surfaces. The inspection image is extracted from the inspection image candidates extracted by the image extraction unit 173 so that at least a part of the region overlaps. In other words, the overlapping area extraction unit 175A extracts the inspection image from the inspection image candidates so that the road surface regions overlap in time between the images of the plurality of road surfaces. Further, in each inspection image extracted by the overlapping region extraction unit 175A, the apex portion and the side portion of the road surface region of the image of a certain road surface are included in the road surface region of the image of another road surface between the images of the plurality of road surfaces. .. The overlapping area extraction unit 175A outputs information representing the extracted inspection image and the corresponding position information to the calculation unit 177.

FIG. 10 is a flowchart showing an example of the damage degree calculation process according to the second embodiment of the present invention.
In step S201, the image acquisition unit 171 acquires an image of one or more road surfaces via the input unit 10 and the communication unit 13.
In step S203, the image extraction unit 173 executes an image extraction process described later.
In step S205, the overlapping area extraction unit 175A sets the road surface area in the image of a certain road surface and the road surface area in the image of another road surface between the images of the plurality of road surfaces based on the position information extracted from the metadata. The inspection image is extracted from the inspection image candidates extracted by the image extraction unit 173 so that at least a part of the overlapping road surface areas overlaps.

In step S207, the state determination unit 1771 determines whether or not the road surface in each inspection image is damaged by deep learning, sets the inspection image in which the road surface damage is detected as “1”, and the road surface damage is detected. The damage presence / absence information is calculated by binarizing the unchecked inspection image as “0”.
In step S209, the degree calculation unit 1773 calculates the degree of damage based on the damage presence / absence information. Specifically, the degree calculation unit 1773 calculates the degree of damage by dividing the sum of the damage presence / absence information in the predetermined section, for example, the 20 m section by the number of inspection images in the predetermined section.

FIG. 11 is a flowchart showing an example of the damage degree calculation process according to the second embodiment of the present invention.
In step S301, the image extraction unit 173 selects an image of the road surface of the frame. Specifically, when the road surface image is not selected, the image extraction unit 173 selects the road surface image of the first frame. On the other hand, when the frame has already been selected, the image extraction unit 173 selects the image of the road surface of the frame next to the selected frame.

In step S303, the image extraction unit 173 refers to the position information corresponding to the image of the selected frame, and calculates the moving distance from the position information corresponding to the previous frame. Hereinafter, this moving distance is referred to as "distance between images".
In step S305, the image extraction unit 173 adds the calculated movement distance to the total movement distance. The total travel distance is initialized at the start of the image extraction process.

In step S307, the image extraction unit 173 compares the total movement distance with the predetermined set distance. When the total movement distance is larger than the set distance (step S307; YES), the control unit 17 executes the process of step S309. On the other hand, when the total movement distance is smaller than the set distance (step S307; NO), the control unit 17 executes the process in step S315.

In step S309, the image extraction unit 173 initializes the total movement distance by setting it to 0.
In step S311 the image extraction unit 173 sets the selected image as an inspection image candidate.

In step S313, the image extraction unit 173 confirms the presence or absence of an image of the road surface of the next frame. When the image exists (step S313; YES), the process of the control unit 17 and step S301 is executed. If the image does not exist (step S313; NO), the control unit 17 executes the process in step S315.
In step S315, the image extraction unit 173 outputs the inspection image candidate to the overlap region extraction unit 175A.

The set distance may be set according to, for example, the recognition accuracy of road surface damage in one inspection image. Further, for example, when it is possible to detect road surface damage not only in the vicinity but also in the distance in one inspection image, the set distance may be increased to reduce the overlap of the road surface in each inspection image. If it is not possible to detect damage to the road surface that appears in the distance in one inspection image, the set distance may be reduced and the road surface that appears in each inspection image may overlap more. As a result, it is possible to maintain the recognition accuracy of the road surface damage while reducing the load required for the analysis of the road surface damage.

As described above, the information processing apparatus 1A according to the second embodiment is based on the image acquisition unit 171 that acquires an image of the region including the surface captured by the image pickup apparatus and the image acquired by the image acquisition unit 171. It is provided with a calculation unit 177 for calculating the degree of damage of the above.
In addition, the calculation unit 177 calculates the degree of damage to the surface in each of the plurality of consecutive images acquired by the image acquisition unit 171. Further, the information processing device 1A further includes a state determination unit 1771 that determines the state of the surface in each image, and the calculation unit 177 is based on the state of the surface in each image in each image. Calculate the degree of surface damage.
As a result, damage to the surface can be automatically detected and quantitative evaluation can be performed, so that convenience in automatic detection of damage to the building can be improved.

<Third embodiment>
In the third embodiment, an example will be described in which the image of one road surface is divided into a plurality of regions, and each of the plurality of regions partially overlaps with other regions in the image. In the third embodiment, a part different from the first embodiment will be mainly described.

FIG. 12 is a schematic block diagram showing an example of the configuration of the information processing apparatus 1B according to the third embodiment of the present invention.
The information processing device 1B includes an input unit 10, a communication unit 13, a storage unit 15, a control unit 17, and an output unit 19.
The control unit 17 includes an image acquisition unit 171, an image extraction unit 173, a partial image extraction unit 175B, and a calculation unit 177. The calculation unit 177 includes a state determination unit 1771 and a degree calculation unit 1773.

The partial image extraction unit 175B extracts a plurality of road surface regions that partially overlap each other from each inspection image candidate based on the position information extracted from the metadata. In other words, the partial image extraction unit 175B extracts a plurality of regions including the road surface in a certain inspection image candidate, and each of the plurality of regions includes the apex portion and the side portion of the certain region in the other region. .. The partial image extraction unit 175B identifies the inspection image candidate obtained by extracting a plurality of regions as an inspection image, and represents information representing the inspection image, region information representing a plurality of regions extracted from the inspection image candidate, and position information. Is output to the calculation unit 177.

The calculation unit 177 analyzes the damage on the road surface in the inspection image by deep learning. Specifically, the state determination unit 1771 determines whether or not the road surface in each inspection image is damaged by deep learning in each of the plurality of regions, and calculates the value of the determination score (confidence).
The degree calculation unit 1773 calculates the degree of damage, for example, the crack rate based on the damage presence / absence information. Specifically, the degree calculation unit 1773 calculates the degree of damage by dividing the sum of the judgment score values in the predetermined section, for example, the 20 m section by the number of regions in the inspection image in the predetermined section. The calculation unit 177 causes the output unit 19 to output the calculated damage degree and the position information. Here, when the moving distance is less than the predetermined section, the calculation unit 177 calculates the degree of damage for each inspection image.

The calculation unit 177 may compare the calculated value of the determination score with the predetermined threshold value. For example, when the value of the determination score is equal to or greater than the predetermined threshold value, it is "1", and the value of the determination score is less than the predetermined threshold value. When is, it may be binarized like "0". Further, when the value of the determination score is equal to or less than the predetermined threshold value, the value of the determination score may be set to "0".

FIG. 13 is a flowchart showing an example of the damage degree calculation process according to the third embodiment of the present invention.
In step S401, the image acquisition unit 171 acquires an image of one or more road surfaces via the input unit 10 and the communication unit 13.
In step S403, the image extraction unit 173 executes an image extraction process described later.
In step S405, the partial image extraction unit 175B extracts a plurality of road surface regions that partially overlap each other from the respective inspection image candidates based on the position information extracted from the metadata. The partial image extraction unit 175B identifies the inspection image candidate obtained by extracting a plurality of regions as an inspection image, and represents information representing the inspection image, region information representing a plurality of regions extracted from the inspection image candidate, and position information. Is output to the calculation unit 177.

In step S407, the state determination unit 1771 determines whether or not the road surface in each inspection image is damaged by deep learning in each of the plurality of regions, and calculates the value of the determination score (confidence).
In step S409, the degree of damage is calculated by dividing the sum of the judgment score values in the predetermined section, for example, the 20 m section by the number of regions in the inspection image in the predetermined section.

FIG. 14 is a flowchart showing an example of the damage degree calculation process according to the third embodiment of the present invention.
In step S501, the image extraction unit 173 selects an image of the road surface of the frame. Specifically, when the road surface image is not selected, the image extraction unit 173 selects the road surface image of the first frame. On the other hand, when the frame has already been selected, the image extraction unit 173 selects the image of the road surface of the frame next to the selected frame.

In step S503, the image extraction unit 173 refers to the position information corresponding to the image of the selected frame, and calculates the moving distance from the position information corresponding to the previous frame. Hereinafter, this moving distance is referred to as "distance between images".
In step S505, the image extraction unit 173 adds the calculated movement distance to the total movement distance. The total travel distance is initialized at the start of the image extraction process.

In step S507, the image extraction unit 173 compares the total movement distance with the predetermined set distance. When the total movement distance is larger than the set distance (step S507; YES), the control unit 17 executes the process of step S509. On the other hand, when the total movement distance is smaller than the set distance (step S507; NO), the control unit 17 executes the process in step S515.

In step S509, the image extraction unit 173 initializes the total movement distance by setting it to 0.
In step S511, the image extraction unit 173 sets the selected image as an inspection image candidate.

In step S513, the image extraction unit 173 confirms the presence or absence of an image of the road surface of the next frame. When the image exists (step S513; YES), the process of the control unit 17 and step S501 is executed. If the image does not exist (step S513; NO), the control unit 17 executes the process in step S515.
In step S515, the image extraction unit 173 outputs the inspection image candidate and the area information to the partial image extraction unit 175B.

As described above, the information processing apparatus 1B according to the third embodiment is based on the image acquisition unit 171 that acquires an image of the region including the surface captured by the image pickup apparatus and the image acquired by the image acquisition unit 171. It is provided with a calculation unit 177 for calculating the degree of damage of the above.
The information processing device 1B further includes a partial image generation unit (partial image extraction unit 175B) that divides the image acquired by the image acquisition unit 171 and generates a plurality of partial images that are a part of the image, and the calculation unit 177 includes a calculation unit 177. For each of the partial images generated by the partial image generation unit (partial image extraction unit 175B), the degree of damage to the surface in the partial image is calculated.
The information processing device 1B further includes a state determination unit 1771 that determines the state of a surface in the partial image for each of the partial images generated by the partial image generation unit (partial image extraction unit 175B), and the calculation unit 177 includes a calculation unit 177. The degree of damage to the surface in the partial image is calculated based on the state of the surface in the partial image.
The partial image generation unit (partial image extraction unit 175B) divides the image in the image acquired by the image acquisition unit 171 and divides the divided first partial image and the apex portion or side portion of the first partial image. A second partial image, including, is generated.
As a result, damage to the surface can be automatically detected and quantitative evaluation can be performed, so that convenience in automatic detection of damage to the building can be improved.

The state determination unit 1771 may calculate the damage class according to the damage type. Specifically, for example, planar cracks such as turtle shell-shaped cracks and multiple linear cracks, single linear cracks such as damage class "1", longitudinal cracks, and transverse cracks, damage class "0.6" , The damage class without damage such as cracks may be calculated for each region by machine learning as in the damage class "0". In this case, the degree of damage may be calculated using the value of the damage class instead of the value of the determination score in the third embodiment.

Although the degree of damage has been described as an example of the crack rate, the degree of damage may be the amount of rut digging. In addition, the degree of damage may be a damage category in addition to the crack rate and the amount of rut digging.

<Fourth Embodiment>
In the first to third embodiments, an example of calculating the degree of damage has been described.
In the fourth embodiment, an example of determining the type of damage will be described. Here, in the fourth embodiment, the first embodiment, the second embodiment, the third embodiment, and the parts different from each modification will be mainly described.
Here, when repairing the road surface, the work process, the number of working days, the cost, etc. related to the repair change depending on the type of damage to the road surface. Therefore, if the type of damage to the road surface can be detected, that is, if the state of the road surface can be evaluated, these estimates can be made, and the convenience of the user can be improved.

In the following description, the state determination unit 1771 is a road surface by combining the first embodiment, the second embodiment, the third embodiment, the contents described in each modification, or a part or all of them. An example will be described when any one of the judgment score value, the damage class value, and the damage presence / absence information for each area is calculated as the damage score.
In addition, the degree of damage calculated by the degree calculation unit 1773 by combining the first embodiment, the second embodiment, the third embodiment, the contents described in each modification, or a part or all of them. It may be used as an injury score. In this case, the road surface of the predetermined section for which the degree of damage has been calculated may be treated as one road surface area, or the predetermined section may be treated as a short section and the road surface of the short predetermined section may be treated as one road surface area.

In the fourth embodiment, the type of road surface damage is determined based on the distribution of the plurality of road surface regions and the damage scores in each region. Types of road surface damage include, for example, longitudinal cracks, transverse cracks, hexagonal cracks, longitudinal cracks and ruts, construction seams, dents, and overall sparse damage. This will be described with reference to FIGS. 15 to 22.

FIG. 15 is a diagram showing an example of a road surface damage type according to a fourth embodiment of the present invention. FIG. 16 is a diagram showing another example of the road surface damage type according to the fourth embodiment of the present invention. FIG. 17 is a diagram showing another example of the road surface damage type according to the fourth embodiment of the present invention. FIG. 18 is a diagram showing another example of the road surface damage type according to the fourth embodiment of the present invention. FIG. 19 is a diagram showing another example of the road surface damage type according to the fourth embodiment of the present invention. FIG. 20 is a diagram showing another example of the road surface damage type according to the fourth embodiment of the present invention. FIG. 21 is a diagram showing another example of the road surface damage type according to the fourth embodiment of the present invention. FIG. 22 is a diagram showing another example of the road surface damage type according to the fourth embodiment of the present invention.
Here, each figure represents a part of the road surface, and the squares in each figure, that is, the direction in which the number of road surface areas is large is the traveling direction, and the direction in which the number of road surface areas is small is the width direction.

The example illustrated in FIG. 15 is a longitudinal crack. The solid line shown represents a crack, and the crack runs through a plurality of regions indicated by hatches. The example illustrated in FIG. 16 is a transverse crack. The solid line shown represents a crack, and the crack traverses a plurality of regions indicated by hatches. The example illustrated in FIG. 17 is a hexagonal crack. The solid line shown represents cracks, and the cracks occur in a vertical and horizontal grid pattern, that is, a hexagonal pattern over a plurality of regions indicated by hatches. The example illustrated in FIG. 18 is a longitudinal crack and rut. The solid line shown in the figure represents a crack, and a plurality of cracks are vertically traversed over a plurality of regions indicated by hatches, and a plurality of vertically traversing cracks are generated substantially in parallel. The example illustrated in FIG. 19 is a construction seam. The solid line shown represents the seam created by the construction, and the seam is generated over a plurality of regions indicated by the hatch, for example, over more regions than the longitudinal cracks.

The example illustrated in FIG. 20 is a depression. The solid line shown in the figure represents a crack, and the crack is generated in the shape of a mouth over a plurality of regions indicated by the hatch. The example illustrated in FIG. 21 is an overall sparsely present damage. The solid line shown represents damage such as cracks, and occurs sparsely in a plurality of regions indicated by hatches and in predetermined sections. The example illustrated in FIG. 22 is a hexagonal crack. The solid line shown represents cracks, and the cracks occur in a vertical and horizontal grid pattern, that is, a hexagonal pattern over a plurality of regions indicated by hatches. Here, the two types of hatches shown in FIG. 22 are a region in which the damage score is equal to or more than the first predetermined value and less than the second predetermined value, and a region in which the damage score is equal to or greater than the second predetermined value. The central portion of the hatch is a region where the damage score is high, that is, a region where the damage score is equal to or higher than the second predetermined value. In the case of the example shown in FIG. 22, the value of the determination score or the value of the damage class may be used as the damage score.

The information processing device 1C classifies and determines the types of road surface damage as shown in FIGS. 18 to 22 according to the damage score for each road surface area and the distribution of the area where the damage score is equal to or higher than a predetermined value.

FIG. 23 is a schematic block diagram showing an example of the configuration of the information processing apparatus 1C according to the fourth embodiment of the present invention.
The information processing device 1C includes an input unit 10, a communication unit 13, a storage unit 15, a control unit 17, and an output unit 19. The control unit 17 includes a damage score calculation unit 177C and a classification unit 179. The classification unit 179 includes an integration unit 1791 and a classification determination unit 1793.

The damage score calculation unit 177C has a part or all of the functions of the control units 17 of the information processing devices 1, 1A and 1B described above, and has the first embodiment, the second embodiment, and the third embodiment. The damage score is calculated by combining the contents described in each modification or a part or all of them. For the specific description, the description of each embodiment and each modification described above will be incorporated, and the description will be omitted.
The damage score calculation unit 177C outputs the calculated damage score and the information of the road surface area corresponding to the damage score to the classification unit 179.

The classification unit 179 classifies the damage type of the road surface according to the distribution of the damage score of each mesh. Specifically, the integration unit 1791 calculates the damage score for each mesh.

The classification determination unit 1793 determines the type of damage by comparing the distribution of damage scores for each region in a predetermined section with the distribution of damage for each known damage type. Specifically, the classification determination unit 1793 determines, for example, a longitudinal crack when a predetermined number, for example, three or more regions having a damage score of a predetermined value or more are continuously distributed in the vertical direction. Further, the classification determination unit 1793 determines, for example, a transverse crack when a predetermined number, for example, three or more regions having a damage score of a predetermined value or more are continuously distributed in the lateral direction. In addition, the classification determination unit 1793 has, for example, a predetermined number of damage score regions having a predetermined value or more, for example, two or more regions, which are continuous in the vertical direction and have a predetermined number or more damage score regions, for example, two or more regions. , If it is continuously distributed in the lateral direction, it is judged to be a hexagonal crack.

Further, the classification determination unit 1793 determines, for example, that when a plurality of longitudinal cracks are distributed in parallel, it is determined to be longitudinal cracks and ruts. Further, the classification determination unit 1793 determines, for example, a construction seam when a predetermined number of areas having a damage score of a predetermined value or more, for example, six or more areas are continuously distributed in the vertical direction. Further, in the classification determination unit 1793, for example, there are a predetermined number of areas with damage scores of a predetermined value or more, for example, two or more, and there are a predetermined number of areas having a damage score of a predetermined value or more, for example, two or more. , If it is continuously distributed in the horizontal direction and distributed in the shape of a mouth, it is judged as a depression. Further, the classification determination unit 1793 determines that, for example, when a predetermined number of areas having a damage score equal to or higher than a predetermined value are sparsely distributed, the damage is sparsely present as a whole.

In this way, the classification determination unit 1793 determines the type of road surface damage according to the distribution of the damage score region in the predetermined section. The classification determination unit 1793 outputs the determination result to the output unit 19.

The predetermined values used by the classification determination unit 1793 for determining the type of damage may be the same value or different values.
It should be noted that the integration unit 1791 does not have to associate the position of the road surface to be determined with the area according to the position information of the area included in the information of each area. That is, the integration unit 1791 does not have to use the position information. In this case, the classification unit 179 may be processed by the integration unit 1791 and the classification determination unit 1793. For example, the classification unit 179 integrates the information of each area included in the information of the road surface area and the damage score of each area, associates each area of the road surface with each damage score, and describes the above-mentioned classification determination unit. The process of 1793 may be performed.

FIG. 24 is a flowchart showing an example of the damage type determination process according to the fourth embodiment of the present invention.
In step S601, the damage score calculation unit 177C has a part or all of the functions of the control units 17 of the information processing devices 1, 1A, and 1B described above, and has the functions of the first embodiment, the second embodiment, and the third. The damage score is calculated by combining the embodiments of the above, the contents described in each modification, or a part or all of them.
In step S603, the classification unit 179 integrates the information of each region included in the information of the region of the road surface and the damage score of each region.
In step S605, the classification unit 179 determines which of the damage types of the plurality of road surfaces is the damage type according to the distribution of the damage score for each region in the predetermined section. Further, the determination based on the distribution may be performed based on a rule based on a known distribution as described above, or may be performed by deep learning based on the teacher data of the distribution.

As described above, according to the fourth embodiment, the information processing apparatus 1C is based on the image acquisition unit 171 that acquires an image of the region including the surface imaged by the image pickup apparatus and the image acquired by the image acquisition unit 171. A calculation unit 177 for calculating the degree of damage to the surface and a type determination unit (classification determination unit 1793) for determining the type of damage to the surface according to the degree of damage calculated by the calculation unit 177 are provided.
Further, the information processing device 1C further includes a partial image generation unit (partial image extraction unit 175B) that divides the image acquired by the image acquisition unit 171 and generates a plurality of partial images that are a part of the image, and is a type determination unit. (Classification determination unit 1793) determines the damage type of the surface based on the degree of damage to the surface in the partial image extracted by the partial image generation unit (partial image extraction unit 175B).

Further, the type determination unit (classification determination unit 1793) determines the damage type of the surface based on the distribution of the partial images in which the degree of damage is equal to or higher than a predetermined value.
In addition, the type determination unit (classification determination unit 1793) integrates the degree of damage to the surface in each of the plurality of partial images generated from one image to determine the damage type of the surface.
Further, the type determination unit (classification determination unit 1793) determines the damage type of the surface based on the degree of damage to the surface in each of the plurality of consecutive images acquired by the image acquisition unit 171.
Further, the type determination unit (classification determination unit 1793) determines the damage type of the surface based on the distribution of a plurality of images whose damage degree is equal to or higher than a predetermined value.
Further, the type determination unit (classification determination unit 1793) determines the damage type of the surface by integrating the degree of damage to the surface in each of the plurality of consecutive images.
In addition, the calculation unit 177 calculates the degree of damage by deep learning.

As a result, it is possible to classify which damage is caused, so that the convenience of the user can be improved.

In each of the above embodiments and modifications, the position information may be associated with the road surface damage type and / or the road surface damage degree and output on a map, for example, or the position information and the road surface may be output. The damage type and / and the degree of damage to the road surface may be associated and output as a list.

In each of the above-described embodiments and modifications, the value of the determination score may be multiplied by the value of the determination class as a weight to be used as the damage score, or the degree of damage may be calculated.

When the amount of rut digging is used as the degree of damage in each of the above embodiments and modifications, the depth of rut digging may be calculated by deep learning.

Although each embodiment and each modification have been described, the present invention is not limited to these examples, and for example, any one of each embodiment or each modification, a part of each embodiment, or each modification One aspect of the present invention may be realized by combining a part with another one or more embodiments or another one or more modifications.

The program for executing each process of the information processing apparatus 1 in the present embodiment is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read and executed by the computer system. Therefore, the above-mentioned various processes related to the information processing apparatus 1 may be performed.

The "computer system" here may include hardware such as an OS and peripheral devices. In addition, the "computer system" includes a homepage providing environment (or display environment) if a WWW system is used. The "computer-readable recording medium" includes a flexible disk, a magneto-optical disk, a ROM, a writable non-volatile memory such as a flash memory, a portable medium such as a CD-ROM, and a hard disk built in a computer system. It refers to the storage device of.

Further, the "computer-readable recording medium" is a volatile memory (for example, DRAM (Dynamic)) inside a computer system that serves as a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. It also includes those that hold the program for a certain period of time, such as Random Access Memory)). Further, the program may be transmitted from a computer system in which this program is stored in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium.

Here, the "transmission medium" for transmitting a program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. Further, the above program may be for realizing a part of the above-mentioned functions. Further, a so-called difference file (difference program) may be used, which can realize the above-mentioned functions in combination with a program already recorded in the computer system.

Although the embodiment of the present invention has been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and the design within a range not deviating from the gist of the present invention is also included.

systems Information processing system 1, 1A, 1B, 1C Information processing device 10 Input unit 13 Communication unit 15 Storage unit 17 Control unit 171 Image acquisition unit 173 Image extraction unit 175 Area extraction unit 175A Overlapping area extraction unit 175B Partial image extraction unit 177 Calculation Unit 1771 Condition determination unit 177 Degree calculation unit 177C Damage score calculation unit 179 Classification unit 1791 Integration unit 1793 Classification determination unit 19 Output unit

Claims (5)

An image acquisition unit that acquires an image of an area including the captured road surface in the image pickup device,
Select multiple images, compare the distance between images, which is the distance of the position information corresponding to each selected image, with the predetermined set distance, and select if the distance between images is larger than the set distance. An image extraction unit that sets the image inside as an inspection image candidate and does not set the selected image as an inspection image candidate when the distance between images is smaller than the set distance.
Based on the position information corresponding to the inspection image candidate, at least a part of the road surface area in the image of one road surface and the road surface area in the image of another road surface overlap between the images of the plurality of road surfaces. , An overlapping area extraction unit that extracts an inspection image from the inspection image candidates extracted by the image extraction unit,
And wherein each for the overlapping area checking image extracted by the extraction section, the inspection of the condition of the road surface is judged state determining section in the image,
For each of the test image, a calculation unit based on the state of the road surface in the inspection image, and calculates the degree of damage of the road surface in the inspection image,
And a type determination unit determines the damage type of the road surface in accordance with the degree of damage that the calculation unit is calculated,
The predetermined set distance is
The higher the recognition accuracy of road surface damage in one inspection image, the larger the setting, so that the overlap of the road surface reflected in each inspection image is reduced.
The lower the recognition accuracy of road surface damage in one inspection image, the smaller the setting, which increases the overlap of the road surface in each inspection image.
In a plurality of road surface regions, the type determination unit includes a continuous number of regions in which the degree of damage calculated by the calculation unit is equal to or higher than a predetermined value, directions between regions in which the degree of continuous damage is equal to or higher than a predetermined value, and a plurality of regions. The damage type is determined based on integrating the dispersion over the entire road surface area.
Information processing device. The information processing device according to claim 1, wherein the calculation unit calculates the degree of damage by deep learning. Imaging device and
An image acquisition unit that acquires an image of an area including the captured road surface by the imaging device,
Select multiple images, compare the distance between images, which is the distance of the position information corresponding to each selected image, with the predetermined set distance, and select if the distance between images is larger than the set distance. An image extraction unit that sets the image inside as an inspection image candidate and does not set the selected image as an inspection image candidate when the distance between images is smaller than the set distance.
Based on the position information corresponding to the inspection image candidate, at least a part of the road surface area in the image of one road surface and the road surface area in the image of another road surface overlap between the images of the plurality of road surfaces. , An overlapping area extraction unit that extracts an inspection image from the inspection image candidates extracted by the image extraction unit,
And wherein each for the overlapping area checking image extracted by the extraction section, the inspection of the condition of the road surface is judged state determining section in the image,
For each of the test image, a calculation unit based on the state of the road surface in the inspection image, and calculates the degree of damage of the road surface in the inspection image,
And a type determination unit determines the damage type of the road surface in accordance with the degree of damage that the calculation unit is calculated,
The predetermined set distance is
The higher the recognition accuracy of road surface damage in one inspection image, the larger the setting, so that the overlap of the road surface reflected in each inspection image is reduced.
The lower the recognition accuracy of road surface damage in one inspection image, the smaller the setting, which increases the overlap of the road surface in each inspection image.
In a plurality of road surface regions, the type determination unit includes a continuous number of regions in which the degree of damage calculated by the calculation unit is equal to or higher than a predetermined value, directions between regions in which the degree of continuous damage is equal to or higher than a predetermined value, and a plurality of regions. The damage type is determined based on integrating the dispersion over the entire road surface area.
Information processing system. The computer of the information processing device
An image acquisition step of acquiring an image of a region including the imaged road surface in the image pickup device,
Select multiple images, compare the distance between images, which is the distance of the position information corresponding to each selected image, with the predetermined set distance, and select if the distance between images is larger than the set distance. The image extraction process, in which the image inside is set as an inspection image candidate and the selected image is not set as an inspection image candidate when the distance between images is smaller than the set distance,
Based on the position information corresponding to the inspection image candidate, at least a part of the road surface area in the image of one road surface and the road surface area in the image of another road surface overlap between the images of the plurality of road surfaces. , An overlapping region extraction process for extracting an inspection image from the inspection image candidates extracted by the image extraction process, and
For each of the test images extracted by the overlap region extraction process, a state determination process determines the state of the road surface in the inspection image,
For each of the test images, a calculation process based on the state of the road surface in the inspection image, and calculates the degree of damage of the road surface in the inspection image,
And a type determination step determines the damage type of the road surface in accordance with the degree of damage was calculated by the calculating step,
The predetermined set distance is
The higher the recognition accuracy of road surface damage in one inspection image, the larger the setting, so that the overlap of the road surface reflected in each inspection image is reduced.
The lower the recognition accuracy of road surface damage in one inspection image, the smaller the setting, which increases the overlap of the road surface in each inspection image.
In the type determination process, in a plurality of road surface regions, the number of consecutive regions in which the degree of damage calculated by the calculation process is equal to or higher than a predetermined value, the direction between regions in which the degree of continuous damage is equal to or higher than a predetermined value, and a plurality of regions. The damage type is determined based on integrating the dispersion over the entire road surface area.
Information processing method. The computer of the information processing device
An image acquisition step of acquiring an image of an area including the captured road surface in the image pickup device,
Select multiple images, compare the distance between images, which is the distance of the position information corresponding to each selected image, with the predetermined set distance, and select if the distance between images is larger than the set distance. An image extraction step in which the image inside is set as an inspection image candidate and the selected image is not set as an inspection image candidate when the distance between images is smaller than the set distance.
Based on the position information corresponding to the inspection image candidate, at least a part of the road surface area in the image of one road surface and the road surface area in the image of another road surface overlap between the images of the plurality of road surfaces. , An overlapping area extraction step for extracting an inspection image from the inspection image candidates extracted by the image extraction step, and
For each of the test images extracted by the overlap region extraction step, a state determination step determines the state of the road surface in the inspection image,
For each of the test images, a calculating step based on the state of the road surface in the inspection image, and calculates the degree of damage of the road surface in the inspection image,
Perform a type determination step of determining damage type of the road surface in accordance with the degree of damage was calculated by the calculation step,
The predetermined set distance is
The higher the recognition accuracy of road surface damage in one inspection image, the larger the setting, so that the overlap of the road surface reflected in each inspection image is reduced.
The lower the recognition accuracy of road surface damage in one inspection image, the smaller the setting, which increases the overlap of the road surface in each inspection image.
In the type determination step, in a plurality of road surface regions, the number of consecutive regions in which the degree of damage calculated by the calculation step is equal to or higher than a predetermined value, the direction between regions in which the degree of continuous damage is equal to or higher than a predetermined value, and a plurality of regions. The damage type is determined based on integrating the dispersion over the entire road surface area.
program.