JP7510472B2 - Information processing device, information processing method, and program - Google Patents

Description

The present invention relates to a technology for determining changes in the state of an abnormality.

There are methods for detecting abnormalities such as cracks from images of an inspection target such as the wall surface of a structure, and for determining changes in condition such as progression or shrinkage of the abnormality from images taken at different times. Patent Document 1 describes a technology that compares first abnormality data created from a first image taken at different times with second abnormality data created from a second image to determine changes in condition such as the length or width of the abnormality.

JP 2019-211277 A

However, while Patent Document 1 makes it possible to confirm changes in the length, width, and other aspects of the deformation, there are cases where the shape or position of a past deformation has changed, and the shape has not simply progressed from the past deformation. For this reason, it may not be possible to create deformation data that can accurately reproduce the parts that have changed and the parts that have not changed from the past deformation.

The present invention was made in consideration of the above problems, and aims to make it possible to create deformation data that can accurately reproduce changes in the state of deformation.

In order to solve the above problems and achieve the object, the information processing device of the present invention has an acquisition means for acquiring first deformation data and second deformation data created based on images taken of the same object, a calculation means for determining a common part between the first deformation data and the second deformation data and a different part between the first deformation data and the second deformation data based on the first deformation data and the second deformation data, and calculating position data corresponding to the boundary between the common part and the different part, and a creation means for creating status change data indicating a change in deformation including the common part and the different part using the position data.

The present invention makes it possible to create deformation data that can accurately reproduce changes in the state of deformation.

FIG. 2 is a block diagram showing a hardware configuration of the information processing apparatus according to the embodiment. FIG. 4 is a diagram illustrating a deformation data table according to the present embodiment. FIG. 13 is a diagram showing an example of a display of a crack drawn based on deformation data of this embodiment. FIG. 4 is a diagram illustrating a UI screen according to the embodiment. 4 is a flowchart showing a state change determination process according to the embodiment. 6 is a diagram showing an example of a display of the state change determination process result of FIG. 5 . 6 is a flowchart showing the state change data creation process of FIG. 5 . FIG. 8 is an explanatory diagram of the expansion process of the deformation data in FIG. 7 . 8 is a flowchart showing a process for creating deformation data of the common part and the difference part of FIG. 7 . FIG. 13 is a diagram illustrating a deformation data table of common parts and different parts. 13 is a diagram illustrating the relationship between the deformation data of the common part and the difference part and the expansion area. 13 is a flowchart showing a correspondence data creating process. FIG. 13 is a diagram illustrating an example of a correspondence data table. 6 is an explanatory diagram of a process for determining comparison data corresponding to reference data. FIG. 4 is a diagram for explaining types of status change data. FIG. 11 is a diagram showing an example of display of state change data. 10 is a flowchart showing a state change data creation process according to the second embodiment. 18 is a flowchart showing a process of examining the common portion of FIG. 17 . 19 is a diagram for explaining a score calculation method in the reconciliation process of FIG. 18 . FIG. 19 is a diagram for explaining the reconciliation process in FIG. 18 . FIG. 20 is a diagram for explaining a final judgment method for comparison target deformation data in the reconciliation process of FIG. 18 .

The following embodiments are described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe multiple features, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicate explanations are omitted.

[Embodiment 1]
In embodiment 1, an example is described in which a computer device operates as an information processing device, determines changes in the state of deformations detected from images taken at different times, such as progression, degeneration, or disappearance due to repair, creates deformation data that can accurately reproduce the changes in the state of the deformation, and displays the changes in the state of the deformation so that they can be understood from multiple perspectives.

Deformations refer to cracks that occur on the concrete surface of concrete structures such as expressways, bridges, tunnels, and dams due to damage, deterioration, or other factors. A crack is linear damage that has a starting point, end point, length, and width and occurs on the wall surface of a structure due to deterioration over time or the impact of an earthquake.

<Hardware Configuration>
First, the hardware configuration of an information processing apparatus according to the present embodiment will be described with reference to FIG.

Figure 1 is a block diagram showing the hardware configuration of the information processing device 100 of this embodiment.

In this embodiment, a computer device operates as the information processing device 100. The processing of the information processing device of this embodiment may be realized by a single computer device, or may be realized by distributing each function among multiple computer devices as necessary. The multiple computer devices are connected to each other so that they can communicate with each other.

The information processing device 100 includes a control unit 101, a non-volatile memory 102, a work memory 103, a storage device 104, an input device 105, an output device 106, a network interface 107, and a system bus 108.

The control unit 101 includes an arithmetic processor such as a CPU or MPU that controls the entire information processing device 100. The non-volatile memory 102 is a ROM that stores programs and parameters executed by the processor of the control unit 101. Here, the program refers to a program for executing the state change determination process described below. The work memory 103 is a RAM that temporarily stores programs and data supplied from an external device. The storage device 104 is an internal device such as a hard disk or memory card built into the information processing device 100, or an external device such as a hard disk or memory card that is detachably connected to the information processing device 100. The storage device 104 includes a memory card or hard disk composed of a semiconductor memory or a magnetic disk. The storage device 104 also includes a storage medium composed of a disk drive that reads/writes data from/to optical disks such as DVDs and Blu-ray (registered trademark) Discs.

The input device 105 is an operating member such as a mouse, keyboard, or touch panel that accepts user operations and outputs operation instructions to the control unit 101. The output device 106 is a display device such as a monitor or a display composed of an LCD or organic EL, and displays data held by the information processing device 100 or data supplied from an external device. The network interface 107 is communicatively connected to a network such as the Internet or a LAN (Local Area Network). The system bus 108 includes an address bus, a data bus, and a control bus that connect the components 101 to 107 of the information processing device 100 so that data can be exchanged.

The non-volatile memory 102 stores an OS (operating system), which is basic software executed by the control unit 101, and applications that work with the OS to realize applied functions. In this embodiment, the non-volatile memory 102 also stores an application that enables the information processing device 100 to realize a state change determination process, which will be described later.

The processing of the information processing device 100 in this embodiment is realized by loading software provided by an application. Note that the application has software for utilizing the basic functions of the OS installed in the information processing device 100. Note that the OS of the information processing device 100 may have software for realizing the processing in this embodiment.

<Deformation Data Table>
Next, the data structure of the deformation data table of this embodiment will be described with reference to FIGS.

In this embodiment, the deformation is represented as vector data, and an example of a deformation is explained using a crack.

Figure 2(a) illustrates a first deformation data table 201 in which first deformation data created from an image of the inspection object taken at a first time period is registered. Figure 2(b) illustrates a second deformation data table 251 in which second deformation data created from an image of the inspection object taken at a second time period after the first time period is registered.

The first deformation data table 201 and the second deformation data table 251 include, as information (deformation data) about cracks, deformation IDs 202, 252, maximum widths 203, 253, number of vertices 204, 254, and vertex coordinate lists 205, 255. The deformation IDs 202, 252 are identification information uniquely assigned to each crack. The maximum widths 203, 253 are the maximum values of the widths (thickness) of the cracks. The number of vertices 204, 254 and the vertex coordinate lists 205, 255 are information on the number of vertices and the coordinates of the vertices corresponding to the start and end points of each line segment for expressing the shape of the crack as a polyline consisting of one or more line segments. The deformation data tables 201, 251 are made up of information acquired from images of the inspection object, and each deformation data is input by a user tracing it on the image with a tablet or the like, automatically generated by image analysis processing, or input by a combination of these. Additionally, image analysis processing may be performed using a learning model created by machine learning or deep learning of AI (artificial intelligence).

Figure 3 (a) illustrates an example of a display screen 300 of a crack drawn based on the first deformation data registered in the first deformation data table 201 shown in Figure 2 (a). In the display screen 300, crack 301 is drawn based on the deformation data with the deformation ID Ca001 in the deformation data table 201. Similarly, cracks 302 to 310 correspond to deformation IDs Ca002 to Ca010, respectively.

Figure 3 (b) illustrates a display screen 350 of a crack drawn based on the second deformation registered in the second deformation data table 251 shown in Figure 2 (b). In the display screen 350, a crack 351 is drawn based on the deformation data with the deformation ID Cb001 in the deformation data table 251. Similarly, cracks 352 to 360 correspond to the deformation IDs Cb002 to Cb010, respectively.

Figure 4 illustrates an example of a GUI (Graphical User Interface) screen 400 of an application that realizes the state change determination process executed in the information processing device 100 of this embodiment.

In the GUI screen 400, the first data input button 401 is a button for selecting the first deformation data registered in the first deformation data table 201 in FIG. 2(a). When the user operates the button 401, a file selection dialog (not shown) is displayed, and the first deformation data in the first deformation data table 201 stored in the storage device 104 is selected. The second data input button 402 is a button for selecting the second deformation data registered in the second deformation data table 251 in FIG. 2(b). When the user operates the button 402, a file selection dialog (not shown) is displayed, and the second deformation data in the second deformation data table 251 stored in the storage device 104 is selected.

The first deformation data display area 411 is an area that displays the first deformation data selected by the first data input button 401. The first deformation data display area 411 displays the first deformation data display screen 300 shown in FIG. 3(a) in which cracks are drawn based on the first deformation data registered in the first deformation data table 201 shown in FIG. 2(a). Similarly, the second deformation data display area 412 is an area that displays the second deformation data selected by the second data input button 402. The second deformation data display area 412 displays the crack display screen 350 drawn based on the second deformation data registered in the second deformation data table 251 shown in FIG. 2(b).

The result display area 421 is an area for displaying the status change data, which is the result of the status change determination process described later in FIG. 6, for the first deformation data table 201 in FIG. 2(a) and the second deformation data table 251 in FIG. 2(b). The criterion selection button 431 is a button for selecting the deformation data used as the criterion in the status change determination process, and in the example of FIG. 4, either the first deformation data or the second deformation data can be selected. The result output button 441 is a button for outputting the result of the status change determination process as a file. The data and files output as the result of the status change determination process will be described later.

<Status Change Determination Process>
Next, a state change determination process executed in the information processing device 100 of this embodiment will be described with reference to FIGS.

Figure 5 is a flowchart showing the state change determination process executed in the information processing device 100 of this embodiment.

The process in FIG. 5 is realized by the control unit 101 of the information processing device 100 shown in FIG. 1 expanding a program stored in the non-volatile memory 102 into the work memory 103, executing it, and controlling each component. The same applies to FIG. 17 described later.

In this embodiment, an example of a process is described in which past first deformation data and most recent second deformation data created at different times are input, the common parts and differences between the past first deformation data and the most recent second deformation data are calculated, and the calculated state change data is displayed.

In S501, the control unit 101 reads past first deformation data from the first deformation data table 201 stored in the storage device 104, and reads the most recent second deformation data from the second deformation data table 251.

In S502, the control unit 101 creates first status change data using the past first deformation data input in S501 as reference data and the most recent second deformation data as comparison data. The first status change data includes information indicating a common portion determined to exist in the past first deformation data and the most recent second deformation data. The first status change data also includes information indicating a difference portion determined to exist in the past first deformation data but not in the most recent second deformation data. Furthermore, the first status change data includes information indicating the correspondence between the common portion of the past first deformation data and the common portion of the most recent second deformation data.

In S503, the control unit 101 creates second state change data using the most recent second deformation data input in S501 as reference data and the past first deformation data as comparison data. The second state change data includes information indicating common parts determined to exist in common between the past first deformation data and the most recent second deformation data. The second state change data also includes information indicating difference parts determined to exist in the most recent second deformation data but not in the past first deformation data. Furthermore, the second state change data includes information indicating the correspondence between the common parts of the past first deformation data and the common parts of the most recent second deformation data.

The information indicating the common parts is information on the deformed parts of the reference data that are also present in the comparison data. Moreover, the information indicating the difference parts is information on the deformed parts that are not present in the comparison data but are present in the reference data. For example, if the comparison data is past first deformation data and the reference data is the most recent second deformation data, the information indicating the difference parts corresponds to newly developed cracks that were not detected during past inspections. Moreover, if the comparison data is the most recent second deformation data and the reference data is past first deformation data, the information indicates the difference parts corresponds to cracks that have disappeared due to repairs, etc.

The state change data creation process and data structure in S502 and S503 will be described later.

In S504, the control unit 101 sets reference data for displaying the state change data created in S502 and S503.

In S505, the control unit 101 uses the deformation data set in S504 as the reference data and displays the first state change data created in S502 or the second state change data created in S503 in the result display area 421 of FIG. 4. If the reference data set in S504 is the most recent second deformation data, the common parts of the second state change data created in S503 between the past first deformation data and the most recent second deformation data, and the difference parts not present in the past first deformation data are displayed. Furthermore, the difference parts of the first state change data created in S502 that are not present in the past first deformation data are displayed.

Figure 6 shows an example of the display of the state change determination process results in the state change data displayed in the result display area 421 of Figure 4 in S505 of Figure 5, where the most recent second deformation data is used as the reference data and the past first deformation data is used as the comparison data. The display area 600 of Figure 6 corresponds to the result display area 421 of Figure 4. The solid line segments 601 to 610 correspond to cracks in the common parts of the period from the past to the most recent. The dotted line segments 621 to 624 correspond to the difference parts when the past first deformation data is used as the comparison data and the most recent second deformation data is used as the reference data, and correspond to cracks that have newly occurred in the period from the past to the most recent. The double line segment 631 corresponds to the difference parts when the most recent second deformation data is used as the comparison data and the past first deformation data is used as the reference data, and corresponds to cracks that have disappeared in the period from the past to the most recent. In the example of Figure 6, the three types of abnormalities are displayed with solid lines, dotted lines, and double lines in order to be distinguishable, but they may also be distinguished by drawing them in different colors.

In S506, the control unit 101 determines whether the user operation received by the input device 105 is a result output instruction using the result output button 441 in FIG. 4, or a criterion switching instruction using the criterion selection button 431. If the control unit 101 determines that the user operation received by the input device 105 is a result output instruction, the process proceeds to S507. If the control unit 101 determines that the user operation received by the input device 105 is a criterion switching instruction, the process proceeds to S508.

In S507, the control unit 101 outputs the state change data created in S502 and S503 as a file and ends the process. The output file format may be any known format such as CSV, JSON, XML, or an original format, or a combination thereof, as long as it can describe the deformation data structure of this embodiment. In addition, for each reference data, three types of state change data are created: common part deformation data (described later in FIG. 10(a)), difference part data (described later in FIG. 10(b)), and correspondence relationship data (described later in FIG. 13), and a total of six types of data are included, which will be described later in FIG. 15. In addition, in the file output in S507, all six types of data may be one file, two files for each of the three types of data, three files grouped by data type, individual files for the six types of data, or other combinations.

In S508, the control unit 101 switches the deformation data to the reference data specified in S506 and returns the process to S505. When the reference data is switched to the past first deformation data, in the example of FIG. 6, the line segment shown in solid line indicates the past first deformation data, i.e., the shape of the crack detected during the past inspection. The line segment shown in dotted line indicates the most recent second deformation data, i.e., the shape of the crack newly detected during the most recent inspection. The line segment shown in double line indicates a crack detected during the past inspection but disappeared during the most recent inspection. This makes it possible to accurately grasp the cracks in the common parts and different parts of the deformation data displayed as the reference data, even if the crack position is shifted or the shape of the polyline is different between the past first deformation data and the most recent second deformation data.

In the process of FIG. 5, instead of steps S501 to S503, the existing state change data file output in S507 may be read and converted into a displayable data structure, and the same process as steps S504 to S508 may be performed.

Figure 7 is a flowchart showing the state change data creation process of S502 and S503 in Figure 5.

In S701, the control unit 101 executes an expansion process to set an expansion area for the deformation data for each deformation ID in the deformation data table that is to be used as comparison data from among the past first deformation data and the most recent second deformation data input in S501.

Figure 8 is an explanatory diagram of the expansion process in S701 of Figure 7. Figure 8 (a) illustrates an expansion area data table 801 created by performing expansion process on the deformation data for each deformation ID in the deformation data table of Figures 2 (a) and (b) that is the comparison data among the deformation data tables of Figures 2 (a) and (b). The expansion area data table 801 includes a deformation ID 802 and expansion area information 803. The deformation ID 802 corresponds to the deformation ID in the deformation data table of Figures 2 (a) and (b). The expansion area information 803 is vector data that represents the outline of the expansion area created as a result of performing expansion process on the deformation data, and corresponds, for example, to the vertex coordinate list of a polygon described below.

Figure 8 (b) illustrates a crack polyline 804 drawn based on the deformation data of the deformation IDCa001 in the first deformation data table 201 in Figure 2 (a). The thin line 805 in Figure 8 (c) illustrates an expansion area of a polygon created by performing an expansion process on the polyline 804 in Figure 8 (b). The black circles in the polyline 804 correspond to the vertex coordinates registered in the vertex coordinate list of the deformation IDCa001. The expansion area 805 is created by connecting the extension of the line segments obtained by translating each line segment constituting the polyline 804 by a predetermined distance D on both sides, and the intersection of the straight lines that are perpendicular to the line segments at both ends of the polyline 804 and pass through the tangent points at both ends. The shape of the expansion area 805 is not limited to a polygon, but may be any closed area. For example, in FIG. 8(c), corners 806, 807 on both sides of the vertex of the bent portion of polyline 804 are longer than a predetermined distance D from polyline 804, so the corners of the outline may be rounded into a circular arc with a radius equal to distance D centered on the vertex of the bent portion. In particular, when the bent portion of polyline 804 is acute, the difference from the predetermined distance D is significant, so the process of rounding the corners contributes to improving the accuracy when creating correspondence relationship data with reference data for the common portion.

In S702, the control unit 101 creates deformation data of the common and different parts of the reference data and the comparison data using the deformation data of the reference data and the expansion area data created from the deformation data of the comparison data created in S701. The process of creating deformation data of the common and different parts will be described later with reference to FIG. 9.

In S703, the control unit 101 creates correspondence data indicating the correspondence between the expansion area data created in S701 and the deformation data of the common part created in S702. The correspondence data creation process will be described later with reference to FIG. 12.

<Creating data on changes in common and different parts>
Next, the process of creating deformation data for common and different parts in S702 of FIG. 7 will be described with reference to FIGS.

Figure 9 is a flowchart showing the process of creating deformation data for common and different parts in S702 of Figure 7.

In S901, the control unit 101 initializes the deformation data table of the common parts and the difference parts that are output as the processing result.

Figure 10(a) illustrates a deformation data table 1001 for common parts. Figure 10(b) illustrates a deformation data table 1051 for different parts. The deformation data table 1001 for common parts in Figure 10(a) includes a deformation ID 1002 for identifying the deformation data for the common parts, a reference deformation ID 1003 for identifying the reference deformation data before being distinguished into the common parts and the different parts, a maximum width 1004, a number of vertices 1005, and a vertex coordinate list 1006. The reference deformation ID 1003, maximum width 1004, number of vertices 1005, and vertex coordinate list 1006 are the same as the deformation ID 202, maximum width 203, number of vertices 204, and vertex coordinate list 205 in Figure 2(a).

The deformation data table 1015 of the difference part in FIG. 10(b) includes a deformation ID 1052 that identifies the deformation data of the difference part, a reference deformation ID 1053 that identifies the deformation data that is the reference before being distinguished between the common part and the difference part, a maximum width 1054, a number of vertices 1055, and a vertex coordinate list 1056. The reference deformation ID 1053, the maximum width 1054, the number of vertices 1055, and the vertex coordinate list 1056 are the same as the deformation ID 252, the maximum width 253, the number of vertices 254, and the vertex coordinate list 255 in FIG. 2(b). When the deformation data table 1001 of the common part in FIG. 10(a) and the deformation data table 1015 of the difference part in FIG. 10(b) are initialized, each item is empty with no data registered. The deformation data table 1001 of the common part in FIG. 10(a) and the deformation data table 1015 of the difference part in FIG. 10(b) each have a pointer indicating the position where deformation data is registered in the processing from S902 onwards, and at the time of initialization, the pointer is located in the first row in both tables.

Next, the process in FIG. 9 will be explained with reference to FIG. 11.

Figure 11 illustrates an example of the relationship between the reference data from the deformation data in Figures 2(a) and (b) and the expansion area data created from the comparison data. In Figure 11(a), polyline 1100 indicates a line segment connecting the vertex coordinates of the reference data. Line segments 1101 to 1106 indicate line segments connecting two adjacent vertices in polyline 1100. Expansion areas 1111 and 1112 indicate the expansion area data created from the comparison data in S701.

In S902, the processes of S903 to S910 are repeated for each of the deformation data in Figures 2(a) and (b) that serve as reference data, that is, for each polyline 1100 shown in Figure 11 as an example.

In S903, the processes of S904 to S909 are repeated for each line segment of the polyline of the deformation data currently being processed in the reference data, that is, for each line segment 1101 to 1106 of the polyline 1100 shown in FIG. 11.

In S904, the control unit 101 searches the expansion area data table 801 in FIG. 8 for deformation data of the comparison data corresponding to the expansion area that overlaps with the line segment of the deformation data currently being processed. The determination of whether the line segment of the deformation data overlaps with the expansion area uses a collision determination technique that is widely known in the field of computer games, etc. If the control unit 101 does not detect deformation data of the comparison data corresponding to the expansion area that overlaps with the line segment of the deformation data currently being processed, the control unit 101 proceeds to S905. If the control unit 101 detects deformation data of the comparison data corresponding to the expansion area that overlaps with the line segment of the deformation data currently being processed, the control unit 101 proceeds to S906. In the example of FIG. 11(a), the line segment 1101 does not overlap with any of the expansion areas 1111 and 1112, so the process proceeds to S905.

In S905, the control unit 101 adds the line segment of the deformation data currently being processed to the pointer position of the deformation data table 1051 of the difference part in FIG. 10(b). Since the line segment 1101 is the line segment to be added to the first row, a new ID is issued and it is registered in the deformation ID of the difference part. In addition, the deformation ID and maximum width of the line segment 1101 are registered in the reference deformation ID 1053 and maximum width 1054, respectively, of the deformation data table 1051 of the difference part in FIG. 10(b). In addition, 2 is registered in the number of vertices 1055, and the vertex coordinates of the line segment 1101 are registered in the vertex coordinate list 1056.

The next line segment 1102 overlaps with the expansion area 1111, so in S904 it is determined that deformation data in the comparison data corresponding to the expansion area that overlaps with the line segment of the deformation data currently being processed has been detected, and processing proceeds to S906.

In S906, the control unit 101 determines whether or not the line segment of the deformation data currently being processed is included in the expansion area. If the control unit 101 determines that part of the line segment of the deformation data currently being processed is included in the expansion area, the process proceeds to S907, and if the control unit 101 determines that the entire line segment is included in the expansion area, the process proceeds to S908. The determination of whether or not the line segment of the deformation data is included in the expansion area and the calculation of the intersection points described below use techniques that are widely known in the field of computer graphics. As part of the line segment 1102 is included in the expansion area, the process proceeds to S907.

In S907, the control unit 101 calculates position data of midpoint coordinates that are not in the vertex coordinate list of the deformation data table to be compared, by dividing the line segments included in the expansion area and the line segments that protrude from the expansion area at the intersection of the line segments with the contour of the expansion area. The calculation of the midpoint coordinates is performed by calculating midpoint 1131, which is the intersection of line segment 1102 with the contour 1111 of the diamond in FIG. 11(b). Midpoint 1131 divides line segment 1102 into line segments 1121 and 1122, and is the boundary between the common portion and the different portion.

In S908, the control unit 101 separates the line segment data divided in S907 into line segment data included in the expansion area and line segment data outside the expansion area. The line segment data included in the expansion area is then added to the deformation data table 1001 of the common part in FIG. 10(a). The line segment data outside the expansion area is added to the deformation data table 1051 of the difference part in FIG. 10(b). The data of line segment 1121 is added to the deformation data table 1001 of the common part in FIG. 10(a). The data of line segment 1122 is added to the deformation data table 1051 of the difference part in FIG. 10(b). In the common part deformation data table 1001 of FIG. 10(a), the vertex coordinates of the line segment 1101 are registered at the pointer position in S905, but because the line segment 1121 is continuous with the line segment 1101, the number of the first vertex of the line segment 1121 adjacent to the line segment 1101 is incremented by one. In addition, the vertex coordinates of the midpoint 1131, which is the second coordinate of the line segment 1121, are added as vertex coordinates not present in the vertex coordinate list. In addition, for the line segment 1122, the vertex coordinates of the midpoint 1131 are newly added to the common part deformation data table 1001 of FIG. 10(a), but the new addition process for the line segment 1122 is the same as in S905.

The next line segment 1103 also overlaps with the expansion area 1111, so processing proceeds to S906 as determined in S904, and because it is entirely contained within the expansion area 1111, processing proceeds to S909.

In S909, the control unit 101 adds the data of the line segment 1103 to the common portion deformation data table 1001 in FIG. 10(a). The process of adding the line segment 1103 to a row in which the vertex coordinates are already registered is the same as the process of adding the line segment 1121 in S908.

The next line segment 1104 overlaps partially with the expansion area 1111, so processing proceeds to S907, where it is split into line segments 1123 and 1124 at midpoint 1132, which is the intersection with the expansion area. In S908, data for line segment 1123, whose end point is midpoint 1132, is added to deformation data table 1001 of the common portion in FIG. 10(a), and data for line segment 1124, whose start point is midpoint 1132, is added to deformation data table 1051 of the difference portion in FIG. 10(b). The process of adding line segment 1123 to a row in which vertex coordinates have already been registered is the same as the process of adding line segment 1121 in S908.

On the other hand, the process of adding the difference part of the data of line segment 1124 in Figure 10 (b) to the deformation data table 1051 is as follows.

At the pointer position of the deformation data table 1051 for the difference part in Figure 10 (b), line segments 1101 and 1121 have been added, but the vertex coordinate list for the current row does not include the coordinates of the start and end points of line segment 1124 to be added. This means that line segment 1124 is not continuous with the polyline registered at the current pointer position. Therefore, the pointer is advanced to the next row, and the addition process to the new row already explained is performed.

Similar processing is performed for line segments 1105 and 1106. Line segment 1106 is divided into line segment 1125 ending at midpoint 1133, line segment 1126 starting at midpoint 1133 and ending at midpoint 1134, and line segment 1127 starting at midpoint 1134. The processing for adding each divided line segment to deformation data table 1001 of the common portion in FIG. 10(a) and deformation data table 1051 of the difference portion in FIG. 10(b) is the same as S905 and S908.

The process of dividing the line segments in S907 will be further explained with reference to FIG. 11(c). FIG. 11(c) illustrates an example in which part of line segment 1150 overlaps with expansion areas 1141 and 1142. Line segment 1150 intersects with the contours of expansion areas 1141 and 1142 near the center, but when the expansion areas overlap as in FIG. 11(c), the process divides the expansion area at the intersections of the contours as a combined area, rather than at the intersections with the contours of each expansion area. In other words, line segment 1150 is divided into three line segments 1151, 1152, and 1153.

<Correspondence Data Creation Process>
Next, the correspondence data creation process in S703 of FIG. 7 will be described with reference to FIGS.

Figure 12 is a flowchart showing the correspondence data creation process in S703 of Figure 7.

In S1201, the control unit 101 initializes the correspondence data table that is output as the processing result.

Figure 13 illustrates a correspondence data table 1301. The correspondence data table 1301 includes a deformation ID 1302 of the deformation data of the common part and correspondence information 1303. The deformation ID 1302 of the deformation data of the common part corresponds to the deformation ID of the deformation data table 1001 of the common part in Figure 10 (a). The correspondence information 1303 is registered as a set of the deformation ID of the comparison data that is determined to match for each line segment that constitutes the polyline of the deformation data of the common part and the result of determining the increase or decrease in the maximum crack width of the comparison data. For example, the deformation ID Cbm001 of the deformation data of the common part in the first row of the correspondence data table 1301 corresponds to Cbm001 in the first row of the deformation data table 1001 of the common part in Figure 10 (a). The correspondence information 1303 indicates the correspondence with the deformation ID of the second deformation data table 251 in FIG. 2(b), which is comparison data for the line segments connecting the vertices in the order registered in the vertex coordinate list 1006 of the deformation data table 1001 of the common part in FIG. 10(a).

In S1202, the processes of S1203 to S1209 are repeated for each row of the deformation data table 1001 in the common portion of FIG. 10(a).

S1203 repeats the processes of S1204 to S1208 for each line segment of the polyline of the deformation data of the common part currently being processed.

In S1204, the control unit 101 acquires deformation data for line segments that are candidates for comparison data corresponding to the line segment currently being processed. Here, the deformation ID of the comparison data that is the source of the expansion area where the line segments overlap is searched for in the expansion area data table 801 in FIG. 8(a). Note that in the process of FIG. 9, only the line segments that overlap with any of the expansion areas are registered in the deformation data table 1001 of the common part in FIG. 10(a), so candidates are always detected.

In S1205, the control unit 101 determines whether there is multiple candidate line segment deformation data acquired in S1204. If the control unit 101 determines that there is one candidate acquired in S1204, the process proceeds to S1206. If the control unit 101 determines that there are multiple candidates acquired in S1204, the process proceeds to S1207.

In S1206, the control unit 101 determines the deformation data acquired in S1204 as a candidate for comparison data corresponding to the line segment currently being processed.

In S1207, the control unit 101 determines, from among the deformation data acquired in S1207, the deformation data with the longest overlap length between the expansion area and the line segment as a candidate for comparison data corresponding to the line segment currently being processed. Note that the user may be able to change the candidate determined in S1207.

Now, referring to Figure 14, we will explain the process of determining the deformation data of the line segment that is a candidate for comparison data corresponding to the line segment currently being processed in steps S1205 to S1207.

In FIG. 14, thin lines 1401 and 1402 are expansion areas created from the deformation data of the comparison data. Lines 1411 to 1415 are lines that make up the polyline of the deformation data registered in the deformation data table 1001 of the common part of FIG. 10(a). Lines 1421 and 1422 are lines that make up the polyline of another deformation data registered in the deformation data table 1001 of the common part of FIG. 10(a). Here, lines 1411, 1412, 1413, and 1422 overlap only with the expansion area 1401, and line segment 1415 overlaps only with the expansion area 1402, so in S1205 it is determined that there is one candidate. Then, in S1206, the deformation ID of the comparison data that is the basis of the expansion areas 1401 and 1402 is determined as the candidate. On the other hand, line segment 1414 overlaps expansion area 1401 and expansion area 1402, but the overlapping length is longer in expansion area 1402, so the candidate determined in S1207 is the deformation ID of the comparison data that is the source of expansion area 1402. Similarly, for line segment 1421, the candidate is determined to be the deformation ID of the comparison data that is the source of expansion area 1401. Note that if a line segment is included in an area where multiple expansion areas overlap, the overlapping length of the line segment will be the same in all expansion areas. In that case, the line segment is extended to the adjacent line segments on both sides of the polyline, and the expansion area with the longest overlapping length with all of the overlapping expansion areas is determined. If there is no difference even if the line segment is extended to either end of the polyline, the line segment is determined by some method, such as random, the width of the expansion area, or the order in the expansion area data table of the deformation data that is the source of the expansion area.

In S1208, the control unit 101 registers in the correspondence data table a set of the deformation ID of the comparison data determined as a candidate in S1206 or S1207 and information regarding the increase or decrease in the maximum crack width. The deformation ID of the comparison data and information regarding the increase or decrease in the maximum crack width are registered in the correspondence information 1303 of the deformation ID 1302 at the pointer position in the correspondence data table 1301 in FIG. 13. Information regarding the increase or decrease in the maximum crack width is created by comparing the maximum crack width 1004 of the deformation ID 1002 currently being processed registered in the deformation data table 1001 of the common part in FIG. 10(a) with the maximum crack widths 203, 253 of the deformation ID of the comparison data in the first deformation data table 201 in FIG. 2(a) and the second deformation data table 251 in FIG. 2(b). If the maximum crack width of the deformation ID currently being processed is greater than the upper limit of the specified threshold for the maximum crack width of the deformation ID registered in the deformation data of the comparison data, Wide is registered; if it is less than the lower limit of the specified threshold, Narrow is registered; if it is within the range between the upper and lower limits of the specified threshold, Same is registered.

By repeating the processes of S1204 to S1209, candidates for deformation data of the comparison data and information regarding the increase or decrease in the maximum crack width are registered for each line segment that constitutes the deformation data of the deformation ID of the common part currently being processed.

By referring to the correspondence data table 1301 in FIG. 13, a correspondence can be obtained even when, for example, a crack branches into a Y-shape and the directions of the main axes are different between the past first deformation data and the most recent second deformation data, as shown in FIG. 14. For example, a relationship can be obtained in which the past first deformation data corresponds to line segments 1411, 1412, 1413, 1421, and 1422, and the most recent second deformation data corresponds to line segments 1414 and 1415.

In addition, in S502 of FIG. 5, a change data table of common parts, a change data table of different parts, and a correspondence data table of the common parts of the past first deformation data and the most recent second deformation data are created as state change data using the past first deformation data as reference data. In addition, in S503 of FIG. 5, a change data table of common parts, a change data table of different parts, and a correspondence data table of the common parts of the most recent second deformation data and the past first deformation data are created as state change data using the most recent second deformation data as reference data.

Now, referring to Figure 15, we will explain the relationship between the six types of data tables that are inputted as the results of the state change determination process by inputting the first deformation data table 1501 (Figure 2(a)) and the second deformation data table 1502 (Figure 2(b)).

Using the second deformation data table 1502 as the reference data and the first deformation data table 1501 as the comparison data, a deformation data table 1511 (Figure 10(a)) of the common portion and a deformation data table 1512 (Figure 10(b)) of the different portion are created. Similarly, using the first deformation data table 1501 as the reference data and the second deformation data table 1502 as the comparison data, a deformation data table 1521 (Figure 10(a)) of the common portion and a deformation data table 1522 (Figure 10(b)) of the different portion are created.

Furthermore, a correspondence data table 1531 (Figure 13) is created between the common deformation data table 1511 and the first deformation data table 1501, with the second deformation data table 1502 as the reference data. Also, a correspondence data table 1532 (Figure 13) is created between the common deformation data table 1521 and the second deformation data table 1502, with the first deformation data table 1501 as the reference data.

<Display example of status change data>
Finally, a display example of the status change data will be described with reference to FIGS.

When the second deformation data table 1502 is used as the reference data, the deformation data table 1511 of the common part, the deformation data table 1512 of the difference part, and the correspondence data table 1531 in FIG. 15, and the deformation data table 1522 of the difference part using the first deformation data table 1501 as the reference data are used during display. Also, when the first deformation data table 1502 is used as the reference data, the deformation data table 1521 of the common part, the deformation data table 1522 of the difference part, and the correspondence data table 1532 in FIG. 15, and the deformation data table 1512 of the difference part using the second deformation data table 1502 as the reference data are used during display.

First, we will explain an example of displaying two cracks with different branching directions in the past first deformation data in Figure 16 (a) and the most recent second deformation data in Figure 16 (b).

Display area 1600 in FIG. 16(a) shows a part of the first deformation data display area 411 in FIG. 4. Deformation data 1601 and deformation data 1602 displayed in display area 1600 are registered in rows with different deformation IDs in the first deformation data table 1501. For deformation data 1601 and 1602, the vertex coordinates of the first deformation data table 1501 are drawn as circles.

The display area 1610 in FIG. 16(b) shows a part of the second deformation data display area 412 in FIG. 4. The deformation data 1611 and deformation data 1612 displayed in the display area 1610 are registered in rows with different deformation IDs in the second deformation data table 1502. The vertex coordinates of the deformation data 1611 and 1612 in the second deformation data table 1502 are drawn as circles.

Display area 1620 in FIG. 16(c) shows a part of display area 600 in FIG. 6. Display area 1620 displays the results of a state change determination process in which second deformation data table 1502 is used as reference data and first deformation data table 1501 is used as comparison data. In FIG. 16(c), common part deformation data 1622 and common part deformation data 1623 and 1624 are registered in rows with different deformation IDs in common part deformation data table 1511. In other words, deformation data 1623 and 1624 are registered with the same deformation ID, and deformation data 1622 is registered with a different deformation ID.

The deformation data 1622 to 1624 are displayed in different colors. The deformation data 1622 to 1624 can change their display colors based on information about the increase or decrease in the maximum crack width registered as the correspondence information 1303 in the correspondence data table 1301 in FIG. 13 created in the correspondence data creation process in FIG. 12.
For example, if the increase or decrease in the maximum crack width is registered as Wide in the corresponding information 1303 of the deformation data 1624, the line width can be made wider or the color can be changed to a darker color than the deformation data 1623 registered as Same, thereby highlighting the part to which the user should be alerted. On the other hand, if the increase or decrease in the maximum crack width is registered as Narrow in the corresponding information 1303 of the deformation data 1622, the part can be displayed so as to be identifiable, for example, by making the color lighter than the deformation data 1623 registered as Same. The display form of the deformation data may be expressed not only by color or width, but also by blinking, animation of width width, etc.

The deformation data 1625 is registered in the deformation data table 1512 of the difference portion, and corresponds to the portion where the maximum width of the crack in the deformation data 1624 has progressed compared to the past first deformation data. The deformation data 1624 of the common portion and the deformation data 1625 of the difference portion are divided by a midpoint 1627 that does not exist in the deformation data 1612 (second deformation data table 1502) in FIG. 16(b).

Furthermore, the deformation data 1621 indicates the parts that have disappeared due to repairs or the like up until the most recent inspection. The deformation data 1621 is registered in a deformation data table 1522 of the difference part, which is created using the first deformation data table 1501 as the reference data and the second deformation data table 1502 as the comparison data. The deformation data 1621 of the difference part is composed of the midpoint 1628 that does not exist in the deformation data 1611 of FIG. 16(b), which was calculated in the state change determination process of FIG. 5. In this way, by displaying the cracks in the common parts and the cracks in the difference parts in different colors, the user can intuitively grasp the changes in the cracks.

Next, we will explain an example of displaying the processing results when switching between reference data and comparison data using the reference selection button 431.

The display area 1630 in FIG. 16(d) shows a part of the display area 600 in FIG. 6. The display area 1630 displays the results of the condition change determination process using the first deformation data table 1501 as the reference data and the second deformation data table 1502 as the comparison data.

The deformation data 1632 and 1633 are registered as the same deformation ID in the deformation data table 1521 of the common part created based on the first deformation data table 1501. In addition, the deformation data 1634 is registered as a different deformation ID. As in the display area 1620 of FIG. 16(c), the deformation data 1632 is registered as having a maximum crack width of Wide in the correspondence data table 1532 of the common part of the past first deformation data and the most recent second deformation data, which uses the first deformation data table 1501 as the reference data. In this case, in the correspondence data table 1532, the deformation data 1632 is displayed more highlighted than the deformation data 1634, whose maximum crack width is registered as Same. Similarly, in the correspondence data table 1532, the deformation data 1633 in which the maximum crack width is registered as Narrow is displayed in a lighter color than the deformation data 1633.

In addition, the deformation data 1631 indicates the progress made from the first deformation data 1632 in the past, is registered in the deformation data table 1522 for the difference part, and corresponds to the deformation data for the same difference part as the deformation data 1621 in FIG. 16(c).

Deformation data 1635 is registered in the deformation data table 1512 for the difference part. Deformation data 1635 corresponds to deformation data for the same difference part as deformation data 1625 in FIG. 16(c), and is deformation data for the difference part that existed in the past first deformation data but does not exist in the most recent second deformation data.

In addition, in FIG. 16(c) and FIG. 16(d), the missing or degenerated deformation data 1621, 1635, which is the difference part of the comparison data with respect to the reference data, may be discontinuous with the deformation data 1622, 1633 of the common part of the reference data. This is because, as in the display area 1650 of FIG. 16(e), when the past first deformation data 1651 represented by a dotted line and the most recent second deformation data 1652 represented by a solid line are superimposed, the positions are shifted or the shapes are different. The reason why the position and shape of the crack change is thought to be the difference in the accuracy of the alignment at the time of shooting and the accuracy at the time of crack detection. In this way, the common part of the most recent second deformation data and the difference part that existed in the past first deformation data, or the common part of the past first deformation data and the difference part of the most recent second deformation data are displayed discontinuously due to the position shift. FIG. 16(f) shows an enlarged view of the deformation data 1633 and the deformation data 1635 in the display area 1630 in FIG. 16(d). As shown in the display area 1660 in FIG. 16(f), the vertex 1661 of the deformation data 1633 and the midpoint 1637 of the deformation data 1635 may be connected by a line segment, and the deformation data 1633 and 1635 may be treated as one deformation data. This allows the deformation data 1633 of the common part of the past first deformation data to be used as reference data, and the deformation data 1635 of the difference part that has progressed from the past first deformation data to be connected to create and display one deformation data.

Note that line segment 1662 is a line segment that does not exist in the most recent second deformation data or the past first deformation data, and therefore may be displayed in a color or line that draws the user's attention. Furthermore, when treating common part deformation data 1633 and different part deformation data 1635 as the same crack, the deformation ID and reference deformation ID of the different part registered in different part deformation data table 1522, which uses first deformation data table 1501 as the reference data, and the reference deformation ID and common part deformation ID of common part deformation data table 1521 are used. Furthermore, it can be identified using the common part deformation ID and correspondence information registered in correspondence data table 1532.

In addition, the shape of the crack may differ between the past first deformation data and the most recent second deformation data for the deformation data of the common part. In this case, by switching the reference data in S508 of FIG. 5, it is possible to check the change part not only based on the most recent crack shape, but also based on the past crack shape. As a result, as shown in FIG. 16(f), the reference data that the user wants to check is the deformation data 1633 of the common part of the past first deformation data, and the comparison data is the deformation data 1635 of the difference part (progress) of the most recent second deformation data. Also, as shown in the display area 1670 of FIG. 16(g), the deformation data 1671 of the common part of the past first deformation data of the comparison data may be displayed superimposed on the display area 1620 of FIG. 16(c). The deformation data 1671 is the deformation data obtained by removing the deformation data 1631 of the difference portion of FIG. 16(d) from the deformation data 1651 of FIG. 16(e). Note that in the example of FIG. 16(g), the deformation data 1671 is the deformation data of the common portion of the past first deformation data that is not the reference data, so the increase or decrease in the maximum crack width is not highlighted, but may be highlighted.

In addition, in the correspondence data table 1301 in FIG. 13, the user may specify the deformation ID of the common part that he or she wishes to display using a mouse or the like, and the deformation data corresponding to the specified deformation ID may be displayed.

According to this embodiment, the user can intuitively grasp changes in the condition of cracks, such as their progression, shrinkage, or disappearance due to repairs. In addition, by switching between the reference data and the comparison data when comparing the past first deformation data and the most recent second deformation data, it becomes possible to confirm changes in cracks using one of the past first deformation data and the most recent second deformation data as the reference data and the other as the comparison data. This allows the user to grasp changes in the condition of the deformation from multiple perspectives.

In addition, by creating deformation data for the common parts and deformation data for the difference parts, it becomes easier to calculate the amount of change in the crack. In addition, a correspondence data table of deformation data for multiple common parts created at different times can be created and managed for each crack, so even if a crack branches in different directions, it is possible to match the common parts of the past first deformation data of the branched crack with the common parts of the most recent second deformation data. Furthermore, by knowing the correspondence between the common parts of the past first deformation data and the common parts of the most recent second deformation data, it is also possible to calculate the increase or decrease in the maximum width of the crack.

[Modification]
In this embodiment, an example has been described in which the determination of whether the line segment and the expansion area overlap, and the calculation of the intersection with the contour are processed as vector data, but this is not limited to this. For example, the line segment and the object in which the expansion area is filled may be rasterized into a binary bitmap by the graphics library, and the overlap determination and intersection calculation may be performed by bit operations on both raster images. By adding a GPU (Graphics Processing Unit) to the information processing device 100 in FIG. 1 and having the graphics library rasterize using the GPU, it is possible to speed up processing.

[Embodiment 2]
In embodiment 1, all of the line segments of the reference data that are contained within the expansion area of the deformation data being compared are determined to be common parts, but in this embodiment, these parts are further examined and narrowed down, and a final determination is made as to whether they are common parts or different parts.

The hardware configuration of the information processing device 100 of embodiment 2 is the same as that shown in FIG. 1.

Figure 17 is a flowchart showing the state change data creation process of this embodiment, in which step S1701 has been added to Figure 7 showing the process of embodiment 1. S1701 is a process for further examining and narrowing down the parts determined to be common in S702, and making a final determination. Figure 18 is a flowchart showing the process of examining common parts in S1701 of Figure 17.

In S1801, the control unit 101 initializes the reserved list and the difference part list to be used in subsequent processing, and repeats the processing from S1802 to S1807 for each row (deformation data to be compared) of the data table in FIG. 2 that was used as comparison data in S701.

In S1802, the control unit 101 acquires the reference deformation data of the common portion created in S702 that overlaps with the area in which the comparison deformation data being processed has been expanded.

In S1803, the control unit 101 calculates a score indicating the degree of match with the deformation data being processed as the comparison target for each line segment of the reference deformation data of the common portion acquired in S1802. The score is set so that the closer the line segment of the comparison deformation data being processed is to the line segment of the reference deformation data and the closer it is to being parallel to the line segment of the reference deformation data, the higher the score. For example, the distance between the lines is the length of a perpendicular line drawn from the midpoint of the line segment of the reference deformation data to the line segment of the comparison deformation data.

Here, an example of score calculation using distance will be described with reference to FIG. 19(a). Line segments 1901-1903 are line segments of the comparison target deformation data being processed, and line segments 1904-1906 and 1907-1908 are line segments of the reference deformation data included in the comparison target deformation data. The length of the perpendicular line drawn from the midpoint (triangle mark) of the line segment of the reference deformation data to the line segment of the comparison target deformation data is the distance for calculating the score of that line segment. For example, perpendicular line 1909 is the distance to the comparison target deformation of line segment 1907, and perpendicular line 1911 is the distance to the comparison target deformation of line segment 1906. Note that if perpendicular lines can be drawn from the midpoint of a line segment of one reference deformation data to multiple line segments of comparison target deformation data, the line segment of the comparison target deformation data with the shortest perpendicular line is used for score calculation. Conversely, if a perpendicular line cannot be drawn to any of the line segments in the comparison deformation data (the foot of the perpendicular line is not on the line segment), the line segment in the reference deformation data is deemed unable to calculate a score and is added to the pending list. Hereinafter, the line segment in the reference deformation data and the line segment in the comparison deformation data used to calculate the score are described as being "nearby."

The following Equation 1 illustrates an example of a formula for calculating a score for one line segment of the reference deformation data.
(Equation 1)
Score = k · (W-d) / W + cos θ
In formula 1, W is the width of the expansion region of the comparison target deformation data, d is the length of the perpendicular line, and θ is the angle (0 to 90°) between the line segment of the reference deformation data and the line segment of the comparison target deformation data. For example, the angles between line segment 1907 and line segment 1902, and line segment 1906 and line segment 1903 are angles 1910 and 1912, respectively. k is a coefficient that determines the weighting of the distance and angle, and in this embodiment, k = 1.

Note that the deformation data detected from the captured image may consist of a series of short line segments arranged in a sawtooth pattern. In such cases, the value of θ in Equation 1 becomes unstable, resulting in a score that is far removed from the overall degree of match. To prevent this, for each line segment in the reference deformation data, the scores may be smoothed by taking a moving average along the arrangement of the line segments, or the score may be calculated using the moving average value of θ.

In S1804, the control unit 101 compares the scores calculated in S1803, and determines that the line segments of the reference deformation data whose score values are outliers are difference parts and adds them to the difference part list. The outlier criteria is determined using existing statistical methods, for example, by setting a threshold value obtained by subtracting the standard deviation from the average score, and determining anything less than this as an outlier. As a result, line segments that branch off and are separated from the deformation data to be compared are determined to be difference parts. Note that a value greater than the outlier criteria may be set as a second threshold, and line segments with scores equal to or greater than the outlier threshold but less than the second threshold may be added to the pending list created in S1803 as possibly being difference parts, although not definite. The second threshold is, for example, the average score, but is not limited to this, and may be any value greater than the outlier threshold.

In the case of FIG. 19(a), of the line segments 1904 to 1908, the score of the line segment 1908 is an outlier and is determined to be a difference portion. FIG. 19(b) is similar to FIG. 19(a), but when the deformation data is created from the image, the reference deformation data is divided into the line segments 1924, 1927, and 1928, and the line segments 1925 to 1926. In the score calculation in S1803 and the estimation of the difference portion in S1804, each line segment is treated independently without considering the flow of the reference deformation data, so in FIG. 19(b) as in FIG. 19(a), the line segment 1928 is added to the difference portion list. The line segments at the root of the branch, such as line segments 1907 and 1927, are added to the difference portion list if the distance d is short but the angle θ is sufficiently large so that the score exceeds the outlier threshold.

In S1805, the control unit 101 further examines the line segments of the reference deformation data that are near each line segment of the comparison target deformation data being processed, using the characteristics of the deformation data, in preparation for the case where the line segment at the root of the branch, such as line segments 1907 and 1927 shown in FIG. 19, is not determined to be a difference portion. Here, an example of the examination process of S1805 is explained using FIG. 20.

In Figure 20, line segment 2000 is one of the lines in the comparison deformation data being processed. The reference deformation consists of two parts: line segments 2001, 2004, 2005, and 2006, and line segments 2002 and 2003 that branch off from them. On the other hand, when deformation data of the common part is created from images of these two reference deformations, the deformation data is divided into three parts: line segments 2001-2003, line segment 2004, and line segments 2005-2006, each of which is registered in the deformation data table of the common part. Also, of the line segments 2002, 2003, 2004, and 2005 of the reference deformation data that are near line segment 2000, line segment 2003, shown as a solid gray line, has an outlier score in S1804 and is determined to be a difference part.

In S1805, first, the line segment 2004 with the highest score is determined to be the common part, and the remaining line segments are identified as the difference part based on the characteristics of the deformation data. For example, since a crack is a series of line segments, the line segments of the same crack do not run parallel to each other in close proximity. Therefore, a line segment running parallel to a line segment determined to be a common part can be determined to be a crack different from the common part and to be a difference part. Whether or not they run parallel is determined, for example, by whether or not the range of the line segment of the reference deformation data overlaps with respect to the coordinate axis of the long side direction of the circumscribing rectangle of the line segment of the comparison target deformation data. In the case of the line segment 2000 of the comparison target deformation data in FIG. 20, since it is long in the X-axis direction (horizontal direction), it is determined by the overlap of the range on the X-axis. The range 2007 of the line segment 2004 determined to be the common part and the range 2009 of the line segment 2005 do not overlap, so the two do not run parallel to each other. On the other hand, a portion of the range 2008 of the line segment 2002 overlaps with the range 2007 of the line segment 2004, so the line segment 2002 runs parallel to the line segment 2004. Therefore, the line segment 2002 is determined to be a different portion and is added to the different portion list.

In S1806, the control unit 101 performs a final judgment on the line segments of the comparison target deformation data that were placed on the reserved list in S1803 and S1804 based on the judgment results of adjacent line segments. The judgment process is performed on consecutive reserved partial lines (partial lines that are included in the reserved list) in the comparison target deformation data, and only if the adjacent lines on both sides are judged to be different parts, the reserved partial lines sandwiched between them are judged to be different parts collectively.

Figure 21 is a diagram explaining an example of the processing of S1806. Figures 21(a) to 21(c) show different cases, with the comparison target deformation data before and after the processing of S1806 lined up vertically. In Figure 21, the comparison target deformation data is composed of four line segments, with those that have not been inspected (provisional common parts) shown with black solid lines, those in the reserved list shown with black dotted lines, and those determined to be different parts shown with gray solid lines. In the comparison target deformation data 2100 in Figure 21(a), the line segments at both ends are provisional common parts, so the two reserved part lines do not change state like the comparison target deformation data 2101. In the comparison target deformation data 2102 in Figure 21(b), the line segment at the right end has been determined to be a different part, but the line segment at the left end is a provisional common part, so the state does not change like the comparison target deformation data 2103. In the comparison target deformation data 2104 in FIG. 21(c), the line segments at both ends are determined to be the difference part, so the reserved part line in between is determined to be the difference part and is removed from the reserved list. Note that if the end of the reference deformation data is a reserved part line (if it is the tip of the deformation data, or if its end is included in the deformation data table of the difference part in FIG. 10(b)), it is treated as if there is a line segment of the difference part beyond that.

The above process is repeated until no new line segments are identified as differences.

In S1807, the control unit 101 reflects the results of the examination in S1802 to S1806 in the deformation data table of the common part and the difference part in FIG. 10. Then, for the line segments in the list of difference parts, it is determined whether they overlap with expansion areas other than the deformation data being compared and being processed. As a result, for the parts that do not overlap with any expansion areas, they are cut out from the deformation data table of the common part and moved to the deformation data table of the difference part. If the part to be moved is in the middle of the deformation data of the original common part, the deformation data of the common part is divided and a new deformation data row is added to the table. Also, in moving the difference part to the deformation data table, if any end of the part to be moved matches an end of the deformation data of the difference part to be moved to, it is joined to that deformation data, and if it does not match the end of any change data, a new deformation data row is added.

As described above, according to this embodiment, the reference deformation data that overlaps with the expansion area of the comparison target deformation data is narrowed down to the common parts and the difference parts based on the degree of agreement with the comparison target deformation data, so that the length and number of the line segments that are the difference can be determined more accurately. Furthermore, by calculating the degree of agreement from the distance and angle with the comparison target data, the accuracy of determining the root parts of the branching and intersection of the deformation data as the difference parts can be improved. Furthermore, by narrowing down the reference deformation data for each line segment that constitutes the deformation data, it is possible to absorb the difference in the connection at the branching point and the subdivision of the same deformation that occurred when generating the deformation data from the image. Furthermore, by narrowing down the data by excluding outliers in the degree of agreement, it is possible to determine the deformation of the reference data at the point moving away from the comparison target data as the difference part. Furthermore, by performing a moving average along the line segment connection of the deformation data when calculating the degree of agreement, it is possible to absorb local line segment disturbance. Furthermore, by examining based on the characteristics of the deformation data, the accuracy of determining the difference part can be further improved.

In this embodiment, the information processing device 100 includes the input device 105 and the output device 106, but the present invention is not limited to this. For example, an information processing system may be constructed in which the input device 105 and the information processing device 100 are provided separately and connected to each other by wire or wirelessly. Also, an information processing system may be constructed in which the output device 106 and the information processing device 100 are provided separately and connected to each other by wire or wirelessly. In this case, the input device 105 and the output device 106 may be tablet terminals. Also, the input device 105 and the output device may be the same tablet terminal.

[Other embodiments]
The present invention can also be realized by a process in which a program for realizing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors of a computer in the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) for realizing one or more of the functions.

The invention is not limited to the above-described embodiment, and various modifications and variations are possible without departing from the spirit and scope of the invention. Therefore, the following claims are appended to disclose the scope of the invention.

The disclosure of this specification includes the following information processing device, information processing method, and program.
[Configuration 1]
An acquisition means for acquiring first deformation data and second deformation data created based on images of the same object;
A calculation means for calculating a common part common to the first deformation data and the second deformation data and a difference part existing in either the first deformation data or the second deformation data based on the first deformation data and the second deformation data, and calculating position data corresponding to the boundary between the common part and the difference part;
and creating means for creating status change data indicating a change in an abnormality including the common portion and the different portion using the position data.
[Configuration 2]
The calculation means sets one of the first deformation data and the second deformation data as reference data and the other as comparison data, and sets an expansion area based on the comparison data,
The information processing device described in configuration 1 is characterized in that the deformation data of the portion of the reference data that is included in the expansion area is defined as the common part, the remaining deformation data is defined as the difference part, and the coordinates of the intersection between the reference data and the expansion area are defined as the position data.
[Configuration 3]
The deformation data includes information on at least one line segment constituting the deformation,
The information processing device described in configuration 2 is characterized in that, when there are multiple line segments of the comparison data for each line segment of the reference data, the creation means selects the line segment of the comparison data that has the longest overlapping length with the expansion area as the line segment of the comparison data.
[Configuration 4]
4. The information processing apparatus according to configuration 3, wherein when a plurality of expansion regions overlap each other, the expansion region is set in all of the overlapping regions.
[Configuration 5]
The deformation data includes identification information for each deformation, the width of the deformation, the number of vertices of the line segments constituting the deformation, and the coordinates of the vertices.
An information processing device according to any one of configurations 2 to 4, characterized in that the data of the expansion area includes identification information for each deformation and coordinates of the contour of the expansion area.
[Configuration 6]
The information processing device described in any one of configurations 2 to 5, characterized in that the creation means creates, as the state change data, deformation data of the common part, deformation data of the difference part, and correspondence data indicating the correspondence between the common part of the reference data and the comparison data.
[Configuration 7]
The acquisition means reads the first deformation data from a first deformation data table in which the first deformation data is registered, and reads the second deformation data from a second deformation data table in which the second deformation data is registered,
The creation means creates a deformation data table of the common part in which the deformation data of the common part is registered, and a deformation data table of the difference part in which the deformation data of the difference part is registered,
The position data is registered in a data table of the common portion and a data table of the difference portion;
7. The information processing apparatus according to configuration 6, wherein a correspondence data table is created in which the correspondence data is registered.
[Configuration 8]
The information processing device according to configuration 6 or 7, wherein the correspondence data includes information indicating a width of the deformation data of the common portion.
[Configuration 9]
The calculation means is
The first deformation data is used as reference data, and the second deformation data is used as comparison data to obtain a common part and a difference part,
The second deformation data is used as reference data, and the first deformation data is used as comparison data to obtain a common part and a difference part;
9. The information processing apparatus according to any one of claims 1 to 8, wherein the creating means creates the state change data for each of the obtained results.
[Configuration 10]
The first deformation data is deformation data created from an image of an inspection object taken at a first time,
The second deformation data is deformation data created from an image of the inspection object taken at a second time period after the first time period. The information processing device according to any one of claims 1 to 9.
[Configuration 11]
11. The information processing device according to any one of claims 1 to 10, wherein the deformation data is data relating to cracks.
[Configuration 12]
The calculation means is
a second calculation means for calculating a degree of agreement between the comparison data and a reference data included in the expansion region;
The information processing device according to any one of configurations 2 to 8, further comprising a narrowing-down unit for further narrowing down the deformation data to be the common part based on the degree of coincidence.
[Configuration 13]
13. The information processing apparatus according to configuration 12, wherein the second calculation means calculates the degree of coincidence from a distance and an angle with respect to the comparison data.
[Configuration 14]
14. The information processing apparatus according to claim 12, wherein the second calculation means calculates the degree of coincidence for each line segment constituting the reference data.
[Configuration 15]
15. The information processing apparatus according to configuration 14, wherein the second calculation means calculates a moving average of the degree of coincidence calculated for each of the line segments along the connection of the reference data.
[Configuration 16]
16. The information processing device according to any one of configurations 12 to 15, wherein the second calculation means performs the narrowing down by excluding outliers whose degree of coincidence is less than a threshold value.
[Configuration 17]
17. The information processing device according to any one of configurations 12 to 16, wherein the narrowing down means narrows down the data based on characteristics of the deformation data.
[Configuration 18]
An input device including an input means for inputting first deformation data and second deformation data created based on images of the same object;
An acquisition means for acquiring the first deformation data and the second deformation data from the input device;
A calculation means for calculating a common part common to the first deformation data and the second deformation data and a difference part existing in either the first deformation data or the second deformation data based on the first deformation data and the second deformation data, and calculating position data corresponding to the boundary between the common part and the difference part;
and a creation means for creating status change data indicating a change in an abnormality including the common part and the different part using the position data.
[Configuration 19]
An information processing method for detecting a change in a state of an abnormality, comprising:
A step in which an acquisition means acquires first deformation data and second deformation data created based on images of the same object;
A calculation means, based on the first deformation data and the second deformation data, obtains a common part common to the first deformation data and the second deformation data and a difference part existing in either the first deformation data or the second deformation data, and calculates position data corresponding to the boundary between the common part and the difference part;
A method for processing information, comprising: a step in which a creating means creates status change data indicating a change in an abnormality including the common part and the different part using the position data.
[Configuration 20]
18. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 17.

100: Information processing device, 101: Control unit, 102: Non-volatile memory, 103: Work memory, 104: Storage device, 105: Input device, 106: Output device, 107: Network interface, 108: System bus

Claims (20)

An acquisition means for acquiring first deformation data and second deformation data created based on images of the same object;
A calculation means for calculating a common part between the first deformation data and the second deformation data and a difference part between the first deformation data and the second deformation data based on the first deformation data and the second deformation data, and calculating position data corresponding to the boundary between the common part and the difference part;
and creating means for creating status change data indicating a change in an abnormality including the common portion and the different portion using the position data. The calculation means sets one of the first deformation data and the second deformation data as reference data and the other as comparison data, and sets an expansion area based on the comparison data,
The information processing device described in claim 1, characterized in that the deformation data of the part of the reference data that is included in the expansion area is defined as the common part, the other deformation data is defined as the difference part, and the coordinates of the intersection between the reference data and the expansion area are defined as the position data. The deformation data includes information on at least one line segment constituting the deformation,
3. The information processing apparatus according to claim 2, wherein when there are a plurality of line segments of the comparison data for the line segment of the reference data, the creation means selects the line segment of the comparison data that has the longest overlapping length with the expansion area as the line segment of the comparison data. The information processing device according to claim 3, characterized in that, when multiple expansion areas overlap, the expansion area is set to all overlapping areas. The deformation data includes identification information for each deformation, the width of the deformation, the number of vertices of the line segments constituting the deformation, and the coordinates of the vertices.
The information processing device according to claim 2, characterized in that the data of the expansion area includes identification information for each deformation and coordinates of the contour of the expansion area. The information processing device according to claim 2, characterized in that the creation means creates, as the state change data, deformation data of the common portion, deformation data of the difference portion, and correspondence data indicating the correspondence between the common portion of the reference data and the comparison data. The acquisition means reads the first deformation data from a first deformation data table in which the first deformation data is registered, and reads the second deformation data from a second deformation data table in which the second deformation data is registered,
The creation means creates a deformation data table of the common part in which the deformation data of the common part is registered, and a deformation data table of the difference part in which the deformation data of the difference part is registered,
The position data is registered in a data table of the common portion and a data table of the difference portion;
7. The information processing apparatus according to claim 6, further comprising: a correspondence data table in which the correspondence data is registered. The information processing device according to claim 6, characterized in that the correspondence data includes information indicating the width of the deformation data of the common portion. The calculation means is
The first deformation data is used as reference data, and the second deformation data is used as comparison data to obtain a common part and a difference part,
The second deformation data is used as reference data, and the first deformation data is used as comparison data to obtain a common part and a difference part;
2. The information processing apparatus according to claim 1, wherein said creating means creates said state change data for each of the obtained results. The first deformation data is deformation data created from an image of an inspection object taken at a first time,
2. The information processing device according to claim 1, wherein the second deformation data is deformation data created from an image of the inspection object taken at a second time period that is later than the first time period. The information processing device according to claim 1, characterized in that the deformation data is data related to cracks. The calculation means is
a second calculation means for calculating a degree of coincidence between the comparison data and a reference data included in the expansion region;
The information processing apparatus according to claim 2 , further comprising a narrowing-down unit for further narrowing down the deformation data to be the common portion based on the degree of coincidence. The information processing device according to claim 12, characterized in that the second calculation means calculates the degree of coincidence from the distance and angle with the comparison data. The information processing device according to claim 12, characterized in that the second calculation means calculates the degree of coincidence for each line segment constituting the reference data. The information processing device according to claim 14, characterized in that the second calculation means calculates a moving average of the degree of match calculated for each line segment along the connection of the reference data. The information processing device according to claim 12, characterized in that the second calculation means performs the narrowing down by excluding outliers whose degree of match is less than a threshold value. The information processing device according to claim 12, characterized in that the narrowing down means narrows down the data based on the characteristics of the deformation data. An input device including an input means for inputting first deformation data and second deformation data created based on images of the same object;
An acquisition means for acquiring the first deformation data and the second deformation data from the input device;
A calculation means for calculating a common part between the first deformation data and the second deformation data and a difference part between the first deformation data and the second deformation data based on the first deformation data and the second deformation data, and calculating position data corresponding to the boundary between the common part and the difference part;
and a creation means for creating status change data indicating a change in an abnormality including the common part and the different part using the position data. An information processing method for detecting a change in a state of a deformation, comprising:
A step in which an acquisition means acquires first deformation data and second deformation data created based on images of the same object;
A step in which a calculation means obtains a common part between the first deformation data and the second deformation data and a difference part between the first deformation data and the second deformation data based on the first deformation data and the second deformation data, and calculates position data corresponding to a boundary between the common part and the difference part;
A method for processing information, comprising: a step in which a creating means creates status change data indicating a change in an abnormality including the common part and the different part using the position data. A program for causing a computer to function as each of the means of an information processing device according to any one of claims 1 to 17.

Images