JP2016176800A - Displacement or strain calculating program, and displacement or strain measuring method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a displacement or strain calculating program and a displacement or strain measuring method that can solve a problem with the prior art so that calculation time for matching images before and after deformation of a concrete structure can significantly be reduced compared to the prior art.SOLUTION: A displacement or strain calculating program is a program to instruct a computer to execute processes to determine a strain of an object by comparing images before and after deformation of the object, and is provided with functions to instruct the computer to execute a pre-deformation characteristic point recognition process, a subset preparation process, a post-deformation characteristic point recognition process, a characteristic point displacement calculation process, a post-deformation gauge point interpolation process, a search area setting process, a subset-equivalent area searching process, and a displacement or strain calculation process. Of these processes, the post-deformation gauge point interpolation process is to interpolate a gauge point in a deformed image on the basis of displacement of three or more characteristic points.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technique for measuring displacement or strain (hereinafter referred to as “strain or the like”) of a concrete structure or the like, and more specifically, comparing an image before deformation and an image after deformation. The present invention relates to a program for calculating strain and the like, and a method for measuring strain and the like.

  For example, when measuring the strain of a concrete structure, a direct method using a strain gauge has been the mainstream in the past, but in recent years, a method using a photographic image has also been adopted. The distortion is obtained by comparing the image before the structure is deformed with the image after the deformation and performing image processing. Specifically, attention is paid to a mark on the surface of the structure, and the strain is grasped by seeing how this mark moves before and after the deformation of the structure.

  The “image correlation method”, which is one of the methods for measuring distortion and the like by using an image, will be briefly described. First, paying attention to the mark in the image before deformation, a subset that is an area including the mark is set. This subset is formed by a plurality of pixels. Next, a specific area that best correlates with the subset is searched from the deformed image, and a mark in the specific area is obtained. Then, the movement (size, direction) of the gauge is grasped by comparing the gauge before deformation with the gauge after deformation. By repeatedly performing this within the measuring range, the distortion of the structure can be obtained.

  Patent Document 1 also proposes a method for measuring the displacement of a measurement object using an image, and the image correlation method is adopted here as well.

JP 2013-083605 A

  The technique proposed in Patent Document 1 calculates the displacement of a measurement object in a red hot state, and divides an image before and after deformation into three components R (Red), G (Green), and B (Blue), In this method, both images are converted with a component having the smallest luminance change, and pattern matching is performed.

  FIG. 9 is an explanatory diagram showing an image matching method in the image correlation method. As shown in this figure, when matching images before and after deformation by the image correlation method, a subset (template image) is cut out from the image before deformation, and a specific area most correlated with the image is searched from the entire image after deformation. It was. Specifically, the image dissimilarity is calculated while moving the subset one pixel (or several pixels) in the transformed image, and the image having the smallest dissimilarity is set as the specific region. . Here, the degree of difference is a value obtained as a result of obtaining physical quantities of colors calculated by RGB or gray scale and comparing the physical quantities.

  According to such a matching method, a specific region can be searched for anywhere in the transformed image, but a considerable number of image comparisons (difference calculation) are required, and it takes a long time to obtain the final result. It was forced to take time. In particular, when the image size is large, an enormous number of image comparisons are performed, and a large calculation cost is required.

  The problem of the present invention is to solve the problems of the prior art, that is, in matching images before and after deformation of a concrete structure or the like, the calculation time can be greatly reduced compared to the prior art, A strain calculation program and a displacement or strain measurement method are provided.

  The invention of the present application has been made by paying attention to the fact that the position of the feature point in the image before and after the deformation is used to estimate the target position after the deformation and the image is matched. It was based on an unprecedented idea.

  The displacement or strain calculation program of the present invention is a program that causes a computer to execute processing for obtaining strain or the like of an object by comparing images before and after the deformation of the object. A function for causing a computer to execute processing, post-deformation feature point recognition processing, feature point displacement calculation processing, post-deformation target point interpolation processing, search region setting processing, subset equivalent region search processing, displacement or strain calculation processing It is. Among these, the pre-deformation feature point recognition process is a process for specifying three or more feature points in the image before the deformation, and the subset creation process is an area formed by three or more feature points (hereinafter referred to as “main area”). It is a process of creating a subset composed of a plurality of pixels including a target point in the inside. The post-deformation feature point recognition process is a process for specifying three or more feature points in the post-deformation image, and the feature point displacement calculation process is performed by comparing the corresponding feature points before and after the deformation. It is the process which calculates | requires the displacement of. The post-deformation point interpolation process is a process of interpolating the point in the image after deformation based on the displacement of three or more feature points, and the search area setting process is a point position in the image after deformation. Is a process of setting a search area based on the above. The subset-corresponding region search process is a process of searching an image that is most approximate to the subset image (hereinafter referred to as “subset-corresponding region”) from the transformed image while moving the reference point in the subset within the search region. is there. The displacement or strain calculation process is a process for obtaining the displacement or strain of the object based on the position of the subset and the position of the subset-corresponding region.

  The displacement or strain calculation program of the present invention may further include a function for causing a computer to execute post-deformation point estimation processing. The post-deformation point estimation process is a process for estimating the position of the mark within the subset-corresponding region. In this case, when the estimated target position is at the periphery of the subset-corresponding region, the search region is expanded and the subset-corresponding region search process is executed again.

  In the displacement or strain calculation program of the present invention, the search area setting process may set the search area in an area smaller than the subset.

  The displacement or strain measurement method of the present invention is a method of measuring strain or the like of an object by comparing images before and after the deformation of the object. The pre-deformation feature point extraction step, the subset creation step, the post-deformation feature point This is a method including an extraction step, a feature point displacement calculation step, a post-transformation point interpolation step, a search region setting step, a subset equivalent region search step, and a displacement or strain calculation step. Among these, in the pre-deformation feature point extraction step, three or more feature points in the pre-deformation image are extracted, and in the subset creation step, a main region is formed and a plurality of mark points are included in the main region. A subset of pixels is created. In the post-deformation feature point extraction step, three or more feature points in the post-deformation image are extracted, and in the feature point displacement calculation step, the feature point displacement is obtained by comparing the corresponding feature points before and after the deformation. It is done. In the post-deformation point interpolation step, the point is interpolated in the image after deformation based on the displacement of three or more feature points, and in the search region setting step, based on the point position in the image after deformation. A search area is set. In the subset-corresponding region search step, the subset-corresponding region is searched from the deformed image while moving the reference point in the subset within the search region. In the displacement or strain calculation step, the displacement or strain of the object is obtained based on the position of the subset and the position of the subset-corresponding region.

  The displacement or strain measurement method of the present invention can be a method further comprising a post-deformation point estimation step. In the post-deformation point estimation step, the position of the point is estimated within the subset equivalent region. In this case, when the estimated position of the target point is at the periphery of the subset-corresponding region, the search region is expanded and the subset-corresponding region searching step is performed again.

The displacement or strain calculation program and the displacement or strain measurement method of the present invention have the following effects.
(1) Since only an image is mainly used, the displacement and strain of a concrete structure or the like can be easily obtained over a wide range at a time compared to a strain gauge.
(2) Since the search range is extremely small as compared with the conventional case, the calculation time is remarkably compressed, that is, the cost for calculation can be greatly suppressed.
(3) Even if the calculation time is shortened, distortion and the like can be obtained with the same accuracy as in the prior art.

The flowchart which shows the flow of the main processes (process) of this invention. Explanatory drawing which shows the image acquired by image | photographing a concrete beam with a digital camera. (A) is a model figure which shows four corner points extracted from the image before a deformation | transformation, (b) is a model figure which shows four corner points extracted from the image after a deformation | transformation. (A) is a model figure which shows the subset created in the image before a deformation | transformation, (b) is a model figure which shows the search area | region set in the image after a deformation | transformation. The model figure which shows the relative positional relationship of a pre-deformation corner point and a post-deformation corner point. The model figure which shows the condition where a subset scans in the image after a deformation | transformation. (A) is a model diagram showing a state in which the position of the post-deformation target point is obtained at the periphery of the search area, and (b) is expanded by adding 11 pixels outside the pixel from which the post-deformation point is obtained. The model figure which shows a search area | region. The flowchart which shows a part of flow of the process (process) in the case of extending a search area | region. Explanatory drawing which shows the image matching method in an image correlation method.

  An example of an embodiment of the “displacement or strain calculation program and displacement or strain measurement method” of the present invention will be described with reference to the flowchart of FIG. FIG. 1 is a flowchart showing the flow of main processes (processes) of the present invention, showing actions (processes, processes) to be performed in the center column, and the left column showing input information necessary for the actions, The right column shows the output information resulting from the action. Here, for convenience, the object to be measured is described as a concrete beam. However, the present invention is not limited to a concrete beam, and various objects can be used as objects.

  First, prepare an image of the concrete beam. This image can be acquired by, for example, photographing with a digital camera. FIG. 2 is an explanatory view showing an image 20 obtained by photographing the concrete beam 10 with a digital camera. In this figure, three images 20 in which the concrete beam 10 is partially copied are obtained. The images may be taken at intervals as shown in this figure, the images may be taken so as to cover the whole while overlapping adjacent images 20, or the whole may be taken with one image.

  The image 20 is prepared as a set of two before and after the concrete beam 10 is deformed. That is, in the case of FIG. 2, at least 2 × 3 sets of images are prepared. For convenience, the pre-deformation image 20 of the concrete beam 10 will be described as “pre-deformation image 21”, and the post-deformation image will be described as “post-deformation image 22”. The corresponding (ie, compared) pre-deformation image 21 and post-deformation image 22 are preferably taken from the same range such as by fixing the position / posture of the digital camera, but change before and after the deformation of the concrete beam 10. If a plurality of fixed points that are not taken are photographed so as to be included in both images, it is not always necessary to have the same range.

  The processing described below can be executed by using a computer device. This computer apparatus can be configured by a personal computer (PC), a tablet terminal such as iPad (registered trademark), a smartphone, or a PDA (Personal Data Assistance). The computer device includes a processor such as a CPU, a memory such as a ROM and a RAM, and further includes an input means such as a mouse and a keyboard and a display. In addition, although it inputs from devices, such as a mouse | mouth and a keyboard, if it is a general PC, it is often input by operation (a tap, pinch in / out, a slide, etc.) using a touch panel with a tablet type terminal or a smart phone.

  For example, the pre-deformation image 21 is read from the image data server, and feature points in the pre-deformation image 21 are extracted (Step 10). Here, the feature point is a point that is characteristic as an image, and is a constituent point of the main area surrounding the target point as will be described later. FIG. 3 is a model diagram showing four corner points extracted from the image 20. FIG. 3A shows the case of the untransformed image 21, and FIG. Here, for the sake of convenience, the corner point obtained in the pre-deformation image 21 is set to “pre-deformation corner point 31” as shown in FIG. 3A, and the corner point obtained in the post-deformation image 22 is shown in FIG. ), “Deformed corner point 32”.

  Since the pre-deformation corner point 31 is a point constituting the main region, three or more points (four points in FIG. 3) are extracted and specified from one pre-deformation image 21. The pre-deformation corner point 31 can be designated by an operator operation, but can also be automatically recognized by an image recognition technique conventionally used. Therefore, it is preferable to mark the pre-deformation corner point 31 on the surface of the concrete beam 10 in advance or to give a random pattern by spraying or the like so that the surface of the concrete beam 10 is not uniform.

  When the pre-deformation corner point 31 can be extracted, as shown in FIG. 3A, the main region 40 surrounded by the pre-deformation corner point 31 is formed in the pre-deformation image 21 (Step 20). Then, a mark is specified from the main area 40 (Step 30). Also in this case, as with the pre-deformation corner point 31, the target point can be designated by an operator operation, but it can also be automatically recognized by an image recognition technique used conventionally. Therefore, a characteristic point that can be distinguished from the other on the surface of the concrete beam 10 may be used as a mark. Here, for the sake of convenience, the standard obtained in the pre-deformation image 21 is “pre-deformation standard 51” as shown in FIG. 3A, and the standard obtained in the post-deformation image 22 is “post-deformation standard”. Point 52 ".

  When the pre-deformation reference point 51 can be extracted, the subset 60 is created in the pre-deformation image 21 (Step 40). The subset 60 is a region including a plurality of pixels (for example, 101 × 101 pixels) and including the pre-deformation reference point 51. For example, as shown in FIG. 4A, a subset 60 centered on the pre-deformation reference point 51 can be created.

  The processing up to this point is the processing for the pre-deformation image 21, and the subsequent processing is processing using the post-deformation image 22. First, for example, the post-deformation image 22 is read from the image data server, and the post-deformation corner point 32 in the post-deformation image 22 is extracted in the same manner as the process of extracting the pre-deformation corner point 31 (Step 50). FIG. 3B shows four post-deformation corner points 32 obtained in the post-deformation image 22.

  Next, as shown in FIG. 5, the position of the pre-deformation corner point 31 and the position of the post-deformation corner point 32 are compared in the post-deformation image 22. As described above, the pre-deformation image 21 and the post-deformation image 22 are images of the same range, or the fixed points that do not change before and after the deformation of the concrete beam 10 are included in both images. The relative positions of the corner point 31 and the post-deformation corner point 32 can be obtained. In addition, the position of the pre-deformation corner point 31 and the post-deformation corner point 32 can be compared in the pre-deformation image 21, or the comparison is performed using another image different from the pre-deformation image 21 and the post-deformation image 22. You can also.

  If the relative positions of the pre-deformation corner point 31 and the post-deformation corner point 32 can be confirmed, how much each corner point is displaced before and after the deformation of the concrete beam 10 is calculated (Step 60). Specifically, paying attention to the same corner point, a displacement amount (length) and a displacement direction, that is, a displacement vector, are obtained from the position coordinate of the pre-deformation corner point 31 and the position coordinate of the post-deformation corner point 32. This is repeated at all corner points (four points in FIG. 5), and the process proceeds to the next process.

  When the displacement vectors of the respective corner points are obtained, as shown in FIG. 5, a post-deformation target point 52 is inserted into the post-deformation image 22 (Step 70). Based on the displacement vectors of the four corner points, the displacement vector of the pre-deformation standard point 51 is estimated, and the position of the post-deformation standard point 52 is calculated. When performing this interpolation processing, it is possible to calculate using a shape function mainly used in the finite element method. Of course, as long as the calculation method is performed based on the displacement vector of the corner points, other methods can be adopted in addition to the shape function. It should be noted that the position of the post-deformation target point 52 calculated here is not final and can be regarded as a temporary position.

  When the post-deformation point 52 is interpolated in the post-deformation image 22, as shown in FIG. 3B, a search area 70 composed of a plurality of pixels is set based on the post-deformation point 52 (Step 80). ). In FIG. 3B, a search area 70 centered on the post-deformation point 52 is set. The search area 70 may be an area smaller than the subset 60 created in the pre-deformation image 21 (Step 40). For example, if the subset 60 is 101 × 101 pixels, the search area 70 can be set to 31 × 31 pixels or 21 × 21 pixels.

  When the search area 70 is set in the post-deformation image 22, the sub-set 60 scans the post-deformation image 22 (Step 90). At this time, scanning is performed such that a predetermined reference point in the subset 60 (for example, the center point of the subset 60) sequentially moves in the search area 70. FIG. 6 is a model diagram showing a situation in which the subset 60 scans the image 22 after deformation, and shows a state in which the center point of the subset 60 is sequentially moved in the search area 70 (from the upper left in the figure). .

  Each time the subset 60 is moved, for example, by one pixel, the luminance (or color) of the pixel of the subset 60 is compared with the luminance (or color) of the pixel of the modified image 22 that overlaps the subset 60. The luminance and the like are compared with all the pixels constituting the subset 60, and the correlation between the subset 60 and the transformed image 22 is obtained. At this time, the degree of difference described above may be used. A plurality of image correlations are obtained while moving the subset 60 in the search area 70, for example, an area where the image is closest (for example, a part of the search area 70) is searched, for example, the dissimilarity is minimized, and this area corresponds to the subset. Specify as an area. Since the position of the post-deformation target point 52 is assumed in advance, the search area 70 can be limited to a small range, and therefore the calculation process for the image search can be significantly shortened.

  When the subset-corresponding area can be specified, a predetermined position (for example, the center point) in the subset-corresponding area is determined as the position of the post-deformation target point 52 (Step 100). Since one pre-deformation image 21 may include a plurality of pre-deformation reference points 51, the process of determining the post-deformation reference point 52 position from the process of identifying the pre-deformation reference point 51 (Step 30) (Step 30). Step 100) is repeated as many times as the number of pre-deformation gauge points 51.

  By the way, as a result of reducing the search area 70, the position of the post-deformation point 52 may be obtained at the end (periphery) of the search area 70 as shown in FIG. In this case, it is considered that the position of the post-deformation target 52 should be obtained outside the set search area 70 in practice. Therefore, in such a case, it is possible to perform an image search after further expanding the search area 70. For example, in FIG. 7B, area expansion is performed in which 11 pixels are added outside the pixel (shaded portion) from which the post-deformation point 52 is obtained.

  With reference to FIG. 8, the process (process) in the case of extending a search area | region is demonstrated in detail. The process described so far is performed to determine the position of the post-deformation point 52 (Step 100). Then, it is determined whether or not the position of the post-deformation point 52 is at the periphery of the set search area 70 (Step 120). When the post-deformation point 52 is not on the periphery of the search area 70 (No), the process proceeds to the next process. That is, when there is another undeformed mark 51 in the same undeformed image 21, the process proceeds to Step 30, and when there is no other undeformed mark 51, the process proceeds to Step 110.

  On the other hand, when the post-deformation point 52 is at the periphery of the search area 70 (Yes), the search area 70 is expanded to the periphery as shown in FIG. 7B (Step 130). Then, the image search (Step 90) to the position of the post-deformation target point 52 (Step 100) is performed using the expanded search area 70, and it is determined again whether or not the post-deformation target point 52 is the periphery of the search area 70 (Step 120). . The process is repeated until the post-deformation point 52 is located inside the search area 70 (a position that is not the periphery), and the process proceeds to the next process (Step 30 / Step 110).

When the processing described so far is performed on all the pre-deformation marks 51 in one pre-deformation image 21, and is further performed on all the obtained pre-deformation images 21 (three in FIG. 2). The displacement or strain of the deformed concrete beam 10 is calculated. At this time, the partial displacement amount of the concrete beam 10 can be obtained based on the change in the position of the subset 60 and the position of the subset equivalent region, and the partial strain can be calculated based on the displacement amount. . Alternatively, the partial displacement amount and strain of the concrete beam 10 can also be calculated based on the changes in the positions of the pre-deformation gauge point 51 and the post-deformation gauge point 52. When the strain is calculated over the entire concrete beam 10, a strain distribution chart such as a maximum main strain distribution chart and a minimum main strain distribution chart can be output.

  The displacement or strain calculation program and the displacement or strain measurement method of the present invention can be used effectively when grasping the main strain distribution in an important concrete structure such as a bridge substructure and a tunnel lining. By grasping the main strain distribution of the important structures in service, it is possible to repair and reinforce at an early stage. Considering the provision of measures, the present invention is an invention that can be used not only industrially but also can be expected to make a great social contribution.

DESCRIPTION OF SYMBOLS 10 Concrete beam 20 Image 21 Pre-deformation image 22 Post-deformation image 31 Pre-deformation corner point 32 Deformation corner point main area 40 Main area 51 Pre-deformation point 52 Deformation point 60 Subset 70 Search area

Claims (5)

In a program for causing a computer to execute a function for obtaining displacement or distortion of an object by comparing images before and after the deformation of the object,
A pre-deformation feature point recognition process for identifying three or more feature points in the image before the deformation;
A subset creation process for creating a subset of a plurality of pixels including a target point in an area formed by three or more feature points;
A post-deformation feature point recognition process that identifies three or more feature points in the post-deformation image;
A feature point displacement calculation process for obtaining a displacement of the feature point by comparing the feature points before and after the corresponding deformation;
A post-transformation point interpolation process for interpolating the base point in the image after deformation based on the displacement of three or more feature points;
A search area setting process for setting a search area based on the mark position in the transformed image;
A subset-corresponding area search process for searching a subset-corresponding area that most closely approximates the image of the subset while moving a reference point in the subset in the search area;
A displacement or strain calculation process for determining a displacement or strain of the object based on the position of the subset and the position of the subset-corresponding region, and a function for causing the computer to execute the displacement or strain Strain calculation program. A function of causing the computer to execute a post-transformation point estimation process for estimating a position of the point in the subset-corresponding region;
When the position of the target point estimated by the post-transformation target point estimation process is at the periphery of the subset equivalent area, the search area is expanded, and the subset equivalent area search process is executed again. The displacement or strain calculation program according to claim 1, wherein:   3. The displacement or strain calculation program according to claim 1, wherein, in the search area setting process, the search area is set in an area smaller than the subset. In a method for measuring displacement or strain of an object by comparing images before and after deformation of the object,
A pre-deformation feature point extracting step of extracting three or more feature points in the image before the deformation;
Forming a main region surrounded by three or more feature points, identifying a target point in the main region, and generating a subset including a plurality of pixels including the target point; and
A post-deformation feature point extraction step of extracting three or more feature points in the image after deformation;
A feature point displacement calculating step for obtaining a displacement of the feature point by comparing the feature points before and after the corresponding deformation;
A post-transformation point interpolation step for interpolating the standard point in the image after deformation based on the displacement of three or more feature points;
A search area setting step for setting a search area based on the mark position in the transformed image,
A subset-corresponding region search step of searching a subset-corresponding region that most closely approximates the image of the subset while moving a reference point in the subset in the search region;
A displacement or strain measurement method comprising: a displacement or strain calculation step for obtaining a displacement or strain of the object based on the position of the subset and the position of the subset-equivalent region. In the subset equivalent region, further comprising a post-transformation point estimation step for estimating the position of the point,
When the position of the target point estimated by the post-transformation target point estimation step is at the periphery of the subset equivalent region, the search region is expanded and the subset equivalent region search step is performed again. The displacement or strain measurement method according to claim 4.

Images