JP7227481B2 - Stone locating program, stone locating system, and stone locating device - Google Patents

Description

The present invention relates to a stone locating program, a stone locating system, and a stone locating device.

Some parts of castles, shrines and temples may collapse due to disasters. Because of their value as cultural assets shared by all humankind, the collapsed parts are repaired by the people concerned.

However, for example, when part of the stone wall of a castle has collapsed due to an earthquake and the part of the collapsed stone wall is to be repaired, the repair work may be proceeded by visual inspection by a stonemason. Such repair work is very troublesome, time-consuming and costly.

Therefore, there are cases where stone walls are repaired using IT (Information Technology) technology. For example, there are the following technologies.

That is, 3D stone wall data obtained by laser measurement of the surface shape of the stone wall before the stone wall collapsed, and the exposed part of the stone wall exposed on the surface of the stone wall before the stone wall collapsed were obtained by laser measurement. There is a technique for identifying the arrangement of stone walls by comparing with 3D exposed part data.

According to this technology, it is said that the stones that have fallen from the surface of the stone wall can be restored to the surface of the stone wall with high precision.

In addition, an approximation line that approximates the ridge line to be restored is obtained by referring to the three-dimensional shape data of each stone that constitutes the stone wall, and the design slope of the stone wall is defined based on the approximation line, and interference between adjacent stone materials is obtained. There is also a technique to determine the placement position of the stone so that it matches the design slope while checking the condition.

According to this technology, it is said that it is possible to use a computer to study the construction of stone wall restoration in advance.

JP 2018-104985 A JP 2008-196231 A

However, the technique of comparing the three-dimensional stone wall data and the three-dimensional exposed part data to specify the arrangement of the stone wall obtains each data by laser measurement. However, disasters are difficult to predict in advance. With such a technique, if the stone wall before the collapse is not measured with a laser, the three-dimensional stone wall data and the three-dimensional exposed portion data cannot be used. Therefore, with such a technique, it may not be possible to accurately match each stone material constituting the stone wall before the collapse with each stone material after the collapse.

In addition, this technique does not describe how to specifically acquire three-dimensional stone wall data and three-dimensional exposed portion data by laser measurement. Therefore, with such a technique, it may not be possible to acquire each data, and it may not be possible to accurately match each stone material constituting the stone wall before the collapse with each stone material after the collapse.

In addition, the technology to determine the placement position of stones so that they match the design slope while checking the interference state of adjacent stones is to make four markings on each stone of the stone wall before restoration, and furthermore, a reference ball The relative position with is also measured by a measuring machine. As described above, it is difficult to predict disasters, and if such measurements cannot be performed in advance, it is not possible to obtain three-dimensional shape data for each stone that constitutes the stone wall. Therefore, even with such a technique, there are cases where it is not possible to accurately match each stone material constituting the stone wall before the collapse with each stone material after the collapse.

Therefore, one disclosure is to provide a stone position specifying program, a stone position specifying system, and a stone position specifying device that can accurately match the stone before and after the stone wall collapsed.

One disclosure is a stone position specifying program executed by a computer of a stone position specifying device having a memory in which image data of a stone wall image before it collapses and image data of a stone image after it has collapsed from the stone wall are stored. and generating image data of divided stone wall images obtained by dividing the stone wall image based on the image data of the stone wall image, and rotating and reducing or rotating and enlarging the stone image based on the image data of the stone wall image. Image data of the rotated stone image is generated, and the image data of the divided stone wall image and the image data of the rotated stone image are used to determine the stone contained in the stone wall image before the collapse and the stone image after the collapse. The object is to perform pattern matching with respect to the stone material to be used and to output the result of the pattern matching.

According to one disclosure, it is possible to accurately match the stone material before the stone wall collapsed and the stone material after the stone wall collapsed.

FIG. 1 is a diagram showing a configuration example of a stone position specifying device. FIG. 2 is a diagram showing a configuration example of a stone location specifying system. FIG. 3 is a flow chart showing an operation example. FIG. 4 is a diagram showing an example of a stone wall image before it collapses. FIG. 5A is a diagram showing an example of a stone wall image before collapse, and FIG. 5B is an example of a stone wall image after trimming. FIG. 6A shows an example of a stone wall image after trimming, and FIG. 6B shows an example of a divided stone wall image. FIGS. 7A and 7B are diagrams showing examples of stone material images after the collapse. FIG. 8 is a diagram showing an example of a stone material image after trimming. 9A and 9B are diagrams showing examples of pattern matching. FIGS. 10A and 10B are diagrams showing examples of pattern matching. FIG. 11 is a diagram showing an example of pattern matching results. FIG. 12 is a diagram showing an output example. FIG. 13 is a diagram showing an example of a stone wall image in which the specified stone material is painted black. FIG. 14 is a diagram showing an example of pattern matching results. FIG. 15 is a diagram showing an example of the number of times of matching and the percentage of correct answers. FIG. 16 is a flow chart showing an operation example. FIGS. 17A to 17C are diagrams showing examples of keystone correction. FIGS. 18A and 18B are diagrams showing an example of dividing a stone image.

EMBODIMENT OF THE INVENTION Hereinafter, the form for implementing this invention is demonstrated. It should be noted that the following examples do not limit the technology disclosed. Further, each embodiment can be appropriately combined within a range in which the processing contents are not inconsistent.

[First embodiment]
<Configuration example of stone position specifying device>
FIG. 1 is a diagram showing a configuration example of a stone position specifying device 100. As shown in FIG.

The stone position specifying device 100 uses the stone wall image before the stone wall collapsed and the stone image after the stone wall collapsed to identify the position of the stone wall after the stone wall collapsed. Specifically, the stone position identifying device 100 can use pattern matching to identify the position of the stone wall before the collapse, where the stone after the collapse was located.

As shown in FIG. 1 , the stone position specifying device 100 includes a memory 101 , an image processing section 102 , a monitor section 103 , an operation section 104 and an IF (Interface) 105 .

The memory 101 is, for example, a storage device such as a RAM (Random Access Memory) or HDD (Hard Disk Drive). The memory 101 stores stone wall image data relating to the image of the stone wall before it collapses and stone image data relating to the image of the stone wall after it has collapsed from the stone wall. In addition, the memory 101 stores image data processed by the image processing unit 102 as appropriate.

The stone wall image data is stored in the memory 101 by, for example, the following processing. That is, the operator scans a photograph of the stone wall image before the collapse with a scanner device connected to the stone position specifying device 100, whereby stone wall image data relating to the stone wall image before the collapse is acquired, and the IF 105 and the image processing unit It is stored in memory 101 via 102 . Alternatively, the IF 105 receives the stone wall image data relating to the pre-collapse stone wall image transmitted from another device connected to the stone position specifying device 100, and the image data is stored in the memory 101 via the image processing unit 102. good.

The stone image data may also be stored in the memory 101 via the IF 105 or transmitted from another device, for example, by the operator scanning a photograph of the stone image using a scanner device. It can be anything.

The image processing unit 102 reads image data from the memory 101 and performs image processing. Specifically, based on the stone wall image data read from the memory 101, the image processing unit 102 acquires the divided stone wall image data related to the divided stone wall images obtained by dividing the stone wall image. In addition, based on the stone image data read from the memory 101, the image processing unit 102 acquires rotated stone image data relating to the rotated stone image obtained by rotating and reducing (or rotating and enlarging) the stone image. The image processing unit 102 uses the divided stone wall image data and the rotating stone image data to match the stones included in the pre-collapse stone wall image and the post-collapse stone wall image, and outputs the result. Details will be described in an operation example.

Note that the functions of the image processing unit 102 may be realized by, for example, an FPGA (Field Programmable Gate Array). For example, the FPGA 110 reads a program from the memory 101 and executes it, thereby implementing the functions of the image processing unit 102 . The FPGA 110 corresponds to the image processing unit 102, for example. A processor such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor), a controller, or the like may be used instead of the FPGA.

The monitor unit 103 displays an output image indicating the matching result based on the image data output from the image processing unit 102 and indicating the matching result. The monitor unit 103 can also appropriately display the output result based on the data indicating the image processing result output from the image processing unit 102 .

The operating unit 104 is, for example, a keyboard. By operating the operation unit 104 , the operator can designate a position within the image displayed on the monitor unit 103 , and output data regarding the designated position to the image processing unit 102 . The image processing unit 102 can perform a process of extracting (or trimming) a part of the image included in the image based on the positional data.

Note that the operation unit 104 may be included in the monitor unit 103, for example, and the operation functions of the operation unit 104 may be realized by screen touch operations.

The IF 105 is connected, for example, via a network to another device (for example, a personal computer or a scanner device), and receives image data transmitted from the other device via the IF 105 . The IF 105 also receives, for example, data about the matching result output from the image processing unit 102, and transmits the input data about the matching result to another device.

In addition, below, each stone which comprises a stone wall may be called a stone material, for example. A stone wall is, for example, a stone wall that is made by stacking stones one by one.

Also, hereinafter, one image, one image, and one image frame may be used without distinction. Furthermore, there are cases where images and image data are used without distinction.

<Stone Positioning System>
FIG. 2 is a diagram showing a configuration example of the stone position specifying system 10. As shown in FIG.

The stone position specifying system 10 includes a stone position specifying device 100 and a personal computer 200, which are connected to each other via a network.

For example, the personal computer 200 transmits instruction information (or request information) instructing (or requesting) to specify the position of the stone wall to the stone position specifying device 100, and the stone position specifying device 100 specifies the stone position according to the instruction information. , and transmits data indicating the result of pattern matching to the personal computer 200 . The personal computer 200 can acquire the matching result from the received data indicating the matching result.

Although FIG. 2 shows an example of one personal computer 200 , a plurality of personal computers 200 may be included in the stone location specifying system 10 .

Further, in FIG. 2, the stone position specifying device 100 may serve as a cloud server for the personal computer 200. FIG. In this case, each image data, matching result data, etc. are stored in the stone position specifying device 100, and the personal computer 200 accesses the stone position specifying device 100 as appropriate to acquire each image data, matching result, etc. good too.

<Operation example>
FIG. 3 is a flow chart showing an operation example. An example of the operation will be described below with specific examples of images.

As shown in FIG. 3, when the stone position specifying device 100 starts processing (S10), it acquires an image of the stone wall before it collapses (S11).

FIG. 4 is a diagram showing an example of a stone wall image of a castle before the stone wall collapses. It is desirable that such a stone wall image is an image photographed from the front with respect to the surface of the stone wall.

Image data of such a stone wall image is obtained by using, for example, a scanner device as described above, and stored as stone wall image data in the memory 101 .

Returning to FIG. 3, next, the stone position specifying device 100 acquires an image of the pre-collapse stone wall image with the background removed (S12).

The image of the stone wall that remains after removing the background is, for example, the portion of the stone wall that has collapsed. The object of specifying the position of the stone material is the stone material included in the collapsed stone wall. In this process, the portion of the stone wall that remained without collapsing is deleted, and the portion of the stone wall that has collapsed is extracted (or trimmed). ) to acquire (or generate) the image data of the stone wall image after trimming.

As a result, compared to, for example, using the entire stone wall image, the amount of image data for the target stone wall image is reduced, so the calculation process can be reduced, and the accuracy of pattern matching in the subsequent stage can be improved. becomes.

FIG. 5A is a diagram showing an example of a stone wall image before it collapses. For example, the stone location identifying device 100 performs the following processing to delete the background and trim the stone wall included in the stone wall image.

That is, the image processing unit 102 outputs the stone wall image data read from the memory 101 to the monitor unit 103 . As a result, for example, the stone wall image shown in FIG. 5A is displayed on the monitor unit 103 . Then, the operator uses the operation unit 104 to identify the position of the collapsed portion of the stone wall while viewing the image of the stone wall displayed on the monitor unit 103 . The black dotted line shown in FIG. 5(A) represents the boundary between the collapsed stone wall and the remaining stone wall. The operation unit 104 outputs data regarding the specified position (for example, data regarding the positional coordinates of the four points) to the image processing unit 102 . The image processing unit 102 deletes the image outside the dotted black frame from the stone wall image and trims the image inside the dotted black frame based on the data regarding the specified position. For example, the image processing unit 102 performs trimming by extracting the image data of the stone wall image included in the position coordinates of the four points using the data on the position coordinates of the four points, and extracts the image data of the stone wall image after the trimming. to generate

FIG. 5B is a diagram showing an example of the stone wall image after trimming. The image processing unit 102 stores, for example, image data related to the stone wall image after trimming in the memory 101 .

Returning to FIG. 3, next, the stone position specifying device 100 acquires divided stone wall images obtained by dividing the stone wall image before the collapse (S13). The stone position specifying device 100 obtains divided stone wall images by dividing the trimmed stone wall image.

FIG. 6A is a diagram showing an example of a stone wall image after trimming. The stone position specifying device 100 acquires divided stone wall images by, for example, performing the following processing.

That is, the image processing unit 102 treats the stone wall image after trimming as one block, each region having a predetermined number of pixels vertically and horizontally. (for example, the first number of pixels), and several pixels in the vertical direction (for example, the second number of pixels. The first number of pixels and the second number of pixels may be the same or different). Get an image of each block. The image processing unit 102 stores the image data of the image thus obtained in the memory 101 as the image data of the divided stone wall image.

FIG. 6B is a diagram showing an example of a divided stone wall image. In the example of FIG. 6A, for example, image data of several thousand divided stone wall images are acquired.

Returning to FIG. 3, on the other hand, the stone position specifying device 100 acquires a stone image after the collapse (S14).

FIGS. 7A and 7B are diagrams showing examples of stone material images after the collapse. The image data of the stone material image is acquired using a scanner device or the like and stored in the memory 101 as described above.

Note that the stone shown by the stone image is, for example, the stone existing on the surface of the stone wall before it collapsed. The part of the stone material facing the surface side of the stone wall has a different brightness (or a different color) compared to the other part because it receives the sun, etc., and the stonemasons can immediately understand it when they see it. It is possible to Therefore, the stone material image used to identify the position of the stone wall may be selected by a masonry worker as having a high possibility of being the surface of the stone wall. For example, an operator takes a picture of a stone selected in such a way using a camera, etc., reads it with a scanner device, etc., or transmits it as stone image data. may be obtained.

Returning to FIG. 3, next, the stone position specifying device 100 acquires an image of the post-collapse stone image with the background removed (S15). The stone position specifying device 100 removes the background of the stone image and extracts the image data of the stone portion included in the stone image.

FIG. 8 is a diagram showing an example of a stone image after trimming with respect to the stone image after collapse in FIG. 7B. For example, the stone position specifying device 100 performs the following processing to generate (or extract) image data of the trimmed stone image.

That is, the image processing unit 102 reads the image data of the stone image after the collapse read out from the memory 101 and outputs it to the monitor unit 103, and the monitor unit 103 displays the stone image after trimming. The operator operates the operation unit 104 while viewing the stone image displayed on the monitor unit 103 to specify the positions of four points including the stone image. The operation unit 104 outputs the positional coordinate data of the designated four points to the image processing unit 102 . The image processing unit 102 extracts the image data of the stone material image contained within the four points based on the data of the positional coordinates of the four points. Then, the image processing unit 102 extracts image data included in the edge by performing edge extraction processing or the like based on the extracted image data. By this extraction, the image processing unit 102 can extract the image data of the trimmed stone image (the image of the stone portion included in the stone image) shown in FIG. 8, for example. The image processing unit 102 stores the image data of the stone image after trimming in the memory 101 .

Returning to FIG. 3, next, the stone position specifying device 100 performs rotation and reduction processing on the trimmed stone image (S16). The stone position specifying device 100 performs rotation and reduction processing, for example, by performing the following processing.

That is, the image processing unit 102 generates a rotated stone image by performing rotation transformation processing using a known rotation matrix on the image data acquired in S15. At this time, the image processing unit 102 changes θ in the rotation matrix by 10 degrees, for example, 10 degrees, 20 degrees, 30 degrees, . Generate image data for each rotated image.

Further, the image processing unit 102 generates a reduced stone image by performing reduction conversion processing using a known reduction matrix on the image data of each rotated image. At this time, the image processing unit 102 generates a reduced image by multiplying each rotated image by, for example, 0.9 times, and then generates a reduced image by further multiplying the obtained image by 0.9, Next, the obtained image is further multiplied by 0.9, such as 0.9, and repeated 13 times to generate a reduced image after rotation.

That is, the image processing unit 102 generates a stone image obtained by rotating the trimmed stone image obtained in S15 by 10 degrees, and multiplies the stone image by 0.9 times. An image is generated, a stone image is generated by multiplying the stone image by 0.9, and a stone image is generated by multiplying the stone image by 0.9. At this time, the image processing unit 102 repeats 0.9 times 13 times. Then, the image processing unit 102 generates a stone image further rotated 10 degrees (20 degrees for the image acquired in S15) by rotation conversion processing, and rotates the stone image rotated 10 degrees. , 0.9 is repeated 13 times to generate a reduced image after rotation. Thereafter, the image processing unit 102 rotates by 10 degrees, repeats up to 360 degrees, and repeats 0.9 times 13 times for each rotated image rotated by 10 degrees.

In this case, the image processing unit 102 generates image data of 36×13=468 stone images for one stone image.

An image obtained by rotation and reduction may be referred to as, for example, a rotated stone image.

Also, in the example of S16, an example in which the stone image is reduced has been described, but instead of the reduction, the stone image may be enlarged. In this case, the image processing unit 102 may perform enlargement conversion processing using a known enlargement matrix on the trimmed stone image. As for the enlargement conversion, the image processing unit 102 may repeat enlargement processing multiple times for one stone image.

Returning to FIG. 3, next, the stone position specifying device 100 uses the image data of the rotated stone image and the image data of the divided stone wall image to determine the pattern for the stone contained in the stone image and the stone contained in the stone wall image. Matching is performed (S17).

FIGS. 9A to 10B are diagrams showing examples of pattern matching.

The images to be pattern matched are the rotating stone image and the divided stone wall image.

On the other hand, as for the rotated stone image, a single trimmed stone image (hereinafter sometimes referred to as a "stone image") is subjected to a combination of rotation processing and reduction processing through the processing from S14 to S16. It is an image (rotating stone image) that has been processed by

In FIG. 9A, the stone image is rotated by +10 degrees and reduced by 0.9 times. In FIG. 9B, the stone image is rotated by +10 degrees. and 0.9×0.9 times reduction processing are respectively shown. In the reduction process, the stone image after rotation is multiplied by 0.9, which is repeated 13 times. 13 turning stone images are obtained.

FIG. 10A shows a stone image obtained by rotating the stone image by +20 degrees and reducing it by 0.9 times, and FIG. 10B shows the stone image of FIG. Each example of a stone material image further subjected to a reduction process of 0.9 times (0.9×0.9 times) is shown. In the reduction process, the stone image after rotation is multiplied by 0.9, which is repeated 13 times. 13 turning stone images are obtained.

On the other hand, by the processing from S11 to S13, x pieces of divided stone wall images obtained by sliding the (trimmed) stone wall image by 20 pixels vertically and horizontally are obtained by sliding one image of 100 pixels vertically and 150 pixels horizontally. can get.

Therefore, as shown in FIGS. 9A to 10B, the image processing unit 102 uses 468 individual rotating stone images and x individual divided stone wall images. , respectively, perform pattern matching.

In the above example, the stone image is rotated by 10 degrees. may Also, the reduction processing for the stone image may be a multiple other than 0.9 (for example, 0.5 or 0.8), and the number of repetitions may be other than 13 ( For example, it may be performed once or multiple times).

The image processing unit 102 performs pattern matching, for example, as follows.

That is, the image processing unit 102 calculates a local feature quantity in a feature region (for example, 48×48 pixels) consisting of a plurality of pixels around a predetermined feature point in each image of the rotated stone image and the divided stone wall image. calculate. As the local feature amount, for example, a plurality of pairs of pixels in the feature region are determined in advance, and a bit value corresponding to the sign of the luminance difference between each pair of pixels (for example, if the luminance difference is a positive value is represented by "1", negative value is represented by "0", etc.). When the image data of each image is represented by RGB, the image processing unit 102 may calculate the brightness value of each pixel using a known conversion formula. Also, the local feature amount may be, for example, a local feature amount represented by a BRIEF (Binary Robust Independent Elementary Features) feature. Then, the image processing unit 102 searches for the feature region in the divided stone wall image that is most similar to the local feature value of a certain feature region in the rotating stone image, and finds the feature in the two feature regions having the closest local feature value. Connect the points to generate a vector. The image processor 102 computes such vectors across the two images (the rotated stone image and the segmented stone wall image) and computes a histogram for the vectors. Then, the image processing unit 102 extracts the vector with the largest number in the histogram, calculates the number of feature points having such a vector in one image, and calculates the number of feature points included in one image. A ratio to the total number of feature points is calculated as the similarity. The image processing unit 102 calculates one degree of similarity between one rotated stone image and one divided stone wall image.

In addition to BRIEF, for example, ORB (Oriented fast and Rotated BRIEF), BRISK (Binary Robust Invariant Scalable Keypoints), etc. may be used as the local feature amount.

Further, the stone position specifying device 100 may perform pattern matching using, for example, the rotated and enlarged rotated stone image and the divided stone wall image. Even when a rotated and enlarged rotated stone image is used, the stone position specifying device 100 can perform pattern matching using the same method as when using the above-described rotated and reduced rotated stone image. .

Returning to FIG. 3, next, the stone position specifying device 100 outputs the result of pattern matching (S18).

FIG. 11 is a diagram showing an output example showing the result of pattern matching. In the first embodiment, the image processing unit 102 can output ten divided stone wall images in descending order of similarity to one stone image. At that time, as shown in FIG. 11, the image processing unit 102 can also output the similarity and the angle together with the top 10 divided stone wall images for one stone image. Here, the angle represents, for example, the angle when the rotation processing (S16) is performed when the degree of similarity shown in FIG. 11 is obtained.

As shown in FIG. 11, for the stone image having the identification number of stone #1, the top 10 divided stone wall images with the highest degree of similarity are shown, and the stone having the identification number of stone #2 is shown. As for the images, it indicates which of the top 10 divided stone wall images with the highest degree of similarity is. The image processing unit 102 is capable of outputting top 10 divided stone wall images with high similarity to all stone images.

The image processing unit 102 outputs the image data of each stone image and the image data of the top ten divided stone wall images for each stone image to the monitor unit 103 . At this time, the image processing unit 102 may also output each data of the identification number of each stone image, the identification number of each divided stone wall image, the degree of similarity, and the angle to the monitor unit 103, as shown in FIG.

Returning to FIG. 3, next, the stone position specifying device 100 performs result determination (S19), and determines whether or not a new stone wall position has been specified (S20). For example, the stone position specifying device 100 performs the following processing.

That is, the monitor unit 103 displays the top ten divided stone wall images having the highest degree of similarity together with the stone material image.

FIG. 12 is a diagram showing an example of a stone material image and a divided stone wall image displayed on the monitor unit 103. FIG. The operator visually confirms the stone image displayed on the monitor unit 103 and the top 10 divided stone wall images, and determines that the stone displayed in the stone image is one of the top 10 divided stone wall images. When it is specified that it is the stone in the image, the operation unit 104 is operated. Through such an operation, the operation unit 104 outputs data indicating that the stone has been specified to the image processing unit 102 . The image processing unit 102 determines S20 depending on whether or not this data has been received.

When determining that the position of the stone wall has been identified (Y in S20), the stone position identifying device 100 deletes the detected stone material from the image of the stone wall before the collapse (S21). When the stone position identifying device 100 identifies that the stone contained in the stone image is the stone contained in the stone wall before the collapse, the stone position identifying device 100 “blacks out” the stone to delete it from the stone wall image before the collapse. I have to. For example, the stone position specifying device 100 performs the following processing.

That is, the operator operates the operation unit 104 and designates the stone specified from the stone wall image displayed on the monitor unit 103 . The operation unit 104 outputs to the image processing unit 102 the identification information of the designated stone material, the position information in the image of the stone wall before the collapse, and the like. Based on the positional information, the image processing unit 102 performs image processing so that the stone contained in the specified position in the stone wall image becomes black, thereby converting the detected stone into the stone wall image. delete from

FIG. 13 is a diagram showing an example of a stone wall image after deleting the detected stone materials. In FIG. 13, blackened stones represent detected stones. The image processing unit 102 reads the image data of the pre-collapse stone wall image from the memory 101, specifies the position in the pre-collapse stone wall image based on the stone position information output from the operation unit 104, and determines the specified position. By applying edge processing and the like to the image data contained in , the stones contained in the specified position are extracted. Then, the image processing unit 102 sets the image data of the extracted stone material to R=0, G=0, and B=0, so that the detected stone material can be "painted black".

In addition to the "black painting", for example, the image processing unit 102 sets the image data of the extracted stones to R=255, G=255, and B=255, so that the detected stones in the stone wall image are You can also set the image to "white". Alternatively, the image processing unit 102 sets the extracted image data of the stone to a value representing a specific color, so that the color of the image of the detected stone in the stone wall image can be easily grasped by the operator. good.

In such identification, candidate divided stone wall images are displayed for each stone image. This is done by identifying any of the stones contained in the image. Therefore, even if the operator sees the 10 divided stone wall images displayed on the monitor unit 103, the operator determines that the stones contained therein and the stone reflected in the stone image are not the same. There may be stone images that cannot be identified.

Returning to FIG. 3, when the detected stone material is deleted from the pre-collapse stone wall image (S21), the stone location identifying apparatus 100 proceeds to S12, and processes the pre-collapse stone wall image from which the detected stone material has been deleted. process. The stone position specifying device 100 performs the processing from S12 onwards on the stone wall image before the collapse that has been "blacked out".

Then, the stone position specifying device 100 performs pattern matching using the divided stone wall images obtained by dividing the "blackened" pre-collapse stone wall image and the rotating stone image (S17).

In this way, when the first pattern matching fails to identify the stone, the stone location identifying device 100 can perform the second pattern matching based on the stone wall image excluding the detected stone. becomes.

For example, the second pattern matching may improve the stone detection rate compared to the first pattern matching.

FIG. 14 is a diagram showing an example of the result output of the first pattern matching for the stone image #10. As shown in FIG. 14, as a result of the first pattern matching, divided stone wall images #10 to #100 are calculated as top ten divided stone wall images having the highest degree of similarity with stone image #10. Also, a divided stone wall image #110 is calculated as the 11th ranked divided stone wall image.

For example, let us consider a case in which the divided stone wall image #30 has a high degree of similarity not to the stone image #10 but to another stone image #11 and becomes a detected stone by the first pattern matching. In this case, the divided stone wall image #30 is excluded from the top ten divided stone wall images with respect to the stone image #10, and the order of the subsequent divided stone wall images is moved up one by one. As a result, in the first pattern matching result, the divided stone wall image #110 ranked 11th is displayed on the monitor unit 103 as a result output that is included in the top 10 in the second pattern matching. It will be. Therefore, the operator can determine that the stone shown in the divided stone wall image #110 is the stone most similar to the stone contained in the stone image #10. Therefore, the detection rate for stone image #10 can be improved more than the first pattern matching. That is, the second re-matching process (S17) may improve the detection rate of stones included in the stone image.

FIG. 15 summarizes the results obtained by the above processing in tabular form. The stone position specifying device 100 can also perform pattern matching a plurality of times, and FIG. 15 shows an example in which pattern matching is repeated five times.

The percentage of correct answers shown in FIG. 15 represents the ratio of the number of identified stones to the total number of collapsed stones.

As shown in FIG. 15, in the first matching, the correct answer rate was x1%, but the correct answer rate increased as the number of times of matching increased, and in the fifth matching process, the correct answer rate was x5% (however, x1<x5). As compared with the correct answer rate of the 1st time, the correct answer rate of the 5th time increased. In addition, in FIG. 15, x1<x2<x3<x4<x5 is established.

In this manner, the stone position specifying device 100 can improve the probability (or detection rate) of detected stones by repeating pattern matching a plurality of times.

Returning to FIG. 3, on the other hand, when the stone position specifying device 100 cannot specify a new stone wall position (N in S20), the series of processing ends (S22). The stone position specifying device 100 terminates a series of processing when it is not possible to specify the position of the stone in the stone wall.

As described above, in the first embodiment, the stone position specifying device 100 rotates and reduces (or rotates and enlarges) the stone image, divides the stone wall image, and then converts the image data of each image into is used for pattern matching. As a result, it is possible to match the direction and size of the stones piled up on the stone wall and the stones included in the stone image in pattern matching. Accurate matching is possible.

In addition, in the stone material position specifying device 100 in the first embodiment, each stone material stacked on the stone wall is measured with a laser or marked to obtain three-dimensional data. It is possible to perform pattern matching without Therefore, the stone position identifying device 100 can efficiently identify the position of the stone wall where the collapsed stone is, without spending time and money.

[Second embodiment]
In the second embodiment, the stone position specifying device 100 corrects the stone image, and rotates and reduces (or rotates and enlarges) the corrected stone image.

FIG. 16 is a flow chart showing an operation example in the second embodiment.

As shown in FIG. 16, the stone position specifying device 100 removes the background of the stone image after the collapse and trims it (S15), and then performs correction processing on the trimmed stone image (S30). Correction processing includes keystone correction and division correction. First, keystone correction will be described.

<Keystone Correction>
FIGS. 17A to 17C are diagrams showing examples of keystone correction.

For example, as shown in FIG. 17A, there are cases where the stone material reflected in the stone material image does not face the front. In such a case, the stone position specifying device 100 performs trapezoidal correction on the stone image to obtain a front facing stone image.

That is, the stone position specifying device 100 deletes the background and trims the image of the stone portion of the collapsed stone image, as shown in FIG. 17(B). Then, as shown in FIG. 17C, the stone position specifying device 100 performs trapezoidal correction on the stone image after trimming, and the stone image in which the stone faces the front (hereinafter referred to as a stone image after trapezoidal correction). ) is generated. Returning to FIG. 16, the stone position specifying device 100 performs rotation processing and reduction processing on the stone image after trapezoidal correction (S31). The rest is the same as in the first embodiment.

Keystone correction may be performed by, for example, a known method. For example, the stone position specifying device 100 performs the following processing. That is, the image processing unit 102 reads out the trimmed image data from the memory 101 and outputs it to the monitor unit 103 . A stone image after trimming is displayed on the monitor unit 103 . The operator uses the operation unit 104 to specify arbitrary four points in the stone image after trimming displayed on the monitor unit 103 . Also, the operator uses the operation unit 104 to specify any four points viewed from the front. The image processing unit 102 calculates a matrix used for projective transformation based on the information regarding the positions of all eight points obtained from the operation unit 104 . The image processing unit 102 uses the calculated matrix to perform projective transformation on the stone image after trimming, so that the image data (for example, the figure 17(C)). Trapezoidal correction is performed by such projective transformation, for example.

<Division Correction>
Next, correction by division (hereinafter sometimes referred to as “division correction”) will be described. Division correction is, for example, division and correction of the stone image. A stone image obtained by division may be referred to as, for example, a post-division stone image.

FIG. 18A is a diagram showing an example of a stone image after trimming, and FIG. 18B is an example of a stone image after division.

The stone position specifying device 100 calculated the degree of similarity between the two images during pattern matching (S17). The degree of similarity represents, for example, the ratio of the number of feature points having the largest number of vectors between two images to all feature points included in one image. Portions with the highest number of such vectors may be concentrated in certain regions within an image. That is, as shown in FIG. 18A, one image (stone image after trimming) may include areas with high similarity and areas with low similarity.

Therefore, if there is an area with such a high degree of similarity, the stone position specifying device 100 calculates the degree of similarity of that area as the degree of similarity of the entire image, thereby distributing the degree of similarity within one entire image. It is possible to avoid a situation where the similarity is lowered, thereby avoiding a situation where the similarity is lowered. For this reason, the stone position specifying device 100 divides the stone image (S30) before pattern matching (S17). For example, the stone position specifying device 100 divides the stone image and generates image data of the divided stone image by the following processing.

That is, the image processing unit 102 extracts the image data of the trimmed stone image from the memory 101 and divides the stone image into predetermined regions. The image processing unit 102 collects the image data extracted for each region (or as one file data), stores it in the memory 101 or outputs it to the monitor unit 103 .

FIG. 18B shows an example of the post-division stone image displayed on the monitor unit 103 based on the image data collectively received for each region. After that, the image processing unit 102 performs rotation processing and reduction processing (or enlargement processing) on each of the divided stone images to generate image data of the rotated stone images (S31), Pattern matching is performed (S17).

Such stone material image correction may be performed, for example, when a new stone wall position could not be specified in the first embodiment (N in S20). Even if the operation example shown in the first embodiment is repeated many times, if a new stone wall position cannot be identified, correction of the stone material image (S30) changes the comparison target of pattern matching. This is because the possibility of specifying a new stone wall position increases.

[Other embodiments]
In the first embodiment, an example has been described in which the stone material image is rotated and the stone wall image is divided. For example, rotation processing may be performed on the stone wall image. In this case, for example, the image processing unit 102 rotates the image data of one divided stone wall image stored in the memory 101 by a predetermined number of degrees (for example, 10 degrees) by the same rotation processing as for the rotated stone image. Image data of divided stone wall images after rotation for a plurality of pieces (for example, 36 pieces) may be generated, and pattern matching may be performed between the rotated stone wall image and the rotated stone wall image after division. Furthermore, the image processing unit 102 may perform reduction processing (or enlargement processing), for example, on the image data of the rotated divided stone wall image for one sheet. In this case, the image processing unit 102 may perform pattern matching between the image data of the divided stone wall images for a plurality of pieces that have undergone rotation processing and reduction processing (or enlargement processing) and the image data of the rotated stone image. .

Further, the stone position specifying device 100 described in the first and second embodiments can be applied, for example, when removing stones from a stone wall in order to repair each of the stacked stones. By specifying the positions of the stone materials in the stone wall with the present stone material position determining device 100, it is also possible to stack the stone materials after restoration on the original position of the stone wall.

Furthermore, in the first and second embodiments, processing for stone wall images has been described. For example, the stone position specifying device 100 can perform similar processing on images of Buddha statues in addition to images of stone walls. In this case, the stone location identifying device 100 replaces the stone image with an image of the collapsed (or collapsed) Buddha statue portion (hereinafter referred to as the Buddha statue portion image), and replaces the stone wall image with the image of the Buddha statue before the collapse (hereinafter referred to as the Buddha statue image). The processing shown in FIG. Specifically, for example, the image processing unit 102 performs rotation processing and reduction processing (or enlargement processing) on the Buddha statue partial image in the same manner as in the first embodiment. As in the first embodiment, division processing is performed. Then, the image processing unit 102 performs pattern matching using the image data of the partial image of the Buddha image that has undergone the rotation process and the image data of the whole image of the Buddha image that has been subjected to the division process. It is also possible to specify the Buddha statue part of the part that has been cut.

Furthermore, the stone position identifying device 100 can also process shrine images, temple images, etc., in addition to stone wall images, in the same manner as stone wall images and Buddha statue images, and identify collapsed portions.

The above is summarized as follows.

(Appendix 1)
A stone position specifying program executed by a computer of a stone position specifying device having a memory in which image data of a stone wall image before it collapses and image data of a stone image after it has collapsed from the stone wall are stored,
Image data of divided stone wall images obtained by dividing the stone wall image is generated based on the image data of the stone wall image, and the stone image is rotated and reduced or rotated and enlarged based on the image data of the stone wall image. Generate the image data of the rotating stone image,
Using the image data of the divided stone wall image and the image data of the rotating stone image, pattern matching is performed on the stone contained in the stone wall image before the collapse and the stone contained in the stone image after the collapse, A stone location identification program characterized by outputting the result of pattern matching.

(Appendix 2)
Based on the image data of the stone wall image, the image data of the stone wall image of the collapsed portion is extracted from the stone wall image before the collapse, and the extracted image data is used to divide the stone wall image into divided stone wall images. Supplementary note 1. The stone location specifying program according to appendix 1, wherein the image data of is generated.

(Appendix 3)
The image data of the divided stone wall image is based on the image data of the stone wall image of the collapsed portion. A stone position specifying program according to appendix 2, wherein the image data are images obtained by sliding a second number of pixels in the horizontal direction.

(Appendix 4)
Image data of an image of a stone portion included in the stone image is extracted based on the image data of the stone image, and image data of the rotated stone image is generated using the extracted image data. A stone location program according to Supplementary Note 1.

(Appendix 5)
Using the image data of the divided stone wall image and the image data of the rotated stone wall image, the first local feature quantity at the first feature point of the rotated stone wall image and the first local feature quantity at the second feature point of the divided stone wall image 2 local feature amounts are calculated for all feature points included in the rotated stone image and the divided stone wall image, and the first feature points having the first local feature amount and the first local feature amount are calculated. A vector connecting the second feature point having the second local feature amount that is most similar is calculated, and for the number of all feature points included in the divided stone wall image, the vector in the divided stone wall image The ratio of the number of the second feature points having the largest number is used as the similarity, and the pattern matching is performed by searching for the divided stone wall image having the highest similarity with respect to the rotated stone image. A stone location program according to Supplementary Note 1.

(Appendix 6)
6. The stone position specifying program according to Supplementary Note 5, wherein the plurality of divided stone wall images are displayed in descending order of similarity with respect to the rotated stone image.

(Appendix 7)
6. The stone position specifying program according to appendix 5, wherein the first and second local feature quantities are local feature quantities represented by BRIEF (Binary Robust Independent Elementary Features) features.

(Appendix 8)
When it is specified by the pattern matching that the stone contained in the rotated stone image is the stone contained in the divided stone wall image, the image data of the detected stone in the stone wall image is set to a value representing a specific color. and generating image data of a divided stone wall image obtained by dividing the stone wall image based on the image data of the stone image including the specific color, and using the image data and the image data of the rotating stone image, Pattern matching is performed on the stone contained in the stone wall image before the collapse and the stone contained in the stone image after the collapse, with the stone contained in the rotated stone image being the stone contained in the divided stone wall image. A stone location specifying program according to appendix 1, characterized in that it repeats until it is impossible to specify something.

(Appendix 9)
Based on the image data of the stone image, trapezoidal correction is performed so that the stone contained in the stone image faces the front, or division correction is performed by dividing the stone image into a plurality of pieces, and the stone image that has undergone trapezoidal correction or division correction is obtained. A stone position specifying program according to appendix 1, wherein image data of the rotated and reduced or rotated and enlarged stone image is generated.

(Appendix 10)
The stone position specifying program according to appendix 9, wherein trapezoidal correction is performed by subjecting the image data of the stone image to projective transformation.

(Appendix 11)
generating image data of a rotated divided stone wall image obtained by rotating the divided stone wall image based on the image data of the divided stone wall image;
The stone position specifying program according to Supplementary Note 1, wherein the pattern matching is performed using the image data of the divided stone wall image after the rotation and the image data of the rotated stone wall image.

(Appendix 12)
In a stone locating system comprising a personal computer and a stone locating device,
The personal computer transmits request information requesting identification of the stone wall position,
The stone position specifying device is
a memory storing image data of a stone wall image before it collapsed and image data of a stone material image after it collapsed from the stone wall;
generating image data of divided stone wall images obtained by dividing the stone wall image based on the image data of the stone wall image according to the request information, and rotating and reducing the stone image based on the image data of the stone wall image, or Image data of the rotated and enlarged rotated stone image is generated, and the image data of the divided stone wall image and the image data of the rotated stone image are used to determine the stone contained in the stone wall image before the collapse and the stone after the collapse. and an image processing unit that performs pattern matching on a stone contained in a stone image and outputs the result of pattern matching to the personal computer.

(Appendix 13)
a memory storing image data of a stone wall image before it collapsed and image data of a stone material image after it collapsed from the stone wall;
Image data of divided stone wall images obtained by dividing the stone wall image is generated based on the image data of the stone wall image, and the stone image is rotated and reduced or rotated and enlarged based on the image data of the stone wall image. Image data of a rotating stone image is generated, and the image data of the divided stone wall image and the image data of the rotating stone image are used to determine the stone included in the stone wall image before the collapse and the stone image after the collapse. and an image processing unit that performs pattern matching on a stone and outputs a pattern matching result.

10: Stone position specifying system 100: Stone position specifying device 101: Memory 102: Image processing unit 103: Monitor unit 104: Operation unit 105: IF 110: FPGA

Claims (7)

A stone position specifying program executed by a computer of a stone position specifying device having a memory in which image data of a stone wall image before it collapses and image data of a stone image after it has collapsed from the stone wall are stored,
Image data of divided stone wall images obtained by dividing the stone wall image is generated based on the image data of the stone wall image, and the stone image is rotated and reduced or rotated and enlarged based on the image data of the stone wall image. Generate the image data of the rotating stone image,
The image data of the divided stone wall image and the image data of the rotating stone image are used for the stone wall image of the collapsed stone contained in the stone wall image before the collapse and the stone contained in the stone image after the collapse. A stone location identification program characterized by performing pattern matching on a stone and outputting the result of pattern matching. Based on the image data of the stone wall image, the image data of the stone wall image of the collapsed portion is extracted from the stone wall image before the collapse, and the extracted image data is used to divide the stone wall image into divided stone wall images. 2. The stone position specifying program according to claim 1, wherein the image data of . Image data of an image of a stone portion included in the stone image is extracted based on the image data of the stone image, and image data of the rotated stone image is generated using the extracted image data. 2. The stone location specifying program according to claim 1. When it is specified by the pattern matching that the stone contained in the rotated stone image is the stone contained in the divided stone wall image, the image data of the detected stone in the stone wall image is set to a value representing a specific color. and generating image data of a divided stone wall image obtained by dividing the stone wall image based on the image data of the stone image including the specific color, and using the image data and the image data of the rotating stone image, Pattern matching is performed on the stone contained in the stone wall image before the collapse and the stone contained in the stone image after the collapse, with the stone contained in the rotated stone image being the stone contained in the divided stone wall image. 2. The stone position specifying program according to claim 1, wherein the program repeats until it becomes impossible to specify something. generating image data of a rotated divided stone wall image obtained by rotating the divided stone wall image based on the image data of the divided stone wall image;
2. The stone position specifying program according to claim 1, wherein the pattern matching is performed using the image data of the divided stone wall image after the rotation and the image data of the rotated stone wall image. In a stone locating system comprising a personal computer and a stone locating device,
The personal computer transmits request information requesting identification of the stone position,
The stone position specifying device is
a memory storing image data of a stone wall image before it collapsed and image data of a stone material image after it collapsed from the stone wall;
generating image data of divided stone wall images obtained by dividing the stone wall image based on the image data of the stone wall image according to the request information, and rotating and reducing the stone image based on the image data of the stone wall image, or Image data of the rotated and enlarged rotated stone image is generated, and the image data of the divided stone wall image and the image data of the rotated stone image are used to generate a stone wall image of the collapsed stone contained in the stone wall image before the collapse. and an image processing unit that performs pattern matching on the stone contained in the stone image after the collapse, and outputs the result of pattern matching to the personal computer. a memory storing image data of a stone wall image before it collapsed and image data of a stone material image after it collapsed from the stone wall;
Image data of divided stone wall images obtained by dividing the stone wall image is generated based on the image data of the stone wall image, and the stone image is rotated and reduced or rotated and enlarged based on the image data of the stone wall image. Image data of the rotated stone image is generated, and image data of the divided stone wall image and image data of the rotated stone wall image are used to generate a stone wall image of the collapsed stone contained in the stone wall image before the collapse and after the collapse. and an image processing unit that performs pattern matching on the stone contained in the stone image and outputs a pattern matching result.

Images