JP2021055266A - Stone wall arrangement assistance program, stone wall arrangement assistance method, and stone wall arrangement assistance device - Google Patents

Abstract

To provide a stone wall arrangement assistance processing program and a stone wall arrangement assistance method capable of facilitating arrangement of stone materials in a stone wall.SOLUTION: A before-collapse state including the position of each of a plurality of collapsed stone materials before collapse and a rotational angle to the attitude of each before the collapse is acquired through pattern matching between each of a plurality of divided stone wall images generated from a stone wall static image and each of a plurality of adjustment stone material images corresponding to each of a plurality of stone material static images. A stone material three-dimensional model of each of a plurality of collapse stone materials is generated from a stone material moving image obtained by photographing each of the collapsed stone materials from all directions. A stone wall restoration three-dimensional model is generated and output by arranging the generated stone material three-dimensional model of each of the collapsed stone materials on the basis of the acquired before-collapse state of each of the collapsed stone materials.SELECTED DRAWING: Figure 15

Description

The present invention relates to a stone wall placement support program, a stone wall placement support method, and a stone wall placement support device.

Due to a disaster, some parts of castles, shrines and temples may collapse. Castles, shrines and temples are valuable as cultural assets common to all humankind, and will be restored to their original state. In the repair work, the repair work may proceed according to the visual judgment of the worker, but if it depends on the visual judgment of the worker, a great deal of labor, time, and cost are required.

Information technology may be used to repair stone walls to reduce labor, time, and cost. For example, the stone wall 3D data obtained by measuring the surface shape of the stone wall before the collapse with a laser and the stone material 3D data obtained by measuring the surface of the collapsed stone material exposed on the surface of the stone wall before the collapse with a laser. There is a technique to identify the arrangement state of stone materials in the stone wall by comparing with.

JP-A-2018-104985

In the technique of specifying the arrangement state of the stone wall by comparing the stone wall three-dimensional data and the stone material three-dimensional data, each data is acquired by laser measurement. However, it is difficult to predict the collapse of the stone material in advance, and the technology cannot be used unless the surface shape of the stone wall before the collapse is measured by a laser.

An object of the present disclosure is to facilitate the work of examining the arrangement of stone materials in the stone wall.

In one embodiment, a plurality of divided stone wall images, which are images in a plurality of blocks in which adjacent blocks have overlapping portions with each other, are generated based on a stone wall still image taken from the stone wall surface side before the collapse. A plurality of adjusted stone images are generated for each of the plurality of stone still images based on the plurality of stone still images taken from the side corresponding to the stone wall surface for each of the plurality of collapsed stones that have collapsed from the stone wall. Each of the plurality of stone-adjusted stone images has a different rotation angle with respect to the stone still image by a predetermined angle, and a magnification with respect to the stone still image differs by a predetermined magnification. By pattern matching each of the generated multiple divided stone wall images and each of the plurality of adjusted stone images corresponding to each of the plurality of stone still images, the positions of the plurality of collapsed stones in the stone wall still image before each collapse and each of them. Acquire the state before collapse including the rotation angle to the posture before collapse. A three-dimensional graphic model of each of the plurality of collapsed stones is generated from the moving images of the stones taken from each of the plurality of collapsed stones from all directions. The stone material 3D graphic model of each of the generated multiple collapsed stone materials is arranged based on the state before the collapse of each of the acquired plurality of collapsed stone materials to generate a stone wall restoration 3D graphic model and output it to the output device.

According to the present disclosure, it is possible to facilitate the work of examining the arrangement of stone materials in Ishigaki.

It is a block diagram which illustrates the functional structure of the stone wall arrangement support device of this embodiment. It is a block diagram which illustrates the hardware composition of the stone wall arrangement support device of 1st Embodiment. It is a flowchart which illustrates the flow of the stone wall arrangement support processing of this embodiment. It is a flowchart which illustrates the flow of the position identification processing of this embodiment. It is a conceptual diagram for demonstrating the taking of the stone moving image of this embodiment. It is a conceptual diagram which illustrates the stone still image. It is a conceptual diagram which illustrates the stone part image. It is a conceptual diagram which illustrates the stone wall image before the collapse. It is a conceptual diagram for explaining the collapsed part stone wall image. It is a conceptual diagram which illustrates the collapsed part stone wall image. It is a conceptual diagram for explaining a divided stone wall image. It is a conceptual diagram which illustrates the divided stone wall image. It is a conceptual diagram which illustrates the rotation reduction stone image. It is a conceptual diagram which illustrates the divided stone wall image. It is a conceptual diagram which illustrates the screen which displays the result of pattern matching. It is a conceptual diagram which illustrates the field contained in the record of the placement support database. It is a conceptual diagram which illustrates the collapsed part stone wall image which deleted the stone part image which specified the position and the angle. It is a conceptual diagram which illustrates the result of pattern matching. It is a flowchart which illustrates the flow of the arrangement support processing of 1st Embodiment. It is a conceptual diagram which illustrates the restored stone wall 3D graphic model of 1st Embodiment. It is a conceptual diagram which illustrates the stone wall restored by using the plane model of the related technology. It is a conceptual diagram for explaining the inside of the stone wall. It is a conceptual diagram for explaining the inside of the stone wall. It is a block diagram which illustrates the hardware composition of the stone wall arrangement support device of 2nd Embodiment. It is a block diagram which illustrates the hardware configuration of the information terminal of 2nd Embodiment. It is a flowchart which illustrates the flow of the arrangement support processing of 2nd Embodiment. It is a flowchart which illustrates the flow of the arrangement support processing of 2nd Embodiment. It is a flowchart which illustrates the flow of the arrangement support processing of an information terminal. It is a conceptual diagram which illustrates the stone 3D model. It is a conceptual diagram which illustrates the arrangement support screen.

Hereinafter, embodiments will be described. The following examples are not intended to be limited. In addition, each embodiment can be appropriately combined as long as the processing contents do not contradict each other.

[First Embodiment]
<Ishigaki placement support device>
FIG. 1 illustrates a functional configuration of the stone wall arrangement support device 10.

The Ishigaki placement support device 10 includes a position identification unit 11 and a placement support unit 12. The position identification unit 11 uses the image of the stone wall before the collapse, which is the image of the stone wall before the stone collapses, and the image of the stone wall after the collapse, which is the image of the stone wall after the collapse, and the collapsed stone material exists in the stone wall before the collapse. Specify the position and angle of the stone. The stone material is an individual stone contained in the stone wall, but unless otherwise specified, here, it represents a relatively large stone material called a stone wall that is piled up on the surface of the stone wall.

The placement support unit 12 arranges the three-dimensional graphic model generated from the stone image using the position and angle specified by the position identification unit 11 so as to restore the stone wall before the collapse. Generate a 3D graphic model and output it to the output device.

As an example, the Ishigaki arrangement support device 10 includes a central processing unit (CPU) 51, a primary storage unit 52, a secondary storage unit 53, an external interface 54, and a communication unit 55, as shown in FIG. The CPU 51 is an example of a processor that is hardware. The CPU 51, the primary storage unit 52, the secondary storage unit 53, the external interface 54, and the communication unit 55 are connected to each other via the bus 59.

The primary storage unit 52 is, for example, a volatile memory such as a Random Access Memory (RAM). The secondary storage unit 53 is, for example, a non-volatile memory such as a Hard Disk Drive (HDD) or a Solid State Drive (SSD).

The secondary storage unit 53 includes a program storage area 53A and a data storage area 53B. The program storage area 53A stores a program such as the Ishigaki placement support program as an example. The data storage area 53B stores, for example, intermediate data generated when a program such as the Ishigaki placement support program is executed.

The CPU 51 reads the Ishigaki placement support program from the program storage area 53A and deploys it in the primary storage unit 52. The CPU 51 operates as the position specifying unit 11 and the arrangement support unit 12 in FIG. 1 by loading and executing the stone wall arrangement support program.

A program such as the Ishigaki placement support program may be stored in an external server and expanded to the primary storage unit 52 via a network. Further, a program such as the Ishigaki arrangement support program may be stored in a non-temporary recording medium such as a Digital Versatile Disc (DVD) and expanded to the primary storage unit 52 via a recording medium reading device.

An external device is connected to the external interface 54, and the external interface 54 controls transmission and reception of various information between the external device and the CPU 51. FIG. 2 shows an example in which an external storage device 71, a camera 72, a display 73, a mouse 74, and a scanner 75 are connected to the external interface 54 by wire or wirelessly.

However, at least one of the external storage device 71, the camera 72, the display 73, the mouse 74, and the scanner 75 may not be connected to the external interface 54. Further, at least one of the external storage device 71, the camera 72, the display 73, the mouse 74, and the scanner 75 may be connected to the external interface 54. Further, at least one of the external storage device 71, the camera 72, the display 73, the mouse 74, and the scanner 75 may be built in the Ishigaki placement support device 10, or the Ishigaki placement support device may be built in the Ishigaki placement support device 10 via a network. It may be arranged at a position separated from 10.

The external storage device 71 may be, for example, a storage device such as an HDD and an SSD. The external storage device 71 may store the pre-collapse stone wall image data acquired by photographing the stone wall before the collapse, the collapsed stone material image data acquired by photographing the collapsed stone material, and the like.

The camera 72 is used to capture a still image or a moving image of the stone wall or the collapsed stone material before the collapse. The display 73 shows the position and angle of the stone wall before the collapse of the stone, which is the processing result of the position identification process, and the stone wall of the stone wall restored by using the stone three-dimensional (3D) graphic model, which is the processing result of the placement support processing. Display 3D graphic model and so on. Instead of the display 73, for example, a stone wall 3D graphic model may be printed by a printer or the like.

The mouse 74 can specify a position in the image displayed on the display 73, for example, by an operation by the user. Instead of the mouse 74, a trackball, a keyboard, or the like may be used, and when the display 73 is a touch panel, a user's finger, a touch pen, or the like may be used. The scanner 75 is used to read a photograph on a paper medium as image data.

The Ishigaki placement support device 10 may be a dedicated device, a workstation, a personal computer, or a tablet. The Ishigaki arrangement support device 10 may be connected to another device (for example, a personal computer, etc.) by wire or wirelessly via a network, and may transmit / receive data to / from the other device.

<Ishigaki placement support processing>
FIG. 3 is an example of a flowchart showing the flow of the stone wall placement support process.

In step 100, the CPU 51 performs a position specifying process for specifying the position and angle of the collapsed stone material in the stone wall before the collapse. In step 200, the CPU 51 performs placement support processing for displaying the stone wall 3D graphic model of the stone wall restored using the stone material 3D graphic model on the display 73 based on the position and angle specified in step 100.

FIG. 4 is an example of a flowchart showing the flow of the position specifying process in step 100. The CPU 51 acquires the collapsed stone image data in step 101. The collapsed stone image data is acquired, for example, by extracting a still image from the collapsed stone moving image data. As illustrated by the arrows S1 and S2 in FIG. 5, the collapsed stone moving image data captures the entire moving image of the stone R1 from all directions while moving the camera 72 held by the user around the stone R1. Is obtained by.

Collapsed stones are often observed visually, and individual features are often recorded, sketched, and photographed. Such observations require a lot of time and effort. However, by taking the entire moving image of the stone material R1 from all directions and saving it, the time and labor required for observation can be reduced, and it is easy to use it for analysis of collapsed stone materials using a computer. It becomes.

The collapsed stone moving image data acquired by the camera 72 is transmitted to the CPU 51 via the external interface 54 or the communication unit 55 by a wired or wireless connection. FIG. 6A is an example of a collapsed stone image.

The surface of the stone that faces the surface of the stone wall has a different brightness (or different color) from the surface that existed inside the stone wall when exposed to sunlight and outside air, according to experts. It can be easily identified. Therefore, as a collapsed stone image used for specifying the position and angle, a still image obtained by photographing a surface that is likely to be a surface facing the surface of the stone wall may be selected by an expert.

In step 102, the CPU 51 generates a stone partial image in which the background of the collapsed stone image is deleted. FIG. 6B shows a stone partial image in which the background of the collapsed stone image of FIG. 6A is deleted. Specifically, the CPU 51 displays the collapsed stone image on the display 73.

The user uses the mouse 74 to specify the positions of the four corners of the rectangle surrounding the stone portion of the collapsed stone image displayed on the display 73. The data indicating the positions of the four corners acquired by the mouse 74 is transmitted to the CPU 51 via the external interface 54 or the communication unit 55 by a wired or wireless connection. The CPU 51 extracts rectangular image data included in the rectangle based on the received data at the four positions.

The CPU 51 acquires the outer edge of the stone portion by performing edge extraction processing or the like on the extracted rectangular image data, and extracts the image data included in the outer edge. By this extraction, the CPU 51 can generate the stone partial image illustrated in FIG. 6B in which the background is deleted from the collapsed stone image illustrated in FIG. 6A. The CPU 51 stores the stone partial image data in the data storage area 53B.

In step 103, the CPU 51 performs rotation processing and reduction processing on the stone partial image data to generate a rotation reduction stone image which is an example of the adjustment stone image. Specifically, the CPU 51 performs a known rotation process such as an Affine transformation on the stone partial image data to generate a rotated stone image to which the rotation process has been performed. At this time, for example, the CPU 51 increases the rotation angle by a predetermined angle to generate a plurality of rotating stone image data for one stone partial image data. The predetermined angle may be, for example, 10 degrees, and by changing the rotation angle by 10 degrees (for example, 10 degrees to 360 (10 × 36) degrees), 36 rotating stone image data are generated.

The CPU 51 performs a known reduction process on the rotating stone image data to generate a rotation reduction stone image that has been reduced. At this time, the CPU 51 generates a plurality of rotation-reduced image data for one rotation stone image data by changing the magnification by a predetermined magnification. The predetermined magnification may be, for example, 0.9, and by ^{changing the predetermined magnification by 0.9 (for example, 0.9 to 0.9 12} ), 12 rotation-reduced stone image data are generated. That is, by changing the rotation angle in 36 ways by 10 degrees and changing the magnification in 12 ways by 0.9, 36 × 12 = 432 rotation-reduced stone image data is generated for one stone partial image data. To.

Although an example of changing the rotation angle by 10 degrees has been described, the change of the rotation angle is not limited to 10 degrees. For example, the change in rotation angle may be 5 degrees. Further, although an example in which the magnification is changed by 0.9 times has been described, the change in the magnification is not limited to 0.9 times. For example, the change in magnification may be 0.8 times, or instead of generating the rotation reduction image, the change in magnification may be 1.1 times and the rotation enlargement image may be generated as the adjustment stone image.

Although an example of generating rotating stone image data from the stone partial image data and generating rotating reduced stone image data from the rotating stone image data has been described, the present embodiment is not limited to this. For example, the reduced stone image data may be generated from the stone partial image data, and the rotation reduced stone image data may be generated from the reduced stone image data.

In step 104, the CPU 51 acquires the stone wall image data before the collapse. FIG. 7A is an example of a stone wall image before the collapse. It is desirable that the stone wall image before the collapse is an image taken from the front with respect to the surface of the stone wall. The stone wall image data before the collapse may be stored in the external storage device 71 in advance, for example.

If the pre-collapse stone wall image data that has been saved in advance does not exist, the pre-collapse stone wall image data may be acquired by scanning a paper medium photograph of the stone wall before the collapse using a scanner 75. .. The acquired pre-collapse stone wall image data is transmitted to the CPU 51 via the external interface 54 or the communication unit 55 via a wired or wireless connection. Even when the pre-collapse stone wall image data exists in a server or cloud connected via a network, it may be transmitted to the CPU 51 via an external interface 54 or a communication unit 55 by a wired or wireless connection. Even if the collapse of the stone wall is not expected, there will be many, for example, a paper photograph taken for tourism promotion or an image taken by a tourist with a digital camera, before the collapse of the stone wall. It can be effectively used as an image.

In step 105, the CPU 51 acquires a stone wall portion image in which unnecessary portions of the stone wall image before collapse are deleted. The stone wall portion image may be an image in which the background image and the image of the stone wall portion in which the stone material has not collapsed are removed, and the image of the collapsed stone wall portion is left. As a result, the amount of image data to be processed is reduced, so that the processing load can be reduced. In addition, since unnecessary data for specifying the position and angle of the collapsed stone material is reduced, the accuracy in pattern matching can be improved.

FIG. 7B is an example of a stone wall image before the collapse. The CPU 51 displays the stone wall image before the collapse on the display 73. The user uses the mouse 74 to specify the positions of the four corners of the rectangle W1 including the portion where the stone material has collapsed in the stone wall image displayed on the display 73.

The data indicating the positions of the four corners acquired by the mouse 74 is transmitted to the CPU 51 via the external interface 54 or the communication unit 55 by a wired or wireless connection. The CPU 51 extracts the rectangular image data included in the rectangle W1 based on the received data indicating the positions of the four corners, thereby generating the collapsed portion Ishigaki image data in which unnecessary parts are deleted from the pre-collapse Ishigaki image data. To do.

FIG. 7C is an example of a collapsed part stone wall image inside the extracted rectangle W1. The CPU 51 stores the collapsed portion stone wall image data in the data storage area 53B. In step 106, the CPU 51 generates a divided stone wall image obtained by dividing the collapsed portion stone wall image.

FIG. 8A is an example of a collapsed part stone wall image. The collapsed portion stone wall image W1 illustrated in FIG. 8A has 2004 pixels in the horizontal direction and 930 pixels in the vertical direction.

The CPU 51 slides the block B1 having a width and height of a predetermined number of pixels laterally with respect to the collapsed portion stone wall image W1 by the number of pixels smaller than the number of pixels of the width of the block B1 in order from the upper left. When the block B1 reaches the right end, the block B1 is returned to the left end, and the process of sliding the block B1 in the vertical direction by a number of pixels smaller than the number of pixels at the height of the block B1 is repeated until the block B1 reaches the lower right end. The CPU 51 acquires an image of each block B1 as a divided stone wall image. The CPU 51 stores the acquired divided stone wall image data in the data storage area 53B. The width of the block B1 may be, for example, 150 pixels, the height may be, for example, 100 pixels, and the block B1 may be slid by 20 pixels each in the horizontal direction and the vertical direction.

FIG. 8B shows an example of the divided stone wall image. The number of pixels in the width and height of the block B1 is an example, and may be, for example, 200 pixels in width and 150 pixels in height. The number of pixels to be slid is also an example. For example, 10 pixels may be slid in the horizontal direction and 30 pixels may be slid in the vertical direction.

In step 107, the CPU 51 acquires the position and angle of the collapsed stone material in the pre-collapse stone wall image by performing pattern matching using the rotation-reduced stone wall image data and the divided stone wall image data. The pattern matching is performed by brute force between the rotation-reduced stone image data for which a part is exemplified in FIG. 9A and the divided stone wall image data for which a part is exemplified in FIG. 9B. Specifically, the CPU 51 determines the local feature amount in a feature region (for example, 48 × 48 pixels) including a plurality of pixels centered on predetermined feature points in each image of the rotation-reduced stone image and the divided stone wall image. calculate.

As the local feature amount, for example, a plurality of pairs of each pixel in the feature region are predetermined, and a bit value corresponding to the sign of the brightness difference between the paired pixels (for example, when the brightness difference is a positive value). It is represented by "1", "0" in the case of a negative value, etc.). When the pixel data is represented by RGB values, the CPU 51 may calculate the luminance value as a local feature amount from the RGB values of each pixel by a known conversion formula. Further, the local feature amount may be, for example, a local feature amount represented by BRIEF (Binary Robust Independent Elementary Features).

The CPU 51 searches for the feature area in the divided stone wall image that most closely resembles the local feature amount of one feature area in the rotation-reduced stone image, and connects the feature points in the two feature areas having the most approximate local feature amount. And generate a vector. The CPU 51 generates vectors as a whole between the two images (that is, between the rotation-reduced stone image and the divided stone wall image), and generates a histogram showing the number of each vector.

The CPU 51 extracts the vector having the largest number in the histogram, calculates the number of feature points corresponding to the extracted vector in the image, and sets the calculated number and the total number of feature points included in the image. The ratio of is calculated as the similarity. The CPU 51 calculates one similarity between one rotation-reduced stone image and one divided stone wall image.

The local feature amount is not limited to BRIEF, and for example, ORB (Oriented FAST and Rotated BRIEF, Oriented FAST and Rotated BRIEF), BRISK (Binary Robust Invariant Scalable Keypoints), or the like may be used.

In step 108, the CPU 51 determines whether or not the position and angle of the collapsed stone material in the pre-collapse stone wall have been specified. Specifically, the CPU 51 displays the result of pattern matching on the display 73. FIG. 10 shows an example of the result of pattern matching displayed on the display 73.

In the present embodiment, the CPU 51 can display a predetermined number of divided stone wall images in descending order of similarity with respect to one stone partial image. The predetermined number may be, for example, 10. In FIG. 10, the divided stone wall images WI1 to WI10 having the highest similarity to the top 10 are displayed for one stone partial image RI. The similarity and angle of each may be displayed together with each of the divided stone wall images. The angle is the rotation angle of the stone partial image when the rotation-reduced stone image used when the displayed similarity was obtained was generated. Further, instead of the angle, the rotation-reduced image used when the displayed similarity is obtained may be displayed.

The user uses the mouse 74 to select the stone portion in the divided stone wall images WI1 to WI10 including the stone portion corresponding to the displayed stone portion image RI. When the corresponding stone portion does not exist in the divided stone wall images WI1 to WI10 on which the corresponding stone portion is displayed, the user does not select the stone portion by, for example, selecting the skip button SB. The selection result is transmitted to the CPU 51 via the external interface 54 or the communication unit 55 by wired or wireless connection.

When the stone portion is selected by the mouse 74, the CPU 51 stores the result of the position identification process in the placement support database in step 109. For example, a stone ID that identifies a stone partial image, a stone moving image ID that identifies a stone moving image used when acquiring the stone partial image, a stone position ID that identifies the position of a stone portion in a collapsed stone wall image, and The angles are associated and stored in the placement support database. FIG. 11 illustrates the fields included in the records of the placement support database. The stone ID is given when, for example, a stone partial image is generated, the stone moving image ID is given when, for example, a stone moving image is acquired, and the stone position ID is given, for example, when the collapsed partial stone wall image is included. It is manually or automatically applied to each of the stones. The placement support database may be stored in the data storage area 53B or may be stored in the external storage device 71, for example.

In step 110, the CPU 51 deletes the image of the stone portion corresponding to the stone position ID stored in the placement support database from the collapsed portion stone wall image. The CPU 51 changes the color of the image of the stone portion corresponding to the stone position ID stored in the placement support database to, for example, "black". FIG. 12 illustrates a stone wall image after deleting the image of the stone material portion corresponding to the stone material whose position is specified in the collapsed portion stone wall image. The present embodiment is not limited to deleting the image of the stone portion by changing the color of the stone portion to "black", and for example, the color of the stone portion may be changed to "red". ..

When the image of the stone part corresponding to the stone whose position is specified in step 110 is deleted from the collapsed part stone wall image, the CPU 51 determines in step 111 whether or not there is a stone part image whose position is not specified. If the determination in step 111 is affirmed, the CPU 51 returns to step 106.

When the position of the stone material in the collapsed part stone wall image cannot be specified for the stone material of all the stone material part images, the CPU 51 uses the collapsed part stone wall image in which the image of the stone material portion corresponding to the stone material whose position has been specified is deleted. Perform pattern matching again. FIG. 13 illustrates a processing result when pattern matching is repeated 5 times. FIG. 13 shows the ratio (“specific / total number”) between the number of stones whose positions are specified in the collapsed part stone wall image and the total number of collapsed stones included in the collapsed part stone wall image. The specific rate is a numerical value expressing the ratio in percent (%) units.

As shown in FIG. 13, in the first matching, the specific rate is 61.8%, but in the fifth matching, the specific rate increases as the number of matchings increases, and the specific rate is 82.1%. Yes, the 5th specific rate has increased by 20% or more compared to the 1st specific rate. In this way, the CPU 51 can improve the specific rate by repeating the pattern matching a plurality of times. This is because by removing the image of the stone part whose position has already been specified from the collapsed part stone wall image, it becomes gradually easier to identify the position of the stone material in the collapsed part stone wall image.

When the determination in step 111 is denied, that is, when the positions of all the stones are specified, and in step 108, the user selects the skip button SB with the mouse 74 for all the stone partial images. If it is determined that the process has been performed, the position identification process is terminated.

FIG. 14 shows an example of the flow of the arrangement support process 200 of FIG. In step 201, the CPU 51 generates point cloud data from the stone moving image data. Specifically, a still image is extracted from the stone moving image data at a predetermined time interval, for example, every second, and point cloud data for generating a stone 3D mesh model is generated using the extracted continuous still image. ..

In step 202, the CPU 51 removes the point cloud data of an unnecessary portion other than the stone portion, for example, the background, from the point cloud data, and in step 203, the stone 3D mesh model is generated. The stone 3D mesh model is an example of a stone 3D model, and for example, the Structure from Motion (SfM) technique can be applied to generate the stone 3D mesh model.

In step 204, the CPU 51 associates the placement support database illustrated in FIG. 11 with the stone moving image ID of the stone moving image data used for generating the point cloud data in step 201, and identifies the stone 3D mesh model. To store. The model ID is given, for example, when a stone 3D mesh model is generated.

The CPU 51 generates a stone 3D graphic model using the generated stone 3D mesh model, and displays the generated stone 3D graphic model on the display 73, for example, as illustrated in FIG. 15 in step 205. .. The stone 3D graphic model is positioned based on the stone position ID corresponding to the model ID of the placement support database, and is rotated based on the corresponding angle to form a stone wall 3D graphic model in which the stone wall before the collapse is restored. To do.

That is, the stone 3D graphic model is rotated so that the surface corresponding to the surface of the stone wall is the front surface and the front image is rotated by the corresponding angle. Instead of the stone 3D graphic model, the stone 3D mesh model may be displayed on the display 73. Further, instead of displaying the image on the display 73, the image may be converted into a two-dimensional image and printed by a printer. The stone 3D graphic model may be a mesh model, an opaque solid model, a transparent solid model, or a wireframe model.

In the related technology, as illustrated in FIG. 16, a flat model DR having a surface shape corresponding to the stone wall surface of the stone is created with a plastic board or the like, and the model DR is attached with a tape TP or the like to the stone material. Consider the placement of. However, as illustrated in FIGS. 17 and 18, when the stone RRs are arranged on the stone wall, the stone RRs are in contact with each other slightly behind the surface of the stone wall. Furthermore, in order to fix the stone RR, the stone KR is placed in the gap between the stones on the back side of the stone, and in order to reduce the unevenness on the surface of the stone wall, the stone AR is placed in the gap between the stones in front of the stone. To.

In the planar model DR as illustrated in FIG. 16, it is not possible to grasp whether the stone RRs are properly joined to each other and the size and amount of the stone KR and the stone AR. On the other hand, when a stone wall 3D graphic model is used to generate a stone wall 3D graphic model in which the stone wall before the collapse is restored, whether the stone RRs are properly joined, the size and amount of the stone KR and the stone wall AR. Can be easily grasped. As a result, it is possible to appropriately formulate a plan for restoring the stone wall using real stone materials, and it is possible to reduce the amount of unnecessary work such as reloading.

As described above, in the first embodiment, a plurality of divisions are images in a plurality of blocks in which adjacent blocks have overlapping portions with each other, based on a still image of the stone wall taken from the surface side of the stone wall before the collapse. Generate a stone wall image. A plurality of adjusted stone images are generated for each of the plurality of stone still images based on the plurality of stone still images taken from the side corresponding to the stone wall surface for each of the plurality of collapsed stones that have collapsed from the stone wall. Each of the plurality of stone-adjusted stone images has a different rotation angle with respect to the stone still image by a predetermined angle, and a magnification with respect to the stone still image differs by a predetermined magnification. By pattern matching each of the generated multiple divided stone wall images and each of the plurality of adjusted stone images corresponding to each of the plurality of stone still images, the positions of the plurality of collapsed stones in the stone wall still image before each collapse and each of them. Acquire the state before collapse including the rotation angle to the posture before collapse. A three-dimensional graphic model of each of the plurality of collapsed stones is generated from the moving images of the stones taken from each of the plurality of collapsed stones from all directions. The stone material 3D graphic model of each of the generated multiple collapsed stone materials is arranged based on the state before the collapse of each of the acquired plurality of collapsed stone materials to generate a stone wall restoration 3D graphic model and output it to the output device.

In the present embodiment, the above makes it easy to grasp whether the joining of stones is appropriate, which is difficult in the examination using a model on a plane, and the size and amount of stones and stones. , It is possible to facilitate the work of examining the arrangement of stone materials in the stone wall.

<Second Embodiment>
Next, the second embodiment will be described. The same operations and configurations as those in the first embodiment are designated by the same reference numerals, and the description thereof will be omitted.

In the second embodiment, the stone wall arrangement support device uses the stone material 3D model by creating a stone material 3D model with a 3D printer and outputting the position and angle in the collapsed part stone wall image of the stone material 3D model to the output device. Support the examination of stone layout.

<Ishigaki placement support device>
FIG. 19 illustrates the hardware configuration of the stone wall arrangement support device 20 of the second embodiment. The stone wall placement support device 20 differs from the stone wall placement support device 10 of the first embodiment in that it includes a 3D printer 76 and a marker printer 77. The 3D printer 76 prints the stone 3D model in 3D, and the marker printer 76 prints a marker sticker containing identification information that identifies each of the stone 3D models. The marker sticker is attached to the stone 3D model.

<Information terminal>
FIG. 20 illustrates the hardware configuration of the information terminal 30 of the second embodiment. As an example, the information terminal 30 includes a CPU 61, a primary storage unit 62, a secondary storage unit 63, an external interface 64, and a communication unit 65, as shown in FIG. The CPU 61 is an example of a processor that is hardware. The CPU 61, the primary storage unit 62, the secondary storage unit 63, the external interface 64, and the communication unit 65 are connected to each other via the bus 69.

The primary storage unit 62 is, for example, a volatile memory such as a RAM. The secondary storage unit 63 is, for example, a non-volatile memory such as an HDD or an SSD.

The secondary storage unit 63 includes a program storage area 63A and a data storage area 63B. The program storage area 63A stores a program such as an arrangement support program as an example. As an example, the data storage area 63B stores intermediate data generated when a program such as an arrangement support program is executed.

The CPU 61 reads the placement support program from the program storage area 63A and deploys it in the primary storage unit 62. The CPU 61 loads and executes the placement support program.

A program such as an arrangement support program may be stored in an external server and expanded in the primary storage unit 62 via a network. Further, a program such as an arrangement support program may be stored in a non-temporary recording medium such as a DVD and expanded in the primary storage unit 62 via a recording medium reading device.

An external device is connected to the external interface 64, and the external interface 64 controls the transmission and reception of various information between the external device and the CPU 61. FIG. 20 shows an example in which the camera 82 and the display 83 are connected to the external interface 64 by wire or wirelessly.

However, the camera 82 and the display 83 may not be connected to the external interface 64, or only one of the camera 82 and the display 83 may be connected to the external interface 64. Further, either or both of the camera 82 and the display 83 may be built in the information terminal 30, or may be arranged at a position separated from the information terminal 30 via a network.

The information terminal 30 may be a portable smart device, a digital camera having a communication function, a personal computer, or the like.

<Placement support processing>
FIG. 21A illustrates the flow of the placement support process # 1 in the stone wall placement support device 20 of the second embodiment. Since steps 201 to 204 are the same as steps 201 to 204 in FIG. 14, the description thereof will be omitted. In step 211, the CPU 51 3D prints the stone 3D model with the 3D printer 76 using the 3D mesh model generated in step 203. FIG. 22 shows an example of a 3D printed stone 3D model. The stone 3D model may be full-scale or may be reduced in size at a constant magnification.

In step 212, the CPU 51 generates a marker representing the stone ID corresponding to the model ID of the 3D mesh model used when generating the stone 3D model stored in the placement support database illustrated in FIG. The CPU 51 prints a marker sticker including the marker with the marker printer 77, and ends the placement support process # 1. The marker may be, for example, a QR code (registered trademark), a barcode, or a character string. The marker sticker is attached to the stone 3D model by the user, for example.

FIG. 21B illustrates the flow of the placement support process # 2 in the Ishigaki placement support device 20. In step 213, the CPU 51 determines whether or not the image data of the marker transmitted by the information terminal 30 has been received via the communication unit 55. The CPU 51 repeats step 213 until the determination in step 213 is affirmed.

If the determination in step 213 is affirmed, the CPU 51 displays the placement support screen on the display 73 in step 214, as illustrated in FIG. The placement support screen includes, for example, a stone ID D1, an image D2 in which the surface of the stone corresponding to the stone wall surface is rotated by an angle, and an image D3 in which the stone position in the stone wall is indicated by a star mark and an arrow. The stone ID is acquired from the image data of the marker using the existing technique, and the angle and the stone position are acquired from the placement support database based on the stone ID.

The user determines the position and angle of the stone 3D model with reference to the displayed information, and by stacking the stone 3D models, the restored stone wall similar to the stone wall 3D graphic model displayed on the screen of the display 73 illustrated in FIG. 15 is restored. You can create a 3D model.

In step 215, the CPU 51 determines whether or not a termination instruction has been given, for example, by selecting a predetermined button by the user. If the determination in step 215 is denied, the CPU 51 returns to step 213. If the determination in step 215 is affirmed, the CPU 51 ends the placement support process # 2.

FIG. 21C illustrates the flow of placement support processing in the information terminal 30 of the second embodiment. The user photographs the marker sticker attached to the stone 3D model with the camera 82. The captured image data of the marker is transmitted to the CPU 61 via the external interface 64 or the communication unit 65 by a wired or wireless connection.

In step 221 the CPU 61 determines whether or not the image data of the marker has been received. The CPU 61 repeats step 221 until the determination in step 221 is affirmed. When the determination in step 221 is affirmed, the CPU 61 transmits the image data of the marker to the Ishigaki placement support device 20 via the communication unit 65, and ends the placement support process.

Instead of the information terminal 30, the camera 72 may take a picture of the marker sticker. The image data of the marker taken by the camera 72 is transmitted to the CPU 51 by wire or wireless connection via the external interface 54 or the communication unit 55. Further, the arrangement support screen illustrated in FIG. 23 is an example, and for example, the position and the angle may be represented numerically.

Instead of printing the marker sticker, the stone ID may be displayed on the display so that the user can refer to the displayed stone ID and handwrite the stone ID on the stone 3D model. Also in this case, by photographing the handwritten stone ID, the stone ID can be acquired by using the existing handwritten character recognition technique.

The flowcharts of FIGS. 3, 4, 14, 21A, 21B, and 21C are examples, and the order of processing may be changed as appropriate.

In the present embodiment, the CPU is exemplified as the processor, but the processor is not limited to the CPU. For example, the processor may be a Programmable Logic Device (PLD) whose circuit configuration can be changed after manufacturing such as a Field-Programmable Gate Array (FPGA), a Graphics Processing Unit (GPU), or the like. One of these various processors may be used alone, or two or more processors of the same type or different types may be used in combination.

In the present embodiment, a plurality of divided stone wall images, which are images in a plurality of blocks in which adjacent blocks have overlapping portions with each other, are generated based on a stone wall still image taken from the stone wall surface side before the collapse. For each of the plurality of collapsed stone materials that have collapsed from the stone wall, a plurality of adjusted stone material images are generated based on the plurality of stone material still images taken from the side corresponding to the stone wall surface. The plurality of adjusted stone images differ in the rotation angle with respect to the stone still image by a predetermined angle and the magnification with respect to the stone still image differ by a predetermined magnification for each of the plurality of stone still images. By pattern matching each of the generated multiple divided stone wall images and each of the plurality of adjusted stone images corresponding to each of the plurality of stone still images, the positions of the plurality of collapsed stones in the stone wall still image before each collapse and each of them. Acquire the state before collapse including the rotation angle to the posture before collapse. A three-dimensional model of each of the multiple collapsed stones is generated from the moving images of the stones taken from all directions of each of the plurality of collapsed stones, and a plurality of three-dimensional stones are generated based on each of the three-dimensional models of the multiple collapsed stones. Make a model. Based on the pre-collapse state, information regarding the position of each of the plurality of three-dimensional models before the collapse and the rotation angle to the posture before the collapse is output to the output device.

In the present embodiment, the above makes it easy to grasp whether the joining of stones, which is difficult to grasp by the examination using a flat model, is appropriate, and the size and amount of the stones and stones. , It is possible to facilitate the work of examining the arrangement of stone materials in the stone wall.

Further, in the present embodiment, since it is shown how to stack the stone three-dimensional models selected by the user, the work can be promoted, and the stacking error due to the user's mistake of the stone three-dimensional model or the like can be achieved. Can also be reduced. Since the stone 3D model is used, even if the user is not familiar with understanding the display of 3D graphics, what is the appropriate size and amount of stones and stones to join the stones together? It is easy to grasp the degree.

The following additional notes will be further disclosed with respect to each of the above embodiments.

(Appendix 1)
Based on the stone wall still image taken from the stone wall surface side before the collapse, a plurality of divided stone wall images, which are images in a plurality of blocks in which adjacent blocks have overlapping parts with each other, are generated.
Each of the plurality of collapsed stones collapsed from the stone wall is rotated with respect to the stone still image for each of the plurality of stone still images based on the plurality of stone still images taken from the side corresponding to the stone wall surface. A plurality of adjusted stone images in which the angles differ by a predetermined angle and the magnification with respect to the stone still image differs by a predetermined magnification are generated.
By pattern matching between each of the generated plurality of divided stone wall images and each of the plurality of adjusted stone image images corresponding to each of the plurality of stone still images, before the collapse of each of the plurality of collapsed stone materials in the stone wall still image. Acquire the pre-collapse state including the position of and the rotation angle to each pre-collapse posture.
A three-dimensional graphic model of each of the plurality of collapsed stones is generated from the moving images of the stones taken from each of the plurality of collapsed stones from all directions, and the three-dimensional graphic model of the stones of each of the plurality of collapsed stones generated. Is arranged based on the state before the collapse of each of the acquired plurality of collapsed stone materials to generate a stone wall restoration 3D graphic model and output it to the output device.
Ishigaki A program that causes a computer to perform placement support processing.
(Appendix 2)
Based on the stone wall still image taken from the stone wall surface side before the collapse, a plurality of divided stone wall images, which are images in a plurality of blocks in which adjacent blocks have overlapping parts with each other, are generated.
Each of the plurality of collapsed stones collapsed from the stone wall is rotated with respect to the stone still image for each of the plurality of stone still images based on the plurality of stone still images taken from the side corresponding to the stone wall surface. A plurality of adjusted stone images in which the angles differ by a predetermined angle and the magnification with respect to the stone still image differs by a predetermined magnification are generated.
By pattern matching between each of the generated plurality of divided stone wall images and each of the plurality of adjusted stone image images corresponding to each of the plurality of stone still images, before the collapse of each of the plurality of collapsed stone materials in the stone wall still image. Acquire the pre-collapse state including the position of and the rotation angle to each pre-collapse posture.
A three-dimensional model of each of the plurality of collapsed stones is generated from the moving images of the stones taken from each of the plurality of collapsed stones from all directions, and a plurality of three-dimensional models of the plurality of collapsed stones are generated based on each of the three-dimensional models of the collapsed stones. Created a three-dimensional model of the stone material of
Based on the pre-collapse state, information regarding the position of each of the plurality of three-dimensional models before the collapse and the rotation angle to the posture before the collapse is output to the output device.
Ishigaki A program that causes a computer to perform placement support processing.
(Appendix 3)
The information regarding the rotation angle is a still image obtained by rotating the stone still image by the rotation angle.
Appendix 2 program.
(Appendix 4)
Information on the position of each of the plurality of collapsed stones before the collapse and the rotation angle to the posture before the collapse is stored in association with the identification information for identifying each of the plurality of collapsed stones.
A plurality of markers representing the identification information corresponding to each of the plurality of collapsed stone materials were created.
Based on the identification information acquired by photographing the markers attached to each of the plurality of stone three-dimensional models, the position before the collapse and the rotation angle to the posture before the collapse corresponding to the identification information. To get,
Appendix 2 or Appendix 3 program.
(Appendix 5)
The computer
Based on the stone wall still image taken from the stone wall surface side before the collapse, a plurality of divided stone wall images, which are images in a plurality of blocks in which adjacent blocks have overlapping parts with each other, are generated.
Each of the plurality of collapsed stones collapsed from the stone wall is rotated with respect to the stone still image for each of the plurality of stone still images based on the plurality of stone still images taken from the side corresponding to the stone wall surface. A plurality of adjusted stone images in which the angles differ by a predetermined angle and the magnification with respect to the stone still image differs by a predetermined magnification are generated.
By pattern matching between each of the generated plurality of divided stone wall images and each of the plurality of adjusted stone image images corresponding to each of the plurality of stone still images, before the collapse of each of the plurality of collapsed stone materials in the stone wall still image. Acquire the pre-collapse state including the position of and the rotation angle to each pre-collapse posture.
A three-dimensional graphic model of each of the plurality of collapsed stones is generated from the moving images of the stones taken from each of the plurality of collapsed stones from all directions, and the three-dimensional graphic model of the stones of each of the plurality of collapsed stones generated. Is arranged based on the state before the collapse of each of the acquired plurality of collapsed stone materials to generate a stone wall restoration 3D graphic model and output it to the output device.
Ishigaki placement support method.
(Appendix 6)
The computer
Based on the stone wall still image taken from the stone wall surface side before the collapse, a plurality of divided stone wall images, which are images in a plurality of blocks in which adjacent blocks have overlapping parts with each other, are generated.
Each of the plurality of collapsed stones collapsed from the stone wall is rotated with respect to the stone still image for each of the plurality of stone still images based on the plurality of stone still images taken from the side corresponding to the stone wall surface. A plurality of adjusted stone images in which the angles differ by a predetermined angle and the magnification with respect to the stone still image differs by a predetermined magnification are generated.
By pattern matching between each of the generated plurality of divided stone wall images and each of the plurality of adjusted stone image images corresponding to each of the plurality of stone still images, before the collapse of each of the plurality of collapsed stone materials in the stone wall still image. Acquire the pre-collapse state including the position of and the rotation angle to each pre-collapse posture.
A three-dimensional model of each of the plurality of collapsed stones is generated from the moving images of the stones taken from each of the plurality of collapsed stones from all directions, and a plurality of three-dimensional models of the plurality of collapsed stones are generated based on each of the three-dimensional models of the collapsed stones. Created a three-dimensional model of the stone material of
Based on the pre-collapse state, information regarding the position of each of the plurality of three-dimensional models before the collapse and the rotation angle to the posture before the collapse is output to the output device.
Ishigaki placement support method.
(Appendix 7)
The information regarding the rotation angle is a still image obtained by rotating the stone still image by the rotation angle.
Appendix 6 Ishigaki placement support method.
(Appendix 8)
Information on the position of each of the plurality of collapsed stones before the collapse and the rotation angle to the posture before the collapse is stored in association with the identification information for identifying each of the plurality of collapsed stones.
A plurality of markers representing the identification information corresponding to each of the plurality of collapsed stone materials were created.
Based on the identification information acquired by photographing the markers attached to each of the plurality of stone three-dimensional models, the position before the collapse and the rotation angle to the posture before the collapse corresponding to the identification information. To get,
The stone wall placement support method of Appendix 6 or Appendix 7.
(Appendix 9)
Based on the stone wall still image of the stone wall before the collapse from the surface side of the stone wall, a plurality of divided stone wall images which are images in a plurality of blocks in which adjacent blocks have overlapping parts are generated, and a plurality of images collapsed from the stone wall. Based on a plurality of stone still images taken from the side corresponding to the stone wall surface, the rotation angle of each of the plurality of stone still images differs by a predetermined angle with respect to the stone still image. In addition, a plurality of adjusted stone images having different magnifications with respect to the stone still image are generated, and the plurality of adjusted stone images corresponding to each of the generated divided stone wall images and each of the plurality of stone still images are generated. By pattern matching with each of the above, a position specifying part for acquiring the position before the collapse of each of the plurality of collapsed stone materials in the stone wall still image and the rotation angle to the posture before the collapse, and the position specifying part including the rotation angle to the posture before the collapse.
A three-dimensional graphic model of each of the plurality of collapsed stones is generated from the moving images of the stones taken from each of the plurality of collapsed stones from all directions, and the three-dimensional graphic model of the stones of each of the plurality of collapsed stones generated. Is arranged based on the state before the collapse of each of the acquired plurality of collapsed stone materials, and the stone wall restoration 3D graphic model is generated and output to the output device.
Ishigaki placement support device, including.
(Appendix 10)
Based on the stone wall still image of the stone wall before the collapse from the surface side of the stone wall, a plurality of divided stone wall images which are images in a plurality of blocks in which adjacent blocks have overlapping parts are generated, and a plurality of images collapsed from the stone wall. Based on a plurality of stone still images taken from the side corresponding to the stone wall surface, the rotation angle of each of the plurality of stone still images differs by a predetermined angle with respect to the stone still image. In addition, a plurality of adjusted stone images having different magnifications with respect to the stone still image are generated, and the plurality of adjusted stone images corresponding to each of the generated divided stone wall images and each of the plurality of stone still images are generated. By pattern matching with each of the above, a position specifying part for acquiring the position before the collapse of each of the plurality of collapsed stone materials in the stone wall still image and the rotation angle to the posture before the collapse, and the position specifying part including the rotation angle to the posture before the collapse.
A three-dimensional stone model of each of the plurality of collapsed stones is generated from the moving images of the stones taken from each of the plurality of collapsed stones from all directions, and a plurality of three-dimensional models of the plurality of collapsed stones are generated based on each of the three-dimensional models of the collapsed stones. A three-dimensional model of the stone material is created, and based on the pre-collapse state, information regarding the position of each of the plurality of three-dimensional models before the collapse and the rotation angle to the posture before the collapse is output to the output device. With the support department
Ishigaki placement support device, including.
(Appendix 11)
The information regarding the rotation angle is a still image obtained by rotating the stone still image by the rotation angle.
Appendix 10 stone wall placement support device.
(Appendix 12)
The position specifying part is
Information on the position of each of the plurality of collapsed stones before the collapse and the rotation angle to the posture before the collapse is stored in association with the identification information for identifying each of the plurality of collapsed stones.
The placement support department
A plurality of markers representing the identification information corresponding to each of the plurality of collapsed stone materials were created.
Based on the identification information acquired by photographing the markers attached to each of the plurality of stone three-dimensional models, the position before the collapse and the rotation angle to the posture before the collapse corresponding to the identification information. To get,
The stone wall placement support device of Appendix 10 or Appendix 11.

10 Ishigaki placement support device 11 Position identification unit 12 Placement support unit 51 CPU
52 Primary storage 53 Secondary storage

Claims (8)

Based on the stone wall still image taken from the stone wall surface side before the collapse, a plurality of divided stone wall images, which are images in a plurality of blocks in which adjacent blocks have overlapping parts with each other, are generated.
Each of the plurality of collapsed stones collapsed from the stone wall is rotated with respect to the stone still image for each of the plurality of stone still images based on the plurality of stone still images taken from the side corresponding to the stone wall surface. A plurality of adjusted stone images in which the angles differ by a predetermined angle and the magnification with respect to the stone still image differs by a predetermined magnification are generated.
By pattern matching between each of the generated plurality of divided stone wall images and each of the plurality of adjusted stone image images corresponding to each of the plurality of stone still images, before the collapse of each of the plurality of collapsed stone materials in the stone wall still image. Acquire the pre-collapse state including the position of and the rotation angle to each pre-collapse posture.
A three-dimensional graphic model of each of the plurality of collapsed stones is generated from the moving images of the stones taken from each of the plurality of collapsed stones from all directions, and the three-dimensional graphic model of the stones of each of the plurality of collapsed stones generated. Is arranged based on the state before the collapse of each of the acquired plurality of collapsed stone materials to generate a stone wall restoration 3D graphic model and output it to the output device.
Ishigaki A program that causes a computer to perform placement support processing. Based on the stone wall still image taken from the stone wall surface side before the collapse, a plurality of divided stone wall images, which are images in a plurality of blocks in which adjacent blocks have overlapping parts with each other, are generated.
Each of the plurality of collapsed stones collapsed from the stone wall is rotated with respect to the stone still image for each of the plurality of stone still images based on the plurality of stone still images taken from the side corresponding to the stone wall surface. A plurality of adjusted stone images in which the angles differ by a predetermined angle and the magnification with respect to the stone still image differs by a predetermined magnification are generated.
By pattern matching between each of the generated plurality of divided stone wall images and each of the plurality of adjusted stone image images corresponding to each of the plurality of stone still images, before the collapse of each of the plurality of collapsed stone materials in the stone wall still image. Acquire the pre-collapse state including the position of and the rotation angle to each pre-collapse posture.
A three-dimensional model of each of the plurality of collapsed stones is generated from the moving images of the stones taken from each of the plurality of collapsed stones from all directions, and a plurality of three-dimensional models of the plurality of collapsed stones are generated based on each of the three-dimensional models of the collapsed stones. Created a three-dimensional model of the stone material of
Based on the pre-collapse state, information regarding the position of each of the plurality of three-dimensional models before the collapse and the rotation angle to the posture before the collapse is output to the output device.
Ishigaki A program that causes a computer to perform placement support processing. The information regarding the rotation angle is a still image obtained by rotating the stone still image by the rotation angle.
The program according to claim 2. Information on the position of each of the plurality of collapsed stones before the collapse and the rotation angle to the posture before the collapse is stored in association with the identification information for identifying each of the plurality of collapsed stones.
A plurality of markers representing the identification information corresponding to each of the plurality of collapsed stone materials were created.
Based on the identification information acquired by photographing the markers attached to each of the plurality of stone three-dimensional models, the position before the collapse and the rotation angle to the posture before the collapse corresponding to the identification information. To get,
The program according to claim 2 or 3. The computer
Based on the stone wall still image taken from the stone wall surface side before the collapse, a plurality of divided stone wall images, which are images in a plurality of blocks in which adjacent blocks have overlapping parts with each other, are generated.
Each of the plurality of collapsed stones collapsed from the stone wall is rotated with respect to the stone still image for each of the plurality of stone still images based on the plurality of stone still images taken from the side corresponding to the stone wall surface. A plurality of adjusted stone images in which the angles differ by a predetermined angle and the magnification with respect to the stone still image differs by a predetermined magnification are generated.
By pattern matching between each of the generated plurality of divided stone wall images and each of the plurality of adjusted stone image images corresponding to each of the plurality of stone still images, before the collapse of each of the plurality of collapsed stone materials in the stone wall still image. Acquire the pre-collapse state including the position of and the rotation angle to each pre-collapse posture.
A three-dimensional graphic model of each of the plurality of collapsed stones is generated from the moving images of the stones taken from each of the plurality of collapsed stones from all directions, and the three-dimensional graphic model of the stones of each of the plurality of collapsed stones generated. Is arranged based on the state before the collapse of each of the acquired plurality of collapsed stone materials to generate a stone wall restoration 3D graphic model and output it to the output device.
Ishigaki placement support method. The computer
Based on the stone wall still image taken from the stone wall surface side before the collapse, a plurality of divided stone wall images, which are images in a plurality of blocks in which adjacent blocks have overlapping parts with each other, are generated.
Each of the plurality of collapsed stones collapsed from the stone wall is rotated with respect to the stone still image for each of the plurality of stone still images based on the plurality of stone still images taken from the side corresponding to the stone wall surface. A plurality of adjusted stone images in which the angles differ by a predetermined angle and the magnification with respect to the stone still image differs by a predetermined magnification are generated.
By pattern matching between each of the generated plurality of divided stone wall images and each of the plurality of adjusted stone image images corresponding to each of the plurality of stone still images, before the collapse of each of the plurality of collapsed stone materials in the stone wall still image. Acquire the pre-collapse state including the position of and the rotation angle to each pre-collapse posture.
A three-dimensional model of each of the plurality of collapsed stones is generated from the moving images of the stones taken from each of the plurality of collapsed stones from all directions, and a plurality of three-dimensional models of the plurality of collapsed stones are generated based on each of the three-dimensional models of the collapsed stones. Created a three-dimensional model of the stone material of
Based on the pre-collapse state, information regarding the position of each of the plurality of three-dimensional models before the collapse and the rotation angle to the posture before the collapse is output to the output device.
Ishigaki placement support method. Based on the stone wall still image of the stone wall before the collapse from the surface side of the stone wall, a plurality of divided stone wall images which are images in a plurality of blocks in which adjacent blocks have overlapping parts are generated, and a plurality of images collapsed from the stone wall. Based on a plurality of stone still images taken from the side corresponding to the stone wall surface, the rotation angle of each of the plurality of stone still images differs by a predetermined angle with respect to the stone still image. In addition, a plurality of adjusted stone images having different magnifications with respect to the stone still image are generated, and the plurality of adjusted stone images corresponding to each of the generated divided stone wall images and each of the plurality of stone still images are generated. By pattern matching with each of the above, a position specifying part for acquiring the position before the collapse of each of the plurality of collapsed stone materials in the stone wall still image and the rotation angle to the posture before the collapse, and the position specifying part including the rotation angle to the posture before the collapse.
A three-dimensional graphic model of each of the plurality of collapsed stones is generated from the moving images of the stones taken from each of the plurality of collapsed stones from all directions, and the three-dimensional graphic model of the stones of each of the plurality of collapsed stones generated. Is arranged based on the state before the collapse of each of the acquired plurality of collapsed stone materials, and the stone wall restoration 3D graphic model is generated and output to the output device.
Ishigaki placement support device, including. Based on the stone wall still image of the stone wall before the collapse from the surface side of the stone wall, a plurality of divided stone wall images which are images in a plurality of blocks in which adjacent blocks have overlapping parts are generated, and a plurality of images collapsed from the stone wall. Based on a plurality of stone still images taken from the side corresponding to the stone wall surface, the rotation angle of each of the plurality of stone still images differs by a predetermined angle with respect to the stone still image. In addition, a plurality of adjusted stone images having different magnifications with respect to the stone still image are generated, and the plurality of adjusted stone images corresponding to each of the generated divided stone wall images and each of the plurality of stone still images are generated. By pattern matching with each of the above, a position specifying part for acquiring the position before the collapse of each of the plurality of collapsed stone materials in the stone wall still image and the rotation angle to the posture before the collapse, and the position specifying part including the rotation angle to the posture before the collapse.
A three-dimensional stone model of each of the plurality of collapsed stones is generated from the moving images of the stones taken from each of the plurality of collapsed stones from all directions, and a plurality of three-dimensional models of the plurality of collapsed stones are generated based on each of the three-dimensional models of the collapsed stones. A three-dimensional model of the stone material is created, and based on the pre-collapse state, information regarding the position of each of the plurality of three-dimensional models before the collapse and the rotation angle to the posture before the collapse is output to the output device. With the support department
Ishigaki placement support device, including.

Images