JPWO2020039470A1 - Image processing system - Google Patents

Abstract

An object of the present invention is to provide an image processing system. An image processing system for a plurality of image information used for modeling an object. The image processing system includes an image information input reception processing unit that accepts input of a plurality of image information and an image information that receives the input. It has an overlap processing unit that specifies image information that satisfies the conditions related to the degree of overlap of the reference image information with the shooting area, and an image quality determination processing unit that determines the image quality of the image information. This is an image processing system that uses the results determined by the image quality determination processing unit to identify the image information that satisfies the conditions related to image quality from the image information specified by the wrap processing unit, and selects that image information as the output target. ..

Description

The present invention relates to an image processing system.

In recent years, structural inspection work has become important from the viewpoint of maintenance and management of aging of various structures such as bridges, dams, and tunnels, and disaster countermeasures. In the inspection work of structures, efforts are being made to utilize inspection robots, UAVs (Unmanned Aerial Vehicles), and information technology.

Then, as one method of inspection work, there is a method of confirming the state of damage or deterioration of the structure on a computer screen by reproducing the state of the structure to be inspected with a three-dimensional model.

When reproducing with a three-dimensional model, the image information obtained by photographing the structure from multiple viewpoints is used, the feature points between the image information are associated with each other, and the three-dimensional coordinates of the feature points are three-dimensionally measured by the principle of triangulation. Use a technique to reproduce the shape. This technique is called SfM (Structure from Motion).

When SfM is used, images of the same part of the same structure must be taken from different viewpoints and from at least two viewpoints. Therefore, in the image information used for SfM, while changing the shooting position, the image information is photographed so as to overlap (overlap) with other image information.

Patent Document 1 and Patent Document 2 disclose a technique for associating feature points between image information. Further, Patent Document 3 discloses a technique for improving the accuracy of generating an object shape from a captured image. Further, Patent Document 4 discloses an image evaluation method for selecting a high-quality image.

Japanese Unexamined Patent Publication No. 2017-021427 JP-A-2017-041141 Japanese Unexamined Patent Publication No. 2014-109819 Japanese Unexamined Patent Publication No. 2008-23459

When an image of a structure is taken using SfM to generate a three-dimensional model of the structure, a huge amount of processing time is required because it is necessary to reconstruct the structure from the two-dimensional image information in three dimensions. Especially in the case of inspection work, high-resolution image information is used, which increases the processing time. For example, even if the overlap rate between image information is about 60%, if the image information is about 5000 sheets, even if a dedicated computer equipped with a GPU (Graphics Processing Unit) is used, it will take several days or 1 It may take about a week to process. Therefore, reduction of processing time is required.

However, even if the techniques of Patent Documents 1 to 3 are applied to SfM, some speedup may be expected, but the processing time cannot be significantly reduced. This is because the processing time in SfM is proportional to the number of combinations of the number of image information, and it is difficult to solve the problem by speeding up the processing in SfM itself.

Further, the "best photograph" of the image selected by the method of Patent Document 4 means an emotional "good photograph". Therefore, even if such a method is used, it is not possible to obtain the best photograph in the sense of highly accurate measurement data.

Therefore, in view of the above problems, the present inventor has invented an image processing system capable of shortening the processing time required for reconstructing a three-dimensional model.

The first invention is an image processing system for a plurality of image information used for modeling an object, wherein the image processing system includes an image information input reception processing unit that accepts input of the plurality of image information. Of the image information received, the overlap processing unit that specifies the image information that satisfies the condition regarding the degree of overlap of the reference image information with the shooting area, and the image quality determination process that determines the image quality of the image information. Of the image information specified by the overlap processing unit, the image information that satisfies the conditions related to image quality is specified by using the result of determination by the image quality determination processing unit, and the image information is specified. Is an image processing system that selects as an output target.

By configuring as in the present invention, among the image information used for modeling the object, a part of the image information having a high degree of duplication can be thinned out, and the number of image information used for modeling can be reduced. Can be reduced. In addition, since the image information is selected in consideration of the image quality, modeling can be performed with high quality image information, so that the accuracy at the time of modeling can be improved.

In the above-described invention, the image processing system includes an image processing unit that estimates shooting position information when information such as shooting position is not associated with the image information that has received the input. It can be configured like a system.

It is preferable to attach a device for measuring the shooting position and shooting direction to the shooting device before shooting the object, but even if this is not the case, the processing of the present invention can be used to shoot the shooting position and shooting direction. Information such as the shooting position can be estimated.

In the above-described invention, the imaging position estimation processing unit calculates the movement amount by associating the feature points of the image information to be compared with each other, and calculates the relative position using the calculated movement amount. It can be configured like an image processing system that estimates information such as the shooting position of image information.

There are various processes for estimating information such as the shooting position, but it is preferable to execute the process as in the process of the present invention.

In the above-described invention, the image processing system can be configured like an image processing system having a shooting area calculation processing unit that calculates a shooting area in the image information by using information such as a shooting position in the image information. ..

The imaging area of image information can be calculated by using the present invention.

A fifth invention is an image processing system, which is an image information input reception processing unit that accepts input of a plurality of image information obtained by photographing an object, and an image information selection processing unit that selects image information from the plurality of image information. The image information selection processing unit is an image processing system that selects image information that satisfies the condition related to image quality from among the image information satisfying the condition related to the degree of duplication of the shooting area of the image information. be.

The present invention may be used when necessary image information is selected from a plurality of image information.

In the above-described invention, the image processing system includes an output processing unit that outputs image information selected by the image information selection processing unit to software that executes modeling processing of an object based on the image information. It can be configured like a processing system.

In each of the above-mentioned inventions, when the image information selected by the processing of the present invention is output to software for modeling an object based on a plurality of image information, a high technical effect can be obtained.

The first invention can be realized by loading and executing the program of the present invention in a computer. That is, the computer is used in the image information input reception processing unit that accepts the input of a plurality of image information used for modeling the object, and the image information that has received the input overlaps with the shooting area of the reference image information. An image processing program that functions as an overlap processing unit that specifies image information that satisfies the conditions related to the degree, and an image quality determination processing unit that determines the image quality of the image information, and is the image information specified by the overlap processing unit. Among these, it is an image processing program that identifies image information that satisfies the conditions related to image quality by using the result of determination by the image quality determination processing unit, and selects the image information as an output target.

The fifth invention can be realized by loading the program of the present invention into a computer and executing it. That is, it is an image processing program that causes a computer to function as an image information input reception processing unit that accepts input of a plurality of image information obtained by photographing an object, and an image information selection processing unit that selects image information from the plurality of image information. The image information selection processing unit is an image processing program that selects image information that satisfies the condition related to image quality from among the image information satisfying the condition relating to the degree of duplication of the imaging area of the image information.

By inventing the image processing system of the present invention, the processing time required for reconstructing the three-dimensional model can be shortened. Further, by selecting the image information in consideration of the image quality, it is possible to improve the accuracy of the reconstruction of the three-dimensional model.

It is a block diagram which shows an example of the processing function of the image processing system of this invention. It is a block diagram which shows an example of the processing function of the photographing position estimation processing part in the image processing system of this invention. It is a block diagram which shows an example of the processing function of the image information selection processing part in the image processing system of this invention. It is a block diagram which shows an example of the hardware composition of the computer used in the image processing system of this invention. It is a flowchart which shows an example of the whole processing process of the image processing system of this invention. It is a flowchart which shows an example of the processing process of the imaging position estimation processing of the image processing system of this invention. It is a flowchart which shows an example of the processing process of the image information selection processing of the image processing system of this invention. It is a figure which shows an example of the concept of processing in an image information selection processing part. It is a table which shows the experimental result which compared the case where the processing of SfM was executed using all the image information, and the case where the processing of SfM was executed using the image information selected by using this invention. It is a figure which shows the 3D model which executed SfM processing using all image information, and 3D model which performed SfM processing using image information selected by using this invention.

An example of the configuration of the entire processing function of the image processing system 1 of the present invention is shown in FIG. 1, and an example of the configuration of the processing function of the photographing position estimation processing unit 21 described later is shown in FIG. FIG. 3 shows an example of the configuration of the processing function, and FIG. 4 shows an example of the hardware configuration of the computer used in the image processing system 1.

The image processing system 1 uses a control computer 2. The control computer 2 is a computer for realizing the processing function in the image processing system 1.

The control computer 2 includes an arithmetic unit 70 such as a CPU that executes arithmetic processing of a program, a storage device 71 such as a RAM or a hard disk that stores information, a display device 72 such as a display, and an input device 73 that inputs information. And a communication device 74 that communicates the processing result of the arithmetic unit 70 and the information stored in the storage device 71. When the computer is provided with a touch panel display, the display device 72 and the input device 73 may be integrally configured. Touch panel displays are often used, for example, in portable communication terminals such as tablet computers and smartphones, but are not limited thereto.

The touch panel display is a device in which the functions of the display device 72 and the input device 73 are integrated in that input can be performed directly on the display with a predetermined input device (such as a pen for a touch panel) or a finger.

The image processing system 1 may be realized by one computer, but some or all of its functions may be realized by a plurality of computers. The computer in this case may be, for example, a cloud server. Further, the computer in the image processing system 1 may be a portable communication terminal such as a mobile phone, a smartphone, or a tablet computer.

Each means in the image processing system 1 of the present invention is only logically distinguished in its function, and may form the same area physically or substantially.

The image processing system 1 includes an image information input reception processing unit 20, a shooting position estimation processing unit 21, a shooting area calculation processing unit 22, an image information selection processing unit 23, and an output processing unit 24.

The image information input reception processing unit 20 receives input of image information obtained by photographing an object such as a structure from an imaging device or the like. The image information may be a moving image or a still image. In the case of a moving image, the process is executed after converting it to a still image for each frame. The image information obtained by photographing an object or the like is continuous, and the image information before and after the image information is close to each other. Therefore, in the image information for which input is received by the image information input reception processing unit 20, the same area is photographed from different positions (different viewpoints) for the previous and next image information. The same area may be photographed from the same position (same viewpoint). Further, the image information may be associated with information indicating the shooting position and the shooting direction (information such as the shooting position) for each image information by attaching a device for recording the shooting position and the shooting direction to the shooting device. ..

The shooting position estimation processing unit 21 executes a process of estimating the shooting position information of the image information when the shooting position information is not associated with the image information. That is, among the image information input received by the image information input reception processing unit 20, the feature points of the two image information to be compared are associated with each other, and the relative position is calculated based on the correspondence, so that the information such as the shooting position can be obtained. presume.

The shooting position estimation processing unit 21 has a feature point extraction processing unit 210, a feature point correspondence search processing unit 211, and a relative position calculation processing unit 212.

The feature point extraction processing unit 210 extracts feature points in the image information. Examples of the processing for extracting feature points include SIFT, SURF, ORB, and KAZE, but are not limited thereto.

The feature point correspondence search processing unit 211 obtains a projection matrix that associates feature points in two image information.

The relative position calculation processing unit 212 calculates the relative position of the image information PY with respect to the image information PX using the movement amount calculated based on the projection matrix specified by the feature point correspondence search processing unit 211, and uses it as information such as the shooting position. presume.

Specifically, one image information P1 for which input is received by the image information input reception processing unit 20, for example, the first image information among the image information for which input is received by the image information input reception processing unit 20 is specified, and the first image information is specified. The feature point extraction processing unit 210 extracts the feature points for the image information P1. Further, the shooting position of the image information P1 is set as the origin. Then, the image information P2 to be compared with the image information P1 (for example, image information having an area overlapping with the image information P1 such as the next image information of the image information P1) is specified, and the feature points of the image information P2 are feature points. The extraction processing unit 210 extracts. Then, the feature point correspondence search processing unit 211 obtains a projection matrix for associating the feature points in the image information P1 with the feature points in the image information P2 by a known method. Since the vector of feature points is mainly translation and rotation, other feature points are deleted as erroneous judgments. Then, the relative position calculation processing unit 212 calculates the amount of movement from the image information P1 to the image information P2 based on the projection matrix obtained by the feature point correspondence search processing unit 211, so that the image information P1 of the image information P2 is calculated. The relative position to the image is obtained and estimated as information such as the shooting position.

The shooting position estimation processing unit 21 may use the image information received by the image information input reception processing unit 20 as it is in the above processing, but for example, conversion to grayscale or the like, file size (vertical or horizontal). After reducing the amount of data by reducing the size of), the feature point extraction processing unit 210, the feature point correspondence search processing unit 211, the relative position calculation processing unit 212, and the like may be processed. This is because the color of the image information and the size of the image information are not required to extract the feature points.

The shooting area calculation processing unit 22 calculates the shooting area of the object based on the shooting position and the like information associated with the image information or the shooting position and the like information estimated by the shooting position estimation processing unit 21. Specifically, it accepts input of the information of the photographing device (image sensor size, focal length of the lens) and the average distance to the object to be photographed, and based on this information, the actual size (large size) of the object reflected in the image information. ) Is calculated. Then, the shooting area is calculated from information such as the shooting position indicating the shooting position and the shooting direction by using the length of mm obtained by converting the length of the shot image in pixel units into the actual size.

The image information selection processing unit 23 executes a process of selecting necessary image information (thinning out unnecessary image information) from the image information in the image information input reception processing unit 20. The image information selection processing unit 23 includes an overlap processing unit 230 and an image quality determination processing unit 231.

The overlap processing unit 230 identifies the image information in which the overlap rate of the image information of interest with the photographing region is equal to or greater than a predetermined threshold value. Then, among the specified image information, the optimum image information by the image quality determination processing in the image quality determination processing unit 231 is specified and added as an output target list. The overlap rate can be calculated by calculating the ratio of how much the shooting areas of the two image information to be compared overlap.

The image quality determination processing unit 231 determines whether or not the image information is the optimum image information. Whether or not the image information is optimal can be determined by scoring factors related to image quality, such as whether the image information has high contrast or the image information has clear edges. Whether or not the image information is optimal is not limited to these, and any process used to judge the good image quality (the image information has no blur or blur) should be used. Can be done. For example, machine learning may be used to learn and judge the quality of image quality. Whether or not the contrast is high can be determined based on the variance of the luminance value in the image information, and a score (evaluation value) according to the magnitude of the variance is given. Image information with a large variance is given a high score as high contrast. Whether or not the image information has edges can be determined from the variance of the Laplacian filter (difference filter), and a score (evaluation value) is given according to the size of the variance. An image quality score, which is an evaluation value obtained by scoring elements related to image quality, is calculated by performing a predetermined calculation using each evaluation value for contrast and edge. If the image quality score is high, it can be determined that the image information has high measurement accuracy suitable for reconstruction of a three-dimensional model, and if it is low, it can be determined that the image information is not. The processing related to contrast and edge is not limited to these, and other processing can be used. Further, the threshold value of the overlap rate may be input at an arbitrary timing or may be set to a predetermined value in advance.

The process of scoring the image quality of each image information in the image quality determination processing unit 231 may be performed at any timing before using it.

FIG. 8 shows an example of the concept of processing in the image information selection processing unit 23. When the image information of interest is P1, the overlap processing unit 230 searches for image information satisfying a predetermined overlap rate (for example, 20%) from the image information after the image information P1. .. Then, as a result of the search, image information P2 (overlap rate to P1 60%), image information P3 (overlap rate to P1 40%,), and image information P4 (overlap rate to P1 20%) were identified. And. Further, it is assumed that the image quality scores of the image information P2, P3, and P4 specified by the image quality determination processing unit 231 are 10, 30, and 20, respectively.

At this time, the overlap processing unit 230 selects the image information P3 having the highest image quality score and adds it to the output target list.

Next, the overlap processing unit 230 searches for image information whose overlap rate with the image information P3 is equal to or greater than a predetermined threshold value from the image information after the image information P3, based on the selected image information P3. .. Then, as a result of the search, image information P4 (overlap rate to P3 80%), image information P5 (overlap rate to P3 40%,), and image information P6 (overlap rate to P3 20%) were identified. And. Further, it is assumed that the image quality scores of the image information P4, P5, and P6 specified by the image quality determination processing unit 231 are 20, 30, and 40. At this time, the overlap processing unit 230 selects the image information P6 having the highest image quality score and adds it to the output target list.

Next, the overlap processing unit 230 repeats the same processing as described above based on the selected image information P6 until there is no subsequent image information.

As described above, conventionally, where there were six image information of image information P1 to P6, only three images of image information P1, image information P3, and image information P6 were selected, and image information P2 and image information were selected. P4 and image information P5 can be thinned out. Moreover, since the three selected image information are all better than the other image information whose image quality overlaps, the accuracy can be improved when reconstructing the three-dimensional model.

The overlap processing unit 230 selected the image information having the best image quality score determined by the image quality determination processing unit 231 among the image information satisfying the overlap rate threshold value, but the number is not limited to one. The image information of may be selected. For example, image information whose image quality score is equal to or higher than a certain threshold value may be added to the output target list. When a plurality of image information is selected, it is preferable, but not limited to, to select the image information at the end of the selected image information as the image information to be used as the reference for the next processing.

In addition, when image information is selected by judgment based on overlap, there is a possibility that image information may be omitted, for example, image information in the corner of a structure. Therefore, the image information selection processing unit 23 may execute a process of adding the image information around the shooting area, which is not added to the output target list, to the output target list. Specifically, for the entire shooting area covered by the image information included in the output target list, the entire current shooting area can be enlarged from the image information not included in the output target list. This is done by adding the information to the output target list.

The output processing unit 24 extracts the image information in the output target list and outputs it as the image information used in the reconstruction processing of the three-dimensional model or the like. For example, the image information in the output target list is input to the software that executes the SfM process.

Next, an example of the processing process using the image processing system 1 of the present invention will be described with reference to the flowcharts of FIGS. 5 to 7.

First, the control computer 2 is made to read the image information P1 to Pn obtained by photographing an object such as a structure to be three-dimensionally modeled, and the image information input reception processing unit 20 receives the input (S100). At this time, if there is information such as the shooting position, the input is accepted in association with the image information. The image information input reception processing unit 20 stores the input-received image information, shooting position, and other information in the storage device 71.

When the information such as the shooting position is not associated with the image information (S110), the shooting position estimation processing unit 21 executes the shooting position estimation process (S120).

Specifically, first, the photographing position estimation processing unit 21 converts the image information stored in the storage device 71 into grayscale, and reduces the vertical and horizontal sizes of the file by the same magnification. Then, the feature point extraction processing unit 210 extracts the feature points of the image information P1 used as the origin of the information such as the shooting position (S200). Further, the shooting position of the image information P1 and the shooting direction thereof are set as a reference origin. The shooting position and shooting direction may be input by the person in charge.

Then, the feature point correspondence search processing unit 211 sets the image information P1 as the first comparison image PX and the next image information P2 of the image information P1 as the second comparison image PY (S220). Further, the feature point extraction processing unit 210 extracts the feature points of the second comparative image PY (image information P2) (S230).

When the feature points of the first comparison image PX and the second comparison image PY can be extracted in this way, the feature point correspondence search processing unit 211 deletes the feature points having a vector other than translation and rotation. Then, the projection matrix that associates the feature points of the first comparative image PX with the feature points of the second comparative image PY is obtained (S240).

Then, the relative position calculation processing unit 212 calculates the amount of movement from the shooting position of the first comparative image PX to the second comparative image PY based on the projection matrix, and calculates the relative position of the second comparative image PY. (S250). Thereby, the relative position of the image information P2 with respect to the shooting position of the image information P1 can be specified.

Then, when there is image information next to the image information P2 (S260), the second comparison image PY is the first comparison image PX, and the second comparison image PY (image information P2) is the next image information P3. It is set as the comparison image PX of 2 (S270), and the same processing as after S230 is repeated.

By executing such processing, the relative positions of adjacent image information can be specified. Therefore, the shooting position estimation processing unit 21 sets the relative position to the absolute position based on the shooting position and other information of the image information P1 as the origin. By converting to, the absolute position of information such as the shooting position of each image information is estimated.

After the shooting position estimation processing unit 21 estimates the shooting position information of each image information as described above, or when the shooting position information is associated with the image information (S110), the shooting area calculation processing unit 22 However, the shooting area (area in which the image information is copied) in each image information P1 to Pn is calculated based on the information such as the shooting position (S130).

After the shooting area calculation processing unit 22 calculates the shooting area for each image information P1 to Pn, the image information selection processing unit 23 thins out unnecessary image information and executes an image information selection process for selecting the necessary image information. (S140).

First, the image information selection processing unit 23 adds the image information P1 as the reference image information to the output target list (S300). Then, the overlap processing unit 230 sets the image information P1 as the attention image information Pα and the shooting area A1 of the image information P1 as the shooting area Aα of the attention image information Pα (S310).

Then, the overlap processing unit 230 obtains image information having a shooting area that is equal to or greater than the threshold value of the overlap rate with the imaging region Aα, for example, 20% or more, from the image information after the attention image information Pα (image information P1). Search and identify (S320). In the case of FIG. 8, it is said that the image information P2 (overlap rate to P1 is 60%), the image information P3 (overlap rate to P1 is 40%,), and the image information P4 (overlap rate to P1 is 20%) are specified. do. Since the image information is continuous, once there is image information that does not satisfy the overlap rate threshold value, it is unlikely that the subsequent image information will satisfy the overlap rate threshold value. Therefore, when the overlap processing unit 230 determines that there is image information that does not satisfy the overlap rate threshold value, the search for the image information that satisfies the overlap rate may be completed at that time.

Then, when the image information satisfying the overlap rate is specified in S320, the image information selection processing unit 23 has the image information having the image quality score satisfying the predetermined condition based on the image quality score scored by the image quality determination processing unit 231. For example, the image information having the highest image quality score is specified as the image information to be added to the output target list, and the image information is added to the output target list (S330). For example, in FIG. 8, assuming that the image quality scores of the image information P2, P3, and P4 specified by the image quality determination processing unit 231 are 10, 30, and 20, respectively, the image information P3 is specified as the image information to be added to the output target list. do. Further, the image information P3 added to the output target list is set as the image information Pβ, and the shooting area A3 of the image information P3 is set as the shooting area Aβ of the image information Pβ (S330).

When there is image information after the image information P3 added to the output target list (S340), the image information Pβ (image information P3) is set to the attention image information Pα, and the shooting area Aβ is set to the shooting area Aα of the attention image information Pα (S340). It is set as the shooting area A3) of the image information P3 (S350). Then, the processing after S320 is repeated.

The overlap processing unit 230 executes the same processing. For example, as shown in FIG. 8, the image information having a shooting area in which the overlap rate with the shooting area Aα (shooting area A3 of the image information P3) is 20% or more is after the attention image information Pα (image information P3). Image information P4 (overlap rate to P3 80%), image information P5 (overlap rate to P3 40%,), image information P6 (overlap rate to P3 20%) are specified by searching from the image information of. (S320). Then, assuming that the image quality scores of the specified image information P4, P5, and P6 are 20, 30, and 40, the image information satisfying the conditions related to the image quality score, for example, the image information P6 having the best image quality score is output. It is specified as image information to be added to the list (S330).

When there is image information after the image information P6 added to the output target list (S340), the image information Pβ (image information P6) is set to the attention image information Pα, and the shooting area Aβ is set to the shooting area Aα of the attention image information Pα (S340). It is set as the shooting area A6) of the image information P6 (S350). Then, the processing after S320 is repeated.

The image information selection processing unit 23 repeats the above processing until there is no subsequent image information.

The output target list information, which is a list of image information selected by the image information selection processing unit 23, is stored in the storage device 71.

When the image information selection processing unit 23 determines in S340 that the subsequent image information has disappeared, the image information selection processing unit 23 adds the image information around the shooting area, which is not added to the output target list, to the output target list. (S360). This process may be performed manually by the person in charge.

After the image information selection process by the image information selection processing unit 23 is completed as described above, the output processing unit 24 extracts the image information in the output target list from the storage device 71 and outputs it (S150). For example, image information is passed to software that executes SfM processing.

Then, the software that executes the SfM processing executes the SfM processing based on the image information received from the image processing system 1 of the present invention, thereby shortening the processing time and having high accuracy in three dimensions. The model can be reconstructed.

By using the image processing system 1 of the present invention, it is possible to perform the processing by dividing the entire object into parts (for example, the pier portion of the bridge) instead of processing the entire object at once.

In the case where all the image information obtained by photographing the object (the pier portion of the bridge) is 141 images, when the image information selected by the image processing system of the present invention with the overlap rate of 70% or more is 70 images, the figure is shown. The experimental results shown in 9 were obtained. In the experiment of FIG. 9, Intel Core i7-6850K 3.6 GHz was used as the CPU, 64 GB was used as the memory, three NVIDIA GTX 1060s were used as the Graphics, Windows 10 64 bits were used as the OS, and PhotoScan Ver. 1.4.1 was used.

According to the experimental results of FIG. 9, the processing time when the SfM processing is executed for all 141 image information is 4 hours 24 minutes 1 second, whereas the processing by the image processing system 1 of the present invention is performed. When used, the processing time of the processing (pre-sorting processing time) of the present invention was 8 minutes and 45 seconds, and the processing time when the SfM processing was executed was 3 hours and 9 minutes and 50 seconds. Therefore, the total processing time is 3 hours 18 minutes 35 seconds, and the processing time of 1 hour 5 minutes 26 seconds (about 30% of the total) can be shortened. In FIG. 9, all the images were 141 images, but in the actual work, for example, about 5000 images of one bridge are used for reconstructing the three-dimensional model. Therefore, it takes about 3 days as the processing time. If the processing time can be shortened by about 30% using the image processing system 1 of the present invention, the processing time of about 2.1 days is sufficient, and the processing time can be significantly reduced.

Further, FIG. 10 shows a three-dimensional model of the pier portion of the bridge when the experimental result of FIG. 9 is output. FIG. 10A shows a case where the SfM process is executed using all 141 image information, and FIG. 10B shows the SfM process using the 70 image information output by the process of the present invention. Is executed. Even if the image information is selected by the process of the present invention, the three-dimensional model can be reconstructed.

The image processing system 1 of the present invention can be applied not only to SfM processing but also to the case of reconstructing one image information or model based on a plurality of image information, for example, the case of panoramic composition of photographs of structures.

It can also be used to create correct answer data for machine learning. That is, in order to perform machine learning for recognizing an object reflected in the image information, it is necessary to prepare correct answer data of what is reflected in the image information. When creating this correct answer data, it is also applied to the process of removing similar image information (image information with a large overlap rate and no change) from the image information of the captured object to streamline the work of creating the correct answer data. can do.

By inventing the image processing system 1 of the present invention, the processing time required for reconstructing the three-dimensional model can be shortened. Further, by selecting the image information in consideration of the image quality, it is possible to improve the accuracy of the reconstruction of the three-dimensional model.

1: Image processing system 2: Control computer 20: Image information input reception processing unit 21: Shooting position estimation processing unit 22: Shooting area calculation processing unit 23: Image information selection processing unit 24: Output processing unit 70: Computing device 71: Storage Device 72: Display device 73: Input device 74: Communication device 210: Feature point extraction processing unit 211: Feature point correspondence search processing unit 212: Relative position calculation processing unit 230: Overlap processing unit 231: Image quality determination processing unit

Claims (8)

An image processing system for multiple image information used to model an object.
The image processing system is
An image information input reception processing unit that accepts the input of a plurality of image information, and an image information input reception processing unit.
Of the image information that has received the input, the overlap processing unit that specifies the image information that satisfies the condition regarding the degree of overlap of the reference image information with the shooting area, and the overlap processing unit.
It has an image quality determination processing unit that determines the image quality of the image information.
Of the image information specified by the overlap processing unit, the image information that satisfies the conditions related to image quality is specified by using the result determined by the image quality determination processing unit, and the image information is selected as an output target.
An image processing system characterized by this. The image processing system is
When the information such as the shooting position is not associated with the image information that has received the input, the shooting position estimation processing unit that estimates the information such as the shooting position,
The image processing system according to claim 1, wherein the image processing system comprises. The shooting position estimation processing unit
The amount of movement is calculated by associating each feature point of the image information to be compared, and the relative position is calculated using the calculated amount of movement to estimate information such as the shooting position of the image information.
The image processing system according to claim 2, wherein the image processing system is characterized by the above. The image processing system is
Shooting area calculation processing unit that calculates the shooting area in the image information using information such as the shooting position in the image information,
The image processing system according to claim 2 or 3, wherein the image processing system has. It is an image processing system
An image information input reception processing unit that accepts input of multiple image information of an object, and
It has an image information selection processing unit that selects image information from the plurality of image information.
The image information selection processing unit is
Select the image information that satisfies the condition related to image quality from the image information that satisfies the condition related to the degree of duplication of the shooting area of the image information.
An image processing system characterized by this. The image processing system is
An output processing unit that outputs the image information selected by the image information selection processing unit to software that executes modeling processing of an object based on the image information.
The image processing system according to any one of claims 1 to 5, wherein the image processing system has. Computer,
Image information input reception processing unit that accepts input of multiple image information used to model an object,
An overlap processing unit that specifies image information that satisfies the condition regarding the degree of overlap of the reference image information with the shooting area among the image information that has received the input.
Image quality determination processing unit that determines the image quality of the image information,
It is an image processing program that functions as
Of the image information specified by the overlap processing unit, the image information that satisfies the conditions related to image quality is specified by using the result determined by the image quality determination processing unit, and the image information is selected as an output target.
An image processing program characterized by this. Computer,
Image information input reception processing unit that accepts input of multiple image information of an object,
Image information selection processing unit that selects image information from the plurality of image information,
It is an image processing program that functions as
The image information selection processing unit is
Select the image information that satisfies the condition related to image quality from the image information that satisfies the condition related to the degree of duplication of the shooting area of the image information.
An image processing program characterized by this.

Images