KR102567163B1 - System for operating an imaging device, damage analysis and evaluation of structures - Google Patents

Abstract

The present invention relates to a system for operating a photographing device and analyzing and evaluating structure damage.
A system for operating a photographing device, analyzing and evaluating structure damage according to the present invention includes an input unit for receiving image information captured by a smart device or camera (eg, a DSLR camera), a memory storing an operating program for the photographing equipment, and the operating program A processor that executes, wherein the processor obtains the captured image information and measurement information of a distance sensor, transmits photographing-related control information, and transmits the photographing-related control information including lighting-related control information.

Description

Operation of filming device, structural damage analysis and evaluation system

The present invention relates to a system for operating a photographing device and analyzing and evaluating structure damage.

According to the prior art, structure inspection by manpower causes deviations in inspection results depending on the visual acuity and proficiency of the inspector, and the reliability of the final result deteriorates due to the accumulation of errors during the process of creating the external survey network.

In addition, when scanning tunnels and road pavements using an industrial vision camera, there is a limit to focusing, and when shooting using a drone, the focus of the digital image shakes and the weight of the sensor mounted to measure information such as the shooting location and distance However, there is a problem that cannot be applied to bridges, buildings, port facilities, and power generation facilities that require long-distance high-resolution photography.

In addition, the existing technology performs damage analysis by applying the gray scale method to the measured image information, which has a high false detection rate of cracks, and it is difficult to detect cross-section damage such as whiteness, exposure of reinforcing bars, peeling, and breakage. There is a problem.

In the gray scale method according to the prior art, there is a problem in that an inspector may not find damage in an image or may mistakenly recognize non-damage as damage, and in this case, an on-site re-examination by manpower is required.

According to the prior art, there are problems in terms of the photographing equipment that has limitations in the photographing angle, cannot acquire many images due to the small photographing area, is heavy and bulky, has complicated connection lines, and cannot photograph in a dark place.

According to the prior art, there is an inconvenience of having to manually move a captured image, there are limitations in that focusing information transmission and location selection are impossible, and there are problems in terms of control software that makes it difficult to check images in real time.

According to the prior art, there are limitations in image merging and the types of damage analysis are limited.

The present invention has been proposed to solve the above problems, and an object of the present invention is to provide a system for operating an imaging device, analyzing and evaluating structural damage, which can improve safety, precision, reliability, economy, and productivity of structure inspection and diagnosis. .

Specifically, the shooting angle and shooting area are enlarged, portability is improved, simple connection is possible, shooting is performed using a shooting device equipped with a lighting system, and images are captured using IoT for the captured data. It transmits wirelessly and automatically transmits focusing information through an Android app, making it easy to select a location and real-time checking of images, increasing the efficiency of image merging and damaging detection targets due to steel bridge paint damage, concrete leaks, material separation, etc. Its purpose is to provide a system capable of expanding the types of

In addition, the structure is divided and photographed at a preset overlapping rate, an image of the entire structure is implemented using the divided and overlapped image, and cracks, whiteness, exfoliation, damage and reinforcing bar exposure are detected using an artificial intelligence algorithm learned from selected data. Damage of steel bridge, damage to steel bridge painting, concrete leakage, material separation, etc. are expanded and detected, the detected damage area is displayed on the image, and the detected damage is quantified using measurement distance and angle information, and the document automation program Its purpose is to provide a photographic device operation, structure damage analysis and evaluation system that creates an external survey network map and damage quantity table through

A system for operating a photographing device, analyzing and evaluating structure damage according to the present invention includes an input unit for receiving image information captured by a smart device or camera (eg, a DSLR camera), a memory storing an operating program for the photographing equipment, and the operating program A processor that executes, wherein the processor obtains the captured image information and measurement information of a distance sensor, transmits photographing-related control information, and transmits the photographing-related control information including lighting-related control information.

According to the present invention, there is an effect of providing operation software capable of automatic precise control of photographing hardware and securing an overlap matching rate by performing overlapping photographing for matching divided images.

According to the present invention, by using an artificial intelligence algorithm that overcomes the limitations of image analysis techniques according to the prior art, there are five types of cracks, efflorescence, peeling, exposure of reinforcing bars, and damage that may occur during use due to aging of concrete, as well as painting of steel bridges. It is possible to perform expanded damage analysis such as damage, concrete leakage, and material separation, and it is possible to automatically generate an external survey network map and damage quantity table using detected and quantified damage information.

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 shows the main functions of operating software according to an embodiment of the present invention.
2 shows an image overlapping portion according to an embodiment of the present invention.
3 illustrates a system for operating a photographing device and analyzing and evaluating structure damage according to an embodiment of the present invention.
4 shows an angle of view and an adjusted distance according to an embodiment of the present invention.
5 illustrates a photographing decision and a photographing sequence according to an embodiment of the present invention.
6 illustrates a structure damage analysis and evaluation system according to an embodiment of the present invention.
7 is an example of image merging according to an embodiment of the present invention.
8 shows the structure of damage detection software according to an embodiment of the present invention.
9 shows a damage detection step according to an embodiment of the present invention.
10 shows an RPN structure according to an embodiment of the present invention.
11 shows a damage probability map through RPN according to an embodiment of the present invention.
12 shows a camera pin hole model according to an embodiment of the present invention.
13 is an example of damage quantification according to an embodiment of the present invention.
14 illustrates a documentation process according to an embodiment of the present invention.
15 is an example of documentation according to an embodiment of the present invention.
16 shows a GUI screen of damage analysis and evaluation software according to an embodiment of the present invention.
17A to 17D show a high-resolution image acquisition device according to an embodiment of the present invention.
18 shows a lighting device according to an embodiment of the present invention.
19 shows a high-resolution image acquisition device in which a lighting unit is disposed according to an embodiment of the present invention.
20 shows photographing equipment disposed on a moving cart according to an embodiment of the present invention.
21 shows a system configuration diagram according to an embodiment of the present invention.
22 shows a software configuration diagram according to an embodiment of the present invention.
23 shows a control and measurement device according to an embodiment of the present invention.
24 shows data flow according to an embodiment of the present invention.
25 illustrates an image merging process according to an embodiment of the present invention.

The foregoing and other objects, advantages and characteristics of the present invention, and a method of achieving them will become clear with reference to the detailed embodiments described below in conjunction with the accompanying drawings.

However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms, and only the following embodiments provide the purpose of the invention, As only provided to easily inform the configuration and effect, the scope of the present invention is defined by the description of the claims.

Meanwhile, terms used in this specification are for describing the embodiments and are not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. As used herein, “comprises” and/or “comprising” means the presence of one or more other components, steps, operations, and/or elements in which a stated component, step, operation, and/or element is present. or added.

The photographing device according to the embodiment of the present invention is capable of photographing a predetermined distance (40 meters) or more between the camera and the structure, secures image precision capable of discriminating the crack width predetermined limit (0.1 mm), and wide area Split shooting is possible.

The photographing device according to the embodiment of the present invention may use a smartphone whose angle is adjusted from 0 to 270 degrees or a DSLR camera. In the case of using a smartphone, the shooting screen can be checked from a remote location through communication, and in the case of using a DSLR camera, a field agent can check the shooting screen. In the case of using a smartphone, the image taken based on the IoT is wirelessly transmitted, and the transmission is made to a laptop with operating SW installed, a NAS with a fixed IP, or a cloud using a wireless LAN.

Through the high-resolution automatically controlled image acquisition system according to an embodiment of the present invention, when a high-resolution image is automatically obtained using a shooting hardware that improves shooting speed, shooting angle, weight, and assembly complexity, and there is no natural light using a lighting device It supports smooth shooting, provides image merging, preview of sequentially merging images, and editing functions, and the effect of easy program handling can be expected.

A high-resolution automatically controlled image acquisition system according to an embodiment of the present invention will be described later with reference to FIGS. 17 to 25 .

The photographing device operating software for analyzing and evaluating damage to a structure according to an embodiment of the present invention includes photographing unit operating software, moving and rotating unit operating software, and sensor unit measurement information storage software.

The photographing unit operation software is a program that automatically controls image capture, and transmits and receives a control command for automatically operating a camera shutter and a control command for automatically focusing an image of a telephoto lens.

The software for operating the moving and rotating parts is a program that automatically controls the 4-axis movement, and consists of a program for transmitting control commands and communication, as well as a program for checking the current position and preventing malfunction.

The moving and rotating part operation software transmits control commands to the motor drive included in the linear transfer device of the photographing hardware and the motor of the rotation device, and transmits communication about the state of the motor, encoder pulse, and the like.

The sensor unit measurement information storage software is a program for automatically storing the measured photographing distance, angle, current location information, and the absolute location of the photographing hardware, and transmits the measurement information and measurement information through communication.

The main functions of the operating software according to an embodiment of the present invention are shown in FIG.

2 shows an image overlapping portion according to an embodiment of the present invention.

The suitability of the shooting sequence function of the operating software according to the embodiment of the present invention is performed on the accuracy of images divided and photographed at a preset overlap ratio.

Install the shooting hardware, connect it with the operating software, and use the operating software to form a sequence with a preset overlapping rate to extract feature points of concrete, and photograph in the "d" direction.

Images that are continuously photographed overlapping at a predetermined overlapping rate are input, and as shown in FIG. 3 , overlapping portions are checked and compared.

The number of diagonal pixels of the overlapping images is calculated, feature points of the overlapping images are calculated, pixel errors between the images are calculated, and error rates are calculated as error pixels/total pixels.

As a result of the shooting sequence function suitability test, it is confirmed that the accuracy of the 50% divided image has 99.15%.

As described above, the operating software according to an embodiment of the present invention transmits a control signal to a smart device installed for taking a concrete image, and the smart device drives a camera mounted therein by the control signal.

The software for operating the moving and rotating parts uses a motor to move the photographing part to the correct position in order to photograph the polyhedral concrete structure, transmits control commands to the motor drive, and receives motor states, encoders, and pulses.

The sensor measurement and information storage software is a program that automatically forms a sequence by using distance and angle information for shooting at a specific image overlap rate after specifying the shooting range.

3 illustrates a system for operating a photographing device and analyzing and evaluating structure damage according to an embodiment of the present invention.

A system for operating a photographing device and analyzing and evaluating structure damage according to an embodiment of the present invention controls an input unit 1110 for receiving image information photographed by a smart device or camera and a smart device for photographing equipment. A memory 1120 storing an evaluation program for evaluating the state of a structure using an operating program and image information stored therein, and a processor 1130 executing the operating program and the evaluation program, the processor 1130 includes a captured image Acquire information and measurement information of the distance sensor, perform focusing control for the camera of the smart device, and perform control for division overlapping shooting.

The processor 1130 synchronizes image information and measurement information and transmits them wirelessly.

In wirelessly transmitting captured image information, the processor 1130 transmits image information after data pre-processing and filtering.

The processor 1130 transmits a control command for the focusing position of the camera according to user settings.

The processor 1130 controls the lighting unit to illuminate the photographing area.

The processor 1130 controls division and overlapping shooting with a preset image overlap ratio.

The processor 1130 designates a shooting range for a preset region of the member, creates a table of angles of view having a preset image overlap ratio, and determines whether to shoot for each table.

When inspecting current structures, crack widths of 0.1 mm must be investigated, so high-resolution images are required for damage analysis using images.

Divisional photographing is performed to capture a high-resolution image, and division and overlapping photographing is controlled to combine the divided images into one image.

It is preferable to automatically calculate the information measured from the distance sensor and the angle sensor according to the determination of the overlap ratio of the image merging software, rotate the photographing device, and take pictures at the determined overlap ratio.

At this time, since the resolution of the image is different for each horizontal and vertical direction, a control command for photographing is transmitted after calculating a movement range in the horizontal and vertical directions, respectively.

As described above, since the horizontal and vertical shooting lengths of the captured image resolution are different, adjustment is performed through the angle of view calculation as shown in FIG. 4 in order to maintain a constant overlap rate of the captured images.

According to an embodiment of the present invention, the processor 1130 controls a motor to designate a shooting range, creates a table of angles of view having an overlap ratio of a predetermined ratio (50%), and determines whether to shoot for each table. .

There are a total of three types of shooting cases: 1) when one or more vertices of the shooting angle are included in the structure, 2) when one or more vertices of the structure are included in the shooting angle of view, and 3) 1) and 2) if all are satisfied.

5 illustrates a photographing decision and a photographing sequence according to an embodiment of the present invention.

Based on the result of determining whether or not to take a picture, the table to be taken is moved from the upper left to the right and taken, and when it reaches the right end, it is controlled to take a picture by moving down.

When acquiring a concrete image by specifying a shooting range in a rectangle, there is a problem in that the surrounding background is also captured. However, according to the embodiment of the present invention, by controlling to designate the end edge of a member when designating a shooting range, the interior of the concrete member of the structure is captured. It has the effect of shortening the shooting time by adjusting it to shoot.

According to an embodiment of the present invention, it is possible to expand the types of damage to be detected to include steel bridge paint damage, concrete leakage, and material separation.

The processor 1130 wirelessly transmits measurement information according to a preset timing.

The processor 1130 transmits a rotation control command signal to the rotation member to control the rotation angle of the smart device.

Referring to FIG. 6, by using an image matching unit 1610 that performs matching by receiving image information from a smartphone, and using a convolutional neural network algorithm based on a mask area, cracks, whiteness, peeling, damage and exposure of reinforcing bars, steel bridge paint damage, A damage detection and quantification unit 1620 that detects damage, including at least one of concrete leak and material separation, and quantifies the detected damage, and a documentation unit 1630 that prepares an external survey network map and a damage quantity table includes

The image matching unit 1610 extracts feature points using images divided and photographed at a predetermined overlapping rate, and implements an image of a structure.

The image matching unit 1610 performs image merging by using information about the shooting angle, distance, and overlap ratio of the smart phone.

The damage detection and quantification unit 1620 detects damage using a convolutional neural network algorithm based on a mask area that performs damage location detection, damage type classification, and pixel-by-pixel detection of the damage area.

The damage detection and quantification unit 1620 obtains a probability map using a Region Proposal Network and determines an object using a preset reference probability.

The damage detection and quantification unit 1620 uses the region proposal results of the Region Proposal Network to extract features of damage from candidate boxes through RoIAlign and perform classification.

The damage detection and quantification unit 1620 detects the shape of the object by supplying the feature map extracted through RoIAlign to a mask branch that divides the object of the image in pixel units.

The damage detection and quantification unit 1620 obtains the actual damage size for the damage size of the image using the camera pinhole model.

The documentation unit 1630 converts the coordinates and extent of damage into length, classifies them based on the type and amount of damage, and documents damage information.

The documentation unit 1630 creates an external survey network map using pixel coordinates, type, and range.

An evaluation program according to an embodiment of the present invention includes image merging software, damage analysis and quantification software, and documentation software.

According to an embodiment of the present invention, at least one of crack, efflorescence, exfoliation, exposure of reinforcing bar, damage, damage to steel bridge painting, concrete leak, and material separation is detected by using a method combining FCN and Mask R-CNN. To develop damage analysis and quantification software, training image datasets are prepared, non-learning factors (hyperparameters) are set, and image augmentation and algorithm accuracy verification are performed.

Hereinafter, preparation of the training image dataset will be described.

Artificial intelligence is not a method in which developers implement algorithms for handling various environments, but a method in which a computer detects damage by itself by finding key features in a given image dataset, and requires a lot of training image data.

According to an embodiment of the present invention, the data acquired through the performance of safety inspection and precise safety diagnosis services are managed in a database, and the image data included in the database is selected and used as learning data for an artificial intelligence algorithm network.

At this time, the training images are divided into damaged images and non-damaged images, and damaged images (200,000 copies) and non-damaged images (100 copies) are extracted and set as learning targets, and the quantity of damaged images and non-damaged images can be adjusted. do.

Describing the learning method for each damage, cracks are classified based on the part where the concrete surface is cracked and the part where it is cut off, and only cracks with one or two lines per label are learned. In the case of reinforcing bar exposure, one exposed reinforcing bar is studied as one, and whether the curved shape is similar to a crack is reviewed. It learns mainly on exfoliation, and learns only the parts damaged by external impact for damage.

Hereinafter, the setting of factors outside learning will be described.

The convolutional network used for learning is a residual network 101 (the number of hidden layers is 101), the learning rate is 0.001, the epoch is 100, and the image batch size is 8.

The learning rate is an alpha value that comes before the differential value of the cost value when using an algorithm that finds the point where the loss function is minimized. It is an indicator.

[Equation 1]

In the learning process, the weight (W in Equation 1) is updated through a feed forward process and a back propagation process between the input, hidden layer, and output.

At this time, the entire data is divided and feed forward and back propagation are repeated, and this one process is defined as an epoch.

The data used for one epoch is defined as a mini-batch, and its size is defined as a batch size.

Hereinafter, image augmentation (augmentation) will be described.

In order to improve the performance of the learned Mask R-CNN and prevent overfitting, image augmentation of horizontal and vertical flips is applied randomly in each iteration of learning.

Hereinafter, algorithm accuracy verification will be described.

The accuracy of the algorithm is verified by the loss function value for each epoch, and the loss function is a function used as an optimization technique to find the critical point with the highest accuracy during model learning.

[Equation 2]

Loss function = L _cls + L _box + L _mask

L _cls is softmax cross entropy, L _box is regression, and L _mask is binary cross entropy.

Epoch learning shows a tendency to converge as the value of the loss function gradually decreases.

As shown in FIG. 7 , the image merging unit 610 receives divided images acquired at a predetermined overlapping rate using photographing hardware and operating software, extracts feature points of each image (aligning image), and create an image of

As shown in FIGS. 8 and 9, the damage detection and quantification unit 1120 detects damage using Mask R-CNN, and preferably uses a method combining FCN and Mask R-CNN to detect damage. it is possible to detect

The damage detection and quantification unit 620 quantifies the detected damage size using a camera pinhole model.

Mask R-CNN is a combination of an object classification model and an object segmentation model, and consists of three steps: detecting the location of damage in the input image, classifying the type of detected damage, and detecting the area of damage in pixel units. do.

Hereinafter, the damage location detection step will be described.

In order to detect the location of damage, RPN (Region Proposal Network) is used to find locations where damage is likely to occur in the image.

RPN is a Fully Convolutional Network (FCN) algorithm that outputs the damage probability score reflecting how similar the image is to the damage to be detected and the location of damage in the form of a bounding box.

RPN scans a feature map extracted from a convolutional neural network (CNN) model using a bounding box to generate region proposals.

Each region obtained from the feature map by the bounding box is converted into a low-dimensional feature, and the feature is supplied to the box regression layer and box classification layer.

RPN detects the damage location area at each sliding window position.

Each bounding box has at most k region proposals at one location, as shown in FIG. 10 .

For k areas, the box regression layer has 4 outputs (x-coordinate, y-coordinate, height, and width of the upper left corner) representing the coordinates of the bounding box, and the box classification layer has 2 outputs to indicate the probability that the bounding box is damaged or intact. It has a softmax layer with two classes.

The k areas are called anchors, and area proposals are performed by constantly changing the horizontal and vertical sizes centered on the bounding box.

11 shows that a probability map is obtained from the RPN after training the RPN for damage detection.

After acquiring the probability map of the object extracted by the RPN, a threshold value is set to determine an area above the preset threshold value as an object.

It is possible to conduct an experiment on damage detection as a method of determining the threshold of the probability map, but it can be somewhat difficult to determine experimentally due to the nature of damage data. determine the threshold of

As an example of using a bounding box-type probability map, there is Faster R-CNN, a type of object detection model.

Faster R-CNN determines the area where the probability of being an object is 0.6 or more 25 on the probability map as an object.

According to an embodiment of the present invention, in order to efficiently detect damage that is more densely distributed than general objects, a part having a damage probability of 0.8 or more in a probability map is determined as damage, and the reference probability may be changed.

Hereinafter, the step of classifying the detected damage will be described.

We take region proposal results from RPN, extract features of impairment using RoIAlign from each candidate box, and perform classification.

RoIAlign is an operation that properly aligns the feature extraction result of the candidate box with the input image. RoIAlign creates a w Х h size feature map (7 Х 7 in the application technology) from the area proposal extracted by RPN.

In this process, there is a problem that the alignment is not correct when the position of the region proposal is indicated on the feature map, which expresses the coordinates as an integer using coordinates expressed as real numbers, and if the position of the region proposal is rounded to an integer, there is a problem. Since the extraction result may not be good, RoIAlign is performed so that the exact location of the region proposal is not lost.

RoIAlign calculates the exact value of each feature extraction point using bilinear interpolation from nearby points in the feature map.

Feature maps extracted from RoIAlign not only contribute to object classification with higher accuracy, but also improve the accuracy of object shape extraction.

Hereinafter, the damage shape detection step will be described.

The feature map extracted by RoIAlign detects the damage shape using the mask branch.

The feature map extracted by RoIAlign is passed to the Classification layer and Bounding-Box regressor layer of Faster R-CNN.

The Bounding-Box regressor layer works in a very similar way to how RPN works, fine-tuning the position and size of the bounding box.

The feature map extracted by RoIAlign is supplied to the mask branch to determine the shape of the object in the bounding box.

A mask branch is an FCN that divides an object in an image in units of pixels.

As an example, the structure of the mask branch can be composed of four consecutive 3x3 convolutional layers, a 1x1 deconvolutional layer, and a 1x1 convolutional layer, and the mask branch placed in parallel to the classification layer is an object without the class of the object specified by the classification layer. There is an effect that it is possible to predict the shape.

Hereinafter, damage quantification of the damage detection and quantification unit 1120 according to an embodiment of the present invention will be described.

As shown in FIG. 12, quantification is performed using a camera pinhole model.

The shooting distance and angle are required to calculate 1 pixel of the image, and the size of the analyzed damage is quantified by calculating the size of 1 pixel using the camera pinhole model using the measured distance and angle.

The camera pinhole model is a proportional expression of the actual damage size to the damage size of the image, and the length of the actual damage ( ) is calculated as follows [Equation 3].

[Equation 3]

W _c represents the actual damage length, D _w represents the distance between the concrete and the camera sensor, _PC represents millimeters per pixel of the camera sensor, L _f represents the focal length of the camera, and D _p represents the pinhole model Indicates the distance corresponding to a single pixel in . 13 is an example of damage quantification according to an embodiment of the present invention.

As shown in FIG. 14, the documentation unit 630 according to an embodiment of the present invention uses image merging software and damage detection and quantification software to analyze the damage type, size, shape, and location information by using damage type, size, shape, and location information. Create a damage quantity table and an external survey network map.

In the damage quantity table, the damage detection and quantification software converts the coordinates and range of the analyzed damage into length, classifies them based on the type and amount of damage, assigns a serial number, and documents the classified damage information.

The external survey network map sets the outline of the damage detection target in the image after merging, and documents it in the form of a CAD file using the coordinates, type, and range of pixels analyzed in the damage detection and quantification software. The damage serial number assigned to the exterior survey network map is also assigned the same number to the damage quantity table.

According to an embodiment of the present invention, an outer perimeter is determined, and location information is extracted only for a target within the determined outer perimeter, and automatic documentation is performed.

In addition, the shape of damage shown in the external survey network map is drawn the same as the shape of actual damage. do.

15 is an example of documentation according to an embodiment of the present invention, and FIG. 16 shows a GUI screen of damage analysis and evaluation software.

A high-resolution automatically controlled image acquisition system according to an embodiment of the present invention includes a photographing device for performing photographing and transmitting images and photographing information, a lighting device for irradiating light on a photographing area, and the photographing device. A computing device that performs integrated control of a photographing device that adjusts a photographing angle of the device and the photographing equipment; and a storage unit for storing high-resolution images.

According to another embodiment of the present invention, it is possible to control a rotation operation by using a smartphone or a DSLR camera as a photographing device and transmitting a control signal to the photographing equipment on which the photographing device is mounted.

The computing device transmits a motor control signal to the photographing equipment, and the photographing equipment performs rotation angle control for two orthogonal axes using the motor control signal.

The photographing equipment measures distance information to a subject and position information of a motor in real time using distance sensor information and limit sensor information.

The photographing equipment is supported by a tripod structure, and each leg constituting the tripod is adjusted in a different height to maintain a horizontal level for photographing according to the topography.

The photographing equipment is disposed on a movable carriage, so that it can be easily moved from a photographing location.

The photographing device transmits camera and video state information in real time.

The storage unit includes at least one of a cloud server and a NAS.

The lighting unit includes a communication module, receives a control signal from the computing device, and changes a lighting irradiation area.

The lighting unit is interlocked when the photographing device is driven by controlling the photographing area, and an angle of the lighting irradiation area is changed when the angle of the photographing area is controlled.

According to the present invention, the photographing speed is increased, a photographing angle of 0 to 270 degrees is secured, the structure is lightened and its assembly is simplified, there are advantages in that focusing information and photographed images can be transmitted, and handling is easy. It provides one image merging program and has the advantage of being able to check damage anywhere in real time by utilizing the cloud or NAS.

17A to 17D show a high-resolution image acquisition device according to an embodiment of the present invention, wherein FIG. 17A is a front view, FIG. 17B is a left view, FIG. 17C is a right view, and FIG. 17D is a rear view.

A high-resolution image acquisition device according to an embodiment of the present invention includes a motor generator 101, a base panel 102, a y-axis rotation base 103, a tripod fixing bracket 104, a bearing 105, and an x-axis rotation column 106. ), tripod 107, x-axis rotation pulley 108, fixing bracket 109, distance sensor 110 for measuring the distance between the camera and subject, x-axis rotation bearing 111, y-axis rotation pulley 112 ), camera fixing part 113, y-axis rotation stopper 114, camera fixing bracket 115, lower motor cover 116, smartphone 117, timing belt 118, x-axis motor rotation shaft assembly 119 ).

According to an embodiment of the present invention, a sensor (encoder) for measuring a rotation angle of a rotation transfer device is provided.

The tripod 107 can be adjusted to a preset height, and the adjustable height is manufactured from a minimum of 100 mm to a maximum of 500 to 700 mm. Each leg of the tripod 107 can be adjusted in height so that it can be leveled without affecting the installation terrain, and is manufactured in a prefabricated manner for easy movement, and can be folded and stored when disassembled.

According to an embodiment of the present invention, the camera is rotated in up, down, left and right directions using a servo stepping motor (a driving motor for precisely controlling equipment), and up, down, left and right angle rotation is possible using a plurality of rotation devices.

The rotation angle is set to move in the range of 0 to 270 degrees up and down, left and right, without interference.

According to an embodiment of the present invention, it is possible to perform up, down, left, right, and tilt using a stepping motor, simplify the electric circuit of the driving unit with power consumption DC 24V, and the rotation radius of each axis is set to 270 degrees.

According to an embodiment of the present invention, there are advantages of simplifying the circuit through unification of 24V power consumption, reducing the volume by integrally configuring the motor and the driver, and minimizing the wiring connection.

According to an embodiment of the present invention, using a drive-integrated stepping motor, it is possible to omit an inverter required for driving a motor, and a motor driver (a motor control device that connects a motor and receives a control signal to drive a motor) is integrated through a configuration It is possible to reduce the volume of the control box and simplify wiring.

According to an embodiment of the present invention, by arranging the stepping motor inside the mechanical unit, the center of gravity is aligned to the center to secure stability when installing the equipment and reduce the overall volume.

Referring to FIG. 18, when there is no natural light, a high-resolution image can be acquired using a separate lighting device, and referring to FIG. 19, a high-resolution image can be obtained even when there is no natural light using a lighting device included in the integrated equipment. do.

18 shows a lighting device according to an embodiment of the present invention.

The lighting device according to the embodiment of the present invention is provided separately from the image acquisition device, and a plurality of lighting units connected to the tripod structure are installed. In this case, the lighting unit may include a communication module, and may control the lighting irradiation area by receiving a control signal from the control device. The control signal is determined by comprehensively examining the shooting area of the camera and the area requiring light irradiation, and is transmitted to the communication module of the lighting unit, so that the lighting area of the lighting unit can be automatically controlled.

19 shows a high-resolution image acquisition device in which a lighting unit is disposed according to an embodiment of the present invention.

The lighting units 310a and 310b may be applied in a manner that is disposed in a predetermined area of the high-resolution image capture device and radiates light to a photographing area.

In this case, the irradiation area of the lighting unit may be controlled by receiving a separate control signal, and when driven by the imaging area control for the imaging unit, the irradiation area may be controlled in conjunction with this.

According to an embodiment of the present invention, lighting (eg, 200W lighting) with a predetermined output is installed on the left/right side of the photographing unit to irradiate light when photographing in the absence of natural light.

Since the lighting units 310a and 310b are disposed in a detachable form, it is possible to reduce the volume when stored.

According to an embodiment of the present invention, installation is simplified by using a tripod, and weight is reduced through a structure made of aluminum. As another example, as shown in FIG. 20 , as described above, by arranging the photographing equipment on the moving cart 2010, there is an effect of facilitating movement.

21 shows a system configuration diagram according to an embodiment of the present invention.

The daytime imaging equipment 401 performs motor control, measures distance, and acquires data. Daytime shooting equipment means shooting equipment used when there is natural light.

The night photographing equipment 402 performs motor control, measures distance, acquires data, and performs lighting control. Night photography equipment means photography equipment used when there is no natural light.

The smartphone 403 performs photographing, image transmission, and photographing information transmission.

The wired/wireless router 402 for daytime shooting equipment and the wired/wireless router 405 for nighttime shooting equipment create a closed Wi-Fi network and are securely connected by SSID and PW.

The notebook 406 is installed with integrated control software and analysis software.

The LTE or 5G modem 407 supports connection to the LTE or 5G network and stores images in the cloud or NAS 408 .

According to an embodiment of the present invention, by configuring Wi-Fi for the system to achieve wireless, through which equipment assembly is simplified and problems of non-coupling and misconnection are solved. A nighttime shooting device 404 is provided separately from the daytime shooting device 401, and as the shooting is performed with a smartphone 403, there is an effect of miniaturization and light weight. The notebook 406 is connected to an LTE or 5G network and automatically backs up captured images to the cloud or NAS 408.

All communication is carried out by TCP/IP communication using a wireless closed Wi-Fi network, and the integrated control program (laptop), shooting equipment control program (embedded PC), and smartphone app attempt to establish a communication connection at the same time as the program runs, and lose access. Attempts to reconnect infinitely until the end of the program.

22 shows a software configuration diagram according to an embodiment of the present invention.

The software configuration is classified into integrated control software and sub-control software that controls and measures sub-devices for the purpose of distributed control.

All subordinate control software transmits and receives control, measurement, and equipment status information through wireless communication with the integrated control software through Wi-Fi, and the integrated control software performs interface with the user and overall control of shooting.

Daytime shooting control software is installed in the embedded computer and the daytime shooting equipment 510 . A night photographing control software is installed in the embedded computer and the night photographing equipment 520 . The embedded computer program is built into and driven in the equipment, and transmits signals and data files between devices. It provides device inspection and notification check function, real-time measurement and motor control function of data acquisition equipment, and real-time signal and file relay function by wirelessly connecting to smartphones and laptops. The photographing equipment program and the lighting equipment program control the motor by receiving movement information of the motor necessary for photographing from the integrated control software, and measure distance information from the structure through a distance sensor. In addition, limit sensor information for checking the state of the motor is obtained. The daytime shooting control software and the nighttime shooting control software receive information on motor operation and transmit measurement information through communication with the integrated control software. In addition, it controls a plurality of motors and receives motor status and encoder (angle) values. In addition, distance information and motor position information are measured in real time through a distance sensor and a limit sensor.

A shooting app is installed in the smartphone 530 . The shooting app captures an image according to the control signal and transmits the measured image. It controls the camera by receiving the real-time control signal, measures the image, and transmits the measured image in real time. Camera and image status (focus, image information) is transmitted in real time. The shooting app runs automatically when the smartphone is turned on and runs in the background. After that, when communication with the integrated control software is connected, it is automatically changed to the shooting mode, and shooting proceeds according to the command of the integrated control software. In the shooting mode, information such as focus information and image files is transmitted. When shooting is completed or the user forcibly stops it, communication is released and it is converted to standby mode again.

In the notebook 540, integrated control software and analysis software are installed. The integrated control software is a program that measures and controls the entire equipment. It controls and measures the equipment and the smartphone, user interface function, real-time monitoring and control function, smartphone linkage and camera control function, and video data automatic monitoring and storage. function.

A cloud or NAS 550 stores the images.

23 shows a control and measurement device according to an embodiment of the present invention.

Components connected to the control and measurement device according to an embodiment of the present invention include a laptop computer, a battery and charger, a chassis, a module, a motor driver, an embedded computer, an access point, and a power device.

The chassis connects the control and measurement modules and measures the data.

The module measures the sensor signal and generates a control signal.

The motor driver delivers control commands to the motor and corrects errors when they occur.

The embedded computer receives the user's command, measures and measures the data of the equipment, and collects video control and video files from the smartphone and transmits them to the laptop computer.

The access point connects the components of the system to the network.

The power device is a device that supplies power to the control device and can drive AC220 or DC24.

The laptop computer has a built-in operating program, receives user information, and wirelessly measures and controls the system in real time.

Batteries and chargers are provided to use the equipment in places without wired power, and run for a maximum preset time (eg 5 hours).

24 shows data flow according to an embodiment of the present invention.

A dotted line indicates a wireless connection, and a solid line indicates a wired connection.

In FIG. 24 , black lines show control signals, blue lines show data flow through wired LAN, red lines show power signals, and purple lines show data flow through Wi-Fi.

The data acquisition device and the embedded computer are connected to the access point and wired LAN.

The power device supplies power to the motor driver, data acquisition device, access point, and embedded computer using AC 220V or battery power.

The motor driver sends control signals to the data acquisition device.

The access point is connected via Wi-Fi communication with smartphones and laptop computers.

Control commands are transmitted between the motor driver and the data acquisition device, and the data acquisition device and the embedded computer transmit and receive data through a wired LAN, and the embedded computer and the laptop computer through Wi-Fi communication.

Image data is transmitted and received through Wi-Fi communication between the smart phone and the embedded computer, and between the embedded computer and the laptop computer.

25 illustrates an image merging process according to an embodiment of the present invention.

According to an embodiment of the present invention, when merging image path information is received after merging starts, image merging (receiving image coordinate information by an analysis program), error checking (determination of result of merging completed image, providing editing function through user interface in case of error) ), preview (sequential merging images are compressed and saved to display images for preview), and user editing processes (image merging and post-processing are performed under user control if the user wants to edit when checking an error), and merging is performed. End (images are compressed and saved, and split images for the damage detection module are saved). That is, the image merging program provides a progress rate and a compressed image of a currently merged image to the user, and provides a handling function through a user interface when there is an error in merging.

Claims (20)

An input unit for receiving image information captured by a smart device or camera;
a memory in which operation programs for photographing equipment are stored; and
Including a processor that executes the operating program,
The processor obtains the captured image information and measurement information of a distance sensor, transmits shooting-related control information, and transmits the shooting-related control information including lighting-related control information;
The processor performs control for divisional overlapping shooting at a preset image overlap rate, designates a shooting range for a preset region of a member, creates a table of angles of view having the preset image overlap rate, and shoots for each table. to judge whether
Phosphor filming device operation, structural damage analysis and evaluation system.
According to claim 1,
The input unit receiving the image information obtained through wireless communication
Phosphor filming device operation, structural damage analysis and evaluation system.
According to claim 1,
The processor transmits the shooting-related control information for determining a shooting range using distance and angle information
Phosphor filming device operation, structural damage analysis and evaluation system.
According to claim 1,
The processor transmits the lighting-related control information to control the lighting angle to the photographed part.
Phosphor filming device operation, structural damage analysis and evaluation system.
According to claim 1,
The processor transmits a control command for a shooting focusing position according to a user setting
Phosphor filming device operation, structural damage analysis and evaluation system.
delete delete According to claim 1,
The processor wirelessly transmits the measurement information according to a predetermined timing
Phosphor filming device operation, structural damage analysis and evaluation system.
According to claim 1,
The processor transmits a rotation control command signal to the rotating member to control the rotation angle of the smart device.
Phosphor filming device operation, structural damage analysis and evaluation system.
According to claim 1,
The processor performs at least one of cracks, efflorescence, exfoliation, damage and exposure of reinforcing bars, damage to steel bridge paint, concrete leakage, and material separation by using an image matching unit that receives image information and performs matching, and a convolutional neural network algorithm based on a mask area. To include a damage detection and quantification unit that detects damage to and quantifies the detected damage, and a documentation unit that prepares an external survey network map and damage quantity table
Phosphor filming device operation, structural damage analysis and evaluation system.
According to claim 10,
The image matching unit extracts feature points using images divided and photographed at a predetermined overlapping rate and implements an image of a structure.
Phosphor filming device operation, structural damage analysis and evaluation system.
According to claim 10,
The damage detection and quantification unit detects damage using a method combining the mask area-based convolutional neural network algorithm and FCN, which perform damage location detection, damage type classification, and pixel-by-pixel detection of damaged areas.
Phosphor filming device operation, structural damage analysis and evaluation system.
According to claim 12,
The damage detection and quantification unit applies weights of neighboring pixels to the damage detected during damage analysis.
Phosphor filming device operation, structural damage analysis and evaluation system.
According to claim 12,
The damage detection and quantification unit acquires a probability map using a Region Proposal Network and determines an object using a preset standard probability.
Phosphor filming device operation, structural damage analysis and evaluation system.
According to claim 14,
The damage detection and quantification unit extracts damage features from candidate boxes through RoIAlign using the region proposal results of the Region Proposal Network and performs classification.
Phosphor filming device operation, structural damage analysis and evaluation system.
According to claim 15,
The damage detection and quantification unit detects the shape of the object by supplying the feature map extracted through the RoIAlign to a mask branch that divides the object of the image in pixel units
Phosphor filming device operation, structural damage analysis and evaluation system.
According to claim 10,
The damage detection and quantification unit obtains the actual damage size for the damage size of the image using the camera pin hole model
Phosphor filming device operation, structural damage analysis and evaluation system.
According to claim 10,
The documentation unit converts the coordinates and range of damage into length, classifies them based on damage type and amount of damage, and performs documentation on damage information.
Phosphor filming device operation, structural damage analysis and evaluation system.
According to claim 10,
The documentation unit generates an external survey network map using pixel coordinates, type, and range.
Phosphor filming device operation, structural damage analysis and evaluation system.
According to claim 19,
The documentation unit determines the perimeter, extracts location information on the target inside the determined perimeter, and performs automatic documentation.
Phosphor filming device operation, structural damage analysis and evaluation system.