CN110473288B - Dam model reconstruction method and device and electronic equipment - Google Patents
Dam model reconstruction method and device and electronic equipment Download PDFInfo
- Publication number
- CN110473288B CN110473288B CN201910782814.8A CN201910782814A CN110473288B CN 110473288 B CN110473288 B CN 110473288B CN 201910782814 A CN201910782814 A CN 201910782814A CN 110473288 B CN110473288 B CN 110473288B
- Authority
- CN
- China
- Prior art keywords
- dam
- semantic segmentation
- binary image
- image data
- reconstructed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/155—Segmentation; Edge detection involving morphological operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention provides a dam model reconstruction method, a dam model reconstruction device and electronic equipment, and relates to the technical field of water conservancy. The dam model reconstruction method comprises the following steps: acquiring a plurality of pieces of initial image data which carry acquisition position information and comprise a dam; processing each piece of initial image data by utilizing a pre-trained semantic segmentation learning model for identifying a dam defect region to obtain a semantic segmentation binary image capable of identifying the dam defect region; fusing each piece of initial image data with the corresponding semantic segmentation binary image to obtain an image to be reconstructed; and performing three-dimensional reconstruction based on the plurality of images to be reconstructed and the corresponding acquisition position information to obtain a dam model marked with the dam defect area. The defect can be accurately positioned, potential safety hazards of the dam can be found in time, and labor cost is saved.
Description
Technical Field
The invention relates to the technical field of water conservancy, in particular to a dam model reconstruction method, a dam model reconstruction device and electronic equipment.
Background
Dykes and dams belong to the hydro-junction infrastructure, and it all has very big influence to the development of livelihood quality guarantee, economy. Along with rapid increase of hydraulic engineering, intelligent routing inspection is carried out on hydro-junction infrastructure, and the trend is mainstream.
The current intelligent inspection is mainly carried out by acquiring dam images, then carrying out three-dimensional model reconstruction based on the acquired dam images, and finally manually inspecting the three-dimensional model of the dam to find the position of the dam with defects. Wherein, the human cost is extremely high and omit some defects easily, makes the potential safety hazard that dykes and dams exist difficult to discover in time.
Disclosure of Invention
In view of the above, the present invention provides a dam model reconstruction method, a dam model reconstruction device, and an electronic apparatus.
In order to achieve the above purpose, the embodiment of the present invention adopts the following technical solutions:
in a first aspect, an embodiment of the present invention provides a dam model reconstruction method, where the dam model reconstruction method includes: acquiring a plurality of pieces of initial image data which carry acquisition position information and comprise a dam; processing each piece of initial image data by utilizing a pre-trained semantic segmentation learning model for identifying a dam defect region to obtain a semantic segmentation binary image capable of identifying the dam defect region; fusing each piece of initial image data and the corresponding semantic segmentation binary image to obtain an image to be reconstructed; and performing three-dimensional reconstruction based on the plurality of images to be reconstructed and the corresponding acquisition position information to obtain a dam model marked with the dam defect area.
In an alternative embodiment, the dam model reconstruction method further includes: performing morphological processing on the obtained semantic segmentation binary image; obtaining quantitative information of the dam defect area based on the processed semantic segmentation binary image; the step of performing three-dimensional reconstruction based on the plurality of images to be reconstructed and the corresponding acquisition position information comprises: and performing three-dimensional reconstruction based on the plurality of images to be reconstructed, the corresponding acquisition position information and the corresponding quantitative information.
In an optional embodiment, the fusing each piece of initial image data with the corresponding semantic segmentation binary image to obtain an image to be reconstructed includes: and respectively fusing each piece of initial image data and the corresponding processed semantic segmentation binary image to obtain the image to be reconstructed.
In an optional embodiment, the step of obtaining the quantitative information of the dam defect area based on the processed semantic segmentation binary map includes: extracting skeleton information of the dam defect region from the processed semantic segmentation binary image by adopting a skeleton algorithm; according to the skeleton information, combining the processed semantic segmentation binary image to calculate image quantitative information of the dam defect area; obtaining the quantitative information by utilizing a preset conversion relation according to the image quantitative information; wherein the conversion relationship is a mapping relationship between a world coordinate system and a camera coordinate system of an acquisition device acquiring the initial image data.
In an alternative embodiment, the morphological treatment comprises: one or a combination of the open operation process and the close operation process.
In an alternative embodiment, the step of processing each of the initial image data using a pre-trained semantic segmentation learning model for identifying a dam defect region includes: reading a pixel point array of the initial image data; dividing the pixel point array into a plurality of sub-arrays to be processed according to the input size of the semantic segmentation learning model; sequentially processing each subarray to be processed by utilizing the semantic segmentation learning model to obtain a plurality of binary image arrays; and obtaining the semantic segmentation binary image according to the plurality of binary image arrays.
In a second aspect, an embodiment of the present invention provides an embankment model reconstruction apparatus, including: the acquisition module is used for acquiring a plurality of pieces of initial image data which carry acquisition position information and contain the dam; the processing module is used for processing each piece of initial image data by utilizing a pre-trained semantic segmentation learning model for identifying the dam defect region to obtain a semantic segmentation binary image capable of identifying the dam defect region; the fusion module is used for respectively fusing each piece of initial image data with the corresponding semantic segmentation binary image to obtain an image to be reconstructed; and the reconstruction module is used for performing three-dimensional reconstruction on the basis of the plurality of images to be reconstructed and the corresponding acquisition position information to obtain a dam model marked with the dam defect region.
In an alternative embodiment, the processing module comprises: a reading sub-module for reading the pixel point array of the initial image data; the division submodule is used for dividing the pixel point array into a plurality of sub arrays to be processed according to the input size of the semantic segmentation learning model; the processing submodule is used for sequentially processing each subarray to be processed by utilizing the semantic segmentation learning model to obtain a plurality of binary image arrays; and the obtaining submodule is used for obtaining the semantic segmentation binary image according to the plurality of binary image arrays.
In a third aspect, an embodiment of the present invention provides an electronic device, including a processor and a memory, where the memory stores machine executable instructions capable of being executed by the processor, and the processor can execute the machine executable instructions to implement the method described in any one of the foregoing embodiments.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the method according to any one of the foregoing embodiments.
According to the dam model reconstruction method provided by the embodiment of the invention, each piece of initial image data is processed by utilizing the semantic segmentation learning model capable of identifying the dam defect region, so that the semantic segmentation binary image capable of identifying the dam defect region is obtained. Namely, the pixel points belonging to the dam defect are respectively identified through the dam pixel level identification. And then respectively fusing each piece of initial image data with the corresponding semantic segmentation binary image to obtain an image to be reconstructed for marking dam defects. And finally, reconstructing a three-dimensional model of the dam according to the image to be reconstructed and the corresponding acquired position information, so that the obtained three-dimensional model can visually and accurately display the defects and the positions of the dam while displaying the dam. The defect can be accurately positioned, potential safety hazards of the dam can be found in time, and labor cost is saved.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 shows an application scenario diagram provided in an embodiment of the present invention.
Fig. 2 shows a schematic diagram of an electronic device provided by an embodiment of the invention.
Fig. 3 is a flowchart illustrating steps of a dam model reconstruction method according to an embodiment of the present invention.
Fig. 4 is a flowchart illustrating sub-steps of step S102 in fig. 3.
Fig. 5 shows another part of the flowchart of the steps of the dam model reconstruction method according to the embodiment of the present invention.
Fig. 6 is a schematic diagram illustrating an embankment model reconstruction apparatus according to an embodiment of the present invention.
Icon: 100-an electronic device; 200-a collection device; 110-a memory; 120-a processor; 130-a communication module; 300-a dam model reconstruction device; 301-an obtaining module; 302-a processing module; 303-a fusion module; 304-a reconstruction module; 305-a morphological processing module; 306-calculation module.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
It is noted that relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Referring to fig. 1, fig. 1 shows an application scenario of the embodiment of the present invention. As shown in fig. 1, the acquisition device 200 is communicatively coupled to the electronic device 100.
The above-mentioned collecting device 200 is a device capable of synchronously positioning the position information of itself when collecting images. Optionally, the above-mentioned collecting device 200 may be a drone, and the drone may determine its own location information. When the unmanned aerial vehicle carries out image acquisition, the position information can be synchronously acquired. The collected position information may be GPS information. The position of the unmanned aerial vehicle in a world coordinate system during image acquisition can be determined according to the acquisition position information. Of course, in some other embodiments, the capturing device 200 may be another capturing device 200 capable of being positioned, such as a camera, a video camera, a mobile phone, and the like.
Fig. 2 is a block diagram of the electronic device 100. The electronic device 100 includes a memory 110, a processor 120, and a communication module 130. The memory 110, the processor 120 and the communication module 130 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, the components may be electrically connected to each other via one or more communication buses or signal lines.
The memory 110 is used to store programs or data. The Memory 110 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like.
The processor 120 is used to read/write data or programs stored in the memory 110 and perform corresponding functions.
The communication module 130 is configured to establish a communication connection between the electronic device 100 and another communication terminal through the network, and to transmit and receive data through the network.
It should be understood that the structure shown in fig. 2 is only a schematic structural diagram of the electronic device 100, and the electronic device 100 may also include more or fewer components than shown in fig. 2, or have a different configuration than shown in fig. 2. The components shown in fig. 2 may be implemented in hardware, software, or a combination thereof.
First embodiment
Referring to fig. 3, fig. 3 illustrates a dam model reconstruction method according to an embodiment of the present invention. The above dam model reconstruction method is applied to the electronic apparatus 100. As shown in fig. 3, the dam model reconstruction method may include the steps of:
step S101, acquiring a plurality of pieces of initial image data which carry acquisition position information and contain dams.
In the embodiment of the invention, initial image data of multiple angles of the dam acquired by the unmanned aerial vehicle from multiple positions is received. It should be noted that, when the unmanned aerial vehicle acquires one piece of image data including a dam, the GPS information of the unmanned aerial vehicle at this time is written in the image attribute of the acquired image data as the acquisition position information, so as to obtain the initial image data.
Step S102, processing each piece of initial image data by utilizing a pre-trained semantic segmentation learning model for identifying the dam defect region to obtain a semantic segmentation binary image capable of identifying the dam defect region.
The semantic segmentation learning model for identifying the dam defect region can be obtained by training in advance. It should be noted that the semantic segmentation learning model can be obtained based on full Convolutional neural Network (FCN) training. FCN is an end-to-end Convolutional Neural Network (CNN) based on semantic segmentation. Specifically, the FCN is obtained by replacing the fully-connected classification layer of the CNN with the deconvolution layer, which can realize pixel-level detection, i.e., can realize identification of each pixel point. That is, the FCN can also be viewed as an extended CNN that can convert the predicted object into a semantically segmented image, i.e., dense prediction. The FCN mainly comprises a down-sampling part and an up-sampling part, wherein the down-sampling part comprises a convolution layer, a pooling layer and a dropout layer, and the up-sampling layer mainly comprises a deconvolution layer. The down-sampling part is mainly a very deep convolution network, and the training time can be obviously shortened by using a deep neural network with better detection effect instead of the deep neural network, such as VGG16, VGG19, GoogleNet, ResNet and the like. VGG19 networks are commonly used for object recognition and classification of large-scale images, while numerous applications in the industry have demonstrated the effectiveness of VGG19 networks. Therefore, the invention adopts the VGG19 network as a down-sampling part to realize the feature extraction of the defect data. The up-sampling process is opposite to the down-sampling process, has stronger prediction capability and can generate a class prediction for each pixel. The up-sampling process not only can realize the accurate positioning of the target, but also can ensure that the output size of the whole network is equal to the input size, and realize the end-to-end target segmentation.
Further, the step of training the selected FCNs into a semantic segmentation learning model capable of identifying dam defect regions comprises: an image with a dam is acquired. The image is cropped into a plurality of sample images having a size that matches the input size of the FCN. The dam defect region (e.g., dam crack region) appearing in the sample image is labeled. And training the FCN by using the labeled sample image so as to obtain a semantic segmentation learning model for identifying the dam defect area.
And processing the initial image data through the semantic segmentation learning model to obtain a semantic segmentation binary image. The semantic segmentation binary image assigns a first value to each pixel point in an image area of the dam defect presented in the initial image data and assigns second values to pixel points of other image areas.
In an embodiment of the present invention, as shown in fig. 4, the step of processing each piece of initial image data by using the pre-trained semantic segmentation learning model for identifying the dam defect region includes:
in the sub-step S1021, the pixel dot array of the initial image data is read.
The size of the pixel point array is the same as that of the initial image data, and each element in the pixel point array corresponds to one pixel point in the initial image data. The value of each element is the pixel value of the corresponding pixel point. For example, initial image data of 5472 × 3648 is read, resulting in an 5472 × 3648 pixel dot array. The element located at (1,1) in the pixel array is the pixel value of the pixel of (1,1) in the initial image data.
In the sub-step S1022, the pixel point array is divided into a plurality of to-be-processed sub-arrays according to the input size of the semantic segmentation learning model.
It can be understood that there is a limit to the size of the image that can be processed by the semantic segmentation learning model at a time. In order to meet the input size requirement of the semantic segmentation learning model, a mode of cutting an input image is adopted in the related technology. However, cropping the input image is time consuming and also consumes system resources.
In the embodiment of the invention, the pixel point array is divided into a plurality of to-be-processed sub-arrays which accord with the input size of the semantic segmentation learning model, and the to-be-processed sub-arrays obtained after division are sequentially input into the semantic segmentation learning model for recognition, so that the initial image does not need to be actually cut in the whole process. By adopting the method for dividing the pixel point array, the requirement of the semantic segmentation learning model on the size of input data is met, the process of cutting initial image data is avoided, and the effect of saving time is achieved.
In the above example, the input size of the semantic segmentation learning model is 912 × 608, and then the 5472 × 3648 pixel point array is divided into 36 to-be-processed sub-arrays of 912 × 608.
And a substep S1023, sequentially processing each subarray to be processed by using a semantic segmentation learning model to obtain a plurality of binary image arrays.
In the embodiment of the invention, a semantic segmentation learning model is used for identifying the pixel points corresponding to each subarray to be processed, the pixel points belonging to the dam defect area are distinguished, and the corresponding binary image array is obtained based on the distinguishing result.
And a substep S1024 of obtaining a semantic segmentation binary image according to the plurality of binary image arrays.
In the embodiment of the invention, the obtained multiple binary image arrays are integrated to obtain the semantic segmentation binary image.
Furthermore, the semantic segmentation binary image obtained by the semantic segmentation learning model has the problems of discontinuous defect information, noise and the like. In order to improve the problems, the integrity of the obtained dam defect area and the reliability of detection are improved. In some embodiments, on the basis of the dam model reconstruction method shown in fig. 3, as shown in fig. 5, the dam model reconstruction method may further include the steps of:
step S201, morphological processing is performed on the obtained semantic segmentation binary image.
In the embodiment of the present invention, the morphological processing may be one of or a combination of an open operation processing and a close operation processing. The above-mentioned opening operation processing is to perform corrosion processing on the semantic segmentation binary image and then perform expansion processing. Noise can be removed by the on-operation process. The closed operation processing is to perform expansion processing on the semantic segmentation binary image and then perform corrosion processing. The identified dam defect regions can be made continuous by the closing operation process. In some embodiments, if it is determined that the semantic segmentation binary image is noisy, then an on operation process is employed. In some embodiments, if it is determined that there is no noise in the semantically segmented binary image, a closed-operation process is employed. Of course, in some embodiments, both the open operation process and the close operation process may also be enabled.
The etching treatment may be: and defining structural elements, and sequentially moving the original points of the defined structural elements to each pixel point of the semantic segmentation binary image. The corrosion result after each movement is determined by the minimum value on the image in the overlapping area of the semantic segmentation binary image and the structural element. Small and meaningless objects can be eliminated by the etching operation, and the method is suitable for noise reduction processing, and the expression is as follows:
wherein, f represents a semantic segmentation binary image, b represents a structural element, and (x, y) represents the coordinate position of a pixel point in the semantic segmentation binary image.
The expansion treatment may be: and defining structural elements, and sequentially moving the original points of the defined structural elements to each pixel point of the semantic segmentation binary image. The expansion result after each movement is determined by the maximum value on the image in the overlapping area of the semantic segmentation binary image and the structural element. The expansion operation can be used for filling some holes in the target area and eliminating the particle eliminating noise contained in the target area, and the expression is as follows:
wherein, f represents a semantic segmentation binary image, b represents a structural element, and (x, y) represents the coordinate position of a pixel point in the semantic segmentation binary image.
Step S202, based on the processed semantic segmentation binary image, obtaining the quantitative information of the dam defect area.
In an embodiment of the present invention, the quantitative information may be size information of the dam defect area. For example, the quantitative information includes morphological characteristics such as length, area, width, and the like.
Alternatively, a skeleton algorithm can be adopted to extract skeleton information of the dam defect region from the processed semantic segmentation binary image. The extracted skeleton information of the dam defect region may include a skeleton curve, which has a single pixel point as a width and substantially coincides with a center line of the dam defect region. The skeleton curve can outline the extending direction of the dam defect area.
Then, according to the skeleton information, combining the processed semantic segmentation binary image to calculate the image quantitative information of the dam defect area. The image quantitative information may be morphological characteristics such as a length, an area, and a width of the dam defect region in the image. Optionally, according to the skeleton information, using a formula:
and calculating the length information of the dam defect area in the image. Where f (x, y) represents a geometric calibration index, which may be determined based on the distortion of the image, and f (x, y) is 1 in the case of an image without geometric distortion. dl represents the length infinitesimal of the skeleton information, and C represents the skeleton curve corresponding to the skeleton information.
It can be understood that the skeleton extraction is performed on the dam defect region by using the central axis transformation algorithm, and the obtained skeleton information includes the distance between the central axis and the background of the dam defect region in addition to the skeleton curve. Based on this, according to the skeleton information, the formula is used:
the area of the dam defect region in the image is obtained. Wherein f (x, y) represents a geometric calibration index, dS represents an area infinitesimal of skeleton information, and S represents a curved surface corresponding to the dam defect region processed by the medial axis transformation algorithm.
According to the skeleton information, using the formula:
the width of the dam defect area in the image is obtained.
And finally, obtaining quantitative information by utilizing a conversion relation according to the image quantitative information. The above-described conversion relationship is a mapping relationship between the world coordinate system and the camera coordinate system of the capturing device 200 that captures the initial image data. It will be appreciated that the same acquisition device 200, the same object distance and the same focal length, when used to photograph an object of known dimensions, will allow a correspondence between the world coordinate system (i.e. the actual dimensions) and the camera coordinate system (i.e. the pixel dimensions). Therefore, the conversion relation can be obtained through testing in advance and stored for use.
And S103, fusing each piece of initial image data and the corresponding semantic segmentation binary image respectively to obtain an image to be reconstructed.
Understandably, each piece of initial image data corresponds to a semantic segmentation binary image through the processing of the semantic segmentation learning model. The semantic segmentation binary image distinguishes pixel points belonging to a dam defect region from pixel points not belonging to the dam defect region in the initial image data by using the first value and the second value.
In the embodiment of the invention, each piece of initial image data and the corresponding semantic segmentation binary image are fused to obtain the image to be reconstructed. Compared with the initial image data, the image to be reconstructed marks the dam defect area existing in the initial image data. At the same time, the acquisition location information in the image attributes is also lost. Therefore, in some embodiments, before each piece of initial image data is fused with the semantic segmentation binary image, the acquisition position information of the initial image data is extracted and written into the designated storage area. And after the initial image data is fused with the corresponding semantic segmentation binary image, the acquisition position information of the initial image data is taken out from the specified storage area and written into the image attribute of the image to be reconstructed.
In some embodiments, for example, in the dam model reconstruction method corresponding to fig. 5, the above respectively fusing each piece of initial image data and the corresponding semantic segmentation binary image to obtain the image to be reconstructed may be: and respectively fusing each piece of initial image data and the corresponding processed semantic segmentation binary image to obtain an image to be reconstructed. And after the image to be reconstructed is obtained, writing the quantitative information of the dam defect area of the image to be reconstructed into the image attribute of the image to be reconstructed.
And step S104, performing three-dimensional reconstruction based on the plurality of images to be reconstructed and the corresponding acquisition position information to obtain a dam model marked with a dam defect area.
In the embodiment of the invention, each image to be reconstructed corresponds to acquisition position information. According to the acquisition position information of the images to be reconstructed, the regions at the same positions on the dam can be determined in the images to be reconstructed, and three-dimensional modeling is carried out on the basis of the regions to be reconstructed to obtain a dam model. The dam defect area can be accurately identified on the three-dimensional reconstructed dam model, and related personnel can monitor the dam defect area.
In some embodiments, after the dam model is reconstructed, the quantified information of each dam defect area on the dam model can be automatically marked and corresponding to the corresponding position, so that the relevant personnel can conveniently learn the information. Therefore, the dam defect area can be accurately positioned and determined.
In order to perform the corresponding steps in the above-described embodiments and various possible manners, an implementation manner of the dam model reconstruction apparatus 300 is given below, and optionally, the dam model reconstruction apparatus 300 may adopt the device structure of the electronic device 100 shown in fig. 2. Further, referring to fig. 6, fig. 6 is a functional block diagram of an embankment model reconstruction apparatus 300 according to an embodiment of the present invention. It should be noted that the basic principle and the generated technical effect of the dam model reconstruction apparatus 300 provided in the present embodiment are the same as those of the above embodiments, and for the sake of brief description, no part of the present embodiment is mentioned, and reference may be made to the corresponding contents in the above embodiments. The embankment model reconstruction device 300 includes: an acquisition module 301, a processing module 302, a fusion module 303, and a reconstruction module 304.
The acquiring module 301 is configured to acquire a plurality of pieces of initial image data that carry the collecting position information and include the dam.
In the embodiment of the present invention, the step S101 may be executed by the obtaining module 301.
The processing module 302 is configured to process each piece of the initial image data by using a pre-trained semantic segmentation learning model for identifying a dam defect region, so as to obtain a semantic segmentation binary image capable of identifying the dam defect region.
In the embodiment of the present invention, the step S102 may be executed by the processing module 302. Preferably, the processing module 302 includes: reading the sub-modules, dividing the sub-modules, processing the sub-modules and obtaining the sub-modules.
And the reading sub-module is used for reading the pixel point array of the initial image data.
In an embodiment of the present invention, the sub-step S1021 may be performed by a reading sub-module.
And the division submodule is used for dividing the pixel point array into a plurality of sub arrays to be processed according to the input size of the semantic segmentation learning model.
In an embodiment of the present invention, the sub-step S1022 described above may be performed by a dividing sub-module.
And the processing sub-module is used for sequentially processing each sub-array to be processed by utilizing the semantic segmentation learning model so as to obtain a plurality of binary image arrays.
In an embodiment of the present invention, the sub-step S1023 may be executed by a processing sub-module.
And the obtaining submodule is used for obtaining the semantic segmentation binary image according to the plurality of binary image arrays.
In an embodiment of the present invention, the sub-step S1024 may be performed by the obtaining sub-module.
The fusion module 303 is configured to fuse each piece of the initial image data with the corresponding semantic segmentation binary image to obtain an image to be reconstructed.
In an embodiment of the present invention, the step S103 may be executed by the fusion module 303.
The reconstruction module 304 is configured to perform three-dimensional reconstruction based on the plurality of images to be reconstructed and the corresponding acquisition position information, so as to obtain a dam model in which the dam defect region is marked.
In an embodiment of the present invention, the step S104 may be performed by the reconstruction module 304.
Further, in some embodiments, the dam model reconstruction apparatus 300 further includes: a morphology processing module 305 and a calculation module 306.
The morphological processing module 305 is configured to perform morphological processing on the obtained semantic segmentation binary image.
In an embodiment of the present invention, the step S201 may be executed by the morphology processing module 305.
The calculating module 306 is configured to obtain quantitative information of the dam defect area based on the processed semantic segmentation binary image.
In the embodiment of the present invention, the step S202 may be executed by the calculating module 306.
Alternatively, the modules may be stored in the memory 110 shown in fig. 2 in the form of software or Firmware (Firmware) or be fixed in an Operating System (OS) of the electronic device 100, and may be executed by the processor 120 in fig. 2. Meanwhile, data, codes of programs, and the like required to execute the above-described modules may be stored in the memory 110.
In summary, the embodiment of the invention provides a dam model reconstruction method, a dam model reconstruction device and electronic equipment. The dam model reconstruction method comprises the following steps: acquiring a plurality of pieces of initial image data which carry acquisition position information and contain dams; processing each piece of initial image data by utilizing a pre-trained semantic segmentation learning model for identifying a dam defect region to obtain a semantic segmentation binary image capable of identifying the dam defect region; and the obtained semantic segmentation binary image is subjected to morphological processing, so that the integrity and reliability of the identified dam defect area are improved. And respectively fusing each piece of initial image data and the corresponding semantic segmentation binary image to obtain an image to be reconstructed. And extracting the quantitative information of the dam defect area from the processed semantic segmentation binary image. And finally, performing three-dimensional reconstruction based on the plurality of images to be reconstructed, the quantification information and the corresponding acquisition position information to obtain a dam model marking the dam defect area. The defect can be accurately positioned, potential safety hazards of the dam can be found in time, and labor cost is saved.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, the functional modules in the embodiments of the present invention may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (8)
1. A dam model reconstruction method, characterized by comprising:
acquiring a plurality of pieces of initial image data which carry acquisition position information and comprise a dam;
processing each piece of initial image data by utilizing a pre-trained semantic segmentation learning model for identifying a dam defect region to obtain a semantic segmentation binary image capable of identifying the dam defect region;
fusing each piece of initial image data with the corresponding semantic segmentation binary image to obtain an image to be reconstructed;
performing three-dimensional reconstruction based on the plurality of images to be reconstructed and the corresponding acquisition position information to obtain a dam model marked with the dam defect area;
the dam model reconstruction method further includes:
performing morphological processing on the obtained semantic segmentation binary image;
extracting skeleton information of the dam defect region from the processed semantic segmentation binary image by adopting a skeleton algorithm;
according to the skeleton information, combining the processed semantic segmentation binary image to calculate image quantitative information of the dam defect region;
obtaining quantitative information by utilizing a preset conversion relation according to the image quantitative information; wherein the conversion relation is a mapping relation between a world coordinate system and a camera coordinate system of an acquisition device acquiring the initial image data;
the step of performing three-dimensional reconstruction based on the plurality of images to be reconstructed and the corresponding acquisition position information comprises: and performing three-dimensional reconstruction based on the plurality of images to be reconstructed, the corresponding acquisition position information and the corresponding quantitative information.
2. A dam model reconstruction method according to claim 1, wherein said fusing each of said initial image data and said corresponding semantic segmentation binary image to obtain an image to be reconstructed comprises:
and respectively fusing each piece of initial image data and the corresponding processed semantic segmentation binary image to obtain the image to be reconstructed.
3. A dam model reconstruction method according to claim 1, wherein said morphological processing comprises: one or a combination of both of the open operation processing and the close operation processing.
4. A dam model reconstruction method according to claim 1, wherein the step of processing each of the initial image data using a pre-trained semantic segmentation learning model for identifying dam defect regions comprises:
reading a pixel point array of the initial image data;
dividing the pixel point array into a plurality of sub-arrays to be processed according to the input size of the semantic segmentation learning model;
sequentially processing each subarray to be processed by utilizing the semantic segmentation learning model to obtain a plurality of binary image arrays;
and obtaining the semantic segmentation binary image according to the plurality of binary image arrays.
5. A embankment-modeling reconstruction apparatus, comprising:
the acquisition module is used for acquiring a plurality of pieces of initial image data which carry acquisition position information and contain the dam;
the processing module is used for processing each piece of initial image data by utilizing a pre-trained semantic segmentation learning model for identifying the dam defect region to obtain a semantic segmentation binary image capable of identifying the dam defect region;
the fusion module is used for respectively fusing each piece of initial image data with the corresponding semantic segmentation binary image to obtain an image to be reconstructed;
the reconstruction module is used for performing three-dimensional reconstruction based on the images to be reconstructed and the corresponding acquisition position information to obtain a dam model marked with the dam defect area;
the reconstruction module is further to: performing morphological processing on the obtained semantic segmentation binary image; extracting skeleton information of the dam defect region from the processed semantic segmentation binary image by adopting a skeleton algorithm; according to the skeleton information, combining the processed semantic segmentation binary image to calculate image quantitative information of the dam defect area; obtaining quantitative information by utilizing a preset conversion relation according to the image quantitative information; wherein the conversion relation is a mapping relation between a world coordinate system and a camera coordinate system of an acquisition device acquiring the initial image data; the step of performing three-dimensional reconstruction based on the plurality of images to be reconstructed and the corresponding acquisition position information comprises: and performing three-dimensional reconstruction based on the plurality of images to be reconstructed, the corresponding acquisition position information and the corresponding quantitative information.
6. An embankment model reconstruction device according to claim 5, wherein said processing module comprises:
a reading sub-module for reading the pixel point array of the initial image data;
the division submodule is used for dividing the pixel point array into a plurality of sub arrays to be processed according to the input size of the semantic segmentation learning model;
the processing submodule is used for sequentially processing each subarray to be processed by utilizing the semantic segmentation learning model to obtain a plurality of binary image arrays;
and the obtaining submodule is used for obtaining the semantic segmentation binary image according to the plurality of binary image arrays.
7. An electronic device comprising a processor and a memory, the memory storing machine executable instructions executable by the processor to perform the method of any one of claims 1 to 4.
8. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910782814.8A CN110473288B (en) | 2019-08-23 | 2019-08-23 | Dam model reconstruction method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910782814.8A CN110473288B (en) | 2019-08-23 | 2019-08-23 | Dam model reconstruction method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110473288A CN110473288A (en) | 2019-11-19 |
| CN110473288B true CN110473288B (en) | 2022-08-05 |
Family
ID=68512104
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910782814.8A Active CN110473288B (en) | 2019-08-23 | 2019-08-23 | Dam model reconstruction method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110473288B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112288741A (en) * | 2020-11-23 | 2021-01-29 | 四川长虹电器股份有限公司 | Product surface defect detection method and system based on semantic segmentation |
| CN118425167B (en) * | 2024-07-02 | 2024-10-15 | 昂坤视觉(北京)科技有限公司 | Defect detection method and system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103413288A (en) * | 2013-08-27 | 2013-11-27 | 南京大学 | LCD general defect detecting method |
| JP2016015638A (en) * | 2014-07-02 | 2016-01-28 | 株式会社デンソー | Fault detection device and fault detection program |
| CN106709568A (en) * | 2016-12-16 | 2017-05-24 | 北京工业大学 | RGB-D image object detection and semantic segmentation method based on deep convolution network |
| CN108876790A (en) * | 2017-09-14 | 2018-11-23 | 北京旷视科技有限公司 | Image, semantic dividing method and device, neural network training method and device |
| CN109884067A (en) * | 2019-01-26 | 2019-06-14 | 北京中水科海利工程技术有限公司 | Dam face slab defect precise positioning and visual processing method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019040288A1 (en) * | 2017-08-25 | 2019-02-28 | Exxonmobil Upstream Researchcompany | Automated seismic interpretation using fully convolutional neural networks |
-
2019
- 2019-08-23 CN CN201910782814.8A patent/CN110473288B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103413288A (en) * | 2013-08-27 | 2013-11-27 | 南京大学 | LCD general defect detecting method |
| JP2016015638A (en) * | 2014-07-02 | 2016-01-28 | 株式会社デンソー | Fault detection device and fault detection program |
| CN106709568A (en) * | 2016-12-16 | 2017-05-24 | 北京工业大学 | RGB-D image object detection and semantic segmentation method based on deep convolution network |
| CN108876790A (en) * | 2017-09-14 | 2018-11-23 | 北京旷视科技有限公司 | Image, semantic dividing method and device, neural network training method and device |
| CN109884067A (en) * | 2019-01-26 | 2019-06-14 | 北京中水科海利工程技术有限公司 | Dam face slab defect precise positioning and visual processing method |
Non-Patent Citations (5)
| Title |
|---|
| Autonomous concrete crack detection using deep fully convolutional neural network;Gao Vu Dung 等;《Automation in Construction》;20190331;第99卷;第52-58页 * |
| Deep Learning-Based Crack Damage Detection Using Convolutional Neural Networks;Young Jin Cha 等;《Computer Aided Civil and Infrastructure Engineering》;20170501;第1-18页 * |
| Recognition, location, measurement, and 3D reconstruction of concealed cracks using convolutional neural networks;Zheng Tong 等;《Construction and Building Materials》;20170430;第146卷;第775-787页 * |
| 基于深度学习的全卷积网络图像裂纹检测;王森 等;《计算机辅助设计与图形学学报》;20180515;第30卷(第5期);第859-866页 * |
| 模块图像的三维重建与缺陷检测;王朝琴;《中国优秀硕士学位论文全文数据库 信息科技辑》;20090315(第03期);第I138-525页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110473288A (en) | 2019-11-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6902611B2 (en) | Object detection methods, neural network training methods, equipment and electronics | |
| CN106530320B (en) | End-to-end image segmentation processing method and system | |
| CN111652218A (en) | Text detection method, electronic device and computer readable medium | |
| WO2017190743A1 (en) | Model construction in a neural network for object detection | |
| CN111145209A (en) | Medical image segmentation method, device, equipment and storage medium | |
| CN110346808B (en) | Point cloud data processing method and system of laser radar | |
| CN110473288B (en) | Dam model reconstruction method and device and electronic equipment | |
| CN111415364B (en) | Conversion method, system and storage medium for image segmentation sample in computer vision | |
| CN111523439B (en) | Method, system, device and medium for target detection based on deep learning | |
| CN113505781B (en) | Target detection method, target detection device, electronic equipment and readable storage medium | |
| CN115797350B (en) | Bridge disease detection method, device, computer equipment and storage medium | |
| CN110942456B (en) | Tamper image detection method, device, equipment and storage medium | |
| CN112329702A (en) | Method and device for rapid face density prediction and face detection, electronic equipment and storage medium | |
| CN111310758A (en) | Text detection method and device, computer equipment and storage medium | |
| CN113239818A (en) | Cross-modal information extraction method of tabular image based on segmentation and graph convolution neural network | |
| CN106845347A (en) | Marine oil film recognizes extracting method and system | |
| CN112507876A (en) | Wired table picture analysis method and device based on semantic segmentation | |
| WO2021240327A1 (en) | Modelling georeferenced data for civil infrastructure asset management | |
| CN110991414A (en) | High-precision traffic element segmentation method, electronic equipment and storage medium | |
| CN117237648B (en) | Training method, device and equipment of semantic segmentation model based on context awareness | |
| CN115115740A (en) | Thinking guide graph recognition method, device, equipment, medium and program product | |
| CN116385889B (en) | Railway identification-based power inspection method and device and electronic equipment | |
| CN116206326A (en) | Training method of missing detection model, missing detection method and device of diversion area | |
| CN113160217B (en) | Method, device, equipment and storage medium for detecting circuit foreign matters | |
| CN114022458A (en) | Skeleton detection method and device, electronic equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |