CN116087036B - Device for identifying images of sediment plume of deep sea mining and image analysis method - Google Patents

Device for identifying images of sediment plume of deep sea mining and image analysis method Download PDF

Info

Publication number
CN116087036B
CN116087036B CN202310108945.4A CN202310108945A CN116087036B CN 116087036 B CN116087036 B CN 116087036B CN 202310108945 A CN202310108945 A CN 202310108945A CN 116087036 B CN116087036 B CN 116087036B
Authority
CN
China
Prior art keywords
image
sediment
deep sea
plume
frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310108945.4A
Other languages
Chinese (zh)
Other versions
CN116087036A (en
Inventor
贾永刚
郭煦
范智涵
王宏威
陈翔
田兆阳
卢龙玉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ocean University of China
Original Assignee
Ocean University of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ocean University of China filed Critical Ocean University of China
Priority to CN202310108945.4A priority Critical patent/CN116087036B/en
Publication of CN116087036A publication Critical patent/CN116087036A/en
Application granted granted Critical
Publication of CN116087036B publication Critical patent/CN116087036B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N13/00Investigating surface or boundary effects, e.g. wetting power; Investigating diffusion effects; Analysing materials by determining surface, boundary, or diffusion effects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/40Image enhancement or restoration using histogram techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N13/00Investigating surface or boundary effects, e.g. wetting power; Investigating diffusion effects; Analysing materials by determining surface, boundary, or diffusion effects
    • G01N2013/003Diffusion; diffusivity between liquids
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10016Video; Image sequence

Landscapes

  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Biochemistry (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Pathology (AREA)
  • Analytical Chemistry (AREA)
  • Chemical & Material Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Immunology (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Image Processing (AREA)

Abstract

The application provides a device for identifying sediment plume images in deep sea mining and an image analysis method, wherein the sediment plume three-dimensional observation device mainly comprises a frame structure, deep sea camera equipment and an auxiliary scale, and can record the morphological change rule of sediment plumes with high precision. The U-net deep learning model is used for segmenting the sediment plume images, so that the morphology of the sediment plumes can be effectively extracted. By the technical scheme, the morphology of the sediment plumes can be recorded from different angles at the deep sea boundary layer, the three-dimensional diffusion morphological characteristics of the sediment plumes can be directly reflected, and the boundary of the sediment plumes is extracted from the image by a deep learning method, so that the morphology change rule of migration diffusion of the sediment plumes is analyzed.

Description

Device for identifying images of sediment plume of deep sea mining and image analysis method
Technical Field
The application relates to the technical field of analysis of submarine sediments, in particular to a device for identifying images of sediment feathers in deep sea mining and an image analysis method.
Background
A sediment plume is a dynamically changing process in which sediment is disturbed, causing it to spread in a body of water. The application aims at disturbing sediment due to mining activities of human beings in the submarine manganese nodule mining process, so that the sediment is suspended and is in migration diffusion, and the suspension and migration diffusion processes of the sediment are sediment plumes.
The sediment is resuspended and transferred and spread on the seabed, so that the concentration of suspended particles of a seabed boundary layer is increased, the chemical property of a water body is changed, the normal life activities of submarine organisms are influenced, the original ecological environment of a seabed area is damaged, and the submarine mining activities are restrained. Therefore, the plume generated by sediment re-suspension in the submarine mining process is necessarily observed and identified, the migration and diffusion rule of the sediment plume is known, the influence of the sediment plume is evaluated, and precious advice is provided for reducing the influence of deep-sea manganese nodule mining on the environment.
Because the deep sea polymetallic nodule mining area is located in a deep sea ocean basin of 4000-6000 meters, the observation and research difficulty is extremely high, and the current research can only be carried out in a numerical simulation mode on the migration and diffusion rule of sediment plumes, but the current numerical simulation research on the migration distribution and diffusion characteristics of plumes is seriously lack of support and inspection of on-site monitoring data, so that the accuracy and reliability of a plumes value simulation model are insufficient. In the field observation research, most researches only use an acoustic instrument to measure the concentration profile of the sediment plume, but concentration data measured based on a turbidity probe still have defects, so that the real situation of migration and diffusion of the sediment plume is difficult to describe, and the morphological characteristics of the sediment plume are difficult to intuitively show, so that the application adopts an observation method of underwater image shooting to analyze the morphological characteristics of the migration and diffusion of the sediment plume, and research the migration and diffusion rule of the sediment plume.
Disclosure of Invention
In order to make up for the defects of the prior art, the application provides an observation device for three-dimensionally observing and recording the shape characteristics of sediment plumes in a deep sea boundary layer and an image analysis method for deep learning for analyzing the migration and diffusion rules of the sediment plumes, so as to overcome the defects of the prior art.
Aiming at the defects of the prior art in sediment plume research, one of the application is to design an imaging device for recording sediment plume migration and diffusion, which can record the morphological characteristics of sediment plume three-dimensional migration and diffusion in multiple angles and can quickly and automatically identify the scale morphology of the sediment plume.
The application is realized by the following technical scheme: the device for identifying the plume images of the deep sea mining sediments comprises an orthocube frame, wherein the four corners of the bottom of the orthocube frame are respectively and fixedly provided with a counterweight, 6 observation scales are fixedly arranged on the frame of the orthocube frame, each 2 observation scales are respectively fixed on the rear face, the left side face and the bottom face of the orthocube frame structure, the observation scales are fixed on the left side face and the lower side frame of the face, the observation scales are fixed on the right side frame and the lower side frame of the face, the observation scales are fixed on the upper side frame and the left side frame of the face, and the observation scales are fixed on the bottom face of the orthocube frame;
the front face, the right side face and the top face of the regular cube frame are respectively and fixedly provided with a marine camera equipment fixing bracket, the marine camera equipment fixing bracket comprises a steel pipe and a stainless steel hoop arranged in the middle of the steel pipe, the steel pipe is welded in the middle of the three face frames, the stainless steel hoop is positioned in the center of the three faces, and marine submarine camera equipment is fixedly arranged on the three stainless steel hoops;
the ocean seabed camera equipment comprises a data acquisition storage pressure-resistant bin, a power supply unit and a data acquisition unit are arranged in the data acquisition storage pressure-resistant bin, a pressure-resistant bin lower end sealing cover is arranged at the lower end of the data acquisition storage pressure-resistant bin, ocean camera auxiliary lighting equipment and an ocean high-resolution camera lens are vertically downwards arranged on the lower surface of the pressure-resistant bin lower end sealing cover, the ocean camera auxiliary lighting equipment and the ocean high-resolution camera lens are respectively surrounded by a stainless steel pressure-resistant bin shell and are electrically connected with the power supply unit and the data acquisition unit in the data acquisition storage pressure-resistant bin, the data acquisition unit comprises a relevant data acquisition plate and data acquisition software, and the ocean camera auxiliary lighting equipment and the ocean high-resolution camera lens of the 3 ocean seabed camera equipment face the inside of an cube frame at vertical angles relative to the surface.
Preferably, the regular cube frame is formed by welding a stainless steel pipe made of 316L material with the length of 1 meter.
Further, the material of observation scale is the scale of stainless steel material, and the length of observation scale is 1 meter, and measurement accuracy is millimeter.
Preferably, the counterweight is a cake-shaped counterweight, and the diameter of the counterweight is 20 cm.
As a preferable scheme, the resolution of the ocean high-resolution imaging lens is 10801920, marine camera auxiliary lighting equipment is the LED light source constitution, and totally three, and three marine camera auxiliary lighting equipment all use marine high-resolution camera lens as central equiangular ring its setting, and power supply unit is high-performance lithium cell, can satisfy and observe at the seabed for a long time.
The application discloses an image analysis method applying a deep sea mining sediment plume image recognition device, which adopts a U-net model to extract characteristics of the deep sea sediment plume image, mainly processes and analyzes the photographed plume image through a deep sea sediment plume three-dimensional observation device, and the flow diagram is shown in fig. 5, and specifically comprises the following steps:
step S1, processing an original sediment plume image video: decomposing the three-dimensional observation video of the acquired submarine sediment plumes through a frame-by-frame decomposition program to obtain frame images of the sediment plumes; a deep sea sediment plume image photographed at the sea floor is 24 frames per second;
step S2, image enhancement and restoration of deep sea sediment plume images: adopting a contrast limited self-adaptive histogram equalization algorithm to limit noise amplification and local contrast enhancement by limiting the height of a local histogram; dividing an image into a plurality of subareas, classifying the histogram of each subarea, respectively carrying out histogram equalization on each subarea, and finally carrying out interpolation operation on each pixel to obtain a converted gray value, thereby realizing contrast limited self-adaptive histogram equalization image enhancement;
step S3, bit depth change of the deep sea sediment plume images: the video image shot by the three-dimensional observation device of the submarine sediment plume in the step S2 is a true color image, the bit depth is 24, and the colors of 24 powers of 2 can be combined; the image is changed to an image with bit depth of 8 and contains 256 colors;
step S4, graying the deep sea sediment plume image: gray processing is carried out on the deep sea sediment plume images, wherein pixels with large gray values are brighter: the maximum pixel value is 255, and the pixel value is white; whereas it is darker: the minimum pixel is 0 and is black; the image graying is realized through a cv2.Color_bgr2gray function in an OpenCV function package of python language:
GRAY = B * 0.114 + G *0.587 + R * 0.299;
step S5, establishing a deep sea sediment plume training data set: marking a plume range in a deep sea sediment plume image by using image marking software LabelMe of a graphical interface, and establishing a training set with a deep sea sediment plume label for image training;
step S6, training image expansion is carried out: increasing the number of training samples, generating a new image using a stochastic transformation method, comprising: randomly rotating the angle, horizontally traversing a certain distance, vertically traversing a certain distance, randomly zooming a certain range, horizontally overturning and vertically overturning;
s7, image segmentation is carried out by utilizing a U-net model: the U-net model consists of a compression part and an expansion part 2; the compression part is formed by 2 convolution kernels to 33 (Conv 2D, activation function is relu), the convolution kernel is 2 +.>2 (maxpoolinglayer) to form a downsampling module, wherein the total number of the downsampling modules is 3; the extension part consists of an up-sampled convolution layer (UpSampling 2D), a feature splicing layer splicing, and 3 convolution kernels of 3 +.>3 (Conv 2D, activation function is relu) to form an up-sampling module, and 4 up-sampling modules are all arranged;
step S8, threshold segmentation: according to the segmentation result of the image segmentation model, setting a threshold value as 174, carrying out threshold segmentation on the segmented image, dividing the image pixel points into target areas and background areas with different gray scales, and displaying the distribution range of deep sea sediment plumes;
and S9, extracting the shape characteristics of the deep sea sediment plumes, and determining the diffusion range of the sediment plumes, so as to analyze the migration and diffusion rules of the deep sea mining sediment plumes.
As a preferred solution, the contrast limited adaptive histogram equalization algorithm in step S2 specifically includes the following steps:
s21, dividing the photographed sea sediment plume images into subareas with the same size, wherein the subareas are required to be adjacent but not overlapped;
step S22, observing pixel information contained in each sub-region, and counting respective gray level histograms H (i), wherein i represents possible gray levels;
s23, enabling gray levels in the sub-region images to have equal pixel numbers, namely, an average value Naver of the pixel numbers:
(1);
wherein ,the pixel number is used for representing the number of pixels of the sub-block image in the horizontal direction; />The number of pixels in the vertical direction; />Representing the number of gray levels in the sub-block image;
step S24, setting a shearing limiting coefficient(default value is 0.01), the value range is 0 to 1, and the closer the value is to 1, the stronger the contrast is; actual clipping limit value->The method comprises the following steps:
(2);
exceeding in H (i)The pixels of the value are truncated, the truncated pixels are re-uniformly allocated to the respective gray levels, assuming the total number of truncated pixels +.>The number of pixels allocated per gray level is determined>The method comprises the following steps:
(3);
wherein ,(4);
by usingRepresenting the newly obtained local histogram after allocation as a piecewise function:
(5)
step S25, new histograms for each sub-area imagePerforming equalization treatment; and obtaining a new gray value by using a bilinear interpolation method.
The application adopts the technical proposal, and compared with the prior art, the application has the following beneficial effects:
1. the device can realize three-dimensional observation record of the sediment plume at the sea bottom, and can completely record the migration and diffusion process of the sediment plume in a three-dimensional space.
2. The algorithm of the application can realize the identification of the image of the submarine sediment plumes and automatically analyze the migration and diffusion rules of the submarine sediment plumes.
3. By automatically analyzing the migration and diffusion form of the sediment plume, the migration and diffusion rule of the sediment plume can be known through morphological analysis, so that the diffusion range and the concentration change rule of the sediment plume can be better estimated.
4. By identifying morphological characteristics of migration and diffusion of the deep sea mining sediment plumes, effective data can be provided for researching the influence and mechanism of the migration and diffusion of the deep sea mining sediment plumes on the bottom layer environment.
5. The observation device and the research method provided by the application have the advantages of simple operation and small influence by other factors, and compared with the common traditional measurement technology, the defect of limited measurement precision caused by the principle defect of measurement can be overcome.
Additional aspects and advantages of the application will be set forth in part in the description which follows, or may be learned by practice of the application.
Drawings
The foregoing and/or additional aspects and advantages of the application will become apparent and may be better understood from the following description of embodiments taken in conjunction with the accompanying drawings in which:
FIG. 1 is a perspective view of a three-dimensional observation device for a plume of a deep-sea seabed sediment;
FIG. 2 is a three view of an orthocube frame in a deep sea mining sediment plume image identifying device;
FIG. 3 is a perspective view of a marine camera device;
fig. 4 is a bottom view (a) and a cross-sectional view (b) of the marine imaging apparatus;
FIG. 5 is a schematic diagram of a process flow of a deep sea bottom sediment plume image;
FIG. 6 is a view showing the pre-enhancement effect of a deep sea bottom sediment plume image;
fig. 7 is a view showing the effect of identifying images of the deep-sea seabed sediment feathered image, (a) an image enhanced by the CLAHE method, (b) a gray-scale image, (c) an image segmented by the U-net model, (d) an image segmented by the threshold,
wherein, the correspondence between the reference numerals and the components in fig. 1 to 4 is:
the marine high-resolution imaging device comprises a right cube frame 1, a counterweight 2, a observing scale 3, a marine imaging device fixing support 4, a steel pipe 5, a stainless steel hoop 6, a marine submarine imaging device 7, a data acquisition and storage pressure-resistant bin 71, a power supply unit 72, a data acquisition unit 73, a pressure-resistant bin lower end sealing cover 74, a marine camera auxiliary lighting device 75 and a marine high-resolution imaging lens 76.
Detailed Description
In order that the above-recited objects, features and advantages of the present application will be more clearly understood, a more particular description of the application will be rendered by reference to the appended drawings and appended detailed description. It should be noted that, without conflict, the embodiments of the present application and features in the embodiments may be combined with each other.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application, however, the present application may be practiced otherwise than as described herein, and therefore the scope of the present application is not limited to the specific embodiments disclosed below.
The apparatus and method for identifying a plume image of a deep sea mining deposit according to an embodiment of the present application will be described in detail with reference to fig. 1 to 7. The device for recording the morphological characteristics of the sediment plumes through the submarine three-dimensional shooting and the analysis method for extracting the morphological characteristics of the sediment plumes through the deep learning method can record the morphologies of the sediment plumes from different angles at a deep sea boundary layer, can directly show the morphological characteristics of the three-dimensional diffusion of the sediment plumes, and extract the boundaries of the sediment plumes from an image through the deep learning method, so that the morphological change rule of the migration diffusion of the sediment plumes is analyzed.
The morphological characteristics of the sediment plume movement diffusion can be accurately and optically observed and recorded through a submarine camera, and the acquired image data of the sediment plume is processed and analyzed by using an image segmentation technology. Image segmentation techniques have found wide application in a variety of industries, such as medical imaging, pedestrian detection, and face recognition. Image segmentation techniques refer to techniques and procedures that divide an image into a number of specific regions with unique properties. In the sediment plume image, the boundary of the sediment plume is identified through an image segmentation technology, and the sediment plume area is segmented from other areas, so that the migration and diffusion rule of the sediment plume is better researched. The common image segmentation techniques mainly include: a threshold-based segmentation method, a region-based segmentation method, an edge-based segmentation method, and a segmentation method based on a specific theory, wherein the image segmentation method is a method based on a deep learning method.
The application provides an observation device for three-dimensionally observing and recording sediment plume form characteristics in a deep sea boundary layer and an image analysis method for deep learning for analyzing sediment plume migration and diffusion rules, so as to overcome the defects of the prior art.
Aiming at the defects of the prior art in sediment plume research, one of the application is to design an imaging device for recording sediment plume migration and diffusion, which can record the morphological characteristics of sediment plume three-dimensional migration and diffusion in multiple angles and can quickly and automatically identify the scale morphology of the sediment plume.
The application provides a device for identifying a sediment plume image of deep sea mining, which is shown in fig. 1 and 2, and comprises an orthocube frame 1, wherein the four corners of the bottom of the orthocube frame 1 are respectively and fixedly provided with a counterweight 2, the counterweight 2 is a round cake-shaped counterweight, and the diameter of the counterweight 2 is 20 cm as shown in fig. 1. The cubic structure frame and the 4 counterweights in the equipment can keep the structural stability of the whole observation device on the sea bottom, and the 4 circular counterweights can be used as the sinking plates, so that the whole observation device can be effectively prevented from sinking deeply due to softer seabed substrate, and the stability of the observation device and the three-dimensional observation effect of deep sea sediment plumes are ensured. The frame of the cube frame 1 is fixedly provided with 6 observation scales 3, each 2 observation scales 3 are respectively fixed on the rear face, the left side face and the bottom face of the frame structure of the cube frame 1, the observation scales 3 are fixed on the left side frame and the lower side frame of the face on the left side face of the cube frame 1, the observation scales 3 are fixed on the right side frame and the lower side frame of the face on the rear face of the cube frame 1, and the observation scales 3 are fixed on the upper side frame and the left side frame of the face on the bottom face of the cube frame 1; the observation scale can be directly shot through the submarine camera device, and the device can play an obvious identification role in the deep sea sediment plume observation process. The regular cube frame 1 is formed by welding a 316L stainless steel pipe with the length of 1 meter. The three-dimensional observation scale of the plume of the deep sea sediment is shown in fig. 2, a scale with 6 scales made of stainless steel is fixed on a frame structure, the observation scale 3 is made of the stainless steel, the length of the observation scale 3 is 1 meter, and the measurement accuracy is millimeter.
The front face, the right side face and the top face of the regular cube frame 1 are respectively fixedly provided with a marine camera equipment fixing support 4, the marine camera equipment fixing support 4 comprises a steel pipe 5 and a stainless steel hoop 6 arranged in the middle of the steel pipe 5, the steel pipe 5 is welded in the middle of the three face frames, the stainless steel hoop 6 is located in the center of the three faces, and the marine camera equipment fixing support structure can support three sets of marine camera equipment with enough strength to shoot and record on the seabed. The three stainless steel hoops 6 are fixedly provided with ocean seabed camera equipment 7.
As shown in fig. 4, the ocean seabed camera device 7 comprises a data collection storage pressure-resistant bin 71, a power supply unit 72 and a data collection unit 73 are installed in the data collection storage pressure-resistant bin 71, a pressure-resistant bin lower end sealing cover 74 is installed at the lower end of the data collection storage pressure-resistant bin 71, an ocean camera auxiliary lighting device 75 and an ocean high-resolution camera lens 76 are vertically installed downwards on the lower surface of the pressure-resistant bin lower end sealing cover 74, the ocean camera auxiliary lighting device 75 and the ocean high-resolution camera lens 76 are respectively surrounded by a stainless steel pressure-resistant bin shell and are electrically connected with the power supply unit 72 and the data collection unit 73 in the data collection storage pressure-resistant bin 71, the data collection unit 73 comprises a relevant data collection plate and data collection software, and the ocean camera auxiliary lighting device 75 and the ocean high-resolution camera lens 76 of the 3 ocean seabed camera devices 7 face the inside the cube frame 1 at an angle vertical to the surface. Ocean high resolution camera lens 76 resolution 10801920, the auxiliary illumination equipment 75 of ocean camera is the LED light source and constitutes, totally three, and the auxiliary illumination equipment 75 of three ocean camera all uses ocean high resolution camera lens 76 as the equiangular surrounding of center its setting, and power supply unit 72 is high-performance lithium cell, can satisfy and observe at the seabed for a long time.
The application discloses an image analysis method using the deep sea mining sediment plume image recognition device as claimed in claim 1, wherein the method for extracting the characteristics of the deep sea sediment plume image is mainly to process and analyze the photographed plume image through a deep sea sediment plume three-dimensional observation device, and the flow diagram is shown in fig. 5, and specifically comprises the following steps:
step S1, processing an original sediment plume image video: decomposing the three-dimensional observation video of the acquired submarine sediment plumes through a frame-by-frame decomposition program to obtain frame images of the sediment plumes; a deep sea sediment plume image photographed at the sea floor is 24 frames per second;
step S2, image enhancement and restoration of deep sea sediment plume images: because the shooting is carried out on the sea bottom of the deep sea, many factors are influenced, such as illumination, scattering refraction of the water body and the like. In order to obtain better image restoration effect, the restoration effect of 5 image restoration methods is compared, and the optimal restoration method for the deep sea sediment plume image is selected through visual comparison and related index statistics. The 5 image restoration methods for effect comparison are respectively as follows: contrast limited adaptive histogram equalization (contrast limited adaptive histogram equalization, CLAHE), histogram Equalization (HE), integrated color model (integrated color model, ICM), relative to global histogram stretching (relative globalhistogram stretching, RGHS), unsupervised color correction (unsupervised color correction method, UCM).
Adopting a contrast limited self-adaptive histogram equalization algorithm to limit noise amplification and local contrast enhancement by limiting the height of a local histogram; dividing an image into a plurality of subareas, classifying the histogram of each subarea, respectively carrying out histogram equalization on each subarea, and finally carrying out interpolation operation on each pixel to obtain a converted gray value, thereby realizing contrast limited self-adaptive histogram equalization image enhancement; the method specifically comprises the following steps:
s21, dividing the photographed sea sediment plume images into subareas with the same size, wherein the subareas are required to be adjacent but not overlapped;
step S22, observing pixel information contained in each sub-region, and counting respective gray level histograms H (i), wherein i represents possible gray levels;
s23, enabling gray levels in the sub-region images to have equal pixel numbers, namely, an average value Naver of the pixel numbers:
(1);
wherein ,the pixel number is used for representing the number of pixels of the sub-block image in the horizontal direction; />The number of pixels in the vertical direction; />Representing the number of gray levels in the sub-block image;
step S24, setting a shearing limiting coefficient(default value is 0.01), the value range is 0 to 1, and the closer the value is to 1, the stronger the contrast is; actual clipping limit value->The method comprises the following steps:
(2);
exceeding in H (i)The pixels of the value are truncated, the truncated pixels are re-uniformly allocated to the respective gray levels, assuming the total number of truncated pixels +.>The number of pixels allocated per gray level is determined>The method comprises the following steps:
(3);
wherein , (4);
by usingRepresenting the newly obtained local histogram after allocation as a piecewise function:
(5)
step S25, new histograms for each sub-area imagePerforming equalization treatment; and obtaining a new gray value by using a bilinear interpolation method.
The image restoration effect of the 5 methods is shown in fig. 6, and according to the manual perception of the image restoration effect, the contrast-limited adaptive histogram equalization (CLAHE) method is finally adopted to restore the deep sea sediment plume observation image.
Step S3, bit depth change of the deep sea sediment plume images: the video image shot by the three-dimensional observation device of the submarine sediment plume in the step S2 is a true color image, the bit depth is 24, and the colors of 24 powers of 2 can be combined; the color is needed to be simpler by utilizing the image segmentation model U-net, so that the image is changed into an image with bit depth of 8 and contains 256 colors;
step S4, graying the deep sea sediment plume image: in digital image processing, various formats of images are generally converted into grayscale images so that the subsequent images are less computationally intensive. The color of each pixel in the color image is determined by three components of R, G and B, and the value of each component can be 0-255, so that one pixel point can have the color change range of 1600 tens of thousands (256×256×256= 1677256), the gray image is a special color image with the same components of R, G and B, one pixel point has the change range of 256, the deep sea sediment plume image is subjected to gray processing for reducing the calculation amount of the subsequent image processing, and the gray image still reflects the distribution and the characteristics of the whole and partial chromaticity and the highlight level of the whole image like the color image. Wherein the pixels with large gray values are brighter: the maximum pixel value is 255, and the pixel value is white; whereas it is darker: the minimum pixel is 0 and is black; the image graying is realized through a cv2.Color_bgr2gray function in an OpenCV function package of python language:
gray=b 0.114+g 0.587+r 0.299, and the GRAY scale image is shown in fig. 7 (B);
step S5, establishing a deep sea sediment plume training data set: marking a plume range in a deep sea sediment plume image by using image marking software LabelMe of a graphical interface, and establishing a training set with a deep sea sediment plume label for image training;
step S6, training image expansion is carried out: the image expansion is also called a data enhancement technology, and is suitable for the condition that the image training sample amount is small, the training set sample used in the training is less, the overfitting of the trained model is easy to cause, the generalization capability is insufficient, so that the number of the training samples is increased through the step, and a random transformation method is used for generating a new image, and the method comprises the following steps: randomly rotating the angle, horizontally traversing a certain distance, vertically traversing a certain distance, randomly zooming a certain range, horizontally overturning and vertically overturning;
s7, image segmentation is carried out by utilizing a U-net model: the U-net model is applied to medical image segmentation at the earliest, has good effect in image segmentation of nature, and is gradually applied. Training of images is performed in a TensorFlow deep learning framework.
FIG. 7 (C) shows an image after image segmentation by a U-net model consisting of a compression part and an expansion part 2; the compression part is formed by 2 convolution kernels to 33 (Conv 2D, activation function is relu), the convolution kernel is 2 +.>2 (2)A maximum pooling layer (maxpoolinglayer) forms a downsampling module, and the total number of the downsampling modules is 3; the extension part consists of an up-sampled convolution layer (UpSampling 2D), a feature splicing layer splicing, and 3 convolution kernels of 3 +.>3 (Conv 2D, activation function is relu) to form an up-sampling module, and 4 up-sampling modules are all arranged;
step S8, threshold segmentation: fig. 7 (d) shows an image subjected to threshold segmentation. According to the segmentation result of the image segmentation model, setting a threshold value as 174, carrying out threshold segmentation on the segmented image, and dividing the image pixel points into target areas and background areas with different gray scales, thereby more clearly showing the distribution range of the deep sea sediment plumes;
and S9, extracting the shape characteristics of the deep sea sediment plumes, and determining the diffusion range of the sediment plumes, so as to analyze the migration and diffusion rules of the deep sea mining sediment plumes.
In the description of the present application, the term "plurality" means two or more, unless explicitly defined otherwise, the orientation or positional relationship indicated by the terms "upper", "lower", etc. are based on the orientation or positional relationship shown in the drawings, merely for convenience of description of the present application and to simplify the description, and do not indicate or imply that the apparatus or elements referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as limiting the present application; the terms "coupled," "mounted," "secured," and the like are to be construed broadly, and may be fixedly coupled, detachably coupled, or integrally connected, for example; can be directly connected or indirectly connected through an intermediate medium. The specific meaning of the above terms in the present application can be understood by those of ordinary skill in the art according to the specific circumstances.
In the description of the present specification, the terms "one embodiment," "some embodiments," "particular embodiments," and the like, mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The above is only a preferred embodiment of the present application, and is not intended to limit the present application, but various modifications and variations can be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims (5)

1. An image analysis method of a device for identifying a sediment plume image of deep sea mining, the device for identifying the sediment plume image of deep sea mining comprises an orthocube frame (1), and is characterized in that four corners of the bottom of the orthocube frame (1) are respectively and fixedly provided with a counterweight (2), 6 observation scales (3) are fixedly arranged on the frame of the orthocube frame (1), each 2 observation scales (3) are respectively fixed on three surfaces of the back, the left side and the bottom of the frame structure of the orthocube frame (1), the observation scales (3) are fixed on the left side and the lower side of the surface, the observation scales (3) are fixed on the right side and the lower side of the surface on the back of the orthocube frame (1), and the observation scales (3) are fixed on the upper side and the left side of the surface on the bottom of the orthocube frame (1);
the ocean photographing equipment fixing support (4) is fixedly arranged in front of, on the right side of and on the top of the regular cube frame (1), the ocean photographing equipment fixing support (4) comprises a steel pipe (5) and stainless steel hoops (6) arranged in the middle of the steel pipe (5), the steel pipe (5) is welded in the middle of the three face frames, the stainless steel hoops (6) are located in the center of the three faces, and ocean seabed photographing equipment (7) is fixedly arranged on the three stainless steel hoops (6);
the ocean seabed camera equipment (7) comprises a data acquisition storage pressure-resistant bin (71), a power supply unit (72) and a data acquisition unit (73) are arranged in the data acquisition storage pressure-resistant bin (71), a pressure-resistant bin lower end sealing cover (74) is arranged at the lower end of the data acquisition storage pressure-resistant bin (71), ocean camera auxiliary lighting equipment (75) and an ocean high-resolution camera lens (76) are vertically downwards arranged on the lower surface of the pressure-resistant bin lower end sealing cover (74), the ocean camera auxiliary lighting equipment (75) and the ocean high-resolution camera lens (76) are respectively surrounded by a stainless steel pressure-resistant bin shell and are electrically connected with the power supply unit (72) and the data acquisition unit (73) in the data acquisition storage pressure-resistant bin (71), the data acquisition unit (73) comprises a relevant data acquisition plate and data acquisition software, and the ocean camera auxiliary lighting equipment (75) and the ocean high-resolution camera lens (76) of 3 ocean seabed camera equipment (7) face the inside a cube frame (1) at a vertical angle relative to the surface;
the method specifically comprises the following steps:
step S1, processing an original sediment plume image video: decomposing the three-dimensional observation video of the acquired submarine sediment plumes through a frame-by-frame decomposition program to obtain frame images of the sediment plumes; a deep sea sediment plume image photographed at the sea floor is 24 frames per second;
step S2, image enhancement and restoration of deep sea sediment plume images: adopting a contrast limited self-adaptive histogram equalization algorithm to limit noise amplification and local contrast enhancement by limiting the height of a local histogram; dividing an image into a plurality of subareas, classifying the histogram of each subarea, respectively carrying out histogram equalization on each subarea, and finally carrying out interpolation operation on each pixel to obtain a converted gray value, thereby realizing contrast limited self-adaptive histogram equalization image enhancement;
the step S2 specifically comprises the following steps:
s21, dividing the photographed submarine sediment plume image into subareas with the same size, wherein the subareas are required to be adjacent but not overlapped;
step S22, observeThe pixel information contained in each sub-region is used to calculate the respective gray level histogram H(i)iRepresenting the gray level that may occur;
s23, making gray level in the sub-region image have equal pixel number, i.e. average value of pixel numberN aver
(1);
wherein ,the pixel number is used for representing the number of pixels of the sub-block image in the horizontal direction; />The number of pixels in the vertical direction;representing the number of gray levels in the sub-block image;
step S24, setting a shearing limiting coefficientThe value range is 0 to 1, and the closer the value is to 1, the stronger the contrast ratio is; actual clipping limit value->The method comprises the following steps:
(2);
handle H(i)Is beyond inThe pixels of the value are truncated, the truncated pixels are re-uniformly allocated to the respective gray levels, assuming the total number of truncated pixels +.>The number of pixels allocated per gray level is determined>The method comprises the following steps:
(3);
wherein , (4);
by usingRepresenting the newly obtained local histogram after allocation as a piecewise function:
(5)
step S25, new histograms for each sub-area imagePerforming equalization treatment; obtaining a new gray value by using a bilinear interpolation method;
step S3, bit depth change of the deep sea sediment plume images: the video image shot by the three-dimensional observation device of the submarine sediment plume in the step S2 is a true color image, the bit depth is 24, and the colors of 24 powers of 2 can be combined; the image is changed to an image with bit depth of 8 and contains 256 colors;
step S4, graying the deep sea sediment plume image: gray processing is carried out on the deep sea sediment plume images, wherein pixels with large gray values are brighter: the maximum pixel value is 255, and the pixel value is white; whereas it is darker: the minimum pixel is 0 and is black; the image graying is realized through a cv2.Color_bgr2gray function in an OpenCV function package of python language:
GRAY = B * 0.114 + G *0.587 + R * 0.299;
step S5, establishing a deep sea sediment plume training data set: marking a plume range in a deep sea sediment plume image by using image marking software LabelMe of a graphical interface, and establishing a training set with a deep sea sediment plume label for image training;
step S6, training image expansion is carried out: increasing the number of training samples, generating a new image using a stochastic transformation method, comprising: randomly rotating the angle, horizontally traversing a certain distance, vertically traversing a certain distance, randomly zooming a certain range, horizontally overturning and vertically overturning;
s7, image segmentation is carried out by utilizing a U-net model: the U-net model consists of a compression part and an expansion part 2; the compression part is formed by 2 convolution kernels to 33, the activation function is relu, the convolution kernel is 2 +.>2, forming a downsampling module by the largest pooling layer, wherein the total number of the downsampling modules is 3; the extension part consists of an up-sampling convolution layer, a characteristic splicing layer splicing, and 3 convolution kernels of 3 +.>3, the convolutional layer Conv2D of the method is activated as relu, and an up-sampling module is formed, and the total number of the up-sampling modules is 4;
step S8, threshold segmentation: according to the segmentation result of the image segmentation model, setting a threshold value as 174, carrying out threshold segmentation on the segmented image, dividing the image pixel points into target areas and background areas with different gray scales, and displaying the distribution range of deep sea sediment plumes;
and S9, extracting the shape characteristics of the deep sea sediment plumes, and determining the diffusion range of the sediment plumes, so as to analyze the migration and diffusion rules of the deep sea mining sediment plumes.
2. The image analysis method of the deep sea mining sediment plume image recognition device according to claim 1, wherein the cubic frame (1) is formed by welding a 316L stainless steel pipe with a length of 1 meter.
3. The image analysis method of the deep sea mining sediment plume image recognition device according to claim 2, wherein the observation scale (3) is made of stainless steel, the length of the observation scale (3) is 1 meter, and the measurement accuracy is millimeter.
4. The image analysis method of a device for identifying a deep sea mining sediment plume image according to claim 1, wherein the counterweight (2) is a pie-shaped counterweight, and the diameter of the counterweight (2) is 20 cm.
5. The image analysis method of a deep sea mining sediment plume image recognition device according to claim 1, wherein the resolution of the marine high resolution camera lens (76) is 10801920, marine camera auxiliary lighting equipment (75) are the LED light source is constituteed, totally three, and three marine camera auxiliary lighting equipment (75) all encircle its setting with marine high-resolution camera lens (76) equiangle as center, and power supply unit (72) are high-performance lithium cell.
CN202310108945.4A 2023-02-14 2023-02-14 Device for identifying images of sediment plume of deep sea mining and image analysis method Active CN116087036B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310108945.4A CN116087036B (en) 2023-02-14 2023-02-14 Device for identifying images of sediment plume of deep sea mining and image analysis method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310108945.4A CN116087036B (en) 2023-02-14 2023-02-14 Device for identifying images of sediment plume of deep sea mining and image analysis method

Publications (2)

Publication Number Publication Date
CN116087036A CN116087036A (en) 2023-05-09
CN116087036B true CN116087036B (en) 2023-09-22

Family

ID=86199063

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310108945.4A Active CN116087036B (en) 2023-02-14 2023-02-14 Device for identifying images of sediment plume of deep sea mining and image analysis method

Country Status (1)

Country Link
CN (1) CN116087036B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117213795B (en) * 2023-11-07 2024-01-30 上海交通大学三亚崖州湾深海科技研究院 Method and system for measuring plume in cross flow
CN117491218B (en) * 2023-12-05 2024-05-03 中国海洋大学 Submarine plume three-dimensional diffusion monitoring device method based on acoustic technology

Citations (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100967683B1 (en) * 2010-02-11 2010-07-07 곽철우 System photographing in water
KR20120003126A (en) * 2010-07-02 2012-01-10 오션테크 주식회사 Real-time camera monitoring bottom sampler
CN203120059U (en) * 2013-03-26 2013-08-07 大连海洋岛水产集团股份有限公司 Deep-sea organism size camera device
KR20160029039A (en) * 2016-01-27 2016-03-14 한국해양과학기술원 Fixed frame type recording device of images and videos
CN110674822A (en) * 2019-09-25 2020-01-10 重庆理工大学 Handwritten digit recognition method based on feature dimension reduction
CN111780727A (en) * 2020-07-21 2020-10-16 烟台仁达自动化装备科技有限公司 Seabed in-situ monitoring buoy system, seabed in-situ monitoring system and seabed in-situ monitoring method
CN111815562A (en) * 2020-06-10 2020-10-23 三峡大学 Retinal vessel segmentation method combining U-Net and self-adaptive PCNN
CN112033383A (en) * 2020-09-11 2020-12-04 中国海洋大学 Deep sea polymetallic nodule mining engineering geological environment monitoring system and method
CN112098289A (en) * 2020-09-23 2020-12-18 中国海洋大学 Device and method for measuring concentration of ocean suspended particulate matters based on digital image processing
CN112164089A (en) * 2020-08-20 2021-01-01 浙江大学 Satellite image-based farmland boundary extraction method and device, electronic equipment and storage medium
CN112507890A (en) * 2020-12-14 2021-03-16 南京林业大学 Bamboo leaf sheet classification and identification method based on SVM classifier
CN113344936A (en) * 2021-07-02 2021-09-03 吉林农业大学 Soil nematode image segmentation and width measurement method based on deep learning
CN113780117A (en) * 2021-08-26 2021-12-10 中国海洋大学 Method for rapidly identifying and extracting relevant parameters of estuary plume profile
CN215340356U (en) * 2021-06-03 2021-12-28 青岛海洋地质研究所 Deep sea movable type ocean bottom seismograph laying and recovering device
CN114419057A (en) * 2022-01-27 2022-04-29 盛视科技股份有限公司 Image-based road surface segmentation method and system
CN114593892A (en) * 2022-03-25 2022-06-07 中国船舶科学研究中心 Underwater test device for marine equipment and operation method thereof
CN115359562A (en) * 2022-08-22 2022-11-18 南京邮电大学 Sign language letter spelling recognition method based on convolutional neural network
CN115588276A (en) * 2022-09-08 2023-01-10 中国海洋大学 Remote monitoring and early warning station and monitoring and early warning method for marine geological disasters

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11458542B2 (en) * 2020-10-30 2022-10-04 Ut-Battelle, Llc Systems and methods for powder bed additive manufacturing anomaly detection
CN115169733B (en) * 2022-08-01 2023-04-25 中国海洋大学 Deep learning-based method for predicting resuspension amount of internal solitary waves on deep sea sediment

Patent Citations (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100967683B1 (en) * 2010-02-11 2010-07-07 곽철우 System photographing in water
KR20120003126A (en) * 2010-07-02 2012-01-10 오션테크 주식회사 Real-time camera monitoring bottom sampler
CN203120059U (en) * 2013-03-26 2013-08-07 大连海洋岛水产集团股份有限公司 Deep-sea organism size camera device
KR20160029039A (en) * 2016-01-27 2016-03-14 한국해양과학기술원 Fixed frame type recording device of images and videos
CN110674822A (en) * 2019-09-25 2020-01-10 重庆理工大学 Handwritten digit recognition method based on feature dimension reduction
CN111815562A (en) * 2020-06-10 2020-10-23 三峡大学 Retinal vessel segmentation method combining U-Net and self-adaptive PCNN
CN111780727A (en) * 2020-07-21 2020-10-16 烟台仁达自动化装备科技有限公司 Seabed in-situ monitoring buoy system, seabed in-situ monitoring system and seabed in-situ monitoring method
CN112164089A (en) * 2020-08-20 2021-01-01 浙江大学 Satellite image-based farmland boundary extraction method and device, electronic equipment and storage medium
CN112033383A (en) * 2020-09-11 2020-12-04 中国海洋大学 Deep sea polymetallic nodule mining engineering geological environment monitoring system and method
CN112098289A (en) * 2020-09-23 2020-12-18 中国海洋大学 Device and method for measuring concentration of ocean suspended particulate matters based on digital image processing
CN112507890A (en) * 2020-12-14 2021-03-16 南京林业大学 Bamboo leaf sheet classification and identification method based on SVM classifier
CN215340356U (en) * 2021-06-03 2021-12-28 青岛海洋地质研究所 Deep sea movable type ocean bottom seismograph laying and recovering device
CN113344936A (en) * 2021-07-02 2021-09-03 吉林农业大学 Soil nematode image segmentation and width measurement method based on deep learning
CN113780117A (en) * 2021-08-26 2021-12-10 中国海洋大学 Method for rapidly identifying and extracting relevant parameters of estuary plume profile
CN114419057A (en) * 2022-01-27 2022-04-29 盛视科技股份有限公司 Image-based road surface segmentation method and system
CN114593892A (en) * 2022-03-25 2022-06-07 中国船舶科学研究中心 Underwater test device for marine equipment and operation method thereof
CN115359562A (en) * 2022-08-22 2022-11-18 南京邮电大学 Sign language letter spelling recognition method based on convolutional neural network
CN115588276A (en) * 2022-09-08 2023-01-10 中国海洋大学 Remote monitoring and early warning station and monitoring and early warning method for marine geological disasters

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
《基于机器视觉的煤矿井下皮带纵向撕裂保护方法研究》;王义涵;《中国优秀硕士学位论文全文数据库 工程科技Ⅰ辑》;全文 *
刘国华.《HALCON数字图像处理》.西安电子科技大学出版社,2018,第173页. *
包米克·维迪雅.《基于GPU加速的计算机视觉编程:使用OpenCV和CUDA实时处理复杂图像数据》.机械工业出版社,2020,第230-232页. *
方志军 等.《TensorFlow应用案例教程》.中国铁道出版社,2020,第119-125页. *
胡众义.《内河航运图像和视频去雾算法》.武汉理工大学出版社,2018,第40-41页. *

Also Published As

Publication number Publication date
CN116087036A (en) 2023-05-09

Similar Documents

Publication Publication Date Title
CN116087036B (en) Device for identifying images of sediment plume of deep sea mining and image analysis method
CN113469177B (en) Deep learning-based drainage pipeline defect detection method and system
CN109460753B (en) Method for detecting floating object on water
CN113450307B (en) Product edge defect detection method
CN110414334B (en) Intelligent water quality identification method based on unmanned aerial vehicle inspection
CN111209876B (en) Oil leakage defect detection method and system
CN107169953B (en) Bridge concrete surface crack detection method based on HOG characteristics
US8508588B2 (en) Methods and systems for identifying well wall boundaries of microplates
CN112308832A (en) Bearing quality detection method based on machine vision
CN108186051B (en) Image processing method and system for automatically measuring double-apical-diameter length of fetus from ultrasonic image
CN112098289B (en) Device and method for measuring concentration of ocean suspended particulate matters based on digital image processing
KR100889997B1 (en) Apparatus and Method for Ship Ballast Water Examination using Image Processing
CN114119526A (en) Steel plate surface defect detection and identification system and method based on machine vision
CN108470338A (en) A kind of water level monitoring method
CN116721391A (en) Method for detecting separation effect of raw oil based on computer vision
CN108364296B (en) Cell population space distribution construction method based on multilayer holographic reconstruction and focusing strategy
CN108388853A (en) The substep that hologram coexists for leucocyte and blood platelet is rebuild and method of counting
CN110853041A (en) Underwater pier component segmentation method based on deep learning and sonar imaging
CN115511814A (en) Image quality evaluation method based on region-of-interest multi-texture feature fusion
CN116434230A (en) Ship water gauge reading method under complex environment
CN116805416A (en) Drainage pipeline defect identification model training method and drainage pipeline defect identification method
CN110348533A (en) A kind of planktonic organism partial size spectrum detection method based on SVM
CN115578695A (en) Water gauge water level machine vision detection method and device with free shooting visual angle
CN115170596A (en) Blood agglutination detection device, system and method based on image edge detection
CN114396919A (en) Method for extracting small water tank wave elements based on photogrammetry

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant