CN110674823A - Sample library construction method based on automatic identification of deep sea large benthonic animals - Google Patents
Sample library construction method based on automatic identification of deep sea large benthonic animals Download PDFInfo
- Publication number
- CN110674823A CN110674823A CN201910915735.XA CN201910915735A CN110674823A CN 110674823 A CN110674823 A CN 110674823A CN 201910915735 A CN201910915735 A CN 201910915735A CN 110674823 A CN110674823 A CN 110674823A
- Authority
- CN
- China
- Prior art keywords
- sample
- library
- category
- video
- samples
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/752—Contour matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
Abstract
The invention discloses a sample library construction method based on automatic identification of deep sea large benthonic animals, which comprises the following steps: preprocessing the acquired deep sea video; extracting a video image from the preprocessed video and carrying out image segmentation to obtain an effective outline; respectively carrying out outline similarity comparison on the effective outline and all samples of each category sub-library of a pre-established image sample library, calculating the average value of the outline similarity of each category, and selecting the category with the minimum average value as a tentative category; the image sample library comprises a plurality of category sub-libraries, and one category sub-library stores a picture sample of one type of large benthonic animals; and if the minimum average value is not larger than the contour similarity threshold of the tentative category, determining that the category of the large benthonic animals in the video image is the tentative category, adding the video into a category sub-library corresponding to the tentative category, and otherwise, storing the video into a sample temporary library for manual identification.
Description
Technical Field
The invention relates to deep sea benthos observation, in particular to a sample library construction method based on automatic identification of deep sea large benthos.
Background
The large benthonic animals in deep sea are the most important components in the aquatic ecosystem, and the growth process and the living condition of the large benthonic animals can not only indicate the change of the deep sea environment, but also reflect the quality condition of the deep sea environment in a specific time to a certain extent, so that the large benthonic animals are always the research focus in the marine field, but the complexity and the diversity of marine organisms cause great difficulty and challenge to the marine research. With the development of deep sea observation technology, many deep sea observation platforms perform deep sea large-scale benthonic animal observation by arranging equipment such as a deep sea Autonomous Underwater Vehicle (AUV) and an underwater camera. However, due to the huge amount of video data obtained by deep sea observation, how to automatically and rapidly identify the deep sea large benthonic animals is always the focus of attention of scholars in the related field. With the rapid development of deep learning, the target identification method based on the neural network has been applied to a plurality of scientific fields and accepted by various industries. The neural network is an intelligent information processing technology simulating the human brain information processing process, and has self-organization, self-adaptability and strong robustness. The representative neural network model comprises a Hopfield network, a feedforward neural network, a Radial Basis Function (RBF) neural network, a Markov Chain (MC), a Convolutional Neural Network (CNN), a support vector machine and the like, the artificial neural network carries out supervised or unsupervised sample training in advance to obtain a proper network weight, and only a trained network structure needs to be called for classification during identification, so that the algorithm is short in time consumption. Meanwhile, the neural network can perform information distribution storage and large-scale self-adaptive parallel processing and has the advantages of high fault tolerance, strong robustness and the like, however, the application basis of all the neural networks is an abundant sample library, and the application of the neural network in the automatic identification field of the large benthonic animals on the seabed is severely restricted due to the loss of the large benthonic animal sample library on the seabed, so that the development of the large benthonic animal identification technology on the seabed is limited. The invention fully considers the urgent need of the deep sea observation field for the automatic identification technology of the large benthonic animals at the sea bottom, and provides a method for constructing a sample library of the large benthonic animals at the deep sea, which takes network pictures and deep sea video images as data sources and assists the identification of the large benthonic animals by comparing the contour similarity of the large benthonic animals.
The traditional marine organism identification technology adopts a visual interpretation method, only human eyes are used for judging organism types in a video, but visual interpretation firstly needs to know the characteristics of various deep sea large benthonic animals, then the animals in the deep sea video are analyzed according to the characteristics, the types of the benthonic animals are determined, the whole process of artificial participation causes over-high subjectivity and has higher requirements on the specialty of an interpreter, and the artificial identification method can not meet the requirements of the marine organism identification technology on identification efficiency, identification accuracy and universality along with the increase of the types and the number of the marine large benthonic animals.
The method for collecting the pictures of the related deep-sea large-scale benthonic animals by utilizing the web crawler technology is a sample library construction method commonly used in the deep-sea observation field at present. The method comprises the steps of firstly reading out network resources specified in a URL address from a network stream, storing the network resources to the local, and then quickly capturing a retrieval picture by utilizing an image retrieval function based on texts provided by search engines such as Google and Baidu. However, because related pictures of deep-sea large-scale benthonic animals on the network are few at present, and the network pictures and the pictures obtained by observing the video have certain differences, the quality of the sample library can be seriously influenced by only using a network crawler technology to obtain samples, and the accuracy of animal identification by using the sample library can be influenced.
Disclosure of Invention
The invention aims to solve the problem of missing of a submarine large-scale benthonic animal sample library in the current submarine observation network, and the video data obtained by submarine observation is taken as the main source of samples in the deep-sea large-scale benthonic animal sample library, so that the difference between picture samples and video data is effectively eliminated, and the quality of the samples in the sample library is improved. According to the method, the images of the large benthonic animals on the seabed collected by the network are used as auxiliary data, the outline data is extracted, and accordingly, the video data samples are preliminarily identified, so that the manual participation is effectively reduced, and the efficiency and the precision of the sample library construction are improved. The invention intercepts video images of different frames as samples, and stores the samples as video images of different forms of the large benthonic animals in the sample library, so that the obtained samples are more comprehensive.
Aiming at the current situation that a technical means for quickly and accurately constructing a seabed large benthonic animal sample library is lacked at present, the invention provides a sample library construction method based on automatic identification of deep sea large benthonic animals, which comprises the following steps:
preprocessing the acquired deep sea video;
extracting a video image from the preprocessed video and carrying out image segmentation to obtain an effective outline;
respectively carrying out outline similarity comparison on the effective outline and all samples of each category sub-library of a pre-established image sample library, calculating the average value of the outline similarity of each category, and selecting the category with the minimum average value as a tentative category; the image sample library comprises a plurality of category sub-libraries, and one category sub-library stores a picture sample of one type of large benthonic animals;
and if the minimum average value is not larger than the contour similarity threshold of the tentative category, determining that the category of the large benthonic animals in the video image is the tentative category, adding the video into a category sub-library corresponding to the tentative category, and otherwise, storing the video into a sample temporary library for manual identification.
As an improvement of the above method, the pre-treatment comprises: video cutting, video image enhancement and video image denoising.
As an improvement of the above method, the method extracts a video image from the preprocessed video and performs image segmentation to obtain an effective contour; the method specifically comprises the following steps:
extracting a frame of video image every 5s, and performing image segmentation on the video image by adopting a segmentation method based on region growing;
if the object of the segmented video image contains the image spots with the pixel number more than 100, the object is the effective outline of the large benthonic animal target;
if the number of the effective outlines in the video image is larger than 2, the video image is marked as a mixed sample, and the sample is noted as the mixed sample when the video image is put in storage.
As an improvement of the above method, the comparing the effective contours with all samples of each category sub-library of a pre-established image sample library respectively, and calculating an average value of the contour similarities of each category specifically includes:
calculating the similarity of the effective contour of the video image and the contour of all samples of each category sub-library:
based on the Hu moment characteristics, comparing the effective outline of the video image with the similarity of a sample of each category sub-library by using an OPEN CV cvMatchShapes function; the function return value is the contour similarity; if the return value of the function is 0, the two contours are completely the same, the larger the return value is, the lower the similarity of the contours is, and the maximum return value is 1;
calculating the mean value
Where n is the number of all samples of a class sublibrary, TiIs the contour similarity of the effective contour of the visual video image and the i-th sample contour.
As an improvement of the above method, the adding the video to the sub-library of the category specifically includes: and selecting adjacent frame video images with the interval of 2s between the front and the back of the video image, and adding the video images as video image samples of different forms of the large benthonic animals into the sub-libraries of the category.
As an improvement of the above method, the step of establishing the category sub-library comprises:
obtaining a picture sample of a type of large benthonic animal;
standardizing all the picture samples to form a picture sample set;
extracting the animal contour of each picture sample in the picture sample set by adopting a segmentation method based on region growth;
and calculating the contour similarity threshold of the class sub-library.
As an improvement of the above method, the extracting an animal contour of each picture in the picture sample set specifically includes:
based on the RGB value of the picture, merging adjacent homogeneous paired pixels from one pixel, and segmenting the image into a spot comprising a plurality of similar or identical pixels through a plurality of iterations, wherein the spot is an animal contour;
the process of merging adjacent homogeneous paired pixels is as follows:
calculate the heterogeneity f of pairs of neighboring pixels:
where c is 3 layers of video image RGB, nmNumber of pixels, σ, of the object after mergingmStandard deviation of the merged object, n1、n2For merging the number of pixels of the first two adjacent objects, σ1、σ2To merge the standard deviations of the first two neighboring objects, EmFor the actual boundary length of the merged object region, E1、E2For merging the actual boundary lengths, L, of the preceding two adjacent object regionsmIs the length of the rectangular boundary, L, containing the range of the merged image area1、L2The lengths of two rectangular boundaries containing the image area range before merging;
and judging whether the heterogeneity of the paired adjacent pixels is less than 50, if so, judging that the adjacent pixels are the same or similar pixels, and merging.
As an improvement of the foregoing method, the calculating a threshold of the similarity of the contours of the category sub-library specifically includes:
wherein, TdIs the contour similarity threshold of the d-th class sub-library, n is the number of samples in the class sub-library, SijAnd the similarity between the ith sample contour and the jth sample contour in the category sub-library, wherein the values of i and j are from 1 to n.
As an improvement of the above method, the method further comprises: the step of verifying the category sub-library comprises the following steps:
calculating the contour similarity of every two picture samples in the category sub-library, and sequencing the picture samples from small to large;
extracting 5% of sample pairs from big to small, carrying out manual verification, and determining whether the categories are uniform;
if there is an incorrect sample class classification, then manual revision is performed; and after the revision is finished, reselecting the sample for manual verification, otherwise, adding the sample into the category sub-library.
As an improvement of the above method, the method further comprises: the method comprises the steps of carrying out periodic retest on a sample library, wherein the sample library comprises a pre-established picture sample library and added video images; the method specifically comprises the following steps: whenever the sample amount in the sample library is increased by 500, performing sampling inspection verification to ensure the accuracy of the samples in the sample library; randomly extracting 5% of samples and samples with 5% of contour similarity from large to small in the samples, wherein the samples for sampling comprise picture samples in a picture sample library and added video images; if the accuracy of the sample is more than or equal to 95%, all samples in the sample library are considered to be valid samples; otherwise, the sample in the sample library is considered to have an error sample, and the sample is corrected.
Compared with the prior art, the invention has the advantages that:
1. the invention relates to a method capable of quickly constructing a sample library of deep sea large benthonic animals, which utilizes the existing deep sea video data, combines pictures obtained by a network crawler, preliminarily determines the types of the deep sea large benthonic animals by comparing the outlines of the deep sea large benthonic animals obtained by a deep sea video image with the outlines of the deep sea large benthonic animals obtained by the network picture, simultaneously performs manual assistance to determine the final video image sample, reduces manual participation, greatly saves manpower, material resources and time, realizes the quick construction of the sample library of the deep sea large benthonic animals, and provides a foundation for the monitoring of the deep sea large benthonic animals;
2. the method ensures the precision of the deep-sea large-scale benthonic animal sample library, adopts the deep-sea video as the basic data constructed by the deep-sea large-scale benthonic animal sample library, eliminates the difference between the picture sample and the video data, and ensures the quality and the practicability of the sample; in consideration of the continuity of the video samples, video images adjacent to the video images of the samples are extracted as samples, the video images of the large benthonic animals in different motion forms are stored as the samples, and the comprehensiveness and the application range of the sample library are improved;
3. the method realizes the rapid construction of the high-precision deep sea large-scale benthonic animal sample library based on the network pictures and the deep sea videos, effectively solves the problem of the loss of the deep sea large-scale benthonic animal sample library at present, and provides data support for realizing the automatic identification of the seabed large-scale benthonic animals;
4. the method of the invention reduces manual participation, improves the recognition efficiency, enriches the sample library and improves the sample accuracy.
Drawings
FIG. 1 is a flow chart of a sample library construction method based on automatic identification of large benthic animals in deep sea according to the present invention.
Detailed Description
According to the large benthonic animal classification system set based on the sample library, relevant images of the large benthonic animals on the sea bottom are collected from the network, the outlines of the large benthonic animals are extracted to serve as auxiliary data, and the average value of the outline similarity of the benthonic animals is calculated, so that the outline similarity threshold values of different types of the benthonic animals are obtained. And processing deep sea video data, identifying effective outlines in the video images, and extracting effective video images. And comparing the outline of the benthonic animals in the picture with the outline of the benthonic animals in the video image, and determining the animal category. After the category is determined, adjacent video images capable of showing the morphology of the large benthonic animals are extracted. And warehousing the extracted images as samples, thereby constructing a seabed large benthonic animal sample library.
As shown in FIG. 1, the method for constructing the sample bank based on the automatic identification of the large benthic animals in deep sea provided by the invention comprises the following steps:
1. and constructing a large benthonic animal classification system.
And setting a classification system of the large benthonic animals in the large benthonic animal sample library according to the door grade.
2. And obtaining a picture of the benthonic animal.
The crawler technology is utilized to crawl pictures of different large zoobenthos from the webpage, clear pictures which can represent various zoobenthos categories are screened, and the clear pictures are stored in a local folder according to the same picture format.
3. And extracting the outline of the benthonic animal.
The method comprises the steps of extracting the outline of the large benthonic animal in the picture by adopting a segmentation method based on region growth, gradually combining adjacent homogeneous paired pixels from one pixel, and segmenting the image into a spot comprising a plurality of similar or identical pixels through a plurality of iterations. Calculating the heterogeneity of adjacent pixels in the picture based on the RGB values of the picture pixels, wherein the calculation method comprises the following steps:
where c is 3 layers of video image RGB, nmNumber of pixels, σ, of the object after mergingmStandard deviation of the merged object, n1、n2For merging the number of pixels of the first two adjacent objects, σ1、σ2To merge the standard deviations of the first two neighboring objects, EmFor the actual boundary length of the merged object region, E1、E2For merging the actual boundary lengths, L, of the preceding two adjacent object regionsmIs the length of the rectangular boundary, L, containing the range of the merged image area1、L2The lengths of the two rectangular boundaries containing the image area before merging. According to the compositionAnd judging whether the adjacent pixels are the same or similar pixels according to the heterogeneity of the adjacent pixels, thereby judging whether the adjacent pixels are combined.
4. And calculating the contour similarity of the benthonic animals.
Based on the Hu moment characteristics, calculating the sample contour similarity of the large benthonic animals of the same class by using an OPEN CV cvMatchShapes function, selecting a sample with lower similarity of 5 percent for manual verification, and determining whether the sample class is correct. If there are incorrect samples, then a manual revision is made. And after the revision is finished, reselecting the sample for manual verification. If all samples are correct, the samples are stored in a sample bank.
5. And calculating a contour similarity threshold of the benthonic animals.
The method takes the average value of the contour similarity of the same benthonic animal sample library as the threshold value of the contour similarity of the same benthonic animal sample library. The method for calculating the average value of the sample contour similarity comprises the following steps:
wherein, TdIs the contour similarity threshold of the d-th class sub-library, n is the number of samples in the class sub-library, SijAnd the similarity between the ith sample contour and the jth sample contour in the category sub-library, wherein the values of i and j are from 1 to n.
6. And preprocessing the deep sea video.
And obtaining a deep sea video, and carrying out preprocessing such as video shearing, video image enhancement, video image denoising and the like.
7. And extracting effective video images.
The video image is extracted at intervals of 5s, the image is segmented by adopting a segmentation method based on region growing, and the segmented target image spots containing more than 100 pixels are regarded as the effective outline of the large benthonic animal, and the video image is regarded as the effective video image.
8. And judging the type of the sample.
If there are 2 or more effective contours in step 7, the video image is considered as a mixed sample, and remarking is performed. Otherwise it is considered a single sample.
9. And comparing the similarity.
Comparing the effective contour extracted from the video image with the contour extracted from the picture sample in the sample library one by one, calculating the similarity by using the OPEN CV cvMatchShapes function and all samples in the sub-library, and calculating the average value of all the similarities (the average value of all the similarities is calculated: (the average value of all the similarities is calculated)
Wherein n is the number of samples in the sample sub-library, SiSimilarity between video sample image contour and ith sample contour in sample sub-library
10. Tentative sample classes.
And calculating the average value of the contour similarity of the large benthonic animals compared with all the sub-libraries, and selecting the most similar sub-library benthonic animal category as the category of the large benthonic animals in the sample.
11. A sample class is determined.
And (3) comparing the average value calculated in the step (10) with a threshold value of a tentative type, and if the average value is smaller than the threshold value, determining the type of the large benthonic animals in the video image sample and warehousing the large benthonic animals. Otherwise, storing the sample in a sample temporary library for manual identification.
12. Sample neighboring video images are extracted.
And selecting adjacent frame video images with the interval of 2s between the front and the back of the video image as video image samples of different forms of the large benthonic animals.
13. And sampling and inspecting the sample library sample.
And (3) periodically and randomly drawing samples of 5% of the samples and 5% of the samples with lower outline similarity, manually verifying whether the accuracy is more than 95%, and if the accuracy is less than 95%, determining that a sample error exists in the sample library and needing manual revision. Otherwise, all the samples in the sample library are considered to be valid samples.
The construction of the deep sea large benthonic animal sample library is a foundation for automatically identifying the deep sea large benthonic animals, and the cameras arranged on the sea floor provide continuous videos of long-time sequences and provide good data foundation. The method is different from the current sample library construction scheme for collecting picture samples from the network, the video shot by the deep-sea camera is taken as a data source to extract the deep-sea large benthonic animal image as a sample, the video image adjacent to the sample video image is extracted as the sample in consideration of the continuity of the video sample, and the video images of the large benthonic animal in different motion forms are stored as the sample. Not only the quality and the precision of the sample are improved, but also the practicability of the sample is improved.
The method comprises the steps of taking pictures of the deep-sea large-scale benthonic animals collected by a network as auxiliary data, extracting video sample images, comparing the video sample images with the profiles of the deep-sea large-scale benthonic animals collected by the network and the profiles of video image samples in a sample library respectively, and preliminarily judging the types of the large-scale benthonic animals.
Specifically, the method of the present invention comprises:
step 1: and constructing a large benthonic animal classification system in the large benthonic animal sample library. The deep sea creatures in the video obtained by deep sea observation refer to large benthonic animals and swimming animals moving in deep sea. The sample library is divided into 13 types of sub-libraries which are respectively as follows: phylum cnidarianae, phylum stellera, phylum enterogills, phylum urechisella, annelids, phylum echinoderma, phylum ctenophyta, phylum mollusca, phylum arthropoda, phylum platyphylla, phylum neomorpha, deep sea fish and others. Among other classes are newly discovered species.
Step 2: and warehousing the network picture samples.
Step 2.1, collecting pictures by the network: adopting a web crawler technology, and taking the subclass name classified by the sample library as the name of the searched picture;
step 2.2, selecting a picture sample: acquiring a URL address of the picture, analyzing the address content, capturing the picture and storing the picture in a local folder;
step 2.3, standardizing pictures: and analyzing the definition of the picture, and deleting picture files smaller than 50 k. And (3) manually and quickly browsing the pictures, screening the pictures which can most represent various zoobenthos categories from the pictures, and storing other reserved pictures into a jpg format to form a picture sample set.
Step 2.4, extracting the outline of the benthonic animal of the picture: and extracting the outline of the large benthonic animals in the picture by adopting a segmentation method based on region growth. Specifically, based on the RGB values of the picture, adjacent homogeneous pairs of pixels are gradually merged from one pixel, and the image is segmented into a patch containing a plurality of similar or identical pixels through a plurality of iterations. The basis for whether pairs of pixels are merged is that the heterogeneity of two pixels is less than 50, and the heterogeneity calculation formula is:
where c is 3 layers of video image RGB, nmNumber of pixels, σ, of the object after mergingmStandard deviation of the merged object, n1、n2For merging the number of pixels of the first two adjacent objects, σ1、σ2To merge the standard deviations of the first two neighboring objects, EmFor the actual boundary length of the merged object region, E1、E2For merging the actual boundary lengths, L, of the preceding two adjacent object regionsmIs the length of the rectangular boundary, L, containing the range of the merged image area1、L2The lengths of the two rectangular boundaries containing the image area before merging. And judging whether the heterogeneity of the paired adjacent pixels is less than 50, if so, judging that the adjacent pixels are the same or similar pixels, and merging.
Step 2.5, calculating the sample contour similarity: based on the Hu moment characteristics, the similarity degree of the sample contour is compared by using an OPEN CV cvMatchShapes function. The function return value is 0, the two sample contours are completely the same, the greater the return value is, the lower the contour similarity is, and the maximum return value is 1.
Step 2.6, extracting a sample, manually verifying and then warehousing: and calculating the Hu moment similarity between every two sample contours in the sample sub-library, and sequencing from small to large. And extracting 5% of sample pairs from big to small for manual verification to determine whether the categories are uniform. If there is an incorrect sample class classification, a manual revision is made. And after the revision is finished, reselecting the sample for manual verification. If the determination is correct, the sample profile data is binned.
Step 2.7, calculating a sample similarity threshold: in order to ensure the accuracy of the post-calculation of the samples, the invention provides a method for calculating the average value of the contour similarity of all the samples in the sample sub-library, which is used as a threshold value for determining whether the video image is of a change category. The method for calculating the sample contour similarity threshold in all the sample sub-libraries comprises the following steps:
wherein, TdIs the contour similarity threshold of the d-th class sub-library, n is the number of samples in the class sub-library, SijAnd the similarity between the ith sample contour and the jth sample contour in the category sub-library, wherein the values of i and j are from 1 to n.
And step 3: deep sea video sample warehousing
Step 3.1, obtaining a deep sea video,
step 3.2, deep sea video preprocessing: the method mainly carries out preprocessing such as video shearing, video image enhancement, video image denoising and the like.
Step 3.3, extracting video images: considering the movement speed of animals and the flow speed of deep sea water, the invention extracts one frame of video image every 5s for processing;
step 3.4, image segmentation: and carrying out image segmentation by adopting a segmentation method based on region growing. Since many zooplankton may exist in the deep sea video, the method considers the image spots with the pixel number more than 100 in the segmented object as the effective outlines of the large benthonic animal targets, and the video images with the effective outlines are considered as the effective video images.
Step 3.5, judging the effective contour: if the number of the effective outlines in the video image is larger than 2, the video image is marked as a mixed sample, and the sample is noted as the mixed sample when the video image is put in storage. And extracting effective outlines to serve as the outlines of the large benthonic animals.
Step 3.6, calculating the average value of the contour similarity: and carrying out similarity comparison on the effective contour obtained from the deep sea video and the samples in the sample sub-library one by one. Similar to step 2.7, the similarity between the video image contour and all the sample contours in the sub-library is compared using the OPEN CV cvMatchShapes function, and then the average value is calculated (Wherein n is the number of samples in the sample sub-library, SiThe similarity between the video sample image contour and the ith sample contour in the sample sub-library).
Step 3.7: tentative sample classification: and respectively carrying out contour similarity comparison on the deep sea video sample and samples of all sample sub-libraries, calculating an average value, and selecting the sample library class with the minimum average value as a tentative class.
Step 3.8: determining the animal class: and if the minimum average value is less than or equal to the threshold value of the tentative category, determining the category of the large benthonic animals in the video image sample, and warehousing the large benthonic animals. And if the minimum average value is larger than the threshold value of the tentative category, storing the minimum average value in a temporary sample library, and storing the minimum average value in the temporary sample library after the animal category is manually identified.
Step 3.9: determining a video image sample: after the video image samples are determined, selecting adjacent frame video images with the interval of 2s between the front and the back of the video images, and warehousing the video images as video image samples of different forms of the large benthonic animals.
And 4, step 4: and (3) regular rechecking: and whenever the sample amount in the sample library is increased by 500, performing sampling verification to ensure the accuracy of the samples in the sample library. The sampling comprises a network picture sample obtained by a network crawler technology and a sample extracted from a deep sea observation video, and 5% of the samples and samples with 5% of contour similarity from large to small are randomly extracted. If the accuracy of the sample is greater than or equal to 95%, all samples in the sample library are considered valid samples. And if the accuracy of the sample is less than 95%, determining that the sample in the sample library has an error sample, and correcting the sample.
And 5: and completing the construction of the sample library.
Finally, it should be noted that the above embodiments are only used for illustrating the technical solutions of the present invention and are not limited. Although the present invention has been described in detail with reference to the embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (10)
1. A sample bank construction method based on automatic identification of large benthic animals in deep sea comprises the following steps:
preprocessing the acquired deep sea video;
extracting a video image from the preprocessed video and carrying out image segmentation to obtain an effective outline;
respectively carrying out outline similarity comparison on the effective outline and all samples of each category sub-library of a pre-established image sample library, calculating the average value of the outline similarity of each category, and selecting the category with the minimum average value as a tentative category; the image sample library comprises a plurality of category sub-libraries, and one category sub-library stores a picture sample of one type of large benthonic animals;
and if the minimum average value is not larger than the contour similarity threshold of the tentative category, determining that the category of the large benthonic animals in the video image is the tentative category, adding the video into a category sub-library corresponding to the tentative category, and otherwise, storing the video into a sample temporary library for manual identification.
2. The deep sea large benthic animal automatic identification based specimen bank construction method according to claim 1, wherein the pretreatment comprises: video cutting, video image enhancement and video image denoising.
3. The method for constructing the specimen bank based on the automatic identification of the large deep sea benthonic animals according to claim 2, wherein the effective contour is obtained by extracting a video image from the preprocessed video and performing image segmentation; the method specifically comprises the following steps:
extracting a frame of video image every 5s, and performing image segmentation on the video image by adopting a segmentation method based on region growing;
if the object of the segmented video image contains the image spots with the pixel number more than 100, the object is the effective outline of the large benthonic animal target;
if the number of the effective outlines in the video image is larger than 2, the video image is marked as a mixed sample, and the sample is noted as the mixed sample when the video image is put in storage.
4. The deep sea large benthic animal automatic identification based sample bank construction method according to claim 3, wherein the comparing the effective contours with all samples of each category sub-bank of the pre-established picture sample bank respectively, and calculating the average value of the contour similarity of each category specifically comprises:
calculating the similarity of the effective contour of the video image and the contour of all samples of each category sub-library:
based on the Hu moment characteristics, comparing the effective outline of the video image with the similarity of a sample of each category sub-library by using an OPEN CV cvMatchShapes function; the function return value is the contour similarity; if the return value of the function is 0, the two contours are completely the same, the larger the return value is, the lower the similarity of the contours is, and the maximum return value is 1;
calculating the mean value
Where n is the number of all samples of a class sublibrary, TiIs the contour similarity of the effective contour of the video image and the contour of the ith sample.
5. The method for constructing the specimen bank based on the automatic identification of the large benthic animals according to claim 1, wherein the video is added into the sub-bank of the category, specifically: and selecting adjacent frame video images with the interval of 2s between the front and the back of the video image, and adding the video images as video image samples of different forms of the large benthonic animals into the sub-libraries of the category.
6. The method for constructing a specimen bank based on the automatic identification of large benthic animals according to claim 5, wherein the step of creating the one category sublibrary comprises:
obtaining a picture sample of a type of large benthonic animal;
standardizing all the picture samples to form a picture sample set;
extracting the animal contour of each picture sample in the picture sample set by adopting a segmentation method based on region growth;
and calculating the contour similarity threshold of the class sub-library.
7. The method for constructing the specimen bank based on the automatic identification of the deep sea large benthonic animals according to claim 6, wherein the method for extracting the animal outlines of each image specimen in the image specimen set by adopting a segmentation method based on region growing; the method specifically comprises the following steps:
based on the RGB value of the picture, merging adjacent homogeneous paired pixels from one pixel, and segmenting the image into a spot comprising a plurality of similar or identical pixels through a plurality of iterations, wherein the spot is an animal contour;
the process of merging adjacent homogeneous paired pixels is as follows:
calculate the heterogeneity f of pairs of neighboring pixels:
where c is 3 layers of video image RGB, nmNumber of pixels, σ, of the object after mergingmStandard deviation of the merged object, n1、n2For merging the number of pixels of the first two adjacent objects, σ1、σ2To merge the standard deviations of the first two neighboring objects, EmFor the actual boundary length of the merged object region, Ex, E2For merging the actual boundary lengths, L, of the preceding two adjacent object regionsmIs the length of the rectangular boundary, L, containing the range of the merged image area1、L2The lengths of two rectangular boundaries containing the image area range before merging;
and judging whether the heterogeneity of the paired adjacent pixels is less than 50, if so, judging that the adjacent pixels are the same or similar pixels, and merging.
8. The method for constructing the sample library based on the automatic identification of the large deep sea benthonic animals according to claim 7, wherein the step of calculating the contour similarity threshold of the class of sub-libraries comprises the following steps:
wherein, TdIs the contour similarity threshold of the d-th class sub-library, n is the number of samples in the class sub-library, SijAnd the similarity between the ith sample contour and the jth sample contour in the category sub-library, wherein the values of i and j are from 1 to n.
9. The method for constructing a specimen bank based on the automatic identification of the large benthic animals according to claim 8, further comprising: the step of verifying the category sub-library comprises the following steps:
calculating the contour similarity of every two picture samples in the category sub-library, and sequencing the picture samples from small to large;
extracting 5% of sample pairs from big to small, carrying out manual verification, and determining whether the categories are uniform;
if there is an incorrect sample class classification, then manual revision is performed; and after the revision is finished, reselecting the sample for manual verification, otherwise, adding the sample into the category sub-library.
10. The method for constructing a specimen bank based on the automatic identification of the large benthic animals according to claim 1, further comprising: the method comprises the steps of carrying out periodic retest on a sample library, wherein the sample library comprises a pre-established picture sample library and added video images; the method specifically comprises the following steps: whenever the sample amount in the sample library is increased by 500, performing sampling inspection verification to ensure the accuracy of the samples in the sample library; randomly extracting 5% of samples and samples with 5% of contour similarity from large to small in the samples, wherein the samples for sampling comprise picture samples in a picture sample library and added video images; if the accuracy of the sample is more than or equal to 95%, all samples in the sample library are considered to be valid samples; otherwise, the sample in the sample library is considered to have an error sample, and the sample is corrected.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910915735.XA CN110674823A (en) | 2019-09-26 | 2019-09-26 | Sample library construction method based on automatic identification of deep sea large benthonic animals |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910915735.XA CN110674823A (en) | 2019-09-26 | 2019-09-26 | Sample library construction method based on automatic identification of deep sea large benthonic animals |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110674823A true CN110674823A (en) | 2020-01-10 |
Family
ID=69079175
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910915735.XA Pending CN110674823A (en) | 2019-09-26 | 2019-09-26 | Sample library construction method based on automatic identification of deep sea large benthonic animals |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110674823A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112907615A (en) * | 2021-01-08 | 2021-06-04 | 中国石油大学(华东) | Submarine landform unit contour and detail identification method based on region growing |
| CN112989086A (en) * | 2021-05-20 | 2021-06-18 | 苏州希格玛科技有限公司 | Intelligent recognition and classification system and method for city management images |
| CN116758580A (en) * | 2023-05-05 | 2023-09-15 | 中国地质大学（北京） | Benthonic organism recognition method, benthonic organism recognition device, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103430179A (en) * | 2011-02-07 | 2013-12-04 | 英特尔公司 | Method, system and computer-readable recording medium for adding new image and information on new image to image database |
| CN104778478A (en) * | 2015-04-22 | 2015-07-15 | 中国石油大学(华东) | Handwritten numeral identification method |
| CN105023025A (en) * | 2015-08-03 | 2015-11-04 | 大连海事大学 | Set opening trace image classification method and system |
| CN106778501A (en) * | 2016-11-21 | 2017-05-31 | 武汉科技大学 | Video human face ONLINE RECOGNITION method based on compression tracking with IHDR incremental learnings |
| CN109781917A (en) * | 2017-11-14 | 2019-05-21 | 中国科学院大连化学物理研究所 | A kind of biological sample intelligent identification Method based on molecule map |
-
2019
- 2019-09-26 CN CN201910915735.XA patent/CN110674823A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103430179A (en) * | 2011-02-07 | 2013-12-04 | 英特尔公司 | Method, system and computer-readable recording medium for adding new image and information on new image to image database |
| CN104778478A (en) * | 2015-04-22 | 2015-07-15 | 中国石油大学(华东) | Handwritten numeral identification method |
| CN105023025A (en) * | 2015-08-03 | 2015-11-04 | 大连海事大学 | Set opening trace image classification method and system |
| CN106778501A (en) * | 2016-11-21 | 2017-05-31 | 武汉科技大学 | Video human face ONLINE RECOGNITION method based on compression tracking with IHDR incremental learnings |
| CN109781917A (en) * | 2017-11-14 | 2019-05-21 | 中国科学院大连化学物理研究所 | A kind of biological sample intelligent identification Method based on molecule map |
Non-Patent Citations (3)
| Title |
|---|
| 余元辉，邓莹: "基于多特征融合的海洋生物图像检索方法的研究", 《河南大学学报（自然科学版）》 * |
| 宋晓阳 等: "基于面向对象的高分影像分类研究", 《遥感技术与应用》 * |
| 金朵 等: "基于HIS颜色空间的图像自动标注技术", 《中国科技博览》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112907615A (en) * | 2021-01-08 | 2021-06-04 | 中国石油大学(华东) | Submarine landform unit contour and detail identification method based on region growing |
| CN112907615B (en) * | 2021-01-08 | 2022-07-26 | 中国石油大学(华东) | Submarine landform unit contour and detail identification method based on region growing |
| CN112989086A (en) * | 2021-05-20 | 2021-06-18 | 苏州希格玛科技有限公司 | Intelligent recognition and classification system and method for city management images |
| CN112989086B (en) * | 2021-05-20 | 2022-04-15 | 苏州元澄科技股份有限公司 | Intelligent recognition and classification system and method for city management images |
| CN116758580A (en) * | 2023-05-05 | 2023-09-15 | 中国地质大学（北京） | Benthonic organism recognition method, benthonic organism recognition device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110414368B (en) | Unsupervised pedestrian re-identification method based on knowledge distillation | |
| CN111368690B (en) | Deep learning-based video image ship detection method and system under influence of sea waves | |
| CN110674823A (en) | Sample library construction method based on automatic identification of deep sea large benthonic animals | |
| CN109492706B (en) | Chromosome classification prediction device based on recurrent neural network | |
| CN113011357A (en) | Depth fake face video positioning method based on space-time fusion | |
| CN111027377A (en) | Double-flow neural network time sequence action positioning method | |
| CN110858276A (en) | Pedestrian re-identification method combining identification model and verification model | |
| CN114749342A (en) | Method, device and medium for identifying coating defects of lithium battery pole piece | |
| CN112418028A (en) | Satellite image ship identification and segmentation method based on deep learning | |
| CN114998603A (en) | Underwater target detection method based on depth multi-scale feature factor fusion | |
| CN114781514A (en) | Floater target detection method and system integrating attention mechanism | |
| CN115115863A (en) | Water surface multi-scale target detection method, device and system and storage medium | |
| CN110738166A (en) | Fishing administration monitoring system infrared target identification method based on PCNN and PCANet and storage medium | |
| Zhang et al. | Surveillance video quality assessment based on quality related retraining | |
| CN110688512A (en) | Pedestrian image search algorithm based on PTGAN region gap and depth neural network | |
| CN113326738A (en) | Pedestrian target detection and re-identification method based on deep network and dictionary learning | |
| McLeay et al. | Deep convolutional neural networks with transfer learning for waterline detection in mussel farms | |
| WO2023197827A1 (en) | Fine-grained visual content search platform | |
| CN116977633A (en) | Feature element segmentation model training method, feature element segmentation method and device | |
| CN112308002B (en) | Submarine organism identification and detection method based on single-stage deep learning network | |
| CN114782875A (en) | Fish fine-grained information acquisition method based on fishway construction | |
| CN114898290A (en) | Real-time detection method and system for marine ship | |
| Ge et al. | Real-time object detection algorithm for Underwater Robots | |
| CN113850166A (en) | Ship image identification method and system based on convolutional neural network | |
| CN114581769A (en) | Method for identifying houses under construction based on unsupervised clustering |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200110 |
|
| RJ01 | Rejection of invention patent application after publication |