CN112132884A - Sea cucumber length measuring method and system based on parallel laser and semantic segmentation - Google Patents

Sea cucumber length measuring method and system based on parallel laser and semantic segmentation Download PDF

Info

Publication number
CN112132884A
CN112132884A CN202011054510.9A CN202011054510A CN112132884A CN 112132884 A CN112132884 A CN 112132884A CN 202011054510 A CN202011054510 A CN 202011054510A CN 112132884 A CN112132884 A CN 112132884A
Authority
CN
China
Prior art keywords
sea cucumber
laser
semantic segmentation
length
pixel distance
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011054510.9A
Other languages
Chinese (zh)
Other versions
CN112132884B (en
Inventor
俞智斌
张心亮
曾慧敏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ocean University of China
Original Assignee
Ocean University of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ocean University of China filed Critical Ocean University of China
Priority to CN202011054510.9A priority Critical patent/CN112132884B/en
Publication of CN112132884A publication Critical patent/CN112132884A/en
Application granted granted Critical
Publication of CN112132884B publication Critical patent/CN112132884B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G06T2207/20132—Image cropping
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
    • Y02A40/80—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in fisheries management
    • Y02A40/81—Aquaculture, e.g. of fish

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Geometry (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to the technical field of sea cucumber length measurement, and discloses a sea cucumber length measurement method and system based on parallel laser and semantic segmentation, wherein the method comprises the following steps: two parallel visible laser beams are emitted to irradiate the front, a laser spot is formed on the surface of an object in front, and a camera shoots a front image in real time; cutting the front image to obtain the pixel distance D between the laser pointspAnd a central region of the laser spot; performing semantic segmentation on the central region to obtain a segmented image containing the approximate contour of the sea cucumber, determining the axial direction of the sea cucumber based on the segmented image, and obtaining the pixel distance L of the sea cucumber in the axial directionp(ii) a According to pixel distance between laser pointsFrom DpActual distance D between laser beamsrThe pixel distance L of the sea cucumber in the axial directionpCalculating the true length of the sea cucumber
Figure DDA0002710530970000011
The invention provides a non-contact sea cucumber measuring system and method, which are characterized in that semantic segmentation is carried out by means of a pair of parallel lasers based on a Mask R-CNN model, so that the measuring process is fast and convenient, and the cost is low.

Description

Sea cucumber length measuring method and system based on parallel laser and semantic segmentation
Technical Field
The invention relates to the technical field of sea cucumber length measurement, in particular to a sea cucumber length measurement method and system based on parallel laser and semantic segmentation.
Background
Sea cucumber measurement is a basic requirement of aquaculture. In situ non-contact measurement greatly reduces cost and harmlessness, making this technique an ideal choice for aquaculture monitoring. Two major challenges are presented to developing an in-situ non-contact measurement system. One challenge is the lack of a referenceable object under non-contact conditions. Another challenge is how to detect and locate sea cucumbers in an underwater environment.
Disclosure of Invention
The invention provides a sea cucumber length measuring method and system based on parallel laser and semantic segmentation, and solves the technical problems that: how to detect and position sea cucumbers in an underwater environment in a non-contact manner so as to measure the length of the sea cucumbers simply, quickly and accurately.
In order to solve the technical problems, the invention provides a sea cucumber length measuring method based on parallel laser and semantic segmentation, which comprises the following steps:
s1, emitting two parallel visible laser beams to irradiate the front, forming a laser spot on the surface of an object in front, and shooting a front image in real time by a camera;
s2, cutting the front image to obtain a pixel distance D between laser pointspAnd a central region of the laser spot;
s3, performing semantic segmentation on the central region to obtain a segmented image containing the approximate outline of the sea cucumber, determining the axial direction of the sea cucumber based on the segmented image, and obtaining the pixel distance L of the sea cucumber in the axial directionp；
S4, according to the pixel distance D between the laser pointspActual distance D between laser beamsrThe pixel distance L of the sea cucumber in the axial directionpCalculating the true length of the sea cucumber
Figure BDA0002710530950000021
Further, the step S2 specifically includes the steps of:
s21, separating out a G channel of the front image, and converting the G channel into a binary image with a preset threshold value;
s22, determining the centers of two laser points by taking the contour with the largest and the second largest area in the binary image as the laser points, and further calculating the distance between the two centers, namely the pixel distance D between the laser pointsp；
S23, taking the pixel distance D between laser pointspAnd determining a rectangle with the length and the preset height as the width to be used as a cutting range to cut the front image to obtain a central area of the laser spot.
Further, in the step S21, the setting range of the preset threshold is 250 ± 3; in the step S23, the preset height is 300 ± 50 pixels.
Further, in the step S21, the preset threshold is 250; in the step S23, the preset height is 300 pixels.
Further, in the step S22, the area of each contour is calculated by using a contourArea () function in OpenCV.
Further, in the step S1, the emitted laser beam is green light.
Further, in the step S3, performing semantic segmentation on the central region by using a deep learning network model based on a Mask R-CNN model; and acquiring a circumscribed rectangle of the sea cucumber outline in the segmentation image by adopting a minArea () function in OpenCV, and further determining the axial direction of the sea cucumber through the circumscribed rectangle.
Further, the deep learning network model is trained by taking a backbone network Res-50-FPN, a background cross-over ratio of 0.3, a foreground cross-over ratio of 0.7, a weight attenuation strategy of a Mask R-CNN model and NMS in a Mask R-CNN model ROI as thresholds, and iterates for ten thousand times.
Further, a 1080Ti GPU using a single card was trained for a total of 900,000 iterations with the batch size set to 2.
Based on the method, the invention also provides a sea cucumber length measuring system based on parallel laser and semantic segmentation, which comprises a local cutting module, a segmentation module and a calculation module, wherein the local cutting module, the segmentation module and the calculation module are respectively used for executing the steps S2, S3 and S4 in the method.
The invention provides a sea cucumber length measuring method and system based on parallel laser and semantic segmentation, which have the beneficial effects that:
(1) providing a distance reference using two parallel laser beams to avoid contacting the sea cucumber;
(2) constructing a deep learning network model based on a Mask R-CNN model and training the deep learning network model to detect and segment the sea cucumber in the central area, wherein the sea cucumber contour in the segmented image is more real;
(3) by the pixel distance D between the laser spotspActual distance D between laser beamsrThe pixel distance L of the sea cucumber in the axial directionpCalculating the true length of the sea cucumber
Figure BDA0002710530950000031
The accuracy is better;
(4) the length of the sea cucumber is estimated at quite low cost, and the result is accurately and effectively verified by a large number of experiments.
Drawings
FIG. 1 is a 3D diagram of a laser frame used in a sea cucumber length measuring system based on parallel laser and semantic segmentation provided by an embodiment of the invention;
FIG. 2 is a block diagram of a sea cucumber length measuring system based on parallel laser and semantic segmentation provided by an embodiment of the present invention;
FIG. 3 is a schematic diagram of an image processing process of a local cropping module in a sea cucumber length measuring system based on parallel laser and semantic segmentation provided by an embodiment of the invention;
FIG. 4 is a schematic diagram of a labeling process of a Mask-RCNN data set used in a sea cucumber length measuring system based on parallel laser and semantic segmentation according to an embodiment of the present invention;
FIG. 5 is a comparison diagram of the segmentation effect of the sea cucumber length measuring system and method based on parallel laser and semantic segmentation provided by the embodiment of the invention;
fig. 6 is a schematic image processing process diagram of a sea cucumber length measuring system and method based on parallel laser and semantic segmentation provided by the embodiment of the invention.
Detailed Description
The embodiments of the present invention will be described in detail below with reference to the accompanying drawings, which are given solely for the purpose of illustration and are not to be construed as limitations of the invention, including the drawings which are incorporated herein by reference and for illustration only and are not to be construed as limitations of the invention, since many variations thereof are possible without departing from the spirit and scope of the invention.
Due to its great economic value, the variety of sea cucumbers is always concerned. In order to protect the ecological environment, the minimum legal fishing size is taken as a supervision measure for the sea cucumber fishing industry in many countries. Minimum fishing size limitations also need to meet the economic demands of the fishery industry. To a certain extent, the size of sea cucumbers determines their economic value. These demands have forced attention to the measurement of the length of sea cucumbers. Since 1991, researchers have noted the benefits and difficulties of measuring the size of live sea cucumbers. With the gradual maturity of the deep learning technology, the sea cucumber measuring technology based on vision is also developed. With the benefit of in-situ data set, multiple sea cucumber tracking and localization methods have also been proposed. Recently, with the rapid development of deep learning technology, in-situ sea cucumber detection methods are also proposed. The embodiment aims to provide an in-situ sea cucumber length measuring method and system based on a deep learning method and applying paired portable lasers.
Underwater laser machining is widely used in underwater engineering including cutting, welding, plating, and the like. The underwater laser system has the advantages of light weight, small focal spot and the like. Therefore, many researchers have applied lasers to underwater 3D reconstruction and laser imaging. On the other hand, it is impossible to apply the underwater laser system to a long-distance detection task due to scattering and attenuation of light. Restoring and enhancing source signals from turbid media is another area of relatively explosive research.
Due to the numerous achievements of deep learning in the field of computers, the embodiment adopts deep learning to locate and segment the sea cucumbers. However, unlike the computationally expensive and time consuming underwater laser processing method, the present embodiment first proposes a simple and fast measurement system based on a pair of portable lasers.
In order to determine an underwater in-situ length measuring device, the embodiment needs a reference object to measure the length of the sea cucumber. At the same time, the underwater reference object should also have sufficient accuracy and portability. In view of the above, the present embodiment employs a pair of parallel green laser beams as a reference standard. In order to fix the laser to the underwater robot, the present embodiment employs a gripping bracket to fix the laser. The 3D schematic of the laser mount is shown in fig. 1, after the laser is fixed, the gap is sealed with AB glue, and then the laser together with the mount is fixed to the underwater robot with screws.
As shown in fig. 2, the measurement system based on laser and underwater robot is mainly composed of 3 parts: the device comprises a local cutting module, a segmentation module and a length calculation module.
(1) Local cutting module
The local cutting module is responsible for cutting out the central area of the laser points and calculating the pixel distance between the laser points. In the system of this embodiment, clipping is a necessary part to sort out the pixels between the laser beams, thereby reducing the unnecessary calculation overhead in the subsequent sea cucumber segmentation process.
After capturing a real-time image as in fig. 3(a), the G channel is separated and converted into a binary image with a threshold of 250 (which can be set to a range of 250 ± 3). From the binary map a series of coordinates approximating the green laser spot profile can be obtained. Due to background noise and interference from other objects, the series of coordinates may include approximate contour coordinates of two or more objects, as shown in fig. 3 (b). To solve this problem, the contourArea () function in OpenCV needs to be called first to calculate the area of each contour; considering that the profile of the impurity is relatively small, these regions are sorted by area as shown in fig. 3(c), and the first 2 profiles having the largest area are selected as laser spots.
By means of the two profiles, the centers of the two laser points can be found, and the distance between the two points (the pixel distance D between the laser points) can be calculatedp) As shown in fig. 3 (d). Once fixed on the underwater robot, the true distance between the laser points can be measured. The range between the two laser points is then cut out with the rectangle shown in fig. 1, and the cut out image is then fed into the segmentation module. The width of the rectangle is equal to the distance between two laser points, and the height of the rectangle is fixed to 300 pixels (which can be set to a range of 300 ± 50).
(2) Segmentation module
The module aims to obtain the axial length of the sea cucumber by calculating the number of pixels in the axial direction of the sea cucumber in the image. In order to achieve the goal, the sea cucumber needs to be segmented from the background to obtain a segmented image containing the approximate contour of the sea cucumber. The model used in the segmentation module is based on He et al[1]A Mask R-CNN proposed in 2017 constructs a deep learning network model suitable for the embodiment. The segmentation result of each sample consists of a series of pixel point coordinates which are approximate to the outline of the sea cucumber and form a closed curve. Then, a minArea () function in OpenCV is adopted to obtain a circumscribed rectangle of the outline, the axial direction of the sea cucumber is determined, and the pixel distance L of the sea cucumber in the axial direction can be further determinedp。
And (3) a training process of the deep learning network model. This embodiment is first aligned to the COCO2014 data set[2]Mask R-CNN model with pre-training[1]Fine adjustment is carried out to adapt to the task of sea cucumber segmentation. The backbone network of the model is Res-50-FPN[3]. The background IoU (cross-over ratio) and foreground IoU were 0.3 and 0.7, respectively. In this embodiment, a default weight attenuation strategy of a Mask R-CNN model and NMS in the ROI are used as thresholds. A1080 Ti GPU using a single card was trained, and the network model was iterated 900,000 times in total, with the batch size set to 2.
(3) Length calculation module
Known as laserTrue distance D between beamsrD (distance between lasers), pixel distance D between laser spotsp. Pixel distance L of sea cucumber in axial directionpCan find LrIs the real length of the sea cucumber. The principle of the calculation is that the ratio of the pixel distance in the axial direction of the sea cucumber to the real length of the sea cucumber should be equal to the ratio of the pixel distance between the laser points to the real distance of the laser beams, namely:
Figure BDA0002710530950000061
therefore, the real length of the sea cucumber is as follows:
Figure BDA0002710530950000062
based on the measurement system, the embodiment further provides a sea cucumber length measurement method based on parallel laser and semantic segmentation, which comprises the following steps:
s1, emitting two parallel green laser beams to irradiate the front, forming a laser spot on the surface of an object in front, and shooting a front image in real time by a camera;
s2, cutting the front image to obtain the pixel distance D between the laser pointspAnd a central region of the laser spot;
s3, performing semantic segmentation on the central region by adopting a Mask R-CNN model-based deep learning network model to obtain a segmented image containing the approximate outline of the sea cucumber, acquiring a circumscribed rectangle of the outline of the sea cucumber in the segmented image by adopting a minArea () function in OpenCV, further determining the axial direction of the sea cucumber by the circumscribed rectangle, and obtaining the pixel distance L of the sea cucumber in the axial directionp；
S4, according to the pixel distance D between the laser pointspActual distance D between laser beamsrThe pixel distance L of the sea cucumber in the axial directionpCalculating the true length of the sea cucumber
Figure BDA0002710530950000071
Further, the step S2 specifically includes the steps of:
s21, separating out a G channel of the front image, and converting the G channel into a binary image with a preset threshold value;
s22, calculating the area of each outline in the binary image by using a contourArea () function in OpenCV, determining the centers of two laser points by using the outline with the largest and the second largest area in the binary image as the laser points, and further calculating the distance between the two centers, namely the pixel distance D between the laser pointsp；
S23, taking the pixel distance D between laser pointspAnd determining a rectangle with the length and the preset height as the width to be used as a cutting range to cut the front image to obtain a central area of the laser spot.
Further, in the step S21, the setting range of the preset threshold is 250 ± 3, and the embodiment is preferably 250; in the step S23, the preset height is 300 ± 50 pixels, and the embodiment is preferably 300 pixels.
In step S3, the deep learning network model is trained with the backbone network Res-50-FPN, the background cross-over ratio of 0.3, the foreground cross-over ratio of 0.7, the weight attenuation policy of the Mask R-CNN model, and the NMS in the Mask R-CNN model ROI as thresholds, and a 1080Ti GPU using a single card is trained, wherein the iterations are performed 900,000 times in total, and the batch size is set to 2.
In step S3, a minArea () function in OpenCV is used to obtain a circumscribed rectangle of the sea cucumber contour in the segmented image, and further, the axis direction of the sea cucumber is determined by the circumscribed rectangle.
In addition, the reason why the green beam is selected by the parallel laser in this embodiment is that the green beam has a higher brightness after the G channel of the front image is separated, so that the profile with higher brightness can be retained by the preset threshold value, and the pixel distance between the laser points can be conveniently determined.
It should also be noted that, in other embodiments, in a system or method:
the implementation of the deep learning network model may be based on other transformations made by the Mask R-CNN model, may adopt other background intersection ratio 0.3 and foreground intersection ratio 0.7, may adopt other GPUs to iterate, may iterate less or more times, and the like, which are suitable transformations made on the basis of the content disclosed in this embodiment;
the contourArea () function in the OpenCV used to calculate the area of each contour in the binary map may be replaced with other functions for calculating the area of the contour area;
the minArea () function in the OpenCV used for obtaining the circumscribed rectangle of the sea parameter outline in the segmentation image can be replaced by other functions for calculating the area of the outline area;
the laser beam can be selected to be light of other colors, such as light which is separated from other image areas after the front image is subjected to R channel, G channel or B channel separation, and the position and the contour of the laser spot are further determined by setting a proper threshold value.
The invention provides a sea cucumber length measuring method and system based on parallel laser and semantic segmentation, which have the beneficial effects that:
(1) providing a distance reference using two parallel laser beams to avoid contacting the sea cucumber;
(2) constructing a deep learning network model based on a Mask R-CNN model and training the deep learning network model to detect and segment the sea cucumber in the central area, wherein the sea cucumber contour in the segmented image is more real;
(3) by the pixel distance D between the laser spotspActual distance D between laser beamsrThe pixel distance L of the sea cucumber in the axial directionpCalculating the true length of the sea cucumber
Figure BDA0002710530950000091
The accuracy is better;
(4) the length of the sea cucumber is estimated at quite low cost, and the result is accurately and effectively verified by a large number of experiments.
The system and method of the present embodiment are experimentally verified.
(1) Mask R-CNN dataset
Since there is no disclosed sea cucumber image segmentation dataset, the present embodiment manually labels 193 datasets in the COCO2014 format and divides the 193 datasets into 20 validation sets and 173 training sets. There are only two categories of labels in this dataset: sea cucumber and background. This 193 data set had 29 from URPC games[4]164 pieces of the test pieces were photographed in the test water tank of this example. Of the 20 pictures in the validation set, 9 were from the URPC game and 20 were from the laboratory pool. Two rules are followed when annotating a data set: a) the outline of the sea cucumber is close to the edge of the sea cucumber as much as possible; b) the profile of the sea cucumber does not include the spurs of the sea cucumber. This embodiment is illustrated by providing a visualization diagram 4 in which the left side (a) is taken for a laboratory pool and the right side (b) is from a URPC race. In the embodiment, IoU of Mask R-CNN is set to be 0.5, the accuracy of image segmentation is 0.84, and the accuracy of the positioning frame is 0.832.
(2) Data set of a measurement system
In order to evaluate the measurement system and method, the embodiment collects 7 segments of videos in the experimental pond and cuts the videos into frames as data sets, however, not all the frames are available, because many frames do not contain sea cucumber images, or the sea cucumber is not in the middle of the laser spot, or the sea cucumber cannot be detected by Mask R-CNN. Therefore, the embodiment extracts the image which contains the sea cucumber and is in the middle of the laser point and can be detected by the Mask R-CNN as the final data set of the embodiment. This embodiment illustrates how the valid frame is extracted using table 1. When extracting valid frame frames, three principles are followed: firstly, the image contains sea cucumber; secondly, the sea cucumber is in the middle of the laser spot; the image finally clipped by the region clipping module can be detected by Mask R-CNN. Only if these three conditions are met is the frame considered valid.
TABLE 1 example of valid frames
Rules 1 2 3 4 ……
Comprises sea cucumber Is that Is that Is that Whether or not ……
In the middle of the laser spot Is that Is that Whether or not Whether or not ……
Can be detected Is that Whether or not Whether or not Whether or not ……
Whether it is a valid frame Is that Whether or not Whether or not Whether or not ……
For each video, the present embodiment classifies the extracted effective frames into 3 types of far, medium and near according to the distance between an ROV (underwater robot) and the sea cucumber. According to the pixel distance D between the laser pointspThe ratio to the image width defines the distance. The video resolution of this embodiment is 1920 x 1080, so the ratio is Dp/1920. The far, middle and near 3 types are defined as: dp/1920 is less than or equal to 0.5 and far; d is more than 0.5p/1920 is less than or equal to 0.7, medium; d is more than 0.7pAnd/1920 is less than or equal to 1 and is close. The true length, valid frame and distance of the sea cucumber in each video are recorded in table 2.
TABLE 2 data set of measurement System
Figure BDA0002710530950000101
(3) Method comparison of partitioned modules
The present embodiment uses the average error and variance as evaluation indexes. In the segmentation module, the embodiment tries 2 methods to obtain the pixel length L of the sea cucumberp: (1) and calculating the distance between the two farthest points in the sea cucumber outline by using a convex hull function in OpenCV as the pixel length of the sea cucumber. (2) And obtaining a minimum circumscribed rectangle of the sea cucumber outline by using a minAreaRect () function in OpenCV, and taking the length of the circumscribed rectangle as the pixel length of the sea cucumber. The present embodiment first calculates the average result, average error, and mean variance for each video segment. Then, the effective frame of each video is used as a weight to obtain the weighted average error and the weighted average variance of the two methods. The results of the experiment are reported in table 3. As can be seen from the experimental results, method (2) is significantly more accurate and stable than method (1).
TABLE 3 method comparison of segmentation modules
Figure BDA0002710530950000111
Note: the subscript "1" represents method (1) of the segmentation module, and the subscript "2" represents method (2); "+" indicates that only valid frame measurements were calculated, as in tables 4 and 5.
(4) Method comparison of region cropping modules
In the region cropping module, the present embodiment attempts another simpler and bolder method of locating the laser spot. OpenCV provides a minMaxLoc () function, which returns the position of the pixel with the largest luminance value. The method has only three steps: 1) obtaining a first point with maximum brightness by using a minMaxLoc () function; 2) zeroing 50 pixels around the point; 3) the second brightest pixel is found by minMaxLoc (). Based on this method, table 4 was obtained. One benefit of this approach is that more valid frames can be obtained, but from experimental results it can be seen that while the average error of this clipping method is smaller than table 3, the variance is larger than table 3. In practice, the clipping method has also been found to be highly random and unstable. However, it can still be seen from table 4 that in the segmentation block, method (2) is better than method (1), i.e. both error and variance are lower.
TABLE 4 segmentation module method comparison based on another method of locating laser spots
Figure BDA0002710530950000121
(5) Results and discussion
The visual output results of the measurement system of the present embodiment visualizing the respective modules are recorded in fig. 6. First, the ROV of this embodiment will take an image 6(a) of the sea cucumber. The region cropping module would then locate the laser spot and crop the middle portion of the laser spot into image 6 (b). The cropped image 6(c) is used as input to a segmentation module to obtain a semantic segmentation result 6(d) of the sea cucumber. Finally, the length calculating module of this embodiment calculates the actual length of the sea cucumber according to the segmented contour thereof, as shown in fig. 6 (e).
In practice, the present embodiment finds that it is difficult for the ROV to maintain a stable posture due to the complicated underwater environment, which makes the distance between the sea cucumber and the ROV always variable. While different distances may result in different measurement results. Therefore, the present embodiment classifies the valid frames into the far, medium, and near 3 classes. The results of the experiment are reported in table 5. From the experimental results, it can be found that the distance has little influence on the error, but the closer the distance, the smaller the variance, which means that the result of the close-range measurement is more stable.
TABLE 5 measurement results at different distances
Distance between two adjacent plates Error of the measurement1 Variance (variance)1 Error of the measurement2 Variance (variance)2 Active frame
Far away 11.09％ 78.03 8.42％ 83.49 361
In 11.31％ 107.71 9.45％ 101.724 1520
Near to 11.18％ 65.312 9.59％ 64.516 2599
In summary, the present embodiment provides a system and a method for measuring sea cucumber in a non-contact manner, which perform semantic segmentation based on a Mask R-CNN model by using a pair of portable parallel lasers, so that the measurement process is fast and convenient, and the cost is low. Experimental results in an underwater environment show that the method and the system protected by the embodiment have robustness in length measurement and good performance in detection and segmentation.
Reference documents:
[1]K.He,G.Gkioxari,P.Dollár,and R.Girshick.Mask r-cnn.In Proceedings of the IEEE international con-ference on computer vision,pages 2961–2969,2017.
[2]T.-Y.Lin,M.Maire,S.Belongie,J.Hays,P.Perona,D.Ramanan,P.Dollár,and C.L.Zitnick.Microsoft coco:Common objects in context.In European con-ference on computer vision,pages 740–755.Springer,2014.
[3]T.-Y.Lin,P.Dollár,R.Girshick,K.He,B.Hariharan,and S.Belongie.Feature pyramid networks for object detection.In Proceedings of the IEEE conference on computer vision and pattern recognition,pages 2117–2125,2017.
[4]Urpc competition.http://http://en.cnurpc.org.
the above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.

Claims (10)

1. The sea cucumber length measuring method based on parallel laser and semantic segmentation is characterized by comprising the following steps of:
s1, emitting two parallel visible laser beams to irradiate the front, forming a laser spot on the surface of an object in front, and shooting a front image in real time by a camera;
s2, cutting the front image to obtain a pixel distance D between laser pointspAnd a central region of the laser spot;
s3, performing semantic segmentation on the central region to obtain a segmented image containing the approximate outline of the sea cucumber, determining the axial direction of the sea cucumber based on the segmented image, and obtaining the pixel distance L of the sea cucumber in the axial directionp；
S4, according to the pixel distance D between the laser pointspActual distance D between laser beamsrThe pixel distance L of the sea cucumber in the axial directionpCalculating the true length of the sea cucumber
Figure FDA0002710530940000011
2. The method for measuring the length of the sea cucumber based on the parallel laser and the semantic segmentation as claimed in claim 1, wherein the step S2 specifically comprises the steps of:
s21, separating out a G channel of the front image, and converting the G channel into a binary image with a preset threshold value;
s22, determining the centers of two laser points by taking the contour with the largest and the second largest area in the binary image as the laser points, and further calculating the distance between the two centers, namely the pixel distance D between the laser pointsp；
S23, taking the pixel distance D between laser pointspAnd determining a rectangle with the length and the preset height as the width to be used as a cutting range to cut the front image to obtain a central area of the laser spot.
3. The method for measuring the length of the sea cucumber based on the parallel laser and the semantic segmentation as claimed in claim 2, wherein: in the step S21, the setting range of the preset threshold is 250 ± 3; in the step S23, the preset height is 300 ± 50 pixels.
4. The sea cucumber length measuring method based on parallel laser and semantic segmentation as claimed in claim 3, wherein: in the step S21, the preset threshold is 250; in the step S23, the preset height is 300 pixels.
5. The sea cucumber length measuring method based on parallel laser and semantic segmentation as claimed in claim 3, wherein: in the step S22, the area of each contour is calculated by using a contourArea () function in OpenCV.
6. The method for measuring the length of the sea cucumber based on the parallel laser and the semantic segmentation as claimed in claim 1, wherein: in the step S1, the emitted laser beam is green light.
7. The sea cucumber length measuring method based on parallel laser and semantic segmentation as claimed in any one of claims 1-6, characterized in that: in the step S3, performing semantic segmentation on the central region by using a deep learning network model based on a Mask R-CNN model; and acquiring a circumscribed rectangle of the sea cucumber outline in the segmentation image by adopting a minArea () function in OpenCV, and further determining the axial direction of the sea cucumber through the circumscribed rectangle.
8. The sea cucumber length measuring method based on parallel laser and semantic segmentation as claimed in claim 7, wherein: the deep learning network model is trained by taking a backbone network Res-50-FPN, a background cross-over ratio of 0.3, a foreground cross-over ratio of 0.7, a weight attenuation strategy of a Mask R-CNN model and NMS in a Mask R-CNN model ROI as thresholds, and iterates for ten thousand times.
9. The sea cucumber length measuring method based on parallel laser and semantic segmentation as claimed in claim 8, wherein: a1080 Ti GPU using a single card was trained for a total of 900,000 iterations with the batch size set to 2.
10. Sea cucumber length measurement system based on parallel laser and semantic segmentation, its characterized in that: the method comprises a local clipping module, a segmentation module and a calculation module, which are respectively used for executing the steps S2, S3 and S4 of any claim from 1 to 9.
CN202011054510.9A 2020-09-29 2020-09-29 Sea cucumber length measurement method and system based on parallel laser and semantic segmentation Active CN112132884B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011054510.9A CN112132884B (en) 2020-09-29 2020-09-29 Sea cucumber length measurement method and system based on parallel laser and semantic segmentation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011054510.9A CN112132884B (en) 2020-09-29 2020-09-29 Sea cucumber length measurement method and system based on parallel laser and semantic segmentation

Publications (2)

Publication Number Publication Date
CN112132884A true CN112132884A (en) 2020-12-25
CN112132884B CN112132884B (en) 2023-05-05

Family

ID=73843222

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011054510.9A Active CN112132884B (en) 2020-09-29 2020-09-29 Sea cucumber length measurement method and system based on parallel laser and semantic segmentation

Country Status (1)

Country Link
CN (1) CN112132884B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113344936A (en) * 2021-07-02 2021-09-03 吉林农业大学 Soil nematode image segmentation and width measurement method based on deep learning
CN117854116A (en) * 2024-03-08 2024-04-09 中国海洋大学 Sea cucumber in-situ length measurement method based on Bezier curve

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107928675A (en) * 2017-11-22 2018-04-20 王华锋 A kind of trunk measuring method being combined based on deep learning and red dot laser
CN110147763A (en) * 2019-05-20 2019-08-20 哈尔滨工业大学 Video semanteme dividing method based on convolutional neural networks
CN110738207A (en) * 2019-09-10 2020-01-31 西南交通大学 character detection method for fusing character area edge information in character image
CN110738642A (en) * 2019-10-08 2020-01-31 福建船政交通职业学院 Mask R-CNN-based reinforced concrete crack identification and measurement method and storage medium
CN110838100A (en) * 2019-10-11 2020-02-25 浙江大学 Colonoscope pathological section screening and segmenting system based on sliding window
CN111652179A (en) * 2020-06-15 2020-09-11 东风汽车股份有限公司 Semantic high-precision map construction and positioning method based on dotted line feature fusion laser

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107928675A (en) * 2017-11-22 2018-04-20 王华锋 A kind of trunk measuring method being combined based on deep learning and red dot laser
CN110147763A (en) * 2019-05-20 2019-08-20 哈尔滨工业大学 Video semanteme dividing method based on convolutional neural networks
CN110738207A (en) * 2019-09-10 2020-01-31 西南交通大学 character detection method for fusing character area edge information in character image
CN110738642A (en) * 2019-10-08 2020-01-31 福建船政交通职业学院 Mask R-CNN-based reinforced concrete crack identification and measurement method and storage medium
CN110838100A (en) * 2019-10-11 2020-02-25 浙江大学 Colonoscope pathological section screening and segmenting system based on sliding window
CN111652179A (en) * 2020-06-15 2020-09-11 东风汽车股份有限公司 Semantic high-precision map construction and positioning method based on dotted line feature fusion laser

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
KAIMING HE 等: "Mask R-CNN", 《ARXIV》 *
李龙星 等: "基于CMOS摄像头和平行激光束测距系统设计", 《山西电子技术》 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113344936A (en) * 2021-07-02 2021-09-03 吉林农业大学 Soil nematode image segmentation and width measurement method based on deep learning
CN117854116A (en) * 2024-03-08 2024-04-09 中国海洋大学 Sea cucumber in-situ length measurement method based on Bezier curve
CN117854116B (en) * 2024-03-08 2024-05-17 中国海洋大学 Sea cucumber in-situ length measurement method based on Bezier curve

Also Published As

Publication number Publication date
CN112132884B (en) 2023-05-05

Similar Documents

Publication Publication Date Title
US11488308B2 (en) Three-dimensional object detection method and system based on weighted channel features of a point cloud
US11763485B1 (en) Deep learning based robot target recognition and motion detection method, storage medium and apparatus
JP2020038658A (en) Learning method and learning device for detecting object by using edge image, and test method and test device using the same
CN110687904A (en) Visual navigation routing inspection and obstacle avoidance method for inspection robot
CN112598713A (en) Offshore submarine fish detection and tracking statistical method based on deep learning
US8396285B2 (en) Estimating vanishing points in images
CN110298281B (en) Video structuring method and device, electronic equipment and storage medium
WO2019071976A1 (en) Panoramic image saliency detection method based on regional growth and eye movement model
Trouvé et al. Single image local blur identification
CN110827312B (en) Learning method based on cooperative visual attention neural network
CN112132884B (en) Sea cucumber length measurement method and system based on parallel laser and semantic segmentation
CN111695373B (en) Zebra stripes positioning method, system, medium and equipment
CN115909268A (en) Dynamic obstacle detection method and device
CN114972333A (en) Road crack detection method and system based on artificial intelligence
CN113128430B (en) Crowd gathering detection method, device, electronic equipment and storage medium
CN112991159B (en) Face illumination quality evaluation method, system, server and computer readable medium
Wietrzykowski et al. Stereo plane R-CNN: Accurate scene geometry reconstruction using planar segments and camera-agnostic representation
CN106446832B (en) Video-based pedestrian real-time detection method
CN112818983A (en) Method for judging character inversion by using picture acquaintance
CN116883483A (en) Fish body measuring method based on laser camera system
CN117058232A (en) Position detection method for fish target individuals in cultured fish shoal by improving YOLOv8 model
CN115995017A (en) Fruit identification and positioning method, device and medium
Juujarvi et al. Digital-image-based tree measurement for forest inventory
CN114140428A (en) Method and system for detecting and identifying larch caterpillars based on YOLOv5
CN114494165A (en) Clustering-based light bar extraction method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant