CN112287913B - Intelligent supervisory system for fish video identification - Google Patents
Intelligent supervisory system for fish video identification Download PDFInfo
- Publication number
- CN112287913B CN112287913B CN202011564844.0A CN202011564844A CN112287913B CN 112287913 B CN112287913 B CN 112287913B CN 202011564844 A CN202011564844 A CN 202011564844A CN 112287913 B CN112287913 B CN 112287913B
- Authority
- CN
- China
- Prior art keywords
- fish
- video
- clustering
- color
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 241000251468 Actinopterygii Species 0.000 title claims abstract description 145
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 29
- 238000000034 method Methods 0.000 claims abstract description 15
- 238000012544 monitoring process Methods 0.000 claims abstract description 14
- 238000009826 distribution Methods 0.000 claims abstract description 10
- 230000011218 segmentation Effects 0.000 claims abstract description 7
- 239000013598 vector Substances 0.000 claims description 16
- 230000005484 gravity Effects 0.000 claims description 6
- 230000000007 visual effect Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 239000003086 colorant Substances 0.000 claims description 5
- 230000006399 behavior Effects 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 4
- 230000033001 locomotion Effects 0.000 claims description 4
- 238000013528 artificial neural network Methods 0.000 claims description 3
- 235000014510 cooky Nutrition 0.000 claims description 3
- 238000013135 deep learning Methods 0.000 claims description 3
- 230000002452 interceptive effect Effects 0.000 claims description 3
- 238000010801 machine learning Methods 0.000 claims description 3
- 238000003860 storage Methods 0.000 claims description 3
- 238000012360 testing method Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 claims description 2
- 238000005070 sampling Methods 0.000 claims description 2
- 238000000926 separation method Methods 0.000 claims description 2
- 238000005286 illumination Methods 0.000 claims 1
- 230000018109 developmental process Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 7
- 238000011161 development Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 238000007726 management method Methods 0.000 description 6
- 238000013461 design Methods 0.000 description 5
- 238000010276 construction Methods 0.000 description 3
- 230000010354 integration Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000013439 planning Methods 0.000 description 1
- 230000033772 system development Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
- G06V10/507—Summing image-intensity values; Histogram projection analysis
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention provides an intelligent monitoring system for fish video identification, which comprises a service layer, a data layer and a presentation layer, wherein the service layer comprises a user system, a service system and an algorithm system; the service system is used for managing the services among the whole platform modules; the algorithm system realizes video identification of the fish through a fish model clustering algorithm. According to the method, a fish model is fixed at the origin of a coordinate system, and an independent fish clustering color library is constructed by utilizing the gray distribution statistics of fish schools and background targets according to the characteristics of color clustering and fish videos; clustering time is reduced, and efficiency is improved; the clustering value is determined in a self-adaptive mode through the valley value of the normalized histogram, and therefore the low efficiency and invalid segmentation caused by manual experience are avoided.
Description
Technical Field
The invention belongs to the field of intelligent supervision systems, and particularly relates to an intelligent supervision system for fish video identification.
Background
Currently, big data has become a fundamental and strategic resource for national economy and social development. With the continuous development of information acquisition technology, various basic data such as regional resource environment background data, management service data, monitoring data and the like rapidly increase, and big data characteristics are gradually presented. The demands of management departments at all levels on real-time and visual display and analysis of big data are increasing. And the method also puts more comprehensive and urgent requirements on the expansion, mining and application of the space resource big data. The current various service management systems are independent of each other, lack integration and shared utilization of information resources, serious 'information isolated island' phenomenon, insufficient deep data application, imperfect data updating mechanism and the like.
Fish classification has been more and more emphasized for decades, is vital to reasonable regulation and control of fish schools, effectively and accurately extracts and classifies fish, is essential for analyzing and identifying the relation between fish behavior characteristics and environmental factors and accurately controlling fish growth environments, is based on artificial identification at present, and lacks an effective algorithm identification means for fish videos.
Disclosure of Invention
In order to solve the problems, particularly to identify the video of the fishes, the invention utilizes a GIS technology and an area projection outline and Cartesian projection profile feature extraction technology to identify the habitat of the fishes and further identify the fishes, and the specific scheme is as follows:
an intelligent monitoring system for fish video identification comprises a service layer, a data layer and a display layer,
the service layer comprises three systems, namely a user system, a service system and an algorithm system, wherein the user system is mainly used for managing platform user operation behaviors and information management; the service system is used for managing the services among the whole platform modules; the algorithm system realizes video identification of the fish through a fish model clustering algorithm;
the data layer is used for storing data and is divided into a data center, a system database and a video database, and the data center is used for storing various service data including the number, date, position and the like of fish identification; the system database stores service relation data among system modules, including maps, video storage addresses and the like; the video database stores video data of all fishes and remote sensing map data;
the display layer outputs the interactive returned result among the functional modules through the WEB end, and the developer of the open API interface calling method can call according to the provided calling rule through the related open interface address.
The service system obtains the fish video through the video extraction equipment, and the algorithm system is used as a background system to realize fish video identification through a fish video identification method.
The system service adopts a lightweight FlaskWeb application framework, a WSGI tool box adopts Werkzeug, a Flask has a built-in server and unit test, adapts RESTful and supports safe cookies; a machine deep learning algorithm Keras artificial neural network and an OpenCV machine learning visual algorithm are used for capturing a dynamic video in real time for recognition; and the data video is automatically acquired, so that accurate and intelligent identification is realized.
The fish model clustering algorithm comprises the following steps:
step 1, regarding the fish as a three-dimensional model of a closed curved surface formed by a plurality of polygonal or triangular meshes, fixing the three-dimensional model at the origin of a coordinate system, regarding a camera as a point, emitting straight lines with various spatial angles, wherein the straight lines are light beam models of the camera, the straight lines emitted by the camera are intersected with the triangular meshes of the three-dimensional fish model, assuming that a straight line passes through the meshes at the centroid, and calculating the backscattering intensity of each mesh on the surface of the fish, wherein the backscattering intensity is expressed by brightness;
step 2, building a three-dimensional fish model;
step 3, setting coordinates;
step 4, calculating the backscattering intensity;
step 5, establishing a model;
step 6, reading the fish model file and moving the fish model file to the origin;
step 7, moving the fish model to a target point through rotation and parallel movement;
step 8, checking the gravity centers and normal vectors of all grids, and judging whether the gravity centers are in the light beam range and meet the vector limitation; if yes, entering step 9, otherwise, not saving and exiting;
step 9, storing the barycentric coordinates, the distance from the barycentric coordinates to the original point and the angle from the barycentric coordinates to the original point, judging whether the cycle times reach a preset number, if not, returning to the step 8, and if so, entering the step 10;
step 10, storing information of all selected grids, and drawing a 2D video;
step 11, color clustering segmentation;
step 12, establishing a fish color library;
step 13, mean value clustering;
and step 14, fish school segmentation.
The invention has the beneficial effects that:
the invention has certain advancement, foresight and expandability on design ideas, system architecture, adopted technology and selected platform. Advancement is one of the main goals of system construction. The advancement is mainly represented by: on the basis of fully understanding and mastering the development trend of information technology and adopting the current advanced database technology, the technology such as data exchange among distributed databases, multi-source heterogeneous data integration and the like is realized, the data maintenance cost is reduced, the data management efficiency is improved, and the system can represent the mainstream development direction of fishery production safety environment guarantee application.
Therefore, the selected software platform is not only an advanced product mature at the present stage, but also a mainstream of international like products, and accords with the development direction in the future; in the software development concept, the system must be designed, managed and developed strictly according to the standards of software engineering and object-oriented theory, and the high starting point of system development is ensured.
The invention provides a simple and convenient operation mode and a visual operation interface by fully considering the convenience and flexibility of application and maintenance, so that a user can easily master and use the operation mode and the visual operation interface. Many software systems often have a contradiction between powerful functions and easy use, i.e., the powerful software with complete functions is often difficult to master because of too many menus; on the contrary, the functions of the software which is easy to use are not perfect. The system should overcome the above two tendencies, and achieve easy use and strong function.
The invention establishes and sets scientific and reasonable data standards, sets and perfects related data operation technical rules, ensures the compatibility and openness of basic geographic data, improves the interoperability of the data level and can effectively support and expand the data platform service.
The invention has flexible and convenient secondary development interface, and can customize service based on components to ensure the expandable capability of the system. The concrete points are as follows: in order to meet the demands of users on system capacity expansion and application range expansion in the future, the system should fully consider the function expansion from the aspects of system structure, function design, management objects and the like; upgrading software: the system should fully consider the scalability and load balancing mechanisms of the platform. The system has flexible and smooth expansion capability; the system is designed and developed by adopting the current popular technology, and the module encapsulation of the service logic is realized, so that the system has excellent reconfigurable capability and extensible capability.
The invention designs and develops the data resource sharing and data security and confidentiality relation which follows the principles of security, confidentiality and sharing. The design of the project database fully considers the overall design and planning of fishery production safety environment guarantee informatization construction, and data sharing with all relevant departments and units is guaranteed on the premise of safety and confidentiality.
The invention fixes the fish model at the origin of the coordinate system, can detect the change of the backscattering intensity of each part of the fish body through the brightness along with the motion of the camera in the space, generates a plane simulation video when the fish is underwater at various positions and angles, and provides the possibility of comparing the simulation video with the plane real video. Based on a K mean + + algorithm, a self-adaptive fast clustering fish shoal color image segmentation algorithm is provided. And a new image is generated to replace the original image through channel color compensation, so that a large amount of complex noise is reduced. Aiming at the characteristics of color clustering and fish videos, an independent fish clustering color library is constructed by utilizing the gray level distribution statistics of fish schools and background targets; clustering time is reduced, and efficiency is improved; the clustering value is determined in a self-adaptive mode through the valley value of the normalized histogram, so that the low efficiency and invalid segmentation caused by manual experience are avoided; compared with other methods, the quality and the accuracy are obviously improved; the algorithm furthest retains the color information of the fish school and eliminates a large amount of irrelevant noise.
Drawings
FIG. 1 is a flow chart of a fish model clustering algorithm method of the present invention.
Detailed Description
The invention is further described with reference to the following figures and examples.
An embodiment of the present invention is illustrated with reference to fig. 1.
An intelligent monitoring system for fish video identification comprises a service layer, a data layer and a display layer,
the service layer comprises three systems, namely a user system, a service system and an algorithm system, wherein the user system is mainly used for managing platform user operation behaviors and information management; the service system is used for managing the services among the whole platform modules; the algorithm system realizes video identification of the fish through a fish model clustering algorithm;
the data layer is used for storing data and is divided into a data center, a system database and a video database, and the data center is used for storing various service data including the number, date, position and the like of fish identification; the system database stores service relation data among system modules, including maps, video storage addresses and the like; the video database stores video data of all fishes and remote sensing map data;
the display layer outputs the interactive returned result among the functional modules through the WEB end, and the developer of the open API interface calling method can call according to the provided calling rule through the related open interface address.
The service system obtains the fish video through the video extraction equipment, and the algorithm system is used as a background system to realize fish video identification through a fish video identification method.
The system service adopts a lightweight FlaskWeb application framework, a WSGI tool box adopts Werkzeug, a Flask has a built-in server and unit test, adapts RESTful and supports safe cookies; a machine deep learning algorithm Keras artificial neural network and an OpenCV machine learning visual algorithm are used for capturing a dynamic video in real time for recognition; and the data video is automatically acquired, so that accurate and intelligent identification is realized.
The fish model clustering algorithm comprises the following steps:
step 1, regarding the fish as a three-dimensional model of a shxiangji closed curved surface formed by a plurality of polygonal or triangular meshes, fixing the three-dimensional model at the origin of a coordinate system, regarding a camera as a point, emitting straight lines of various spatial angles, wherein the straight lines are a light beam model of the camera, intersecting the straight lines emitted by the camera with the triangular meshes of the three-dimensional fish model, assuming that a straight line passes through the meshes at the centroid, and calculating the backscattering intensity of each mesh on the surface of the fish, wherein the backscattering intensity is expressed by brightness;
step 2, the construction of a three-dimensional fish model,
the model is a dense point cloud model of fish, the coordinates of all dense cloud points are derived into txt files, a three-dimensional model of the fish consisting of triangular meshes is obtained after mesh processing, the resolution of the three-dimensional model is about 2 mm, normal vectors and coordinates of 3 vertexes of each mesh and the number of the meshes are derived into stl files, and the stl files are read and processed through a program;
step 3, setting the coordinates,
setting the center of the fish model to the origin of the coordinate system, comparing the x, y and z coordinates of all vertexes to obtain xmin、xmax、ymin、ymax、zmin、zmaxCoordinate x of center of fish modelcenter、ycenter、zcenterThe calculation formula is as follows:
wherein the new coordinates
All vertices of the fish model replace the old coordinates
By the following formula, the center of the fish model will move to the origin,
the fish model is parallel to the x-axis, the calculation should be from right to left, x, y, z represent transformed coordinates, alpha, beta, gamma represent the angle of rotation of the model around the x, y, z-axis, when the fish is parallel to the x-axis, the values of alpha, beta, gamma are adjusted and fixed,
moving around the fish model by adjusting the values of phi and theta, P representing the vector from the centroid of the mesh to the camera, N being the normal vector to the mesh, and P and N having coordinates of
R represents the distance from the origin, and the values of phi and theta are changed by controlling the position of the camera
The angle δ between the two vectors P and N is:
，
when in use
When the fish is in a fish-;
step 4, calculating the back scattering intensity,
when the light beam meets different objects, the light beam is back-scattered, the surface of the fish skin is rough, the roughness of the fish skin is less than the wavelength, and the back-scattering intensity S is determined according to the Lambert ruleBComprises the following steps:
wherein mu is the scattering coefficient of the fish skin;
step 5, establishing a model,
calculating the backscattering intensity of each grid on the surface of the three-dimensional fish model according to the included angle delta, wherein the backscattering intensity is represented by grid brightness, the brighter the grid is, the fish model is fixed at the original point, the camera is moved around the fish model, and the position of the camera is adjusted by changing the space angles phi and theta so that the backscattering intensity on the surface of the fish model is changed; since the real video of the camera is a flat video, the new simulator must be able to generate a flat simulated video for comparison and matching with the real video, the camera is considered as a point and fixed at the origin, emitting a 30 x 3 light beam, when the fish moves to the range of this light beam, a flat simulated video of the fish will be formed, the resolution of the simulated video can be adjusted;
step 6, reading the fish model file and moving the fish model file to the origin;
step 7, moving the fish model to a target point through rotation and parallel movement;
step 8, checking the gravity centers and normal vectors of all grids, and judging whether the gravity centers are in the light beam range and meet the vector limitation; if yes, entering step 9, otherwise, not saving and exiting;
step 9, storing the barycentric coordinates, the distance from the barycentric coordinates to the original point and the angle from the barycentric coordinates to the original point, judging whether the cycle times reach a preset number, if not, returning to the step 8, and if so, entering the step 10;
step 10, storing information of all selected grids, and drawing a 2D video;
step 11, the color clustering segmentation is carried out,
the cluster is composed of objects with close Euclidean spatial distance, and takes compactness and independence as a final target, and pixel groups in the video are assumed to be represented as follows:
where N is the total number of pixels,
randomly selecting k clustering centroids
D of color similarity of two pixels without considering spatial position of a pointijThe formula is calculated according to the principle of similar color values as follows:
wherein, OiRepresents a sample xiAnd the closest point, λ, between k cluster centersjThe method is a guess value of the centers of the same type of samples, and each point is replaced by a clustering center point after clustering is finished;
step 12, establishing a fish color library,
analyzing the influence of different channels on video brightness and contrast, randomly selecting a plurality of fish videos, carrying out channel separation operation on the fish videos, carrying out statistics on the average value of gray scale distribution of the different channels, generating an average normalized histogram aiming at the gray scale video of the R, G, B channel, obtaining the brightness and contrast characteristics of the different channels, and expressing the overall brightness by the peak value position of the histogram;
constructing a fish shoal color library L according to the brightness contributions of different channelsiI =0,1,2, the fish school color library comprises three parts, l0The method is characterized in that only brightness information of an R channel video is reserved in a fish school color library; l1Only the brightness information of the G channel video is reserved; l2Only the brightness information of the B channel video is reserved; wherein l1The range of gray values in the library is 0, 255]The gray level compensation for the remaining two channels is zero, and the table below shows l1Colors in the libraryInformation, wherein R _ P represents the gray value of the R channel pixel, G _ P represents the value of the G channel, and B _ P represents the B channel;
TABLE 1 color information base
Step 13, mean value clustering, which comprises the following steps:
step 13.1, randomly selecting k samples from the data set as initial clustering centers;
wherein,
represents the center of a cluster that is randomly selected,
a set representing a cluster center;
step 13.2, for each sample x in the datasetiCalculating the distances from the k clustering centers to the k clustering centers, and dividing the distances into categories corresponding to the clustering centers with the minimum distances;
step 13.3, for each OiThe class, its cluster center is calculated as follows:
wherein the sample set is xiN, N being the total number of pixels;
step 13.4, returning to step 13.2 until the clustering center is not changed any more;
step 14, the fish school is divided into a plurality of fish schools,
step 14.1, generating a characteristic video;
in the RGB color space, the channel with the maximum average brightness in the three channels of videos is taken as a target channel, other channels with lower brightness are compensated to be zero, the generated new video replaces the original video, and three components of the color video are represented in a vector form:
wherein
Representing the original video, MiIs a different compensation operation, M0Indicating that the brightness information of the R channel is reserved, and the rest is zero; miIndicating that the brightness information of the G channel is reserved, and the rest is zero; indicating that the brightness information of the B channel is reserved, and the rest is zero;
step 14.2, determining a clustering value;
step 14.3, selecting a color library;
by pairs
Analyzing the brightness of different channels, and generating a new color compensation video Seg (x, y) to replace an original video by adopting different color compensation methods; miRepresents different compensation operations, where i =0,1, 2;
if i =0, it means "onOver M0Color compensation of the operation, the resulting video will be color clusters of the R library,
if i =1, indicates a pass through M1Color compensation of the operation, the generated video is color clustering through a G library,
if i =2, indicates a pass through M2Color compensation of the operation, the generated video is clustered by colors of a B library,
step 14.4, clustering the colors of the videos,
according to the determined clustering value K and the fish color library as LiI =0,1,2, clustering the video, from a cluster color library LiRandomly selecting a sample point as an initial clustering center OiAnd the remaining cluster centers satisfy the following condition:
calculate each sample point at LiAnd OiThe probability is as follows:
wherein, D (L)i) Selecting a point with the highest probability as the next initial clustering center from the distance from each sampling point to the nearest center, and repeating the step 14.4 until k initial centers are selected; lyIs a sample point located in the color library; according to k initialsAnd the heart segments the video image to further obtain the number of fishes in the video.
Wherein, the step 14.2 specifically comprises the following steps:
step 14.2.1, the gray interval is set,
setting the gray level video of the fish school after color compensation as gradient (x, y), wherein (x, y) is the space coordinate of the pixel, comprising
The video is divided into T gray intervals according to the gray levels to form a new pixel space R' represented as:
for each small gray interval, the number of pixels is:
r(ci,cj)=║Segray(x,y,ci)+ Segray(x,y,ci+1)+…+ Segray(x,y,cj)║ (i<j)，
wherein R' is the new pixel space, | represents ciAnd cjThe norm of the sum of the number of pixels in the gray scale interval of (2), Segray (x, y, c)i) Means that the gray value is cjIs the spatial coordinate of the pixel, r (c)i,cj) Denotes ciAnd cjThe number of pixels between the gray scale intervals of (a);
step 14.2.2, traversing the pixels, calculating the total number of pixels in each gray scale interval, and taking any rtE to R, calculating the total number of pixels in the gray scale interval according to the formula, calculating the percentage of the pixels in each gray scale interval to the total number of the pixels, and using ptDenotes, T =1, 2, 3.. T,
wherein (x, y) is the spatial coordinate of the pixel and (c)i,cj) Denotes ciAnd cjIn (2) pixel assemblyNumber, N is the total number of pixels in the video, t is a positive integer;
step 14.2.3, sorting the pixel distribution probability value of each gray scale interval in step 14.2.2, and setting the reference probability value as p0By traversing the probability value of each pixel distribution, a probability distribution value p satisfying the following equation is foundt:
The clustering value K is determined as follows:
wherein (x, y) is the spatial coordinate of the pixel and (c)i,cj) Denotes ciAnd cjN is the total number of pixels in the video, and is the reference probability value.
The above-described embodiment merely represents one embodiment of the present invention, but is not to be construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention.
Claims (10)
1. An intelligent monitoring system for fish video identification, which comprises a service layer, a data layer and a display layer, and is characterized in that,
the service layer consists of three systems, namely a user system, a service system and an algorithm system, wherein the user system is used for managing platform user operation behaviors and information management; the service system is used for managing the services among the whole platform modules; the algorithm system realizes video identification of the fish through a fish model clustering algorithm;
the data layer is used for storing data and is divided into a data center, a system database and a video database, and the data center is used for storing various service data including the number, date and position of fish identification; the system database stores service relation data among system modules, including maps and video storage addresses; the video database stores video data of all fishes and remote sensing map data;
the display layer outputs the interactive returned result among the functional modules through a WEB end, and an open API interface calling method developer can call according to the provided calling rule through a related open interface address;
the service system acquires fish videos through video extraction equipment, and the algorithm system serves as a background system to realize fish video identification through a fish video identification method;
the system service adopts a lightweight FlaskWeb application framework, a WSGI tool box adopts Werkzeug, a Flask has a built-in server and unit test, adapts RESTful and supports safe cookies; a machine deep learning algorithm Keras artificial neural network and an OpenCV machine learning visual algorithm are used for capturing a dynamic video in real time for recognition; automatically acquiring a data video to realize accurate and intelligent identification;
the fish model clustering algorithm comprises the following steps:
step 1, regarding the fish as a three-dimensional model of a closed curved surface formed by a plurality of polygonal or triangular meshes, fixing the three-dimensional model at the origin of a coordinate system, regarding a camera as a point, emitting straight lines with various spatial angles, wherein the straight lines are light beam models of the camera, the straight lines emitted by the camera are intersected with the triangular meshes of the three-dimensional fish model, assuming that a straight line passes through the meshes at the centroid, and calculating the backscattering intensity of each mesh on the surface of the fish, wherein the backscattering intensity is expressed by brightness;
step 2, building a three-dimensional fish model;
step 3, setting coordinates;
step 4, calculating the backscattering intensity;
step 5, establishing a model;
step 6, reading the fish model file and moving the fish model file to the origin;
step 7, moving the fish model to a target point through rotation and parallel movement;
step 8, checking the gravity centers and normal vectors of all grids, judging whether the gravity centers are in the light beam range and meet the vector limitation, if so, entering step 9, otherwise, not storing and exiting;
step 9, storing the barycentric coordinates, the distance from the barycentric coordinates to the original point and the angle from the barycentric coordinates to the original point, judging whether the cycle times reach a preset number, if not, returning to the step 8, and if so, entering the step 10;
step 10, storing information of all selected grids, and drawing a 2D video;
step 11, color clustering segmentation;
step 12, establishing a fish color library;
step 13, mean value clustering;
and step 14, fish school segmentation.
2. The intelligent monitoring system for fish video identification according to claim 1, wherein the step 2 is specifically: the model is a dense point cloud model of fish, the coordinates of all dense cloud points are derived into txt files, after grid processing, a three-dimensional model of the fish consisting of triangular grids is obtained, the resolution of the three-dimensional model is 2 mm, normal vectors and coordinates of 3 vertexes of each grid and the grid number are derived into stl files, and the files are read and processed through a program.
3. The intelligent monitoring system for fish video identification according to claim 1, wherein step 3 specifically comprises:
setting the center of the fish model to the origin of the coordinate system, comparing the X, y and z coordinates of all vertexes to obtain Xmin、Xmax、Ymin、Ymax、Zmin、ZmaxCoordinate X of center of fish modelcenter、Ycenter、ZcenterThe calculation formula is as follows:
wherein the new coordinates
、
、
All vertices of the fish model replace the old coordinates
、
、
By the following formula, the center of the fish model will move to the origin,
the fish model is parallel to the X-axis, the calculation should be from right to left, X, Y, Z represent transformed coordinates, alpha, beta, gamma represent the angle of rotation of the model around the X, Y, Z-axis, when the fish is parallel to the X-axis, the values of alpha, beta, gamma are adjusted and fixed,
by adjusting
And
is moved around the fish model, P represents the vector from the centroid of the mesh to the camera, N is the normal vector to the mesh, and the coordinates of P and N are
，
R represents the distance to the origin, modified by controlling the position of the camera
And
the value of (a) is,
angle between two vectors P and N
Comprises the following steps:
，
when in use
The backscatter intensity of the grid will not be calculated, and the surface of the fish opposite the camera, the illumination beam cannot reach.
4. According to claim 3The intelligent monitoring system for fish video identification is characterized in that the step 4 specifically comprises the following steps: calculation of backscattering intensity: when the light beam meets different objects, the light beam is back-scattered, the surface of the fish skin is rough, the roughness of the fish skin is less than the wavelength, and the back-scattering intensity is determined according to the Lambert rule
Comprises the following steps:
wherein,
the scattering coefficient of the fish skin is shown.
5. The intelligent monitoring system for fish video identification according to claim 3, wherein the step 5 specifically comprises: according to the included angle
Calculating the backscattering intensity of each grid on the surface of the three-dimensional fish model, wherein the backscattering intensity is represented by grid brightness, the higher the backscattering intensity is, the brighter the grid is, fixing the fish model at the original point, moving a camera around the fish model, and changing the space angle
And
the position of the camera is adjusted to change the backscattering intensity of the surface of the fish model; since the real video of the camera is a flat video, the new simulator must be able to generate a flat simulated video for comparison and matching with the real video, the camera being considered as a point and fixed at the origin, emitting a signal30 x 3, when the fish moves to the beam range, a planar analog video of the fish will be formed, and the resolution of the analog video can be adjusted.
6. The intelligent monitoring system for fish video identification according to claim 1, wherein step 11 specifically comprises: the cluster is composed of objects with close Euclidean spatial distance, and takes compactness and independence as a final target, and pixel groups in the video are assumed to be represented as follows:
where N is the total number of pixels,
randomly selecting k clustering centroids
The colour similarity of two pixels being independent of the spatial position of the point
The formula is calculated according to the principle of similar color values as follows:
(ii) a The following formula is satisfied:
wherein,
represents a sample xiAnd the closest point between the k cluster centers,
is the guess value of the center of the same type of sample, and each point is replaced by the center point of the cluster after the clustering is finished.
7. The intelligent monitoring system for fish video identification according to claim 1, wherein step 12 specifically comprises:
analyzing the influence of different channels on video brightness and contrast, randomly selecting a plurality of fish videos, carrying out channel separation operation on the fish videos, carrying out statistics on the average value of gray scale distribution of the different channels, generating an average normalized histogram aiming at the gray scale video of the R, G, B channel, obtaining the brightness and contrast characteristics of the different channels, and expressing the overall brightness by the peak value position of the histogram;
constructing a fish shoal color library according to the brightness contributions of different channels as
The fish color library comprises three parts, L0The method is characterized in that only brightness information of an R channel video is reserved in a fish school color library; l is1Only the brightness information of the G channel video is reserved; l is2Only the brightness information of the B channel video is reserved; wherein,
the range of gray values in the library is 0, 255]The gray level compensation for the remaining two channels is zero, and the following table shows
Color information in the library, wherein R _ P represents the gray value of the R channel pixel, G _ P represents the value of the G channel, and B _ P represents the B channel;
8. the intelligent monitoring system for fish video identification according to claim 1, wherein step 13 specifically comprises: step 13.1, randomly selecting k samples from the data set as initial clustering centers;
wherein,
represents the center of a cluster that is randomly selected,
a set representing a cluster center;
step 13.2, for each sample x in the datasetiCalculating the distances from the k clustering centers to the k clustering centers, and dividing the distances into categories corresponding to the clustering centers with the minimum distances;
step 13.3, for each OiThe class, its cluster center is calculated as follows:
wherein the sample set is xiN, N being the total number of pixels;
and step 13.4, returning to the step 13.2 until the cluster center is not changed any more.
9. The intelligent monitoring system for fish video identification according to claim 1, wherein step 14 specifically comprises: step 14.1, generating a characteristic video;
in the RGB color space, the channel with the maximum average brightness in the three channels of videos is taken as a target channel, other channels with lower brightness are compensated to be zero, the generated new video replaces the original video, and three components of the color video are represented in a vector form:
wherein
Representing the original video, MiIs a different compensation operation, M0Indicating that the brightness information of the R channel is reserved, and the rest is zero; m1Indicating that the brightness information of the G channel is reserved, and the rest is zero; m2Indicating that the brightness information of the B channel is reserved, and the rest is zero;
step 14.2, determining a clustering value;
step 14.3, selecting a color library;
by pairs
Analyzing the brightness of different channels, and generating a new color compensation video Seg (x, y) to replace an original video by adopting different color compensation methods;
if i =0, it is indicated as passing
Color compensation of the operation, the resulting video will be color clusters of the R library,
if i =1, it is indicated as passing
Color compensation of the operation, the generated video is color clustering through a G library,
if i =2, indicate passing
Color compensation of the operation, the generated video is clustered by colors of a B library,
step 14.4, clustering the colors of the videos,
according to the determined clustering value K and the fish color library as
Clustering videos from a clustered color library
Randomly selecting sample points as initial clustering centers
And the remaining cluster centers satisfy the following condition:
calculate each sample point at
And
the probability is as follows:
wherein,
selecting a point with the highest probability as the next initial clustering center from the distance from each sampling point to the nearest center, and repeating the step 14.4 until k initial centers are selected;
is a sample point located in the color library; and segmenting the video image according to the k initial centers to further obtain the number of fishes in the video.
10. The intelligent monitoring system for fish video identification according to claim 9, wherein step 14.2 is specifically:
step 14.2.1, the gray interval is set,
setting the gray level video of the fish school after color compensation as gradient (x, y), wherein (x, y) is the space coordinate of the pixel, comprising
The video is divided into T gray intervals according to the gray levels to form a new pixel space R' represented as:
for each small gray interval, the number of pixels is:
wherein R' is the new pixel space, | represents
And
the norm of the sum of the number of pixels in the gray scale interval of (a),
means a gray value of
(x, y) are the spatial coordinates of the pixel,
to represent
And
the number of pixels between the gray scale intervals of (a);
step 14.2.2, traversing the pixels, calculating the total number of pixels in each gray scale interval, and selecting any pixel
Calculating the total number of pixels in the gray scale interval according to the formula, and calculating the total number of pixels in each gray scale intervalPercentage of number, use
Denotes, T =1, 2, 3.. T,
where (x, y) is the spatial coordinate of the pixel,
to represent
And
n is the total number of pixels in the video, t is a positive integer;
step 14.2.3, sorting the pixel distribution probability value of each gray scale interval in the step 14.2.2, and setting the reference probability value asP 0 By traversing the probability value of each pixel distribution, the probability distribution value satisfying the following equation is found
:
The clustering value K is determined as follows:
where (x, y) is the spatial coordinate of the pixel,
to represent
And
and N is the total number of pixels in the video.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011564844.0A CN112287913B (en) | 2020-12-25 | 2020-12-25 | Intelligent supervisory system for fish video identification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011564844.0A CN112287913B (en) | 2020-12-25 | 2020-12-25 | Intelligent supervisory system for fish video identification |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112287913A CN112287913A (en) | 2021-01-29 |
| CN112287913B true CN112287913B (en) | 2021-04-06 |
Family
ID=74426356
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011564844.0A Active CN112287913B (en) | 2020-12-25 | 2020-12-25 | Intelligent supervisory system for fish video identification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112287913B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112966698A (en) * | 2021-03-23 | 2021-06-15 | 西安电子科技大学 | Freshwater fish image real-time identification method based on lightweight convolutional network |
| CN115690570B (en) * | 2023-01-05 | 2023-03-28 | 中国水产科学研究院黄海水产研究所 | Fish shoal feeding intensity prediction method based on ST-GCN |
| CN117274651B (en) * | 2023-11-17 | 2024-02-09 | 北京亮道智能汽车技术有限公司 | Object detection method and device based on point cloud and computer readable storage medium |
| CN118314407A (en) * | 2024-06-05 | 2024-07-09 | 中国水产科学研究院珠江水产研究所 | Shrimp sorting optimization method and system based on biological characteristics |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101493524A (en) * | 2009-02-19 | 2009-07-29 | 浙江海洋学院 | Sonar detecting method for large yellow croaker escaping from deep water net cage and special equipment thereof |
| CN109064497A (en) * | 2018-07-16 | 2018-12-21 | 南京信息工程大学 | A kind of video tracing method based on color cluster accretion learning |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101753101B1 (en) * | 2015-11-28 | 2017-07-03 | 계명대학교 산학협력단 | Automatic source classification method and apparatus using mean-shift clustering and stepwise merging in color image |
| CN106997592A (en) * | 2017-03-30 | 2017-08-01 | 鲁东大学 | Fuzzy clustering medical image cutting method with real-time |
| CN109784378A (en) * | 2018-12-27 | 2019-05-21 | 南京芊玥机器人科技有限公司 | A kind of underwater fishing method based on machine vision |
| CN111401308B (en) * | 2020-04-08 | 2023-03-24 | 蚌埠学院 | Fish behavior video identification method based on optical flow effect |
| CN112102288B (en) * | 2020-09-15 | 2023-11-07 | 应急管理部大数据中心 | Water body identification and water body change detection method, device, equipment and medium |
-
2020
- 2020-12-25 CN CN202011564844.0A patent/CN112287913B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101493524A (en) * | 2009-02-19 | 2009-07-29 | 浙江海洋学院 | Sonar detecting method for large yellow croaker escaping from deep water net cage and special equipment thereof |
| CN109064497A (en) * | 2018-07-16 | 2018-12-21 | 南京信息工程大学 | A kind of video tracing method based on color cluster accretion learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112287913A (en) | 2021-01-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112287913B (en) | Intelligent supervisory system for fish video identification | |
| WO2022121645A1 (en) | Method for generating sense of reality of virtual object in teaching scene | |
| US20180225539A1 (en) | Systems and methods for extracting information about objects from scene information | |
| KR101555347B1 (en) | Apparatus and method for generating video-guided facial animation | |
| US20190087985A1 (en) | Differentiable rendering pipeline for inverse graphics | |
| CN114155299B (en) | Building digital twinning construction method and system | |
| CN108648269A (en) | The monomerization approach and system of three-dimensional building object model | |
| Poux et al. | Unsupervised segmentation of indoor 3D point cloud: Application to object-based classification | |
| CN105631892B (en) | It is a kind of that detection method is damaged based on the aviation image building of shade and textural characteristics | |
| AU2014241771A1 (en) | Mapping augmented reality experience to various environments | |
| CN110378997A (en) | A kind of dynamic scene based on ORB-SLAM2 builds figure and localization method | |
| CN110379020A (en) | A kind of laser point cloud painting methods and device based on generation confrontation network | |
| CN111753698A (en) | Multi-mode three-dimensional point cloud segmentation system and method | |
| CN113538574B (en) | Pose positioning method, device and equipment and computer readable storage medium | |
| CN107871143B (en) | Image-recognizing method and device, computer installation and computer readable storage medium | |
| CN115035260A (en) | Indoor mobile robot three-dimensional semantic map construction method | |
| Pound et al. | A patch-based approach to 3D plant shoot phenotyping | |
| Li | Film and TV animation production based on artificial intelligence AlphaGd | |
| CN112102480A (en) | Image data processing method, apparatus, device and medium | |
| CN115937461B (en) | Multi-source fusion model construction and texture generation method, device, medium and equipment | |
| Ishikawa et al. | Semantic segmentation of 3D point cloud to virtually manipulate real living space | |
| Zhang et al. | 3D viewpoint estimation based on aesthetics | |
| CN105957148A (en) | Granularity balance data organization method of complicated three-dimensional building model | |
| CN117496086A (en) | Indoor geometric reconstruction method for semantic perception | |
| CN107909111A (en) | A kind of multilevel scheme clustering method of settlement place polygon |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |