CN111967473A - Grain depot storage condition monitoring method based on image segmentation and template matching - Google Patents
Grain depot storage condition monitoring method based on image segmentation and template matching Download PDFInfo
- Publication number
- CN111967473A CN111967473A CN202010603388.XA CN202010603388A CN111967473A CN 111967473 A CN111967473 A CN 111967473A CN 202010603388 A CN202010603388 A CN 202010603388A CN 111967473 A CN111967473 A CN 111967473A
- Authority
- CN
- China
- Prior art keywords
- grain
- grain surface
- picture
- semantic segmentation
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 238000003860 storage Methods 0.000 title claims abstract description 51
- 238000012544 monitoring process Methods 0.000 title claims abstract description 47
- 238000003709 image segmentation Methods 0.000 title claims abstract description 36
- 230000011218 segmentation Effects 0.000 claims abstract description 131
- 230000008859 change Effects 0.000 claims abstract description 116
- 235000013339 cereals Nutrition 0.000 claims description 391
- 238000012549 training Methods 0.000 claims description 16
- 238000012360 testing method Methods 0.000 claims description 15
- 235000013312 flour Nutrition 0.000 claims description 12
- 238000012795 verification Methods 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 9
- 238000002372 labelling Methods 0.000 claims description 8
- 238000013135 deep learning Methods 0.000 claims description 6
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 239000000126 substance Substances 0.000 claims description 3
- 238000004458 analytical method Methods 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 11
- 238000007689 inspection Methods 0.000 description 8
- 238000004590 computer program Methods 0.000 description 7
- 238000012423 maintenance Methods 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005303 weighing Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000011056 performance test Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/68—Food, e.g. fruit or vegetables
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a grain depot warehousing condition monitoring method, equipment and medium based on image segmentation and template matching. The method comprises the steps of obtaining a first grain picture and a second grain picture which are shot at different time points at the same preset position of the same grain depot; performing semantic segmentation on the first grain surface picture and the second grain surface picture by adopting an image semantic segmentation model to obtain a first grain surface semantic segmentation map and a second grain surface semantic segmentation map; comparing the first grain surface semantic segmentation graph with the second grain surface semantic segmentation graph to obtain a grain surface external edge change area; carrying out similarity comparison on the first grain picture and the second grain picture by adopting template matching to obtain a grain surface internal change area; and calculating to obtain the storage change condition of the grain depot according to the edge change area outside the grain surface and the change area inside the grain surface. The method adopts a non-contact mode to monitor the grain storage state of the granary, can remotely operate and timely find the change situation and position the change area, reduces the working intensity and is convenient for daily analysis and supervision.
Description
Technical Field
The application relates to the technical field of Artificial Intelligence (AI) and computer vision, in particular to a grain depot storage condition monitoring method, equipment and medium based on image segmentation and template matching.
Background
At present, the grain inventory inspection usually adopts a direct measurement calculation method and a weighing method. These methods are not only labor intensive, long inspection cycle, and expensive, but also require the use of large amounts of manpower and materials. Meanwhile, the storage condition of all warehouses is difficult to be checked one day.
In recent years, in order to improve the detection accuracy and speed of the storage condition of the grain depot and accelerate the intelligent upgrade of the grain depot, related scientific researchers have proposed some advanced measurement methods, such as a grain inventory inspection method based on a pressure sensor or based on laser scanning. However, these methods often have the disadvantages of high deployment cost, difficult maintenance, single deployment environment, and the like. Under the prerequisite that reduces deployment cost, convenient maintenance, improvement monitoring accuracy, how long-range realization carry out one a day to the grain depot storage condition and examine, in time discover storage change condition and fix a position the change region to reduce supervisory personnel's in the storehouse working strength, promote the technical problem that needs to solve urgently among the prior art at present.
Therefore, there is a need to develop a new method, apparatus and medium for monitoring grain depot storage conditions to overcome and ameliorate one or more of the above disadvantages of the prior art, or at least to provide an effective alternative to solve the above problems.
Disclosure of Invention
The embodiment of the specification provides a grain depot storage condition monitoring method, equipment and medium based on image segmentation and template matching, and is used for solving the following technical problems in the prior art: the existing grain inventory inspection method based on a pressure sensor or based on laser scanning and the like has the problems of high deployment cost, difficult maintenance, single deployment environment and the like.
The embodiment of the specification adopts the following technical scheme:
a grain depot warehousing condition monitoring method based on image segmentation and template matching comprises the following steps:
acquiring a first grain surface picture and a second grain surface picture which are shot at different time points at the same preset position of the same grain depot;
performing semantic segmentation on the first grain surface picture and the second grain surface picture by adopting an image semantic segmentation model to obtain a first grain surface semantic segmentation map and a second grain surface semantic segmentation map;
comparing the first grain surface semantic segmentation graph with the second grain surface semantic segmentation graph to obtain a grain surface external edge change area;
carrying out similarity comparison on the first grain surface picture and the second grain surface picture by adopting template matching to obtain a grain surface internal change area;
and calculating to obtain the storage change condition of the grain depot according to the edge change area outside the grain surface and the change area inside the grain surface.
Further, the semantic segmentation of the first grain picture and the second grain picture by using an image semantic segmentation model comprises:
and respectively dividing the first grain picture and the second grain picture into a grain class and a background class.
Further, the grain depot warehousing condition monitoring method based on image segmentation and template matching further comprises the following steps:
acquiring an original grain surface picture data set;
performing picture labeling processing on the grain surface original picture data set by adopting a picture labeling tool to obtain a corresponding grain surface labeled picture data set;
obtaining a grain division data set according to the grain original image data set and the corresponding grain annotation image data set;
dividing the grain surface segmentation data set into a training data set, a verification data set and a test data set;
adopting a deep learning framework to build an image semantic segmentation model;
training the image semantic segmentation model by adopting the training data set;
judging the image semantic segmentation model by adopting the verification data set so as to adjust the capacity of the image semantic segmentation model;
and testing the image semantic segmentation model by adopting the test data set.
Further, before the grain surface segmentation data set is divided into a training data set, a verification data set and a test data set, the grain depot storage condition monitoring method based on image segmentation and template matching further comprises the following steps:
and enhancing the grain and flour segmentation data set so as to increase the size of the grain and flour segmentation data set.
Further, the picture marking tool adopts LABELME software.
Further, the matching of the template is adopted to compare the similarity of the first grain picture and the second grain picture, and the obtaining of the internal change area of the grain comprises the following steps:
respectively segmenting the first grain surface picture and the second grain surface picture by adopting the first grain surface semantic segmentation picture and the second grain surface semantic segmentation picture to obtain a first grain surface partial image in the first grain surface picture and a second grain surface partial image in the second grain surface picture;
and performing similarity comparison on the first grain surface partial image and the second grain surface partial image by adopting template matching to obtain a grain surface internal change area.
Further, the performing similarity comparison on the first grain portion image and the second grain portion image by adopting template matching to obtain a grain surface internal change region includes:
dividing the first grain part image and the second grain part image into a plurality of corresponding units by adopting the same scale;
and performing similarity comparison on each corresponding unit by adopting template matching to obtain the internal change area of the grain surface.
Further, the calculation of the grain depot storage change condition according to the grain surface outer edge change area and the grain surface inner change area comprises:
calculating the area of the grain surface outer edge change area to obtain a first area value, and if the first area value is larger than a preset first threshold value, judging that the grain surface outer edge changes;
and calculating the area of the edge change area inside the grain surface to obtain a second area value, and if the second area value is larger than a preset second threshold value, judging that the grain surface has a change inside.
An equipment of grain depot storage condition monitoring based on image segmentation and template matching, wherein includes:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to:
acquiring a first grain surface picture and a second grain surface picture which are shot at different time points at the same preset position of the same grain depot;
performing semantic segmentation on the first grain surface picture and the second grain surface picture by adopting an image semantic segmentation model to obtain a first grain surface semantic segmentation map and a second grain surface semantic segmentation map;
comparing the first grain surface semantic segmentation graph with the second grain surface semantic segmentation graph to obtain a grain surface external edge change area;
carrying out similarity comparison on the first grain surface picture and the second grain surface picture by adopting template matching to obtain a grain surface internal change area;
and calculating to obtain the storage change condition of the grain depot according to the edge change area outside the grain surface and the change area inside the grain surface.
A non-transitory computer storage medium for grain depot warehousing condition monitoring based on image segmentation and template matching, storing computer-executable instructions configured to:
acquiring a first grain surface picture and a second grain surface picture which are shot at different time points at the same preset position of the same grain depot;
performing semantic segmentation on the first grain surface picture and the second grain surface picture by adopting an image semantic segmentation model to obtain a first grain surface semantic segmentation map and a second grain surface semantic segmentation map;
comparing the first grain surface semantic segmentation graph with the second grain surface semantic segmentation graph to obtain a grain surface external edge change area;
carrying out similarity comparison on the first grain surface picture and the second grain surface picture by adopting template matching to obtain a grain surface internal change area;
and calculating to obtain the storage change condition of the grain depot according to the edge change area outside the grain surface and the change area inside the grain surface.
The embodiment of the specification adopts at least one technical scheme which can achieve the following beneficial effects:
(1) the grain depot warehousing condition monitoring method based on image segmentation and template matching disclosed by the embodiment of the specification utilizes an image semantic segmentation model and a template matching algorithm based on deep learning to automatically monitor and analyze the warehousing condition of grains in a grain depot, so that the warehousing change condition can be found in time and the change area can be positioned.
(2) Compared with the existing method, the grain depot warehousing condition monitoring method based on image segmentation and template matching, which is exemplified by the embodiment of the specification, can obtain the change condition of the outer edge of the grain surface, can also obtain the change condition of the inner part of the grain surface, and can position the change area, so that the grain depot warehousing condition can be monitored more comprehensively and efficiently, and the grain depot warehousing condition monitoring method has good popularization and application values.
(3) The grain depot warehousing condition monitoring method based on image segmentation and template matching disclosed by the embodiment of the specification has the advantages of simplicity in deployment, high efficiency, high precision, low cost and the like, can realize remote automatic monitoring of grain depot warehousing only by integrating a cost model on the basis of the existing grain depot monitoring equipment (such as a camera), and has important significance for intelligent upgrading and transformation of a grain depot.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
fig. 1 is a schematic flowchart of a grain depot warehousing condition monitoring method based on image segmentation and template matching according to an embodiment of the present disclosure;
fig. 2 is a block flow diagram of a grain depot warehousing condition monitoring method based on image segmentation and template matching according to an embodiment of the present disclosure;
fig. 3 is a schematic diagram of an external edge variation region of a grain surface in a grain depot warehousing condition monitoring method based on image segmentation and template matching provided in an embodiment of the present disclosure;
fig. 4 is a schematic diagram of a grain surface internal change region in the grain depot storage condition monitoring method based on image segmentation and template matching provided in the embodiment of the present specification.
Detailed Description
In order to make the objects, technical solutions and advantages of the present disclosure more apparent, the technical solutions of the present disclosure will be clearly and completely described below with reference to the specific embodiments of the present disclosure and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person skilled in the art without making any inventive step based on the embodiments in the description belong to the protection scope of the present application.
First, the technical concept of the technical solution disclosed in the present invention will be explained. The existing grain inventory inspection methods include direct measurement calculation methods and weighing methods, and the methods have the disadvantages of large workload, long inspection period, high cost, large amount of manpower and material resources, and difficulty in achieving one-day inspection of the storage condition. In recent years, some advanced detection methods, such as a grain inventory inspection method based on a pressure sensor or based on laser scanning, have been proposed by researchers. However, these methods have the disadvantages of high deployment cost, difficult maintenance, single deployment environment, and the like. Under the prerequisite that reduces deployment cost, convenient maintenance, improvement monitoring accuracy, how long-range realization carry out one a day to the grain depot storage condition and examine, in time discover storage change condition and fix a position the change region to reduce supervisory personnel's in the storehouse working strength, promote the technical problem that needs to solve urgently among the prior art at present.
The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings. Fig. 1 is a schematic flow chart of a grain depot warehousing condition monitoring method based on image segmentation and template matching according to an embodiment of the present disclosure.
As shown in fig. 1, a grain depot warehousing condition monitoring method based on image segmentation and template matching includes acquiring a first grain surface picture and a second grain surface picture of the same grain depot at different time points and at the same preset position; performing semantic segmentation on the first grain surface picture and the second grain surface picture by adopting an image semantic segmentation model to obtain a first grain surface semantic segmentation map and a second grain surface semantic segmentation map; comparing the first grain surface semantic segmentation graph with the second grain surface semantic segmentation graph to obtain a grain surface external edge change area; carrying out similarity comparison on the first grain picture and the second grain picture by adopting template matching to obtain a grain surface internal change area; and calculating to obtain the storage change condition of the grain depot according to the edge change area outside the grain surface and the change area inside the grain surface.
The grain depot storage condition monitoring method disclosed by the embodiment utilizes the deep learning-based image semantic segmentation model and the template matching algorithm to automatically monitor and analyze the storage condition of grains in the grain depot, so that the storage change condition can be found in time and the change area can be positioned. Compared with the prior art, the method can obtain the change condition of the outer edge of the grain surface, can also obtain the change condition of the inner part of the grain surface, and can position the change area, thereby monitoring the grain depot storage condition more comprehensively and efficiently, and having good popularization and application values. The system has the advantages of simple deployment, high efficiency, high precision, low cost and the like, can realize remote automatic monitoring of grain depot storage only by integrating a cost model on the basis of the existing grain depot monitoring equipment (such as a camera), and has important significance for intelligent upgrading and reconstruction of the grain depot.
In this embodiment, performing semantic segmentation on the first grain picture and the second grain picture by using an image semantic segmentation model includes segmenting the first grain picture and the second grain picture into a grain class and a background class, respectively.
As shown in fig. 2, in this embodiment, the grain depot warehousing condition monitoring method based on image segmentation and template matching further includes obtaining an original grain surface image dataset; performing picture labeling processing on the grain surface original picture data set by adopting a picture labeling tool to obtain a corresponding grain surface labeled picture data set; obtaining a grain and flour segmentation data set according to the grain and flour original image data set and the corresponding grain and flour annotation image data set; dividing the grain surface segmentation data set into a training data set, a verification data set and a test data set; adopting a deep learning framework to build an image semantic segmentation model; training the image semantic segmentation model by adopting a training data set; judging the image semantic segmentation model by adopting a verification data set so as to adjust the capacity of the image semantic segmentation model; and testing the image semantic segmentation model by adopting a test data set. When the grain surface segmentation data set is created, the monitoring equipment in the existing grain depot, such as a camera and the like, can be adopted to capture images in the grain depot, and then a picture marking tool is adopted to mark the captured original pictures, namely the positions of the grain surfaces in the pictures are marked.
In this embodiment, before the grain surface segmentation data set is divided into the training data set, the verification data set and the test data set, the grain depot warehousing condition monitoring method based on image segmentation and template matching further includes enhancing processing of the grain surface segmentation data set to increase the size of the grain surface segmentation data set. Wherein, the picture marking tool adopts LABELME software.
In this embodiment, performing similarity comparison on the first grain surface picture and the second grain surface picture by using template matching to obtain an internal grain surface change region includes respectively segmenting the first grain surface picture and the second grain surface picture by using a first grain surface semantic segmentation picture and a second grain surface semantic segmentation picture to obtain a first grain surface partial image in the first grain surface picture and a second grain surface partial image in the second grain surface picture; and performing similarity comparison on the first grain surface partial image and the second grain surface partial image by adopting template matching to obtain a grain surface internal change area.
The method comprises the steps that a first grain part image and a second grain part image are subjected to similarity comparison through template matching, and an internal change area of the grain surface is obtained; and (4) carrying out similarity comparison on each corresponding unit by adopting template matching to obtain the internal change area of the grain surface.
In this embodiment, the calculation of the grain depot warehousing change condition according to the grain surface external edge change region and the grain surface internal change region includes calculating the area of the grain surface external edge change region to obtain a first area value, and if the first area value is greater than a preset first threshold value, it is determined that the grain surface external edge has a change; and calculating the area of the edge change area in the grain surface to obtain a second area value, and if the second area value is larger than a preset second threshold value, judging that the grain surface has change.
For ease of understanding, the details of the above method are further described below:
a grain depot warehousing condition monitoring method based on image segmentation and template matching comprises the following contents:
1. collecting a plurality of original grain surface pictures in a grain depot, marking the original grain surface pictures by using a picture marking tool, and creating a grain surface segmentation data set;
2. constructing an image semantic segmentation model, and training and testing the image semantic segmentation model by using a grain surface segmentation data set;
3. and segmenting two pictures captured by the same camera in the same grain depot at the same preset position and at different time points through a trained image semantic segmentation model to obtain two grain surface semantic segmentation maps. The grain surface semantic segmentation graph is a mask graph, wherein the region where the grain is located is red, and the other regions are black. Then subtracting the two grain surface semantic segmentation maps to obtain a grain surface external edge change area;
4. matching similarity of the grain surface parts through template matching to obtain a grain surface internal change area;
5. and combining the grain surface outer edge change area with the grain surface inner change area to obtain the storage change condition in the grain depot.
Wherein, the concrete content of the step 1 is as follows:
capturing the internal picture of the granary by using a camera inside the granary;
labeling the collected pictures by using picture labeling tools such as labelme software and the like;
performing post-processing on the original grain surface picture and the corresponding marked picture to obtain a grain surface segmentation data set;
performing enhancement processing on the grain surface segmentation data set to increase the size of the data set;
and dividing the data set, namely dividing the grain surface segmentation data set into a training data set, a verification data set and a test data set.
Wherein, the specific content of the step 2 is as follows:
constructing an image semantic segmentation model by using a deep learning framework;
training the constructed image semantic segmentation model by using a training data set in the grain surface segmentation data set; judging whether the network model is over-fit or under-fit through the verification data set, and adjusting the capacity of the network model; and performing performance test on the trained image semantic segmentation model through a test data set.
Wherein, the specific content of step 3 is:
inputting grain surface pictures captured by a camera in a grain depot into a trained image semantic segmentation model to obtain a grain surface semantic segmentation picture corresponding to the grain surface semantic segmentation model, namely a mask picture;
two grain surface pictures and corresponding grain surface semantic segmentation pictures which are captured at the same preset position and different time points of the same granary are selected, and then the two grain surface semantic segmentation pictures are subtracted to obtain a grain surface external edge change area.
Wherein, the specific content of the step 4 is as follows:
segmenting the original grain and flour picture through a grain and flour semantic segmentation picture, wherein original information of the grain and flour part in the original picture is reserved, and other parts are uniformly processed into black (RGB (0,0,0)) so as to eliminate irrelevant factors;
the method comprises the steps of dividing two grain surface partial pictures processed by the same presetting position and different time points of the same granary into a plurality of units with the same size by the same scale, and then carrying out similarity matching on each unit through template matching to obtain the internal change area of the grain surface. Preferably, the division is (20 pixels) × (20 pixels) sized cells.
Wherein, the concrete content of step 5 is:
calculating the area of the edge change area outside the grain surface, and if the area exceeds a threshold value 1 (used for judging the change of the outer edge), judging the change of the outer edge, wherein the anti-regularization is not established;
calculating the area of the internal change area of the grain surface, and if the area exceeds a threshold value 2 (used for judging the internal change of the grain surface), judging the internal change of the grain surface, wherein the anti-regularization is not established;
combine grain face outside edge change region and grain face inside change region, obtain storage change situation in the grain depot and fix a position it: firstly, only the outer edge of the grain surface is changed; secondly, only the inner part of the grain surface changes; both change; both are unchanged.
Second, design scheme
In order to realize the grain depot storage condition monitoring method based on image segmentation and template matching, the method is mainly completed by the following steps in design:
1. defining an image semantic segmentation interface and parameters thereof:
the method is used for performing semantic segmentation on the captured grain and flour picture: grain class (grain) and background class (background).
Inputting: grain flour picture
And (3) outputting: grain surface semantic segmentation graph (mask graph)
2. Defining template matching interfaces and parameters thereof
The method is used for comparing the similarity of the two pictures and positioning the dissimilar area exceeding a set threshold, wherein the positioning adopts a sliding window principle.
Inputting: two grain pictures processed according to the mask picture (pictures needing to be captured by the same camera at the same preset position, wherein one picture is set as a base line picture and used for comparing with the other picture)
And (3) outputting: changing area in grain picture
3. Interface for defining variable map of grain warehouse and its parameters
And the method is used for combining the grain surface segmentation graph with the template matching graph to obtain a grain surface storage variation graph.
Inputting: grain surface semantic segmentation graph and template matching result graph
And (3) outputting: grain storage changing map
The grain depot warehousing condition monitoring method based on image segmentation and template matching comprises the following models and algorithms:
1. the convolutional neural network-based image semantic segmentation model depeplab v3+ can be replaced by other semantic segmentation models.
2. The image template matching algorithm based on the image similarity measurement inputs two pictures and outputs the picture similarity:
this embodiment also provides a grain depot storage condition monitoring's equipment based on image segmentation and template matching, includes wherein:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to:
acquiring a first grain surface picture and a second grain surface picture which are shot at different time points at the same preset position of the same grain depot;
performing semantic segmentation on the first grain surface picture and the second grain surface picture by adopting an image semantic segmentation model to obtain a first grain surface semantic segmentation map and a second grain surface semantic segmentation map;
comparing the first grain surface semantic segmentation graph with the second grain surface semantic segmentation graph to obtain a grain surface external edge change area;
carrying out similarity comparison on the first grain surface picture and the second grain surface picture by adopting template matching to obtain a grain surface internal change area;
and calculating to obtain the storage change condition of the grain depot according to the edge change area outside the grain surface and the change area inside the grain surface.
The embodiment also provides a non-volatile computer storage medium for grain depot warehousing condition monitoring based on image segmentation and template matching, which stores computer executable instructions, wherein the computer executable instructions are set as follows:
acquiring a first grain surface picture and a second grain surface picture which are shot at different time points at the same preset position of the same grain depot;
performing semantic segmentation on the first grain surface picture and the second grain surface picture by adopting an image semantic segmentation model to obtain a first grain surface semantic segmentation map and a second grain surface semantic segmentation map;
comparing the first grain surface semantic segmentation graph with the second grain surface semantic segmentation graph to obtain a grain surface external edge change area;
carrying out similarity comparison on the first grain surface picture and the second grain surface picture by adopting template matching to obtain a grain surface internal change area;
and calculating to obtain the storage change condition of the grain depot according to the edge change area outside the grain surface and the change area inside the grain surface.
The embodiments in the present application are described in a progressive manner, and the same and similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the device and media embodiments, the description is relatively simple as it is substantially similar to the method embodiments, and reference may be made to some descriptions of the method embodiments for relevant points.
The device and the medium provided by the embodiment of the application correspond to the method one to one, so the device and the medium also have the similar beneficial technical effects as the corresponding method, and the beneficial technical effects of the method are explained in detail above, so the beneficial technical effects of the device and the medium are not repeated herein.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is directed to methods, apparatus (systems), and computer program products according to embodiments of the present invention
A flowchart and/or block diagram of an article. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (10)
1. A grain depot storage condition monitoring method based on image segmentation and template matching is characterized by comprising the following steps:
acquiring a first grain surface picture and a second grain surface picture which are shot at different time points at the same preset position of the same grain depot;
performing semantic segmentation on the first grain surface picture and the second grain surface picture by adopting an image semantic segmentation model to obtain a first grain surface semantic segmentation map and a second grain surface semantic segmentation map;
comparing the first grain surface semantic segmentation graph with the second grain surface semantic segmentation graph to obtain a grain surface external edge change area;
carrying out similarity comparison on the first grain surface picture and the second grain surface picture by adopting template matching to obtain a grain surface internal change area;
and calculating to obtain the storage change condition of the grain depot according to the edge change area outside the grain surface and the change area inside the grain surface.
2. The grain depot warehousing condition monitoring method based on image segmentation and template matching according to claim 1, wherein the performing semantic segmentation on the first grain surface picture and the second grain surface picture by adopting an image semantic segmentation model comprises:
and respectively dividing the first grain picture and the second grain picture into a grain class and a background class.
3. The grain depot warehousing condition monitoring method based on image segmentation and template matching as claimed in claim 1, wherein the method further comprises:
acquiring an original grain surface picture data set;
performing picture labeling processing on the grain surface original picture data set by adopting a picture labeling tool to obtain a corresponding grain surface labeled picture data set;
obtaining a grain division data set according to the grain original image data set and the corresponding grain annotation image data set;
dividing the grain surface segmentation data set into a training data set, a verification data set and a test data set;
adopting a deep learning framework to build an image semantic segmentation model;
training the image semantic segmentation model by adopting the training data set;
judging the image semantic segmentation model by adopting the verification data set so as to adjust the capacity of the image semantic segmentation model;
and testing the image semantic segmentation model by adopting the test data set.
4. The grain depot warehousing condition monitoring method based on image segmentation and template matching according to claim 2, wherein before the grain face segmentation dataset is divided into a training dataset, a verification dataset and a test dataset, the method further comprises:
and enhancing the grain and flour segmentation data set so as to increase the size of the grain and flour segmentation data set.
5. The grain depot warehousing condition monitoring method based on image segmentation and template matching as claimed in claim 2, characterized in that:
the picture marking tool adopts LABELME software.
6. The grain depot warehousing condition monitoring method based on image segmentation and template matching as claimed in claim 2, wherein the obtaining of the grain surface internal change region by comparing the similarity of the first grain surface picture and the second grain surface picture by template matching comprises:
respectively segmenting the first grain surface picture and the second grain surface picture by adopting the first grain surface semantic segmentation picture and the second grain surface semantic segmentation picture to obtain a first grain surface partial image in the first grain surface picture and a second grain surface partial image in the second grain surface picture;
and performing similarity comparison on the first grain surface partial image and the second grain surface partial image by adopting template matching to obtain a grain surface internal change area.
7. The grain depot warehousing condition monitoring method based on image segmentation and template matching as claimed in claim 6, wherein the performing similarity comparison on the first grain surface partial image and the second grain surface partial image by adopting template matching to obtain a grain surface internal change region comprises:
dividing the first grain part image and the second grain part image into a plurality of corresponding units by adopting the same scale;
and performing similarity comparison on each corresponding unit by adopting template matching to obtain the internal change area of the grain surface.
8. The grain depot warehousing condition monitoring method based on image segmentation and template matching as claimed in claim 1, wherein the step of calculating grain depot warehousing change conditions according to the grain surface external edge change region and the grain surface internal change region comprises:
calculating the area of the grain surface outer edge change area to obtain a first area value, and if the first area value is larger than a preset first threshold value, judging that the grain surface outer edge changes;
and calculating the area of the edge change area inside the grain surface to obtain a second area value, and if the second area value is larger than a preset second threshold value, judging that the grain surface has a change inside.
9. The utility model provides an equipment of grain depot storage condition monitoring based on image segmentation and template matching which characterized in that includes:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to:
acquiring a first grain surface picture and a second grain surface picture which are shot at different time points at the same preset position of the same grain depot;
performing semantic segmentation on the first grain surface picture and the second grain surface picture by adopting an image semantic segmentation model to obtain a first grain surface semantic segmentation map and a second grain surface semantic segmentation map;
comparing the first grain surface semantic segmentation graph with the second grain surface semantic segmentation graph to obtain a grain surface external edge change area;
carrying out similarity comparison on the first grain surface picture and the second grain surface picture by adopting template matching to obtain a grain surface internal change area;
and calculating to obtain the storage change condition of the grain depot according to the edge change area outside the grain surface and the change area inside the grain surface.
10. A non-transitory computer storage medium for grain depot warehousing condition monitoring based on image segmentation and template matching, storing computer-executable instructions configured to:
acquiring a first grain surface picture and a second grain surface picture which are shot at different time points at the same preset position of the same grain depot;
performing semantic segmentation on the first grain surface picture and the second grain surface picture by adopting an image semantic segmentation model to obtain a first grain surface semantic segmentation map and a second grain surface semantic segmentation map;
comparing the first grain surface semantic segmentation graph with the second grain surface semantic segmentation graph to obtain a grain surface external edge change area;
carrying out similarity comparison on the first grain surface picture and the second grain surface picture by adopting template matching to obtain a grain surface internal change area;
and calculating to obtain the storage change condition of the grain depot according to the edge change area outside the grain surface and the change area inside the grain surface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010603388.XA CN111967473B (en) | 2020-06-29 | 2020-06-29 | Grain depot storage condition monitoring method, equipment and medium based on image segmentation and template matching |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010603388.XA CN111967473B (en) | 2020-06-29 | 2020-06-29 | Grain depot storage condition monitoring method, equipment and medium based on image segmentation and template matching |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111967473A true CN111967473A (en) | 2020-11-20 |
| CN111967473B CN111967473B (en) | 2023-01-20 |
Family
ID=73360716
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010603388.XA Active CN111967473B (en) | 2020-06-29 | 2020-06-29 | Grain depot storage condition monitoring method, equipment and medium based on image segmentation and template matching |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111967473B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112712021A (en) * | 2020-12-29 | 2021-04-27 | 华信咨询设计研究院有限公司 | Grain surface abnormal state identification method based on perceptual hash and connected domain analysis algorithm |
| CN114049481A (en) * | 2022-01-12 | 2022-02-15 | 安徽高哲信息技术有限公司 | Grain kernel detection alignment method, device, equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103808721A (en) * | 2014-03-06 | 2014-05-21 | 山东理工大学 | Capsule barn detection device and detection method thereof |
| WO2015151553A1 (en) * | 2014-03-31 | 2015-10-08 | Ｎｅｃソリューションイノベータ株式会社 | Change detection assistance device, change detection assistance method, and computer-readable recording medium |
| CN108759980A (en) * | 2018-04-03 | 2018-11-06 | 长沙沃哲信息技术有限公司 | A kind of method of grain depot management, equipment and computer readable storage medium |
| CN109472261A (en) * | 2018-06-15 | 2019-03-15 | 河南工业大学 | A kind of quantity of stored grains in granary variation automatic monitoring method based on computer vision |
| CN109886238A (en) * | 2019-03-01 | 2019-06-14 | 湖北无垠智探科技发展有限公司 | Unmanned plane Image Change Detection algorithm based on semantic segmentation |
| CN110008947A (en) * | 2019-04-12 | 2019-07-12 | 河南工业大学 | A kind of silo Grain Quantity monitoring method and device based on convolutional neural networks |
-
2020
- 2020-06-29 CN CN202010603388.XA patent/CN111967473B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103808721A (en) * | 2014-03-06 | 2014-05-21 | 山东理工大学 | Capsule barn detection device and detection method thereof |
| WO2015151553A1 (en) * | 2014-03-31 | 2015-10-08 | Ｎｅｃソリューションイノベータ株式会社 | Change detection assistance device, change detection assistance method, and computer-readable recording medium |
| CN108759980A (en) * | 2018-04-03 | 2018-11-06 | 长沙沃哲信息技术有限公司 | A kind of method of grain depot management, equipment and computer readable storage medium |
| CN109472261A (en) * | 2018-06-15 | 2019-03-15 | 河南工业大学 | A kind of quantity of stored grains in granary variation automatic monitoring method based on computer vision |
| CN109886238A (en) * | 2019-03-01 | 2019-06-14 | 湖北无垠智探科技发展有限公司 | Unmanned plane Image Change Detection algorithm based on semantic segmentation |
| CN110008947A (en) * | 2019-04-12 | 2019-07-12 | 河南工业大学 | A kind of silo Grain Quantity monitoring method and device based on convolutional neural networks |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112712021A (en) * | 2020-12-29 | 2021-04-27 | 华信咨询设计研究院有限公司 | Grain surface abnormal state identification method based on perceptual hash and connected domain analysis algorithm |
| CN112712021B (en) * | 2020-12-29 | 2022-06-17 | 华信咨询设计研究院有限公司 | Grain surface abnormal state identification method based on perceptual hash and connected domain analysis algorithm |
| CN114049481A (en) * | 2022-01-12 | 2022-02-15 | 安徽高哲信息技术有限公司 | Grain kernel detection alignment method, device, equipment and storage medium |
| CN114049481B (en) * | 2022-01-12 | 2022-04-22 | 安徽高哲信息技术有限公司 | Grain kernel detection alignment method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111967473B (en) | 2023-01-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Che et al. | Multi-scan segmentation of terrestrial laser scanning data based on normal variation analysis | |
| CN110689037A (en) | Method and system for automatic object annotation using deep networks | |
| CN111967473B (en) | Grain depot storage condition monitoring method, equipment and medium based on image segmentation and template matching | |
| CN111415106A (en) | Truck loading rate identification method, device, equipment and storage medium | |
| Kim et al. | Automatic photovoltaic panel area extraction from UAV thermal infrared images | |
| CN110188495B (en) | Method for generating three-dimensional house type graph based on two-dimensional house type graph of deep learning | |
| US20210214765A1 (en) | Methods and systems for automated counting and classifying microorganisms | |
| CN111914767A (en) | Scattered-pollution enterprise detection method and system based on multi-source remote sensing data | |
| CN114872574B (en) | Acquisition and monitoring system and method based on charging pile | |
| CN111680704A (en) | Automatic and rapid extraction method and device for newly-increased human active plaque of ocean red line | |
| WO2023185234A1 (en) | Image processing method and apparatus, and electronic device and storage medium | |
| CN115063584A (en) | Pipeline steam leakage detection method and detection system based on deep learning | |
| CN114078127B (en) | Object defect detection and counting method, device, equipment and storage medium | |
| Devereux et al. | A new approach for crack detection and sizing in nuclear reactor cores | |
| CN110276759B (en) | Mobile phone screen bad line defect diagnosis method based on machine vision | |
| Li et al. | Electronic product surface defect detection based on a MSSD network | |
| Košcevic et al. | Automatic visual reading of meters using deep learning | |
| CN113468905A (en) | Graphic code identification method and device, computer equipment and storage medium | |
| CN117237353A (en) | Flaw detection method, device, equipment and storage medium for appearance of mobile phone | |
| Wagner et al. | IndustrialEdgeML-End-to-end edge-based computer vision systemfor Industry 5.0 | |
| CN113298755B (en) | Method and device for rapidly detecting ecological environment change patch based on time sequence image | |
| CN114240924A (en) | Power grid equipment quality evaluation method based on digitization technology | |
| CN114119594A (en) | Oil leakage detection method and device based on deep learning | |
| Mandal et al. | Automated management of pothole related disasters using image processing and geotagging | |
| CN110222575B (en) | Equipment comprehensive efficiency analysis method, device, system, equipment and storage medium based on attention mechanism target detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230315 Address after: 250101 Inspur science and Technology Park, 1036 Inspur Road, hi tech Zone, Jinan City, Shandong Province Patentee after: Inspur Genersoft Co.,Ltd. Address before: 250101 Inspur science and Technology Park, 1036 Inspur Road, hi tech Zone, Jinan City, Shandong Province Patentee before: SHANDONG INSPUR GENESOFT INFORMATION TECHNOLOGY Co.,Ltd. |
|
| TR01 | Transfer of patent right |