CN116703820B - High-density bean counting and center point positioning method and system based on thermodynamic diagram - Google Patents

High-density bean counting and center point positioning method and system based on thermodynamic diagram Download PDF

Info

Publication number
CN116703820B
CN116703820B CN202310406884.XA CN202310406884A CN116703820B CN 116703820 B CN116703820 B CN 116703820B CN 202310406884 A CN202310406884 A CN 202310406884A CN 116703820 B CN116703820 B CN 116703820B
Authority
CN
China
Prior art keywords
thermodynamic diagram
bean
center point
center
point
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310406884.XA
Other languages
Chinese (zh)
Other versions
CN116703820A (en
Inventor
贺菁菁
徐晓刚
冯献忠
王军
何鹏飞
李萧缘
陈若晨
张耀华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Northeast Institute of Geography and Agroecology of CAS
Zhejiang Lab
Original Assignee
Northeast Institute of Geography and Agroecology of CAS
Zhejiang Lab
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Northeast Institute of Geography and Agroecology of CAS, Zhejiang Lab filed Critical Northeast Institute of Geography and Agroecology of CAS
Priority to CN202310406884.XA priority Critical patent/CN116703820B/en
Publication of CN116703820A publication Critical patent/CN116703820A/en
Application granted granted Critical
Publication of CN116703820B publication Critical patent/CN116703820B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Databases & Information Systems (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

A high-density bean counting and center point positioning method based on thermodynamic diagrams uses a Gaussian function to generate a Gaussian kernel template, and combines marked bean center point positions to generate a true value thermodynamic diagram for bean counting; by adopting CSRNet based on cavity convolution as a density map estimation module, the original image and the true value thermodynamic diagram are input into a model to be calculated into the thermodynamic diagram with the same size as the original image, and the L2 loss of the predicted thermodynamic diagram and the true value thermodynamic diagram is compared to perform parameter learning, so that high-quality thermodynamic diagram estimation is realized. And for the image to be tested, CSRNet is used for predicting the thermodynamic diagram, then the position coordinates of all the center points are obtained from the thermodynamic diagram by judging the local maximum position point, and the number of beans is obtained by rounding the values of the local center point thermodynamic diagram. Also comprises a high-density bean counting and center point positioning system based on thermodynamic diagram. The invention can improve the counting accuracy of the bean counting model in high-density and severe shielding scenes.

Description

High-density bean counting and center point positioning method and system based on thermodynamic diagram
Technical Field
The invention relates to the field of machine learning, in particular to a high-density bean counting method and system based on thermodynamic diagrams.
Background
The soybean is an important grain crop in the world, the quantity of soybean grains in the mature period is an important agronomic character of soybean seed test, and can help breeders estimate the yield conditions of different soybean varieties, thereby providing important basis for research of selecting excellent varieties. The traditional bean counting mode is manual measurement, which results in low bean seed checking speed, large error and high cost. With the rapid development and application of deep learning and image processing techniques, the use of automatic shot counting using shot images is increasing. The bean counting method based on deep learning comprises two methods, namely a method based on target frame detection and a method based on thermodynamic diagram. The detection method is to use the pod or the bean as a detection unit and use a detection model such as fast-RCNN, YOLO, SSD to detect the target frame. In this way, the target frame is a rectangular target frame, so that more background information is easy to detect, the calculation method is complex due to the limitation of anchor point design, and more missed detection is easy to generate under the conditions of high bean overlapping degree, shielding and high density. In recent years, the thermodynamic diagram-based counting method is a mainstream method in which targets are detected as one point, anchor point design is not required, and calculation is simplified. However, the thermodynamic diagram-based method has poor effect on the detection of overlapping targets, and can only output the number of targets, so that the positions of the targets cannot be simultaneously positioned.
Disclosure of Invention
In order to solve the defects in the prior art, the invention provides a high-density bean counting and center point positioning method and system for whole plants of pods in a mature period, a high-quality thermodynamic diagram of a bean center point is generated based on a CSRNet model, and the coordinate position of a bean center key point is positioned from the thermodynamic diagram by combining bean characteristics. The invention adopts the following technical scheme:
a high-density bean counting and center point positioning method based on thermodynamic diagram comprises the following steps:
S1, marking the center point of the bean. Marking the central point positions of all beans in the image, and recording coordinates (x i,yi) in the image, wherein the coordinates respectively represent the abscissa and the ordinate of the ith bean;
s2, generating a bean thermodynamic diagram. Generating a thermodynamic diagram H (x, y) of the beans by using a Gaussian function at the center point position of the beans marked in the step S1, wherein the thermodynamic diagram H (x, y) comprises the following steps:
S21, initializing a thermodynamic diagram H (x, y) of an original image, wherein the thermodynamic diagram H is a two-dimensional image which has the same size as the original image, the channel number of which is 1 and the pixel value of which is 0;
S22, a Gaussian kernel template is obtained by using a Gaussian function. Based on two parameters, size ksize and standard deviation σ, of a given gaussian kernel, a gaussian kernel template of size ksize and sum of 1 is generated. In the Gaussian kernel template, the pixel position value of the center point is maximum, and the value far from the center pixel point is gradually reduced. Specifically, the gaussian function is:
wherein a and b are position coordinates in the Gaussian kernel template obtained according to ksize.
And S23, adding the Gaussian kernel template obtained in the step S22 to the position with the marked bean center point, and adding the pixel value of the position to obtain a new pixel value if the pixel value is added to the position.
S3, constructing a bean thermodynamic diagram generation model. Learning all training images by adopting CSRNet model based on cavity convolution, and predicting thermodynamic diagramBy calculating H (x, y) and/>, in the S2Updating CSRNet network parameters until the model converges, and finally outputting a thermodynamic diagram estimation model;
S4, positioning the center point of the bean from the thermodynamic diagram. And outputting a thermodynamic diagram by using the thermodynamic diagram estimation model obtained in the step S3, selecting a point with the maximum value in the current area as the center point position of the bean based on the characteristics of the bean, and outputting the position coordinates, thereby obtaining the number of the bean in the image and the center position coordinates of the bean.
Further, the step S3 includes the following steps:
s31, carrying out data enhancement on an original image and a density map by adopting random horizontal overturning and matched cutting;
S32, inputting the image in the S31 into a built CSRNet, and predicting to obtain a thermodynamic diagram of the pod
S33, calculating the thermodynamic diagram obtained in the S32The distance loss between the true thermodynamic diagram H (x, y) obtained from the mark is as follows:
Where N represents the pixel-by-pixel position correspondence calculation error.
S34, judging whether the model reaches a convergence condition, if so, ending training of the model, and outputting to obtain a final thermodynamic diagram estimation model; if the convergence condition is not reached, the sequential execution of S31, S32, S33, S34 is continued.
Further, the CSRNet model constructed in S32 uses VGG16 as a base module, and replaces the full connection layer with a layer including 6 layers of hole convolution layers. And finally, directly adopting 8 times of up-sampling to obtain a prediction graph with the same size as the original input image and 1 channel number. Wherein CSRNet is initialized using a model trained on ImageNet.
Further, the step S4 is to input the original image to be tested into the thermodynamic diagram estimation model finally obtained after reaching the convergence condition in the step S3, so as to obtain the thermodynamic diagram with the length and width equal to those of the original image and the channel number equal to 1.
Further, in S4, a point having the maximum value in the current area is selected as the center point position of the bean, and is obtained from the predicted thermodynamic diagram by the following steps:
s41, inputting the to-be-tested try sheet into the trained thermodynamic diagram model to obtain a thermodynamic diagram with the same size as the input image and 1 channel number.
And S42, analyzing pixel points larger than four directions of up, down, left and right pixel by pixel from the obtained thermodynamic diagram, and taking the largest point in the area as an alternative center point.
Further, setting the pixel point smaller than a preset threshold value in the predicted thermodynamic diagram as 0, and respectively expanding the upper, lower, left and right boundaries in the thermodynamic diagram by 2 pixels;
further, the pixel value P up,Pdown,Pleft,Pright of the upper, lower, left and right positions of each center pixel P center is obtained, and the value of P center is re-represented as the sum of the original P center and the four position pixels:
further, by determining whether the center point is locally maximum, all local maximum point positions P all are obtained.
S43, sorting the alternative center points, and removing the weight according to the distance between the pixel points to obtain the final position of the center point of the bean.
Further, P all is ordered in the order from smaller to larger on the abscissa, the euclidean distance between the P all and the subsequent point is calculated from the first point, if the distance is smaller than the set threshold, the position smaller than the threshold is regarded as the coincident point, and the final counting of beans and the calculation of the central point are not participated. Until all the local maximum point positions are calculated, obtaining the center point position of the bean after center point de-duplication
S44, upwards rounding the thermal value of the final bean center point position to obtain the bean number of the center point position, and adding the bean numbers of all the center point positions to obtain the final total number of the bean.
Further, forBy/>, for that locationThe values are rounded upwards to obtain the number of beans at the center point:
Further, the position of the center point of the bean is determined Combining the number of beans to obtain the number Q of the beans in the image and the center point position/>, of all the beansThe number of beans is the sum of the number of beans at each center point:
the invention also relates to a high-density bean counting and center point positioning system based on thermodynamic diagram, which comprises the following steps:
The bean center point marking module is used for marking the center points of all beans in the image to obtain the coordinates of the center points of all the beans;
The bean thermodynamic diagram generating module is used for generating a Gaussian kernel template through a Gaussian function at the marked position of the center point of the bean, and obtaining a true value thermodynamic diagram through pixel value addition in the initial thermodynamic diagram;
The bean thermodynamic diagram generation model construction module is used for inputting the original image and the true thermodynamic diagram into the constructed thermodynamic diagram generation model, and training to obtain a final thermodynamic diagram generation model;
And the bean center point positioning module is used for inputting the picture to be tested into a final trained thermodynamic diagram generation model, obtaining the thermodynamic diagram of the picture to be tested, and obtaining the center point coordinates and the corresponding bean number by calculating the maximum value of the local area.
The invention also relates to a computer readable storage medium having stored thereon a program which, when executed by a processor, implements a thermodynamic diagram based high density bean counting and centre point positioning method of the invention.
The invention also relates to a computing device, which comprises a memory and a processor, wherein executable codes are stored in the memory, and the processor realizes the high-density bean counting and center point positioning method based on thermodynamic diagram when executing the executable codes.
The invention has the advantages that:
According to the technical scheme, the high-quality thermodynamic diagram estimation model is designed for solving the problems that the bean counting method based on thermodynamic diagrams has poor effect on detection of overlapped targets and the positions of beans cannot be accurately positioned, and pod counting and center point positioning are realized. First, CSRNet based on cavity convolution is used as a density map estimation module, a thermodynamic diagram with the same size as the original image is obtained through calculation, the high quality characteristic of the density map is guaranteed, and bean particles at partial shielding positions can be accurately predicted. And then, the position coordinates of the center point are obtained from the thermodynamic diagram through the determination of the point position with the maximum local point position, and rounding is carried out by reserving the value of the thermodynamic diagram of the local center point, so that the prediction accuracy of the center point positions of a plurality of beans under the serious shielding condition is avoided.
Drawings
FIG. 1 is a flow chart of the steps of a method implementation of the present invention.
FIG. 2 is a schematic diagram of the present invention for generating a truth thermodynamic diagram using a Gaussian kernel template in this example.
FIG. 3 is a schematic flow chart of a thermodynamic model training phase in the method of the present invention.
Fig. 4 is a diagram showing the effect of the method of the present invention on the post-cropping image authentication data in this example.
Fig. 5 is a graph of the effect of the implementation of the method of the invention in the original large image in this example.
Fig. 6 is a system configuration diagram of the present invention.
Detailed Description
The following describes specific embodiments of the present invention in detail with reference to the drawings. It should be understood that the detailed description and specific examples, while indicating and illustrating the invention, are not intended to limit the invention.
Example 1
As shown in FIG. 1, the high-density bean counting and center point positioning method based on thermodynamic diagrams comprises two stages, namely a training stage of a thermodynamic diagram model and a center point positioning stage. In the center point positioning stage, the thermodynamic diagram model is obtained by training, the thermodynamic diagram of the image to be measured is obtained in the verification set or the test set, and the center point position of the bean is obtained in the thermodynamic diagram. The following describes in detail the implementation procedure of a high-density bean counting and center point positioning method based on thermodynamic diagram according to the present invention with reference to the flowchart shown in fig. 1:
step S1: and marking the center point of the bean. Marking the central point positions of all beans in the image, and recording coordinates (x i,yi) in the image, wherein the coordinates respectively represent the abscissa and the ordinate of the ith bean. Marking is carried out by using a marking tool, and the coordinates (x i,yi) of the center points of all the beans in the image are recorded, wherein the coordinates respectively represent the abscissa and the ordinate of the ith bean.
Preferred examples: the labeling tool is LableMe, the elements in the coordinates are floating point numbers, and the labeled data are stored in a 'json' format. For example, if 2 beans are included in image 1.Jpg, the result can be saved as follows:
{"imagePath":"1.jpg",
"imageHeight":576,
"imageWidth":576,
"points":[
[257.1724137931035,63.793103448275865],
[282.6034482758621,84.48275862068967]
]}
step S2: and (3) generating a thermodynamic diagram H (x, y) of the bean by using the position of the center point of the bean marked in the step SS1 and using a Gaussian function.
Specifically, the thermodynamic diagram is an image with the same length and width as the original image and the channel number of 1, each pixel point in the image is a floating point number between 0 and 1, and the larger the value is, the higher the probability that the point exists in the center point of the bean is.
The thermodynamic diagram initialization values are all 0.
As shown in fig. 2, a gaussian kernel template is obtained through gaussian function calculation, and the gaussian kernel template is attached to the position of the center point of the bean marked in the SS1, so that a true value thermodynamic diagram of the current image is obtained.
Specifically, the gaussian kernel size ksize of the gaussian function is set to 3, the standard deviation σ is 0.5, the size of the gaussian kernel template is a 3×3 two-dimensional array, the sum of elements is 1, and the element value at the center point position is the largest:
[[0.01134374,0.08381951,0.01134374],
[0.08381951,0.61934703,0.08381951],
[0.01134374,0.08381951,0.01134374]]
And aligning the center position of the Gaussian kernel template with the position of the center point marked by the bean, adding the center position to the initialized thermodynamic diagram, and repeating the steps until all the center points are processed. In particular, an additional way of gaussian kernel templates and thermodynamic diagrams is pixel element addition.
And S3, constructing a bean thermodynamic diagram generation model. And training the constructed bean thermodynamic diagram network by utilizing the generated true thermodynamic diagram data.
The construction of the high bean thermodynamic diagram network is based on a CSRNet model of cavity convolution, VGG16 is a basic module, 6 layers of cavity convolution layers are connected to the back, the convolution kernel size is 3 multiplied by 3, the cavity rate is 2, and the channel number is [512,512,512,256,128,64]. Then a convolution layer with the convolution kernel size of 1 multiplied by 1 and the channel number of 1 is connected to obtain a thermodynamic diagram which is reduced by 8 times, finally, the thermodynamic diagram is directly subjected to 8 times up-sampling, the original diagram is restored to the size, and the up-sampling function adopts nearest neighbor.
In a preferred example, the VGG16 base module uses a model trained on ImageNet for weight initialization.
Training is performed on the constructed CSRNet model using training data, as shown in figure 3.
First, the original input image and the true value thermodynamic diagram are subjected to data enhancement by adopting two modes of random horizontal overturn and pairing clipping. The probability value of random horizontal overturn is 0.5, a floating point number is randomly generated during each training, and if the probability value is smaller than the value, overturn is not carried out. And setting the factor of the paired clipping to be 16, and if the length and width of the image do not meet the integral multiple of 16, clipping the original image and the thermodynamic diagram to the pixel part which is less than the integral multiple.
Then, using the enhanced data, inputting into the constructed model, predicting thermodynamic diagrams of pod countsSpecifically, the obtained thermodynamic diagram is a characteristic diagram having the same size as the original diagram to be input and having the number of channels of 1.
Then, a predicted thermodynamic diagram is calculated from the loss function LAnd (3) carrying out gradient transfer by utilizing a back propagation algorithm in the neural network with errors between the true thermodynamic diagram H (x, y) obtained through the Gaussian kernel template, and completing one-time CSRNet updating of model parameters.
Finally, if the convergence condition is reached, ending the iteration; if the convergence condition is not reached, execution is continued from the beginning. The convergence condition refers to the number of sample parameter learning iterations, and if all training samples are learned, the iteration is generally set to 100.
And S4, positioning the center point of the bean from the thermodynamic diagram. And (3) outputting a thermodynamic diagram by using the thermodynamic diagram estimation model obtained in the step (S3), selecting a point with the maximum value in the current area as the center point position of the bean based on the characteristics of the bean, and outputting the position coordinates, thereby obtaining the number of the bean in the image and the center position coordinates of the bean.
Specifically, the test piece may be the same as the training data, that is, the picture on the whole plant after cutting, as shown in fig. 4, or may be a complete soybean picture, as shown in fig. 5. The cut-out picture refers to a soybean partial graph obtained by cutting out the whole soybean plant according to the steps of 256 and 512×512.
The test piece to be tested is input into the trained model to generate a thermodynamic diagram, as shown in fig. 4 (c). Wherein each pixel in the thermodynamic diagram represents the possible number of beans in the area, and the value is 0-1.
The maximum point in the thermodynamic diagram region is obtained by the following steps:
Firstly, setting the pixel point smaller than the preset threshold value in the predicted thermodynamic diagram as 0, and respectively expanding the upper, lower, left and right boundaries in the thermodynamic diagram by 2 pixels, wherein the operation is to calculate the pixel value of the edge conveniently, and the expanded pixel value is 0. Specifically, the preset threshold is 0.2, below which noise interference is considered.
Next, the pixel value P up,Pdown,Pleft,Pright at the four positions of up, down, left, and right of each center pixel P center is obtained. In particular, the pixel values of the region range are added to obtain the possible bean values for the partial region, i.e., the value of P center is re-expressed as the sum of the original P center and the four position pixels:
Then, by judging whether the center point is locally maximum, all the locally maximum point positions P all are obtained. Whether the local is maximum or not is judged according to And/>And/> And is also provided with
Further, all the obtained center point positions are subjected to deduplication to obtain final all the center positions of the beans, as shown in fig. 4 (d). Specifically, P all is ordered in the order from smaller to larger on the abscissa, the euclidean distance between the P all and the subsequent point is calculated from the first point, if the distance is smaller than the set threshold value 30, the P all is considered as the coincident point, and the final counting of beans and the calculation of the central point position are not participated. Until all the local maximum point positions are calculated, obtaining the center point position of the bean after center point de-duplication
The counting of the number of the beans at the center point of the beans is thatBy/>, for that locationAnd (5) carrying out upward rounding on the value to obtain the bean number of the center point. The Round-up means that the floating point number is added with 1 as long as the floating point number has a decimal number, so as to obtain a corresponding integer, for example, round (1.610293) =2 and Round (2.022882) =3.
All the center point positions in each image can be obtained, and the number of beans can be obtained. If in 1.Jpg, 2 center points of beans are predicted, the number of beans is 1 and 2, respectively, the results are (273,82,1), (352,88,1), respectively, where the first two elements represent the abscissa and ordinate in the image, respectively, and the last element is the number of beans at that location. In this picture there are total 3 beans.
Example 2
Referring to fig. 6, a thermodynamic diagram-based high density bean counting and center point positioning system of the present invention is used to implement the method of embodiment 1, comprising:
The bean center point marking module is used for marking the center points of all beans in the image to obtain the coordinates of the center points of all the beans;
The bean thermodynamic diagram generating module is used for generating a Gaussian kernel template through a Gaussian function at the marked position of the center point of the bean, and obtaining a true value thermodynamic diagram through pixel value addition in the initial thermodynamic diagram;
The bean thermodynamic diagram generation model construction module is used for inputting the original image and the true thermodynamic diagram into the constructed thermodynamic diagram generation model, and training to obtain a final thermodynamic diagram generation model;
And the bean center point positioning module is used for inputting the picture to be tested into a final trained thermodynamic diagram generation model, obtaining the thermodynamic diagram of the picture to be tested, and obtaining the center point coordinates and the corresponding bean number by calculating the maximum value of the local area.
Example 3
The present invention also relates to a computer-readable storage medium having stored thereon a program which, when executed by a processor, implements a thermodynamic diagram-based high-density bean counting and center point positioning method of embodiment 1.
Example 4
The invention also relates to a computing device for implementing the method of embodiment 1, comprising a memory and a processor, wherein the memory stores executable code, and the processor implements the high-density bean counting and center point positioning method based on thermodynamic diagram of the invention when executing the executable code.
At the hardware level, the computing device includes a processor, internal bus, network interface, memory, and non-volatile storage, although other services may be required. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs to implement the method described above with respect to fig. 1. Of course, other implementations, such as logic devices or combinations of hardware and software, are not excluded from the present invention, that is, the execution subject of the following processing flows is not limited to each logic unit, but may be hardware or logic devices.
Improvements to one technology can clearly distinguish between improvements in hardware (e.g., improvements to circuit structures such as diodes, transistors, switches, etc.) and software (improvements to the process flow). However, with the development of technology, many improvements of the current method flows can be regarded as direct improvements of hardware circuit structures. Designers almost always obtain corresponding hardware circuit structures by programming improved method flows into hardware circuits. Therefore, an improvement of a method flow cannot be said to be realized by a hardware entity module. For example, a programmable logic device (Programmable Logic Device, PLD) (e.g., field programmable gate array (Field Programmable GATE ARRAY, FPGA)) is an integrated circuit whose logic functions are determined by user programming of the device. A designer programs to "integrate" a digital system onto a PLD without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Moreover, nowadays, instead of manually manufacturing integrated circuit chips, such programming is mostly implemented with "logic compiler (logic compiler)" software, which is similar to the software compiler used in program development and writing, and the original code before being compiled is also written in a specific programming language, which is called hardware description language (Hardware Description Language, HDL), but HDL is not just one, but a plurality of kinds, such as ABEL(Advanced Boolean Expression Language)、AHDL(Altera Hardware Description Language)、Confluence、CUPL(Cornell University Programming Language)、HDCal、JHDL(Java Hardware Description Language)、Lava、Lola、MyHDL、PALASM、RHDL(Ruby Hardware Description Language), and VHDL (Very-High-SPEED INTEGRATED Circuit Hardware Description Language) and Verilog are currently most commonly used. It will also be apparent to those skilled in the art that a hardware circuit implementing the logic method flow can be readily obtained by merely slightly programming the method flow into an integrated circuit using several of the hardware description languages described above.
The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, application SPECIFIC INTEGRATED Circuits (ASICs), programmable logic controllers, and embedded microcontrollers, examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, atmel AT91SAM, microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic of the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller in a pure computer readable program code, it is well possible to implement the same functionality by logically programming the method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Such a controller may thus be regarded as a kind of hardware component, and means for performing various functions included therein may also be regarded as structures within the hardware component. Or even means for achieving the various functions may be regarded as either software modules implementing the methods or structures within hardware components.
The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each element may be implemented in the same piece or pieces of software and/or hardware when implementing the present invention.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The invention may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
The embodiments of the present invention are described in a progressive manner, and the same and similar parts of the embodiments are all referred to each other, and each embodiment is mainly described in the differences from the other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
The foregoing is merely exemplary of the present invention and is not intended to limit the present invention. Various modifications and variations of the present invention will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the invention are to be included in the scope of the claims of the present invention.

Claims (14)

1. A high-density bean counting and center point positioning method based on thermodynamic diagram is characterized by comprising the following steps:
S1, marking a center point of a bean; marking the central point positions of all beans in the image to obtain the central point coordinates of all the beans;
s2, generating a bean thermodynamic diagram; generating a Gaussian kernel template at the marked position of the center point of the bean by a Gaussian function, and obtaining a true value thermodynamic diagram by adding pixel values in the initial thermodynamic diagram;
S3, constructing a bean thermodynamic diagram generation model; inputting the original image and the true thermodynamic diagram into a constructed thermodynamic diagram generating model, and training to obtain a final thermodynamic diagram generating model;
S4, positioning the center point of the bean from the thermodynamic diagram; inputting a picture to be tested into a final trained thermodynamic diagram generation model, obtaining a thermodynamic diagram of the picture to be tested, and obtaining a center point coordinate and a corresponding bean number by calculating a local area maximum value, wherein the method comprises the following steps of:
S41, inputting an attempt sheet to be tested into the trained thermodynamic diagram model to obtain a thermodynamic diagram with the same size as an input image and 1 channel number;
S42, analyzing pixel points larger than four directions of up, down, left and right pixel by pixel from the obtained thermodynamic diagram, and taking the largest point in the area as an alternative central point;
S43, sorting the alternative center points, and performing de-duplication according to the distance between the pixel points to obtain the final bean center point position;
S44, upwards rounding the thermal value of the final bean center point position to obtain the bean number of the bean center point position, and adding the bean numbers of all the bean center point positions to obtain the final total number of the bean.
2. The high-density bean counting and center positioning method based on thermodynamic diagram as claimed in claim 1, wherein step S1 marks the coordinates of the center points of all beans, and one bean marks only one position.
3. The method for counting and positioning the center point of high-density beans based on thermodynamic diagrams as recited in claim 1, wherein the step S2 comprises the following steps:
s21, initializing a thermodynamic diagram H (x, y) of an original image, and carrying out two-dimensional image with the same width and height as the original image, 1 channel number and 0 pixel value;
S22, generating a Gaussian kernel template by using a Gaussian function; generating a gaussian kernel template with a size of ksize and a sum of 1 based on two parameters of a size ksize and a standard deviation sigma of a given gaussian kernel; in the Gaussian kernel template, the pixel position value of the center point is maximum, and the value far away from the center pixel point is gradually reduced;
And S23, adding the Gaussian kernel template obtained in the step S22 to the position with the marked bean center point, and adding the pixel value of the position to obtain a new pixel value if the pixel value is added to the position.
4. The method for counting and positioning center points of high-density beans based on thermodynamic diagrams as claimed in claim 3, wherein the gaussian function in step S22 is a common two-dimensional gaussian function, and the calculation method is as follows:
5. A method for high density bean counting and center point positioning based on thermodynamic diagrams as claimed in claim 3, wherein said adding the gaussian kernel template obtained in step S23 to the position of the marked bean center point comprises: and aligning the center point of the Gaussian kernel template with the marked bean center point in the initialized thermodynamic diagram, and adding a Gaussian model to the corresponding positions in the thermodynamic diagram one by one aiming at the marked bean points to obtain a true thermodynamic diagram.
6. The high-density bean counting and center point positioning method based on thermodynamic diagram as claimed in claim 1, wherein the thermodynamic diagram generating model already constructed in step S3 adopts CSRNet model based on hole convolution, the basic network is VGG16, then the hole convolution of 6 layers is adopted, and the original input image size is recovered through up-sampling; initializing weights by using a model trained on an ImageNet;
after the hole convolution, an output characteristic diagram is obtained by adopting a convolution layer with the number of 1×1 and the output channels being 1, and the size of the original input image is restored by adopting 8 times of nearest neighbor up-sampling.
7. The method for counting and positioning the center point of high-density beans based on thermodynamic diagrams as recited in claim 1, wherein the step S3 comprises the steps of:
s31, carrying out data enhancement on an original image and a density map by adopting random horizontal overturning and matched cutting;
S32, inputting the image in the S31 into a built CSRNet, and predicting to obtain a thermodynamic diagram of the pod;
s33, calculating the distance loss between the thermodynamic diagram obtained in the S32 and the true thermodynamic diagram obtained by the marking in the S2;
S34, judging whether the model reaches a convergence condition, if so, ending training of the model, and outputting to obtain a final thermodynamic diagram estimation model; if the convergence condition is not reached, the sequential execution of S31, S32, S33, S34 is continued.
8. The method of claim 7, wherein the distance loss in step S33 is L2 loss:
9. The method for counting and positioning center points of high-density beans based on thermodynamic diagrams as claimed in claim 1, wherein the pixel-by-pixel analysis in step S42 is larger than the pixel points in the four directions of up, down, left and right, and the largest point in the area is taken as the alternative center point, comprising the following steps:
Acquiring pixel point values P up,Pdown,Pleft,Pright of four positions of an upper position, a lower position, a left position and a right position of each central pixel point P center;
The value of P center is re-represented as the sum of the original P center and four position pixels:
Judging Whether the position is the largest point in the area or not, if so, taking the position as an alternative central point; the judgment basis is as follows:
And/> And/>And/>
10. The method for counting and positioning center points of high-density beans based on thermodynamic diagram as set forth in claim 1, wherein the step S43 of sorting the candidate center points includes sorting in descending order according to the order of the abscissa from small to large, and then de-weighting according to the distance between pixels, and specifically includes: and respectively calculating Euclidean distances between the first and the subsequent points from the first point, if the distance is smaller than the set threshold value, the position smaller than the threshold value is regarded as a coincident point, and the final calculation of the counting and the central point position of the bean is not participated until all the local maximum point positions are calculated, and the central point position of the bean after the central point is de-weighted is obtained.
11. The method for counting and positioning the center point of high-density beans based on thermodynamic diagrams as recited in claim 1, wherein step S44 specifically comprises: further, forBy/>, for that locationThe values are rounded upwards to obtain the number of beans at the center point:
Further, the position of the center point of the bean is determined Combining the number of beans to obtain the number Q of the beans in the image and the center point position/>, of all the beansThe number of beans is the sum of the number of beans at each center point:
12. A thermodynamic diagram-based high-density bean counting and center point positioning system, comprising:
The bean center point marking module is used for marking the center points of all beans in the image to obtain the coordinates of the center points of all the beans;
The bean thermodynamic diagram generating module is used for generating a Gaussian kernel template through a Gaussian function at the marked position of the center point of the bean, and obtaining a true value thermodynamic diagram through pixel value addition in the initial thermodynamic diagram;
The bean thermodynamic diagram generation model construction module is used for inputting the original image and the true thermodynamic diagram into the constructed thermodynamic diagram generation model, and training to obtain a final thermodynamic diagram generation model;
And the bean center point positioning module is used for inputting the picture to be tested into a final trained thermodynamic diagram generation model, obtaining the thermodynamic diagram of the picture to be tested, and obtaining the center point coordinates and the corresponding bean number by calculating the maximum value of the local area.
13. A computer readable storage medium having stored thereon a program which, when executed by a processor, implements a thermodynamic diagram based high density bean counting and centre point positioning method according to any one of claims 1-11.
14. A computing device comprising a memory and a processor, wherein the memory has executable code stored therein, and wherein the processor, when executing the executable code, implements a thermodynamic diagram-based high density bean counting and center point positioning method as claimed in any one of claims 1 to 11.
CN202310406884.XA 2023-04-17 2023-04-17 High-density bean counting and center point positioning method and system based on thermodynamic diagram Active CN116703820B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310406884.XA CN116703820B (en) 2023-04-17 2023-04-17 High-density bean counting and center point positioning method and system based on thermodynamic diagram

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310406884.XA CN116703820B (en) 2023-04-17 2023-04-17 High-density bean counting and center point positioning method and system based on thermodynamic diagram

Publications (2)

Publication Number Publication Date
CN116703820A CN116703820A (en) 2023-09-05
CN116703820B true CN116703820B (en) 2024-05-03

Family

ID=87831829

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310406884.XA Active CN116703820B (en) 2023-04-17 2023-04-17 High-density bean counting and center point positioning method and system based on thermodynamic diagram

Country Status (1)

Country Link
CN (1) CN116703820B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111626349A (en) * 2020-05-22 2020-09-04 中国科学院空天信息创新研究院 Target detection method and system based on deep learning
US11200446B1 (en) * 2020-08-31 2021-12-14 Element Biosciences, Inc. Single-pass primary analysis
CN113947188A (en) * 2021-10-14 2022-01-18 北京百度网讯科技有限公司 Training method of target detection network and vehicle detection method
CN115063410A (en) * 2022-08-04 2022-09-16 中建电子商务有限责任公司 Steel pipe counting method based on anchor-free target detection

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111626349A (en) * 2020-05-22 2020-09-04 中国科学院空天信息创新研究院 Target detection method and system based on deep learning
US11200446B1 (en) * 2020-08-31 2021-12-14 Element Biosciences, Inc. Single-pass primary analysis
CN113947188A (en) * 2021-10-14 2022-01-18 北京百度网讯科技有限公司 Training method of target detection network and vehicle detection method
CN115063410A (en) * 2022-08-04 2022-09-16 中建电子商务有限责任公司 Steel pipe counting method based on anchor-free target detection

Also Published As

Publication number Publication date
CN116703820A (en) 2023-09-05

Similar Documents

Publication Publication Date Title
US9734587B2 (en) Long term object tracker
CN109978035B (en) Pedestrian detection method based on improved k-means and loss function
CN111292377B (en) Target detection method, device, computer equipment and storage medium
US20190279022A1 (en) Object recognition method and device thereof
EP3872761A2 (en) Analysing objects in a set of frames
CN112906816A (en) Target detection method and device based on optical differential and two-channel neural network
CN108446588B (en) Double-temporal remote sensing image change detection method and system
CN112802108A (en) Target object positioning method and device, electronic equipment and readable storage medium
CN113887608B (en) Model training method, image detection method and device
CN109902641B (en) Semantic alignment-based face key point detection method, system and device
US10089764B2 (en) Variable patch shape synthesis
CN116703820B (en) High-density bean counting and center point positioning method and system based on thermodynamic diagram
CN113221855A (en) Small target detection method and system based on scale sensitive loss and feature fusion
CN107077617B (en) Fingerprint extraction method and device
CN117197781A (en) Traffic sign recognition method and device, storage medium and electronic equipment
CN110503631B (en) Remote sensing image change detection method
CN116805393A (en) Hyperspectral image classification method and system based on 3DUnet spectrum-space information fusion
CN111368599B (en) Remote sensing image sea surface ship detection method and device, readable storage medium and equipment
CN112949498B (en) Target key point detection method based on heterogeneous convolutional neural network
CN114998755A (en) Method and device for matching landmarks in remote sensing image
KR102398260B1 (en) Method and apparatus for camera calibration
CN116721412B (en) Pod key point detection method and system based on structural priori from bottom to top
CN117079060B (en) Intelligent blade classification method and system based on photosynthetic signals
CN116363390B (en) Infrared dim target detection method and device, storage medium and electronic equipment
CN116188919B (en) Test method and device, readable storage medium and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant