CN117197148B - Thermal imaging-based wood boring pest detection method, system and medium - Google Patents
Thermal imaging-based wood boring pest detection method, system and medium Download PDFInfo
- Publication number
- CN117197148B CN117197148B CN202311477169.1A CN202311477169A CN117197148B CN 117197148 B CN117197148 B CN 117197148B CN 202311477169 A CN202311477169 A CN 202311477169A CN 117197148 B CN117197148 B CN 117197148B
- Authority
- CN
- China
- Prior art keywords
- image data
- wood
- boring
- probability value
- detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000002023 wood Substances 0.000 title claims abstract description 252
- 241000607479 Yersinia pestis Species 0.000 title claims abstract description 172
- 238000001931 thermography Methods 0.000 title claims abstract description 89
- 238000001514 detection method Methods 0.000 title claims description 244
- 241000238631 Hexapoda Species 0.000 claims abstract description 170
- 238000000034 method Methods 0.000 claims abstract description 56
- 230000002159 abnormal effect Effects 0.000 claims description 107
- 230000000694 effects Effects 0.000 claims description 42
- 238000011156 evaluation Methods 0.000 claims description 23
- 238000012544 monitoring process Methods 0.000 claims description 16
- 238000004458 analytical method Methods 0.000 claims description 14
- 230000007547 defect Effects 0.000 claims description 14
- 238000012937 correction Methods 0.000 claims description 13
- 238000007781 pre-processing Methods 0.000 claims description 12
- 239000003550 marker Substances 0.000 claims description 11
- 230000007246 mechanism Effects 0.000 claims description 11
- 238000003860 storage Methods 0.000 claims description 11
- 230000008859 change Effects 0.000 claims description 6
- 238000012795 verification Methods 0.000 claims description 4
- 238000007405 data analysis Methods 0.000 claims description 3
- 238000003958 fumigation Methods 0.000 description 14
- 238000005553 drilling Methods 0.000 description 12
- 238000012545 processing Methods 0.000 description 12
- 210000001124 body fluid Anatomy 0.000 description 10
- 239000010839 body fluid Substances 0.000 description 10
- 238000004422 calculation algorithm Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 10
- 238000002372 labelling Methods 0.000 description 6
- 238000012549 training Methods 0.000 description 6
- 230000000007 visual effect Effects 0.000 description 6
- 230000003631 expected effect Effects 0.000 description 5
- 238000010438 heat treatment Methods 0.000 description 5
- 238000005070 sampling Methods 0.000 description 5
- 238000005520 cutting process Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 4
- 238000001704 evaporation Methods 0.000 description 4
- 230000008020 evaporation Effects 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 230000009466 transformation Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000009193 crawling Effects 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 239000006185 dispersion Substances 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 239000002316 fumigant Substances 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000033001 locomotion Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000004060 metabolic process Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000005855 radiation Effects 0.000 description 2
- 239000002994 raw material Substances 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 239000010875 treated wood Substances 0.000 description 2
- 229910000831 Steel Inorganic materials 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000009545 invasion Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 239000010959 steel Substances 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Landscapes
- Image Processing (AREA)
- Catching Or Destruction (AREA)
Abstract
The invention discloses a method, a system and a medium for detecting wood boring insect pests based on thermal imaging. Under the condition that pests exist in the wood, quarantine treatment is carried out on the wood, and whether a thermal imaging image marked as the boring pests is changed is judged by comparing detected image data before and after the quarantine treatment, so that the effectiveness of the quarantine treatment of the wood is judged, and the quarantine efficiency of the boring pests in the wood is effectively improved.
Description
Technical Field
The present application relates to the field of data processing and data transmission, and more particularly, to a method, system, and medium for detecting wood boring pests based on thermal imaging.
Background
Along with the continuous development of global economy integration, world trade flow is accelerated, and the influence of people on the increase of consumption demands, the diversification of consumption channels (such as foreign purchasing and shipping) and the like is added, so that the speed of a large number of goods products entering China is increased, meanwhile, the risk that harmful organisms enter China along with the goods entering China is increased, and once the harmful organisms enter China along with the goods, the harmful organisms can be planted or bred in China quickly, the influence on the regional ecological environment in China is great, and great economic losses can be suffered.
In order to protect domestic forest resources and maintain ecological balance development in recent years, many forest areas in China are gradually forbidden to be cut off commercially, so that the wood yield in China is greatly reduced year by year. The consumption demand of China on wood products is large in China, and the export of the wood products is large, so that the supply and demand gap of wood raw materials is further enlarged. The import of a large amount of wood not only fills the gap of industrial raw materials, but also plays a great role in promoting the economic construction of China, reducing the local resource demand and indirectly protecting the ecological environment, but also has a great variety of harmful organisms entering along with the wood, and the risk pressure of invasion of external organisms is upward.
At present, the on-site quarantine of port entry timber, wooden package and wood products is based on manual auxiliary cutting, sampling and detection, and the time from unpacking and checking to sampling and checking is long, and the efficiency is low. Up to now, no technology exists which can perform on-site nondestructive detection primary screening on the steel plate, so that on-site quarantine can be rapidly and effectively implemented.
Therefore, the prior art has defects, and improvement is needed.
Disclosure of Invention
In view of the problems, the invention aims to provide a method, a system and a medium for detecting wood boring insect pests based on thermal imaging, which can overcome the problems of time and labor waste of manual sampling, breakage and cutting of ports, can meet the requirements of on-site quarantine nondestructive detection of ports, can save time and labor, and can improve the quarantine inspection efficiency. The invention has great economic and social benefits for preventing harmful organisms from being transferred into national borders along with wood and products thereof, protecting the production safety of agriculture and forestry in China and guaranteeing the biological safety of national gates, and has certain popularization value.
The first aspect of the invention provides a method for detecting wood boring insect pests based on thermal imaging, which comprises the following steps:
detecting wood to be detected to obtain first detection image data;
inputting the first detection image data into a preset boring pest detection model for analysis, marking an abnormal region to obtain marked image data, and calculating to obtain a first probability value according to the image characteristics of the marked image data;
analyzing according to the position relation between the cavity area and the marked image data to obtain a second probability value;
calculating according to the first probability value and the second probability value to obtain a third probability value, and comparing the third probability value with a first preset threshold interval to obtain the boring pest detection data in the marked image data;
integrating the detected data of the wood boring insect pests in all the marked image data to obtain the detected data of the wood boring insect pests;
quarantine treatment is carried out on the wood with the moths according to the wood moths detection data, the wood subjected to quarantine treatment is detected to obtain third detection image data, and the third detection image data and the first detection image data are compared to obtain wood quarantine treatment effect evaluation data.
In this scheme, still include:
performing image preprocessing on the first detection image data;
the image preprocessing includes non-uniform correction, blind pixel compensation and image enhancement.
In this scheme, input the first detection image data to the detection model of the class of boring pest of predetermineeing and carry out the analysis, mark abnormal region, obtain mark image data to according to the image feature calculation of mark image data obtains first probability value, include:
analyzing the first detection image data through an attention mechanism module to obtain a first abnormal region, and performing marking frame selection on the first abnormal region to obtain marked image data;
analyzing according to the marked image data, and determining the center coordinates of the first abnormal region;
extracting features of the marked image data through a depth residual error learning network module to obtain image features of the marked image data;
and comparing and calculating the image characteristics of the marked image data with the image characteristics of the sample image data in the preset model database to obtain a first probability value.
In this scheme, the analyzing according to the positional relationship between the hole area and the marked image data, obtaining the second probability value includes:
Analyzing according to the first detection image data, and marking the wormhole area in the wood to obtain wormhole area coordinate data;
calculating the minimum distance between the center coordinates of the first abnormal region in the marked image data and the cavity region coordinate data;
comparing the minimum distance with a second preset threshold value, and if the minimum distance is larger than the second preset threshold value, indicating that the marked object is a wood defect, wherein the second probability value is 0%;
otherwise, calculating the minimum distance by a preset method to obtain a second probability value;
and if the minimum distance is 0, the center coordinate of the abnormal region is in the cavity region, and the second probability value is 100%.
In this scheme, calculate according to first probability value and second probability value, obtain the third probability value to compare the third probability value with first default threshold interval, obtain the boring class pest detection data in the mark image data, include:
multiplying the first probability value and the second probability value by corresponding influence weights respectively to obtain a first weight score and a second weight score;
adding the first weight score and the second weight score to obtain a third probability value;
Judging whether the third probability value is in a first preset threshold interval or not;
if the third probability value is smaller than the minimum value of the first preset threshold interval, the fact that no boring insect exists in the marked image data is indicated;
if the third probability value is larger than the maximum value of the first preset threshold interval, indicating that the marked image data contain boring insects;
if the third probability value is in the first preset threshold interval, detecting the wood to be detected again after a preset time interval to obtain second detection image data;
comparing the second detection image data with the first detection image data, and judging whether the marked image data contains boring insects or not;
and integrating the comparison results of all the marked image data to obtain wood boring pest detection data.
In this scheme, will second detection image data compares with first detection image data, judges whether there is boring class pest in the mark image data, includes:
analyzing according to the second detection image data to obtain the center coordinates of a second abnormal region;
comparing the central coordinates of the second abnormal region with the central coordinates of the first abnormal region, and judging whether the central coordinates of the second abnormal region and the first abnormal region are consistent;
If not, judging the abnormal area in the marked image data as a boring pest;
if yes, judging the abnormal area in the marked image data as wood defect.
In this scheme, according to timber boring pest detection data is carried out quarantine treatment to the timber that has boring pest to detect the timber after quarantine treatment, obtain third detection image data, and with the third detection image data compares with first detection image data, obtain timber quarantine treatment effect evaluation data, include:
analyzing according to the wood boring pest detection data, and drawing a circle by taking the central coordinate of an abnormal region where the wood boring pest is located as the center of a circle and a preset radius to obtain a key monitoring region;
and comparing the third detection image data with the first detection image data based on the key monitoring area to obtain wood quarantine treatment effect evaluation data.
In this scheme, still include:
establishing a three-dimensional model of the boring insect according to the sample image data of the boring insect;
simulating the thermal imaging image of the boring insect in the wood by the three-dimensional model of the boring insect to obtain simulated sample image data;
And adding the simulated sample image data to a preset model database.
In a second aspect, the present invention provides a thermal imaging-based wood boring pest detection system, comprising:
the data acquisition module is used for detecting the wood to be detected to obtain first detection image data;
the data analysis module is used for inputting the first detection image data into a preset boring pest detection model for analysis, marking the abnormal region to obtain marked image data, and calculating to obtain a first probability value according to the image characteristics of the marked image data; analyzing according to the position relation between the cavity area and the marked image data to obtain a second probability value; calculating according to the first probability value and the second probability value to obtain a third probability value, and comparing the third probability value with a first preset threshold interval to obtain the boring pest detection data in the marked image data; integrating the detected data of the wood boring insect pests in all the marked image data to obtain the detected data of the wood boring insect pests;
and the quarantine treatment effect verification module is used for quarantining the wood with the boring insect according to the wood boring insect detection data, detecting the quarantined wood to obtain third detection image data, and comparing the third detection image data with the first detection image data to obtain wood quarantine treatment effect evaluation data.
In this scheme, still include:
performing image preprocessing on the first detection image data;
the image preprocessing includes non-uniform correction, blind pixel compensation and image enhancement.
In this scheme, input the first detection image data to the detection model of the class of boring pest of predetermineeing and carry out the analysis, mark abnormal region, obtain mark image data to according to the image feature calculation of mark image data obtains first probability value, include:
analyzing the first detection image data through an attention mechanism module to obtain a first abnormal region, and performing marking frame selection on the first abnormal region to obtain marked image data;
analyzing according to the marked image data, and determining the center coordinates of the first abnormal region;
extracting features of the marked image data through a depth residual error learning network module to obtain image features of the marked image data;
and comparing and calculating the image characteristics of the marked image data with the image characteristics of the sample image data in the preset model database to obtain a first probability value.
In this scheme, the analyzing according to the positional relationship between the hole area and the marked image data, obtaining the second probability value includes:
Analyzing according to the first detection image data, and marking the wormhole area in the wood to obtain wormhole area coordinate data;
calculating the minimum distance between the center coordinates of the first abnormal region in the marked image data and the cavity region coordinate data;
comparing the minimum distance with a second preset threshold value, and if the minimum distance is larger than the second preset threshold value, indicating that the marked object is a wood defect, wherein the second probability value is 0%;
otherwise, calculating the minimum distance by a preset method to obtain a second probability value;
and if the minimum distance is 0, the center coordinate of the abnormal region is in the cavity region, and the second probability value is 100%.
In this scheme, calculate according to first probability value and second probability value, obtain the third probability value to compare the third probability value with first default threshold interval, obtain the boring class pest detection data in the mark image data, include:
multiplying the first probability value and the second probability value by corresponding influence weights respectively to obtain a first weight score and a second weight score;
adding the first weight score and the second weight score to obtain a third probability value;
Judging whether the third probability value is in a first preset threshold interval or not;
if the third probability value is smaller than the minimum value of the first preset threshold interval, the fact that no boring insect exists in the marked image data is indicated;
if the third probability value is larger than the maximum value of the first preset threshold interval, indicating that the marked image data contain boring insects;
if the third probability value is in the first preset threshold interval, detecting the wood to be detected again after a preset time interval to obtain second detection image data;
comparing the second detection image data with the first detection image data, and judging whether the marked image data contains boring insects or not;
and integrating the comparison results of all the marked image data to obtain wood boring pest detection data.
In this scheme, will second detection image data compares with first detection image data, judges whether there is boring class pest in the mark image data, includes:
analyzing according to the second detection image data to obtain the center coordinates of a second abnormal region;
comparing the central coordinates of the second abnormal region with the central coordinates of the first abnormal region, and judging whether the central coordinates of the second abnormal region and the first abnormal region are consistent;
If not, judging the abnormal area in the marked image data as a boring pest;
if yes, judging the abnormal area in the marked image data as wood defect.
In this scheme, according to timber boring pest detection data is carried out quarantine treatment to the timber that has boring pest to detect the timber after quarantine treatment, obtain third detection image data, and with the third detection image data compares with first detection image data, obtain timber quarantine treatment effect evaluation data, include:
analyzing according to the wood boring pest detection data, and drawing a circle by taking the central coordinate of an abnormal region where the wood boring pest is located as the center of a circle and a preset radius to obtain a key monitoring region;
and comparing the third detection image data with the first detection image data based on the key monitoring area to obtain wood quarantine treatment effect evaluation data.
In this scheme, still include:
establishing a three-dimensional model of the boring insect according to the sample image data of the boring insect;
simulating the thermal imaging image of the boring insect in the wood by the three-dimensional model of the boring insect to obtain simulated sample image data;
And adding the simulated sample image data to a preset model database.
A third aspect of the present invention provides a computer-readable storage medium having embodied therein a thermal imaging-based wood boring pest detection method program which, when executed by a processor, implements the steps of a thermal imaging-based wood boring pest detection method as described in any one of the above.
The invention discloses a method, a system and a medium for detecting wood boring insect pests based on thermal imaging. Under the condition that pests exist in the wood, quarantine treatment is carried out on the wood, and whether a thermal imaging image marked as the boring pests is changed is judged by comparing detected image data before and after the quarantine treatment, so that the effectiveness of the quarantine treatment of the wood is judged, and the quarantine efficiency of the boring pests in the wood is effectively improved.
Drawings
FIG. 1 shows a flow chart of a method for detecting wood boring insect pests based on thermal imaging according to the present invention;
FIG. 2 is a flow chart of a first probability value calculation method of the present invention;
FIG. 3 shows a flow chart of a method of wood quarantine treatment and treatment effect verification of the present invention;
FIG. 4 shows a block diagram of a thermal imaging-based wood boring pest detection system of the present invention;
fig. 5 shows a schematic diagram of a wood boring pest detection flow according to the present invention.
List of reference numerals:
1: wood to be detected; 2: quarantine treated wood; 3: a thermal imaging detection device; 4: the cloud server of the Internet of things; 5: and presetting a model database.
Detailed Description
In order that the above-recited objects, features and advantages of the present invention will be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description. It should be noted that, in the case of no conflict, the embodiments of the present application and the features in the embodiments may be combined with each other.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those described herein, and therefore the scope of the present invention is not limited to the specific embodiments disclosed below.
FIG. 1 shows a flow chart of a method for detecting wood boring insect pests based on thermal imaging according to the invention.
As shown in fig. 1, the invention discloses a method for detecting wood boring insect pests based on thermal imaging, which comprises the following steps:
s102, detecting wood to be detected to obtain first detection image data;
s104, inputting the first detection image data into a preset boring pest detection model for analysis, marking an abnormal region to obtain marked image data, and calculating to obtain a first probability value according to the image characteristics of the marked image data;
s106, analyzing according to the position relation between the cavity area and the marked image data to obtain a second probability value;
s108, calculating according to the first probability value and the second probability value to obtain a third probability value, and comparing the third probability value with a first preset threshold interval to obtain the detected data of the boring insect in the marked image data;
s110, integrating the detected data of the wood boring insect in all the marked image data to obtain the detected data of the wood boring insect;
s112, quarantining the wood with the boring insects according to the wood boring insects detection data, detecting the quarantined wood to obtain third detection image data, and comparing the third detection image data with the first detection image data to obtain wood quarantining effect evaluation data.
According to the embodiment of the invention, as shown in fig. 5, the wood 1 to be detected is detected by the thermal imaging detection device 3, the detected image data obtained by infrared thermal imaging is transmitted to the internet of things cloud server 4 by the communication module, real-time image processing is performed by the internet of things cloud server 4, the abnormal areas in the detected image data are marked by the preset boring pest detection model, one or more abnormal areas possibly exist in the marking process, and therefore, one or more marked image data can be obtained by area marking. And then, sample image data of thermal imaging of the boring insect in the preset model database 5 is called, each marked image data is compared with the sample image data of the thermal imaging of the boring insect, the probability value of the boring insect in each marked image data is determined by combining the minimum distance between the marked image data and the boring hole area, and whether the boring insect exists in the current marked image data is judged according to the probability value of the boring insect, so that boring insect detection data in the marked image data is obtained. And integrating the detected data of the wood boring insect pests in all the marked image data after the analysis of all the marked image data is completed, so as to obtain the detected data of the wood boring insect pests. The wood boring pest detection data comprise the number, relative positions and the like of the boring pests. The wood boring pest detection data are fed back to the detection equipment terminal and other preset terminals (such as a mobile terminal or a PC terminal) and the results are stored, so that the data are convenient to inquire, download and statistically analyze, and sample image data in the preset model database 5 are expanded.
In addition, the user can select to quarantine the wood according to the wood boring pest detection data (such as fumigation treatment, heat treatment and the like), detect the quarantined wood 2 again, judge whether the wood quarantine treatment achieves the expected effect or not by comparing the infrared thermal imaging images before and after the quarantine treatment, and obtain the wood quarantine treatment effect evaluation data.
The method comprises the steps of obtaining a preset boring pest detection model through training historical detection image data, firstly marking the existence position of boring pests in the historical detection image data in a manual marking mode to obtain sample image data, and storing the obtained sample image data into a preset model database. And then carrying out data enhancement on the sample image data by carrying out image transformation modes such as rotation, overturn and the like on the sample image data. Training the initial boring pest detection model through sample image data to finally obtain a preset boring pest detection model.
According to the scheme, the temperature difference between the temperature of the insect bodies in the wood and the temperature of the wood is utilized, the metabolism and the vital activities of the boring insects mainly derived from living bodies can be maintained in the wood, and certain heat is generated through the cell activities, so that the temperature difference between the boring insects and the temperature of the wood and the temperature of the environment is shown. Once living boring insects exist in the wood, the change of the thermal conductivity of the wood is caused, so that the temperature conduction nonuniformity appears, and the temperature conduction nonuniformity is reflected on a thermal imaging image, so that the internal distinction from other normal wood is shown, and the effective detection of whether the living boring insects exist in the wood is achieved.
According to an embodiment of the present invention, further comprising:
performing image preprocessing on the first detection image data;
the image preprocessing includes non-uniform correction, blind pixel compensation and image enhancement.
It should be noted that, due to the limitations of development technology, level and materials, problems such as non-uniformity, blind pixels, and image blurring occur in the imaging process of the thermal imaging detection device, and the existence of the problems seriously affects the visual effect of the image. The pixel response rate of the thermal imaging detection equipment is inconsistent to cause the problems of non-uniformity and blind pixels of an image, and noise introduced by the thermal imaging detection equipment in the acquisition process can cause blurring of image details, so that acquired image signals are required to be further processed, and a series of processing procedures such as non-uniform correction, blind pixel compensation, image enhancement and the like are completed in the infrared image processing procedure through interaction with data between the external memory. The two-point correction algorithm is adopted, and the gain correction coefficient and the offset correction coefficient of each pixel in the focal plane array are compensated, so that the pixel response rate is consistent under a uniform radiation source. And adopting a neighborhood averaging method to infer and replace information at the blind pixel position.
Because the body state of the borer is extremely small, and the data acquired by thermal imaging is not rich in visible light level, the difficulty of identifying the target is high, the preprocessing can be carried out through the image enhancement module, and the image feature identification degree is improved
The pixels in the image are directly processed according to the gray mapping relation, which is the core of processing the image in the spatial domain; the image is modified based on Fourier transformation, so that the aim of enhancing the image is fulfilled, namely, the key of processing the image in the frequency domain is realized, and the processed image is more prominent and is easy to identify. The histogram equalization algorithm is selected to enhance the thermal imaging image, and the core is the gray level of the image, wherein the gray level contains less pixels which are compressed and more pixels which are expanded, so that the contrast of the image is improved. And training the model by using a CNN convolution network to obtain a deep learning model.
Fig. 2 is a flowchart showing a method for calculating a probability value of the presence of pests in wood according to the present invention.
As shown in fig. 2, according to an embodiment of the present invention, the inputting the first detection image data into a preset model for detecting the boring pest, analyzing the abnormal region, marking the abnormal region to obtain marked image data, and calculating a first probability value according to the image features of the marked image data includes:
S202, analyzing the first detection image data through an attention mechanism module to obtain a first abnormal region, and performing marking frame selection on the first abnormal region to obtain marked image data;
s204, analyzing according to the marked image data, and determining the center coordinates of the first abnormal region;
s206, extracting features of the marked image data through a depth residual error learning network module to obtain image features of the marked image data;
s208, comparing and calculating the image characteristics of the marked image data with the image characteristics of the sample image data in the preset model database to obtain a first probability value.
The method is characterized in that a preset boring pest detection model is a CNN bilinear model, a CNN bilinear model is used, a depth residual error learning network module and an attention mechanism module are introduced, and the improvement of the depth residual error network can improve a characteristic weighting network, so that the improved model achieves stronger characteristic learning capacity; the visual attention mechanism is introduced, the channel attention and the spatial attention module are fused into a new feature extraction network, and the accuracy of the model on image recognition is further improved. Aiming at the data uploaded by a user in the image input module, the image recognition module transmits the data to a server end and invokes the trained bilinear CNN recognition model to calculate the similarity between the marked image data and the thermal imaging sample image data of the boring insect pests so as to judge the existence condition of the boring insect pests in the wood, namely, the output result of the image recognition module.
Traversing the input image through the visual attention mechanism module, extracting an interested region in the image to obtain one or more first abnormal regions, namely regions suspected of having pests, carrying out frame selection on the regions through the marking frame to obtain marking image data, and taking the central point coordinate of each marking image data as the central coordinate of the corresponding first abnormal region. And then analyzing the marked image data through the depth residual error learning network module, extracting image characteristics of the marked image data, comparing the marked image data with sample image data in a preset model database, selecting sample image data with highest similarity as a comparison standard based on the image characteristics of the marked image data, comparing based on image characteristics such as color characteristics, texture characteristics, shape characteristics and the like of the image, calculating the probability of existence of boring pests in the marked image data, and outputting a first probability value.
The method comprises the steps of acquiring the types of wood and internal environment parameters (moisture content, temperature and the like) while detecting wood boring insect pests due to different thermal imaging effects of the wood in different environments, and preferentially selecting sample image data of the same types of wood and similar internal environment parameters of the wood for comparison when the characteristics of the wood boring insect pests are compared through a model, so that the detection efficiency of the model is improved.
According to an embodiment of the present invention, further comprising:
and amplifying the marked image data by a bilinear interpolation algorithm.
It should be noted that, the body state of the borer is small, for example, the borer is mostly about 3-8mm, and it is difficult to directly obtain the image characteristics of the borer, so that after the marking image data is obtained, the image is amplified. The marking image data is amplified through the bilinear interpolation algorithm, the real-time performance is good, the infrared image is amplified through the algorithm, the resolution of the image is improved, and meanwhile, the definition of the image can be ensured.
According to an embodiment of the present invention, the analyzing according to the positional relationship between the hole area and the marker image data, to obtain the second probability value includes:
analyzing according to the first detection image data, and marking the wormhole area in the wood to obtain wormhole area coordinate data;
calculating the minimum distance between the center coordinates of the first abnormal region in the marked image data and the cavity region coordinate data;
comparing the minimum distance with a second preset threshold value, and if the minimum distance is larger than the second preset threshold value, indicating that the marked object is a wood defect, wherein the second probability value is 0%;
Otherwise, calculating the minimum distance by a preset method to obtain a second probability value;
and if the minimum distance is 0, the center coordinate of the abnormal region is in the cavity region, and the second probability value is 100%.
When the boring pest moves in the target, a hole is generated, and the positional relationship between the marker image data and the hole area is determined by calculating the minimum distance between the center coordinates of the first abnormal area and the hole area coordinate data in the marker image data.
When the cavity exists in the wood, the heat conduction performance of the wood can be changed, and the edge characteristics of the cavity are extracted by analyzing the preprocessed image data, so that the cavity image is extracted, and the cavity area coordinate data are obtained. And calculating the linear distance between the center coordinates of the first abnormal region and each coordinate point in the cavity region coordinate data, and selecting a minimum value of the linear distance as the minimum distance between the marked image data and the cavity region.
Due to the influence of the internal environment of the wood and other factors, a deviation may occur between the detected target position and the actual target position, so that a certain distance difference exists between the marked image data and the cavity area. Comparing the minimum distance between the marked image data and the cavity area with a second preset threshold value, if the minimum distance is smaller than the second preset threshold value, indicating that the drilling and eating insect exists in the current marked image data, calculating through a preset method to obtain a second probability value of the drilling and eating insect existing in the marked image, wherein the second probability value indicates whether the drilling and eating insect exists in the wood or not according to the position relation between the marked image data and the cavity area. The second preset threshold is set through the system, and is dynamically adjusted according to analysis results of the historical detection image data in the using process of the system.
The preset method is expressed as a= (B-C)/B by a formula, where a represents a second probability value, B represents a second preset threshold value, and C represents a minimum distance between a center coordinate of the first abnormal region and the cavity region coordinate data in the marker image data.
According to an embodiment of the present invention, the calculating according to the first probability value and the second probability value to obtain a third probability value, and comparing the third probability value with a first preset threshold interval to obtain the detected data of the boring pest in the marked image data includes:
multiplying the first probability value and the second probability value by corresponding influence weights respectively to obtain a first weight score and a second weight score;
adding the first weight score and the second weight score to obtain a third probability value;
judging whether the third probability value is in a first preset threshold interval or not;
if the third probability value is smaller than the minimum value of the first preset threshold interval, the fact that no boring insect exists in the marked image data is indicated;
if the third probability value is larger than the maximum value of the first preset threshold interval, indicating that the marked image data contain boring insects;
if the third probability value is in the first preset threshold interval, detecting the wood to be detected again after a preset time interval to obtain second detection image data;
Comparing the second detection image data with the first detection image data, and judging whether the marked image data contains boring insects or not;
and integrating the comparison results of all the marked image data to obtain wood boring pest detection data.
The different parameters (image characteristics of the boring thermal imaging image, behavior habit of the insect, etc.) have different weights of influence on judging whether the boring insect exists in the marked image data, and the third probability value is obtained by calculating according to the weights of influence of the different parameters. The method is mainly used for judging whether the boring insect exists in the wood based on the image characteristics of the thermal imaging image of the boring insect, calculating the position relationship and the minimum distance between the marking image data and the boring hole image data in the wood, and playing an auxiliary role in judging whether the boring insect exists in the wood. Thus, the impact weight of the first probability value is greater than the impact weight of the second probability value. The system dynamically adjusts the influence weights of the first probability value and the second probability value based on the size of the first probability value, wherein the larger the first probability value is, the larger the influence weights of the first probability value and the second probability value are, but the maximum value of the sum of the influence weights of the first probability value and the second probability value is 1. And analyzing by combining a first preset threshold interval, judging whether the marked image data contains the boring insects, wherein the initial value of the first preset threshold interval is 30% -70%, and dynamically adjusting the first preset threshold according to the detection precision level selected by the user and the recognition precision of the boring insects in the historical detection data.
When the third probability value is smaller than the minimum value of the first preset threshold interval, the image characteristics of the marked image data, which basically do not contain the boring insect pests, are represented, and the marked image data are judged to contain no boring insect pests; when the third probability value is larger than the maximum value of the first preset threshold interval, the image characteristics of a large number of boring insects exist in the marked image data, and the boring insects exist in the marked image data; if the third probability value is between the first preset threshold values, the third probability value indicates that certain image features of the boring insect pests exist in the marked image, but the third probability value cannot be used as a basis for judging the boring insect pests. In this case, the wood to be detected may be detected again according to a preset time interval (e.g., one hour), and whether or not the boring insect exists in the marker image data may be judged by comparing the characteristic differences of the second detection image data and the first detection image data.
According to an embodiment of the present invention, the comparing the second detection image data with the first detection image data to determine whether the boring pest exists in the marking image data includes:
analyzing according to the second detection image data to obtain the center coordinates of a second abnormal region;
Comparing the central coordinates of the second abnormal region with the central coordinates of the first abnormal region, and judging whether the central coordinates of the second abnormal region and the first abnormal region are consistent;
if not, judging the abnormal area in the marked image data as a boring pest;
if yes, judging the abnormal area in the marked image data as wood defect.
Under the condition that whether the wood is damaged by the drilling or the boring is not determined according to the probability value of the damage, the wood to be detected can be detected again after a system preset time interval, the second detection image data is analyzed according to a preset drilling or boring pest detection model, the center coordinates of a second abnormal area in the second detection image are determined, and whether the drilling or boring is detected in the marked image data is judged by comparing the center coordinates of the abnormal areas in the two detection image data. In addition, in order to improve the recognition accuracy, multiple detections can be performed on the basis of two detections. If the central coordinates of the abnormal areas are consistent, the abnormal areas are knots or other wood defects; if the position coordinates of the marked object are changed, the living body of the abnormal region is represented, and meanwhile, certain image characteristics of the boring insect pests exist in the abnormal region when the first detection image data are analyzed, so that the abnormal region in the marked image data can be judged to be the boring insect pests.
Fig. 3 shows a flow chart of a method for verifying the fumigation effect of wood according to the invention.
As shown in fig. 3, according to an embodiment of the present invention, quarantining a wood with a boring pest according to the wood boring pest detection data, detecting the quarantined wood to obtain third detection image data, and comparing the third detection image data with the first detection image data to obtain wood quarantining effect evaluation data, where the method includes:
s302, analyzing according to the wood boring pest detection data, and drawing a circle by taking the central coordinate of an abnormal region where the boring pest is located as the center of a circle and a preset radius to obtain a key monitoring region;
s302, comparing the third detection image data with the first detection image data based on the key monitoring area to obtain wood quarantine treatment effect evaluation data.
The quarantine treatment includes fumigation treatment, heat treatment and the like, taking fumigation treatment as an example, after fumigation treatment is carried out on wood by using a fumigant, image acquisition is carried out on the wood after the quarantine treatment by using a thermal imaging detection device again at intervals after air dispersion is finished, body fluid evaporation is carried out after death of insects, an insect body heat imaging image cannot be detected by using the detection device, and the quarantine treatment effect is determined by comparing detected image data before and after fumigation. In order to avoid the influence of the movement of the boring insect in the quarantine process on the detection result, one or more key monitoring areas are defined according to the number of the insect and the central coordinates of the area where each insect is located, wherein the preset radius is set by combining the crawling speed of the system simulation boring insect in the wood with the quarantine treatment time.
When detecting the fumigated detection image data, the detection focus is placed on the focus monitoring area, and whether the expected effect of the wood quarantine treatment is achieved is determined by judging whether the pest mark image in the focus monitoring area has change or not.
In addition, a certain time is needed for evaporation of insect body fluid, and when the fumigated wood is detected, as the insect body fluid is evaporated completely and a certain amount of heat exists in residual body fluid, a new thermal imaging image appears in fumigated detection image data. Comparing the newly-appearing thermal imaging image with the pest marking image in the detected image data before fumigation, analyzing by combining sample image data in a preset model database, judging according to the image attribute (shape, size, brightness and the like) of the newly-appearing thermal imaging image, whether the image features of the thermal imaging image of the boring pest exist or not and determining that the newly-appearing thermal imaging image is the thermal imaging image after the boring pest moves or the thermal imaging image generated by residual body fluid after the boring pest dies. Labeling the suspicious data, and detecting again after a period of time or verifying by a manual cutting sampling detection mode to eliminate the suspicious data so as to obtain the wood quarantine treatment effect evaluation data. And performing the wood quarantine treatment again with poor quarantine treatment effect according to the obtained wood quarantine treatment effect evaluation data until the wood quarantine treatment effect meets the expected requirement.
According to an embodiment of the present invention, further comprising:
establishing a three-dimensional model of the boring insect according to the sample image data of the boring insect;
simulating a thermal imaging image of pests in the wood through the three-dimensional model of the boring pests to obtain simulated sample image data;
and adding the simulated sample image data to a preset model database.
It should be noted that, the thermal imaging sample images of the boring insects in the timber in the preset model database are fewer, the sample image data can be basically collected only by collecting the historical detection image data and labeling the historical detection image data by manual labeling, on the basis, the image data of the boring insects can be obtained through the internet, the three-dimensional model of the boring insects is built through the image data of the boring insects, the thermal imaging effect of the insects in the timber is simulated based on the parameters of the moisture content, the temperature, the type, the depth of the insects and the like of the timber, the simulated sample image is obtained, and the simulated sample image is added into the preset model database, so that the accuracy and the generalization capability of the model are improved.
Fig. 4 shows a block diagram of a wood boring pest detection system based on thermal imaging according to the present invention.
As shown in fig. 4, a second aspect of the present invention provides a wood boring pest detection system based on thermal imaging, comprising:
The data acquisition module is used for detecting the wood to be detected to obtain first detection image data;
the data analysis module is used for inputting the first detection image data into a preset boring pest detection model for analysis, marking the abnormal region to obtain marked image data, and calculating to obtain a first probability value according to the image characteristics of the marked image data; analyzing according to the position relation between the cavity area and the marked image data to obtain a second probability value; calculating according to the first probability value and the second probability value to obtain a third probability value, and comparing the third probability value with a first preset threshold interval to obtain the boring pest detection data in the marked image data; integrating the detected data of the wood boring insect pests in all the marked image data to obtain the detected data of the wood boring insect pests;
and the quarantine treatment effect verification module is used for quarantining the wood with the boring insect according to the wood boring insect detection data, detecting the quarantined wood to obtain third detection image data, and comparing the third detection image data with the first detection image data to obtain wood quarantine treatment effect evaluation data.
According to the embodiment of the invention, as shown in fig. 5, the wood 1 to be detected is detected by the thermal imaging detection device 3, the detected image data obtained by infrared thermal imaging is transmitted to the internet of things cloud server 4 by the communication module, real-time image processing is performed by the internet of things cloud server 4, the abnormal areas in the detected image data are marked by the preset boring pest detection model, one or more abnormal areas possibly exist in the marking process, and therefore, one or more marked image data can be obtained by area marking. And then, sample image data of thermal imaging of the boring insect in the preset model database 5 is called, each marked image data is compared with the sample image data of the thermal imaging of the boring insect, the probability value of the boring insect in each marked image data is determined by combining the minimum distance between the marked image data and the boring hole area, and whether the boring insect exists in the current marked image data is judged according to the probability value of the boring insect, so that boring insect detection data in the marked image data is obtained. And integrating the detected data of the wood boring insect pests in all the marked image data after the analysis of all the marked image data is completed, so as to obtain the detected data of the wood boring insect pests. The wood boring pest detection data comprise the number, relative positions and the like of the boring pests. The wood boring pest detection data are fed back to the detection equipment terminal and other preset terminals (such as a mobile terminal or a PC terminal) and the results are stored, so that the data are convenient to inquire, download and statistically analyze, and sample image data in the preset model database 5 are expanded.
In addition, the user can select to quarantine the wood according to the wood boring pest detection data (such as fumigation treatment, heat treatment and the like), detect the quarantined wood 2 again, judge whether the wood quarantine treatment achieves the expected effect or not by comparing the infrared thermal imaging images before and after the quarantine treatment, and obtain the wood quarantine treatment effect evaluation data.
In addition, the user can select to quarantine the wood (such as fumigation treatment, heat treatment and the like) according to the wood boring pest detection data, detect the quarantined wood again, judge whether the wood quarantine treatment achieves the expected effect or not by comparing the infrared thermal imaging images before and after the quarantine treatment, and obtain the wood quarantine treatment effect evaluation data.
The method comprises the steps of obtaining a preset boring pest detection model through training historical detection image data, firstly marking the existence position of boring pests in the historical detection image data in a manual marking mode to obtain sample image data, and storing the obtained sample image data into a preset model database. And then carrying out data enhancement on the sample image data by carrying out image transformation modes such as rotation, overturn and the like on the sample image data. Training the initial boring pest detection model through sample image data to finally obtain a preset boring pest detection model.
According to the scheme, the temperature difference between the temperature of the insect bodies in the wood and the temperature of the wood is utilized, the metabolism and the vital activities of the boring insects mainly derived from living bodies can be maintained in the wood, and certain heat is generated through the cell activities, so that the temperature difference between the boring insects and the temperature of the wood and the temperature of the environment is shown. Once living boring insects exist in the wood, the change of the thermal conductivity of the wood is caused, so that the temperature conduction nonuniformity appears, and the temperature conduction nonuniformity is reflected on a thermal imaging image, so that the internal distinction from other normal wood is shown, and the effective detection of whether the living boring insects exist in the wood is achieved.
According to an embodiment of the present invention, further comprising:
performing image preprocessing on the first detection image data;
the image preprocessing includes non-uniform correction, blind pixel compensation and image enhancement.
It should be noted that, due to the limitations of development technology, level and materials, problems such as non-uniformity, blind pixels, and image blurring occur in the imaging process of the thermal imaging detection device, and the existence of the problems seriously affects the visual effect of the image. The pixel response rate of the thermal imaging detection equipment is inconsistent to cause the problems of non-uniformity and blind pixels of an image, and noise introduced by the thermal imaging detection equipment in the acquisition process can cause blurring of image details, so that acquired image signals are required to be further processed, and a series of processing procedures such as non-uniform correction, blind pixel compensation, image enhancement and the like are completed in the infrared image processing procedure through interaction with data between the external memory. The two-point correction algorithm is adopted, and the gain correction coefficient and the offset correction coefficient of each pixel in the focal plane array are compensated, so that the pixel response rate is consistent under a uniform radiation source. And adopting a neighborhood averaging method to infer and replace information at the blind pixel position.
Because the body state of the borer is extremely small, and the data acquired by thermal imaging is not rich in visible light level, the difficulty of identifying the target is high, the preprocessing can be carried out through the image enhancement module, and the image feature identification degree is improved
The pixels in the image are directly processed according to the gray mapping relation, which is the core of processing the image in the spatial domain; the image is modified based on Fourier transformation, so that the aim of enhancing the image is fulfilled, namely, the key of processing the image in the frequency domain is realized, and the processed image is more prominent and is easy to identify. The histogram equalization algorithm is selected to enhance the thermal imaging image, and the core is the gray level of the image, wherein the gray level contains less pixels which are compressed and more pixels which are expanded, so that the contrast of the image is improved. And training the model by using a CNN convolution network to obtain a deep learning model.
According to an embodiment of the present invention, the inputting the first detection image data into a preset boring pest detection model for analysis, marking an abnormal region to obtain marked image data, and calculating according to image features of the marked image data to obtain a first probability value includes:
analyzing the first detection image data through an attention mechanism module to obtain a first abnormal region, and performing marking frame selection on the first abnormal region to obtain marked image data;
Analyzing according to the marked image data, and determining the center coordinates of the first abnormal region;
extracting features of the marked image data through a depth residual error learning network module to obtain image features of the marked image data;
and comparing and calculating the image characteristics of the marked image data with the image characteristics of the sample image data in the preset model database to obtain a first probability value.
The method is characterized in that a preset boring pest detection model is a CNN bilinear model, a CNN bilinear model is used, a depth residual error learning network module and an attention mechanism module are introduced, and the improvement of the depth residual error network can improve a characteristic weighting network, so that the improved model achieves stronger characteristic learning capacity; the visual attention mechanism is introduced, the channel attention and the spatial attention module are fused into a new feature extraction network, and the accuracy of the model on image recognition is further improved. Aiming at the data uploaded by a user in the image input module, the image recognition module transmits the data to a server end and invokes the trained bilinear CNN recognition model to calculate the similarity between the marked image data and the thermal imaging sample image data of the boring insect pests so as to judge the existence condition of the boring insect pests in the wood, namely, the output result of the image recognition module.
Traversing the input image through the visual attention mechanism module, extracting an interested region in the image to obtain one or more first abnormal regions, namely regions suspected of having pests, carrying out frame selection on the regions through the marking frame to obtain marking image data, and taking the central point coordinate of each marking image data as the central coordinate of the corresponding first abnormal region. And then analyzing the marked image data through the depth residual error learning network module, extracting image characteristics of the marked image data, comparing the marked image data with sample image data in a preset model database, selecting sample image data with highest similarity as a comparison standard based on the image characteristics of the marked image data, comparing based on image characteristics such as color characteristics, texture characteristics, shape characteristics and the like of the image, calculating the probability of existence of boring pests in the marked image data, and outputting a first probability value.
The method comprises the steps of acquiring the types of wood and internal environment parameters (moisture content, temperature and the like) while detecting wood boring insect pests due to different thermal imaging effects of the wood in different environments, and preferentially selecting sample image data of the same types of wood and similar internal environment parameters of the wood for comparison when the characteristics of the wood boring insect pests are compared through a model, so that the detection efficiency of the model is improved.
According to an embodiment of the present invention, further comprising:
and amplifying the marked image data by a bilinear interpolation algorithm.
It should be noted that, the body state of the borer is small, for example, the borer is mostly about 3-8mm, and it is difficult to directly obtain the image characteristics of the borer, so that after the marking image data is obtained, the image is amplified. The marking image data is amplified through the bilinear interpolation algorithm, the real-time performance is good, the infrared image is amplified through the algorithm, the resolution of the image is improved, and meanwhile, the definition of the image can be ensured.
According to an embodiment of the present invention, the analyzing according to the positional relationship between the hole area and the marker image data, to obtain the second probability value includes:
analyzing according to the first detection image data, and marking the wormhole area in the wood to obtain wormhole area coordinate data;
calculating the minimum distance between the center coordinates of the first abnormal region in the marked image data and the cavity region coordinate data;
comparing the minimum distance with a second preset threshold value, and if the minimum distance is larger than the second preset threshold value, indicating that the marked object is a wood defect, wherein the second probability value is 0%;
Otherwise, calculating the minimum distance by a preset method to obtain a second probability value;
and if the minimum distance is 0, the center coordinate of the abnormal region is in the cavity region, and the second probability value is 100%.
When the boring pest moves in the target, a hole is generated, and the positional relationship between the marker image data and the hole area is determined by calculating the minimum distance between the center coordinates of the first abnormal area and the hole area coordinate data in the marker image data.
When the cavity exists in the wood, the heat conduction performance of the wood can be changed, and the edge characteristics of the cavity are extracted by analyzing the preprocessed image data, so that the cavity image is extracted, and the cavity area coordinate data are obtained. And calculating the linear distance between the center coordinates of the first abnormal region and each coordinate point in the cavity region coordinate data, and selecting a minimum value of the linear distance as the minimum distance between the marked image data and the cavity region.
Due to the influence of the internal environment of the wood and other factors, a deviation may occur between the detected target position and the actual target position, so that a certain distance difference exists between the marked image data and the cavity area. Comparing the minimum distance between the marked image data and the cavity area with a second preset threshold value, if the minimum distance is smaller than the second preset threshold value, indicating that the drilling and eating insect exists in the current marked image data, calculating through a preset method to obtain a second probability value of the drilling and eating insect existing in the marked image, wherein the second probability value indicates whether the drilling and eating insect exists in the wood or not according to the position relation between the marked image data and the cavity area. The second preset threshold is set through the system, and is dynamically adjusted according to analysis results of the historical detection image data in the using process of the system.
The preset method is expressed as a= (B-C)/B by a formula, where a represents a second probability value, B represents a second preset threshold value, and C represents a minimum distance between a center coordinate of the first abnormal region and the cavity region coordinate data in the marker image data.
According to an embodiment of the present invention, the calculating according to the first probability value and the second probability value to obtain a third probability value, and comparing the third probability value with a first preset threshold interval to obtain the detected data of the boring pest in the marked image data includes:
multiplying the first probability value and the second probability value by corresponding influence weights respectively to obtain a first weight score and a second weight score;
adding the first weight score and the second weight score to obtain a third probability value;
judging whether the third probability value is in a first preset threshold interval or not;
if the third probability value is smaller than the minimum value of the first preset threshold interval, the fact that no boring insect exists in the marked image data is indicated;
if the third probability value is larger than the maximum value of the first preset threshold interval, indicating that the marked image data contain boring insects;
if the third probability value is in the first preset threshold interval, detecting the wood to be detected again after a preset time interval to obtain second detection image data;
Comparing the second detection image data with the first detection image data, and judging whether the marked image data contains boring insects or not;
and integrating the comparison results of all the marked image data to obtain wood boring pest detection data.
The different parameters (image characteristics of the boring thermal imaging image, behavior habit of the insect, etc.) have different weights of influence on judging whether the boring insect exists in the marked image data, and the third probability value is obtained by calculating according to the weights of influence of the different parameters. The method is mainly used for judging whether the boring insect exists in the wood based on the image characteristics of the thermal imaging image of the boring insect, calculating the position relationship and the minimum distance between the marking image data and the boring hole image data in the wood, and playing an auxiliary role in judging whether the boring insect exists in the wood. Thus, the impact weight of the first probability value is greater than the impact weight of the second probability value. The system dynamically adjusts the influence weights of the first probability value and the second probability value based on the size of the first probability value, wherein the larger the first probability value is, the larger the influence weights of the first probability value and the second probability value are, but the maximum value of the sum of the influence weights of the first probability value and the second probability value is 1. And analyzing by combining a first preset threshold interval, judging whether the marked image data contains the boring insects, wherein the initial value of the first preset threshold interval is 30% -70%, and dynamically adjusting the first preset threshold according to the detection precision level selected by the user and the recognition precision of the boring insects in the historical detection data.
When the third probability value is smaller than the minimum value of the first preset threshold interval, the image characteristics of the marked image data, which basically do not contain the boring insect pests, are represented, and the marked image data are judged to contain no boring insect pests; when the third probability value is larger than the maximum value of the first preset threshold interval, the image characteristics of a large number of boring insects exist in the marked image data, and the boring insects exist in the marked image data; if the third probability value is between the first preset threshold values, the third probability value indicates that certain image features of the boring insect pests exist in the marked image, but the third probability value cannot be used as a basis for judging the boring insect pests. In this case, the wood to be detected may be detected again according to a preset time interval (e.g., one hour), and whether or not the boring insect exists in the marker image data may be judged by comparing the characteristic differences of the second detection image data and the first detection image data.
According to an embodiment of the present invention, the comparing the second detection image data with the first detection image data to determine whether the boring pest exists in the marking image data includes:
analyzing according to the second detection image data to obtain the center coordinates of a second abnormal region;
Comparing the central coordinates of the second abnormal region with the central coordinates of the first abnormal region, and judging whether the central coordinates of the second abnormal region and the first abnormal region are consistent;
if not, judging the abnormal area in the marked image data as a boring pest;
if yes, judging the abnormal area in the marked image data as wood defect.
Under the condition that whether the wood is damaged by the drilling or the boring is not determined according to the probability value of the damage, the wood to be detected can be detected again after a system preset time interval, the second detection image data is analyzed according to a preset drilling or boring pest detection model, the center coordinates of a second abnormal area in the second detection image are determined, and whether the drilling or boring is detected in the marked image data is judged by comparing the center coordinates of the abnormal areas in the two detection image data. In addition, in order to improve the recognition accuracy, multiple detections can be performed on the basis of two detections. If the central coordinates of the abnormal areas are consistent, the abnormal areas are knots or other wood defects; if the position coordinates of the marked object are changed, the living body of the abnormal region is represented, and meanwhile, certain image characteristics of the boring insect pests exist in the abnormal region when the first detection image data are analyzed, so that the abnormal region in the marked image data can be judged to be the boring insect pests.
According to the embodiment of the invention, quarantine treatment is performed on wood with boring pests according to the wood boring pest detection data, the quarantine treated wood is detected to obtain third detection image data, and the third detection image data is compared with the first detection image data to obtain wood quarantine treatment effect evaluation data, which comprises the following steps:
analyzing according to the wood boring pest detection data, and drawing a circle by taking the central coordinate of an abnormal region where the wood boring pest is located as the center of a circle and a preset radius to obtain a key monitoring region;
and comparing the third detection image data with the first detection image data based on the key monitoring area to obtain wood quarantine treatment effect evaluation data.
The quarantine treatment includes fumigation treatment, heat treatment and the like, taking fumigation treatment as an example, after fumigation treatment is carried out on wood by using a fumigant, image acquisition is carried out on the wood after the quarantine treatment by using a thermal imaging detection device again at intervals after air dispersion is finished, body fluid evaporation is carried out after death of insects, an insect body heat imaging image cannot be detected by using the detection device, and the quarantine treatment effect is determined by comparing detected image data before and after fumigation. In order to avoid the influence of the movement of the boring insect in the quarantine process on the detection result, one or more key monitoring areas are defined according to the number of the insect and the central coordinates of the area where each insect is located, wherein the preset radius is set by combining the crawling speed of the system simulation boring insect in the wood with the quarantine treatment time.
When detecting the fumigated detection image data, the detection focus is placed on the focus monitoring area, and whether the expected effect of the wood quarantine treatment is achieved is determined by judging whether the pest mark image in the focus monitoring area has change or not.
In addition, a certain time is needed for evaporation of insect body fluid, and when the fumigated wood is detected, as the insect body fluid is evaporated completely and a certain amount of heat exists in residual body fluid, a new thermal imaging image appears in fumigated detection image data. Comparing the newly-appearing thermal imaging image with the pest marking image in the detected image data before fumigation, analyzing by combining sample image data in a preset model database, judging according to the image attribute (shape, size, brightness and the like) of the newly-appearing thermal imaging image, whether the image features of the thermal imaging image of the boring pest exist or not and determining that the newly-appearing thermal imaging image is the thermal imaging image after the boring pest moves or the thermal imaging image generated by residual body fluid after the boring pest dies. Labeling the suspicious data, and detecting again after a period of time or verifying by a manual cutting sampling detection mode to eliminate the suspicious data so as to obtain the wood quarantine treatment effect evaluation data. And performing the wood quarantine treatment again with poor quarantine treatment effect according to the obtained wood quarantine treatment effect evaluation data until the wood quarantine treatment effect meets the expected requirement.
According to an embodiment of the present invention, further comprising:
establishing a three-dimensional model of the boring insect according to the sample image data of the boring insect;
simulating a thermal imaging image of pests in the wood through the three-dimensional model of the boring pests to obtain simulated sample image data;
and adding the simulated sample image data to a preset model database.
It should be noted that, the thermal imaging sample images of the boring insects in the timber in the preset model database are fewer, the sample image data can be basically collected only by collecting the historical detection image data and labeling the historical detection image data by manual labeling, on the basis, the image data of the boring insects can be obtained through the internet, the three-dimensional model of the boring insects is built through the image data of the boring insects, the thermal imaging effect of the insects in the timber is simulated based on the parameters of the moisture content, the temperature, the type, the depth of the insects and the like of the timber, the simulated sample image is obtained, and the simulated sample image is added into the preset model database, so that the accuracy and the generalization capability of the model are improved.
A third aspect of the present invention provides a computer-readable storage medium having embodied therein a thermal imaging-based wood boring pest detection method program which, when executed by a processor, implements the steps of a thermal imaging-based wood boring pest detection method as described in any one of the above.
The invention discloses a method, a system and a medium for detecting wood boring insect pests based on thermal imaging. Under the condition that pests exist in the wood, quarantine treatment is carried out on the wood, and whether a thermal imaging image marked as the boring pests is changed is judged by comparing detected image data before and after the quarantine treatment, so that the effectiveness of the quarantine treatment of the wood is judged, and the quarantine efficiency of the boring pests in the wood is effectively improved.
In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above described device embodiments are only illustrative, e.g. the division of the units is only one logical function division, and there may be other divisions in practice, such as: multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. In addition, the various components shown or discussed may be coupled or directly coupled or communicatively coupled to each other via some interface, whether indirectly coupled or communicatively coupled to devices or units, whether electrically, mechanically, or otherwise.
The units described above as separate components may or may not be physically separate, and components shown as units may or may not be physical units; can be located in one place or distributed to a plurality of network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present invention may be integrated in one processing unit, or each unit may be separately used as one unit, or two or more units may be integrated in one unit; the integrated units may be implemented in hardware or in hardware plus software functional units.
Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the above method embodiments may be implemented by hardware related to program instructions, and the foregoing program may be stored in a computer readable storage medium, where the program, when executed, performs steps including the above method embodiments; and the aforementioned storage medium includes: a mobile storage device, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk or an optical disk, or the like, which can store program codes.
Alternatively, the above-described integrated units of the present invention may be stored in a computer-readable storage medium if implemented in the form of software functional modules and sold or used as separate products. Based on such understanding, the technical solutions of the embodiments of the present invention may be embodied in essence or a part contributing to the prior art in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, ROM, RAM, magnetic or optical disk, or other medium capable of storing program code.
Claims (9)
1. The method for detecting wood boring insect based on thermal imaging is characterized by comprising the following steps:
detecting wood to be detected to obtain first detection image data;
inputting the first detection image data into a preset boring pest detection model for analysis, marking an abnormal region to obtain marked image data, and calculating to obtain a first probability value according to the image characteristics of the marked image data;
Analyzing according to the position relation between the cavity area and the marked image data to obtain a second probability value;
calculating according to the first probability value and the second probability value to obtain a third probability value, and comparing the third probability value with a first preset threshold interval to obtain the boring pest detection data in the marked image data;
integrating the detected data of the wood boring insect pests in all the marked image data to obtain the detected data of the wood boring insect pests;
quarantine treatment is carried out on the wood with the boring insects according to the wood boring insects detection data, the wood subjected to quarantine treatment is detected to obtain third detection image data, and the third detection image data and the first detection image data are compared to obtain wood quarantine treatment effect evaluation data;
analyzing according to the position relation between the cavity area and the marked image data to obtain a second probability value, wherein the second probability value comprises:
analyzing according to the first detection image data, and marking the wormhole area in the wood to obtain wormhole area coordinate data;
calculating the minimum distance between the center coordinates of the first abnormal region in the marked image data and the cavity region coordinate data;
Comparing the minimum distance with a second preset threshold value, and if the minimum distance is larger than the second preset threshold value, indicating that the marked object is a wood defect, wherein the second probability value is 0%;
otherwise, calculating the minimum distance by a preset method to obtain a second probability value;
if the minimum distance is 0, the central coordinate of the abnormal region is in the cavity region, and the second probability value is 100%;
when a wormhole exists in the wood, the heat conduction performance of the wood can change, and the edge characteristics of the wormhole are extracted by analyzing the first detection image data, so that the wormhole area is extracted, and the wormhole area coordinate data are obtained.
2. The thermal imaging-based wood boring pest detection method according to claim 1, further comprising:
performing image preprocessing on the first detection image data;
the image preprocessing includes non-uniform correction, blind pixel compensation and image enhancement.
3. The method for detecting wood boring insect pests based on thermal imaging according to claim 1, wherein inputting the first detection image data into a preset boring insect pest detection model for analysis, marking an abnormal region to obtain marked image data, and calculating a first probability value according to image features of the marked image data comprises:
Analyzing the first detection image data through an attention mechanism module to obtain a first abnormal region, and performing marking frame selection on the first abnormal region to obtain marked image data;
analyzing according to the marked image data, and determining the center coordinates of the first abnormal region;
extracting features of the marked image data through a depth residual error learning network module to obtain image features of the marked image data;
and comparing and calculating the image characteristics of the marked image data with the image characteristics of the sample image data in the preset model database to obtain a first probability value.
4. The method for detecting wood boring insect pests based on thermal imaging according to claim 1, wherein the calculating according to the first probability value and the second probability value to obtain a third probability value, and comparing the third probability value with a first preset threshold interval to obtain boring insect pest detection data in the marked image data comprises:
multiplying the first probability value and the second probability value by corresponding influence weights respectively to obtain a first weight score and a second weight score;
adding the first weight score and the second weight score to obtain a third probability value;
Judging whether the third probability value is in a first preset threshold interval or not;
if the third probability value is smaller than the minimum value of the first preset threshold interval, the fact that no boring insect exists in the marked image data is indicated;
if the third probability value is larger than the maximum value of the first preset threshold interval, indicating that the marked image data contain boring insects;
if the third probability value is in the first preset threshold interval, detecting the wood to be detected again after a preset time interval to obtain second detection image data;
comparing the second detection image data with the first detection image data, and judging whether the marked image data contains boring insects or not;
and integrating the comparison results of all the marked image data to obtain wood boring pest detection data.
5. The thermal imaging-based wood boring pest detection method according to claim 4, wherein comparing the second detection image data with the first detection image data to determine whether boring pest is present in the marker image data, comprises:
analyzing according to the second detection image data to obtain the center coordinates of a second abnormal region;
Comparing the central coordinates of the second abnormal region with the central coordinates of the first abnormal region, and judging whether the central coordinates of the second abnormal region and the first abnormal region are consistent;
if not, judging the abnormal area in the marked image data as a boring pest;
if yes, judging the abnormal area in the marked image data as wood defect.
6. The method for detecting wood boring insect pests based on thermal imaging according to claim 1, wherein the quarantining treatment is performed on wood having boring insect pests according to the wood boring insect pest detection data, the quarantined wood is detected to obtain third detection image data, and the third detection image data is compared with the first detection image data to obtain wood quarantining treatment effect evaluation data, and the method comprises the steps of:
analyzing according to the wood boring pest detection data, and drawing a circle by taking the central coordinate of an abnormal region where the wood boring pest is located as the center of a circle and a preset radius to obtain a key monitoring region;
and comparing the third detection image data with the first detection image data based on the key monitoring area to obtain wood quarantine treatment effect evaluation data.
7. The thermal imaging-based wood boring pest detection method according to claim 1, further comprising:
Establishing a three-dimensional model of the boring insect according to the sample image data of the boring insect;
simulating the thermal imaging image of the boring insect in the wood by the three-dimensional model of the boring insect to obtain simulated sample image data;
and adding the simulated sample image data to a preset model database.
8. A thermal imaging-based wood boring pest detection system, comprising:
the data acquisition module is used for detecting the wood to be detected to obtain first detection image data;
the data analysis module is used for inputting the first detection image data into a preset boring pest detection model for analysis, marking the abnormal region to obtain marked image data, and calculating to obtain a first probability value according to the image characteristics of the marked image data; analyzing according to the position relation between the cavity area and the marked image data to obtain a second probability value; calculating according to the first probability value and the second probability value to obtain a third probability value, and comparing the third probability value with a first preset threshold interval to obtain the boring pest detection data in the marked image data; integrating the detected data of the wood boring insect pests in all the marked image data to obtain the detected data of the wood boring insect pests;
The quarantine treatment effect verification module is used for quarantining the wood with the boring insect according to the wood boring insect detection data, detecting the quarantined wood to obtain third detection image data, and comparing the third detection image data with the first detection image data to obtain wood quarantine treatment effect evaluation data;
analyzing according to the position relation between the cavity area and the marked image data to obtain a second probability value, wherein the second probability value comprises:
analyzing according to the first detection image data, and marking the wormhole area in the wood to obtain wormhole area coordinate data;
calculating the minimum distance between the center coordinates of the first abnormal region in the marked image data and the cavity region coordinate data;
comparing the minimum distance with a second preset threshold value, and if the minimum distance is larger than the second preset threshold value, indicating that the marked object is a wood defect, wherein the second probability value is 0%;
otherwise, calculating the minimum distance by a preset method to obtain a second probability value;
if the minimum distance is 0, the central coordinate of the abnormal region is in the cavity region, and the second probability value is 100%;
When a wormhole exists in the wood, the heat conduction performance of the wood can change, and the edge characteristics of the wormhole are extracted by analyzing the first detection image data, so that the wormhole area is extracted, and the wormhole area coordinate data are obtained.
9. A computer-readable storage medium, wherein the computer-readable storage medium includes therein a thermal imaging-based wood boring pest detection method program, which when executed by a processor, implements the steps of a thermal imaging-based wood boring pest detection method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311477169.1A CN117197148B (en) | 2023-11-08 | 2023-11-08 | Thermal imaging-based wood boring pest detection method, system and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311477169.1A CN117197148B (en) | 2023-11-08 | 2023-11-08 | Thermal imaging-based wood boring pest detection method, system and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117197148A CN117197148A (en) | 2023-12-08 |
| CN117197148B true CN117197148B (en) | 2024-02-02 |
Family
ID=89003843
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311477169.1A Active CN117197148B (en) | 2023-11-08 | 2023-11-08 | Thermal imaging-based wood boring pest detection method, system and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117197148B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113033520A (en) * | 2021-05-25 | 2021-06-25 | 华中农业大学 | Tree nematode disease wood identification method and system based on deep learning |
| CN115457373A (en) * | 2022-09-19 | 2022-12-09 | 中科合肥智慧农业协同创新研究院 | Anchor-frame-free one-stage insect pest image detection method and device and storage medium |
| CN115728326A (en) * | 2022-11-10 | 2023-03-03 | 中北大学 | Method for detecting underground boring insects of trees based on X-ray |
| CN115861729A (en) * | 2022-11-07 | 2023-03-28 | 东北林业大学 | Small sample forestry pest identification method based on deep learning algorithm |
| CN116237270A (en) * | 2023-01-31 | 2023-06-09 | 广东省农业科学院蚕业与农产品加工研究所 | Grading method of pedicellus et pericarpium citri reticulatae fruits |
| CN116721060A (en) * | 2023-08-08 | 2023-09-08 | 中国热带农业科学院椰子研究所 | Method for detecting red palm weevil beetles based on attention mechanism |
-
2023
- 2023-11-08 CN CN202311477169.1A patent/CN117197148B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113033520A (en) * | 2021-05-25 | 2021-06-25 | 华中农业大学 | Tree nematode disease wood identification method and system based on deep learning |
| CN115457373A (en) * | 2022-09-19 | 2022-12-09 | 中科合肥智慧农业协同创新研究院 | Anchor-frame-free one-stage insect pest image detection method and device and storage medium |
| CN115861729A (en) * | 2022-11-07 | 2023-03-28 | 东北林业大学 | Small sample forestry pest identification method based on deep learning algorithm |
| CN115728326A (en) * | 2022-11-10 | 2023-03-03 | 中北大学 | Method for detecting underground boring insects of trees based on X-ray |
| CN116237270A (en) * | 2023-01-31 | 2023-06-09 | 广东省农业科学院蚕业与农产品加工研究所 | Grading method of pedicellus et pericarpium citri reticulatae fruits |
| CN116721060A (en) * | 2023-08-08 | 2023-09-08 | 中国热带农业科学院椰子研究所 | Method for detecting red palm weevil beetles based on attention mechanism |
Non-Patent Citations (2)
| Title |
|---|
| 基于轻量级神经网络的2种害虫钻蛀振动识别方法;孙钰;脱小倩;蒋琦;张海燕;陈志泊;宗世祥;骆有庆;;林业科学(03);103-111页 * |
| 木材检疫处理技术研究进展;李玉广 等;《植物检疫》;第7-13页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117197148A (en) | 2023-12-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Thapa et al. | The Plant Pathology Challenge 2020 data set to classify foliar disease of apples | |
| Abrego et al. | Measuring and predicting the influence of traits on the assembly processes of wood‐inhabiting fungi | |
| Gutiérrez et al. | Deep learning for the differentiation of downy mildew and spider mite in grapevine under field conditions | |
| Yang et al. | Tea diseases detection based on fast infrared thermal image processing technology | |
| Kumar et al. | GCrop: Internet-of-Leaf-Things (IoLT) for monitoring of the growth of crops in smart agriculture | |
| CN109490306B (en) | Pork freshness detection method based on color and smell data fusion | |
| Cao et al. | Identification of species and geographical strains of Sitophilus oryzae and Sitophilus zeamais using the visible/near‐infrared hyperspectral imaging technique | |
| WO2023071091A1 (en) | Anthracnose pathogen infection stage detection method with pre-analysis capability | |
| CN113657294A (en) | Crop disease and insect pest detection method and system based on computer vision | |
| CN110188657A (en) | Corn arid recognition methods based on crimping blade detection | |
| Jokela et al. | Cross-taxon congruence and relationships to stand characteristics of vascular plants, bryophytes, polyporous fungi and beetles in mature managed boreal forests | |
| Suo et al. | Casm-amfmnet: A network based on coordinate attention shuffle mechanism and asymmetric multi-scale fusion module for classification of grape leaf diseases | |
| CN117197148B (en) | Thermal imaging-based wood boring pest detection method, system and medium | |
| CN117172430B (en) | Deep learning-based water body environment assessment and prediction method and system | |
| Xu et al. | An automatic wheat ear counting model based on the minimum area intersection ratio algorithm and transfer learning | |
| CN117371604A (en) | Agricultural production prediction method and system based on intelligent perception | |
| Wardius et al. | Terrestrial laser scanning vs. manual methods for assessing complex forest stand structure: a comparative analysis on plenter forests | |
| CN109993071B (en) | Method and system for automatically identifying and investigating color-changing forest based on remote sensing image | |
| CN113792715B (en) | Granary pest monitoring and early warning method, device, equipment and storage medium | |
| Chen et al. | Preliminary research on total nitrogen content prediction of sandalwood using the error-in-variable models based on digital image processing | |
| Sams et al. | Characterising spatio‐temporal variation in fruit composition for improved winegrowing management in California Cabernet Sauvignon | |
| Moloo et al. | An app for fungal disease detection on plants | |
| Rößner et al. | Quantifying bark-stripping damages to address the relationship between external and internal wood defects | |
| Schmidtke | Developing a phone-based imaging tool to inform on fruit volume and potential optimal harvest time | |
| CN115424143B (en) | Water source pollution tracing method and device, storage medium and computer equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |