CN116453004A - Agricultural target detection data set construction method, system and electronic equipment - Google Patents
Agricultural target detection data set construction method, system and electronic equipment Download PDFInfo
- Publication number
- CN116453004A CN116453004A CN202310366283.0A CN202310366283A CN116453004A CN 116453004 A CN116453004 A CN 116453004A CN 202310366283 A CN202310366283 A CN 202310366283A CN 116453004 A CN116453004 A CN 116453004A
- Authority
- CN
- China
- Prior art keywords
- agricultural
- data set
- model
- scene
- target detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 96
- 238000010276 construction Methods 0.000 title claims abstract description 64
- 238000012795 verification Methods 0.000 claims abstract description 84
- 238000000034 method Methods 0.000 claims abstract description 31
- 238000005457 optimization Methods 0.000 claims abstract description 17
- 238000012549 training Methods 0.000 claims description 67
- 238000003860 storage Methods 0.000 claims description 16
- 238000005286 illumination Methods 0.000 claims description 13
- 238000013136 deep learning model Methods 0.000 claims description 11
- 238000010200 validation analysis Methods 0.000 claims description 6
- 238000007689 inspection Methods 0.000 claims 5
- 238000004458 analytical method Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 241001465754 Metazoa Species 0.000 description 4
- 239000003086 colorant Substances 0.000 description 4
- 238000011160 research Methods 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000004088 simulation Methods 0.000 description 3
- 238000012876 topography Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 239000011435 rock Substances 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000002689 soil Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of agricultural data set construction, in particular to an agricultural target detection data set construction method, an agricultural target detection data set construction system and electronic equipment. The method comprises the following steps: obtaining an initial agricultural target detection data set based on the agricultural virtual scene; obtaining a trained first model and a trained second model, verifying the trained first model by using a verification data set to obtain a first verification result, and verifying the trained second model by using the verification data set to obtain a second verification result; when the second verification result is better than the first verification result, the agricultural virtual scene is optimized, and when the preset condition is met, the current agricultural virtual scene is subjected to image acquisition to obtain a plurality of target virtual scene images, and a final agricultural target detection data set is formed. Through continuous feedback optimization of the agricultural virtual scene, the problem of insufficient data in the agricultural scene data set is solved, and meanwhile, the acquisition quality is improved.
Description
Technical Field
The invention relates to the technical field of agricultural data set construction, in particular to an agricultural target detection data set construction method, an agricultural target detection data set construction system and electronic equipment.
Background
The artificial intelligence technology is widely applied in modern agriculture, and unmanned machinery based on visual obstacle avoidance can efficiently operate under the condition of ensuring personnel safety. Since visual obstacle avoidance is typically implemented based on a deep learning approach, the deep learning model is highly dependent on training of the data set. When the training data set of deep learning is smaller, the insufficient learning amount of the model can lead to lower recognition rate of the model, for example, the number of training sets of people in the training set is far smaller than that of training sets of vehicles can lead to that the recognition of the trained model on detection is obviously lower than that of the recognition of the vehicles; when the training data volume is enough but the training data distribution is unbalanced, the situation that false detection occurs to the trained model is caused, for example, the number of four-wheel automobiles in the data set is far greater than the number of tricycles, and the trained model can be used for recognizing the tricycles as the four-wheel automobiles in recognition detection. Therefore, the number and quality of data sets used in training a deep learning based model is of paramount importance.
The traditional image video data set acquisition mode mainly relies on manual outgoing acquisition, can not guarantee the integrity and uniformity of the acquired image video data set, is more difficult to acquire when acquiring certain specific scenes and climates, and is long in acquisition period and high in manpower and material resource cost. The development of the virtual reality technology lays a foundation for virtual simulation, and the software-hardware-based ray tracing technology can simulate illumination and shadow effects under real conditions to the greatest extent, and especially, the real virtual micro polygon geometry and the Lumen dynamic global illumination brought by the Unreal Engine 5 can present high-quality and vivid scene representation, so that the scene construction gets rid of the limitation of the real conditions. Along with the development of virtual technology in the fields of scene restoration, digital twinning, simulation and the like, the application of the virtual technology in the agricultural field becomes reality.
Corresponding virtual data sets and methods have been constructed internationally for different tasks. AirSim serving the autopilot domain; the Benchmark dataset is provided with visual task data labels such as optical flow estimation, semantic segmentation, target detection, target tracking and the like; virtual KITTI data set constructed by Virtual 3D scene; the Injeida is a NVIDIA DRIVE Sim virtual platform built in the field of automatic driving. However, these methods are mostly used in the field of roaded autopilot or only in academic research, the construction method is not suitable for autopilot in the field of agricultural scenes, and the scene construction lacks analysis and scene optimization of necessary layout elements. The current virtual scene generation method based on the virtual technology mainly relates to three fields, in particular:
1) The first is a virtual data set construction method applied to the field of road automatic driving, because automatic driving focuses more on the complexity of road traffic, the layout of objects is preferentially considered when a virtual road scene is constructed, and the objects such as vehicles, pedestrians, street lamps and the like in the scene constructed by the method are rich in categories and have certain illumination, shadow and climate simulation, but the situation outside a lane is ignored, and the method cannot be directly applied to the agricultural field.
2) The second is a virtual scene construction method applied to academic research, the method is mostly used for pattern recognition algorithm verification, as the algorithm is focused more on the object recognition itself in academic, the virtual scene construction is simpler, the background of the object to be recognized is mostly solid color or randomly generated, illumination, shadow or climate change in the real scene cannot be simulated, and the construction method is also difficult to apply to the agricultural field.
3) The third is a virtual scene construction method applied to the fields of design and construction, which is mainly used for constructing the whole or detail of construction and indoor, and although the method also relates to a certain outdoor scenery, the simulation effect cannot be achieved, and the method cannot be applied to complex agricultural environments.
In summary, the virtual scene methods applied to the automatic driving field, the academic research, the design and the building field are all characteristic methods aiming at the respective fields. Compared with a complex agricultural scene, the virtual scene construction method applied to the automatic driving field lacks a model and a scenery mode related to agricultural scene construction; the virtual scene construction method applied to academic research lacks not only the model and the scene setting mode related to agricultural scene construction, but also the illumination, shadow and climate change of the simulated real scene. The virtual scene construction method applied to the design and construction fields has better detail texture restoration, but lacks of an agricultural scene construction related model and scene mode, and does not have the variation of illumination, shadow and climate for simulating a real scene. The lack of feedback correction process in the data set constructed by the methods leads to overlarge difference between the constructed virtual scene and the real scene, and the acquired virtual data set cannot be used as an extension of the real data set.
Disclosure of Invention
The invention aims to solve the technical problem of providing a method, a system and electronic equipment for constructing an agricultural target detection data set aiming at the defects of the prior art.
The technical scheme of the agricultural target detection data set construction method is as follows:
s1, carrying out image acquisition on an agricultural virtual scene to obtain a plurality of virtual scene images, and forming an initial agricultural target detection data set;
s2, mixing an initial agricultural target detection data set and a target detection data set constructed based on a real agricultural scene in proportion to obtain a training data set, training a first model based on the training data set to obtain a trained first model, training a second model based on the target detection data set constructed based on the real agricultural scene to obtain a trained second model, wherein the first model and the second model are deep learning models with the same network structure;
s3, verifying the trained first model by using a verification data set to obtain a first verification result, and verifying the trained second model by using the verification data set to obtain a second verification result;
and S4, when the second verification result is better than the first verification result, optimizing the agricultural virtual scene, returning to the step S1, and executing the step S5 until a preset condition is met, wherein the preset condition comprises: the second verification result is not better than the first verification result;
s5, image acquisition is carried out on the current agricultural virtual scene, a plurality of target virtual scene images are obtained, and a final agricultural target detection data set is formed.
The technical scheme of the agricultural target detection data set construction system is as follows:
the system comprises an acquisition construction module, a training module, a verification module and an optimization module;
the acquisition and construction module is used for: image acquisition is carried out on the agricultural virtual scene to obtain a plurality of virtual scene images, and an initial agricultural target detection data set is formed;
the training module is used for: mixing an initial agricultural target detection data set and a target detection data set constructed based on a real agricultural scene in proportion to obtain a training data set, training a first model based on the training data set to obtain a trained first model, training a second model based on the target detection data set constructed based on the real agricultural scene to obtain a trained second model, wherein the first model and the second model are deep learning models with the same network structure;
the verification module is used for: verifying the trained first model by using a verification data set to obtain a first verification result, and verifying the trained second model by using the verification data set to obtain a second verification result;
the optimization module is used for: when the second verification result is better than the first verification result, optimizing the agricultural virtual scene, calling the acquisition construction module, the training module and the verification module until a preset condition is met, calling the acquisition construction module, and enabling the acquisition construction module to acquire images of the current agricultural virtual scene to obtain a plurality of target virtual scene images to form a final agricultural target detection data set, wherein the preset condition comprises the following steps: the second validation result is not better than the first validation result.
A storage medium of the present invention has stored therein instructions which, when read by a computer, cause the computer to execute an agricultural object detection dataset construction method as set forth in any one of the above.
An electronic device of the present invention includes a processor and the storage medium described above, where the processor executes instructions in the storage medium.
The beneficial effects of the invention are as follows:
through continuous feedback optimization of the agricultural virtual scene, the problem of insufficient data in the agricultural scene data set is solved, and the acquisition quality of a final agricultural target detection data set is further improved.
Drawings
FIG. 1 is a schematic flow chart of a method for constructing an agricultural target detection dataset according to an embodiment of the present invention;
FIG. 2 is a second flow chart of a method for constructing an agricultural target detection dataset according to an embodiment of the present invention;
FIG. 3 is a schematic illustration of agricultural scene analysis;
FIG. 4 is a schematic illustration of agricultural virtual scene construction;
FIG. 5 is an agricultural virtual scene effect diagram;
FIG. 6 is a schematic diagram of a scene feedback optimization process;
fig. 7 is a schematic structural diagram of an agricultural target detection dataset construction system according to an embodiment of the present invention.
Detailed Description
As shown in fig. 1, the method for constructing the agricultural target detection data set according to the embodiment of the invention includes the following steps:
s1, carrying out image acquisition on an agricultural virtual scene to obtain a plurality of virtual scene images, and forming an initial agricultural target detection data set;
s2, mixing an initial agricultural target detection data set and a target detection data set constructed based on a real agricultural scene in proportion to obtain a training data set, training a first model based on the training data set to obtain a trained first model, training a second model based on the target detection data set constructed based on the real agricultural scene to obtain a trained second model, wherein the first model and the second model are deep learning models with the same network structure;
s3, verifying the trained first model by using a verification data set to obtain a first verification result, and verifying the trained second model by using the verification data set to obtain a second verification result;
and S4, when the second verification result is better than the first verification result, optimizing the agricultural virtual scene, and returning to the step S1 until the preset condition is met, and executing the step S5, wherein the preset condition comprises: the second verification result is not better than the first verification result; s5, image acquisition is carried out on the current agricultural virtual scene, a plurality of target virtual scene images are obtained, and a final agricultural target detection data set is formed.
The first verification result is the recognition precision of the trained first model, the second verification result is the recognition precision of the trained second model, when the recognition precision of the trained second model is larger than the recognition precision of the trained first model, the second verification result is judged to be better than the first verification result, and when the recognition precision of the trained second model is not larger than the recognition precision of the trained first model, the second verification result is judged to be not better than the first verification result. Optionally, in the above technical solution, the method further includes:
when S1 is executed back, increasing the duty ratio of the initial agricultural target detection data set in the training data set according to a preset step length;
the preset conditions further include: the initial agricultural target detection dataset has a duty cycle in the training dataset exceeding a preset duty cycle threshold.
The preset step length may be set according to practical situations, for example, the preset step length is 1%, 2%, and the like, and the preset step length is 1% for illustration:
when the S2 is executed for the first time, mixing an initial agricultural target detection data set and a target detection data set constructed based on a real agricultural scene according to the proportion of 30:70 to obtain a training data set; when the S2 is executed for the second time, mixing the initial agricultural target detection data set and the target detection data set constructed based on the real agricultural scene according to the ratio of 31:69; when the S2 is executed for the third time, mixing the initial agricultural target detection data set and the target detection data set constructed based on the real agricultural scene according to the proportion of 31:69; and so on.
In this embodiment, the following preset conditions refer to: the second verification result is not superior to the first verification result, and the duty ratio of the initial agricultural target detection data set in the training data set exceeds a preset duty ratio threshold, wherein the preset duty ratio threshold can be set according to actual conditions, such as 50% or 60%.
Optionally, in the above technical solution, the building process of the agricultural virtual scene includes:
and constructing an agricultural virtual scene by utilizing an agricultural scene model library constructed based on basic agricultural scene elements and utilizing a fantasy engine platform.
Optionally, in the above technical solution, S1 includes:
the illumination intensity, the direction, the color temperature and the weather condition in the agricultural virtual scene are adjusted, and the image acquisition is carried out through a camera component placed in the agricultural virtual scene.
Optionally, in the above technical solution, the deep learning model is a YOLOv5 model.
The method for constructing the agricultural target detection data set mainly comprises agricultural scene analysis, agricultural virtual scene construction and scene feedback optimization. Specifically:
as shown in fig. 2, the agricultural scene analysis provides the necessary agricultural scene model library for the construction of the agricultural virtual scene; the virtual engine platform is used as a building platform of the agricultural virtual scene, the real agricultural scene is restored to the maximum extent according to the agricultural scene model library, the built agricultural virtual scene simulates image acquisition under the real condition through a camera component of the engine, and an image video acquired through a camera is used for obtaining a virtual scene image to form an initial agricultural target detection data set.
The method comprises the steps that data screening and target identification detection frame marking are carried out on an initial agricultural target detection data set according to a real data set, namely a target detection data set constructed based on a real agricultural scene, the initial agricultural target detection data set and the real data set are combined into an agricultural target detection data set, namely a training data set in proportion, then a first model is trained based on the training data set to obtain a trained first model, a second model is trained based on the target detection data set constructed based on the real agricultural scene to obtain a trained second model, the first model and the second model are deep learning models with the same network structure, S3 is utilized for reasoning and verification, and an agricultural virtual scene is optimized and adjusted in a manual mode according to a verification result, namely a first verification result and a second verification result, so that scene feedback optimization is completed;
when S1 is executed, the duty ratio of the initial agricultural target detection data set in the training data set is increased according to a preset step length; when the second verification result is not superior to the first verification result and the duty ratio of the initial agricultural target detection data set in the training data set exceeds a preset duty ratio threshold, determining an optimal agricultural virtual scene, namely a current agricultural virtual scene in S5, and performing image acquisition on the current agricultural virtual scene to obtain a plurality of target virtual scene images to form a final agricultural target detection data set, wherein the method specifically comprises the following steps:
s100, obtaining basic agricultural scene elements through agricultural scene analysis, and constructing an agricultural scene model library:
as shown in fig. 3, elements such as figures, vehicles, animals, plants, buildings, terrains, weather, other facilities and the like contained in the real agricultural scene are extracted through the real agricultural scene illustration or video, wherein the figures comprise elements such as gender, clothes color, morphology and the like; the vehicle comprises motor vehicles, non-motor vehicles, vehicle types, colors, textures and other elements; animals include elements such as type, color, morphology, etc.; plants include crops including type, texture, color, growth state, morphology, etc. elements and other plants including type, color, texture, etc.; the building comprises elements such as height, color, texture and the like; the topography comprises soil color, texture, pothole degree and the like, and rock color, texture and the like; weather includes elements such as illumination intensity, direction, rainy day, snowy day, etc.; other facilities include facilities such as roads, streetlamps, fire hydrants, bridges, and the like, and elements such as colors, shapes, and the like.
All the elements form basic agricultural scene elements together, meanwhile, the relative position relation among objects in a real agricultural scene is determined, an agricultural scene model library is formed by utilizing the basic agricultural scene elements, models of characters, vehicles, animals, plants, buildings and other facilities required for building an agricultural virtual scene are contained in the agricultural scene model library, tools required for manufacturing the models comprise common 3D modeling software such as Autodesk Maya, 3DStudio Max and Blender, model manufacturing is carried out according to related information such as colors and textures acquired before, and the output model format is a mainstream 3D model format such as OBJ and FBX. In order to accelerate the model generation efficiency, the agricultural scene model library also comprises an existing model provided by a third party, and meanwhile, the manufactured model is saved to realize model multiplexing, so that the manufacturing efficiency of the agricultural virtual scene is improved.
S101, building an agricultural virtual scene:
as shown in fig. 4, the virtual agricultural scene is built by using the illusion engine platform, and fig. 5 shows the actual virtual agricultural scene building effect. The illusion engine platform provides a blueprint visual script, can complete scene construction through the functional layout of the existing elements of the scene, and is convenient for rapid scene forming. The Nanite virtualization micro polygon geometry system and the virtual shadow map provided by the engine can ensure real-time frame rate and no obvious distortion under the condition of film-level scene representation, and the Lumen dynamic global illumination can establish a more real illumination reflection effect, so that the established virtual scene simulates the real condition to the maximum extent. Then:
firstly, editing topography is completed in a fictive engine platform according to information of topography elements in basic agricultural scene elements, then, related models such as characters, animals, vehicles, plants, buildings, other facilities and the like are obtained from an agricultural scene model library and are arranged in a scene to complete scene construction, and finally, illumination intensity, direction, color temperature and other weather conditions in a virtual scene are adjusted according to weather element information in the basic agricultural scene elements. The camera assembly of the engine is placed in the agricultural virtual scene to complete image acquisition of the virtual scene, the image acquisition process can be adjusted according to acquisition height, speed and whether jitter is generated, target detection labeling is carried out on the acquired virtual scene image, and the construction of the initial agricultural target detection data set based on the illusion engine platform is completed.
S102, scene feedback optimization:
mixing an initial agricultural target detection data set and a target detection data set constructed based on a real agricultural scene according to a ratio of 3:7 to form a training data set, for example, 10000 images in the training data set are shared, 3000 images are acquired from the initial agricultural target detection data set, 7000 images are acquired from the target detection data set constructed based on the real agricultural scene, the first model and the second model are YOLOv5 models with the same network structure, and continuing to explain:
training the YOLOv5 model based on a training data set to obtain a trained YOLOv5 model, namely a trained first model, and training the YOLOv5 model based on a target detection data set constructed by a real agricultural scene to obtain a trained YOLOv5 model, namely a trained second model;
verifying the trained first model by using a verification data set comprising 2000 real agricultural scenes to obtain a first verification result, and verifying the trained second model by using the verification data set to obtain a second verification result;
the method comprises the steps of determining that a first verification result is the recognition precision of a trained first model, determining that a second verification result is better than the first verification result when the recognition precision of a trained second model is greater than the recognition precision of the trained first model, indicating that similar data is absent in an agricultural virtual scene, feeding back to essential elements of the scene, adjusting information such as colors of clothes of people and the like according to real scene conditions to change the performance of the person models in a scene model library, feeding back to a fictive engine platform to increase the number of people at a distance, acquiring virtual scene images again for training and verification, and feeding back to acquire the virtual scene images to increase the duty ratio of an initial agricultural target detection data set in a training data set according to a preset step length. And S103, executing until a preset condition is met, wherein the preset condition comprises the following steps: the second verification result is not superior to the first verification result, and the duty ratio of the initial agricultural target detection data set in the training data set exceeds a preset duty ratio threshold;
s5, image acquisition is carried out on the current agricultural virtual scene, a plurality of target virtual scene images are obtained, and a final agricultural target detection data set is formed. After feedback optimization, the reasoning characteristics of the virtual agricultural scene and the real agricultural scene are relatively close in the target detection task layer. In the subsequent scene expansion, the early-stage optimization process can be used as a building follow principle, so that the efficiency and quality of the scene expansion are improved.
In the above embodiments, although steps S1, S2, etc. are numbered, only specific embodiments are given herein, and those skilled in the art may adjust the execution sequence of S1, S2, etc. according to the actual situation, which is also within the scope of the present invention, and it is understood that some embodiments may include some or all of the above embodiments.
As shown in fig. 7, an agricultural target detection dataset construction system 200 according to an embodiment of the present invention includes an acquisition construction module 210, a training module 220, a verification module 230, and an optimization module 240;
the acquisition construction module 210 is configured to: image acquisition is carried out on the agricultural virtual scene to obtain a plurality of virtual scene images, and an initial agricultural target detection data set is formed;
the training module 220 is configured to: mixing an initial agricultural target detection data set and a target detection data set constructed based on a real agricultural scene in proportion to obtain a training data set, training a first model based on the training data set to obtain a trained first model, training a second model based on the target detection data set constructed based on the real agricultural scene to obtain a trained second model, wherein the first model and the second model are deep learning models with the same network structure;
the verification module 230 is configured to: verifying the trained first model by using a verification data set to obtain a first verification result, and verifying the trained second model by using the verification data set to obtain a second verification result;
the optimization module 240 is configured to: when the second verification result is better than the first verification result, the agricultural virtual scene is optimized, the acquisition construction module 210, the training module 220 and the verification module 230 are called, until a preset condition is met, the acquisition construction module 210 is called, the acquisition construction module 210 performs image acquisition on the current agricultural virtual scene to obtain a plurality of target virtual scene images, a final agricultural target detection data set is formed, and the preset condition comprises: the second validation result is not better than the first validation result.
Optionally, in the above technical solution, when the optimization module 240 invokes the acquisition and construction module 210, the training module 220, and the verification module 230, the training module 220 increases the duty ratio of the initial agricultural target detection data set in the training data set according to a preset step length;
the preset conditions further include: the initial agricultural target detection dataset has a duty cycle in the training dataset exceeding a preset duty cycle threshold.
Optionally, in the above technical solution, the system further includes a scene building module, where the scene building module is configured to:
and constructing an agricultural virtual scene by utilizing an agricultural scene model library constructed based on basic agricultural scene elements and utilizing a fantasy engine platform.
Optionally, in the above technical solution, the acquisition and construction module 210 is specifically configured to:
the illumination intensity, the direction, the color temperature and the weather condition in the agricultural virtual scene are adjusted, and the image acquisition is carried out through a camera component placed in the agricultural virtual scene.
Optionally, in the above technical solution, the deep learning model is a YOLOv5 model.
The steps for implementing the corresponding functions of the parameters and the unit modules in the agricultural target detection dataset construction system 200 according to the present invention are referred to in the above embodiments of the agricultural target detection dataset construction method, and are not described herein.
A storage medium according to an embodiment of the present invention is characterized in that instructions are stored in the storage medium, and when the instructions are read by a computer, the computer is caused to execute a method for constructing an agricultural target detection data set according to any one of the above.
The electronic equipment comprises a processor and the storage medium, wherein the processor executes instructions in the storage medium, the electronic equipment can be selected from a computer, a mobile phone and the like, and correspondingly, the program is computer software or mobile phone APP and the like.
Those skilled in the art will appreciate that the present invention may be implemented as a system, method, or computer program product.
Accordingly, the present disclosure may be embodied in the following forms, namely: either entirely hardware, entirely software (including firmware, resident software, micro-code, etc.), or entirely software, or a combination of hardware and software, referred to herein generally as a "circuit," module "or" system. Furthermore, in some embodiments, the invention may also be embodied in the form of a computer program product in one or more computer-readable media, which contain computer-readable program code.
Any combination of one or more computer readable media may be employed. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of the computer-readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
While embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the invention, and that variations, modifications, alternatives and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the invention.
Claims (10)
1. A method of constructing an agricultural object detection dataset, comprising:
s1, carrying out image acquisition on an agricultural virtual scene to obtain a plurality of virtual scene images, and forming an initial agricultural target detection data set;
s2, mixing an initial agricultural target detection data set and a target detection data set constructed based on a real agricultural scene in proportion to obtain a training data set, training a first model based on the training data set to obtain a trained first model, training a second model based on the target detection data set constructed based on the real agricultural scene to obtain a trained second model, wherein the first model and the second model are deep learning models with the same network structure;
s3, verifying the trained first model by using a verification data set to obtain a first verification result, and verifying the trained second model by using the verification data set to obtain a second verification result;
and S4, when the second verification result is better than the first verification result, optimizing the agricultural virtual scene, returning to the step S1, and executing the step S5 until a preset condition is met, wherein the preset condition comprises: the second verification result is not better than the first verification result;
s5, image acquisition is carried out on the current agricultural virtual scene, a plurality of target virtual scene images are obtained, and a final agricultural target detection data set is formed.
2. The agricultural target inspection dataset construction method of claim 1, further comprising:
when S1 is executed back, increasing the duty ratio of the initial agricultural target detection data set in the training data set according to a preset step length;
the preset conditions further include: the duty cycle of the initial agricultural target detection dataset in the training dataset exceeds a preset duty cycle threshold.
3. The method for constructing an agricultural target detection data set according to claim 1, wherein the construction process of the agricultural virtual scene comprises:
and building the agricultural virtual scene by utilizing an agricultural scene model library built based on basic agricultural scene elements and utilizing a illusion engine platform.
4. The agricultural target inspection dataset construction method of claim 1, wherein S1 comprises:
the illumination intensity, the direction, the color temperature and the weather condition in the agricultural virtual scene are adjusted, and the image acquisition is carried out through a camera component placed in the agricultural virtual scene.
5. An agricultural object detection dataset construction method as claimed in any one of claims 1 to 4, wherein the deep learning model is a YOLOv5 model.
6. The agricultural target detection data set construction system is characterized by comprising an acquisition construction module, a training module, a verification module and an optimization module;
the acquisition and construction module is used for: image acquisition is carried out on the agricultural virtual scene to obtain a plurality of virtual scene images, and an initial agricultural target detection data set is formed;
the training module is used for: mixing an initial agricultural target detection data set and a target detection data set constructed based on a real agricultural scene in proportion to obtain a training data set, training a first model based on the training data set to obtain a trained first model, training a second model based on the target detection data set constructed based on the real agricultural scene to obtain a trained second model, wherein the first model and the second model are deep learning models with the same network structure;
the verification module is used for: verifying the trained first model by using a verification data set to obtain a first verification result, and verifying the trained second model by using the verification data set to obtain a second verification result;
the optimization module is used for: when the second verification result is better than the first verification result, optimizing the agricultural virtual scene, calling the acquisition construction module, the training module and the verification module until a preset condition is met, calling the acquisition construction module, and enabling the acquisition construction module to acquire images of the current agricultural virtual scene to obtain a plurality of target virtual scene images to form a final agricultural target detection data set, wherein the preset condition comprises the following steps: the second validation result is not better than the first validation result.
7. The agricultural target inspection dataset construction system of claim 6, wherein when the optimization module invokes the acquisition construction module, the training module, and the verification module, the training module increases a duty cycle of an initial agricultural target inspection dataset in the training dataset according to a preset step size;
the preset conditions further include: the duty cycle of the initial agricultural target detection dataset in the training dataset exceeds a preset duty cycle threshold.
8. The agricultural target inspection dataset construction system of claim 7, further comprising a scene building module for:
and building the agricultural virtual scene by utilizing an agricultural scene model library built based on basic agricultural scene elements and utilizing a illusion engine platform.
9. A storage medium having instructions stored therein, which when read by a computer, cause the computer to perform an agricultural object detection dataset construction method as claimed in any one of claims 1 to 5.
10. An electronic device comprising a processor and the storage medium of claim 9, the processor executing instructions in the storage medium.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310366283.0A CN116453004A (en) | 2023-04-03 | 2023-04-03 | Agricultural target detection data set construction method, system and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310366283.0A CN116453004A (en) | 2023-04-03 | 2023-04-03 | Agricultural target detection data set construction method, system and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116453004A true CN116453004A (en) | 2023-07-18 |
Family
ID=87126990
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310366283.0A Pending CN116453004A (en) | 2023-04-03 | 2023-04-03 | Agricultural target detection data set construction method, system and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116453004A (en) |
-
2023
- 2023-04-03 CN CN202310366283.0A patent/CN116453004A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10019652B2 (en) | Generating a virtual world to assess real-world video analysis performance | |
| US20190065933A1 (en) | Augmenting Real Sensor Recordings With Simulated Sensor Data | |
| US20190228571A1 (en) | Realistic 3d virtual world creation and simulation for training automated driving systems | |
| CN112199991B (en) | Simulation point cloud filtering method and system applied to vehicle-road cooperation road side perception | |
| US11113864B2 (en) | Generative image synthesis for training deep learning machines | |
| CN111123920A (en) | Method and device for generating automatic driving simulation test scene | |
| CN108984741B (en) | Map generation method and device, robot and computer-readable storage medium | |
| Li et al. | Photo-realistic simulation of road scene for data-driven methods in bad weather | |
| CN115357006A (en) | Intelligent networking automobile virtual and actual testing method, equipment and medium based on digital twins | |
| WO2021146905A1 (en) | Deep learning-based scene simulator construction method and apparatus, and computer device | |
| CN116342783B (en) | Live-action three-dimensional model data rendering optimization method and system | |
| Gao et al. | Large-scale synthetic urban dataset for aerial scene understanding | |
| CN111316324A (en) | Automatic driving simulation system, method, equipment and storage medium | |
| CN116453004A (en) | Agricultural target detection data set construction method, system and electronic equipment | |
| CN111816022A (en) | Simulation method and device for simulation scene, storage medium and electronic equipment | |
| Zhuo et al. | A novel vehicle detection framework based on parallel vision | |
| CN115393822A (en) | Method and equipment for detecting obstacle in driving in foggy weather | |
| EP3855399A1 (en) | Photo-realistic image generation using geo-specific data | |
| Chen et al. | A study of parking-slot detection with the aid of pixel-level domain adaptation | |
| CN110717982A (en) | VR virtual manufacturing method for real scene | |
| Lu et al. | LiDAR-Forest Dataset: LiDAR Point Cloud Simulation Dataset for Forestry Application | |
| CN117593470B (en) | Street view reconstruction method and system based on AI model | |
| CN112288884B (en) | Method for realizing regional historical gymnasium experience based on virtual reality VR technology | |
| CN115937421B (en) | Method for generating simulated video data, image generating device and readable storage medium | |
| CN116188933B (en) | Method and device for predicting target direction of aerial view based on group-wise change |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |