CN117036599A - Virtual data set generation method and device, electronic equipment and storage medium - Google Patents
Virtual data set generation method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN117036599A CN117036599A CN202310943962.XA CN202310943962A CN117036599A CN 117036599 A CN117036599 A CN 117036599A CN 202310943962 A CN202310943962 A CN 202310943962A CN 117036599 A CN117036599 A CN 117036599A
- Authority
- CN
- China
- Prior art keywords
- data set
- virtual data
- virtual
- initial
- target virtual
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000003860 storage Methods 0.000 title claims abstract description 17
- 238000012805 post-processing Methods 0.000 claims abstract description 23
- 238000009877 rendering Methods 0.000 claims description 35
- 230000011218 segmentation Effects 0.000 claims description 24
- 238000012549 training Methods 0.000 claims description 20
- 239000007787 solid Substances 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 8
- 238000012795 verification Methods 0.000 claims description 3
- 238000001514 detection method Methods 0.000 description 28
- 238000010586 diagram Methods 0.000 description 9
- 238000012545 processing Methods 0.000 description 8
- 238000004422 calculation algorithm Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 6
- 238000004519 manufacturing process Methods 0.000 description 5
- 239000000463 material Substances 0.000 description 5
- 239000011159 matrix material Substances 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000011084 recovery Methods 0.000 description 2
- 229910001018 Cast iron Inorganic materials 0.000 description 1
- 239000008186 active pharmaceutical agent Substances 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 229910002056 binary alloy Inorganic materials 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Processing Or Creating Images (AREA)
Abstract
The application relates to a virtual data set generation method, a device, electronic equipment and a storage medium, wherein the virtual data set generation method comprises the following steps: loading a three-dimensional model of an object to be identified into a virtual space; shooting the three-dimensional model at a preset shooting point by controlling a virtual camera in the virtual space to obtain an initial virtual data set; and carrying out post-processing on the initial virtual data set to obtain a target virtual data set. Compared with the prior art, the application has the advantages of high efficiency, strong applicability and the like.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a virtual data set generating method, device, electronic apparatus, and storage medium.
Background
Target detection is the most challenging problem in the field of machine vision, and has the task of finding a target object in an image and determining the type and position of the target object, so that the target object has wide application scenes and task requirements in industrial production.
At present, the target detection algorithm based on the artificial neural network can gradually overtake the traditional detection algorithm in terms of accuracy and universality. Unlike traditional detection algorithms, artificial neural networks not only rely on constantly iterated and refined algorithm models, but also require tens of thousands of high-quality data as a model training set to ensure detection effects, which makes the production of the data set an important ring for training a detection model.
At present, the related art still adopts a manual or semi-automatic mode to manufacture a training data set, so that the training efficiency of the detection model is low.
Disclosure of Invention
The application aims to overcome the defects of the prior art and provide a virtual data set generating method, a device, electronic equipment and a storage medium with high efficiency.
The aim of the application can be achieved by the following technical scheme:
according to a first aspect of the present application, there is provided a virtual data set generation method, the method comprising:
loading a three-dimensional model of an object to be identified into a virtual space;
controlling a virtual camera in the virtual space to shoot the three-dimensional model at a preset shooting point to acquire an initial virtual data set;
and carrying out post-processing on the initial virtual data set to obtain a target virtual data set.
As a preferred technical solution, the target virtual data set includes: a single target virtual dataset;
the single-target virtual data set comprises an initial single-target virtual image and a mask map of the initial single-target virtual image;
the post-processing the initial virtual data set to obtain a target virtual data set includes:
acquiring a rendering outline of the initial single-target virtual image;
and obtaining a mask map of the initial single-target virtual image according to the rendering outline map.
As a preferred technical solution, the target virtual data set includes: a multi-target virtual dataset;
the multi-target virtual dataset includes an initial multi-target virtual image and a mask map of the initial multi-target virtual image;
the post-processing the initial virtual data set to obtain a target virtual data set includes:
obtaining a quasi-solid color rendering map of the initial multi-target virtual image;
and obtaining a mask map of the initial multi-target virtual image according to the quasi-solid color rendering map.
As a preferable technical scheme, the three-dimensional model includes material information, appearance information and label information of the object to be identified.
As a preferable technical solution, after the post-processing is performed on the initial virtual data set to obtain a target virtual data set, the method further includes:
and generating a tag file of the target virtual data set according to the tag information in the three-dimensional model.
As a preferable technical solution, the preset shooting points are:
a point of the virtual space, which is a preset distance away from the object to be identified;
and/or a point in the virtual space on a preset longitude;
and/or points in the virtual space at a preset latitude.
According to a second aspect of the present application, there is provided an example segmentation model training method, the method comprising:
training an instance segmentation model by using a virtual data set generated by the virtual data set generation method provided in the first aspect or any one of possible implementation manners of the first aspect;
verifying the trained example segmentation model by adopting a real data set;
and iterating the steps circularly until the example segmentation model meets the verification requirement.
According to a third aspect of the present application, there is provided a virtual data set generating apparatus comprising:
the model loading module is used for loading the three-dimensional model of the object to be identified into the virtual space;
the initial virtual data set acquisition module is used for controlling a virtual camera in the virtual space to shoot the three-dimensional model at a preset shooting point to acquire an initial virtual data set;
and the post-processing module is used for carrying out post-processing on the initial virtual data set to obtain a target virtual data set.
According to a fourth aspect of the present application there is provided an electronic device comprising a memory and a processor, the memory having stored thereon a computer program which when executed by the processor implements the method provided by the first aspect or any one of the possible implementations of the first aspect.
According to a fourth aspect of the present application there is provided a computer readable storage medium having stored thereon a computer program which when executed by a processor implements the method provided by the first aspect or any one of the possible implementations of the first aspect.
Compared with the prior art, the application has the following beneficial effects:
1. the efficiency is high: the virtual data set generation method can realize full-automatic generation of the virtual data set, can generate a large number of virtual data sets for training of the target detection model in a short time, can save a large number of manpower and time cost, and improves the generation efficiency of the virtual data set, thereby being beneficial to improving the training efficiency of the target detection model.
2. The applicability is strong: the virtual data set generation method can be applied to not only detection and identification scenes of objects in daily life, but also scenes needing to obtain outline data of a target object, such as industrial grabbing, object obstacle avoidance and the like; in addition, the virtual data set can replace an actual scene, so that the virtual data set generation method can be applied to scenes such as medical operation and underwater detection, in which the actual data set is difficult to acquire, and the applicability of the virtual data set generation method is effectively improved.
3. The target detection model with good effect can be obtained: when the virtual data set generated by the virtual data set generation method is used for training the target detection model such as the instance segmentation model, a better target detection effect can be obtained.
Drawings
FIG. 1 is a flow chart of a virtual data set generating method according to an embodiment of the application;
FIG. 2 is a model of a part 1 for single-target virtual dataset fabrication in accordance with an embodiment of the present application;
FIG. 3 is a model of a part used for multi-objective virtual dataset fabrication in accordance with an embodiment of the present application;
FIG. 4 is a schematic diagram of an initial virtual dataset acquisition process according to an embodiment of the present application;
FIG. 5 is a schematic diagram of a mask diagram of an initial single-target virtual image according to an embodiment of the present application;
FIG. 6 is a hybrid diagram of a virtual image and its corresponding mask diagram in an embodiment of the present application;
wherein, fig. 6 (a) and 6 (b) are respectively a single-object mixed graph of initial single-object virtual images of the part 1 and the part 2 and corresponding mask graphs, and fig. 6 (c) is a mixed graph containing the part 1 in fig. 6 (a) and a mixed effect graph of the part 2 in fig. 6 (b);
FIG. 7 is a training loss curve of an example segmentation model in an embodiment of the present application;
FIG. 8 is a graph showing the detection result of a real image using an example segmentation model in an embodiment of the present application;
fig. 8 (a) shows the detection result of a single target, and fig. 8 (b) to 8 (e) show the detection results of a plurality of targets;
fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having" and any variations thereof in the description of the application and the claims and the description of the drawings above are intended to cover a non-exclusive inclusion.
In the description of embodiments of the present application, the technical terms "first," "second," and the like are used merely to distinguish between different objects and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated, a particular order or a primary or secondary relationship. In the description of the embodiments of the present application, the meaning of "plurality" is two or more unless explicitly defined otherwise.
Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
In the description of the embodiments of the present application, the term "and/or" is merely an association relationship describing an association object, and indicates that three relationships may exist, for example, a and/or B may indicate: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship.
Fig. 1 is a flow chart of a virtual data set generating method according to an embodiment of the present application. The present application provides method operational steps as described in the examples or flowcharts, but may include more or fewer operational steps based on conventional or non-inventive labor. The order of steps recited in the embodiments is merely one way of performing the order of steps and does not represent a unique order of execution. The method should be implemented in software and/or hardware. Referring to fig. 1, the method may include:
step S110: loading a three-dimensional model of an object to be identified into a virtual space;
step S120: shooting the three-dimensional model at a preset shooting point by controlling a virtual camera in the virtual space to obtain an initial virtual data set;
step S130: and carrying out post-processing on the initial virtual data set to obtain a target virtual data set.
Alternatively, the virtual space in step S110 may be constructed using the SolidWorks software, and the three-dimensional model of the object to be identified may be a three-dimensional digital model previously constructed by the SolidWorks software. Of course, the three-dimensional model in step S110 may be constructed by other software.
It is understood that the three-dimensional model in the step S110 may be directly constructed by the electronic device performing the virtual data set generating method, or may be constructed by another electronic device and then the constructed three-dimensional model is sent to the electronic device performing the virtual data set generating method.
Optionally, the three-dimensional model in step S110 includes texture information, appearance information, and tag information of the object to be identified, where the tag information may be used to generate a tag file of the target virtual dataset.
Optionally, step S110 may set a virtual background of the virtual space before loading the three-dimensional model of the object to be identified into the virtual space, where the virtual background includes a background environment, an object position, an illumination parameter, and the like.
It is understood that the three-dimensional model is also classified into a single-target three-dimensional model including only one target to be recognized and a multi-target three-dimensional model including a plurality of targets to be recognized. The single-object three-dimensional model may be a part model as shown in fig. 2, the multi-object three-dimensional model may be an assembly model as shown in fig. 3, and the relative positions between the multiple objects may be determined by the relative positions between the parts in the assembly model.
Alternatively, step S120 may use the self-contained rendering plug-in photoview360 in the SolidWorks to perform rendering shooting, and the rendering parameter settings in performing rendering shooting may refer to table 1. The contour color of the rendering contour map in the single-target virtual data set can be set according to the virtual background color and the three-dimensional model color, and the contour can be well distinguished from the virtual background and the three-dimensional model.
Table 1 rendering shooting parameters
Optionally, the preset shooting points in step S120 are: a point of a virtual space, which is a preset distance away from the object to be identified; and/or a point in the virtual space that is on a preset longitude; and/or points in the virtual space that are at a preset latitude.
It can be understood that, in this embodiment, the longitude and latitude of the camera in space and the distance from the object may be modified to implement omnibearing continuous shooting of the target object, and a specific implementation manner is shown in fig. 4. Wherein L (unit: mm) is the distance from the camera to the target object; a1 The unit is the longitude value in the spherical coordinate system where the camera is located; a2 The unit is the latitude value in the spherical coordinate system where the camera is located; q is a counter, counting the number of virtual pictures generated by rendering and providing serial numbers for the virtual pictures as file names for storage. It can be understood that the implementation manner shown in fig. 4 is also equivalent to controlling the virtual camera to shoot at a preset shooting point, and the change process of the longitude and latitude value and the distance value between the camera and the object to be identified is the process of moving from one preset shooting point to another preset shooting point.
Optionally, the target virtual data set comprises: a single target virtual dataset; the single-target virtual data set comprises an initial single-target virtual image and a mask map of the initial single-target virtual image;
at this time, step S130 performs post-processing on the initial virtual data set to obtain a target virtual data set, including: acquiring a rendering outline of an initial single-target virtual image; and obtaining a mask map of the initial single-target virtual image according to the rendering contour map.
It can be appreciated that the initial single-target virtual image and the rendering profile map can be obtained by performing two renderings in the same camera view, and are respectively stored in a JPG format and a BMP format. In the SolidWorks software, the rendering parameters ContourLineTheickness and ContourLineContolor can be set, so that the rendered picture contains surface feature contours, a target object can be conveniently distinguished from the background, and convenience is provided for subsequent data processing.
Alternatively, the mask map of the initial single-target virtual image may be an index map file with a bit depth of 8, and the acquiring method may be: as shown in fig. 5, after the rendering contour map of the initial single-target virtual image is acquired, the rendering contour map contains a contour with color (for example, a red contour), and the RGB three-channel can be traversed according to the color category, so that the contour is extracted from the rendering contour map; then performing binarization operation on the image, and filling holes in the outline by judging connectivity of the target pixel points; and finally, converting the picture into a required format, converting each pixel point in the image into 8-bit depth from binary system on the basis of retaining original data, then loading the image into an index table, rewriting the image into an index map format, and finally, modifying the image format of the image into a PNG format from BMP to obtain a mask map of the initial single-target virtual image.
Optionally, the target virtual data set comprises: a multi-target virtual dataset; the multi-target virtual dataset includes an initial multi-target virtual image and a mask map of the initial multi-target virtual image;
at this time, step S130 performs post-processing on the initial virtual data set to obtain a target virtual data set, including: obtaining a quasi-solid color rendering diagram of an initial multi-target virtual image; and obtaining a mask map of the initial multi-target virtual image according to the quasi-solid color rendering map.
It can be appreciated that the multi-target data includes a plurality of target objects in a picture, and the objects need to be associated with respective labels, and the embodiment uses a pure-color-like rendering mode to provide data support for the subsequent generation of the mask file. The method for rendering the initial multi-target virtual image is characterized in that the same group of parts are selected to be subjected to batch processing, one group is used for rendering to generate an original image, the other group is used for pure color rendering, the two groups keep the same part placement mode and ensure that the camera angle of each rotation is completely consistent with the changed distance, so that matching of the original image and a mask is realized, and the running time of an algorithm is shortened.
According to the quasi-solid color rendering graph, the method for acquiring the mask graph of the initial multi-target virtual image can be as follows:
the initial multi-target virtual image respectively identifies the colors corresponding to all target parts according to the color sequence in the index table, and by combining the generated label information, all the pixel points in the image are traversed and judged to determine which part the pixel point belongs to, and if not, the pixel point is judged to be the background;
the coordinates of the pixel points after judgment are respectively stored in each matrix, the independent mask of each target can be obtained in three different matrixes after binarization operation, hole filling and format conversion are carried out on each matrix, at the moment, the background pixel value of each independent mask is 0, the mask pixel value is 1, and the target area pixel values in each color matrix are modified according to the sequence of an index table and then are combined;
after the data matrix is manufactured, the index table is loaded and modified into PNG format for storage.
It can be understood that the present embodiment designs the overlap detection algorithm, and performs data overlap once before modifying each matrix data, and the mask pixel values are all 1 at this time, so that the portion with the pixel value greater than 1 after overlap is the overlap portion. The overlapping portion occurs only at the edge transition of two or more parts, and the pixel value of the overlapping portion is set to zero, so that the overlapping portion becomes a black background.
Optionally, after obtaining the mask map of the initial multi-target virtual image of each target according to the quasi-solid color rendering map, creating a mixed superimposed image by calling an imfuse function in MATLAB, using alpha to mix the overlay virtual original map and the mask map, and jointly scaling the intensity values in the image to mix them in the same image, wherein the mixed map of the original map and the mask map is shown in fig. 6, fig. 6 (a) and fig. 6 (b) are respectively a single target mixed map of the initial single target virtual image of the part 1 and the part 2 and the corresponding mask map, and fig. 6 (c) is a mixed map containing the part 1 in fig. 6 (a) and a mixed effect map of the part 2 in fig. 6 (b);
optionally, after post-processing the initial virtual data set in step S130 to obtain a target virtual data set, the method further includes: and generating a tag file of the target virtual data set according to the tag information in the three-dimensional model.
In this embodiment, the part label is directly written into the Excel file, and the label source is a preset part name in the three-dimensional model.
The part names of the multi-objective data are subordinate to the assembly, and similar parts may exist. And aiming at the condition that a plurality of part names with uncertain number exist in the assembly body, reading the number and name information of the visible parts under the current camera view angle through an API interface, and sequentially reading name labels and writing the name labels into Excel according to the number of the visible parts. When a plurality of similar parts exist, different parts of the same type are distinguished by the form of tail. The method comprises the steps of loading Pattern Recognition Toolbox additional functional resources in MATLAB, reading the content of an Excel intermediate file stored with tag information by MATLAB, and finally storing the content in YML file.
The foregoing describes an embodiment of a virtual data set generating method according to the present application, and the following describes an embodiment of an example segmentation model training method to further describe the scheme according to the present application.
The embodiment of the application also provides an example segmentation model training method, which comprises the following steps:
training the instance segmentation model by adopting the virtual data set generated by the virtual data set generation method;
verifying the trained example segmentation model by adopting a real data set;
and iterating the steps circularly until the example segmentation model meets the verification requirement.
The example segmentation model may employ a Mask R-CNN model, with a virtual dataset composed of 504 virtual images of part 1, 504 virtual images of part 2, and 504 multi-target virtual images containing part 1 and part 2.
In the embodiment, a deep learning computing platform based on a GPU is built, and hardware parameters of the platform are shown in a table 2. The software is run by Windows10 system, the deep learning framework by TensorFlow and Keras, and the programming language by Python3.6 version. A data processing library of Python and an image processing library such as OpenCV, numpy, pillow, scikit are configured.
Table 2 hardware configuration of computing platform
Training of a Mask R-CNN detection model is carried out by using the virtual data set, the network architecture is ResNet-101, the pre-training weight of COCO is adopted, the learning rate is 0.001, and 100epochs are trained altogether. The model training loss curve is shown in figure 7.
The real pictures containing part 1 and part 2 were examined using the trained example segmentation model, with the results shown in fig. 8. By analyzing the detection result, the method can obtain: the detection model trained by the virtual data set can better carry out example segmentation on the target object in the real environment. Fig. 8 (a) shows that the model has good segmentation capability for a single-target real picture. The backgrounds of fig. 8 (b) to 8 (e) are dot patterns which do not appear in the data set, and background reflections exist in different degrees, and in fig. 8 (d), a part 2 of cast iron material with slightly different shapes is detected, the material is not appearing in the virtual data set for training, the detection result shows that the detection model has good generalization capability, and the part 2 with different materials or slightly different shapes can be identified. Non-target parts in FIG. 8 (e) were not detected, indicating that the detection model has better rejection capability for the interference term.
In this embodiment, statistical analysis is performed on the case division precision of two types of parts, and the case division precision and the classification precision of two targets are shown in table 3. The example segmentation accuracy IoU refers to the ratio of the intersection of the segmentation detection result and the physical result group trunk of the object to the union thereof, which is a dimensionless number, the theoretical maximum value is 1, and the closer to 1, the higher the segmentation accuracy is; accuracy precision refers to the ratio of the number of samples for which the predicted value is true to the actual true; recall recovery refers to the detected proportion of all samples that are actually true. If the single index is higher, the other index is too low, which indicates that the detection model has classification problem.
Table 3 example segmentation accuracy and classification accuracy for two targets
| Target object | Division accuracy IoU (%) | Accuracy precision (%) | Recall rate recovery (%) |
| Part 1 | 82.4 | 97.6 | 96.4 |
| Part 2 | 85.5 | 98.8 | 96.5 |
The analysis result shows that the virtual data set manufactured by the scheme can be converged in the Mask R-CNN network model training process, meanwhile, the virtual data set has a good detection effect in real picture-oriented detection, and the feasibility of the virtual data set manufacturing method is verified.
The above description of the method embodiments further describes the solution of the present application by means of device embodiments.
The embodiment of the application also provides a virtual data set generating device, which comprises:
the model loading module is used for loading the three-dimensional model of the object to be identified into the virtual space;
the initial virtual data set acquisition module is used for controlling a virtual camera in the virtual space to shoot the three-dimensional model at a preset shooting point to acquire an initial virtual data set;
and the post-processing module is used for carrying out post-processing on the initial virtual data set to obtain a target virtual data set.
Optionally, the target virtual data set acquired by the post-processing module includes: a single target virtual dataset;
the single-target virtual data set comprises an initial single-target virtual image and a mask map of the initial single-target virtual image;
at this time, the post-processing module is specifically configured to: acquiring a rendering outline of the initial single-target virtual image; and obtaining a mask map of the initial single-target virtual image according to the rendering outline map.
Optionally, the target virtual data set acquired by the post-processing module includes: a multi-target virtual dataset; the multi-target virtual dataset comprises an initial multi-target virtual image and a mask map of the initial multi-target virtual image;
at this time, the post-processing module is specifically configured to: obtaining a quasi-solid color rendering map of the initial multi-target virtual image; and obtaining a mask map of the initial multi-target virtual image according to the quasi-solid color rendering map.
Optionally, the three-dimensional model loaded into the virtual space by the model loading module includes material information, appearance information and label information of the object to be identified.
Optionally, the virtual data set generating device further includes:
and the tag file generation module is used for generating a tag file of the target virtual data set according to the tag information in the three-dimensional model.
Optionally, the preset shooting points in the initial virtual data set acquisition module are: a point of the virtual space, which is a preset distance away from the object to be identified; and/or a point in the virtual space on a preset longitude; and/or points in the virtual space at a preset latitude.
Fig. 9 shows a schematic block diagram of an electronic device that may be used to implement embodiments of the present disclosure. As shown in fig. 9, the electronic device of the present application includes a Central Processing Unit (CPU) that can perform various appropriate actions and processes according to computer program instructions stored in a Read Only Memory (ROM) or computer program instructions loaded from a storage unit into a Random Access Memory (RAM). In the RAM, various programs and data required for the operation of the device can also be stored. The CPU, ROM and RAM are connected to each other by a bus. An input/output (I/O) interface is also connected to the bus.
A plurality of components in a device are connected to an I/O interface, comprising: an input unit such as a keyboard, a mouse, etc.; an output unit such as various types of displays, speakers, and the like; a storage unit such as a magnetic disk, an optical disk, or the like; and communication units such as network cards, modems, wireless communication transceivers, and the like. The communication unit allows the device to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The processing unit performs the respective methods and processes described above, such as the inventive method steps S110 to S130. For example, in some embodiments, method steps S110-S130 of the present application may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device via the ROM and/or the communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more of the steps of the method steps S110 to S130 of the application described above may be performed. Alternatively, in other embodiments, the CPU may be configured to perform the inventive method steps S110-S130 by any other suitable means (e.g. by means of firmware).
The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.
Program code for carrying out methods of the present application may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present application, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
While the application has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the application. Therefore, the protection scope of the application is subject to the protection scope of the claims.
Claims (10)
1. A method of generating a virtual dataset, the method comprising:
loading a three-dimensional model of an object to be identified into a virtual space;
controlling a virtual camera in the virtual space to shoot the three-dimensional model at a preset shooting point to acquire an initial virtual data set;
and carrying out post-processing on the initial virtual data set to obtain a target virtual data set.
2. The method of claim 1, wherein the target virtual data set comprises: a single target virtual dataset;
the single-target virtual data set comprises an initial single-target virtual image and a mask map of the initial single-target virtual image;
the post-processing the initial virtual data set to obtain a target virtual data set includes:
acquiring a rendering outline of the initial single-target virtual image;
and obtaining a mask map of the initial single-target virtual image according to the rendering outline map.
3. The method of claim 1, wherein the target virtual data set comprises: a multi-target virtual dataset;
the multi-target virtual dataset includes an initial multi-target virtual image and a mask map of the initial multi-target virtual image;
the post-processing the initial virtual data set to obtain a target virtual data set includes:
obtaining a quasi-solid color rendering map of the initial multi-target virtual image;
and obtaining a mask map of the initial multi-target virtual image according to the quasi-solid color rendering map.
4. A virtual data set generating method according to any one of claims 1 to 3, wherein the three-dimensional model includes texture information, appearance information, and tag information of the object to be identified.
5. The method of generating a virtual data set according to claim 4, further comprising, after said post-processing said initial virtual data set to obtain a target virtual data set:
and generating a tag file of the target virtual data set according to the tag information in the three-dimensional model.
6. A virtual data set generating method according to any one of claims 1 to 3, wherein the preset shooting points are:
a point of the virtual space, which is a preset distance away from the object to be identified;
and/or a point in the virtual space on a preset longitude;
and/or points in the virtual space at a preset latitude.
7. An example segmentation model training method, the method comprising:
training an instance segmentation model by using the virtual data set generated by the virtual data set generation method according to any one of claims 1 to 6;
verifying the trained example segmentation model by adopting a real data set;
and iterating the steps circularly until the example segmentation model meets the verification requirement.
8. A virtual data set generating apparatus, the apparatus comprising:
the model loading module is used for loading the three-dimensional model of the object to be identified into the virtual space;
the initial virtual data set acquisition module is used for controlling a virtual camera in the virtual space to shoot the three-dimensional model at a preset shooting point to acquire an initial virtual data set;
and the post-processing module is used for carrying out post-processing on the initial virtual data set to obtain a target virtual data set.
9. An electronic device comprising a memory and a processor, the memory having stored thereon a computer program, characterized in that the processor, when executing the program, implements the method according to any of claims 1-6 or 7.
10. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which is executed by a processor by the method according to any of claims 1-6 or 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310943962.XA CN117036599A (en) | 2023-07-28 | 2023-07-28 | Virtual data set generation method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310943962.XA CN117036599A (en) | 2023-07-28 | 2023-07-28 | Virtual data set generation method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117036599A true CN117036599A (en) | 2023-11-10 |
Family
ID=88638295
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310943962.XA Pending CN117036599A (en) | 2023-07-28 | 2023-07-28 | Virtual data set generation method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117036599A (en) |
-
2023
- 2023-07-28 CN CN202310943962.XA patent/CN117036599A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108764358B (en) | Terahertz image identification method, device and equipment and readable storage medium | |
| CN104778721B (en) | The distance measurement method of conspicuousness target in a kind of binocular image | |
| CN104966085B (en) | A kind of remote sensing images region of interest area detecting method based on the fusion of more notable features | |
| CN108710916B (en) | Picture classification method and device | |
| WO2022179549A1 (en) | Calibration method and apparatus, computer device, and storage medium | |
| CN105678318A (en) | Traffic label matching method and apparatus | |
| CN111310690B (en) | Forest fire recognition method and device based on CN and three-channel capsule network | |
| CN112836625A (en) | Face living body detection method and device and electronic equipment | |
| CN112215861A (en) | Football detection method and device, computer readable storage medium and robot | |
| CN114581646A (en) | Text recognition method and device, electronic equipment and storage medium | |
| CN111353325A (en) | Key point detection model training method and device | |
| CN117523087B (en) | Three-dimensional model optimization method based on content recognition | |
| CN113223098A (en) | Preprocessing optimization method for image color classification | |
| CN116071625B (en) | Training method of deep learning model, target detection method and device | |
| CN109934152B (en) | Improved small-bent-arm image segmentation method for sign language image | |
| CN112396638A (en) | Image processing method, terminal and computer readable storage medium | |
| CN114118127B (en) | Visual scene sign detection and recognition method and device | |
| CN115984712A (en) | Multi-scale feature-based remote sensing image small target detection method and system | |
| CN117036599A (en) | Virtual data set generation method and device, electronic equipment and storage medium | |
| CN114373144B (en) | Automatic identification method for circular identification points in high-speed video | |
| CN115018820A (en) | Breast cancer multi-classification method based on texture enhancement | |
| CN114463534A (en) | Target key point detection method, device, equipment and storage medium | |
| CN112906629A (en) | Training of facial expression classifier and facial expression recognition method and device | |
| CN111325209B (en) | License plate recognition method and system | |
| CN102938156B (en) | Planar note configuration method based on integral images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |