CN114049536A - Virtual sample generation method and device, storage medium and electronic equipment - Google Patents

Abstract

The invention discloses a virtual sample generation method, device, storage medium and electronic equipment. The method includes: acquiring background scene samples and data samples to be detected; constructing a sample relationship model according to the information of the data samples; Perform data enhancement to obtain enhanced data samples; arrange the enhanced data samples according to the arrangement rules constructed by the sample relationship model, and then embed the background scene samples to generate the first sample; according to the adversarial network, the first sample is styled Migrate to get a dummy sample. By implementing the present invention, in the process of generating high-quality virtual samples, by introducing relational modeling, the arrangement of samples in the generated virtual samples is more in line with the actual application scenario, and at the same time, the generative adversarial network is used to transfer the style of the data samples, so that the The final generated sample style is consistent, and the sample image can be smoothly embedded in the background, making the image more realistic.

Description

A virtual sample generation method, device, storage medium and electronic device

technical field

The invention relates to the technical field of computer vision and pattern recognition, and in particular to a virtual sample generation method, device, storage medium and electronic device.

Background technique

In recent years, artificial intelligence technology has developed rapidly in the industry. People use deep learning methods to perform image recognition and target detection to help people meet various needs. Among them, the classification and detection model constructed by neural network algorithm has achieved good results in industry. However, technologies such as image recognition need to rely on the drive of big data, and only after relying on a large amount of training data to train and learn the model can achieve better practical application results.

In practical application scenarios, there is often insufficient data, and the model cannot learn to perform well with a small amount of data. Therefore, in order to obtain better algorithm effects, it is necessary to obtain more pictures with rich features by artificial means. . However, a larger sample size will not only increase the cost, but also increase the difficulty of sample acquisition.

SUMMARY OF THE INVENTION

In view of this, embodiments of the present invention provide a virtual sample generation method, device, storage medium and electronic device to solve the technical problem of few training samples in the prior art when training and learning a model.

The technical scheme proposed by the present invention is as follows:

A first aspect of the embodiments of the present invention provides a method for generating virtual samples, including: acquiring background scene samples and data samples to be detected; constructing a sample relationship model according to information of the data samples; performing data enhancement on the data samples to be detected to obtain enhanced After the enhanced data samples are arranged according to the arrangement rules constructed by the sample relationship model, the background scene samples are embedded to generate the first sample; the style transfer is performed on the first sample according to the adversarial network, Get a dummy sample.

Optionally, the data enhancement method includes: random scale jitter, data enhancement based on HSV space, image sharpness enhancement, and random flip and rotation; data enhancement is performed on the data samples to be detected, including: for each data enhancement method Data enhancement is performed randomly on the data samples to be detected according to a preset probability.

Optionally, constructing a sample relationship model according to the information of the data samples includes: acquiring sales information of the data samples; determining product specification information of the data samples according to the sales information; based on a data mining algorithm, according to the sales information and the product specification information Calculate the correlation between the sales volume of each data sample brand and other brands, and obtain the data specifications with the correlation greater than the threshold; build the sample relationship model according to the quality information of the data samples and the data specifications with the correlation greater than the threshold.

Optionally, the enhanced data samples are arranged according to the arrangement rules constructed by the sample relationship model and then embedded in the background scene samples to generate the first sample, including: according to the random arrangement method, product specification information or correlation The information arranges the data samples to obtain the arranged data samples; and embeds the arranged data samples into the background scene samples to generate a first sample.

Optionally, the loss function of the adversarial network is represented by the following formula:

表示对抗性损失，G表示生成器，D_Y表示判别器，

表示循环一致性损失。in,

represents the adversarial loss, G represents the generator, D _Y represents the discriminator,

represents the cycle consistency loss.

Optionally, performing style transfer on the first sample according to the adversarial network to obtain a virtual sample, comprising: cutting each data sample image in the first sample and inputting it to the adversarial network to perform style transfer, and obtaining a transferred image. Embed the migrated data sample image into the first sample to obtain a virtual sample.

Optionally, the data sample to be detected is a sample of the outer packaging of a cigarette pack.

A second aspect of the embodiments of the present invention provides a virtual sample generation device, including: a sample acquisition module for acquiring background scene samples and data samples to be detected; a relational model building module for constructing a sample relational model according to information of the data samples The enhancement module is used to perform data enhancement on the data samples to be detected to obtain the enhanced data samples; the arrangement module is used for arranging the enhanced data samples according to the arrangement rules constructed by the sample relationship model and then embedding the enhanced data samples. The background scene sample is used to generate a first sample; the style transfer module is used for performing style transfer on the first sample according to the adversarial network to obtain a virtual sample.

A third aspect of the embodiments of the present invention provides a computer-readable storage medium, where the computer-readable storage medium stores computer instructions, and the computer instructions are used to cause the computer to execute the first and first aspects of the embodiments of the present invention. The virtual sample generation method of any one of the aspects.

A fourth aspect of the embodiments of the present invention provides an electronic device, including: a memory and a processor, the memory and the processor are connected in communication with each other, the memory stores computer instructions, and the processor executes the computer instructions, so as to execute the virtual sample generation method according to any one of the first aspect and the first aspect of the embodiments of the present invention.

The technical scheme provided by the invention has the following effects:

In the virtual sample generation method, device, storage medium and electronic device provided by the embodiments of the present invention, in the process of generating high-quality virtual samples, by introducing relationship modeling, the arrangement of samples in the generated virtual samples is more in line with actual application scenarios, and at the same time Generative adversarial network is used to transfer the style of the data samples, so that the final generated sample style is consistent, and the sample image can be smoothly embedded in the background, making the image more realistic.

Description of drawings

In order to illustrate the specific embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the specific embodiments or the prior art. Obviously, the accompanying drawings in the following description The drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

1 is a flowchart of a method for generating a virtual sample according to an embodiment of the present invention;

2 is a schematic diagram of a data enhancement method based on random dithering of image size according to an embodiment of the present invention;

3 is a schematic diagram of a data enhancement method based on HSV space according to an embodiment of the present invention;

4 is a schematic diagram of a data enhancement method based on image sharpness enhancement according to an embodiment of the present invention;

5 is a schematic diagram of a data enhancement method based on random flip and rotation according to an embodiment of the present invention;

6 is a schematic diagram of a virtual sample generated by a method for generating a virtual sample according to an embodiment of the present invention;

7 is a flowchart of a method for generating a virtual sample according to another embodiment of the present invention;

8 is a structural block diagram of a virtual sample generating apparatus according to an embodiment of the present invention;

9 is a schematic structural diagram of a computer-readable storage medium provided according to an embodiment of the present invention;

FIG. 10 is a schematic structural diagram of an electronic device provided according to an embodiment of the present invention.

Detailed ways

As described in the background art, the model cannot learn to obtain better performance with a small amount of data, so in order to obtain better algorithm effects, it is necessary to obtain more pictures containing rich features by artificial means. At present, there are two ways to increase the sample. One is to adjust the sampling frequency to expand the number of times the sample is trained. However, this method may cause the model to overfit the training sample and reduce the robustness of the model. It is possible that On the contrary, it reduces the actual performance of the model. Another way is to perform data enhancement on a small number of data samples, by using existing dataset images to make minor adjustments, such as rotation, displacement, flip, etc., which is achieved by introducing a priori knowledge semantic invariance, that is, these operations Without changing the semantic information of the image, these operations can make the network more robust, and at the same time make a complement to those images with a smaller number of categories.

Data enhancement in the field of image classification, that is, using the second method to increase the sample, is to transform the entire image. Since there is only one category in the entire image, no matter how the change is made, its label will not change. , in the field of target detection, an image contains different types of samples to be identified, and the location information of the target needs to be located at the same time, that is, the quality of the samples generated when the samples are increased by the second method is poor. Therefore, increasing the sample size also requires more complex processing.

In view of this, an embodiment of the present invention provides a method for generating a virtual sample, including: acquiring a background scene sample and a data sample to be detected; constructing a sample relationship model according to the product specification information of the data sample; performing data enhancement on the data sample to be detected , obtain the enhanced data samples; arrange the enhanced data samples according to the arrangement rules constructed by the sample relation model to obtain the arranged data samples; embed the arranged data samples into the background scene samples to generate the first sample; cut each data sample image in the first sample and input it to the adversarial network for style transfer to obtain the migrated data sample image; embed the migrated data sample image into the first sample, Get a dummy sample.

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

An embodiment of the present invention provides a virtual sample generation method. As shown in FIG. 1 , the method includes the following steps:

Step S101: Obtain background scene samples and data samples to be detected.

Specifically, the data sample may be data for which the sample needs to be increased. In one embodiment, the data is a sample of cigarette pack outer packaging. At present, when a neural network is used to detect a cigarette image, an image of a cigarette retail terminal is usually obtained. However, due to the disclosure rate of some cigarette product specifications (brands and specifications), the number of collected images is small, so that the category cannot be fully trained during model training, which affects the detection effect of the model. Therefore, the virtual sample can be used to generate a virtual sample for the outer package image of the cigarette pack, thereby increasing the sample size. In addition, the data sample may also be other data that needs to be sampled.

In one embodiment, when the data sample to be detected is a sample of the outer packaging of a cigarette pack, the background scene sample may be a sample including an image of a retail cabinet where cigarettes are placed.

Step S102: Build a sample relationship model according to the information of the data samples. Specifically, in order to make the generated virtual samples more suitable for actual application scenarios and improve the quality of the generated virtual samples, a sample relationship model can be constructed according to the relevant information of the data samples. For example, in order to make the virtual sample of cigarettes more suitable for the retail scene, the retail or sales information of cigarettes can be obtained to build a relationship model of cigarettes.

In one embodiment, constructing a sample relationship model according to the information of the data samples includes: acquiring the sales information of the data samples; determining the product specification information of the data samples according to the sales information; According to the regulation information, the correlation between the sales volume of each data sample brand and the sales volume of other brands is calculated, and the data regulation with the correlation greater than the threshold is obtained.

For cigarette samples, by collecting cigarette product specification information and cigarette sales related data in retail stores, data mining algorithms such as Apriori algorithm can be used to calculate the difference between the sales volume of each cigarette brand and the sales volume of other cigarette brands on the collected cigarette sales information. Correlation, by inputting the cigarette brand to be calculated into the data mining algorithm model, after calculation, the 5 remaining cigarette specifications with the greatest correlation with each cigarette specification in the cigarette sample are obtained, and they are stored in the database together with the cigarette specification information. , get the relational model of cigarettes. Therefore, by querying the database, several cigarette brand specifications that are most related to a specified cigarette brand specification can be obtained, and information such as the cigarette brand and manufacturer to which the cigarette of this specification belongs can also be obtained by querying.

Step S103: Perform data enhancement on the data samples to be detected to obtain enhanced data samples. Specifically, due to the need to generate a large number of high-quality virtual samples with a small amount of image data, the samples generated by simply cutting and mapping methods cannot meet the needs of large quantities of image samples for cigarette packs, and also lack the variety of images. sex. In order to enhance the diversity of generated images and improve the quality of augmented data, random data augmentation methods for samples are used in each data augmentation process to increase the richness of images. Therefore, when performing data enhancement on the data samples to be detected, a variety of data amplification methods can be selected, and for each data amplification method, the data samples to be detected are randomly enhanced according to a preset probability.

In one embodiment, for cigarette samples, the data augmentation strategy is mainly carried out in two general directions: color channel and geometric transformation. Therefore, the data enhancement method includes: image scale random dithering, data enhancement based on HSV color space, image sharpness enhancement, random flip and rotation and other enhancement methods, so as to increase the richness of virtual samples during generation, so that each generated The images are all different.

Among them, as shown in Figure 2, the random scale jitter is to randomly select a coefficient between 0.5 and 1.5 to scale the size of the image to be amplified, and the size of the amplified image is the length and width of the original image multiplied by the coefficient. ; As shown in Figure 3, the data enhancement based on HSV space is to convert the image from RGB space to HSV space, and then fine-tune the image from the three dimensional values of hue, saturation and brightness, and the adjustment range is kept in the range of 0.8-1.2 In order to imitate the image acquisition under different lighting conditions, in order not to generate too distorted images, adjustments are made in a small range, thereby further enhancing the diversity of the amplified images. The RGB to HSV conversion formula is expressed by the following formula:

H=0°, max(R,G,B)=min(R,G,B)

As shown in Figure 4, image sharpness enhancement is to adjust the edge characteristics and overall sharpness of the image through image sharpening; as shown in Figure 5, random flip and rotation is to randomly perform horizontal flip operations and The tilt and rotation of a small angle is used to simulate various placement methods of cigarettes, which enhances the robustness of the model.

Based on the four data amplification methods mentioned above, ten methods for sample amplification can be obtained. Specifically, it includes: 1. Horizontal movement of the bounding box 2. Vertical movement of the bounding box 3. Size jitter of the bounding box 4. Brightness adjustment of the bounding box 5. Saturation adjustment of the bounding box 6. Hue adjustment of the bounding box 7. Bounding box The sharpening adjustment of 8, the horizontal flip of the bounding box 9, the rotation of the bounding box 10, the histogram equalization.

During data enhancement, the above-mentioned ten data augmentation methods can be used to augment images randomly based on preset probabilities. For example, if the preset probability is 0.5, then, through ten data augmentation methods, theoretically, each image can be augmented to obtain 2 ¹⁰ =1024 different samples. However, in practical applications, cigarette data samples are usually obtained by clipping images of cigarette containers collected by stores. When a cigarette sample is obtained by clipping, not only the cigarette image but also the background information around the cigarette will be clipped. Therefore, even if the cigarette images are the same, the surrounding background information is different (for example, the containers used are different). And, for the cropped cigarette samples, during the actual processing, dozens of different cigarette samples will be pasted into one image by means of maps. In this way, even for the same cigarette image, the surrounding background information and the surrounding cigarette arrangement are different.

Therefore, when data enhancement is performed on each cigarette sample in an image, the location information around each cigarette sample (including surrounding background information and surrounding cigarette arrangement) is different. Therefore, data enhancement in this way can prevent the network from overfitting and enhance its robustness compared with direct oversampling. At the same time, through data enhancement, a very small number of cigarette pack outer packaging images are used to generate a large number of virtual cigarette pack outer packaging samples, which can expand the training data set for balanced cigarette pack outer packaging identification, thereby realizing rich data diversity and cigarette packaging. The purpose of class balance.

Step S104: Arrange the enhanced data samples according to the arrangement rule constructed by the sample relationship model, and then embed the background scene samples to generate a first sample. Specifically, although the enhanced data samples meet the requirements for increasing the samples, the obtained increased samples or the quality of the expanded training data set cannot meet the requirements. Therefore, in order to make the enhanced data samples more in line with the actual situation, the enhanced data samples can be arranged using an arrangement rule constructed based on the sample relationship model.

In one embodiment, the arrangement rule includes a random arrangement manner, a manner of arranging the data samples based on product specification information or correlation information. Among them, the random placement method refers to completely randomly selecting different types of cigarette samples, and randomly selecting different positions for placement. When placing, it is necessary to pay attention to the position and size not exceeding the limit. Placement based on product specification information means randomly selecting a cigarette product specification in each row, inputting the relationship model and selecting all the cigarette product specifications of the manufacturer for placement, ensuring that the cigarette product specifications in each row are the same. Placement based on correlation information means that for a certain type of cigarette sample, several cigarette specifications with the highest degree of correlation are obtained from the relationship model, and the specifications are selected and placed on the left and right sides.

After the enhanced samples are placed in the above-mentioned placement manner, they can be embedded in the background scene samples to obtain the first sample. Specifically, for the placed sample, it can be embedded in one of the background scene samples or embedded in the retail cabinet of the background scene samples. Thus, the constraints on placement can be determined based on the background scene samples. By arranging the enhanced data samples according to the arrangement rules constructed by the sample relationship model, it is possible to simulate various cigarette placement methods in a real retail scene, so that the first sample formed is closer to the actual scene and has higher quality .

Step S105: Perform style transfer on the first sample according to the adversarial network to obtain a virtual sample.

When collecting images, the lighting and viewing angle of each image are different, so the image style of each image is also different. In the process of generating virtual samples, the styles of different cigarette quality images may not be the same. Unification, this will cause the generated virtual sample to be quite different from the real collected image. At the same time, when the enhanced cigarette image is directly embedded in the background image, there will be a large difference between the edge of the cigarette image and the original background image. Differences, the edges are not smooth enough, which also affects the quality of the generated virtual samples. In order to solve this problem, an adversarial network (CycleGAN) is used to transfer the style of the image, so that the image maintains a relatively consistent style.

Before adopting the adversarial network for style transfer, the CycleGAN network was first trained with real cigarette display images and artificially generated texture images, and the trained adversarial network was obtained. The loss function of this network is as follows:

表示对抗性损失，通过生成器G输入域X内的样本得到伪造样本G(X)，目标是使得G(X)与域Y内的样本y尽可能相似，判别器D_Y的目标是尽可能的区分出y与G(X)。

表示循环一致性损失，用来约束生成器得到的样本G(X)在内容上能与原来的样本x保持一致。in,

Represents an adversarial loss, and obtains a fake sample G(X) through the sample in the input domain X of the generator G. The goal is to make G(X) as similar as possible to the sample y in the domain Y. The goal of the discriminator D _Y is to be as similar as possible. distinguishes y from G(X).

Represents the cycle consistency loss, which is used to constrain the sample G(X) obtained by the generator to be consistent with the original sample x in content.

Therefore, during the style transfer, the trained CycleGAN network is used to cut each cigarette pack outer sample and its surrounding context area in the virtual sample, and send it to the CycleGAN network, and the network outputs the image after style transfer. This image aligns the overall style of the sample cigarette pack with the background image. For the data sample image after style transfer, it can be re-embedded into the background scene sample of the first sample, so as to obtain a high-quality virtual sample. As shown in FIG. 6 , it is the virtual sample result generated by the virtual sample generation method.

In the method for generating virtual samples provided by the embodiments of the present invention, in the process of generating high-quality virtual samples, by introducing relational modeling, the arrangement of samples in the generated virtual samples is more in line with the actual application scenario, and the generative adversarial network is used to analyze the data samples. Perform style transfer, so that the final generated sample style is consistent, and the sample image can be smoothly embedded in the background, making the image more realistic.

In one embodiment, the virtual sample generation method can be applied to the virtual sample generation process of the outer packaging image of the cigarette pack; as shown in FIG. The packaging image is used as the basis for the generation of virtual samples. According to the arrangement rules of retail cigarette images and the characteristics of fixed scenes, the collected images of rare cigarette categories are used to generate enhanced training images through mapping methods and image data enhancement methods. Add to the data set to train the model to improve the ability to identify cigarette specifications with few images. In order to ensure that the artificially generated virtual samples have high quality, a relationship model of cigarette categories is constructed to model different cigarette types on sale. , to analyze the relationship between cigarette varieties when cigarettes are displayed, and to use this rule to match cigarette arrangements when generating virtual samples. In order to eliminate the style inconsistency and the difference between the image edges during mapping, the CycleGAN algorithm is used to learn the transformation method of style transfer, and through the style transformation module, a more realistic cigarette packaging sample image is generated.

The virtual sample generation method provided by the embodiment of the present invention aims at the problem that the number of samples of some cigarette packs provided in the image collection of the retail counter is small and the training is insufficient. The method of mapping is to artificially generate virtual samples of cigarette packs. Specifically, the images of retail cigarette packs are arranged regularly and the scene is relatively fixed. A large number of high-quality virtual samples are artificially generated by a small number of rare categories of cigarette pack outer packaging image samples and a variety of data enhancement methods. At the same time, in order to generate a more realistic virtual sample, use the correlation between different cigarette categories to generate a more targeted cigarette position placement strategy, and then use the CycleGAN network to unify the background of the cigarette pack in each virtual sample. Image style, the smooth embedded cigarette image boundary makes the generated samples more realistic and more in line with the cigarette pack outer packaging images in real retail scenarios. Therefore, the method can provide a high-quality method for expanding the training samples of the outer packaging of the small box for data-driven deep learning methods such as image recognition and target detection of cigarette packets.

An embodiment of the present invention further provides a virtual sample generation device, as shown in FIG. 8 , the device includes:

The sample acquisition module is used to acquire background scene samples and data samples to be detected; for details, refer to the corresponding parts of the above method embodiments, which will not be repeated here.

The relational model building module is used for constructing a sample relational model according to the quality regulation information of the data sample; for details, refer to the corresponding part of the above method embodiment, which will not be repeated here.

The enhancement module is used to perform data enhancement on the data samples to be detected to obtain the enhanced data samples; for details, refer to the corresponding parts of the above method embodiments, which will not be repeated here.

an arrangement module, configured to arrange the enhanced data samples according to the arrangement rules constructed by the sample relationship model, and then embed the background scene samples to generate a first sample; for details, refer to the corresponding part of the above method embodiment, which is not described here. Repeat.

The style transfer module is used for performing style transfer on the first sample according to the adversarial network to obtain a virtual sample; for details, refer to the corresponding part of the above method embodiment, which will not be repeated here.

In the virtual sample generation device provided by the embodiment of the present invention, in the process of generating high-quality virtual samples, by introducing relational modeling, the arrangement of samples in the generated virtual samples is more in line with the actual application scenario, and the generative adversarial network is used to analyze the data samples. Perform style transfer, so that the final generated sample style is consistent, and the sample image can be smoothly embedded in the background, making the image more realistic.

For a functional description of the virtual sample generating apparatus provided by the embodiment of the present invention, refer to the description of the virtual sample generating method in the foregoing embodiment for details.

An embodiment of the present invention further provides a storage medium, as shown in FIG. 9 , on which a computer program 601 is stored, and when the instruction is executed by a processor, implements the steps of the virtual sample generation method in the foregoing embodiment. The storage medium also stores audio and video stream data, feature frame data, interaction request signaling, encrypted data, preset data size, and the like. Wherein, the storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, Abbreviation: HDD) or Solid-State Drive (Solid-State Drive, SSD), etc.; the storage medium may also include a combination of the above-mentioned types of memories.

Those skilled in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. During execution, the processes of the embodiments of the above-mentioned methods may be included. Wherein, the storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive) , abbreviation: HDD) or solid-state drive (Solid-State Drive, SSD), etc.; the storage medium may also include a combination of the above-mentioned types of memories.

An embodiment of the present invention further provides an electronic device. As shown in FIG. 10 , the electronic device may include a processor 51 and a memory 52, where the processor 51 and the memory 52 may be connected by a bus or in other ways. Take bus connection as an example.

The processor 51 may be a central processing unit (Central Processing Unit, CPU). The processor 51 may also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA) or Other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components and other chips, or a combination of the above types of chips.

As a non-transitory computer-readable storage medium, the memory 52 can be used to store non-transitory software programs, non-transitory computer-executable programs and modules, such as corresponding program instructions/modules in the embodiments of the present invention. The processor 51 executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory 52, ie, implements the virtual sample generation method in the above method embodiments.

The memory 52 may include a storage program area and a storage data area, wherein the storage program area may store an operating device, an application program required for at least one function; the storage data area may store data created by the processor 51 and the like. Additionally, memory 52 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 52 may optionally include memory located remotely from processor 51 , which may be connected to processor 51 via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The one or more modules are stored in the memory 52, and when executed by the processor 51, execute the virtual sample generation method in the embodiments shown in FIGS. 1-7.

The specific details of the above electronic device can be understood by referring to the corresponding descriptions and effects in the embodiments shown in FIG. 1 to FIG. 7 , and details are not repeated here.

Although the embodiments of the present invention have been described with reference to the accompanying drawings, various modifications and variations can be made by those skilled in the art without departing from the spirit and scope of the present invention, and such modifications and variations fall within the scope of the appended claims within the limits of the requirements.

Claims (10)

1. a virtual sample generation method, is characterized in that, comprises: Obtain background scene samples and data samples to be detected; Build a sample relationship model based on the information of the data sample; Perform data enhancement on the data samples to be detected to obtain enhanced data samples; Arrange the enhanced data samples according to the arrangement rule constructed by the sample relationship model, and then embed the background scene samples to generate a first sample; Perform style transfer on the first sample according to the adversarial network to obtain a virtual sample. 2. The virtual sample generation method according to claim 1, wherein the data enhancement method comprises: random scale jitter, data enhancement based on HSV space, image sharpness enhancement, and random flip and rotation; Perform data enhancement on the data samples to be detected, including: Data augmentation is performed randomly on the data samples to be detected for each data augmentation method according to a preset probability. 3. virtual sample generation method according to claim 1, is characterized in that, builds sample relation model according to the information of data sample, comprises: Obtain sales information for data samples; Determine the product specification information of the data sample according to the sales information; Based on the data mining algorithm, the correlation between the sales volume of each data sample brand and the sales volume of other brands is calculated according to the sales information and the product regulation information, and the data product regulation with the correlation greater than the threshold is obtained; The sample relationship model is constructed according to the quality information of the data samples and the data quality whose correlation is greater than the threshold. 4. The method for generating virtual samples according to claim 1, characterized in that, after arranging the enhanced data samples according to the arrangement rule constructed by the sample relation model, the background scene samples are embedded to generate the first sample, include: Arrange the data samples according to the random placement method, product regulation information or correlation information, and obtain the arranged data samples; Embed the arranged data samples into the background scene samples to generate a first sample. 5. The virtual sample generation method according to claim 1, wherein the loss function of the adversarial network is represented by the following formula:

in,

represents the adversarial loss, G represents the generator, D _Y represents the discriminator,

represents the cycle consistency loss. 6. The virtual sample generation method according to claim 1, wherein the style transfer is performed on the first sample according to an adversarial network to obtain a virtual sample, comprising: Cut each data sample image in the first sample and input it to the adversarial network for style transfer to obtain a migrated data sample image; Embed the migrated data sample image into the first sample to obtain a virtual sample. 7 . The virtual sample generation method according to claim 1 , wherein the data sample to be detected is a sample of the outer packaging of a cigarette pack. 8 . 8. A device for generating virtual samples, comprising: The sample acquisition module is used to acquire background scene samples and data samples to be detected; The relational model building module is used to construct the sample relational model according to the information of the data sample; The enhancement module is used to perform data enhancement on the data samples to be detected to obtain the enhanced data samples; an arrangement module, configured to arrange the enhanced data samples according to the arrangement rules constructed by the sample relationship model, and then embed the background scene samples to generate a first sample; The style transfer module is used for performing style transfer on the first sample according to the adversarial network to obtain a virtual sample. 9 . A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, and the computer instructions are used to cause the computer to execute the virtual machine according to any one of claims 1 to 7 . Sample generation method. 10. An electronic device, comprising: a memory and a processor, wherein the memory and the processor are connected in communication with each other, the memory stores computer instructions, and the processor executes the computer instructions by executing the computer instructions. , so as to execute the virtual sample generation method according to any one of claims 1-7.

Images