WO2025001765A1 - 模型构建方法、图像生成方法、装置、设备、介质 - Google Patents

模型构建方法、图像生成方法、装置、设备、介质 Download PDF

Info

Publication number
WO2025001765A1
WO2025001765A1 PCT/CN2024/097164 CN2024097164W WO2025001765A1 WO 2025001765 A1 WO2025001765 A1 WO 2025001765A1 CN 2024097164 W CN2024097164 W CN 2024097164W WO 2025001765 A1 WO2025001765 A1 WO 2025001765A1
Authority
WO
WIPO (PCT)
Prior art keywords
model
teacher
student
generator
sample image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2024/097164
Other languages
English (en)
French (fr)
Inventor
任玉羲
吴捷
张朋
肖学锋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zitiao Network Technology Co Ltd
Original Assignee
Beijing Zitiao Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zitiao Network Technology Co Ltd filed Critical Beijing Zitiao Network Technology Co Ltd
Publication of WO2025001765A1 publication Critical patent/WO2025001765A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0495—Quantised networks; Sparse networks; Compressed networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—Two-dimensional [2D] image generation
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Definitions

  • the present disclosure relates to a model building method, an image generating method, a device, a equipment, and a medium.
  • the task can be implemented with the help of pre-built models, such as the Generative adversarial networks (GAN) model.
  • GAN Generative adversarial networks
  • the present disclosure provides a model building method, an image generating method, an apparatus, a device, and a medium.
  • the present disclosure provides a model building method, the method comprising:
  • the training data set includes a first sample image and label information corresponding to the first sample image
  • Model distillation is performed based on the training data set, the teacher model, and the student model to obtain an image generation model.
  • the training process of the hypernetwork includes:
  • Model distillation is performed based on the training data set, the teacher generator and the student generator to obtain a trained super network.
  • determining the teacher generator and the student generator from the hypernetwork includes:
  • the student generator is determined.
  • the first subnetwork includes a largest subnetwork
  • the second sub-network includes at least one of a minimum sub-network and a random sub-network.
  • the training data set further includes a second sample image
  • the performing of model distillation according to the training data set, the teacher generator and the student generator to obtain a trained super network includes:
  • the hypernetwork, the teacher generator, and the student generator are updated according to the loss characterization data of the teacher generator and the loss characterization data of the student generator.
  • the method further includes:
  • the training data set further includes a second sample image
  • the performing of model distillation according to the training data set, the teacher generator and the student generator to obtain a trained super network includes:
  • the loss representation data of the student generator; the second pseudo label is obtained by processing the second sample image by the updated teacher generator;
  • the hypernetwork, the teacher generator and the student generator are updated, and the step of updating the teacher generator using the first sample image and the label information corresponding to the first sample image is continued until the second stop condition is reached.
  • searching for a teacher model and a student model from a trained hypernetwork includes:
  • At least one teacher model is searched from the trained hypernetwork so that each of the teacher models satisfies the second model constraint and the at least one teacher model satisfies a preset model architecture condition.
  • the at least one teacher model includes a teacher model that satisfies a preset model depth condition and a teacher model that satisfies a preset model width condition;
  • the second model constraint is determined based on the first model constraint so that each of the teacher model and the student model satisfies a preset multiple condition on one or more model indicators.
  • the training data set further includes a second sample image
  • the performing model distillation according to the training data set, the teacher model and the student model to obtain an image generation model includes:
  • the student model is updated using the second sample image and the third pseudo label corresponding to the second sample image, and the step of updating the teacher model using the first sample image and the label information corresponding to the first sample image is continued until the image generation model is determined according to the student model when the third stop condition is reached; the third pseudo label is obtained by processing the second sample image by the updated teacher model.
  • the updating the student model by using the second sample image and the third pseudo label corresponding to the second sample image includes:
  • the student model is updated using the first sample image, the fourth pseudo label corresponding to the first sample image, the second sample image, and the third pseudo label corresponding to the second sample image; the fourth pseudo label is obtained by processing the first sample image with the updated teacher model.
  • the updating process of the student model includes:
  • the student model is updated according to the loss characterization data of the student model.
  • determining the loss characterization data of the student model according to the first distillation loss includes:
  • the process of determining the weighted weight includes:
  • a preset value is determined as the weighted weight.
  • the process of determining the confidence threshold includes:
  • Data analysis is performed on at least one of the discrimination results corresponding to the generated image to obtain the confidence threshold.
  • the present disclosure provides an image generation method, the method comprising:
  • the image to be processed is processed using a pre-constructed image generation model to obtain an image generation result; the image generation model is constructed using the model construction method provided in the present disclosure.
  • the present disclosure provides a model building device, comprising:
  • a first acquisition unit is used to acquire a training data set;
  • the training data set includes a first sample image and label information corresponding to the first sample image;
  • a first training unit used to train the super network using the training data set
  • a network search unit used to search for a teacher model and a student model from a trained hypernetwork
  • the second training unit is used to perform model distillation based on the training data set, the teacher model and the student model to obtain an image generation model.
  • the present disclosure provides an image generating device, comprising:
  • a second acquisition unit used for acquiring an image to be processed
  • the image generation unit is used to process the image to be processed using a pre-constructed image generation model to obtain an image generation result; the image generation model is constructed using the model construction method provided by the present disclosure.
  • the present disclosure provides an electronic device, the device comprising: a processor and a memory;
  • the memory is used to store instructions or computer programs
  • the processor is used to execute the instructions or computer programs in the memory so that the electronic device executes the model building method or image generating method provided by the present disclosure.
  • the present disclosure provides a computer-readable medium, in which instructions or computer programs are stored.
  • the device executes the model building method or image generating method provided by the present disclosure.
  • the present disclosure provides a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program codes for executing the model building method or image generating method provided by the present disclosure.
  • FIG1 is a flow chart of a model building method provided by an embodiment of the present disclosure.
  • FIG2 is a schematic diagram of a model building process provided by an embodiment of the present disclosure.
  • FIG3 is a flow chart of an image generation method provided by an embodiment of the present disclosure.
  • FIG4 is a schematic diagram of the structure of a model building device provided in an embodiment of the present disclosure.
  • FIG5 is a schematic diagram of the structure of an image generating device provided by an embodiment of the present disclosure.
  • FIG. 6 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure.
  • the model building method provided by the present disclosure includes the following S101-S104.
  • Figure 1 is a flow chart of a model building method provided by an embodiment of the present disclosure.
  • S101 Acquire a training data set; the training data set includes a first sample image and label information corresponding to the first sample image.
  • the training data set refers to the data set required to be used in the model training process (for example, the data set composed of the labeled training data and the unlabeled training data shown in FIG1 ).
  • the present disclosure is not limited to the labeled training data in the above paragraph.
  • the Represents a set of labeled training data ⁇ xi , yi ⁇ represents the ith labeled training data, i is a positive integer, 1 ⁇ i ⁇ M, M is a positive integer, and M represents the number of labeled training data;
  • xi represents the image data in the ith labeled training data;
  • yi represents the label information corresponding toxi , and the label information can be obtained by manual annotation or by other methods, and the present disclosure does not make specific limitations on this.
  • the present disclosure does not limit the expression ofyi , for example, it can be expressed by image data.
  • yi may refer to the real depth map ofxi .
  • yi in the field of face recognition of images, yi may refer to the real face ofxi.
  • the face region label map is used to make yi represent the actual region of the face region in xi .
  • yi in the field of image style change, yi may refer to an image in a target image domain, and the domain style of the target image domain is different from the data domain style of the image data xi , and the present disclosure does not limit the target image domain.
  • it may be an image domain with a watercolor style, an image domain with a cartoon style, an image domain with a stick figure style, or an image domain with a black and white style.
  • the present disclosure is not limited to the unlabeled training data mentioned above.
  • the represents a set of unlabeled training data
  • z k represents the kth unlabeled training data
  • k is a positive integer
  • 1 ⁇ k ⁇ NM 1 ⁇ k ⁇ NM
  • N is a positive integer
  • N represents the number of training data in the above training data set.
  • the present disclosure does not limit the implementation of the above training data set.
  • the training data set may include a first sample image (e.g., x i ) and label information corresponding to the first sample image (e.g., y i ); wherein the first sample image refers to image data with label information required for use in the model training process; the label information corresponding to the first sample image refers to a label pre-annotated for the first sample image.
  • the present disclosure does not limit the number of the first sample images.
  • some labeled training data and some unlabeled training data can be used for model training, so that the model can not only learn knowledge from these labeled training data, but also learn structural features under a certain distribution domain from these unlabeled training data.
  • the present disclosure also provides a possible implementation of the above training data set, in which the training data set may include not only a first sample image and label information corresponding to the first sample image, but also a second sample image (for example, z k ).
  • the second sample image refers to image data without label information required for use in the model training process. It should be noted that the present disclosure does not limit the number of the second sample images.
  • the training data set mentioned above not only includes some labeled training data, but also some unlabeled training data, so that these unlabeled training data can bring some additional supervision information to the model training process, thereby enhancing the generalization of the model, which is beneficial for the model to learn the structural features under a certain distribution domain, so as to increase the amount of training data with as little labeling cost as possible.
  • the present disclosure does not limit the method for obtaining the above training data set.
  • the hypernetwork is used to represent a search space (e.g., the search space shown in FIG. 2 ), so that the hypernetwork can provide some subnetworks with different network structures (e.g., some subnetworks with different depths and widths, etc.).
  • the Gt represents a sub-network sampled from the search space (ie, the super-network), t is a positive integer, and t ⁇ the total number of sub-networks that the super-network can provide.
  • the present disclosure provides a possible implementation of the above hypernetwork, in which the hypernetwork has dynamically adjustable depth and width, so that the depth and width of the hypernetwork can be dynamically adjusted along with the training process of the hypernetwork.
  • the present disclosure does not limit the implementation of S102 above.
  • the training data set above only includes labeled training data
  • any existing or future method that can perform supervised training on a hypernetwork can be used for implementation.
  • the training data set above includes some labeled training data and some unlabeled training data
  • any existing or future method that can perform semi-supervised training on a hypernetwork can be used for implementation.
  • a distillation algorithm (such as an online distillation algorithm) can be used to implement the training process for the hypernetwork.
  • the present disclosure also provides a possible implementation of the above S102, which can specifically include the following steps 11-12.
  • Step 11 Determine the teacher generator and student generator from the hypernetwork.
  • the teacher generator refers to a sub-network with a larger network size sampled from a hypernetwork; and the teacher generator has an image generation function (e.g., image generation function, etc.); in addition, the present disclosure does not limit the teacher generator, for example, it can be the largest sub-network shown in Figure 2.
  • the network size is used to describe the size of a network in at least one dimension (e.g., depth and/or width) so that the network size can characterize the size of the network. It should be noted that the present disclosure does not limit the implementation method of "at least one dimension", for example, it can include depth and/or width.
  • the student generator refers to another or more sub-networks sampled from the super network; and the student generator has an image generation function (for example, an image generation function, etc.); in addition, the network size of the student generator can be smaller than the network size of the above-mentioned teacher generator, so that the final trained student generator can represent the compression result for the teacher generator.
  • image generation function for example, an image generation function, etc.
  • the present disclosure does not limit the implementation of the above student generator.
  • the student generator It can be the random subnetwork shown in Figure 2.
  • the student generator can include the minimum subnetwork and the random subnetwork shown in Figure 2.
  • the random subnetwork refers to the subnetwork randomly sampled from the supernetwork.
  • the minimum subnetwork is a subnetwork sampled from the supernetwork according to a pre-set minimum subnetwork sampling rule. It should be noted that the present disclosure does not limit the minimum subnetwork sampling rule.
  • the minimum subnetwork sampling rule can be set in advance according to the application scenario.
  • the present disclosure does not limit the implementation method of the above step 11.
  • it can be implemented by any existing or future method that can extract some sub-networks with different network sizes from the super network.
  • the present disclosure also provides a possible implementation of the above step 11, which may specifically include the following steps 111 to 113.
  • Step 111 sampling a first sub-network and a second sub-network from a super-network; the network size of the first sub-network is larger than the network size of the second sub-network.
  • the first subnetwork refers to a subnetwork sampled from the supernetwork and having a larger network size; and the present disclosure does not limit the first subnetwork.
  • the first subnetwork may include the largest subnetwork sampled from the supernetwork.
  • the second subnetwork refers to a subnetwork sampled from the supernetwork and having a smaller network size; and the present disclosure does not limit the second subnetwork.
  • the second subnetwork may include at least one of a minimum subnetwork and a random subnetwork sampled from the supernetwork. It can be seen that in a possible implementation, the second subnetwork may include a minimum subnetwork and one or more random subnetworks.
  • Step 112 Determine the teacher generator based on the first sub-network above.
  • step 112. can be specifically: after obtaining the first sub-network above, the first sub-network can be directly determined as the teacher generator. It can be seen that in one possible implementation, the teacher generator can refer to the largest sub-network sampled from the hypernetwork.
  • Step 113 Determine the student generator based on the second sub-network above.
  • step 113 does not limit the implementation of step 113.
  • the second sub-network can be directly determined as the student generator.
  • the student generator can include a minimum sub-network sampled from the super-network and one or more random sub-networks.
  • step 11 Based on the relevant content of step 11 above, it can be known that after obtaining the hypernetwork used to represent the search space, a teacher generator and a student generator can be sampled from the hypernetwork so that model distillation can be performed based on the teacher generator and the student generator to obtain a trained hypernetwork.
  • Step 12 Perform model distillation based on the training dataset, teacher generator, and student generator to obtain a trained hypernetwork.
  • step 12 can be implemented by any existing or future model distillation method (such as an online distillation algorithm, etc.). It can be seen that in a possible implementation, when the above training data set includes some labeled training data, step 12 can specifically be: using these labeled training data, the teacher generator and the student generator to perform model distillation to obtain a trained hypernetwork.
  • step 12 when the training data set above includes a first sample image, label information corresponding to the first sample image, and a second sample image, step 12 can specifically include the following steps 121-123.
  • Step 121 Determine loss representation data of the teacher generator using the first sample image and the label information corresponding to the first sample image.
  • the loss characterization data of the teacher generator is used to characterize the image generation performance of the teacher generator (for example, image generation performance); and the present disclosure does not limit the determination process of the loss characterization data of the teacher generator.
  • it can be implemented using any existing or future method that can determine the loss presented by the teacher generator on labeled training data.
  • the present disclosure also provides a possible implementation method of the process of determining the loss characterization data of the above teacher generator.
  • the process of determining the loss characterization data of the teacher generator may specifically include the following steps 1211-1213.
  • Step 1211 Determine the adversarial loss corresponding to the teacher generator by using the first sample image, the label information corresponding to the first sample image, the teacher generator, and the discriminator corresponding to the teacher generator.
  • the discriminator corresponding to the teacher generator refers to the discriminator required to evaluate the performance of the teacher generator; and the discriminator corresponding to the teacher generator can be combined with the teacher generator to form a into a generative adversarial network.
  • the adversarial loss corresponding to the teacher generator refers to the loss presented by the generative adversarial network composed of the teacher generator and the discriminator corresponding to the teacher generator.
  • the present disclosure does not limit the determination process of the adversarial loss corresponding to the above teacher generator.
  • it can be implemented by using any existing or future method for determining the adversarial loss of a GAN (for example, the determination method shown in formula (1) below).
  • L GAN ( GT , D) represents the adversarial loss corresponding to the teacher generator above; GT represents the teacher generator; D represents the discriminator corresponding to the teacher generator; x represents sample image data (for example, the first sample image above); y represents the label information corresponding to the sample image data (for example, the label information corresponding to the first sample image); GT (x) represents the image data output by the teacher generator GT after the sample image data is input into the teacher generator GT ; represents the expected function under the training data ⁇ x, y ⁇ ; Represents the expected function under the sample image data.
  • step 1211 After obtaining the teacher generator GT , we can use some labeled training data
  • the teacher generator GT and the discriminator D corresponding to the teacher generator GT determine the adversarial loss corresponding to the teacher generator GT , so that the teacher generator GT and the discriminator D corresponding to the teacher generator GT can be optimized based on the adversarial loss, so that the final trained teacher generator GT can achieve the goal of mapping xi to yi , and the final trained discriminator D can distinguish the generated image output by the teacher generator GT from the real image.
  • Step 1212 Using the first sample image, the label information corresponding to the first sample image, and the teacher generator, determine the reconstruction loss corresponding to the teacher generator.
  • the reconstruction loss corresponding to the teacher generator is used to characterize the image generation performance of the teacher generator, so that the output image of the teacher generator can be made close to the true label with the help of the reconstruction loss.
  • the present disclosure does not limit the process of determining the reconstruction loss corresponding to the above-mentioned teacher generator.
  • it can be implemented by adopting a method for determining the reconstruction loss of any existing or future image generator (for example, the determination method shown in formula (2) below).
  • L recon (G T ) represents the reconstruction loss corresponding to the teacher generator;
  • G T represents the teacher generator;
  • x represents sample image data (e.g., the first sample image above);
  • y represents label information corresponding to the sample image data (e.g., label information corresponding to the first sample image);
  • GT (x) represents image data output by the teacher generator GT after the sample image data is input into the teacher generator GT ;
  • step 1212 above Based on the relevant content of step 1212 above, it can be known that when the label information corresponding to the first sample image above is image data, and the teacher generator above belongs to the generator in GAN, the first sample image above can be input into the teacher generator to obtain the generated image output by the teacher generator; and then based on the degree of similarity between the generated image and the label information corresponding to the first sample image, the reconstruction loss corresponding to the teacher generator is determined, so that the reconstruction loss can represent the image generation performance of the teacher generator, so that the teacher generator can be optimized based on the reconstruction loss subsequently, so that the optimized teacher generator has better image generation performance.
  • Step 1213 Based on the adversarial loss corresponding to the above teacher generator and the reconstruction loss corresponding to the teacher generator, determine the loss representation data of the teacher generator (for example, the supervision loss 1 shown in Figure 2).
  • the loss representation data of the teacher generator can be determined based on the two losses (as shown in the determination method of formula (3) below), so that the loss representation data can more comprehensively and accurately represent the image generation performance presented by the teacher generator.
  • L GAN ( GT , D) represents the adversarial loss corresponding to the teacher generator above
  • GT represents the teacher generator
  • D represents the discriminator corresponding to the teacher generator
  • Lrecon ( GT ) represents the reconstruction loss corresponding to the teacher generator.
  • some labeled training data for example, the first sample image above and the label information corresponding to the first sample image
  • the loss that is, the supervision loss
  • Step 122 Determine loss representation data of the student generator using the second sample image and a first pseudo label corresponding to the second sample image; the first pseudo label is determined by the teacher generator through processing the second sample image.
  • the first pseudo label corresponding to the second sample image refers to the pseudo label generated by the teacher generator above for the first sample image.
  • the processing result of the two sample images is obtained so that the processing result can be used as the pseudo label of the second sample image to guide the optimization process of the student generator. In this way, the purpose of the teacher generator guiding the student generator can be achieved, thereby completing the optimization processing for the student generator without relying on the discriminator.
  • the loss characterization data of the student generator is used to characterize the image generation performance of the student generator (for example, image generation performance); and the present disclosure does not limit the determination process of the loss characterization data of the student generator. For example, it can be implemented using any existing or future method that can calculate the loss for the student generator (for example, online distillation loss, etc.).
  • the student generator above may include at least two sub-networks sampled from the super network. Based on this, the present disclosure also provides a possible implementation of step 122 above. In this implementation, when the student generator includes a minimum sub-network and a random sub-network, step 122 may specifically include the following steps 1221-1223.
  • step 1221 does not limit the implementation of step 1221.
  • it can be implemented by any existing or future method that can calculate the online distillation loss of a model.
  • Step 1222 Determine the online distillation loss of the random sub-network using the second sample image and the first pseudo-label corresponding to the second sample image, so that the online distillation loss can represent the image generation performance of the random sub-network.
  • Step 1223 Determine the loss representation data of the student generator based on the online distillation loss of the minimum sub-network and the online distillation loss of the random sub-network.
  • the loss characterization data of the student generator can be determined based on these two losses (for example, the following formula can be used): Formula (4) is used to implement it).
  • Equation (4) represents the loss representation data of the student generator; Gr represents the random subnetwork; Gs represents the minimum subnetwork; LOD ( GT ( xu ), Gr ( xu )) represents the online distillation loss of the random subnetwork; xu represents unlabeled training data (e.g., the second sample image); GT ( xu ) represents the image data generated when xu is processed by the above teacher generator (e.g., the first pseudo label corresponding to the second sample image); Gr ( xu ) represents the image data generated when xu is processed by the random subnetwork; LOD ( GT ( xu ), Gs ( xu )) represents the online distillation loss of the minimum subnetwork; Gs ( xu ) represents the image data generated when xu is processed by the minimum subnetwork.
  • step 1221 to 1223 Based on the relevant contents of steps 1221 to 1223 above, it can be known that in some application scenarios, if the student generator above includes multiple sub-networks, then under the guidance of the teacher generator above, the online distillation loss presented by each sub-network on these unlabeled training data can be first determined; and then the online distillation losses corresponding to all sub-networks are added together to obtain the loss representation data of the student generator, so that the loss representation data can better represent the image generation performance of the student generator.
  • Step 123 Update the hypernetwork, the teacher generator, and the student generator according to the loss characterization data of the teacher generator and the loss characterization data of the student generator.
  • the hypernetwork, teacher generator and student generator can be updated based on these two losses, so that the updated hypernetwork, updated teacher generator and updated student generator all have better image generation performance.
  • the present disclosure does not limit the implementation of the above step 123.
  • the step 123 may specifically include the following steps 1231 to 1233.
  • Step 1231 Determine the loss characterization data of the hypernetwork based on the loss characterization data of the teacher generator and the loss characterization data of the student generator, so that the loss characterization data can better represent the image generation performance of the hypernetwork.
  • step 1231 does not limit the implementation of step 1231.
  • it can be implemented using the following formula (5).
  • L hypernetwork represents the loss representation data of the hypernetwork; Show teaching Loss characterization data for the teacher generator; Represents the loss representation data of the student generator.
  • Step 1232 Update the hypernetwork according to the loss characterization data of the hypernetwork.
  • the hypernetwork after obtaining the loss characterization data of the hypernetwork, the hypernetwork can be updated (especially, the depth and/or width of the hypernetwork is updated) based on the loss characterization data of the hypernetwork, so that the updated hypernetwork has better image generation performance. It should be noted that the present disclosure does not limit the update method of the hypernetwork.
  • Step 1233 Determine the teacher generator and the student generator from the updated hypernetwork to obtain an updated teacher generator and an updated student generator.
  • some subnetworks e.g., the maximum subnetwork, the minimum subnetwork, and the random subnetwork
  • an updated teacher generator and an updated student generator can be determined (e.g., the maximum subnetwork is determined as the updated teacher generator, and the minimum subnetwork and the random subnetwork are determined as the updated student generators).
  • the loss characterization data of the above hypernetwork can be determined based on these two losses, so that the loss characterization data can better represent the image generation performance of the hypernetwork; and then based on the loss characterization data of the hypernetwork, the hypernetwork, the teacher generator and the student generator are updated, so that the updated hypernetwork, the updated teacher generator and the updated student generator all have better image generation performance.
  • the super network can be trained with the help of a semi-supervised method, and the process is roughly as follows: determine the loss of the largest subnetwork in the super network on some labeled training data sets, and use the largest subnetwork to guide the determination of the losses of some other subnetworks (such as the minimum subnetwork and random subnetwork, etc.) in the super network on some unlabeled training data sets, so that the super network can be optimized based on these two losses in the future. In this way, the super network can be optimized with the help of the mutual assistance between some subnetworks with relatively small network sizes and the largest subnetwork.
  • step 12 when the training data set above includes a first sample image, label information corresponding to the first sample image, and a second sample image, step 12 not only The process includes the above steps 121 to 123 and may also include the following step 124.
  • Step 124 determine whether the first stop condition is met, if so, end the training process for the hypernetwork; if not, return to continue to execute the above step 121 and subsequent steps.
  • the first stop condition refers to the condition that needs to be achieved when the training process of the hypernetwork ends; and the present disclosure does not limit the first stop condition, for example, it can be that the loss characterization data of the hypernetwork is lower than the first threshold, or the change rate of the loss characterization data of the hypernetwork is lower than the second threshold, or the number of updates of the hypernetwork reaches the third threshold.
  • the first threshold, the second threshold and the third threshold can all be preset.
  • the present disclosure does not limit the execution time of the above step 124.
  • the execution time of the above step 124 may be later than the execution time of the above step 123.
  • the super network can be trained with the help of a semi-supervised + iterative method.
  • the super network can be continuously optimized with the help of the mutual assistance + progressive method between some sub-networks with relatively small network sizes and the largest sub-network in the super network, so that the trained super network has better image generation performance, and each sub-network in the trained super network also has better image generation performance.
  • step 12 when the training data set above includes a first sample image, label information corresponding to the first sample image, and a second sample image, step 12 can specifically include the following steps 21 to 24.
  • Step 21 Update the teacher generator using the first sample image and the label information corresponding to the first sample image.
  • step 21 does not limit the implementation method of step 21.
  • it can be specifically as follows: first, use the first sample image and the label information corresponding to the first sample image to determine the loss characterization data of the teacher generator; then, based on the loss characterization data of the teacher generator, update the teacher generator so that the updated teacher generator has better performance.
  • the hypernetwork can be updated along with the update of the teacher generator.
  • the present disclosure does not limit the implementation of step 21 above.
  • it can be specifically: using the first sample image and the label information corresponding to the first sample image to update the teacher generator and the hypernetwork. That is, in a possible implementation, the first sample image can be first used.
  • the label information corresponding to the first sample image is used to determine the loss representation data of the teacher generator; then the hypernetwork is updated based on the loss representation data of the teacher generator so that the updated model has better performance; then, a subnetwork (for example, the largest subnetwork) is sampled from the updated hypernetwork and determined as the updated teacher generator, so that the hypernetwork can be updated while the teacher generator is updated.
  • a subnetwork for example, the largest subnetwork
  • Step 22 Determine the loss representation data of the student generator using the second sample image and the second pseudo label corresponding to the second sample image; the second pseudo label is obtained by processing the second sample image by the updated teacher generator.
  • the second pseudo-label corresponding to the second sample image refers to the processing result of the second sample image by the updated teacher generator, so that the processing result can be used as the pseudo-label of the second sample image to guide the optimization process of the student generator.
  • the purpose of guiding the student generator by the updated teacher generator can be achieved, thereby completing the optimization processing of the student generator without relying on the discriminator.
  • step 22 is similar to the implementation of step 122 above, and for the sake of brevity, it will not be repeated here.
  • Step 23 Determine whether the second stop condition is met. If so, end the training process for the hypernetwork; if not, update the hypernetwork, teacher generator and student generator according to the loss representation data of the student generator, and return to continue executing step 21 and its subsequent steps.
  • the second stop condition refers to the condition that needs to be achieved when the training process of the hypernetwork ends; and the present disclosure does not limit the second stop condition, for example, it can be that the loss characterization data of the hypernetwork is lower than the first threshold, or the change rate of the loss characterization data of the hypernetwork is lower than the second threshold, or the number of updates of the hypernetwork reaches the third threshold.
  • the first threshold, the second threshold and the third threshold can all be preset.
  • the present disclosure does not limit the implementation method of the above step "updating the super network, teacher generator and student generator according to the loss characterization data of the student generator".
  • the update process can specifically include: first updating the super network according to the loss characterization data of the student generator; then sampling some sub-networks (for example, the maximum sub-network, the minimum sub-network and the random sub-network) from the updated super network, and determining the updated teacher generator and the updated student generator based on these sub-networks.
  • another semi-supervised method can be used to train the super network.
  • the process is roughly as follows: first, the largest subnetwork in the super network is updated on some labeled training data sets, and then the updated largest subnetwork is used to guide the training process of some other subnetworks in the super network (for example, the minimum subnetwork and random subnetwork, etc.) on some unlabeled training data sets, so that these other subnetworks can better absorb knowledge from these unlabeled training data and further optimize the super network through the weight sharing mechanism.
  • the super network can be optimized by means of this alternating update method between some subnetworks with relatively small network sizes and the largest subnetwork.
  • supervised training can be performed on the largest subnetwork in the supernetwork, and under the guidance of the largest subnetwork, the training process of some other subnetworks in the supernetwork (for example, the smallest subnetwork and the random subnetwork, etc.) on some unlabeled training data can be completed, so that these other subnetworks can absorb knowledge from these unlabeled training data.
  • the supernetwork can be continuously optimized by means of the mutual assistance strategy and progressive collaboration between some subnetworks with smaller network sizes in the supernetwork and the largest subnetwork, so that the trained supernetwork has better image generation performance, and thus each subnetwork in the trained supernetwork also has better image generation performance.
  • the teacher model refers to a sub-network searched from a trained super-network so that the teacher model can guide the training process of the student model.
  • the student model refers to another sub-network searched from the trained super-network so that the structure of the student model is simpler than that of the teacher model.
  • the present disclosure does not limit the implementation of the above S103.
  • it may specifically include the following steps 31 and 32.
  • Step 31 According to the first model constraint, searching for a student model from the trained hypernetwork so that the student model satisfies the first model constraint.
  • the first model constraint refers to the constraint information required to search for the student model from the trained hypernetwork; and the first model constraint can be constraint information set in advance based on the application scenario (for example, in advance based on the configuration information of the terminal device deployed by the image generation model below, etc.), for example, it can be that the model running time is lower than a pre-set fourth threshold.
  • step 31 it can be specifically: according to the first model constraint, using the evolutionary search algorithm to search for a student model from the trained hypernetwork so that the student model satisfies the first model constraint. It should be noted that the present disclosure does not limit the implementation of the evolutionary search algorithm.
  • Step 32 According to the second model constraint, search for at least one teacher model from the trained hypernetwork so that each teacher model satisfies the second model constraint, and the at least one teacher model satisfies the preset model architecture condition.
  • the second model constraint refers to the constraint information required to search for the teacher model from the trained hypernetwork; and the second model constraint can be the constraint information set in advance according to the application scenario, for example, it can be that the model running time is lower than the pre-set fifth threshold.
  • the present disclosure also provides a determination process of the second model constraint above, for example, it can be specifically: according to the first model constraint above, the second model constraint is determined so that the teacher model satisfying the second model constraint and the student model satisfying the first model constraint meet the preset multiple conditions on one or more model indicators (for example, the computational cost of the teacher model is approximately 21 times the computational cost of the student model, etc.).
  • the model indicator can be pre-set, for example, it can be the computational cost.
  • the preset multiple condition can be determined based on the actual application scenario.
  • "at least one teacher model” is used to represent the teacher model that is searched from the trained hypernetwork according to the second model constraint and is used to guide the training process of the student model; and the at least one teacher model satisfies the preset model architecture conditions so that the at least one teacher model can guide the training process of the student model in at least one dimension (for example, depth + width and other dimensions).
  • the "at least one dimension” refers to one or more dimensions that are pre-set for the student model and need to be guided by the teacher model; and the present disclosure does not limit the "at least one dimension", for example, it may include depth and/or width.
  • the teacher model for model depth guidance and the teacher model for model width guidance can be used to participate in the training process for the student model so that the trained student model has better performance.
  • the preset model architecture condition can be set according to the application scenario, and the present disclosure does not limit the preset model architecture condition.
  • the model architecture of at least one teacher model that meets the preset model architecture condition can present a complementary state.
  • the model architecture condition may include a preset model depth condition and a preset model width condition, so that at least one teacher model that satisfies the preset model architecture condition includes a teacher model that satisfies the preset model depth condition (for example, the above "teacher model for model depth guidance") and a teacher model that satisfies the preset model width condition (for example, the above "teacher model for model width guidance”), so that the at least one teacher model presents an architectural complementary state.
  • the preset model depth condition refers to the condition required to be based on when searching for a teacher model for guiding the training process of a student model under the model depth; and the present disclosure does not limit the preset model depth condition.
  • the preset model depth condition can be set in advance according to the actual application scenario, so that the teacher model that satisfies the preset model depth condition can be used later to guide the training process of the student model under the dimension of model depth.
  • the preset model width condition refers to the condition required when searching for a teacher model for guiding the training process of the student model under the model width; and the present disclosure does not limit the preset model width condition.
  • the preset model width condition can be set in advance based on the actual application scenario, so that the teacher model that meets the preset model width condition can be used later to guide the training process of the student model under the dimension of model width.
  • an evolutionary search algorithm can be used to search out a depth-based teacher model, a width-based teacher model, and a student model from the super network, so that the depth-based teacher model can be used to guide the optimization adjustment of the student model in depth, and the width-based teacher model can be used to guide the optimization adjustment of the student model in width.
  • the trained super network has good image generation performance
  • each sub-network in the super network also has good image generation performance, so that the two teacher models and the student model searched from the super network have good performance, so that the adverse effects caused by optimizing the model from scratch (for example, the model convergence speed is relatively slow), so that the model training effect can be effectively improved (for example, the model convergence speed is accelerated).
  • the depth-based teacher model refers to the teacher model that meets the preset model depth conditions above, so that the depth-based teacher model is used to guide the training process of the student model under the dimension of model depth.
  • the width-based teacher model refers to the teacher model that meets the preset model depth conditions above, so that the width-based teacher model is used to guide the training process of the student model under the dimension of model width.
  • the trained hypernetwork can provide subnetworks that meet different constraints
  • the present disclosure can use the hypernetwork to determine various teacher models and student models that meet different constraints, so that It can effectively avoid the adverse effects (such as long time consumption, etc.) caused by manually designing multiple teacher models and student models, thereby helping to improve the model building effect.
  • S104 Perform model distillation based on the training data set, the teacher model, and the student model to obtain an image generation model.
  • the image generation model is used to generate images for the input data of the image generation model; and the present disclosure does not limit the implementation method of the image generation model.
  • the image generation model belongs to an image generator, so that the image generation model can generate a new image data based on the input data of the image generation model.
  • the present disclosure does not limit the implementation method of the model distillation in S104.
  • it can be implemented using any existing or future model distillation method (eg, an online distillation method).
  • the present disclosure also provides a possible implementation of the above S104.
  • the S104 may specifically include the following steps 41-45.
  • Step 41 Update the teacher model using the first sample image and the label information corresponding to the first sample image.
  • step 41 does not limit the implementation method of step 41.
  • it can be specifically: first use the first sample image and the label information corresponding to the first sample image to determine the loss representation data of the teacher model (for example, the supervision loss 2 shown in Figure 2); then update the teacher model based on the loss representation data of the teacher model, so that the updated teacher model has better performance.
  • the loss representation data of the teacher model for example, the supervision loss 2 shown in Figure 2
  • Step 42 Update the student model using the second sample image and the third pseudo label corresponding to the second sample image; the third pseudo label is obtained by processing the second sample image by the updated teacher model.
  • the third pseudo-label corresponding to the second sample image refers to the processing result of the second sample image by the updated teacher model, so that the processing result can be used as the pseudo-label of the second sample image to guide the optimization process of the student model.
  • the purpose of guiding the student model by the updated teacher model can be achieved, thereby completing the optimization processing of the student model without relying on the discriminator.
  • step 42 can adopt existing or The student model update process involved in any model distillation method (for example, an online distillation method) that appears in the future is implemented. It can be seen that under one possible implementation, step 42 can specifically be: first use the second sample image and the third pseudo-label corresponding to the second sample image to determine the loss representation data of the student model; then update the student model based on the loss representation data of the student model. It should be noted that the process of determining the loss representation data of the student model is similar to the process of determining the loss representation data of the student generator mentioned above. For the sake of brevity, it will not be repeated here.
  • step 42 can be specifically: using the first sample image, the fourth pseudo label corresponding to the first sample image, the second sample image and the third pseudo label corresponding to the second sample image to update the student model, which can effectively increase the number of training data of the student model, thereby helping to improve the training effect of the student model; the fourth pseudo label is obtained by processing the first sample image by the updated teacher model.
  • the present disclosure does not limit the updating process of the above student model, for example, it may specifically include the following steps 421 to 423.
  • Step 421 Determine a first distillation loss of the student model using the second sample image and the third pseudo label corresponding to the second sample image.
  • the first distillation loss is used to characterize the loss presented by the student model on unlabeled training data.
  • the present disclosure does not limit the determination process of the first distillation loss.
  • the first distillation loss can be implemented using the following formula (6).
  • formula (6) represents the first distillation loss of the student model;
  • G * represents the student model; represents the online distillation loss presented by the student model on the unlabeled training data under the guidance of the depth-based teacher model; represents the online distillation loss of the student model on the unlabeled training data under the guidance of the width-based teacher model; represents unlabeled training data; represents the image data generated when the depth-based teacher model processes the x u ; represents the image data generated when the student model processes x u ; represents the value generated by the width-based teacher model when processing x u image data.
  • Step 422 Determine loss characterization data of the student model according to the first distillation loss of the student model.
  • step 422. it may specifically be: directly determining the first distillation loss of the student model as the loss representation data of the student model.
  • the process of determining the loss representation data of the student model may include the following steps 51 and 52.
  • Step 51 Determine a second distillation loss of the student model using the first sample image and a fourth pseudo label corresponding to the first sample image.
  • the fourth pseudo-label corresponding to the first sample image refers to the processing result of the first sample image by the updated teacher model, so that the processing result can be used as the pseudo-label of the first sample image to guide the optimization process of the student model.
  • the purpose of guiding the student model by the updated teacher model can be achieved, thereby completing the optimization processing of the student model without relying on the discriminator.
  • the second distillation loss is used to characterize the loss exhibited by the student model on labeled training data.
  • the present disclosure does not limit the determination process of the second distillation loss.
  • the second distillation loss can be implemented using the following formula (7).
  • formula (7) represents the second distillation loss of the student model
  • G * represents the student model
  • Step 52 Determine loss characterization data of the student model according to the first distillation loss of the student model and the second distillation loss of the student model.
  • the present disclosure does not limit the implementation method of the above step 52.
  • it may specifically be: adding the first distillation loss of the student model and the second distillation loss of the student model to obtain the loss characterization data of the student model.
  • the distillation loss presented by the student model in the unlabeled training data and the distillation loss presented by the student model in the labeled training data can be first determined; and then the two losses are combined to determine the loss representation data of the student model, so that the loss representation data can better represent the performance of the student model.
  • step 422 can be specifically: determining the loss characterization data of the student model based on the first distillation loss of the student model and the weighted weight corresponding to the first distillation loss. Among them, the weighted weight is determined according to the confidence of the third pseudo label; the confidence of the third pseudo label is determined based on the updated teacher model, the discriminator corresponding to the updated teacher model, and the second sample image.
  • the confidence of the third pseudo-label is used to characterize the credibility of the third pseudo-label; and the present disclosure does not limit the determination process of the confidence of the third pseudo-label.
  • it can be specifically: first input the second sample image into the updated teacher model to obtain the third pseudo-label output by the teacher model; then the discriminator corresponding to the updated teacher model performs a discriminant process on the third pseudo-label to obtain a discriminant result of the third pseudo-label, and determines the discriminant result as the confidence of the third pseudo-label.
  • the weighted weight corresponding to the first distillation loss is used to characterize the influence degree of the first distillation loss.
  • the present disclosure does not limit the determination process of the weighted weight corresponding to the first distillation loss. For example, it may specifically include the following steps 61 and 62.
  • Step 61 If the confidence of the third pseudo-label is lower than the confidence threshold, a value is randomly selected from a preset interval and determined as the weighted weight corresponding to the first distillation loss above.
  • the confidence threshold is used to describe the boundary between credible and uncredible.
  • the confidence threshold may be determined in advance, for example, according to an application scenario.
  • the present disclosure also provides a method for automatically determining a confidence threshold, which may specifically include the following steps 71 to 73.
  • Step 71 Process at least one image data using the updated teacher model to obtain a generated image corresponding to each image data; the at least one image data includes a second sample image and/or a first sample image.
  • At least one image data refers to image data required for reference when automatically determining the confidence threshold; and the present disclosure does not limit the implementation method of the at least one image data, for example, it may include the second sample image and/or the first sample image mentioned above.
  • Step 72 Use the discriminator corresponding to the updated teacher model to perform discriminative processing on each generated image to obtain a discriminative result corresponding to each generated image.
  • the discriminator corresponding to the updated teacher model refers to the discriminator involved in the update process of the teacher model.
  • Step 73 Perform data analysis and processing on the discrimination result corresponding to at least one generated image to obtain a confidence threshold.
  • data analysis processing refers to a pre-set data analysis method; and the present disclosure does not limit the implementation method of the data analysis processing.
  • it can be implemented by any existing or future data analysis method (for example, maximum value, minimum value, mean, median, or exponential moving average, etc.).
  • the confidence threshold can refer to the result obtained by performing an exponential moving average (EMA) analysis on the discrimination results corresponding to all generated images.
  • EMA exponential moving average
  • the teacher model can be used to process each sample image in the training data set to obtain the generated image corresponding to each sample image; then the discriminator corresponding to the teacher model is used to perform discrimination processing on each generated image to obtain the discrimination results of each generated image; finally, all these discrimination results are analyzed to obtain a confidence threshold, so that the confidence threshold can better describe the boundary between credible and unreliable.
  • the preset interval refers to a preset value selection interval, so that the above “randomly select a value in the preset interval" is greater than or equal to the minimum value in the preset interval, and the "randomly select a value in the preset interval” is less than or equal to the maximum value in the preset interval; and the present disclosure does not limit
  • the preset interval for example, any value in the preset interval is less than or equal to a preset first value (e.g., 1, etc.).
  • a preset second value e.g., 0, etc.
  • the preset interval can be a data interval of [second value, first value] (e.g., a data interval of [0, 1]).
  • a value can be randomly selected from a preset interval (for example, a value is randomly selected between 0 and 1) as the weighted weight corresponding to the third pseudo-label, so that the weighted weight can represent the impact of the loss determined based on the third pseudo-label on the update process of the student model.
  • Step 62 If the confidence of the third pseudo-label is not lower than the confidence threshold, the preset value is determined as the weighted weight corresponding to the first distillation loss above.
  • a preset value for example, 1 can be directly used as the weighted weight corresponding to the third pseudo-label, so that the weighted weight can represent the impact of the loss determined based on the third pseudo-label on the update process of the student model.
  • the present disclosure does not limit the implementation of the above preset value.
  • the preset value is not lower than the highest value in the above preset range.
  • the preset value may be 1.
  • the weighted weight corresponding to the first distillation loss can be determined based on the confidence of the third pseudo-label. In this way, it is possible to weaken the impact of pseudo-labels with lower confidence on the update process of the student model, thereby better improving the model update effect.
  • the present disclosure does not limit the implementation method of the above step of "determining the loss characterization data of the student model according to the first distillation loss of the student model and the weighted weight corresponding to the first distillation loss". For example, it can be implemented using the following formulas (8)-(10).
  • the loss characterization data of the student model can be determined based on the first distillation loss, so that the loss characterization data can better represent the image generation performance of the student model.
  • Step 423 Update the student model according to the loss characterization data of the student model.
  • the updating process of the student model can be guided based on all training data (for example, the first sample image and the second sample image) and using these teacher models, so that the updated student model has better image generation performance.
  • step 42 Based on the relevant content of step 42 above, it can be seen that since the update process of the student model does not need to use real label information, all training data (for example, the first sample image and the second sample image) can be used to participate in the optimization process of the student model, which can better improve the model optimization effect.
  • all training data for example, the first sample image and the second sample image
  • Step 43 Determine whether the third stop condition is met, if so, execute the following step 45; if not, execute the following step 44.
  • the third stop condition refers to the condition that needs to be met when the training process of the student model ends; and the present disclosure does not limit the third stop condition, for example, it can be that the loss characterization data of the student model is lower than the sixth threshold, or the change rate of the loss characterization data of the student model is lower than the seventh threshold, or the number of updates of the student model reaches the eighth threshold.
  • the sixth threshold, the seventh threshold and the eighth threshold can all be preset.
  • Step 44 If the third stop condition is not met, return to continue executing the above step 41 and its subsequent steps.
  • the above step 41 and its subsequent steps can be re-executed based on the updated student model to start the next round of training process for the student model.
  • Step 45 If the third stop condition is reached, the image generation model is determined based on the student model.
  • the image generation model can be determined based on the student model (for example, directly determining the student model as the image generation model, etc.), so that the image generation model can have better performance.
  • the training process for the student model can be implemented by alternately updating the teacher model and the student model, so that the final trained student model has better performance.
  • the training data set is first used to train the super network so that each sub-network in the trained super network has good image generation performance.
  • image generation performance for example, image generation performance, etc.
  • search for a teacher model and a student model from the trained hypernetwork so that the teacher model and the student model also have good image generation performance
  • model distillation based on the training data set, the teacher model and the student model to obtain an image generation model, so that the image generation model not only has good image generation performance, but also has a relatively low computational complexity, so that the image generation model has a relatively high model efficiency (for example, fewer resources are required when deploying the image generation model, etc.) while ensuring performance, so that the image generation model can be deployed on hardware-constrained devices.
  • the teacher model and student model searched from the super-network have relatively good image generation performance, so that only fine-tuning of the teacher model and the student model is required in the future. This can effectively avoid the adverse effects caused by optimizing the teacher model and the student model from scratch (for example, slow model convergence speed, etc.), which is conducive to improving the efficiency of model training.
  • the hypernetwork can provide a large number of various network structures
  • the present disclosure can search for student models and teacher models that meet different constraints from the hypernetwork, which can effectively eliminate the additional resource consumption (such as human resource consumption, time consumption, etc.) caused by manual design and configuration of the teacher model, thereby helping to save resources.
  • additional resource consumption such as human resource consumption, time consumption, etc.
  • this method can achieve the synergy of seamlessly promoting model efficiency and data efficiency learning by sequentially executing the semi-supervised driven network architecture search stage and the adaptive online semi-supervised distillation stage, thereby achieving the purpose of unified compression of network data + model.
  • the present disclosure does not limit the execution subject of the above model building method.
  • the model building method provided in the embodiment of the present disclosure can be applied to a device with data processing function such as a terminal device or a server.
  • a device with data processing function such as a terminal device or a server.
  • the terminal device may be a smart phone, a computer, a personal digital assistant (PDA) or a tablet computer.
  • the server may be an independent server, a cluster server or a cloud server.
  • the present disclosure also provides an image generation method, which is described below in conjunction with the accompanying drawings.
  • the image generation method provided by the embodiment of the present disclosure includes the following S301-S302.
  • Figure 3 is a flowchart of an image generation method provided by an embodiment of the present disclosure.
  • S301 Acquire an image to be processed.
  • the image to be processed refers to image data that needs to be generated with the help of the above-mentioned image generation model.
  • S302 Processing the image to be processed using an image generation model to obtain an image generation result; the image generation model is constructed using any implementation of the model construction method provided in the present disclosure.
  • the image generation result refers to the result output by the image generation model after processing the above image to be processed.
  • the image to be processed can be processed using the pre-built image generation model to obtain an image generation result.
  • the image generation model has good performance, the image generation result determined based on the image generation model is better, which is conducive to improving the image generation effect.
  • the present disclosure does not limit the execution subject of the above image generation method.
  • the image generation method provided in the embodiment of the present disclosure can be applied to a device with a data processing function such as a terminal device or a server.
  • the image generation method provided in the embodiment of the present disclosure can also be implemented by means of a data communication process between different devices (for example, a terminal device and a server, two terminal devices, or two servers).
  • the embodiment of the present disclosure also provides a model building device, which is explained and illustrated in conjunction with Figure 4.
  • Figure 4 is a structural schematic diagram of a model building device provided in the embodiment of the present disclosure. It should be noted that for the technical details of the model building device provided in the embodiment of the present disclosure, please refer to the relevant content of the model building method above.
  • the model building device 400 provided in the embodiment of the present disclosure includes:
  • a first acquisition unit 401 is used to acquire a training data set; the training data set includes a first sample image and label information corresponding to the first sample image;
  • the second training unit 404 is used to perform model distillation based on the training data set, the teacher model and the student model to obtain an image generation model.
  • the first training unit 402 includes:
  • a first determination subunit used to determine a teacher generator and a student generator from the super network
  • the model distillation subunit is used to perform model distillation based on the training data set, the teacher generator and the student generator to obtain a trained super network.
  • the first determination subunit is specifically used to: sample a first subnetwork and a second subnetwork from the supernetwork; the network size of the first subnetwork is larger than the network size of the second subnetwork; determine the teacher generator based on the first subnetwork; and determine the student generator based on the second subnetwork.
  • the first subnetwork includes a maximum subnetwork; and the second subnetwork includes at least one of a minimum subnetwork and a random subnetwork.
  • the training data set further includes a second sample image
  • the model distillation subunit is specifically used to: determine the loss representation data of the teacher generator using the first sample image and the label information corresponding to the first sample image; determine the loss representation data of the student generator using the second sample image and the first pseudo label corresponding to the second sample image; the first pseudo label is obtained by the teacher generator processing the second sample image; and update the hypernetwork, the teacher generator and the student generator according to the loss representation data of the teacher generator and the loss representation data of the student generator.
  • the model distillation subunit is further used to: after updating the hypernetwork, the teacher generator, and the student generator, continue to perform the step of determining the loss representation data of the teacher generator using the first sample image and the label information corresponding to the first sample image until a first stopping condition is reached.
  • the training data set further includes a second sample image
  • the model distillation subunit is specifically used to: use the first sample image and the first sample According to the label information corresponding to the first sample image, the teacher generator is updated; using the second sample image and the second pseudo label corresponding to the second sample image, the loss representation data of the student generator is determined; the second pseudo label is obtained by processing the second sample image by the updated teacher generator; according to the loss representation data of the student generator, the hypernetwork, the teacher generator and the student generator are updated, and the step of updating the teacher generator using the first sample image and the label information corresponding to the first sample image is continued until the second stop condition is reached.
  • the network search unit 403 is specifically used to: search for the student model from the trained super network based on the first model constraint, so that the student model satisfies the first model constraint; and search for at least one from the trained super network based on the second model constraint, so that each of the teacher models satisfies the second model constraint, and the at least one teacher model satisfies the preset model architecture condition.
  • the at least one teacher model includes a teacher model that satisfies a preset model depth condition and a teacher model that satisfies a preset model width condition; and/or, the second model constraint is determined based on the first model constraint so that each of the teacher models and the student model satisfies a preset multiple condition on one or more model indicators.
  • the training data set further includes a second sample image
  • the second training unit 404 is specifically used to: update the teacher model using the first sample image and the label information corresponding to the first sample image; update the student model using the second sample image and the third pseudo label corresponding to the second sample image, and continue to execute the step of updating the teacher model using the first sample image and the label information corresponding to the first sample image until the third stop condition is reached, and determine the image generation model according to the student model; the third pseudo label is obtained by processing the second sample image by the updated teacher model.
  • the second training unit 404 is specifically used to update the student model using the first sample image, the fourth pseudo label corresponding to the first sample image, the second sample image, and the third pseudo label corresponding to the second sample image; the fourth pseudo label is obtained by processing the first sample image by the updated teacher model.
  • the second training unit 404 is specifically configured to: determine the student model using the second sample image and the third pseudo label corresponding to the second sample image. a first distillation loss; determining loss characterization data of the student model according to the first distillation loss; and updating the student model according to the loss characterization data of the student model.
  • the second training unit 404 is specifically used to: determine the loss representation data of the student model based on the first distillation loss and the weighted weight corresponding to the first distillation loss; the weighted weight is determined based on the confidence of the third pseudo label; the confidence is determined based on the updated teacher model, the discriminator corresponding to the updated teacher model, and the second sample image.
  • the process of determining the weighted weight includes: if the confidence level is lower than a confidence threshold, randomly selecting a value from a preset interval and determining it as the weighted weight; if the confidence level is not lower than the confidence threshold, determining the preset value as the weighted weight.
  • the process of determining the confidence threshold includes: processing at least one image data using an updated teacher model to obtain a generated image corresponding to each image data; the at least one image data includes the second sample image and/or the first sample image; using the discriminator corresponding to the updated teacher model to perform discriminant processing on each of the generated images to obtain a discrimination result corresponding to each of the generated images; performing data analysis processing on the discrimination result corresponding to at least one of the generated images to obtain the confidence threshold.
  • the super network is first trained using the training data set so that each sub-network in the trained super network has good image generation performance (for example, image generation performance, etc.); then, a teacher model and a student model are searched from the trained super network so that the teacher model and the student model also have good image generation performance; finally, model distillation is performed based on the training data set, the teacher model and the student model to obtain an image generation model, so that the image generation model not only has good image generation performance, but also has a relatively low computational complexity, so that the image generation model has a relatively high model efficiency (for example, fewer resources are required when deploying the image generation model, etc.) while ensuring performance, so that the image generation model can be deployed on hardware-constrained devices.
  • a training data set for example, some image samples with label information and some image samples without label information
  • the embodiment of the present disclosure also provides an image generation device, which is explained and illustrated in conjunction with FIG5. It should be noted that for the technical details of the image generating device provided in the embodiment of the present disclosure, please refer to the relevant contents of the image generating method above.
  • an image generating device 500 provided in an embodiment of the present disclosure includes:
  • the second acquisition unit 501 is used to acquire an image to be processed
  • the image generation unit 502 is used to process the image to be processed using a pre-constructed image generation model to obtain an image generation result; the image generation model is constructed using any implementation of the model construction method provided in the embodiments of the present disclosure.
  • the image generation device 500 Based on the above-mentioned contents of the image generation device 500, it can be known that, for the image generation device 500 provided by the embodiment of the present disclosure, after obtaining the image to be processed, the image to be processed can be processed using the pre-built image generation model to obtain an image generation result. Among them, because the image generation model has good performance, the image generation result determined based on the image generation model is better, which is conducive to improving the image generation effect.
  • an embodiment of the present disclosure also provides an electronic device, which includes a processor and a memory: the memory is used to store instructions or computer programs; the processor is used to execute the instructions or computer programs in the memory, so that the electronic device executes any implementation of the model building method or image generation method provided in the embodiment of the present disclosure.
  • the terminal device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc.
  • mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc.
  • PDAs personal digital assistants
  • PADs tablet computers
  • PMPs portable multimedia players
  • vehicle-mounted terminals such as vehicle-mounted navigation terminals
  • fixed terminals such as digital TVs, desktop computers, etc.
  • the electronic device shown in FIG6 is only an example and should not bring any limitation to the functions and scope of use of
  • the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603.
  • a processing device e.g., a central processing unit, a graphics processing unit, etc.
  • RAM random access memory
  • various programs and data required for the operation of the electronic device 600 are also stored.
  • the processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604.
  • An input/output (I/O) interface 605 is also connected to the bus 604.
  • input devices 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.
  • Output device 607 such as a liquid crystal display (LCD), a speaker, a vibrator, etc.
  • storage device 608 such as a magnetic tape, a hard disk, etc.
  • communication device 609 can allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data.
  • FIG. 6 shows an electronic device 600 with various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or provided instead.
  • an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart.
  • the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602.
  • the processing device 601 the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
  • the electronic device provided by the embodiment of the present disclosure and the method provided by the above embodiment belong to the same inventive concept.
  • the technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.
  • the embodiments of the present disclosure also provide a computer-readable medium, in which instructions or computer programs are stored.
  • the instructions or computer programs are executed on a device, the device executes any implementation of the model building method or image generation method provided in the embodiments of the present disclosure.
  • the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above.
  • Computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device.
  • a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein the carrier
  • the computer-readable program code is a computer-readable program code.
  • This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above.
  • the computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or device.
  • the program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
  • the client and server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network).
  • HTTP Hyper Text Transfer Protocol
  • Examples of communication networks include a local area network ("LAN”), a wide area network ("WAN”), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
  • the computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
  • the computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device can execute the method.
  • Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages.
  • the program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server.
  • the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).
  • LAN local area network
  • WAN wide area network
  • Internet service provider e.g., AT&T, MCI, Sprint, EarthLink, MSN, GTE, etc.
  • each box in the flowchart or block diagram may represent a module, a program segment, or a portion of code, which contains one or more executable programs for implementing the specified logical functions. Instructions. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two blocks shown in succession may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved.
  • each block in the block diagram and/or flow chart, and combinations of blocks in the block diagram and/or flow chart may be implemented using a dedicated hardware-based system that performs the specified functions or operations, or may be implemented using a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit/module does not, in some cases, constitute a limitation on the unit itself.
  • exemplary types of hardware logic components include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
  • FPGAs field programmable gate arrays
  • ASICs application specific integrated circuits
  • ASSPs application specific standard products
  • SOCs systems on chips
  • CPLDs complex programmable logic devices
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment.
  • a machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • a machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing.
  • a more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
  • RAM random access memory
  • ROM read-only memory
  • EPROM or flash memory erasable programmable read-only memory
  • CD-ROM portable compact disk read-only memory
  • CD-ROM compact disk read-only memory
  • magnetic storage device or any suitable combination of the foregoing.
  • At least one (item) means one or more, and “more” means two or more.
  • “And/or” is used to describe the association relationship of associated objects, indicating that three relationships may exist.
  • a and/or B can mean: only A exists, only B exists, and both A and B exist, where A and B can be singular or plural.
  • the character “/” generally indicates the relationship between the preceding and following.
  • the objects of association are in an “or” relationship.
  • At least one of the following” or similar expressions refers to any combination of these items, including any combination of single items or plural items.
  • At least one of a, b, or c can mean: a, b, c, "a and b", “a and c", “b and c", or "a and b and c", where a, b, c can be single or multiple.
  • the steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two.
  • the software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Biophysics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

本公开提供了一种模型构建方法、图像生成方法、装置、设备、介质，该方法包括：在获取到训练数据集之后，先利用训练数据集对超网络进行训练，以使已训练的超网络中的各个子网络均具有较好的图像生成性能；再从已训练的超网络中搜索教师模型和学生模型，以使教师模型与学生模型也具有较好的图像生成性能；最后，依据训练数据集、教师模型以及学生模型进行模型蒸馏，得到图像生成模型。

Description

模型构建方法、图像生成方法、装置、设备、介质
本申请要求于2023年6月30日递交的中国专利申请第202310798179.9号的优先权，在此全文引用上述中国专利申请公开的内容以作为本申请的一部分。
技术领域
本公开涉及一种模型构建方法、图像生成方法、装置、设备、介质。
背景技术
对于一些图像生成任务来说，该任务可以借助预先构建的模型，比如生成对抗网络(Generative adversarial networks，GAN)模型，进行实现。
另外，对于具有较好的图像生成性能的模型来说，其通常需要依赖于爆炸性的计算复杂性，如此导致部署这种具有大量资源需求的模型是一项艰巨的任务，尤其是在硬件受限的情况下，从而导致如何缓解这个问题成为一项亟待解决的技术问题。
发明内容
本公开提供了一种模型构建方法、图像生成方法、装置、设备、介质。
为了实现上述目的，本公开提供的技术方案如下：
本公开提供一种模型构建方法，所述方法包括：
获取训练数据集；所述训练数据集包括第一样本图像和所述第一样本图像对应的标签信息；
利用所述训练数据集对超网络进行训练；
从已训练的超网络中搜索教师模型和学生模型；
依据所述训练数据集、所述教师模型以及所述学生模型进行模型蒸馏，得到图像生成模型。
在一种可能的实施方式下，所述超网络的训练过程，包括：
从所述超网络中确定教师生成器和学生生成器；
依据所述训练数据集、所述教师生成器和所述学生生成器进行模型蒸馏，得到已训练的超网络。
在一种可能的实施方式下，所述从所述超网络中确定教师生成器和学生生成器，包括：
从所述超网络中采样第一子网络和第二子网络；所述第一子网络的网络尺寸大于所述第二子网络的网络尺寸；
根据所述第一子网络，确定所述教师生成器；
根据所述第二子网络，确定所述学生生成器。
在一种可能的实施方式下，所述第一子网络包括最大子网络；
所述第二子网络包括最小子网络和随机子网络中的至少一个。
在一种可能的实施方式下，所述训练数据集还包括第二样本图像；
所述依据所述训练数据集、所述教师生成器和所述学生生成器进行模型蒸馏，得到已训练的超网络，包括：
利用所述第一样本图像和所述第一样本图像对应的标签信息，确定所述教师生成器的损失表征数据；
利用所述第二样本图像和所述第二样本图像对应的第一伪标签，确定所述学生生成器的损失表征数据；所述第一伪标签是由所述教师生成器针对所述第二样本图像进行处理所得到的；
根据所述教师生成器的损失表征数据和所述学生生成器的损失表征数据，更新所述超网络、所述教师生成器以及所述学生生成器。
在一种可能的实施方式下，所述更新所述超网络、所述教师生成器以及所述学生生成器之后，所述方法还包括：
继续执行所述利用所述第一样本图像和所述第一样本图像对应的标签信息，确定所述教师生成器的损失表征数据的步骤，直至达到第一停止条件。
在一种可能的实施方式下，所述训练数据集还包括第二样本图像；
所述依据所述训练数据集、所述教师生成器和所述学生生成器进行模型蒸馏，得到已训练的超网络，包括：
利用所述第一样本图像和所述第一样本图像对应的标签信息，更新所述教师生成器；
利用所述第二样本图像和所述第二样本图像对应的第二伪标签，确定所 述学生生成器的损失表征数据；所述第二伪标签是由更新后的教师生成器针对所述第二样本图像进行处理所得到的；
根据所述学生生成器的损失表征数据，更新所述超网络、所述教师生成器以及所述学生生成器，并继续执行所述利用所述第一样本图像和所述第一样本图像对应的标签信息，更新所述教师生成器的步骤，直至达到第二停止条件。
在一种可能的实施方式下，所述从已训练的超网络中搜索教师模型和学生模型，包括：
依据第一模型约束，从已训练的所述超网络中搜索所述学生模型，以使所述学生模型满足所述第一模型约束；
依据所述第二模型约束，从已训练的所述超网络中搜索至少一个教师模型，以使各所述教师模型均满足所述第二模型约束，所述至少一个教师模型满足预设模型架构条件。
在一种可能的实施方式下，所述至少一个教师模型包括满足预设模型深度条件的教师模型和满足预设模型宽度条件的教师模型；
和/或，
所述第二模型约束是根据所述第一模型约束所确定的，以使各所述教师模型与所述学生模型在一个或者多个模型指标上满足预设倍数条件。
在一种可能的实施方式下，所述训练数据集还包括第二样本图像；
所述依据所述训练数据集、所述教师模型以及所述学生模型进行模型蒸馏，得到图像生成模型，包括：
利用所述第一样本图像和所述第一样本图像对应的标签信息，更新所述教师模型；
利用所述第二样本图像和所述第二样本图像对应的第三伪标签，更新所述学生模型，并继续执行所述利用所述第一样本图像和所述第一样本图像对应的标签信息，更新所述教师模型的步骤，直至在达到第三停止条件时，根据所述学生模型，确定所述图像生成模型；所述第三伪标签是由更新后的教师模型针对所述第二样本图像进行处理所得到的。
在一种可能的实施方式下，所述利用所述第二样本图像和所述第二样本图像对应的第三伪标签，更新所述学生模型，包括：
利用所述第一样本图像、所述第一样本图像对应的第四伪标签、所述第二样本图像以及所述第二样本图像对应的第三伪标签，更新所述学生模型；所述第四伪标签是由更新后的教师模型针对所述第一样本图像进行处理所得到的。
在一种可能的实施方式下，所述学生模型的更新过程，包括：
利用所述第二样本图像和所述第二样本图像对应的第三伪标签，确定所述学生模型的第一蒸馏损失；
根据所述第一蒸馏损失，确定所述学生模型的损失表征数据；
根据所述学生模型的损失表征数据，更新所述学生模型。
在一种可能的实施方式下，所述根据所述第一蒸馏损失，确定所述学生模型的损失表征数据，包括：
根据所述第一蒸馏损失以及所述第一蒸馏损失对应的加权权重，确定所述学生模型的损失表征数据；所述加权权重是根据所述第三伪标签的置信度所确定的；所述置信度是根据更新后的教师模型、更新后的教师模型对应的判别器以及所述第二样本图像所确定的。
在一种可能的实施方式下，所述加权权重的确定过程，包括：
若所述置信度低于置信度阈值，则从预设区间中随机选择一个数值，确定为所述加权权重；
若所述置信度不低于所述置信度阈值，则将预设数值确定为所述加权权重。
在一种可能的实施方式下，所述置信度阈值的确定过程，包括：
利用更新后的教师模型对至少一个图像数据进行处理，得到各个图像数据对应的生成图像；所述至少一个图像数据包括所述第二样本图像和/或所述第一样本图像；
利用更新后的教师模型对应的判别器对各所述生成图像进行判别处理，得到各所述生成图像对应的判别结果；
针对至少一个所述生成图像对应的判别结果进行数据分析处理，得到所述置信度阈值。
本公开提供了一种图像生成方法，所述方法包括：
获取待处理图像；
利用预先构建的图像生成模型对所述待处理图像进行处理，得到图像生成结果；所述图像生成模型是利用本公开提供的模型构建方法所构建的。
本公开提供了一种模型构建装置，包括：
第一获取单元，用于获取训练数据集；所述训练数据集包括第一样本图像和所述第一样本图像对应的标签信息；
第一训练单元，用于利用所述训练数据集对超网络进行训练；
网络搜索单元，用于从已训练的超网络中搜索教师模型和学生模型；
第二训练单元，用于依据所述训练数据集、所述教师模型以及所述学生模型进行模型蒸馏，得到图像生成模型。
本公开提供了一种图像生成装置，包括：
第二获取单元，用于获取待处理图像；
图像生成单元，用于利用预先构建的图像生成模型对所述待处理图像进行处理，得到图像生成结果；所述图像生成模型是利用本公开提供的模型构建方法所构建的。
本公开提供了一种电子设备，所述设备包括：处理器和存储器；
所述存储器，用于存储指令或计算机程序；
所述处理器，用于执行所述存储器中的所述指令或计算机程序，以使得所述电子设备执行本公开提供的模型构建方法或者图像生成方法。
本公开提供了一种计算机可读介质，所述计算机可读介质中存储有指令或计算机程序，当所述指令或计算机程序在设备上运行时，使得所述设备执行本公开提供的模型构建方法或者图像生成方法。
本公开提供了一种计算机程序产品，其包括承载在非暂态计算机可读介质上的计算机程序，该计算机程序包含用于执行本公开提供的模型构建方法或者图像生成方法的程序代码。
附图说明
为了更清楚地说明本公开实施例的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本公开中记载的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其它的附图。
图1为本公开实施例提供的一种模型构建方法的流程图；
图2为本公开实施例提供的一种模型构建过程的示意图；
图3为本公开实施例提供的一种图像生成方法的流程图；
图4为本公开实施例提供的一种模型构建装置的结构示意图；
图5为本公开实施例提供的一种图像生成装置的结构示意图；以及
图6为本公开实施例提供的一种电子设备的结构示意图。
具体实施方式
为了使本技术领域的人员更好地理解本公开方案，下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅是本公开一部分实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
为了更好地理解本公开所提供的技术方案，下面先结合一些附图对本公开提供的模型构建方法进行说明。如图1所示，本公开实施例提供的模型构建方法，包括下文S101-S104。其中，该图1为本公开实施例提供的一种模型构建方法的流程图。
S101：获取训练数据集；训练数据集包括第一样本图像和第一样本图像对应的标签信息。
其中，训练数据集是指在模型训练过程中所需使用的数据集(比如，由图1所示的有标签的训练数据与无标签的训练数据所构成的数据集)。
需要说明的是，本公开不限定上段中的有标签的训练数据，例如，其可以采用进行实施。其中，该表示一个有标签的训练数据的集合；该{xi,yi}表示第i个有标签的训练数据，i为正整数，1≤i≤M，M为正整数，M表示有标签的训练数据的个数；xi表示第i个有标签的训练数据中的图像数据；yi表示该xi对应的标签信息，该标签信息可以借助人工标注方式进行获取，也可以借助其他方式进行获取，本公开对此不做具体限定。另外，本公开不限定该yi的表达方式，例如，其可以采用图像数据进行表达。此外，本公开不限定该yi的实施方式，例如，在图像的深度估计领域中，yi可以是指xi的真实深度图。又如，在图像的脸部识别领域中，yi可以是指xi的真实脸 部标记图，以使该yi能够表示出脸部区域在该xi中实际所处区域。还如，在图像风格变化领域中，yi可以是指目标图像域的图像，该目标图像域的域风格与xi这一图像数据所具有的数据域风格不同，而且本公开不限定该目标图像域，比如，其可以是具有水彩风格的图像域、具有卡通风格的图像域、具有简笔画风格的图像域、或者具有黑白风格的图像域等。
还需要说明的是，本公开不限定上文无标签的训练数据，例如，其可以采用进行实施。其中，该表示一个无标签的训练数据的集合；zk表示第k个无标签的训练数据，k为正整数，1≤k≤N-M，N为正整数，N表示上文训练数据集中的训练数据的个数。
另外，本公开不限定上文训练数据集的实施方式，例如，在一些应用场景(比如，全监督训练场景)下，该训练数据集可以包括第一样本图像(比如，xi)和该第一样本图像对应的标签信息(比如，yi)；其中，该第一样本图像是指在模型训练过程中所需使用的具有标签信息的图像数据；该第一样本图像对应的标签信息是指预先针对该第一样本图像所标注的标签。需要说明的是，本公开不限定该第一样本图像的个数。
实际上，为了尽可能地降低训练数据的获取难度，可以使用一些有标签的训练数据以及一些无标签的训练数据进行模型训练，以使模型不仅能够从这些有标签的训练数据中学习到知识，还能够从这些无标签的训练数据中学习到某一个分布域下的结构性特征。基于此，本公开还提供了上文训练数据集的一种可能的实施方式，在该实施方式中，该训练数据集可以不仅包括第一样本图像以及该第一样本图像对应的标签信息，可以还包括第二样本图像(比如，zk)。其中，该第二样本图像是指在模型训练过程中所需使用的不具有标签信息的图像数据。需要说明的是，本公开不限定该第二样本图像的个数。
可见，在一些应用场景下，上文的训练数据集不仅包括一些有标签的训练数据，还可以包括一些无标签的训练数据，以使这些无标签的训练数据能够为模型训练过程带来一些额外的监督信息，从而增强模型泛化性，进而有利于模型学习到某一个分布域下的结构性特征，如此能够实现在尽可能少地付出标注成本的前提下提高训练数据量。
此外，本公开不限定上文训练数据集的获取方式。
S102：利用训练数据集对超网络进行训练。
其中，超网络用于表示一个搜索空间(比如，图2所示的搜索空间)，以使该超网络可以提供一些具有不同网络结构的子网络(比如，一些具有不同深度和宽度的子网络等)。例如，该超网络用于表示搜索空间该Gt表示从该搜索空间(也就是，该超网络)中采样所得的一个子网络，t为正整数，t≤该超网络所能提供的子网络的总数。
实际上，为了更好地提高模型构建效果，本公开提供了上文超网络的一种可能的实施方式，在该实施方式中，该超网络具有动态可调的深度和宽度，以使该超网络的深度和宽度能够随着针对该超网络的训练过程进行动态地调整处理。
另外，本公开不限定上文S102的实施方式，例如，当上文训练数据集只包括有标签的训练数据时，可以采用现有的或者未来出现的任意一种能够针对超网络进行监督训练的方法进行实施。又如，当上文训练数据集包括一些有标签的训练数据和一些无标签的训练数据时，可以采用现有的或者未来出现的任意一种能够针对超网络进行半监督训练的方法进行实施。
实际上，为了更好地提高模型构建效果，可以借助蒸馏算法(比如，在线蒸馏算法)实现针对超网络的训练过程。基于此，本公开还提供了上文S102的一种可能的实施方式，其具体可以包括下文步骤11-步骤12。
步骤11：从超网络中确定教师生成器和学生生成器。
教师生成器是指从超网络中采样所得的一种具有较大网络尺寸的子网络；而且该教师生成器具有图像生成功能(例如，图像生成功能等)；还有，本公开不限定该教师生成器，例如，其可以为图2所示的最大子网络。其中，该网络尺寸用于描述一个网络在至少一个维度(比如，深度和/或宽度)上所呈现的尺寸，以使该网络尺寸能够表征出该网络的大小。需要说明的是，本公开不限定该“至少一个维度”的实施方式，例如，其可以包括深度和/或宽度。
学生生成器是指从超网络中采样所得的另一种或者多种子网络；而且该学生生成器具有图像生成功能(例如，图像生成功能等)；还有，该学生生成器的网络尺寸可以小于上文教师生成器的网络尺寸，以使最终已训练的学生生成器能够表示出针对该教师生成器的压缩结果。
另外，本公开不限定上文学生生成器的实施方式，例如，该学生生成器 可以为图2所示的随机子网络。又如，该学生生成器可以包括图2所示的最小子网络和随机子网络。其中，该随机子网络是指从该超网络中随机采样所得的子网络。该最小子网络是按照预先设定的最小子网络采样规则从该超网络中采样所得的子网络。需要说明的是，本公开不限定该最小子网络采样规则，例如，该最小子网络采样规则可以预先依据应用场景所设定。
此外，本公开不限定上文步骤11的实施方式，例如，其具体可以采用现有的或者未来出现的任意一种能够从超网络中抽取出一些网络尺寸不同的子网络的方法进行实施。
实际上，为了更好地提高超网络的训练效果，本公开还提供了上文步骤11的一种可能的实施方式，其具体可以包括下文步骤111-步骤113。
步骤111：从超网络中采样第一子网络和第二子网络；该第一子网络的网络尺寸大于该第二子网络的网络尺寸。
其中，第一子网络是指从超网络中采样所得、具有较大网络尺寸的子网络；而且本公开不限定该第一子网络，例如，该第一子网络可以包括从该超网络中采样所得的最大子网络。
第二子网络是指从超网络中采样所得、具有较小网络尺寸的子网络；而且本公开不限定该第二子网络，例如，该第二子网络可以包括从该超网络中采样所得的最小子网络和随机子网络中的至少一个。可见，在一种可能的实施方式下，该第二子网络可以包括一个最小子网络以及一个或者多个随机子网络。
步骤112：根据上文第一子网络，确定教师生成器。
需要说明的是，本公开不限定步骤112的实施方式，例如，其具体可以为：在获取到上文第一子网络之后，可以将该第一子网络直接确定为教师生成器。可见，在一种可能的实施方式下，该教师生成器可以是指从超网络中采样所得的最大子网络。
步骤113：根据上文第二子网络，确定学生生成器。
需要说明的是，本公开不限定步骤113的实施方式，例如，其具体可以为：在获取到上文第二子网络之后，可以将该第二子网络直接确定为学生生成器。可见，在一种可能的实施方式下，该学生生成器可以包括从超网络中采样所得的一个最小子网络以及一个或者多个随机子网络。
基于上文步骤11的相关内容可知，在获取到用于表示搜索空间的超网络之后，可以从该超网络中采样教师生成器以及学生生成器，以便后续能够基于该教师生成器以及学生生成器进行模型蒸馏，以得到已训练的超网络。
步骤12：依据训练数据集、教师生成器和学生生成器进行模型蒸馏，得到已训练的超网络。
需要说明的是，本公开不限定步骤12的实施方式，例如，其可以采用现有的或者未来出现的任意一种模型蒸馏方法(比如，在线蒸馏算法等)进行实施。可见，在一种可能的实施方式下，当上文训练数据集包括一些有标签的训练数据时，该步骤12具体可以为：利用这些有标签的训练数据、教师生成器和学生生成器进行模型蒸馏，得到已训练的超网络。
实际上，为了更好地降低超网络的训练过程针对标签信息的依赖，可以采用半监督方式训练该超网络。基于此，本公开还提供了上文步骤12的一种可能的实施方式，在该实施方式中，当上文训练数据集包括第一样本图像、该第一样本图像对应的标签信息、以及第二样本图像时，该步骤12具体可以包括下文步骤121-步骤123。
步骤121：利用第一样本图像和该第一样本图像对应的标签信息，确定教师生成器的损失表征数据。
其中，教师生成器的损失表征数据用于表征该教师生成器的图像生成性能(例如，图像生成性能)；而且本公开不限定该教师生成器的损失表征数据的确定过程，例如，其可以采用现有的或者未来出现的任一种能够确定出该教师生成器在有标签的训练数据上所呈现的损失的方法进行实施。
实际上，为了更好地提高针对教师生成器的损失确定效果，本公开还提供了上文教师生成器的损失表征数据的确定过程的一种可能的实施方式，在该实施方式中，当该教师生成器为生成对抗网络(Generative Adversarial Networks，GAN)中的生成器时，该教师生成器的损失表征数据的确定过程，具体可以包括下文步骤1211-步骤1213。
步骤1211：利用第一样本图像、该第一样本图像对应的标签信息、教师生成器以及该教师生成器对应的判别器，确定该教师生成器对应的对抗损失。
其中，教师生成器对应的判别器是指在评估该教师生成器的性能时所需借助的判别器；而且可以利用该教师生成器对应的判别器与该教师生成器组 成一个生成对抗网络。
教师生成器对应的对抗损失是指由该教师生成器以及该教师生成器对应的判别器所构成的生成对抗网络所呈现的损失。
另外，本公开不限定上文教师生成器对应的对抗损失的确定过程，例如，其可以采用现有的或者未来出现的任一种GAN的对抗损失的确定方法(例如，下文公式(1)所示的确定方式)进行实施。
式(1)中，LGAN(GT,D)表示上文教师生成器对应的对抗损失；GT表示该教师生成器；D表示该教师生成器对应的判别器；x表示样本图像数据(例如，上文第一样本图像)；y表示该样本图像数据对应的标签信息(例如，该第一样本图像对应的标签信息)；GT(x)表示将该样本图像数据输入该教师生成器GT之后由该教师生成器GT所输出的图像数据；表示在训练数据{x,y}下的期望函数；表示在样本图像数据下的期望函数。
基于上文步骤1211的相关内容可知，在获取到教师生成器GT之后，可以利用一些有标签的训练数据该教师生成器GT、以及该教师生成器GT对应的判别器D，确定该教师生成器GT对应的对抗损失，以便后续能够基于该对抗损失，优化该教师生成器GT以及该教师生成器GT对应的判别器D，以使最终已训练的教师生成器GT能够实现将xi映射到yi这一目标，并使得最终已训练的判别器D能够实现将由该教师生成器GT所输出的生成图像与真实图像区分开。
步骤1212：利用第一样本图像、该第一样本图像对应的标签信息以及教师生成器，确定该教师生成器对应的重建损失。
其中，教师生成器对应的重建损失用于表征该教师生成器的图像生成性能，以便后续能够借助该重建损失促使该教师生成器的输出图像与真实标签相接近。
另外，本公开不限定上文教师生成器对应的重建损失的确定过程，例如，其可以采用现有的或者未来出现的任一种图像生成器的重建损失的确定方法(例如，下文公式(2)所示的确定方式)进行实施。
式(2)中，Lrecon(GT)表示教师生成器对应的重建损失；GT表示该教师 生成器；x表示样本图像数据(例如，上文第一样本图像)；y表示该样本图像数据对应的标签信息(例如，该第一样本图像对应的标签信息)；GT(x)表示将该样本图像数据输入该教师生成器GT之后由该教师生成器GT所输出的图像数据；表示在训练数据{x,y}下的期望函数。
基于上文步骤1212的相关内容可知，当上文第一样本图像对应的标签信息为图像数据，而且上文教师生成器属于GAN中的生成器时，可以将上文第一样本图像输入该教师生成器，以得到由该教师生成器所输出的生成图像；再依据该生成图像与该第一样本图像对应的标签信息之间的相似程度，确定该教师生成器对应的重建损失，以使该重建损失能够表示出该教师生成器所具有的图像生成性能，以便后续能够基于该重建损失优化该教师生成器，以使优化后的教师生成器具有更好的图像生成性能。
步骤1213：依据上文教师生成器对应的对抗损失和该教师生成器对应的重建损失，确定该教师生成器的损失表征数据(例如，图2所示的监督损失1)。
本公开中，在获取到上文教师生成器对应的对抗损失和该教师生成器对应的重建损失之后，可以依据两项损失确定该教师生成器的损失表征数据(如下文公式(3)所示的确定方式)，以使该损失表征数据能够更全面且准确地表示出该教师生成器所呈现的图像生成性能。
式(3)中，表示上文教师生成器的损失表征数据；LGAN(GT,D)表示上文教师生成器对应的对抗损失；GT表示该教师生成器；D表示该教师生成器对应的判别器；Lrecon(GT)表示教师生成器对应的重建损失。
基于上文步骤1211至步骤1213的相关内容可知，对于教师生成器来说，可以利用一些有标签的训练数据(比如，上文第一样本图像和该第一样本图像对应的标签信息)，确定该教师生成器在这些有标签的训练数据上所呈现的损失(也就是，监督损失)，以便后续能够基于该损失优化该教师生成器。
步骤122：利用第二样本图像和该第二样本图像对应的第一伪标签，确定学生生成器的损失表征数据；该第一伪标签是由教师生成器针对该第二样本图像进行处理所确定的。
其中，第二样本图像对应的第一伪标签是指由上文教师生成器针对该第 二样本图像的处理结果，以便后续能够将该处理结果作为该第二样本图像的伪标签指导学生生成器的优化过程，如此能够实现由该教师生成器指导学生生成器的目的，从而能够实现在不依赖判别器的情况下完成针对该学生生成器的优化处理。
学生生成器的损失表征数据用于表征该学生生成器的图像生成性能(比如，图像生成性能)；而且本公开不限定该学生生成器的损失表征数据的确定过程，例如，其可以采用现有的或者未来出现的任意一种能够针对该学生生成器计算损失的方法(比如，在线蒸馏损失等)进行实施。
实际上，在一些应用场景下，为了更好地提高超网络的训练效果，上文学生生成器可以包括从该超网络中采样所得的至少两个子网络。基于此，本公开还提供了上文步骤122的一种可能的实施方式，在该实施方式中，当该学生生成器包括最小子网络和随机子网络时，该步骤122具体可以包括下文步骤1221-步骤1223。
步骤1221：利用第二样本图像和该第二样本图像对应的第一伪标签，确定最小子网络的在线蒸馏损失，以使该在线蒸馏损失能够表示出该最小子网络的图像生成性能。
需要说明的是，本公开不限定步骤1221的实施方式，例如，其可以采用现有的或者未来出现的任意一种能够计算出一个模型的在线蒸馏损失的方法进行实施。
步骤1222：利用第二样本图像和该第二样本图像对应的第一伪标签，确定随机子网络的在线蒸馏损失，以使该在线蒸馏损失能够表示出该随机子网络的图像生成性能。
需要说明的是，本公开不限定步骤1222的实施方式，例如，其可以采用现有的或者未来出现的任意一种能够计算出一个模型的在线蒸馏损失的方法进行实施。
步骤1223：依据最小子网络的在线蒸馏损失和随机子网络的在线蒸馏损失，确定学生生成器的损失表征数据。
本公开中，当上文学生生成器包括最小子网络和随机子网络时，在获取到该最小子网络的在线蒸馏损失和该随机子网络的在线蒸馏损失之后，可以依据这两项损失确定该学生生成器的损失表征数据(例如，可以采用下文公 式(4)进行实现)。
式(4)中，表示学生生成器的损失表征数据；Gr表示随机子网络；Gs表示最小子网络；LOD(GT(xu),Gr(xu))表示随机子网络的在线蒸馏损失；xu表示无标签的训练数据(例如，第二样本图像)；GT(xu)表示由上文教师生成器处理该xu时所生成的图像数据(例如，该第二样本图像对应的第一伪标签)；Gr(xu)表示由随机子网络处理该xu时所生成的图像数据；LOD(GT(xu),Gs(xu))表示最小子网络的在线蒸馏损失；Gs(xu)表示由最小子网络处理该xu时所生成的图像数据。
基于上文步骤1221至步骤1223的相关内容可知，在一些应用场景下，如果上文学生生成器包括多个子网络，则可以在上文教师生成器的指导下，先确定出各个子网络在这些无标签的训练数据上所呈现的在线蒸馏损失；再将所有子网络对应的在线蒸馏损失进行加和，以得到该学生生成器的损失表征数据，以使该损失表征数据能够更好地表示出该学生生成器所具有的图像生成性能。
步骤123：根据教师生成器的损失表征数据和学生生成器的损失表征数据，更新超网络、教师生成器以及学生生成器。
本公开中，在获取到教师生成器的损失表征数据和学生生成器的损失表征数据之后，可以依据这两项损失，更新超网络、教师生成器以及学生生成器，以使更新后的超网络、更新后的教师生成器以及更新后的学生生成器均具有较好的图像生成性能。
另外，本公开不限定上文步骤123的实施方式，例如，在一些应用场景下，该步骤123具体可以包括下文步骤1231-步骤1233。
步骤1231：根据教师生成器的损失表征数据和学生生成器的损失表征数据，确定超网络的损失表征数据，以使该损失表征数据能够比较好的表示出该超网络的图像生成性能。
需要说明的是，本公开不限定步骤1231的实施方式，例如，其可以采用下文公式(5)进行实施。
式(5)中，Lhypernetwork表示超网络的损失表征数据；表示教 师生成器的损失表征数据；表示学生生成器的损失表征数据。
步骤1232：依据超网络的损失表征数据，更新该超网络。
本公开中，在获取到超网络的损失表征数据之后，可以依据该超网络的损失表征数据，更新该超网络(尤其是，更新该超网络的深度和/或宽度)，以使更新后的超网络具有更好的图像生成性能。需要说明的是，本公开不限定该超网络的更新方式。
步骤1233：从更新后的超网络中确定教师生成器以及学生生成器，以得到更新后的教师生成器以及更新后的学生生成器。
本公开中，在获取到更新后的超网络之后，可以从该更新后的超网络中重新采样一些子网络(比如，最大子网络、最小子网络以及随机子网络)，并基于这些采样所得子网络，确定更新后的教师生成器以及更新后的学生生成器(比如，将该最大子网络确定为更新后的教师生成器，并将该最小子网络以及随机子网络确定为更新后的学生生成器)。
基于上文步骤1231至步骤1233的相关内容可知，在获取到教师生成器的损失表征数据和学生生成器的损失表征数据之后，可以基于这两项损失确定上文超网络的损失表征数据，以使该损失表征数据能够比较好的表示出该超网络的图像生成性能；再基于该超网络的损失表征数据，更新该超网络、教师生成器以及学生生成器，以使更新后的超网络、更新后的教师生成器以及更新后的学生生成器均具有较好的图像生成性能。
基于上文步骤121至步骤123的相关内容可知，在一些应用场景下，可以借助半监督方式训练超网络，其过程大致为：在一些有标签的训练数据集上确定该超网络中的最大子网络的损失，并利用该最大子网络在一些无标签的训练数据集上指导确定该超网络中一些其他子网络(比如，最小子网络和随机子网络等)的损失，以便后续能够基于这两种损失优化超网络，如此能够借助这种由一些网络尺寸比较小的子网络与最大子网络之间的互助方式优化该超网络。
实际上，为了更好地提高超网络的图像生成性能，可以通过多次迭代方式完成针对该超网络的训练过程。基于此，本公开还提供了上文步骤12的另一种可能的实施方式，在该实施方式中，当上文训练数据集包括第一样本图像、该第一样本图像对应的标签信息、以及第二样本图像时，该步骤12不仅 包括上文步骤121-步骤123，可以还包括下文步骤124。
步骤124：判断是否达到第一停止条件，若是，则结束针对超网络的训练过程；若否，则返回继续执行上文步骤121及其后续步骤。
其中，第一停止条件是指上文超网络的训练过程结束时所需达到的条件；而且本公开不限定该第一停止条件，例如，其可以是该超网络的损失表征数据低于第一阈值，或者该超网络的损失表征数据的变化率低于第二阈值，或者该超网络的更新次数达到第三阈值。该第一阈值、第二阈值以及该第三阈值均可以预先设定。
另外，本公开不限定上文步骤124的执行时间，例如，该步骤124的执行时间可以晚于上文步骤123的执行时间。
基于上文步骤121至步骤124的相关内容可知，在一些应用场景下，可以借助半监督方式+迭代方式训练超网络，如此能够借助这种由该超网络中一些网络尺寸比较小的子网络与最大子网络之间的互助+渐进方式不断地优化该超网络，从而使得已训练的超网络具有较好的图像生成性能，从而使得已训练的超网络中各个子网络也具有较好的图像生成性能。
在一些应用场景下，可以借助针对教师生成器以及学生生成器进行交替更新的方式实现针对超网络的训练过程。基于此，本公开还提供了上文步骤12的又一种可能的实施方式，在该实施方式中，当上文训练数据集包括第一样本图像、该第一样本图像对应的标签信息、以及第二样本图像时，该步骤12具体可以包括下文步骤21-步骤24。
步骤21：利用第一样本图像和该第一样本图像对应的标签信息，更新教师生成器。
需要说明的是，本公开不限定步骤21的实施方式，例如，其具体可以为：先利用第一样本图像和该第一样本图像对应的标签信息，确定教师生成器的损失表征数据；再依据该教师生成器的损失表征数据，更新该教师生成器，以使更新后的教师生成器具有更好地性能。
实际上，在一些应用场景下，可以超网络可以随着教师生成器的更新也进行更新。基于此，本公开不限定上文步骤21的实施方式，例如，其具体可以为：利用第一样本图像和该第一样本图像对应的标签信息，更新教师生成器和超网络。也就是，在一种可能的实施方式下，可以先利用第一样本图像 和该第一样本图像对应的标签信息，确定教师生成器的损失表征数据；再依据该教师生成器的损失表征数据，更新该超网络，以使更新后的具有更好的模型性能；然后，从更新后的超网络中采样出一个子网络(比如，最大子网络)，确定为更新后的教师生成器，如此能够实现在更新教师生成器的同时也更新超网络。
步骤22：利用第二样本图像和该第二样本图像对应的第二伪标签，确定学生生成器的损失表征数据；该第二伪标签是由更新后的教师生成器针对该第二样本图像进行处理所得到的。
其中，第二样本图像对应的第二伪标签是指由更新后的教师生成器针对该第二样本图像的处理结果，以便后续能够将该处理结果作为该第二样本图像的伪标签指导学生生成器的优化过程，如此能够实现由更新后的教师生成器指导学生生成器的目的，从而能够实现在不依赖判别器的情况下完成针对该学生生成器的优化处理。
另外，步骤22的实施方式类似于上文步骤122的实施方式，为了简要起见，在此不再赘述。
步骤23：判断是否达到第二停止条件，若是，则结束针对超网络的训练过程；若否，则根据学生生成器的损失表征数据，更新超网络、教师生成器以及学生生成器，并返回继续执行上文步骤21及其后续步骤。
其中，第二停止条件是指上文超网络的训练过程结束时所需达到的条件；而且本公开不限定该第二停止条件，例如，其可以是该超网络的损失表征数据低于第一阈值，或者该超网络的损失表征数据的变化率低于第二阈值，或者该超网络的更新次数达到第三阈值。该第一阈值、第二阈值以及该第三阈值均可以预先设定。
另外，本公开不限定上文步骤“根据学生生成器的损失表征数据，更新超网络、教师生成器以及学生生成器”的实施方式，例如，其可以采用上文步骤123所示的实施方式进行实施。又如，如果超网络会随时教师生成器的更新也进行更新，则为了更好地提高模型更新效率，则该更新过程具体可以包括：先根据学生生成器的损失表征数据，更新超网络；再从更新后的超网络中采样出一些子网络(比如，最大子网络、最小子网络以及随机子网络)，并基于这些子网络确定出更新后的教师生成器和更新后的学生生成器。
基于上文步骤21至步骤23的相关内容可知，在一些应用场景下，可以借助另一种半监督方式训练超网络，其过程大致为：先在一些有标签的训练数据集上更新该超网络中的最大子网络，再利用更新后的最大子网络在一些无标签的训练数据集上指导该超网络中一些其他子网络(比如，最小子网络和随机子网络等)的训练过程，以使这些其他子网络能够从这些无标签的训练数据中更好地吸收知识并通过权重共享机制进一步地优化该超网络，如此能够借助这种由一些网络尺寸比较小的子网络与最大子网络之间的交替更新方式优化该超网络。
基于上文步骤11至步骤12的相关内容可知，在构建出一个深度和宽度可以动态变化的超网络之后，可以通过针对该超网络中最大子网络进行监督训练，并在该最大子网络指导下，完成该超网络中一些其他子网络(比如，最小子网络和随机子网络等)在一些无标签的训练数据上的训练过程，以使这些其他子网络能够从这些无标签的训练数据中吸收知识，如此能够借助这种由该超网络中一些网络尺寸比较小的子网络与最大子网络之间的互助策略以及渐进式协作的方式不断地优化该超网络，从而使得已训练的超网络具有较好的图像生成性能，从而使得已训练的超网络中各个子网络也具有较好的图像生成性能。
S103：从已训练的超网络中搜索教师模型和学生模型。
其中，教师模型是指从已训练的超网络中搜索到的一个子网络，以使该教师模型能够指导学生模型的训练过程。
学生模型是指从已训练的超网络中搜索到的另一个子网络，以使该学生模型的结构比教师模型的结构更简单。
另外，本公开不限定上文S103的实施方式，例如，其具体可以包括下文步骤31-步骤32。
步骤31：依据第一模型约束，从已训练的超网络中搜索学生模型，以使该学生模型满足第一模型约束。
其中，第一模型约束是指从已训练的超网络中搜索学生模型时所需依据的约束信息；而且该第一模型约束可以预先依据应用场景(比如，预先依据下文图像生成模型所部署的终端设备的配置信息等)所设定的约束信息，比如，其可以是模型运行时长低于预先设定的第四阈值。
另外，本公开不限定步骤31的实施方式，例如，其具体可以为：依据第一模型约束，利用进化搜索算法从已训练的超网络中搜索学生模型，以使该学生模型满足第一模型约束。需要说明的是，本公开不限定该进化搜索算法的实施方式。
步骤32：依据第二模型约束，从已训练的超网络中搜索至少一个教师模型，以使各教师模型均满足第二模型约束，该至少一个教师模型满足预设模型架构条件。
其中，第二模型约束是指从已训练的超网络中搜索教师模型时所需依据的约束信息；而且该第二模型约束可以预先依据应用场景所设定的约束信息，比如，其可以是模型运行时长低于预先设定的第五阈值。
实际上，为了更好地提高在教师模型的指导下针对学生模型的训练效果，本公开还提供了上文第二模型约束的一种确定过程，例如，其具体可以为：根据上文第一模型约束，确定该第二模型约束，以使满足该第二模型约束的教师模型与满足该第一模型约束的学生模型在一个或者多个模型指标上满足预设倍数条件(比如，该教师模型的计算成本大约是该学生模型的计算成本的21倍等)。其中，模型指标可以预先设定，比如，其可以是计算成本。该预设倍数条件可以基于实际的应用场景确定。
上述步骤32中“至少一个教师模型”用于表示按照第二模型约束从已训练的超网络中搜索到的、用于指导学生模型的训练过程的教师模型；而且该至少一个教师模型满足预设模型架构条件，以使该至少一个教师模型能够在至少一个维度(比如，深度+宽度等维度)下指导学生模型的训练过程。需要说明的是，该“至少一个维度”是指预先针对学生模型所设定的、需要进行由教师模型进行指导的一个或者多个维度；而且本公开不限定该“至少一个维度”，比如，其可以包括深度和/或宽度。可见，在一种可能的实施方式下，可以借助用于模型深度指导的教师模型以及用于模型宽度指导的教师模型，参与针对该学生模型的训练过程，以使已训练的学生模型具有更好地性能。
其中，预设模型架构条件可以依据应用场景设定，而且本公开不限定该预设模型架构条件，比如，满足该预设模型架构条件的至少一个教师模型的模型架构能够呈现出互补状态。
另外，本公开不限定上段中预设模型架构条件的实施方式，比如，该预 设模型架构条件可以包括预设模型深度条件和预设模型宽度条件，以使满足该预设模型架构条件的至少一个教师模型包括满足预设模型深度条件的教师模型(比如，上文“用于模型深度指导的教师模型”)和满足预设模型宽度条件的教师模型(比如，上文“用于模型宽度指导的教师模型”)，从而使得该至少一个教师模型呈现出架构互补状态。其中，该预设模型深度条件是指当搜索用于在模型深度下指导学生模型的训练过程的教师模型时所需依据的条件；而且本公开不限定该预设模型深度条件，比如，该预设模型深度条件可以预先依据实际的应用场景设定，以便后续能够利用满足该预设模型深度条件的教师模型，在模型深度这一维度下指导学生模型的训练过程。该预设模型宽度条件是指当搜索用于在模型宽度下指导学生模型的训练过程的教师模型时所需依据的条件；而且本公开不限定该预设模型宽度条件，比如，该预设模型宽度条件可以预先依据实际的应用场景设定，以便后续能够利用满足该预设模型宽度条件的教师模型，在模型宽度这一维度下指导学生模型的训练过程。
基于上文步骤31至步骤32的相关内容可知，对于一些应用场景来说，在获取到已训练的超网络之后，可以利用进化搜索算法，从该超网络中搜索出一个基于深度的教师模型、一个基于宽度的教师模型、以及一个学生模型，以便后续能够借助该基于深度的教师模型指导该学生模型在深度方面的优化调整，并借助该基于宽度的教师模型指导该学生模型在宽度方面的优化调整。其中，因已训练的超网络具有较好的图像生成性能，使得该超网络中各个子网络也具有较好的图像生成性能，从而使得从该超网络中搜索所得的两个教师模型以及学生模型均具有较好的性能，如此能够有效地避免从头开始优化模型所导致的不良影响(比如，模型收敛速度比较慢)，从而能够有效地提高模型训练效果(比如，加快模型收敛速度)。其中，该基于深度的教师模型是指上文满足预设模型深度条件的教师模型，以使该基于深度的教师模型用于在模型深度这一维度下指导学生模型的训练过程。该基于宽度的教师模型是指上文满足预设模型深度条件的教师模型，以使该基于宽度的教师模型用于在模型宽度这一维度下指导学生模型的训练过程。
还因已训练的超网络能够提供出符合不同约束的子网络，使得本公开可以利用该超网络确定出满足不同约束的各个教师模型以及学生模型，如此能 够有效地避免因手动设计多个教师模型以及学生模型所导致的不良影响(比如，时耗比较长等)，从而有利于提高模型构建效果。
S104：依据训练数据集、教师模型以及学生模型进行模型蒸馏，得到图像生成模型。
其中，图像生成模型用于针对该图像生成模型的输入数据进行图像生成；而且本公开不限定该图像生成模型的实施方式，例如，在一些应用场景下，该图像生成模型属于图像生成器，以使该图像生成模型能够基于该图像生成模型的输入数据生成一个新的图像数据。
另外，本公开不限定S104中模型蒸馏的实施方式，例如，其可以采用现有的或者未来出现的任意一种模型蒸馏方法(比如，在线蒸馏方法)进行实施。
实际上，为了更好地降低模型蒸馏过程针对标签信息的依赖，本公开还提供了上文S104的一种可能的实施方式，在该实施方式中，当上文训练数据集包括第一样本图像、该第一样本图像对应的标签信息、以及第二样本图像时，该S104具体可以包括下文步骤41-步骤45。
步骤41：利用第一样本图像和该第一样本图像对应的标签信息，更新教师模型。
需要说明的是，本公开不限定步骤41的实施方式，例如，其具体可以为：先利用第一样本图像和该第一样本图像对应的标签信息，确定教师模型的损失表征数据(例如，图2所示的监督损失2)；再依据该教师模型的损失表征数据，更新该教师模型，以使更新后的教师模型具有更好地性能。
步骤42：利用第二样本图像和该第二样本图像对应的第三伪标签，更新学生模型；该第三伪标签是由更新后的教师模型针对该第二样本图像进行处理所得到的。
其中，第二样本图像对应的第三伪标签是指由更新后的教师模型针对该第二样本图像的处理结果，以便后续能够将该处理结果作为该第二样本图像的伪标签指导学生模型的优化过程，如此能够实现由更新后的教师模型指导学生模型的目的，从而能够实现在不依赖判别器的情况下完成针对该学生模型的优化处理。
另外，本公开不限定步骤42的实施方式，例如，其可以采用现有的或者 未来出现的任意一种模型蒸馏方法(比如，在线蒸馏方法)所涉及的学生模型更新过程进行实施。可见，在一种可能的实施方式下，该步骤42具体可以为：先利用第二样本图像和该第二样本图像对应的第三伪标签，确定该学生模型的损失表征数据；再基于该学生模型的损失表征数据，更新该学生模型。需要说明的是，该学生模型的损失表征数据的确定过程类似于上文学生生成器的损失表征数据的确定过程，为了简要起见，在此不再赘述。
实际上，因针对学生模型的更新过程无需使用真实的标签信息，故为了更好地提高该学生模型的训练效果，本公开还提供了上文步骤42的另一种可能的实施方式，其具体可以为：利用第一样本图像、该第一样本图像对应的第四伪标签、第二样本图像以及该第二样本图像对应的第三伪标签，更新学生模型，如此能够有效地提高该学生模型的训练数据的个数，从而有利于提高该学生模型的训练效果；该第四伪标签是由更新后的教师模型针对该第一样本图像进行处理所得到的。
此外，本公开不限定上文学生模型的更新过程，例如，其具体可以包括下文步骤421-步骤423。
步骤421：利用第二样本图像和该第二样本图像对应的第三伪标签，确定学生模型的第一蒸馏损失。
其中，第一蒸馏损失用于表征学生模型在无标签的训练数据上所呈现的损失。
另外，本公开不限定该第一蒸馏损失的确定过程，例如，当上文“至少一个教师模型”包括基于深度的教师模型和基于宽度的教师模型时，该第一蒸馏损失可以采用下文公式(6)进行实施。
式(6)中，表示学生模型的第一蒸馏损失；G*表示学生模型；表示在基于深度的教师模型的指导下，该学生模型在无标签的训练数据上所呈现的在线蒸馏损失；表示在基于宽度的教师模型的指导下，该学生模型在无标签的训练数据上所呈现的在线蒸馏损失；表示无标签的训练数据；表示由该基于深度的教师模型处理该xu时所生成的图像数据；表示由该学生模型处理该xu时所生成的图像数据；表示由该基于宽度的教师模型处理该xu时所生成 的图像数据。
步骤422：根据学生模型的第一蒸馏损失，确定该学生模型的损失表征数据。
需要说明的是，本公开不限定该步骤422的实施方式，例如，其具体可以为：直接将该学生模型的第一蒸馏损失，确定为该学生模型的损失表征数据。
又如，当上文学生模型借助有标签的训练数据以及无标签的训练数据进行训练时，该学生模型的损失表征数据的确定过程，可以包括下文步骤51-步骤52。
步骤51：利用第一样本图像和该第一样本图像对应的第四伪标签，确定学生模型的第二蒸馏损失。
其中，第一样本图像对应的第四伪标签是指由更新后的教师模型针对该第一样本图像的处理结果，以便后续能够将该处理结果作为该第一样本图像的伪标签指导学生模型的优化过程，如此能够实现由更新后的教师模型指导学生模型的目的，从而能够实现在不依赖判别器的情况下完成针对该学生模型的优化处理。
第二蒸馏损失用于表征学生模型在有标签的训练数据上所呈现的损失。
另外，本公开不限定该第二蒸馏损失的确定过程，例如，当上文“至少一个教师模型”包括基于深度的教师模型和基于宽度的教师模型时，该第二蒸馏损失可以采用下文公式(7)进行实施。
式(7)中，表示学生模型的第二蒸馏损失；G*表示学生模型；表示在基于深度的教师模型的指导下，该学生模型在有标签的训练数据上所呈现的在线蒸馏损失；表示在基于宽度的教师模型的指导下，该学生模型在有标签的训练数据上所呈现的在线蒸馏损失；表示有标签的训练数据；表示由该基于深度的教师模型处理该时所生成的图像数据；表示由该学生模型处理该时所生成的图像数据；表示由该基于宽度的教师模型处理该时所生成的图像数据。
需要说明的是，本公开不限定步骤51与上文步骤421之间的执行顺序。
步骤52：根据学生模型的第一蒸馏损失以及该学生模型的第二蒸馏损失，确定该学生模型的损失表征数据。
需要说明的是，本公开不限定上文步骤52的实施方式，例如，其具体可以为：将该学生模型的第一蒸馏损失以及该学生模型的第二蒸馏损失进行加和，得到该学生模型的损失表征数据。
基于上文步骤51至步骤52的相关内容可知，在一些应用场景下，可以先在多个教师模型的指导下，确定学生模型在无标签的训练数据所呈现的蒸馏损失以及该学生模型在有标签的训练数据所呈现的蒸馏损失；再综合这两种损失，确定该学生模型的损失表征数据，以使该损失表征数据能够更好地表示出该学生模型所具有的性能。
实际上，因教师模型的图像生成能力可能会发生波动，导致基于该教师模型所生成的某些伪标签的可信性不大，故为了避免这些可信性不大的伪标签对学生模型的更新过程造成干扰，可以尽可能的削弱这些可信性不大的伪标签对该学生模型的更新过程所造成的影响。基于此，本公开还提供了上文步骤422的一种可能的实施方式，其具体可以为：根据该学生模型的第一蒸馏损失以及该第一蒸馏损失对应的加权权重，确定该学生模型的损失表征数据。其中，该加权权重是根据该第三伪标签的置信度所确定的；该第三伪标签的置信度是根据更新后的教师模型、更新后的教师模型对应的判别器以及第二样本图像所确定的。
其中，第三伪标签的置信度用于表征该第三伪标签的可信性；而且本公开不限定该第三伪标签的置信度的确定过程，例如，其具体可以为：先将第二样本图像输入更新后的教师模型，得到该教师模型输出的第三伪标签；再由该更新后的教师模型对应的判别器针对该第三伪标签进行判别处理，得到该第三伪标签的判别结果，并将该判别结果确定为该第三伪标签的置信度。
第一蒸馏损失对应的加权权重用于表征该第一蒸馏损失的影响程度；而且本公开不限定该第一蒸馏损失对应的加权权重的确定过程，例如，其具体可以包括下文步骤61-步骤62。
步骤61：若第三伪标签的置信度低于置信度阈值，则从预设区间中随机选择一个数值，确定为上文第一蒸馏损失对应的加权权重。
其中，置信度阈值用于描述可信与不可信的划分边界；而且本公开不限 定该置信度阈值的确定过程，例如，其可以依据应用场景预先设定。
另外，为了更好地提高权重合理性，本公开还提供了一种自动地确定置信度阈值的方式，其具体可以包括下文步骤71-步骤73。
步骤71：利用更新后的教师模型对至少一个图像数据进行处理，得到各个图像数据对应的生成图像；该至少一个图像数据包括第二样本图像和/或第一样本图像。
其中，至少一个图像数据是指在自动地确定置信度阈值时所需参考的图像数据；而且本公开不限定该至少一个图像数据的实施方式，例如，其可以包括上文第二样本图像和/或第一样本图像。
步骤72：利用更新后的教师模型对应的判别器对各生成图像进行判别处理，得到各生成图像对应的判别结果。
其中，更新后的教师模型对应的判别器是指在针对该教师模型进行更新处理时所涉及的判别器。
步骤73：针对至少一个生成图像对应的判别结果进行数据分析处理，得到置信度阈值。
其中，数据分析处理是指一种预先设定的数据分析方式；而且本公开不限定该数据分析处理的实施方式，例如，其可以采用现有的或者未来出现的任意一种数据分析方式(例如，最大值、最小值、均值、中值、或者指数移动平均值等)进行实施。可见，在一种可能的实施方式下，该置信度阈值可以是指针对所有生成图像对应的判别结果进行指数移动平均(Exponential Moving Average，EMA)分析所得到的。
基于上文步骤71至步骤73的相关内容可知，在一些应用场景下，在获取到更新后的教师模型之后，可以利用该教师模型针对训练数据集中各个样本图像进行处理，以得到各个样本图像对应的生成图像；再利用该教师模型对应的判别器对各个生成图像进行判别处理，得到各个生成图像的判别结果；最后，针对所有这些判别结果进行分析，以得到置信度阈值，以使该置信度阈值能够更好地描述出可信与不可信之间的边界线。
预设区间是指预先设定的数值选择区间，以使上文“预设区间中随机选择一个数值”大于或者等于该预设区间中的最小值，并使得该“预设区间中随机选择一个数值”小于或者等于该预设区间中的最大值；而且本公开不限定 该预设区间，例如，该预设区间中的任意一个数值均小于或者等于预先设定的第一数值(比如，1等)。又如，该预设区间中的任意一个数值均大于或者等于预先设定的第二数值(比如，0等)。还如，当该第二数值小于该第一数值时，该预设区间可以为[第二数值，第一数值]这一数据区间(比如，[0，1]这一数据区间)。
基于上文步骤61的相关内容可知，对于第二样本图像对应的第三伪标签来说，如果该第三伪标签的置信度低于置信度阈值，可以确定该第三伪标签的可信性不太高，故为了削弱该第三伪标签对学生模型的更新过程所造成的影响，可以从预设区间随机选择一个数值(比如，从0和1之间随机选择一个数值)，作为该第三伪标签对应的加权权重，以使该加权权重能够表示出基于该第三伪标签所确定的损失对该学生模型的更新过程所造成的影响。
步骤62：若第三伪标签的置信度不低于置信度阈值，则将预设数值确定为上文第一蒸馏损失对应的加权权重。
本公开中，对于第二样本图像对应的第三伪标签来说，如果该第三伪标签的置信度高于或者等于置信度阈值，可以确定该第三伪标签的可信性比较高，故为了保持甚至提高该第三伪标签对学生模型的更新过程所造成的影响，可以直接将预设数值(比如，1)，作为该第三伪标签对应的加权权重，以使该加权权重能够表示出基于该第三伪标签所确定的损失对该学生模型的更新过程所造成的影响。
需要说明的是，本公开不限定上文预设数值的实施方式，例如，该预设数值不低于上文预设区间内的最高值。又如，该预设数值可以为1。
基于上文步骤61至步骤62的相关内容可知，在一些应用场景下，当上文第一蒸馏损失是根据第二样本图像对应的第三伪标签所确定时，该第一蒸馏损失对应的加权权重可以依据该第三伪标签的置信度进行确定，如此能够实现削弱具有较低置信度的伪标签针对学生模型的更新过程所造成的影响，从而能够更好地提高模型更新效果。
另外，本公开不限定上文步骤“根据该学生模型的第一蒸馏损失以及该第一蒸馏损失对应的加权权重，确定该学生模型的损失表征数据”的实施方式，例如，其可以采用下文公式(8)-(10)进行实施。


式(8)-(10)中，表示学生模型的第一蒸馏损失；G*表示学生模型；表示在基于深度的教师模型的指导下，该学生模型在无标签的训练数据上所呈现的在线蒸馏损失；表示无标签的训练数据；表示由该基于深度的教师模型处理该xu时所生成的图像数据；表示由该学生模型处理该xu时所生成的图像数据；表示对应的加权权重；表示在基于宽度的教师模型的指导下，该学生模型在无标签的训练数据上所呈现的在线蒸馏损失；表示对应的加权权重；表示由该基于宽度的教师模型处理该xu时所生成的图像数据；表示由基于深度的教师模型处理对应的判别器针对这一生成图像所给出的判定结果(比如，预测分数)；表示由基于宽度的教师模型对应的判别器针对这一生成图像所给出的判定结果；表示由基于深度的教师模型对应的判别器针对由该基于深度的教师模型输出的生成图像给出的判定结果的EMA值；表示由基于宽度的教师模型对应的判别器针对由该基于宽度的教师模型输出的生成图像给出的判定结果的EMA值；RandInt([0,1])表示从0到1这一区间随机选择一个数值。
基于上文步骤422的相关内容可知，在一些应用场景下，在获取到学生模型的第一蒸馏损失之后，可以依据该第一蒸馏损失，确定该学生模型的损失表征数据，以使该损失表征数据能够更好地表示出该学生模型的图像生成性能。
步骤423：根据学生模型的损失表征数据，更新学生模型。
需要说明的是，本公开不限定上文步骤423的实施方式。
基于上文步骤421至步骤423的相关内容可知，在一种可能的实施方式下，在获取到更新后的多个教师模型之后，可以依据所有训练数据(比如，第一样本图像和第二样本图像)并且利用这些教师模型指导学生模型的更新过程，以使更新后的学生模型具有更好的图像生成性能。
基于上文步骤42的相关内容可知，由于针对学生模型的更新过程无需使用真实的标签信息，故可以利用所有的训练数据(比如，第一样本图像和第二样本图像)参与针对该学生模型的优化过程，如此能够更好地提高模型优化效果。
步骤43：判断是否达到第三停止条件，若是，则执行下文步骤45；若否，则执行下文步骤44。
其中，第三停止条件是指上文学生模型的训练过程结束时所需达到的条件；而且本公开不限定该第三停止条件，例如，其可以是该学生模型的损失表征数据低于第六阈值，或者该学生模型的损失表征数据的变化率低于第七阈值，或者该学生模型的更新次数达到第八阈值。该第六阈值、第七阈值以及该第八阈值均可以预先设定。
步骤44：若未达到第三停止条件，则返回继续执行上文步骤41及其后续步骤。
本公开中，对于针对学生模型的当前轮训练过程来说，如果确定没有达到第三停止条件，则可以确定该学生模型的性能仍需改善，故可以基于更新后的学生模型，重新执行上文步骤41及其后续步骤，以开启针对该学生模型的下一轮训练过程。
步骤45：若达到第三停止条件，则根据学生模型，确定图像生成模型。
本公开中，对于针对学生模型的当前轮训练过程来说，如果确定达到第三停止条件，则可以确定该学生模型的性能比较好，故可以依据该学生模型确定图像生成模型(比如，直接将该学生模型确定为图像生成模型等)，以使该图像生成模型能够具有较好的性能。
基于上文步骤41至步骤45的相关内容可知，在从已训练的超网络中搜索出多个教师模型以及一个学生模型之后，可以通过教师模型与学生模型进行交替更新的方式，实现针对该学生模型的训练过程，以使最终已训练的学生模型具有较好的性能。
基于上文S101至S104的相关内容可知，对于本公开实施例提供的模型构建方法来说，在获取到训练数据集(比如，一些具有标签信息的图像样本和一些不具有标签信息的图像样本)之后，先利用该训练数据集对超网络进行训练，以使已训练的超网络中的各个子网络均具有较好的图像生成性能 (比如，图像生成性能等)；再从已训练的超网络中搜索教师模型和学生模型，以使该教师模型与该学生模型也具有较好的图像生成性能；最后，依据该训练数据集、该教师模型以及该学生模型进行模型蒸馏，得到图像生成模型，以使该图像生成模型不仅具有较好的图像生成性能，而且该图像生成模型的计算复杂性比较低，从而使得该图像生成模型在确保性能的前提下具有比较高的模型效率(比如，在部署该图像生成模型时所需的资源比较少等)，从而使得该图像生成模型可以部署在硬件受限的设备上。
另外，因已训练的超网络中的各个子网络均具有较好的图像生成性能，使得从该超网络中搜索所得的教师模型和学生模型均具有比较好的图像生成性能，从而使得后续只需针对该教师模型和学生模型进行微调处理即可，如此能够有效地避免从头开始优化教师模型和学生模型所导致的不良影响(比如，模型收敛速度比较慢等)，从而有利于提高模型训练效率。
另外，因超网络能够提供各种各样的大量的网络结构，以使本公开可以从该超网络中搜索到满足不同约束的学生模型和教师模型，如此能够有效地消除因手动设计和配置教师模型而带来的额外资源消耗(比如，人力资源消耗、时间消耗等)，从而有利于节省资源。
另外，对于本公开提供的模型构建方法来说，因该方法可以通过依次执行半监督驱动的网络架构搜索阶段和自适应在线半监督蒸馏阶段的方式，实现无缝地促进模型高效和数据高效学习的协同作用，如此能够达到针对网络的数据+模型进行统一压缩的目的。
此外，本公开提供了一种新的学习范式，该学习范式具有统一的优化目标，可以无缝地促进模型高效和数据高效学习的协同作用。在该学习范式中，可以依次设置半监督驱动的网络架构搜索和自适应在线半监督蒸馏阶段，制定异构互学习方案以获得架构灵活、数据高效、性能优异的模型。大量实验表明，即使标签少于，该学习范式也可以获得最先进的轻量级模型，以使该学习范式能够为在硬件受限和低数据环境中部署GAN模型提供了可行的解决方案。
此外，本公开不限定上文模型构建方法的执行主体，例如，本公开实施例提供的模型构建方法可以应用于终端设备或服务器等具有数据处理功能的设备。又如，本公开实施例提供的模型构建方法也可以借助不同设备(例 如，终端设备与服务器、两个终端设备、或者两个服务器)之间的数据通信过程进行实现。其中，终端设备可以为智能手机、计算机、个人数字助理(Personal Digital Assistant，PDA)或平板电脑等。服务器可以为独立服务器、集群服务器或云服务器。
基于上文模型构建方法的相关内容，本公开还提供了一种图像生成方法，下面结合附图进行说明。如图3所示，本公开实施例提供的图像生成方法包括下文S301-S302。其中，该图3为本公开实施例提供的一种图像生成方法的流程图。
S301：获取待处理图像。
其中，待处理图像是指需要借助上文图像生成模型进行图像生成的图像数据。
S302：利用图像生成模型对待处理图像进行处理，得到图像生成结果；该图像生成模型是利用本公开提供的模型构建方法的任一实施方式所构建的。
其中，图像生成模型的相关内容请参见上文。
图像生成结果是指由图像生成模型针对上文待处理图像进行处理所输出的结果。
基于上文S301至S302的相关内容可知，在获取到待处理图像之后，可以利用预先构建的图像生成模型对待处理图像进行处理，得到图像生成结果。其中，因该图像生成模型具有较好的性能，使得基于该图像生成模型所确定的图像生成结果比较好，如此有利于提高图像生成效果。
另外，本公开不限定上文图像生成方法的执行主体，例如，本公开实施例提供的图像生成方法可以应用于终端设备或服务器等具有数据处理功能的设备。又如，本公开实施例提供的图像生成方法也可以借助不同设备(例如，终端设备与服务器、两个终端设备、或者两个服务器)之间的数据通信过程进行实现。
基于本公开实施例提供的模型构建方法，本公开实施例还提供了一种模型构建装置，下面结合图4进行解释和说明。其中，图4为本公开实施例提供的一种模型构建装置的结构示意图。需要说明的是，本公开实施例提供的模型构建装置的技术详情，请参照上文模型构建方法的相关内容。
如图4所示，本公开实施例提供的模型构建装置400，包括：
第一获取单元401，用于获取训练数据集；所述训练数据集包括第一样本图像和所述第一样本图像对应的标签信息；
第一训练单元402，用于利用所述训练数据集对超网络进行训练；
网络搜索单元403，用于从已训练的超网络中搜索教师模型和学生模型；
第二训练单元404，用于依据所述训练数据集、所述教师模型以及所述学生模型进行模型蒸馏，得到图像生成模型。
在一种可能的实施方式下，所述第一训练单元402，包括：
第一确定子单元，用于从所述超网络中确定教师生成器和学生生成器；
模型蒸馏子单元，用于依据所述训练数据集、所述教师生成器和所述学生生成器进行模型蒸馏，得到已训练的超网络。
在一种可能的实施方式下，所述第一确定子单元，具体用于：从所述超网络中采样第一子网络和第二子网络；所述第一子网络的网络尺寸大于所述第二子网络的网络尺寸；根据所述第一子网络，确定所述教师生成器；根据所述第二子网络，确定所述学生生成器。
在一种可能的实施方式下，所述第一子网络包括最大子网络；所述第二子网络包括最小子网络和随机子网络中的至少一个。
在一种可能的实施方式下，所述训练数据集还包括第二样本图像；
所述模型蒸馏子单元，具体用于：利用所述第一样本图像和所述第一样本图像对应的标签信息，确定所述教师生成器的损失表征数据；利用所述第二样本图像和所述第二样本图像对应的第一伪标签，确定所述学生生成器的损失表征数据；所述第一伪标签是由所述教师生成器针对所述第二样本图像进行处理所得到的；根据所述教师生成器的损失表征数据和所述学生生成器的损失表征数据，更新所述超网络、所述教师生成器以及所述学生生成器。
在一种可能的实施方式下，所述模型蒸馏子单元，还用于：在所述更新所述超网络、所述教师生成器以及所述学生生成器之后，继续执行所述利用所述第一样本图像和所述第一样本图像对应的标签信息，确定所述教师生成器的损失表征数据的步骤，直至达到第一停止条件。
在一种可能的实施方式下，所述训练数据集还包括第二样本图像；
所述模型蒸馏子单元，具体用于：利用所述第一样本图像和所述第一样 本图像对应的标签信息，更新所述教师生成器；利用所述第二样本图像和所述第二样本图像对应的第二伪标签，确定所述学生生成器的损失表征数据；所述第二伪标签是由更新后的教师生成器针对所述第二样本图像进行处理所得到的；根据所述学生生成器的损失表征数据，更新所述超网络、所述教师生成器以及所述学生生成器，并继续执行所述利用所述第一样本图像和所述第一样本图像对应的标签信息，更新所述教师生成器的步骤，直至达到第二停止条件。
在一种可能的实施方式下，所述网络搜索单元403，具体用于：依据第一模型约束，从已训练的所述超网络中搜索所述学生模型，以使所述学生模型满足所述第一模型约束；依据所述第二模型约束，从已训练的所述超网络中搜索至少一个，以使各所述教师模型均满足所述第二模型约束，所述至少一个教师模型满足预设模型架构条件。
在一种可能的实施方式下，所述至少一个教师模型包括满足预设模型深度条件的教师模型和满足预设模型宽度条件的教师模型；和/或，所述第二模型约束是根据所述第一模型约束所确定的，以使各所述教师模型与所述学生模型在一个或者多个模型指标上满足预设倍数条件。
在一种可能的实施方式下，所述训练数据集还包括第二样本图像；
所述第二训练单元404，具体用于：利用所述第一样本图像和所述第一样本图像对应的标签信息，更新所述教师模型；利用所述第二样本图像和所述第二样本图像对应的第三伪标签，更新所述学生模型，并继续执行所述利用所述第一样本图像和所述第一样本图像对应的标签信息，更新所述教师模型的步骤，直至在达到第三停止条件时，根据所述学生模型，确定所述图像生成模型；所述第三伪标签是由更新后的教师模型针对所述第二样本图像进行处理所得到的。
在一种可能的实施方式下，所述第二训练单元404，具体用于：利用所述第一样本图像、所述第一样本图像对应的第四伪标签、所述第二样本图像以及所述第二样本图像对应的第三伪标签，更新所述学生模型；所述第四伪标签是由更新后的教师模型针对所述第一样本图像进行处理所得到的。
在一种可能的实施方式下，所述第二训练单元404，具体用于：利用所述第二样本图像和所述第二样本图像对应的第三伪标签，确定所述学生模型 的第一蒸馏损失；根据所述第一蒸馏损失，确定所述学生模型的损失表征数据；根据所述学生模型的损失表征数据，更新所述学生模型。
在一种可能的实施方式下，所述第二训练单元404，具体用于：根据所述第一蒸馏损失以及所述第一蒸馏损失对应的加权权重，确定所述学生模型的损失表征数据；所述加权权重是根据所述第三伪标签的置信度所确定的；所述置信度是根据更新后的教师模型、更新后的教师模型对应的判别器以及所述第二样本图像所确定的。
在一种可能的实施方式下，所述加权权重的确定过程，包括：若所述置信度低于置信度阈值，则从预设区间中随机选择一个数值，确定为所述加权权重；若所述置信度不低于所述置信度阈值，则将预设数值确定为所述加权权重。
在一种可能的实施方式下，所述置信度阈值的确定过程，包括：利用更新后的教师模型对至少一个图像数据进行处理，得到各个图像数据对应的生成图像；所述至少一个图像数据包括所述第二样本图像和/或所述第一样本图像；利用更新后的教师模型对应的判别器对各所述生成图像进行判别处理，得到各所述生成图像对应的判别结果；针对至少一个所述生成图像对应的判别结果进行数据分析处理，得到所述置信度阈值。
基于上述模型构建装置400的相关内容可知，对于本公开提供的模型构建装置400来说，在获取到训练数据集(比如，一些具有标签信息的图像样本和一些不具有标签信息的图像样本)之后，先利用该训练数据集对超网络进行训练，以使已训练的超网络中的各个子网络均具有较好的图像生成性能(比如，图像生成性能等)；再从已训练的超网络中搜索教师模型和学生模型，以使该教师模型与该学生模型也具有较好的图像生成性能；最后，依据该训练数据集、该教师模型以及该学生模型进行模型蒸馏，得到图像生成模型，以使该图像生成模型不仅具有较好的图像生成性能，而且该图像生成模型的计算复杂性比较低，从而使得该图像生成模型在确保性能的前提下具有比较高的模型效率(比如，在部署该图像生成模型时所需的资源比较少等)，从而使得该图像生成模型可以部署在硬件受限的设备上。
基于本公开实施例提供的图像生成方法，本公开实施例还提供了一种图像生成装置，下面结合图5进行解释和说明。其中，图5为本公开实施例提 供的一种图像生成装置的结构示意图。需要说明的是，本公开实施例提供的图像生成装置的技术详情，请参照上文图像生成方法的相关内容。
如图5所示，本公开实施例提供的图像生成装置500，包括：
第二获取单元501，用于获取待处理图像；
图像生成单元502，用于利用预先构建的图像生成模型对所述待处理图像进行处理，得到图像生成结果；所述图像生成模型是利用本公开实施例提供的模型构建方法的任一实施方式所构建的。
基于上述图像生成装置500的相关内容可知，对于本公开实施例提供的图像生成装置500来说，在获取到待处理图像之后，可以利用预先构建的图像生成模型对待处理图像进行处理，得到图像生成结果。其中，因该图像生成模型具有较好的性能，使得基于该图像生成模型所确定的图像生成结果比较好，如此有利于提高图像生成效果。
另外，本公开实施例还提供了一种电子设备，所述设备包括处理器以及存储器：所述存储器，用于存储指令或计算机程序；所述处理器，用于执行所述存储器中的所述指令或计算机程序，以使得所述电子设备执行本公开实施例提供的模型构建方法或者图像生成方法的任一实施方式。
参见图6，其示出了适于用来实现本公开实施例的电子设备600的结构示意图。本公开实施例中的终端设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。图6示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图6所示，电子设备600可以包括处理装置(例如中央处理器、图形处理器等)601，其可以根据存储在只读存储器(ROM)602中的程序或者从存储装置608加载到随机访问存储器(RAM)603中的程序而执行各种适当的动作和处理。在RAM603中，还存储有电子设备600操作所需的各种程序和数据。处理装置601、ROM 602以及RAM 603通过总线604彼此相连。输入/输出(I/O)接口605也连接至总线604。
通常，以下装置可以连接至I/O接口605：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置606；包括例如 液晶显示器(LCD)、扬声器、振动器等的输出装置607；包括例如磁带、硬盘等的存储装置608；以及通信装置609。通信装置609可以允许电子设备600与其他设备进行无线或有线通信以交换数据。虽然图6示出了具有各种装置的电子设备600，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在非暂态计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置609从网络上被下载和安装，或者从存储装置608被安装，或者从ROM602被安装。在该计算机程序被处理装置601执行时，执行本公开实施例的方法中限定的上述功能。
本公开实施例提供的电子设备与上述实施例提供的方法属于同一发明构思，未在本实施例中详尽描述的技术细节可参见上述实施例，并且本实施例与上述实施例具有相同的有益效果。
本公开实施例还提供了一种计算机可读介质，所述计算机可读介质中存储有指令或计算机程序，当所述指令或计算机程序在设备上运行时，使得所述设备执行本公开实施例提供的模型构建方法或者图像生成方法的任一实施方式。
需要说明的是，本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载 了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、RF(射频)等等，或者上述的任意合适的组合。
在一些实施方式中，客户端、服务器可以利用诸如HTTP(Hyper Text Transfer Protocol，超文本传输协议)之类的任何当前已知或未来研发的网络协议进行通信，并且可以与任意形式或介质的数字数据通信(例如，通信网络)互连。通信网络的示例包括局域网(“LAN”)，广域网(“WAN”)，网际网(例如，互联网)以及端对端网络(例如，ad hoc端对端网络)，以及任何当前已知或未来研发的网络。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备可以执行上述方法。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括但不限于面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行 指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，单元/模块的名称在某种情况下并不构成对该单元本身的限定。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上系统(SOC)、复杂可编程逻辑设备(CPLD)等等。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
需要说明的是，本说明书中各个实施例采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似部分互相参见即可。对于实施例公开的系统或装置而言，由于其与实施例公开的方法相对应，所以描述的比较简单，相关之处参见方法部分说明即可。
应当理解，在本公开中，“至少一个(项)”是指一个或者多个，“多个”是指两个或两个以上。“和/或”，用于描述关联对象的关联关系，表示可以存在三种关系，例如，“A和/或B”可以表示：只存在A，只存在B以及同时存在A和B三种情况，其中A，B可以是单数或者复数。字符“/”一般表示前后关 联对象是一种“或”的关系。“以下至少一项(个)”或其类似表达，是指这些项中的任意组合，包括单项(个)或复数项(个)的任意组合。例如，a，b或c中的至少一项(个)，可以表示：a，b，c，“a和b”，“a和c”，“b和c”，或“a和b和c”，其中a，b，c可以是单个，也可以是多个。
还需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
结合本文中所公开的实施例描述的方法或算法的步骤可以直接用硬件、处理器执行的软件模块，或者二者的结合来实施。软件模块可以置于随机存储器(RAM)、内存、只读存储器(ROM)、电可编程ROM、电可擦除可编程ROM、寄存器、硬盘、可移动磁盘、CD-ROM、或技术领域内所公知的任意其它形式的存储介质中。
对所公开的实施例的上述说明，使本领域专业技术人员能够实现或使用本公开。对这些实施例的多种修改对本领域的专业技术人员来说将是显而易见的，本文中所定义的一般原理可以在不脱离本公开的精神或范围的情况下，在其它实施例中实现。因此，本公开将不会被限制于本文所示的这些实施例，而是要符合与本文所公开的原理和新颖特点相一致的最宽的范围。

Claims (19)

  1. 一种模型构建方法，包括：
    获取训练数据集；其中，所述训练数据集包括第一样本图像和所述第一样本图像对应的标签信息；
    利用所述训练数据集对超网络进行训练；
    从已训练的超网络中搜索教师模型和学生模型；以及
    依据所述训练数据集、所述教师模型以及所述学生模型进行模型蒸馏，得到图像生成模型。
  2. 根据权利要求1所述的方法，其中，所述超网络的训练过程，包括：
    从所述超网络中采样第一子网络和第二子网络；其中，所述第一子网络的网络尺寸大于所述第二子网络的网络尺寸；
    根据所述第一子网络，确定教师生成器；
    根据所述第二子网络，确定学生生成器；
    依据所述训练数据集、所述教师生成器和所述学生生成器进行模型蒸馏，得到所述已训练的超网络。
  3. 根据权利要求2所述的方法，其中，所述第一子网络包括最大子网络；
    所述第二子网络包括最小子网络和随机子网络中的至少一个。
  4. 根据权利要求2所述的方法，其中，所述训练数据集还包括第二样本图像；
    所述依据所述训练数据集、所述教师生成器和所述学生生成器进行模型蒸馏，得到所述已训练的超网络，包括：
    利用所述第一样本图像和所述第一样本图像对应的标签信息，确定所述教师生成器的损失表征数据；
    利用所述第二样本图像和所述第二样本图像对应的第一伪标签，确定所述学生生成器的损失表征数据；其中，所述第一伪标签是由所述教师生成器针对所述第二样本图像进行处理所得到的；
    根据所述教师生成器的损失表征数据和所述学生生成器的损失表征数据，更新所述超网络、所述教师生成器以及所述学生生成器。
  5. 根据权利要求4所述的方法，其中，在所述更新所述超网络、所述教 师生成器以及所述学生生成器之后，所述方法还包括：
    继续执行所述利用所述第一样本图像和所述第一样本图像对应的标签信息，确定所述教师生成器的损失表征数据的步骤，直至达到第一停止条件。
  6. 根据权利要求2所述的方法，其中，所述训练数据集还包括第二样本图像；
    所述依据所述训练数据集、所述教师生成器和所述学生生成器进行模型蒸馏，得到所述已训练的超网络，包括：
    利用所述第一样本图像和所述第一样本图像对应的标签信息，更新所述教师生成器；
    利用所述第二样本图像和所述第二样本图像对应的第二伪标签，确定所述学生生成器的损失表征数据；其中，所述第二伪标签是由更新后的教师生成器针对所述第二样本图像进行处理所得到的；
    根据所述学生生成器的损失表征数据，更新所述超网络、所述教师生成器以及所述学生生成器，并继续执行所述利用所述第一样本图像和所述第一样本图像对应的标签信息，更新所述教师生成器的步骤，直至达到第二停止条件。
  7. 根据权利要求1所述的方法，其中，所述从已训练的超网络中搜索教师模型和学生模型，包括：
    依据第一模型约束，从所述已训练的超网络中搜索所述学生模型，以使所述学生模型满足所述第一模型约束；
    依据第二模型约束，从所述已训练的超网络中搜索至少一个教师模型，以使各所述教师模型均满足所述第二模型约束，所述至少一个教师模型满足预设模型架构条件。
  8. 根据权利要求7所述的方法，其中，所述至少一个教师模型包括满足预设模型深度条件的教师模型和满足预设模型宽度条件的教师模型；
    和/或，
    所述第二模型约束是根据所述第一模型约束所确定的，以使各所述教师模型与所述学生模型在一个或者多个模型指标上满足预设倍数条件。
  9. 根据权利要求1所述的方法，其中，所述训练数据集还包括第二样本图像；
    所述依据所述训练数据集、所述教师模型以及所述学生模型进行模型蒸馏，得到图像生成模型，包括：
    利用所述第一样本图像和所述第一样本图像对应的标签信息，更新所述教师模型；
    利用所述第二样本图像和所述第二样本图像对应的第三伪标签，更新所述学生模型，并继续执行所述利用所述第一样本图像和所述第一样本图像对应的标签信息，更新所述教师模型的步骤，直至在达到第三停止条件时，根据所述学生模型，确定所述图像生成模型；其中，所述第三伪标签是由更新后的教师模型针对所述第二样本图像进行处理所得到的。
  10. 根据权利要求9所述的方法，其中，所述利用所述第二样本图像和所述第二样本图像对应的第三伪标签，更新所述学生模型，包括：
    利用所述第一样本图像、所述第一样本图像对应的第四伪标签、所述第二样本图像以及所述第二样本图像对应的第三伪标签，更新所述学生模型；其中，所述第四伪标签是由所述更新后的教师模型针对所述第一样本图像进行处理所得到的。
  11. 根据权利要求9或者10所述的方法，其中，所述学生模型的更新过程，包括：
    利用所述第二样本图像和所述第二样本图像对应的第三伪标签，确定所述学生模型的第一蒸馏损失；
    根据所述第一蒸馏损失，确定所述学生模型的损失表征数据；
    根据所述学生模型的损失表征数据，更新所述学生模型。
  12. 根据权利要求11所述的方法，其中，所述根据所述第一蒸馏损失，确定所述学生模型的损失表征数据，包括：
    根据所述第一蒸馏损失以及所述第一蒸馏损失对应的加权权重，确定所述学生模型的损失表征数据；其中，所述加权权重是根据所述第三伪标签的置信度所确定的；所述置信度是根据所述更新后的教师模型、所述更新后的教师模型对应的判别器以及所述第二样本图像所确定的。
  13. 根据权利要求12所述的方法，其中，所述加权权重的确定过程，包括：
    若所述置信度低于置信度阈值，则从预设区间中随机选择一个数值，确 定为所述加权权重；
    若所述置信度不低于所述置信度阈值，则将预设数值确定为所述加权权重。
  14. 根据权利要求13所述的方法，其中，所述置信度阈值的确定过程，包括：
    利用所述更新后的教师模型对至少一个图像数据进行处理，得到各个图像数据对应的生成图像；其中，所述至少一个图像数据包括所述第二样本图像和/或所述第一样本图像；
    利用所述更新后的教师模型对应的判别器对各所述生成图像进行判别处理，得到各所述生成图像对应的判别结果；
    针对至少一个所述生成图像对应的判别结果进行数据分析处理，得到所述置信度阈值。
  15. 一种图像生成方法，其中，所述方法包括：
    获取待处理图像；
    利用预先构建的图像生成模型对所述待处理图像进行处理，得到图像生成结果；其中，所述图像生成模型是利用权利要求1-14中任一项所述的模型构建方法所构建的。
  16. 一种模型构建装置，包括：
    第一获取单元，配置为获取训练数据集；其中，所述训练数据集包括第一样本图像和所述第一样本图像对应的标签信息；
    第一训练单元，配置为利用所述训练数据集对超网络进行训练；
    网络搜索单元，配置为从已训练的超网络中搜索教师模型和学生模型；以及
    第二训练单元，配置为依据所述训练数据集、所述教师模型以及所述学生模型进行模型蒸馏，得到图像生成模型。
  17. 一种图像生成装置，包括：
    第二获取单元，配置为获取待处理图像；以及
    图像生成单元，配置为利用预先构建的图像生成模型对所述待处理图像进行处理，得到图像生成结果；其中，所述图像生成模型是利用权利要求1-14中任一项所述的模型构建方法所构建的。
  18. 一种电子设备，包括：处理器和存储器；
    所述存储器，用于存储指令或计算机程序；
    所述处理器，用于执行所述存储器中的所述指令或所述计算机程序，以使得所述电子设备执行如权利要求1-15中任一项所述的方法。
  19. 一种计算机可读介质，其中，所述计算机可读介质中存储有指令或计算机程序，当所述指令或所示计算机程序在设备上运行时，使得所述设备执行如权利要求1-15中任一项所述的方法。
PCT/CN2024/097164 2023-06-30 2024-06-04 模型构建方法、图像生成方法、装置、设备、介质 Ceased WO2025001765A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310798179.9 2023-06-30
CN202310798179.9A CN119227756A (zh) 2023-06-30 2023-06-30 一种模型构建方法、图像生成方法、装置、设备、介质

Publications (1)

Publication Number Publication Date
WO2025001765A1 true WO2025001765A1 (zh) 2025-01-02

Family

ID=93937315

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/097164 Ceased WO2025001765A1 (zh) 2023-06-30 2024-06-04 模型构建方法、图像生成方法、装置、设备、介质

Country Status (2)

Country Link
CN (1) CN119227756A (zh)
WO (1) WO2025001765A1 (zh)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112001364A (zh) * 2020-09-22 2020-11-27 上海商汤临港智能科技有限公司 图像识别方法及装置、电子设备和存储介质
CN113177888A (zh) * 2021-04-27 2021-07-27 北京有竹居网络技术有限公司 超分修复网络模型生成方法、图像超分修复方法及装置
CN113496512A (zh) * 2021-09-06 2021-10-12 北京字节跳动网络技术有限公司 用于内窥镜的组织腔体定位方法、装置、介质及设备
CN113962272A (zh) * 2021-06-28 2022-01-21 北京旷视科技有限公司 模型蒸馏方法、装置和系统及存储介质
US20220129731A1 (en) * 2021-05-27 2022-04-28 Beijing Baidu Netcom Science Technology Co., Ltd. Method and apparatus for training image recognition model, and method and apparatus for recognizing image
CN114548288A (zh) * 2022-02-24 2022-05-27 京东科技信息技术有限公司 模型训练、图像识别方法和装置
CN115937071A (zh) * 2022-05-13 2023-04-07 青岛海信电子技术服务有限公司 一种图像检测方法、装置、设备及介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112001364A (zh) * 2020-09-22 2020-11-27 上海商汤临港智能科技有限公司 图像识别方法及装置、电子设备和存储介质
CN113177888A (zh) * 2021-04-27 2021-07-27 北京有竹居网络技术有限公司 超分修复网络模型生成方法、图像超分修复方法及装置
US20220129731A1 (en) * 2021-05-27 2022-04-28 Beijing Baidu Netcom Science Technology Co., Ltd. Method and apparatus for training image recognition model, and method and apparatus for recognizing image
CN113962272A (zh) * 2021-06-28 2022-01-21 北京旷视科技有限公司 模型蒸馏方法、装置和系统及存储介质
CN113496512A (zh) * 2021-09-06 2021-10-12 北京字节跳动网络技术有限公司 用于内窥镜的组织腔体定位方法、装置、介质及设备
CN114548288A (zh) * 2022-02-24 2022-05-27 京东科技信息技术有限公司 模型训练、图像识别方法和装置
CN115937071A (zh) * 2022-05-13 2023-04-07 青岛海信电子技术服务有限公司 一种图像检测方法、装置、设备及介质

Also Published As

Publication number Publication date
CN119227756A (zh) 2024-12-31

Similar Documents

Publication Publication Date Title
WO2024174911A1 (zh) 代码生成方法、装置、存储介质及电子设备
CN115114439B (zh) 多任务模型推理、多任务信息处理的方法和装置
CN111738010B (zh) 用于生成语义匹配模型的方法和装置
CN113240108B (zh) 模型训练方法、装置和电子设备
CN114494709B (zh) 特征提取模型的生成方法、图像特征提取方法和装置
CN112380883B (zh) 模型训练方法、机器翻译方法、装置、设备及存储介质
CN117290477B (zh) 一种基于二次检索增强的生成式建筑知识问答方法
CN114239853B (zh) 模型训练方法、装置、设备、存储介质以及程序产品
WO2023142399A1 (zh) 信息搜索方法、装置、电子设备
WO2022250609A1 (zh) 数据保护方法、网络结构训练方法、装置、介质及设备
CN116483891A (zh) 一种信息预测方法、装置、设备和存储介质
CN114153959B (zh) 键值匹配方法、装置、可读介质及电子设备
WO2024183593A1 (zh) 一种图像分类方法、装置、电子设备及存储介质
CN114117219A (zh) 会话推荐方法和推荐模型的训练方法、装置、设备及介质
CN111581455B (zh) 文本生成模型的生成方法、装置和电子设备
CN113868481A (zh) 组件获取方法、装置及电子设备和存储介质
WO2023185896A1 (zh) 一种文本生成方法、装置、计算机设备及存储介质
US20220391425A1 (en) Method and apparatus for processing information
CN115688808A (zh) 翻译方法、装置、可读介质及电子设备
CN111626044B (zh) 文本生成方法、装置、电子设备及计算机可读存储介质
CN114201674A (zh) 模型训练方法、推广内容的处理方法及相关装置
WO2025001765A1 (zh) 模型构建方法、图像生成方法、装置、设备、介质
CN118299022B (zh) 手术设备信息化管理系统及方法
CN115409044B (zh) 翻译方法、装置、可读介质及电子设备
CN115619847A (zh) 一种几何估计图像处理方法、装置、电子设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24830412

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE