CN110276811A - Image conversion method, device, electronic equipment and readable storage medium storing program for executing - Google Patents
Image conversion method, device, electronic equipment and readable storage medium storing program for executing Download PDFInfo
- Publication number
- CN110276811A CN110276811A CN201910588382.7A CN201910588382A CN110276811A CN 110276811 A CN110276811 A CN 110276811A CN 201910588382 A CN201910588382 A CN 201910588382A CN 110276811 A CN110276811 A CN 110276811A
- Authority
- CN
- China
- Prior art keywords
- latent space
- source domain
- coding
- network
- aiming field
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Compression Of Band Width Or Redundancy In Fax (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The embodiment of the present application provides a kind of image conversion method, device, electronic equipment and readable storage medium storing program for executing, by obtaining source domain image to be converted, and encoded by the source domain latent space of the encoder of autoencoder network available source domain image to latent space and pass through circulation generate confrontation network obtain latent space source domain latent space be encoded to aiming field latent space coding between mapping, finally aiming field latent space coding input aiming field decoder is decoded, target area image is obtained, realizes the conversion between source domain and aiming field.In this way, can obtain the target area image of more high quality for the simple generation network training using confrontation compared to the prior art, and reduce network depth and training difficulty, the requirement under a variety of environment can be better adapted to.
Description
Technical field
This application involves field of image processing, in particular to a kind of image conversion method, device, electronic equipment and
Readable storage medium storing program for executing.
Background technique
User no longer meets only when modifying to captured image to the contrast of image, saturation degree
The simple adjustment of equal numerical value, and the advanced adjustment demand such as more need the conversion to the stylistic category of image.For example, by true
Real self-timer image is converted to cartoon figure's image, and landscape image is converted to pencil sketch image, glassy-eyed face is schemed
Face image as being converted to smiling face, is converted to child's image etc. for adult image.
The methods for image conversion most of at present, which are all based on, generates confrontation network implementations, and such methods use one
Source domain image is mapped on target image domain by a generator, the image converted.Then, by the image of conversion and target
In image input discriminator on image area, while both training, the image that then will be mapped to aiming field pass through a life again
It grows up to be a useful person and maps back in source domain, obtain secondary generation image, finally minimize certain between secondary generation image and original image
Constraint is to reach the conversion of source domain image to target area image.However, inventor it has been investigated that, above scheme often network knot
Structure is complicated and network depth is higher, and the training difficulty for generating confrontation network is higher, leads to the picture quality after finally converting
It is often poor.
Summary of the invention
In view of this, the embodiment of the present application is designed to provide a kind of image conversion method, device, electronic equipment and can
Storage medium is read, to solve or improve the above problem.
According to the one aspect of the embodiment of the present application, a kind of electronic equipment is provided, may include that one or more storages are situated between
Matter and one or more processors communicated with storage medium.One or more storage mediums are stored with the executable machine of processor
Device executable instruction.When electronic equipment operation, processor executes the machine-executable instruction, to execute image conversion side
Method.
According to the another aspect of the embodiment of the present application, a kind of image conversion method is provided, is applied to electronic equipment, the electricity
It is stored with autoencoder network in sub- equipment and circulation generates confrontation network, the autoencoder network includes encoder and aiming field solution
Code device, which comprises
Obtain source domain image to be converted；
The encoder that the source domain image inputs the autoencoder network is encoded, source domain latent space coding is obtained；
By source domain in circulation generation confrontation network described in the source domain latent space coding input to the generation network of aiming field
It is handled, obtains aiming field latent space coding；
Aiming field decoder described in the aiming field latent space coding input is decoded, target area image is obtained.
In a kind of possible embodiment, the method also includes:
Configure initial autoencoder network, the initial autoencoder network include initial encoder, initial source domain decoder and
Initial target domain decoder；
Training sample set is obtained, the training sample set includes source domain image pattern and aiming field image pattern；
According to the source domain image pattern and the aiming field image pattern training initial encoder, the initial source domain solution
Code device and initial target domain decoder, obtain the autoencoder network, and the autoencoder network is stored in the electricity
In sub- equipment.
It is described according to the source domain image pattern and aiming field image pattern training institute in a kind of possible embodiment
Initial encoder, the initial source domain decoder and initial target domain decoder are stated, the step of the autoencoder network is obtained
Suddenly, comprising:
The source domain image pattern and aiming field image pattern are encoded respectively by the initial encoder, obtained
The corresponding source domain latent space of the source domain image pattern encodes aiming field latent space corresponding with aiming field image pattern and encodes；
Source domain latent space coding is decoded by the initial source domain decoder, obtains the source domain latent space
Corresponding source decoding image is encoded, and aiming field latent space coding is solved by initial target domain decoder
Code obtains the aiming field latent space and encodes corresponding target decoder image；
Image, the source domain image pattern, the target decoder image and the aiming field figure are decoded according to the source
The network parameter of the initial encoder as described in Sample Refreshment, initial source domain decoder and initial target domain decoder, and return logical
The step of initial encoder respectively encodes the source domain image pattern and aiming field image pattern is crossed, until described
When initial encoder, the initial source domain decoder and initial target domain decoder meet training termination condition, output instruction
The autoencoder network got.
It is described that image, the source domain image pattern, the mesh are decoded according to the source in a kind of possible embodiment
Mark decoding image and the aiming field image pattern update the initial encoder, initial source domain decoder and initial target domain
The step of network parameter of decoder, comprising:
Calculate the first-loss functional value between the source decoding image and the source domain image pattern and the target
Decode the second loss function value between image and the aiming field image pattern；
The initial encoder, initial source domain are updated according to the first-loss functional value and the second loss function value
The network parameter of decoder and initial target domain decoder.
In a kind of possible embodiment, the method also includes:
It configures initial cycle and generates confrontation network, it includes source domain to the of aiming field that the initial cycle, which generates confrontation network,
One, which is initially generated network and the first initial differentiation network and aiming field, is initially generated network and second initially to the second of source domain
Differentiate network；
Network is initially generated to the encoder of the autoencoder network of input to source domain image pattern by described first
The source domain latent space coding obtained after being encoded is handled, and obtains first object domain latent space coding, and pass through described the
Two, which are initially generated network, handles first object domain latent space coding, obtains the first source domain latent space coding；
Network is initially generated to the encoder of the autoencoder network of input to target area image sample by described second
This aiming field latent space obtained after being encoded coding is handled, and obtains the second source domain latent space coding, and by described
First, which is initially generated network, handles the second source domain latent space coding, obtains the second aiming field latent space coding；
According to the first source domain latent space coding, the second source domain latent space coding, the hidden sky in first object domain
Between encode and the second aiming field latent space coding training initial cycle generates confrontation network, obtain the circulation and give birth to
Confrontation network storage is generated in the electronic equipment at confrontation network, and by the circulation.
In a kind of possible embodiment, it is described by described first be initially generated network to described in input from encode
The source domain latent space coding that the encoder of network obtains after encoding to source domain image pattern is handled, and obtains first object
Domain latent space coding, and be initially generated network by described second and latent space coding in the first object domain is handled, it obtains
The step of being encoded to the first source domain latent space, comprising:
Network is initially generated to the encoder of the autoencoder network of input to source domain image pattern by described first
The source domain latent space coding obtained after being encoded is handled, and is obtained the source domain latent space and is encoded to aiming field latent space volume
Each in coding mapping and the latent space coding of code encodes upper weighting coefficient corresponding with the source domain latent space coding；
The coding mapping of the aiming field latent space coding is encoded to according to the source domain latent space and the latent space is compiled
Each in code encodes upper weighting coefficient corresponding with the source domain latent space coding and obtains first object domain latent space volume
Code；
Network is initially generated by described second to handle the first object domain latent space coding of input, is obtained
First object domain latent space is encoded in the coding mapping and latent space coding of source domain latent space coding on each coding
Weighting coefficient corresponding with first object domain latent space coding；
It is encoded in the coding mapping and latent space coding of source domain latent space coding according to first object domain latent space
Each encodes upper weighting coefficient corresponding with first object domain latent space coding and obtains the first source domain latent space volume
Code；
It is described to be initially generated network to the encoder of the autoencoder network of input to aiming field figure by described second
The decent aiming field latent space coding obtained after being encoded is handled, and obtains the second source domain latent space coding, and pass through
Described first, which is initially generated network, handles the second source domain latent space coding, obtains the second aiming field latent space coding
The step of, comprising:
Network is initially generated to the encoder of the autoencoder network of input to target area image sample by described second
This aiming field latent space obtained after being encoded coding is handled, and is obtained the aiming field latent space and is encoded to the hidden sky of source domain
Between in the coding mapping that encodes and latent space coding the upper weighting corresponding with the aiming field latent space coding of each coding be
Number；
It is encoded to according to the aiming field latent space each in the coding mapping and latent space coding of source domain latent space coding
Position encodes upper weighting coefficient corresponding with the aiming field latent space coding and obtains the second source domain latent space coding；
Network is initially generated by described first to handle the second source domain latent space coding of input, obtains institute
It states the second source domain latent space and is encoded to that each coding in the coding mapping and latent space coding of source domain latent space coding is upper and institute
It states the second source domain latent space and encodes corresponding weighting coefficient；
It is encoded to according to the second source domain latent space every in the coding mapping and latent space coding of source domain latent space coding
One encodes upper weighting coefficient corresponding with the second source domain latent space coding and obtains the second aiming field latent space coding.
It is described hidden according to the first source domain latent space coding, second source domain in a kind of possible embodiment
Space encoding, first object domain latent space coding and the second aiming field latent space coding training initial cycle
Confrontation network is generated, the step of circulation generates confrontation network is obtained, comprising:
By the described first initial differentiation network to first object domain latent space coding and the aiming field latent space
Coding is differentiated, obtains the first differentiation as a result, and by the described second initial differentiation network to the second source domain latent space
Coding and source domain latent space coding are differentiated, the second differentiation result is obtained；
Calculate the described first discriminant value and the aiming field latent space for differentiating latent space coding in first object domain in result
Third loss function value and the first source domain latent space coding between the discriminant value of coding are compiled with the source domain latent space
The 4th loss function value between code, and calculate described second and differentiate the discriminant value of the second source domain latent space coding and institute in result
It states the 5th loss function value between the discriminant value of source domain latent space coding and the second aiming field latent space encodes and institute
State the 6th loss function value between aiming field latent space coding；
Differentiate that result, second differentiate result, third loss function value, the 4th loss function value, the 5th according to described first
Loss function value and the 6th loss function value update the initial cycle and generate the network parameter of confrontation network, and pass back through
Described first is initially generated after network encodes the encoder of the autoencoder network of input to source domain image pattern
To source domain latent space coding handled the step of, until the initial cycle generate confrontation network meet training termination condition
When, the circulation that output training obtains generates confrontation network.
According to the another aspect of the embodiment of the present application, a kind of image conversion apparatus is provided, is applied to electronic equipment, the electricity
It is stored with autoencoder network in sub- equipment and circulation generates confrontation network, the autoencoder network includes encoder and aiming field solution
Code device, described device include:
Module is obtained, for obtaining source domain image to be converted；
Coding module is encoded in the encoder for the source domain image to be inputted to the autoencoder network, is obtained
Source domain latent space coding；
Input module is used for source domain in circulation generation confrontation network described in the source domain latent space coding input to target
The generation network in domain is handled, and aiming field latent space coding is obtained；
Decoder module is obtained for aiming field decoder described in the aiming field latent space coding input to be decoded
Target area image.
According to the another aspect of the embodiment of the present application, a kind of readable storage medium storing program for executing is provided, is stored on the readable storage medium storing program for executing
The step of having machine-executable instruction, above-mentioned image conversion method can be executed when which is run by processor.
Based on any of the above-described aspect, the embodiment of the present application passes through coding net certainly by obtaining source domain image to be converted
The source domain latent space of the encoder of network available source domain image to latent space encodes and passes through circulation generation confrontation network and obtains
The source domain latent space of latent space is encoded to the mapping between aiming field latent space coding, finally by aiming field latent space coding input
Aiming field decoder is decoded, and obtains target area image, realizes the conversion between source domain and aiming field.In this way, compared to existing
For thering is technology to generate network training using confrontation merely, the target area image of more high quality can be obtained, and reduce
Network depth and training difficulty, can better adapt to the requirement under a variety of environment.
Detailed description of the invention
Technical solution in ord to more clearly illustrate embodiments of the present application, below will be to needed in the embodiment attached
Figure is briefly described, it should be understood that the following drawings illustrates only some embodiments of the application, therefore is not construed as pair
The restriction of range for those of ordinary skill in the art without creative efforts, can also be according to this
A little attached drawings obtain other relevant attached drawings.
Fig. 1 shows one of the flow diagram of image conversion method provided by the embodiment of the present application；
Fig. 2 shows the two of the flow diagram of image conversion method provided by the embodiment of the present application；
Fig. 3 shows the training schematic diagram of autoencoder network provided by the embodiment of the present application；
Fig. 4 shows the three of the flow diagram of image conversion method provided by the embodiment of the present application；
Fig. 5 shows the training schematic diagram that circulation provided by the embodiment of the present application generates confrontation network；
Fig. 6 shows the structural schematic diagram block diagram of electronic equipment provided by the embodiment of the present application.
Specific embodiment
To keep the purposes, technical schemes and advantages of the embodiment of the present application clearer, below in conjunction with the embodiment of the present application
In attached drawing, the technical scheme in the embodiment of the application is clearly and completely described, it should be understood that attached drawing in the application
The purpose of illustration and description is only played, is not used to limit the protection scope of the application.In addition, it will be appreciated that schematical attached
Figure does not press scale.Process used herein shows real according to some embodiments of the embodiment of the present application
Existing operation.It should be understood that the operation of flow chart can be realized out of order, the step of context relation of logic can be with
Reversal order is implemented simultaneously.In addition, those skilled in the art under the guide of teachings herein, can add to flow chart
Other one or more operations, can also remove one or more operations from flow chart.
In addition, described embodiments are only a part of embodiments of the present application, instead of all the embodiments.Usually exist
The component of the embodiment of the present application described and illustrated in attached drawing can be arranged and be designed with a variety of different configurations herein.Cause
This, is not intended to limit claimed the application's to the detailed description of the embodiments herein provided in the accompanying drawings below
Range, but it is merely representative of the selected embodiment of the application.Based on embodiments herein, those skilled in the art are not being done
Every other embodiment obtained under the premise of creative work out, shall fall in the protection scope of this application.
Fig. 1 shows the flow diagram of image conversion method provided by the embodiments of the present application, it should be understood that in other realities
It applies in example, the sequence of the image conversion method part step of the present embodiment can not be with Fig. 1 and following specific embodiments
Sequence is limitation, such as can be exchanged with each other according to actual needs or part steps therein also can be omitted or delete.It should
The detailed step of image conversion method is described below.
Step S110 obtains source domain image to be converted.
The encoder of source domain image input autoencoder network is encoded, obtains source domain latent space coding by step S120.
Step S130, by source domain in source domain latent space coding input circulation generation confrontation network to the generation network of aiming field
It is handled, obtains aiming field latent space coding.
Aiming field latent space coding input aiming field decoder is decoded, obtains target area image by step S140.
In the present embodiment, for step S110, the source domain image to be converted can be user by take pictures, network downloading
Etc. the image information that any feasible pattern is got, such as it can be the figure shot by the camera application program of installation
Picture, or by network from the image of the downloadings such as website, chat record, cloud service, the present embodiment is not intended to be limited in any this.
For step S120, is encoded by the encoder that source domain image is inputted autoencoder network, source can be obtained
Mapping namely source domain latent space coding of the area image to latent space.
For step S130, aiming field is arrived by the way that source domain latent space coding input is recycled source domain in generation confrontation network
It generates network to be handled, available source domain latent space is encoded to the coding mapping of aiming field latent space coding and latent space is compiled
Each in code encodes upper weighting coefficient corresponding with source domain latent space coding, and it is hidden according to source domain latent space to be encoded to aiming field
Each coding is upper in coding mapping and the latent space coding of space encoding encodes corresponding weighting coefficient with source domain latent space
Aiming field latent space coding can be obtained in weighting.
After obtaining aiming field latent space coding, it can be decoded by inputting aiming field decoder, obtain aiming field
Image.
Based on above-mentioned design, the present embodiment is by obtaining source domain image to be converted, and the coding for passing through autoencoder network
The source domain latent space of device available source domain image to latent space, which encodes and passes through circulation generation confrontation network, obtains latent space
Source domain latent space is encoded to the mapping between aiming field latent space coding, finally by aiming field latent space coding input aiming field solution
Code device is decoded, and obtains target area image, realizes the conversion between source domain and aiming field.In this way, single compared to the prior art
For the pure generation network training using confrontation, the target area image of more high quality can be obtained, and reduce network depth
With training difficulty, the requirement under a variety of environment can be better adapted to.
As a kind of possible embodiment, referring to Fig. 2, before abovementioned steps S110, figure provided in this embodiment
As conversion method can also include the following steps:
Step S101 configures initial autoencoder network.
In the present embodiment, initial autoencoder network may include initial encoder, initial source domain decoder and initial target
Domain decoder.
Step S102 obtains training sample set.
In the present embodiment, training sample set includes source domain image pattern and aiming field image pattern.Source domain image pattern can
To be any image for needing to be converted to aiming field, aiming field image pattern is the image for being converted to aiming field.
Step S103 is decoded according to source domain image pattern and aiming field image pattern training initial encoder, initial source domain
Device and initial target domain decoder obtain autoencoder network, and in the electronic device by autoencoder network storage.
Fig. 3 is please referred to, is carried out below based on training process of the Fig. 3 to autoencoder network in this step S103 exemplary
It illustrates.
Firstly, encoding respectively to source domain image pattern and aiming field image pattern by initial encoder, source is obtained
The corresponding source domain latent space of area image sample encodes aiming field latent space coding corresponding with aiming field image pattern.Lead to as a result,
Crossing same initial encoder can guarantee that source domain image pattern and aiming field image pattern can be mapped in the same latent space.
Then, source domain latent space coding is decoded by initial source domain decoder, obtains source domain latent space coding pair
The source decoding image answered, and aiming field latent space coding is decoded by initial target domain decoder, it is hidden to obtain aiming field
The corresponding target decoder image of space encoding；
Then, image, source domain image pattern, target decoder image and aiming field image pattern is decoded according to source to update just
The network parameter of beginning encoder, initial source domain decoder and initial target domain decoder.Such as a kind of possible embodiment party
Formula can calculate first-loss functional value and target decoder image and target between source decoding image and source domain image pattern
The second loss function value between area image sample, and initial compile is updated according to first-loss functional value and the second loss function value
The network parameter of code device, initial source domain decoder and initial target domain decoder.On this basis, initial encoder is passed back through
The step of source domain image pattern and aiming field image pattern are encoded respectively, until initial encoder, initial source domain decode
When device and initial target domain decoder meet training termination condition, obtained autoencoder network is trained in output.
Wherein, above-mentioned training termination condition may include at least one of following three kinds of conditions:
1) repetitive exercise number reaches setting number；2) first-loss functional value and the second loss function value are lower than setting threshold
Value；3) first-loss functional value and the second loss function value no longer decline.
Wherein, in condition 1) in, in order to save operand, the maximum value of the number of iterations can be set, if the number of iterations
Reach setting number, the iteration of this iteration cycle can be stopped, using the network finally obtained as autoencoder network.In condition 2)
In, if first-loss functional value and the second loss function value are lower than given threshold, illustrate current autoencoder network base
Originally it can satisfy condition, iteration can be stopped at this time.In condition 3) in, first-loss functional value and the second loss function value are no longer
Decline, shows to have formd optimal autoencoder network, can stop iteration.
It should be noted that above-mentioned iteration stopping condition can be used in combination, a use can also be selected, for example, can be
First-loss functional value and the second loss function value, which no longer decline, stops iteration, alternatively, when the number of iterations reaches setting number
Stop iteration, alternatively, stopping iteration when first-loss functional value and the second loss function value no longer decline.Alternatively, can be with
It is lower than given threshold, and first-loss functional value and the second loss function in first-loss functional value and the second loss function value
When value no longer declines, stop iteration.
In addition, in the actual implementation process, can also be not limited to using above-mentioned example as training termination condition, this field
Technical staff can design the training termination condition different from above-mentioned example according to actual needs.
On the basis of foregoing description, further referring to Fig. 4, after step S103, figure provided in this embodiment
As conversion method can also include the following steps:
Step S104, configuration initial cycle generate confrontation network.
In the present embodiment, it may include that source domain to the first of aiming field is initially generated network that initial cycle, which generates confrontation network,
Network and the second initial differentiation network are initially generated to the second of source domain with the first initial differentiation network and aiming field.
Fig. 5 is please referred to, is shown below based on Fig. 5 training process for generating confrontation network to circulation in the present embodiment
Example property illustrates.
Step S105 is initially generated network to the encoder of the autoencoder network of input to source domain image pattern by first
The source domain latent space coding obtained after being encoded is handled, and obtains first object domain latent space coding, and by the beginning of second
Beginning generates network and handles first object domain latent space coding, obtains the first source domain latent space coding.
For example, as a kind of possible embodiment, network can be initially generated to the coding net certainly of input by first
The source domain latent space coding that the encoder of network obtains after encoding to source domain image pattern is handled, and obtains source domain latent space
It is encoded to each in the coding mapping and latent space coding of aiming field latent space coding and encodes upper and source domain latent space coding pair
The weighting coefficient answered.On this basis, the coding mapping and hidden sky of aiming field latent space coding are encoded to according to source domain latent space
Between each in coding encode upper weighting coefficient corresponding with source domain latent space coding and obtain first object domain latent space and encode, and
Network is initially generated by second to handle the first object domain latent space coding of input, obtains first object domain latent space
Each coding in the coding mapping and latent space coding of source domain latent space coding is encoded to above to compile with first object domain latent space
The corresponding weighting coefficient of code.Then, the coding mapping and hidden of source domain latent space coding is encoded to according to first object domain latent space
Each in space encoding encodes upper weighting coefficient corresponding with first object domain latent space coding and obtains the first source domain latent space
Coding.
Step S106 is initially generated network to the encoder of the autoencoder network of input to target area image sample by second
This aiming field latent space obtained after being encoded coding is handled, and obtains the second source domain latent space coding, and pass through first
It is initially generated network to handle the second source domain latent space coding, obtains the second aiming field latent space coding.
For example, as a kind of possible embodiment, network can be initially generated to the coding net certainly of input by second
The aiming field latent space coding that the encoder of network obtains after encoding to aiming field image pattern is handled, and obtains aiming field
Latent space is encoded to each in the coding mapping and latent space coding of source domain latent space coding and encodes upper and aiming field latent space
Encode corresponding weighting coefficient.On this basis, the coding mapping of source domain latent space coding is encoded to according to aiming field latent space
Upper weighting coefficient corresponding with aiming field latent space coding, which is encoded, with each in latent space coding obtains the second source domain latent space
Coding, and be initially generated network by first and the second source domain latent space coding of input is handled, it is hidden to obtain the second source domain
Each in the coding mapping and latent space coding that space encoding is encoded to source domain latent space encodes upper and the second source domain latent space
Encode corresponding weighting coefficient.Then, the coding mapping and hidden of source domain latent space coding is encoded to according to the second source domain latent space
Each in space encoding encodes upper weighting coefficient corresponding with the second source domain latent space coding and obtains the second aiming field latent space
Coding.
Step S107 is compiled according to the first source domain latent space coding, the second source domain latent space coding, first object domain latent space
Code and the second aiming field latent space encode training initial cycle and generate confrontation network, obtain circulation and generate confrontation network, and will
Circulation generates confrontation network storage in the electronic device.
For example, as a kind of possible embodiment, it can be by the first initial differentiation network to the hidden sky in first object domain
Between coding and aiming field latent space coding differentiated, obtain the first differentiation as a result, and initial differentiating network to the by second
Two source domain latent spaces coding and source domain latent space coding are differentiated, the second differentiation result is obtained.On this basis, first is calculated
Differentiate that the third in result between the discriminant value of first object domain latent space coding and the discriminant value of aiming field latent space coding is damaged
The 4th loss function value between functional value (L2 LOSS) and the first source domain latent space coding and source domain latent space coding is lost,
And it calculates second and differentiates in result between the discriminant value of the second source domain latent space coding and the discriminant value of source domain latent space coding
The 6th between 5th loss function value (L2 LOSS) and the second aiming field latent space coding and aiming field latent space coding
Loss function value.Then, according to first differentiate result, second differentiate result, third loss function value, the 4th loss function value,
5th loss function value and the 6th loss function value update initial cycle and generate the network parameter of confrontation network, and pass back through
First is initially generated the source domain obtained after network encodes the encoder of the autoencoder network of input to source domain image pattern
The step of latent space coding is handled, when initial cycle, which generates confrontation network, meets training termination condition, output training
Obtained circulation generates confrontation network.
Wherein, above-mentioned training termination condition may include at least one of following three kinds of conditions:
1) repetitive exercise number reaches setting number；2) third loss function value, the 4th loss function value, the 5th loss letter
Numerical value and the total losses functional value of the 6th loss function value are lower than given threshold；3) total losses functional value no longer declines.
Based on above-mentioned steps, the quality of latent space mapping can be promoted, the target area image being subsequently generated thus is improved
Quality.
Fig. 6 shows the schematic diagram of electronic equipment 100 provided by the embodiments of the present application, in the present embodiment, the electronic equipment
100 may include storage medium 110, processor 120 and image conversion apparatus 130.
Wherein, processor 120 can be a general central processing unit (Central Processing Unit,
CPU), microprocessor, application-specific integrated circuit (Application-Specific Integrated Circuit, ASIC),
Or the integrated circuit that one or more programs for controlling the image conversion method of above method embodiment offer execute.
Storage medium 110 can be ROM or can store the other kinds of static storage device of static information and instruction,
RAM or the other kinds of dynamic memory that can store information and instruction, are also possible to the read-only storage of electric erazable programmable
Device (Electrically Erasable Programmabler-Only Memory, EEPROM), CD-ROM
(Compactdisc Read-Only Memory, CD-ROM) or other optical disc storages, optical disc storage (including compression optical disc, swash
Optical disc, optical disc, Digital Versatile Disc, Blu-ray Disc etc.), magnetic disk storage medium or other magnetic storage apparatus or can use
In carry or storage have instruction or data structure form desired program code and can by computer access it is any its
His medium, but not limited to this.Storage medium 110, which can be, to be individually present, and is connected by communication bus with processor 120.It deposits
Storage media 110 can also be integrated with processor.Wherein, storage medium 110, which is used to store, executes answering for application scheme
With program code, such as image conversion apparatus 130 shown in Fig. 6, and execution is controlled by processor 120.Processor 120 is used
In executing the application code stored in storage medium 110, such as image conversion apparatus 130, to execute above method implementation
The image conversion method of example.
The application can carry out the division of functional module according to above method embodiment to image conversion apparatus 130, for example,
The each functional module of each function division can be corresponded to, two or more functions can also be integrated in a processing mould
In block.Above-mentioned integrated module both can take the form of hardware realization, can also be realized in the form of software function module.
It should be noted that being schematical, only a kind of logical function partition to the division of module in the application, in actual implementation
There may be another division manner.For example, Fig. 6 is shown in the case where each function division of use correspondence each functional module
Image conversion apparatus 130 be a kind of schematic device, separately below to each functional module of the image conversion apparatus 130
Function be described in detail.
Module 131 is obtained, for obtaining source domain image to be converted.It is appreciated that the acquisition module 131 can be used for holding
Row above-mentioned steps S110, the detailed implementation about the acquisition module 131 are referred to above-mentioned related to step S110 interior
Hold.
It is hidden to obtain source domain for will encode in the encoder of source domain image input autoencoder network for coding module 132
Space encoding.It is appreciated that the coding module 132 can be used for executing above-mentioned steps S120, about the detailed of the coding module 132
Thin implementation is referred to above-mentioned to the related content of step S120.
Input module 133 arrives aiming field for source domain latent space coding input to be recycled source domain in generation confrontation network
It generates network to be handled, obtains aiming field latent space coding.It is appreciated that the input module 133 can be used for executing it is above-mentioned
Step S130, the detailed implementation about the input module 133 are referred to above-mentioned to the related content of step S130.
Decoder module 134 obtains aiming field for aiming field latent space coding input aiming field decoder to be decoded
Image.It is appreciated that the decoder module 134 can be used for executing above-mentioned steps S140, the detailed reality about the decoder module 134
Existing mode is referred to above-mentioned to the related content of step S140.
Since image conversion apparatus 130 provided by the embodiments of the present application is the another kind of image conversion method shown in FIG. 1
Way of realization, and image conversion apparatus 130 can be used for executing method provided by embodiment shown in FIG. 1, therefore it can be obtained
The technical effect obtained can refer to above method embodiment, and details are not described herein.
Further, based on the same inventive concept, the embodiment of the present application also provides a kind of computer readable storage medium,
It is stored with computer program on the computer readable storage medium, which executes above-mentioned image when being run by processor
The step of conversion method.
Specifically, which can be general storage medium, such as mobile disk, hard disk, on the storage medium
Computer program when being run, be able to carry out above-mentioned image conversion method.
The embodiment of the present application be referring to according to the method for the embodiment of the present application, equipment (electronic equipment 100 of such as Fig. 6) and
The flowchart and/or the block diagram of computer program product describes.It should be understood that flow chart can be realized by computer program instructions
And/or the knot of the process and/or box in each flow and/or block and flowchart and/or the block diagram in block diagram
It closes.These computer program instructions be can provide to general purpose computer, special purpose computer, Embedded Processor or other programmable numbers
According to the processor of processing equipment to generate a machine, so that passing through the processing of computer or other programmable data processing devices
The instruction that device executes generates for realizing in one box of one or more flows of the flowchart and/or block diagram or multiple sides
The device for the function of being specified in frame.
Although the application is described in conjunction with each embodiment herein, however, implementing the application claimed
In the process, those skilled in the art are by checking the attached drawing, disclosure and the appended claims, it will be appreciated that and it is real
Other variations of the existing open embodiment.In the claims, one word of " comprising " is not excluded for other components or step,
"a" or "an" is not excluded for multiple situations.Single processor or other units may be implemented to enumerate in claim several
Item function.Mutually different has been recited in mutually different dependent certain measures, it is not intended that these measures cannot group close
To generate good effect.
The above, the only various embodiments of the application, but the protection scope of the application is not limited thereto, it is any
Those familiar with the art within the technical scope of the present application, can easily think of the change or the replacement, and should all contain
Lid is within the scope of protection of this application.Therefore, the protection scope of the application shall be subject to the protection scope of the claim.
Claims (10)
1. a kind of image conversion method, which is characterized in that be applied to electronic equipment, be stored in the electronic equipment from coding net
Network and circulation generate confrontation network, and the autoencoder network includes encoder and aiming field decoder, which comprises
Obtain source domain image to be converted；
The encoder that the source domain image inputs the autoencoder network is encoded, source domain latent space coding is obtained；
The generation network that circulation described in the source domain latent space coding input generates source domain to aiming field in confrontation network is carried out
Processing obtains aiming field latent space coding；
Aiming field decoder described in the aiming field latent space coding input is decoded, target area image is obtained.
2. image conversion method according to claim 1, which is characterized in that the method also includes:
Configure initial autoencoder network, the initial autoencoder network includes initial encoder, initial source domain decoder and initial
Aiming field decoder；
Training sample set is obtained, the training sample set includes source domain image pattern and aiming field image pattern；
According to the source domain image pattern and the aiming field image pattern training initial encoder, the initial source domain decoder
With initial target domain decoder, the autoencoder network is obtained, and the autoencoder network is stored in the electronics and is set
In standby.
3. image conversion method according to claim 2, which is characterized in that described according to the source domain image pattern and mesh
Initial encoder, the initial source domain decoder and initial target domain decoder described in area image sample training are marked, is obtained
The step of autoencoder network, comprising:
The source domain image pattern and aiming field image pattern are encoded respectively by the initial encoder, obtained described
The corresponding source domain latent space of source domain image pattern encodes aiming field latent space coding corresponding with aiming field image pattern；
Source domain latent space coding is decoded by the initial source domain decoder, obtains the source domain latent space coding
Corresponding source decodes image, and is decoded by initial target domain decoder to aiming field latent space coding, obtains
Corresponding target decoder image is encoded to the aiming field latent space；
Image, the source domain image pattern, the target decoder image and the target area image sample are decoded according to the source
This updates the network parameter of the initial encoder, initial source domain decoder and initial target domain decoder, and passes back through institute
The step of initial encoder respectively encodes the source domain image pattern and aiming field image pattern is stated, until described initial
When encoder, the initial source domain decoder and initial target domain decoder meet training termination condition, export trained
The autoencoder network arrived.
4. image conversion method according to claim 3, which is characterized in that described to decode image, described according to the source
Source domain image pattern, the target decoder image and the aiming field image pattern update the initial encoder, initial source
The step of network parameter of domain decoder and initial target domain decoder, comprising:
Calculate the first-loss functional value and the target decoder between the source decoding image and the source domain image pattern
The second loss function value between image and the aiming field image pattern；
The initial encoder, the decoding of initial source domain are updated according to the first-loss functional value and the second loss function value
The network parameter of device and initial target domain decoder.
5. image conversion method according to claim 2, which is characterized in that the method also includes:
It configures initial cycle and generates confrontation network, at the beginning of the initial cycle generation confrontation network includes source domain to the first of aiming field
Beginning generates network and the first initial differentiation network and aiming field and is initially generated network and the second initial differentiation to the second of source domain
Network；
Network is initially generated by described first to carry out source domain image pattern the encoder of the autoencoder network of input
The source domain latent space coding obtained after coding is handled, and obtains first object domain latent space coding, and by the beginning of described second
Beginning generates network and handles first object domain latent space coding, obtains the first source domain latent space coding；
By described second be initially generated network to the encoder of the autoencoder network of input to aiming field image pattern into
The aiming field latent space coding obtained after row coding is handled, and obtains the second source domain latent space coding, and pass through described first
It is initially generated network to handle the second source domain latent space coding, obtains the second aiming field latent space coding；
It is compiled according to the first source domain latent space coding, the second source domain latent space coding, first object domain latent space
Code and the second aiming field latent space coding training initial cycle generate confrontation network, obtain the circulation generation pair
Anti- network, and the circulation is generated into confrontation network storage in the electronic equipment.
6. image conversion method according to claim 5, which is characterized in that described to be initially generated network by described first
The source domain latent space obtained after being encoded to the encoder of the autoencoder network of input to source domain image pattern encode into
Row processing, obtains first object domain latent space coding, and to be initially generated network hidden to the first object domain by described second
The step of space encoding is handled, and the first source domain latent space coding is obtained, comprising:
Network is initially generated by described first to carry out source domain image pattern the encoder of the autoencoder network of input
The source domain latent space coding obtained after coding is handled, and is obtained the source domain latent space and is encoded to aiming field latent space coding
Each in coding mapping and latent space coding encodes upper weighting coefficient corresponding with the source domain latent space coding；
It is encoded to according to the source domain latent space in the coding mapping and latent space coding of the aiming field latent space coding
Each encodes upper weighting coefficient corresponding with the source domain latent space coding and obtains first object domain latent space coding；
Network is initially generated by described second to handle the first object domain latent space coding of input, is obtained described
First object domain latent space is encoded to each in the coding mapping and latent space coding of source domain latent space coding and encodes upper and institute
It states first object domain latent space and encodes corresponding weighting coefficient；
It is encoded to according to first object domain latent space each in the coding mapping and latent space coding of source domain latent space coding
Position encodes upper weighting coefficient corresponding with first object domain latent space coding and obtains the first source domain latent space coding；
It is described to be initially generated network to the encoder of the autoencoder network of input to target area image sample by described second
This aiming field latent space obtained after being encoded coding is handled, and obtains the second source domain latent space coding, and by described
First, which is initially generated network, handles the second source domain latent space coding, obtains the step of the second aiming field latent space coding
Suddenly, comprising:
By described second be initially generated network to the encoder of the autoencoder network of input to aiming field image pattern into
The aiming field latent space coding obtained after row coding is handled, and is obtained the aiming field latent space and is encoded to source domain latent space volume
Each in coding mapping and the latent space coding of code encodes upper weighting coefficient corresponding with the aiming field latent space coding；
Each volume in the coding mapping and latent space coding of source domain latent space coding is encoded to according to the aiming field latent space
Weighting coefficient corresponding with the aiming field latent space coding obtains the second source domain latent space coding on code；
Network is initially generated by described first to handle the second source domain latent space of input coding, obtains described the
It is upper with described that two source domain latent spaces are encoded to each coding in the coding mapping and latent space coding of source domain latent space coding
Two source domain latent spaces encode corresponding weighting coefficient；
Each in the coding mapping and latent space coding of source domain latent space coding is encoded to according to the second source domain latent space
Weighting coefficient corresponding with the second source domain latent space coding obtains the second aiming field latent space coding on coding.
7. image conversion method according to claim 5, which is characterized in that described to be compiled according to the first source domain latent space
Code, the second source domain latent space coding, first object domain latent space coding and the second aiming field latent space are compiled
The code training initial cycle generates confrontation network, obtains the step of circulation generates confrontation network, comprising:
First object domain latent space coding and the aiming field latent space are encoded by the described first initial differentiation network
Differentiated, obtains the first differentiation as a result, and encoding by the described second initial differentiation network to the second source domain latent space
Differentiated with source domain latent space coding, obtains the second differentiation result；
It calculates described first and differentiates that the discriminant value of first object domain latent space coding and the aiming field latent space encode in result
Discriminant value between third loss function value and the first source domain latent space coding with the source domain latent space encode it
Between the 4th loss function value, and calculate it is described second differentiate result in the second source domain latent space coding discriminant value and the source
The 5th loss function value and the second aiming field latent space coding and the mesh between the discriminant value of domain latent space coding
Mark the 6th loss function value between domain latent space coding；
Differentiate that result, second differentiate result, third loss function value, the 4th loss function value, the 5th loss according to described first
Functional value and the 6th loss function value update the initial cycle and generate the network parameter of confrontation network, and pass back through described
First is initially generated and obtains after network encodes the encoder of the autoencoder network of input to source domain image pattern
The step of source domain latent space coding is handled, when the initial cycle, which generates confrontation network, meets training termination condition,
The circulation that output training obtains generates confrontation network.
8. a kind of image conversion apparatus, which is characterized in that be applied to electronic equipment, be stored in the electronic equipment from coding net
Network and circulation generate confrontation network, and the autoencoder network includes encoder and aiming field decoder, and described device includes:
Module is obtained, for obtaining source domain image to be converted；
Coding module encodes in the encoder for the source domain image to be inputted to the autoencoder network, obtains source domain
Latent space coding；
Input module arrives aiming field for circulation described in the source domain latent space coding input to be generated source domain in confrontation network
It generates network to be handled, obtains aiming field latent space coding；
Decoder module obtains target for aiming field decoder described in the aiming field latent space coding input to be decoded
Area image.
9. a kind of electronic equipment, which is characterized in that the electronic equipment includes one or more storage mediums and one or more
The processor communicated with storage medium, one or more storage mediums are stored with the executable machine-executable instruction of processor,
When electronic equipment operation, the processor executes the machine-executable instruction, to realize any one of claim 1-7
The image conversion method.
10. a kind of readable storage medium storing program for executing, which is characterized in that the readable storage medium storing program for executing is stored with machine-executable instruction, described
Machine-executable instruction, which is performed, realizes image conversion method described in any one of claim 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910588382.7A CN110276811B (en) | 2019-07-02 | 2019-07-02 | Image conversion method and device, electronic equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910588382.7A CN110276811B (en) | 2019-07-02 | 2019-07-02 | Image conversion method and device, electronic equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110276811A true CN110276811A (en) | 2019-09-24 |
| CN110276811B CN110276811B (en) | 2022-11-01 |
Family
ID=67963796
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910588382.7A Active CN110276811B (en) | 2019-07-02 | 2019-07-02 | Image conversion method and device, electronic equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110276811B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111539961A (en) * | 2019-12-13 | 2020-08-14 | 山东浪潮人工智能研究院有限公司 | Target segmentation method, device and equipment |
| CN111582449A (en) * | 2020-05-07 | 2020-08-25 | 广州视源电子科技股份有限公司 | Training method, device, equipment and storage medium for target domain detection network |
| CN113761831A (en) * | 2020-11-13 | 2021-12-07 | 北京沃东天骏信息技术有限公司 | Method, device and equipment for generating style calligraphy and storage medium |
| CN114359490A (en) * | 2021-11-23 | 2022-04-15 | 北京邮电大学 | Electromagnetic map construction method based on multi-mode fusion and related device |
| WO2022199225A1 (en) * | 2021-03-26 | 2022-09-29 | 北京沃东天骏信息技术有限公司 | Decoding method and apparatus, and computer-readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108171320A (en) * | 2017-12-06 | 2018-06-15 | 西安工业大学 | A kind of image area switching network and conversion method based on production confrontation network |
| CN108182657A (en) * | 2018-01-26 | 2018-06-19 | 深圳市唯特视科技有限公司 | A kind of face-image conversion method that confrontation network is generated based on cycle |
| US20180307947A1 (en) * | 2017-04-25 | 2018-10-25 | Nec Laboratories America, Inc. | Cyclic generative adversarial network for unsupervised cross-domain image generation |
| CN108875935A (en) * | 2018-06-11 | 2018-11-23 | 兰州理工大学 | Based on the natural image target materials visual signature mapping method for generating confrontation network |
-
2019
- 2019-07-02 CN CN201910588382.7A patent/CN110276811B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180307947A1 (en) * | 2017-04-25 | 2018-10-25 | Nec Laboratories America, Inc. | Cyclic generative adversarial network for unsupervised cross-domain image generation |
| CN108171320A (en) * | 2017-12-06 | 2018-06-15 | 西安工业大学 | A kind of image area switching network and conversion method based on production confrontation network |
| CN108182657A (en) * | 2018-01-26 | 2018-06-19 | 深圳市唯特视科技有限公司 | A kind of face-image conversion method that confrontation network is generated based on cycle |
| CN108875935A (en) * | 2018-06-11 | 2018-11-23 | 兰州理工大学 | Based on the natural image target materials visual signature mapping method for generating confrontation network |
Non-Patent Citations (1)
| Title |
|---|
| 何剑华等: "基于改进的CycleGAN模型非配对的图像到图像转换", 《玉林师范学院学报》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111539961A (en) * | 2019-12-13 | 2020-08-14 | 山东浪潮人工智能研究院有限公司 | Target segmentation method, device and equipment |
| CN111582449A (en) * | 2020-05-07 | 2020-08-25 | 广州视源电子科技股份有限公司 | Training method, device, equipment and storage medium for target domain detection network |
| CN113761831A (en) * | 2020-11-13 | 2021-12-07 | 北京沃东天骏信息技术有限公司 | Method, device and equipment for generating style calligraphy and storage medium |
| CN113761831B (en) * | 2020-11-13 | 2024-05-21 | 北京沃东天骏信息技术有限公司 | Style handwriting generation method, device, equipment and storage medium |
| WO2022199225A1 (en) * | 2021-03-26 | 2022-09-29 | 北京沃东天骏信息技术有限公司 | Decoding method and apparatus, and computer-readable storage medium |
| CN114359490A (en) * | 2021-11-23 | 2022-04-15 | 北京邮电大学 | Electromagnetic map construction method based on multi-mode fusion and related device |
| CN114359490B (en) * | 2021-11-23 | 2024-08-20 | 北京邮电大学 | Electromagnetic map construction method and related device based on multi-mode fusion |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110276811B (en) | 2022-11-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110276811A (en) | Image conversion method, device, electronic equipment and readable storage medium storing program for executing | |
| US8265407B2 (en) | Method for coding and decoding 3D data implemented as a mesh model | |
| WO2020172907A1 (en) | Method and device for selecting context model of quantization coefficient end flag bit | |
| EP2916543A1 (en) | Method for coding/decoding depth image and coding/decoding device | |
| CN110211119A (en) | Image quality measure method, apparatus, electronic equipment and readable storage medium storing program for executing | |
| CN110378350A (en) | A kind of method, apparatus and system of Text region | |
| US10614129B2 (en) | Colocation and anticolocation in colocation data centers via elastic nets | |
| US20130142251A1 (en) | Syntax extension of adaptive loop filter in hevc | |
| CN107180443A (en) | A kind of Freehandhand-drawing animation producing method and its device | |
| JP2005065227A5 (en) | ||
| Gu et al. | Compression of human motion capture data using motion pattern indexing | |
| JP2016510454A (en) | Method and apparatus for vertex error correction | |
| CN111429566A (en) | Reconstruction method and device of virtual home decoration scene and electronic equipment | |
| CN110519601A (en) | The coding method of digital video and device | |
| CN114065771A (en) | Pre-training language processing method and device | |
| Váša et al. | Rate‐distortion optimized compression of motion capture data | |
| CN110503964B (en) | Encoding method, encoding device, and recording medium | |
| Grassucci et al. | Enhancing Semantic Communication with Deep Generative Models--An ICASSP Special Session Overview | |
| CN109164999A (en) | A kind of method, apparatus that table is adaptively shown, equipment and readable storage medium storing program for executing | |
| Grassucci et al. | Enhancing Semantic Communication with Deep Generative Models: An Overview | |
| CN110069757A (en) | Special efficacy word edit methods and equipment | |
| GB2551387A (en) | Improved encoding and decoding of geometry data in 3D mesh models | |
| Váša et al. | Efficient encoding of texture coordinates guided by mesh geometry | |
| CN116134815A (en) | Coefficient symbol prediction based on neural network | |
| CN114386390B (en) | Data processing method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |