CN109151443A - High degree of comfort three-dimensional video-frequency generation method, system and terminal device - Google Patents
High degree of comfort three-dimensional video-frequency generation method, system and terminal device Download PDFInfo
- Publication number
- CN109151443A CN109151443A CN201811198022.8A CN201811198022A CN109151443A CN 109151443 A CN109151443 A CN 109151443A CN 201811198022 A CN201811198022 A CN 201811198022A CN 109151443 A CN109151443 A CN 109151443A
- Authority
- CN
- China
- Prior art keywords
- video data
- stereo video
- training set
- training
- generator
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Processing Or Creating Images (AREA)
Abstract
The application is suitable for three-dimensional video-frequency technical field, provide a kind of high degree of comfort three-dimensional video-frequency generation method, system and terminal device, by establishing training set according to the stereo video data of input, confrontation network is generated by training set training, generate new stereo video data, then pass through training set and new stereo video data training arbiter, and it will differentiate that result feeds back to generator, to update the parameter of generator, and it returns to generate confrontation network again by training set training, generate new stereo video data, when fighting network convergence until generating, it obtains generating the stereo video data that the comfort level for all frame images that confrontation network generates reaches unanimity, so as to improve the comfort level of stereo video data, obtain the stereo video data of high degree of comfort.
Description
Technical field
The application belongs to three-dimensional video-frequency technical field more particularly to a kind of high degree of comfort three-dimensional video-frequency generation method, system
And terminal device.
Background technique
With the continuous development of 3D display technology, various 3D display equipment emerge one after another, and realize planar video picture and arrive
The transformation of three-dimensional video-frequency picture is brought well so that the display effect of video pictures more levels off to real scene for people
Visual enjoyment.
However, due to being to realize stereoscopic picture plane in visual effect using the right and left eyes parallax of human eye by 3D imaging technique
Display watches three-dimensional video-frequency for a long time and be easy to cause human eye tired, the bad problem of the generally existing viewing comfort level of three-dimensional video-frequency.
Summary of the invention
It is set in view of this, the embodiment of the present application provides a kind of high degree of comfort three-dimensional video-frequency generation method, system and terminal
It is standby, to solve in the prior art due to being to realize to stand in visual effect using the right and left eyes parallax of human eye by 3D imaging technique
Body picture is shown, is watched three-dimensional video-frequency for a long time and is be easy to cause human eye tired, the generally existing viewing comfort level of three-dimensional video-frequency is bad
The problem of.
The first aspect of the embodiment of the present application provides a kind of high degree of comfort three-dimensional video-frequency generation method comprising:
Training set is established according to the stereo video data of input；
Confrontation network is generated by training set training, generates new stereo video data；Wherein, the generation confrontation
Network includes generator and arbiter；
By the training set and the new stereo video data training arbiter, and it will differentiate that result is fed back to
The generator, to update the parameter of the generator；
It passes back through the training set training and generates confrontation network, new stereo video data is generated, until the generation
Fight network convergence；
Obtain the new stereo video data for generating and generating when fighting network convergence, the stereopsis as high degree of comfort
Frequency evidence.
The second aspect of the embodiment of the present application provides a kind of high degree of comfort three-dimensional video-frequency generation system comprising:
Module is established, for establishing training set according to the stereo video data of input；
Generation module generates new stereo video data for generating confrontation network by training set training；Its
In, the generation confrontation network includes generator and arbiter；
Update module, for training the arbiter by the training set and the new stereo video data, and will
Differentiate that result feeds back to the generator, to update the parameter of the generator；
Return module generates confrontation network for passing back through the training set training, generates new stereo video data,
Until the generation fights network convergence；
Module is obtained, the new stereo video data generated when for obtaining the generation confrontation network convergence, as height
The stereo video data of comfort level.
The third aspect of the embodiment of the present application provides a kind of terminal device, including memory, processor and is stored in
In the memory and the computer program that can run on the processor, when the processor executes the computer program
It realizes such as the step of the above method.
The fourth aspect of the embodiment of the present application provides a kind of computer readable storage medium, the computer-readable storage
The step of media storage has computer program, and the above method is realized when the computer program is executed by processor.
The embodiment of the present application passes through training set training generation pair by establishing training set according to the stereo video data of input
Anti- network generates new stereo video data, then by training set and new stereo video data training arbiter, and will sentence
Other result feeds back to generator, to update the parameter of generator, and returns to generate confrontation network again by training set training,
New stereo video data is generated, when fighting network convergence until generating, obtains generating all frame images that confrontation network generates
The stereo video data that reaches unanimity of comfort level obtained high comfortable so as to improve the comfort level of stereo video data
The stereo video data of degree.
Detailed description of the invention
It in order to more clearly explain the technical solutions in the embodiments of the present application, below will be to embodiment or description of the prior art
Needed in attached drawing be briefly described, it should be apparent that, the accompanying drawings in the following description is only some of the application
Embodiment for those of ordinary skill in the art without any creative labor, can also be according to these
Attached drawing obtains other attached drawings.
Fig. 1 is the flow diagram for the high degree of comfort three-dimensional video-frequency generation method that the embodiment of the present application one provides；
Fig. 2 is the flow diagram for the high degree of comfort three-dimensional video-frequency generation method that the embodiment of the present application two provides；
Fig. 3 is the schematic diagram for the image sequence that the embodiment of the present application two provides；
Fig. 4 is the structural schematic diagram that the high degree of comfort three-dimensional video-frequency that the embodiment of the present application three provides generates system；
Fig. 5 is the structural schematic diagram for the terminal device that the embodiment of the present application four provides.
Specific embodiment
In order to make those skilled in the art more fully understand application scheme, below in conjunction in the embodiment of the present application
Attached drawing, technical solutions in the embodiments of the present application are explicitly described, it is clear that described embodiment is the application one
The embodiment divided, instead of all the embodiments.Based on the embodiment in the application, those of ordinary skill in the art are not doing
Every other embodiment obtained under the premise of creative work out, shall fall within the protection scope of the present application.
The description and claims of this application and term " includes " and their any deformations in above-mentioned attached drawing, meaning
Figure, which is to cover, non-exclusive includes.Such as process, method or system comprising a series of steps or units, product or equipment do not have
It is defined in listed step or unit, but optionally further comprising the step of not listing or unit, or optionally also wrap
Include the other step or units intrinsic for these process, methods, product or equipment.In addition, term " first ", " second " and
" third " etc. is for distinguishing different objects, not for description particular order.
Embodiment one
The present embodiment provides a kind of high degree of comfort three-dimensional video-frequency generation methods, can be applied to mobile phone, tablet computer, notes
This computer, desktop PC, palm PC, cloud server, intelligent TV set etc. have setting for video data processing function
It is standby.
As shown in Figure 1, high degree of comfort three-dimensional video-frequency generation method provided in this embodiment, comprising:
Step S101, training set is established according to the stereo video data of input.
In a particular application, the stereo video data of input is that comfort level is lower, needs to be improved the three-dimensional video-frequency of comfort level
Data.Stereo video data includes according to the tactic multiple image of acquisition time, and every frame image includes the left side to match
Eye image and eye image.Pixel of the same name is (i.e. for showing identical picture in all left-eye images and eye image to match
The pixel of element) the distance between set, the view of as left-eye image and eye image is poor.The view of left-and right-eye images
Official post obtains stereo video data and shows 3D effect in visual effect, while also bringing the problem of parallax experienced of images of left and right eyes, thus
Causing long-time to watch three-dimensional video-frequency can make human eye tired, and comfort level is bad.
In a particular application, training set refers to by the image set of all frame image constructions in the stereo video data that inputs
It closes.
In one embodiment, step S101 includes:
The training set with mark is established according to the stereo video data of input.
In a particular application, the training set with mark refers to that every frame image in training set all has mark.Make image
With the true picture and the subsequent image generated by generating confrontation network that mark is to distinguish input.
Step S102, confrontation network is generated by training set training, generates new stereo video data；Wherein, institute
Stating generation confrontation network includes generator and arbiter.
In a particular application, confrontation network (Generative Adversarial is generated by training set training
Networks, GAN), the stereo video data of the true stereo video data close to input can be generated.Generator
(Generator) effect is to receive a random noise, is generated by this noise close to true stereopsis frequency
According to stereo video data.The effect of arbiter (Discriminator) is to judge whether the data of its input are true.
In one embodiment, the generation confrontation network is that depth convolution generates confrontation network (Deep
Convolutional Generative Adversarial Networks, DCGAN).
In one embodiment, the formula of the training process in step S102 is expressed as follows:
Wherein, V (D, G) is cost function, and D is arbiter, and G is generator, and E is encoder, and x indicates a frame image,
pdata(x) true stereo video data is indicated, D (x) indicates x from true stereo video data rather than from p_g (x)
Probability function, p_g (x) is sample distribution, the parameter of the parameter of x or generator that generator generates, and z indicates latent space
One sample or the noise of generator input, pzIt (z) is input noise variable, G (z) indicates that latent space or generator are raw according to z
At stereo video data.
Step S103, by the training set and the new stereo video data training arbiter, and will differentiate
As a result the generator is fed back to, to update the parameter of the generator.
In a particular application, whether arbiter is true stereopsis for discriminative training collection and new stereo video data
Frequency evidence, i.e. discriminative training integrate and the new stereo video data is the probability of truthful data, differentiate that result is described general
Rate.Assuming that the value range of probability is 0~100%, then arbiter output differentiates that the data that result is 0 expression input are not true
Data differentiate that the data that result is 100% expression input are truthful datas.The target of arbiter is just to try to the instruction being inputted
Lian Ji and new stereo video data distinguish.
In a particular application, the parameter of generator is the p_g in the formula of the training process in above-mentioned steps S102
(x)。
Step S104, it passes back through the training set training and generates confrontation network, generate new stereo video data, until
The generation fights network convergence.
In a particular application, to make generator and the mutual game of arbiter, generate generator close to truthful data
Data make the data of arbiter study difference truthful data and generation, finally achieve generator and generate close to truthful data
Ability and arbiter differentiate the balance between the ability of true and false data, realize the convergence for generating confrontation network.
In one embodiment, step S104 includes:
It passes back through the training set training and generates confrontation network, generate new stereo video data, until p_g (x) is received
It holds back to level off to pdata(x) or equal to pdata(x)。
In a particular application, generate confrontation network convergence be instigate training process in above-mentioned steps S102 formula
In p_g (x) converge to and level off to pdata(x) or equal to pdata(x).Work as pdata(x)=pdata(x) when, D (x)=1/2, training
Process stops, and reaches the balance of decision errors between generator and arbiter.
Step S105, the new stereo video data for generating and generating when fighting network convergence is obtained, as high comfortable
The stereo video data of degree.
In a particular application, the image in new stereoscopic video images and training set generated when generating confrontation network convergence
Comfort level reach unanimity or identical, the comfort level of stereo video data can be improved, obtain the stereopsis frequency of high degree of comfort
According to.
The present embodiment generates confrontation net by establishing training set according to the stereo video data of input, by training set training
Network generates new stereo video data, then by training set and new stereo video data training arbiter, and ties differentiating
Fruit feeds back to generator, to update the parameter of generator, and returns to generate confrontation network again by training set training, generates
New stereo video data when fighting network convergence until generating, obtains relaxing for all frame images for generating confrontation network generation
The stereo video data that appropriateness reaches unanimity obtains high degree of comfort so as to improve the comfort level of stereo video data
Stereo video data.
Embodiment two
As shown in Fig. 2, in the present embodiment, the step S101 in embodiment one includes:
Step S202, the stereo video data of input is converted to image sequence, described image sequence is by the solid
Every frame image in video data is according to sequence made of the arrangement of acquisition time sequence.
In a particular application, by stereo video data be converted into image sequence be for the ease of it is subsequent to each frame image into
Row down-sampling.Since frame image every in stereo video data all acquires sequentially in time, it can be according to every frame figure
All frame graphical arrangements of stereo video data are image sequence by the acquisition time sequence of picture.
As shown in figure 3, illustratively showing image sequence made of the acquisition time sequence arrangement according to every frame image.
In a particular application, the frame per second of stereo video data is determined by acquired image quantity in the unit time, Fig. 3 institute
The being merely examples property of image sequence shown, the density of image does not represent the size of frame per second, and time shaft t is without in restriction
Specific time span.
Step S203, down-sampling is carried out to every frame image in described image sequence according to the sampling window for presetting size.
In a particular application, default size can be set according to actual needs, and default size should be less than or equal to solid
The smallest frame image in video data.For example, the size of the smallest frame image is 700 × 700 (pixel sizes), then preset
Size should be less than or equal to 700 × 700.
In a particular application, there are two the main purposes of down-sampling (subsampled): first is that image is made to meet display
The size in region；Second is that, generate correspondence image thumbnail.Down-sampling can make the big of every frame image in stereo video data
It is small to reach unanimity, so that the size of every frame image in the new stereo video data ultimately generated also reaches unanimity, thus
The display effect of the new stereo video data ultimately generated is improved, comfort level when human eye viewing is improved.In some cases,
Down-sampling can also increase the information of image, so that the picture quality after down-sampling is more than original map quality.
Step S204, training set is established according to the described image sequence after down-sampling.
In one embodiment, step S204 includes:
Every frame image in described image sequence after marking down-sampling；
It establishes by the training set of all frame image constructions with mark.
In a particular application, by every frame image after mark down-sampling, convenient for distinguishing the true of input in subsequent step
The image that the image and generator of stereo data generate.
In a particular application, it in order to improve annotating efficiency, can all be used for every frame image in image sequence identical
Notation methods, Batch labeling is carried out to all frame images of image sequence；It can also be according to frame image every in image sequence
Acquisition time sequence, all stamps timestamp to every frame image, timestamp is the mark of image.
The present embodiment by according to acquisition time sequence by the graphical arrangement in the stereo video data of input at image sequence
Then column carry out down-sampling to every frame image, the size of every frame image in stereo video data are made all to reach unanimity, be conducive to
After being trained to generation confrontation network, the consistent stereo video data of display effect is generated, thus when improving human eye viewing
Comfort level；By every frame image in the stereo video data of mark input, convenient for distinguishing truthful data and the generation of input
Fight the data that network generates.
It should be understood that the size of the serial number of each step is not meant that the order of the execution order in above-described embodiment, each process
Execution sequence should be determined by its function and internal logic, the implementation process without coping with the embodiment of the present application constitutes any limit
It is fixed.
Embodiment three
The present embodiment provides a kind of high degree of comfort three-dimensional video-frequencies to generate system, for executing in embodiment one or embodiment two
Method and step, the system can be mobile phone, tablet computer, laptop, desktop PC, palm PC, cloud clothes
Business device, intelligent TV set etc. have the software program system in the equipment of video data processing function.
As shown in figure 4, high degree of comfort three-dimensional video-frequency provided in this embodiment generates system 4, comprising:
Module 401 is established, for establishing training set according to the stereo video data of input；
Generation module 402 generates new stereo video data for generating confrontation network by training set training；
Wherein, the generation confrontation network includes generator and arbiter；
Update module 403, for training the arbiter by the training set and the new stereo video data, and
It will differentiate that result feeds back to the generator, to update the parameter of the generator；
Return module 404 generates confrontation network for passing back through the training set training, generates new stereopsis frequency
According to until the generation fights network convergence；
Module 405 is obtained, the new stereo video data generated when for obtaining the generation confrontation network convergence, as
The stereo video data of high degree of comfort.
In one embodiment, establishing module 401 includes:
Conversion unit, for the stereo video data of input to be converted to image sequence, described image sequence is will be described
Every frame image in stereo video data is according to sequence made of the arrangement of acquisition time sequence；
Sampling unit, for adopt to every frame image in described image sequence according to the sampling window for presetting size
Sample；
Unit is established, for establishing training set according to the described image sequence after down-sampling.
In one embodiment, the unit of establishing includes:
Subelement is marked, for marking every frame image in the described image sequence after down-sampling；
Subelement is established, for establishing by the training set of all frame image constructions with mark.
The present embodiment generates confrontation net by establishing training set according to the stereo video data of input, by training set training
Network generates new stereo video data, then by training set and new stereo video data training arbiter, and ties differentiating
Fruit feeds back to generator, to update the parameter of generator, and returns to generate confrontation network again by training set training, generates
New stereo video data when fighting network convergence until generating, obtains relaxing for all frame images for generating confrontation network generation
The stereo video data that appropriateness reaches unanimity obtains high degree of comfort so as to improve the comfort level of stereo video data
Stereo video data.
Example IV
As shown in figure 5, the present embodiment provides a kind of terminal devices 5 comprising: processor 50, memory 51 and storage
In the memory 51 and the computer program 52 that can run on the processor 50, such as high degree of comfort three-dimensional video-frequency are raw
At program.The processor 50 realizes above-mentioned each high degree of comfort three-dimensional video-frequency generation method when executing the computer program 52
Step in embodiment, such as step S101 to S105 shown in FIG. 1.Alternatively, the processor 50 executes the computer journey
The function of each module/unit in above-mentioned each Installation practice, such as the function of module 401 to 405 shown in Fig. 4 are realized when sequence 52.
Illustratively, the computer program 52 can be divided into one or more module/units, it is one or
Multiple module/units are stored in the memory 51, and are executed by the processor 50, to complete the application.Described one
A or multiple module/units can be the series of computation machine program instruction section that can complete specific function, which is used for
Implementation procedure of the computer program 52 in the terminal device 5 is described.For example, the computer program 52 can be divided
It is cut into and establishes module, generation module, update module, return module and obtain module, each module concrete function is as follows:
Module is established, for establishing training set according to the stereo video data of input；
Generation module generates new stereo video data for generating confrontation network by training set training；Its
In, the generation confrontation network includes generator and arbiter；
Update module, for training the arbiter by the training set and the new stereo video data, and will
Differentiate that result feeds back to the generator, to update the parameter of the generator；
Return module generates confrontation network for passing back through the training set training, generates new stereo video data,
Until the generation fights network convergence；
Module is obtained, the new stereo video data generated when for obtaining the generation confrontation network convergence, as height
The stereo video data of comfort level.
The terminal device 5 can be the calculating such as desktop PC, notebook, palm PC and cloud server and set
It is standby.The terminal device may include, but be not limited only to, processor 50, memory 51.It will be understood by those skilled in the art that Fig. 5
The only example of terminal device 5 does not constitute the restriction to terminal device 5, may include than illustrating more or fewer portions
Part perhaps combines certain components or different components, such as the terminal device can also include input-output equipment, net
Network access device, bus etc..
Alleged processor 50 can be central processing unit (Central Processing Unit, CPU), can also be
Other general processors, digital signal processor (Digital Signal Processor, DSP), specific integrated circuit
(Application Specific Integrated Circuit, ASIC), ready-made programmable gate array (Field-
Programmable Gate Array, FPGA) either other programmable logic device, discrete gate or transistor logic,
Discrete hardware components etc..General processor can be microprocessor or the processor is also possible to any conventional processor
Deng.
The memory 51 can be the internal storage unit of the terminal device 5, such as the hard disk or interior of terminal device 5
It deposits.The memory 51 is also possible to the External memory equipment of the terminal device 5, such as be equipped on the terminal device 5
Plug-in type hard disk, intelligent memory card (Smart Media Card, SMC), secure digital (Secure Digital, SD) card dodge
Deposit card (Flash Card) etc..Further, the memory 51 can also both include the storage inside list of the terminal device 5
Member also includes External memory equipment.The memory 51 is for storing needed for the computer program and the terminal device
Other programs and data.The memory 51 can be also used for temporarily storing the data that has exported or will export.
It is apparent to those skilled in the art that for convenience of description and succinctly, only with above-mentioned each function
Can unit, module division progress for example, in practical application, can according to need and by above-mentioned function distribution by different
Functional unit, module are completed, i.e., the internal structure of described device is divided into different functional unit or module, more than completing
The all or part of function of description.Each functional unit in embodiment, module can integrate in one processing unit, can also
To be that each unit physically exists alone, can also be integrated in one unit with two or more units, it is above-mentioned integrated
Unit both can take the form of hardware realization, can also realize in the form of software functional units.In addition, each function list
Member, the specific name of module are also only for convenience of distinguishing each other, the protection scope being not intended to limit this application.Above system
The specific work process of middle unit, module, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.
In the above-described embodiments, it all emphasizes particularly on different fields to the description of each embodiment, is not described in detail or remembers in some embodiment
The part of load may refer to the associated description of other embodiments.
Those of ordinary skill in the art may be aware that list described in conjunction with the examples disclosed in the embodiments of the present disclosure
Member and algorithm steps can be realized with the combination of electronic hardware or computer software and electronic hardware.These functions are actually
It is implemented in hardware or software, the specific application and design constraint depending on technical solution.Professional technician
Each specific application can be used different methods to achieve the described function, but this realization is it is not considered that exceed
Scope of the present application.
In embodiment provided herein, it should be understood that disclosed device/terminal device and method, it can be with
It realizes by another way.For example, device described above/terminal device embodiment is only schematical, for example, institute
The division of module or unit is stated, only a kind of logical function partition, there may be another division manner in actual implementation, such as
Multiple units or components can be combined or can be integrated into another system, or some features can be ignored or not executed.Separately
A bit, shown or discussed mutual coupling or direct-coupling or communication connection can be through some interfaces, device
Or the INDIRECT COUPLING or communication connection of unit, it can be electrical property, mechanical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme
's.
It, can also be in addition, each functional unit in each embodiment of the application can integrate in one processing unit
It is that each unit physically exists alone, can also be integrated in one unit with two or more units.Above-mentioned integrated list
Member both can take the form of hardware realization, can also realize in the form of software functional units.
If the integrated module/unit be realized in the form of SFU software functional unit and as independent product sale or
In use, can store in a computer readable storage medium.Based on this understanding, the application realizes above-mentioned implementation
All or part of the process in example method, can also instruct relevant hardware to complete, the meter by computer program
Calculation machine program can be stored in a computer readable storage medium, the computer program when being executed by processor, it can be achieved that on
The step of stating each embodiment of the method.Wherein, the computer program includes computer program code, the computer program
Code can be source code form, object identification code form, executable file or certain intermediate forms etc..Computer-readable Jie
Matter may include: can carry the computer program code any entity or device, recording medium, USB flash disk, mobile hard disk,
Magnetic disk, CD, computer storage, read-only memory (ROM, Read-Only Memory), random access memory (RAM,
Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium etc..It should be noted that described
The content that computer-readable medium includes can carry out increasing appropriate according to the requirement made laws in jurisdiction with patent practice
Subtract, such as in certain jurisdictions, according to legislation and patent practice, computer-readable medium do not include be electric carrier signal and
Telecommunication signal.
Embodiment described above is only to illustrate the technical solution of the application, rather than its limitations；Although referring to aforementioned reality
Example is applied the application is described in detail, those skilled in the art should understand that: it still can be to aforementioned each
Technical solution documented by embodiment is modified or equivalent replacement of some of the technical features；And these are modified
Or replacement, the spirit and scope of each embodiment technical solution of the application that it does not separate the essence of the corresponding technical solution should all
Comprising within the scope of protection of this application.
Claims (10)
1. a kind of high degree of comfort three-dimensional video-frequency generation method characterized by comprising
Training set is established according to the stereo video data of input；
Confrontation network is generated by training set training, generates new stereo video data；Wherein, the generation fights network
Including generator and arbiter；
By the training set and the new stereo video data training arbiter, and it is described to differentiate that result is fed back to
Generator, to update the parameter of the generator；
It passes back through the training set training and generates confrontation network, generate new stereo video data, until the generation is fought
Network convergence；
Obtain the new stereo video data for generating and generating when fighting network convergence, the stereopsis frequency as high degree of comfort
According to.
2. high degree of comfort three-dimensional video-frequency generation method as described in claim 1, which is characterized in that according to the three-dimensional video-frequency of input
Data establish training set, comprising:
The stereo video data of input is converted to image sequence, described image sequence is will be every in the stereo video data
Sequence made of frame image is arranged according to acquisition time sequence；
Down-sampling is carried out to every frame image in described image sequence according to the sampling window of default size；
Training set is established according to the described image sequence after down-sampling.
3. high degree of comfort three-dimensional video-frequency generation method as claimed in claim 2, which is characterized in that according to after down-sampling
Image sequence establishes training set, comprising:
Every frame image in described image sequence after marking down-sampling；
It establishes by the training set of all frame image constructions with mark.
4. high degree of comfort three-dimensional video-frequency generation method as described in claim 1, which is characterized in that pass through training set training
Confrontation network is generated, the formula for generating the training process of new stereo video data is expressed as follows:
Wherein, V (D, G) is cost function, and D is arbiter, and G is generator, and E is encoder, and x indicates a frame image, pdata(x)
Indicate true stereo video data, D (x) indicates probability of the x from true stereo video data rather than from p_g (x)
Function, p_g (x) is sample distribution, the parameter of the parameter of x or generator that generator generates, and z indicates a sample of latent space
The noise of this or generator input, pzIt (z) is input noise variable, it is vertical that G (z) indicates that latent space or generator are generated according to z
Volumetric video data.
5. high degree of comfort three-dimensional video-frequency generation method as claimed in claim 4, which is characterized in that pass back through the training set
Training generates confrontation network, generates new stereo video data, until the generation fights network convergence, comprising:
It passes back through the training set training and generates confrontation network, new stereo video data is generated, until p_g (x) is converged to
Level off to pdata(x) or equal to pdata(x)。
6. a kind of high degree of comfort three-dimensional video-frequency generates system characterized by comprising
Module is established, for establishing training set according to the stereo video data of input；
Generation module generates new stereo video data for generating confrontation network by training set training；Wherein, institute
Stating generation confrontation network includes generator and arbiter；
Update module for training the arbiter by the training set and the new stereo video data, and will differentiate
As a result the generator is fed back to, to update the parameter of the generator；
Return module generates confrontation network for passing back through the training set training, generates new stereo video data, until
The generation fights network convergence；
Module is obtained, the new stereo video data generated when for obtaining the generation confrontation network convergence, as high comfortable
The stereo video data of degree.
7. high degree of comfort three-dimensional video-frequency as claimed in claim 6 generates system, which is characterized in that the module of establishing includes:
Conversion unit, for the stereo video data of input to be converted to image sequence, described image sequence is by the solid
Every frame image in video data is according to sequence made of the arrangement of acquisition time sequence；
Sampling unit, for carrying out down-sampling to every frame image in described image sequence according to the sampling window for presetting size；
Unit is established, for establishing training set according to the described image sequence after down-sampling.
8. high degree of comfort three-dimensional video-frequency generation method as claimed in claim 7, which is characterized in that the unit of establishing includes:
Subelement is marked, for marking every frame image in the described image sequence after down-sampling；
Subelement is established, for establishing by the training set of all frame image constructions with mark.
9. a kind of terminal device, including memory, processor and storage are in the memory and can be on the processor
The computer program of operation, which is characterized in that the processor realizes such as claim 1 to 5 when executing the computer program
The step of any one the method.
10. a kind of computer readable storage medium, the computer-readable recording medium storage has computer program, and feature exists
In when the computer program is executed by processor the step of any one of such as claim 1 to 5 of realization the method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811198022.8A CN109151443A (en) | 2018-10-15 | 2018-10-15 | High degree of comfort three-dimensional video-frequency generation method, system and terminal device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811198022.8A CN109151443A (en) | 2018-10-15 | 2018-10-15 | High degree of comfort three-dimensional video-frequency generation method, system and terminal device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109151443A true CN109151443A (en) | 2019-01-04 |
Family
ID=64811727
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811198022.8A Pending CN109151443A (en) | 2018-10-15 | 2018-10-15 | High degree of comfort three-dimensional video-frequency generation method, system and terminal device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109151443A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111669563A (en) * | 2020-06-19 | 2020-09-15 | 福州大学 | Stereo image visual comfort enhancement method based on reinforcement learning |
| CN112329932A (en) * | 2020-10-30 | 2021-02-05 | 深圳市优必选科技股份有限公司 | Training method and device for generating countermeasure network and terminal equipment |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106952239A (en) * | 2017-03-28 | 2017-07-14 | 厦门幻世网络科技有限公司 | image generating method and device |
| CN106961603A (en) * | 2017-03-07 | 2017-07-18 | 腾讯科技（深圳）有限公司 | Intracoded frame code rate allocation method and device |
| CN107358626A (en) * | 2017-07-17 | 2017-11-17 | 清华大学深圳研究生院 | A kind of method that confrontation network calculations parallax is generated using condition |
| CN107578386A (en) * | 2017-09-01 | 2018-01-12 | 荆门程远电子科技有限公司 | A kind of optimization defogging processing method of unmanned plane shooting image |
| CN107945282A (en) * | 2017-12-05 | 2018-04-20 | 洛阳中科信息产业研究院(中科院计算技术研究所洛阳分所) | The synthesis of quick multi-view angle three-dimensional and methods of exhibiting and device based on confrontation network |
| CN108022213A (en) * | 2017-11-29 | 2018-05-11 | 天津大学 | Video super-resolution algorithm for reconstructing based on generation confrontation network |
| CN108021131A (en) * | 2017-11-28 | 2018-05-11 | 王智华 | A kind of robot vision recognition methods, device and computer-readable recording medium |
| CN108171769A (en) * | 2018-01-15 | 2018-06-15 | 成都睿码科技有限责任公司 | The faceform's generation method and human face generating method of a kind of sequence based on DNA |
| CN108416324A (en) * | 2018-03-27 | 2018-08-17 | 百度在线网络技术（北京）有限公司 | Method and apparatus for detecting live body |
| CN108495110A (en) * | 2018-01-19 | 2018-09-04 | 天津大学 | A kind of virtual visual point image generating method fighting network based on production |
| CN108537152A (en) * | 2018-03-27 | 2018-09-14 | 百度在线网络技术（北京）有限公司 | Method and apparatus for detecting live body |
| CN108629823A (en) * | 2018-04-10 | 2018-10-09 | 北京京东尚科信息技术有限公司 | The generation method and device of multi-view image |
-
2018
- 2018-10-15 CN CN201811198022.8A patent/CN109151443A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106961603A (en) * | 2017-03-07 | 2017-07-18 | 腾讯科技（深圳）有限公司 | Intracoded frame code rate allocation method and device |
| CN106952239A (en) * | 2017-03-28 | 2017-07-14 | 厦门幻世网络科技有限公司 | image generating method and device |
| CN107358626A (en) * | 2017-07-17 | 2017-11-17 | 清华大学深圳研究生院 | A kind of method that confrontation network calculations parallax is generated using condition |
| CN107578386A (en) * | 2017-09-01 | 2018-01-12 | 荆门程远电子科技有限公司 | A kind of optimization defogging processing method of unmanned plane shooting image |
| CN108021131A (en) * | 2017-11-28 | 2018-05-11 | 王智华 | A kind of robot vision recognition methods, device and computer-readable recording medium |
| CN108022213A (en) * | 2017-11-29 | 2018-05-11 | 天津大学 | Video super-resolution algorithm for reconstructing based on generation confrontation network |
| CN107945282A (en) * | 2017-12-05 | 2018-04-20 | 洛阳中科信息产业研究院(中科院计算技术研究所洛阳分所) | The synthesis of quick multi-view angle three-dimensional and methods of exhibiting and device based on confrontation network |
| CN108171769A (en) * | 2018-01-15 | 2018-06-15 | 成都睿码科技有限责任公司 | The faceform's generation method and human face generating method of a kind of sequence based on DNA |
| CN108495110A (en) * | 2018-01-19 | 2018-09-04 | 天津大学 | A kind of virtual visual point image generating method fighting network based on production |
| CN108416324A (en) * | 2018-03-27 | 2018-08-17 | 百度在线网络技术（北京）有限公司 | Method and apparatus for detecting live body |
| CN108537152A (en) * | 2018-03-27 | 2018-09-14 | 百度在线网络技术（北京）有限公司 | Method and apparatus for detecting live body |
| CN108629823A (en) * | 2018-04-10 | 2018-10-09 | 北京京东尚科信息技术有限公司 | The generation method and device of multi-view image |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111669563A (en) * | 2020-06-19 | 2020-09-15 | 福州大学 | Stereo image visual comfort enhancement method based on reinforcement learning |
| CN111669563B (en) * | 2020-06-19 | 2021-06-25 | 福州大学 | Stereo image visual comfort enhancement method based on reinforcement learning |
| CN112329932A (en) * | 2020-10-30 | 2021-02-05 | 深圳市优必选科技股份有限公司 | Training method and device for generating countermeasure network and terminal equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20180332222A1 (en) | Method and apparatus for obtaining binocular panoramic image, and storage medium | |
| CN107945282A (en) | The synthesis of quick multi-view angle three-dimensional and methods of exhibiting and device based on confrontation network | |
| CN109472764B (en) | Method, apparatus, device and medium for image synthesis and image synthesis model training | |
| CN107852487A (en) | Method for the electronic equipment of 360 degree of 3-D views of generation and for the electronic equipment | |
| CN102510508B (en) | Detection-type stereo picture adjusting device and method | |
| JP2014532941A (en) | Generation of a depth map for an input image using an exemplary approximate depth map associated with an exemplary similar image | |
| KR20120049997A (en) | Image process device, display apparatus and methods thereof | |
| US9355436B2 (en) | Method, system and computer program product for enhancing a depth map | |
| CN103248855B (en) | Course recording system based on fixing seat in the plane, video process apparatus and record class method | |
| CN108230384A (en) | Picture depth computational methods, device, storage medium and electronic equipment | |
| CN106296781A (en) | Specially good effect image generating method and electronic equipment | |
| CN115272570A (en) | Virtual expression generation method and device, electronic equipment and storage medium | |
| CN108833877A (en) | Image processing method and device, computer installation and readable storage medium storing program for executing | |
| CN106303491A (en) | Image processing method and device | |
| CN106228530A (en) | A kind of stereography method, device and stereophotography equipment | |
| CN109151443A (en) | High degree of comfort three-dimensional video-frequency generation method, system and terminal device | |
| CN109920043A (en) | The three-dimensional rendering of virtual 3D object | |
| WO2024056020A1 (en) | Binocular image generation method and apparatus, electronic device and storage medium | |
| CN106303493A (en) | Image processing method and device | |
| CN103026387B (en) | Method for generating multiple view picture from single image | |
| CN110659440B (en) | Method for rapidly and dynamically displaying different detail levels of point cloud data large scene | |
| Jiang et al. | Visual comfort assessment for stereoscopic images based on sparse coding with multi-scale dictionaries | |
| CN102521876A (en) | Method and system for realizing three dimensional (3D) stereoscopic effect of user interface | |
| CN106358006B (en) | The bearing calibration of video and device | |
| CN106249857B (en) | A kind of display converting method, device and terminal device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190104 |