CN110223228A - Image processing method, device, terminal device and storage medium - Google Patents
Image processing method, device, terminal device and storage medium Download PDFInfo
- Publication number
- CN110223228A CN110223228A CN201910407795.0A CN201910407795A CN110223228A CN 110223228 A CN110223228 A CN 110223228A CN 201910407795 A CN201910407795 A CN 201910407795A CN 110223228 A CN110223228 A CN 110223228A
- Authority
- CN
- China
- Prior art keywords
- textural characteristics
- image
- resolution
- model
- texture feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 17
- 230000002708 enhancing effect Effects 0.000 claims abstract description 47
- 238000000034 method Methods 0.000 claims description 36
- 230000000007 visual effect Effects 0.000 claims description 24
- 238000004590 computer program Methods 0.000 claims description 17
- 238000009877 rendering Methods 0.000 claims description 17
- 238000003062 neural network model Methods 0.000 claims description 16
- 239000012141 concentrate Substances 0.000 claims description 2
- 230000008569 process Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 238000007654 immersion Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/005—General purpose rendering architectures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/04—Texture mapping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Graphics (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
Abstract
This application provides a kind of image processing method, device, terminal device and storage mediums.In this application, first textural characteristics of original image can be loaded on the surface of panorama model, then the second textural characteristics loaded on the surface of panorama model are obtained, first textural characteristics include the second textural characteristics, super-resolution enhancing processing is carried out to the second textural characteristics again, third texture feature is obtained, renders third texture feature on the display interface of panoramic apparatus later.In this application, due to carrying out super-resolution enhancing processing to the second textural characteristics, so that the resolution ratio of the obtained corresponding image of third texture feature is greater than the resolution ratio of the corresponding image of the second textural characteristics, the high resolution of the corresponding image of third texture feature is in the corresponding image of the second textural characteristics in the prior art, it is thus possible to improve the clarity of image.
Description
Technical field
This application involves Internet technical fields, more particularly to a kind of image processing method, device, terminal device and deposit
Storage media.
Background technique
VR (Virtual Reality, virtual reality) technology is a kind of computer simulation technique, it uses headset equipment
The feeling such as image and sound true to nature is generated, the experience of immersion is provided the user with.Panoramic video is that one kind of VR technology is important
Application scenarios, it allows viewer that can see 360 degree of comprehensive images, can experience feeling on the spot in person.
The generic way of panoramic video is that the textural characteristics of original image are mapped on the surface of panorama model, such as ball
Model or day sylphon etc., the texture then rendered on the display interface of headset equipment on the surface for being mapped to panorama model are special
Sign.
However, since panorama model is spherical model, and since the field angle of headset equipment can only cover spherical model
Part of the surface, therefore, when the textural characteristics of original image are mapped on the surface of panorama model, usually by original graph
The textural characteristics of picture are mapped on the surface of the part of panorama model, so as to cause the textural characteristics pair on panoramic surface are mapped to
The resolution ratio for the image answered is lower than the resolution ratio of original image, can reduce the clarity of image in this way.
For example, it is assumed that original image is the resolution ratio of 1080*720P, the resolution ratio of all surfaces of panorama model is also
The textural characteristics of original image can only be mapped to panorama mould if the field angle of headset equipment is 90 ° by 1080*720P
On 1/4 surface of type, the resolution ratio on 1/4 surface of panorama model is only 640*360P, therefore, is mapped to panorama model
The resolution ratio of the corresponding image of textural characteristics on 1/4 surface is 640*360P, finally to render in the display interface
The resolution ratio of image out is only 640*360P, compared to the resolution ratio of the 1080*720P of original image, reduces 4 times of point
Resolution, so that the clarity of image can be reduced.
Summary of the invention
In order to solve the above technical problems, the embodiment of the present application show a kind of image processing method, device, terminal device and
Storage medium.
In a first aspect, the embodiment of the present application shows a kind of image processing method, which comprises
First textural characteristics of original image are loaded on the surface of panorama model；
The second textural characteristics loaded on the surface of the panorama model are obtained, first textural characteristics include described
Second textural characteristics；
Super-resolution enhancing processing is carried out to second textural characteristics, obtains third texture feature, the third texture
The resolution ratio of the corresponding image of feature is greater than the resolution ratio of the corresponding image of second textural characteristics；
The third texture feature is rendered on the display interface of panoramic apparatus.
It is described to obtain the second texture spy loaded on the surface of the panorama model in an optional implementation
Sign, comprising:
Determine the current visual angle of the panoramic apparatus；
Determine view field of the current visual angle on the surface of the panorama model；
On the surface of the panorama model, the textural characteristics being located in the view field are obtained, and as described the
Two textural characteristics.
In an optional implementation, the current visual angle of the determination panoramic apparatus, comprising:
Obtain the parameter of the gyroscope in the panoramic apparatus；
The current visual angle of the panoramic apparatus is determined according to the parameter of the gyroscope.
It is described that super-resolution enhancing processing is carried out to second textural characteristics in an optional implementation, it obtains
To third texture feature, comprising:
Second textural characteristics are input in super-resolution enhancing model, it is defeated to obtain the super-resolution enhancing model
Textural characteristics out, and as third texture feature.
In an optional implementation, the method also includes:
Multiple sample graph image sets are obtained, each sample image concentrates the sample image including multiple same scenes, and more
It include the sample of at least one high-resolution sample image and at least one low resolution in the sample image of a same scene
Image；
The sample image concentrated using the multiple sample image is trained preset neural network model, until institute
The weight stated in preset neural network model restrains, and obtains the super-resolution enhancing model.
It is described that the third texture spy is rendered on the display interface of panoramic apparatus in an optional implementation
Sign, comprising:
Obtain the anti-distortion parameter of the panoramic apparatus；
Anti- distortion processing is carried out to the third texture feature based on the anti-distortion parameter, obtains the 4th textural characteristics；
The 4th textural characteristics are rendered on the display interface of panoramic apparatus.
Second aspect, the embodiment of the present application show a kind of image processing apparatus, and described device includes:
Loading module is loaded onto the surface of panorama model for the first textural characteristics by original image；
Module is obtained, for obtaining the second textural characteristics loaded on the surface of the panorama model, first line
Managing feature includes second textural characteristics；
Enhance processing module, for carrying out super-resolution enhancing processing to second textural characteristics, obtains third texture
The resolution ratio of feature, the corresponding image of the third texture feature is greater than the resolution of the corresponding image of second textural characteristics
Rate；
Rendering module, for rendering the third texture feature on the display interface of panoramic apparatus.
In an optional implementation, the acquisition module includes:
First determination unit, for determining the current visual angle of the panoramic apparatus；
Second determination unit, for determining view field of the current visual angle on the surface of the panorama model；
First acquisition unit, for obtaining the texture being located in the view field on the surface of the panorama model
Feature, and as second textural characteristics.
In an optional implementation, first determination unit includes:
Subelement is obtained, for obtaining the parameter of the gyroscope in the panoramic apparatus；
Subelement is determined, for determining the current visual angle of the panoramic apparatus according to the parameter of the gyroscope.
In an optional implementation, the enhancing processing module includes:
Input unit obtains the oversubscription for second textural characteristics to be input in super-resolution enhancing model
Resolution enhances the textural characteristics of model output, and as third texture feature.
In an optional implementation, the enhancing processing module further include:
Second acquisition unit, for obtaining multiple sample graph image sets, it includes multiple identical fields that each sample image, which is concentrated,
The sample image of scape, and include at least one high-resolution sample image and at least one in the sample image of multiple same scenes
The sample image of a low resolution；
Training unit, sample image for being concentrated using the multiple sample image to preset neural network model into
Row training obtains the super-resolution enhancing model until the weight in the preset neural network model restrains.
In an optional implementation, the rendering module includes:
Third acquiring unit, for obtaining the anti-distortion parameter of the panoramic apparatus；
Anti- distortion processing unit, for being carried out at anti-distortion based on the anti-distortion parameter to the third texture feature
Reason, obtains the 4th textural characteristics；
Rendering unit, for rendering the 4th textural characteristics on the display interface of panoramic apparatus.
The third aspect, the embodiment of the present application show a kind of terminal device, comprising: memory, processor and are stored in institute
The computer program that can be run on memory and on the processor is stated, when the computer program is executed by the processor
Realize image processing method described in any of the above embodiments.
Fourth aspect, the embodiment of the present application show a kind of computer readable storage medium, the computer-readable storage
Computer program is stored on medium, the computer program is realized when being executed by processor at image described in any of the above embodiments
Reason method.
Compared with prior art, the embodiment of the present application includes the following advantages:
In this application, the first textural characteristics of original image can be loaded on the surface of panorama model, is then obtained
The second textural characteristics loaded on the surface of panorama model are taken, the first textural characteristics include the second textural characteristics, then to second
Textural characteristics carry out super-resolution enhancing processing, obtain third texture feature, render on the display interface of panoramic apparatus later
Third texture feature.
It in the prior art, is that the second textural characteristics are directly rendered on the display interface of panoramic apparatus.
And in this application, due to carrying out super-resolution enhancing processing to the second textural characteristics, so that obtain the
The resolution ratio of the corresponding image of three textural characteristics is greater than the resolution ratio of the corresponding image of the second textural characteristics, third texture feature pair
The high resolution for the image answered is in the corresponding image of the second textural characteristics in the prior art, it is thus possible to improve image is clear
Clear degree.
For example, it is assumed that original image is the resolution ratio of 1080*720P, the resolution ratio of all surfaces of panorama model is also
The textural characteristics of original image can only be mapped to panorama mould if the field angle of headset equipment is 90 ° by 1080*720P
On 1/4 surface of type, the resolution ratio on 1/4 surface of panorama model is only 640*360P, therefore, is mapped to panorama model
The resolution ratio of the corresponding image of textural characteristics on 1/4 surface is 640*360P, finally to render in the display interface
The resolution ratio of image out is only 640*360P.
And in this application, super-resolution enhancing processing is carried out to the textural characteristics on the surface for being mapped to panorama model,
The high resolution of treated the corresponding image of textural characteristics is possibly even higher than 1080*720P in 640*360P, therefore,
User in compared with the prior art is only capable of the image for watching resolution ratio to be 640*360P, and the application can enable a user to
High resolution is watched to be above the image of 1080*720P in the image or even resolution ratio of 640*360P, so as to improve figure
The clarity of picture.
Detailed description of the invention
Fig. 1 is a kind of step flow chart of image processing method embodiment of the application；
Fig. 2 is a kind of step flow chart of the embodiment of the method for rendering third texture feature of the application；
Fig. 3 is a kind of step flow chart of the embodiment of the method for the second textural characteristics of acquisition of the application；
Fig. 4 is a kind of structural block diagram of image processing apparatus embodiment of the application.
Specific embodiment
In order to make the above objects, features, and advantages of the present application more apparent, with reference to the accompanying drawing and it is specific real
Applying mode, the present application will be further described in detail.
Referring to Fig.1, a kind of step flow chart of image processing method embodiment of the application is shown, this method is applied to
Panoramic apparatus, such as VR headset equipment etc., this method can specifically include following steps:
In step s101, the first textural characteristics of original image are loaded on the surface of panorama model；
In this application, when needing to show panoramic video to user, due to including multiframe original image in video, because
This, each frame original image can also be shown respectively in the way of the application, so that user can watch panoramic video.
Wherein, for any one frame original image in video, the first texture of the available original image of panoramic apparatus
Feature, for example, generating OES (OpenGL for Embedded System, the opening figure of embedded system according to original image
Library) type the first textural characteristics, then the first textural characteristics of original image are loaded on the surface of panorama model, at this time
The textural characteristics being loaded on the surface of panorama model are the textural characteristics of RGB (rgb color mode) format.
In this application, panoramic apparatus has a FOV (Field of View, field angle), for example, 90 °, 110 ° or
120 ° etc., a loading area can be determined on the surface of panorama model according to field angle, then by the first of original image
Textural characteristics are loaded onto the loading area on the surface of panorama model.
Wherein, the resolution ratio of the loading area on the surface of panorama model is often below the resolution ratio of original image,
The reason is that: the process in the loading area on the surface that the first textural characteristics of original image are loaded onto panorama model
In, it generally requires to delete the element in the first textural characteristics, the textural characteristics after deleting then is loaded onto the load
On region, the resolution ratio so as to cause load image corresponding to the textural characteristics on the surface of panorama model is less than original graph
The resolution ratio of picture.
In step s 102, the second textural characteristics loaded on the surface of panorama model, the first textural characteristics packet are obtained
Include the second textural characteristics；
In the application one embodiment, available all first textures loaded on the surface of panorama model are special
Sign, and as the second textural characteristics.
Alternatively, in order to save system resource, can also be obtained according to actual needs in another embodiment of the application
The first textural characteristics of part loaded on the surface of panorama model, wherein the textural characteristics of every load Board Lot just need to consume
Take certain system resource, for example, memory or processor etc., compared to the first all text features is loaded, only loading section
First textural characteristics can reduce the quantity of the textural characteristics of load, so as to save system resource.
And as the second textural characteristics, embodiment shown in Fig. 3 specifically may refer to, it is not described here in detail.
The second textural characteristics obtained can be the texture of RGB (Red, red, Green, green, Blue, blue) format
Feature.
In step s 103, super-resolution enhancing processing is carried out to the second textural characteristics, obtains third texture feature, third
The resolution ratio of the corresponding image of textural characteristics is greater than the resolution ratio of the corresponding image of the second textural characteristics；
In this application, since the resolution ratio of load image corresponding to the textural characteristics on the surface of panorama model is small
In the resolution ratio of original image, therefore the resolution ratio of image corresponding to the second textural characteristics obtained is less than point of original image
Resolution,
If directly rendering the second textural characteristics on the display interface of panoramic apparatus, image shown by panoramic apparatus
Resolution ratio be lower than original image resolution ratio, image display effect is reduced, it is, what user saw in the display interface
The resolution ratio of image is lower than the resolution ratio of original image, and the clarity that will lead to image in this way is lower.
Therefore, it in order to improve the clarity of image, needs to carry out super-resolution enhancing processing to the second textural characteristics, obtain
Third texture feature, wherein the resolution ratio of the corresponding image of third texture feature is greater than the corresponding image of the second textural characteristics
Resolution ratio.
Specifically, available multiple sample graph image sets, each sample image are concentrated including multiple same scenes in advance
Sample image, and in the sample image of multiple same scenes include at least one high-resolution sample image and at least one
The sample image of low resolution, the sample image concentrated using multiple sample images instruct preset neural network model
Practice, until the weight in preset neural network model restrains, obtains super-resolution enhancing model.
Wherein, the scene that different sample images is concentrated can be different.
Low resolution and high-resolution in the application can not limit the numerical value of specific resolution ratio, but by sample graph
The resolution ratio of sample image in image set distinguishes.
Certainly, the difference between the high-resolution and low resolution of the image that sample image is concentrated is bigger, then super-resolution
Enhancing the image that the image enhancement of low resolution can be high resolution ratio by model, increase effect is more significant, therefore, training
The resolution ratio enhancing performance of super-resolution enhancing model out is stronger, the high-resolution and low resolution of the image that sample image is concentrated
Difference between rate is smaller, then the resolution ratio enhancing performance of the super-resolution enhancing model trained is weaker.
For example, for any one sample set, using preset neural network model to the low resolution in the sample set
Sample image carry out super-resolution enhancing processing, sample image after handle, then determine treated sample image and
The difference between high-resolution sample image in the sample set, and according in the preset neural network model of the discrepancy adjustment
Weight, sample set can be used carry out more wheel above process other each sample sets are equally executed above-mentioned
Process, until the weight in preset neural network model restrains, using finally obtained neural network model as super-resolution
Rate enhances model.
In this way, in this step, the second textural characteristics can be input in super-resolution enhancing model, obtain super-resolution
Rate enhances the textural characteristics of model output, and as third texture feature.
In one example, zoomwithfilter function and blend function are included at least in super-resolution enhancing model,
In this way, zoomwithfilter function and blend function at least can be used successively to RGB in super-resolution enhancing model
Second textural characteristics of format are handled, and obtain the third texture feature of rgb format.
In step S104, third texture feature is rendered on the display interface of panoramic apparatus.
In this application, the first textural characteristics of original image can be loaded on the surface of panorama model, is then obtained
The second textural characteristics loaded on the surface of panorama model are taken, the first textural characteristics include the second textural characteristics, then to second
Textural characteristics carry out super-resolution enhancing processing, obtain third texture feature, render on the display interface of panoramic apparatus later
Third texture feature.
It in the prior art, is that the second textural characteristics are directly rendered on the display interface of panoramic apparatus.
And in this application, due to carrying out super-resolution enhancing processing to the second textural characteristics, so that obtain the
The resolution ratio of the corresponding image of three textural characteristics is greater than the resolution ratio of the corresponding image of the second textural characteristics, third texture feature pair
The high resolution for the image answered is in the corresponding image of the second textural characteristics in the prior art, it is thus possible to improve image is clear
Clear degree.
For example, it is assumed that original image is the resolution ratio of 1080*720P, the resolution ratio of all surfaces of panorama model is also
The textural characteristics of original image can only be mapped to panorama mould if the field angle of headset equipment is 90 ° by 1080*720P
On 1/4 surface of type, the resolution ratio on 1/4 surface of panorama model is only 640*360P, therefore, is mapped to panorama model
The resolution ratio of the corresponding image of textural characteristics on 1/4 surface is 640*360P, finally to render in the display interface
The resolution ratio of image out is only 640*360P.
And in this application, super-resolution enhancing processing is carried out to the textural characteristics on the surface for being mapped to panorama model,
The high resolution of treated the corresponding image of textural characteristics is possibly even higher than 1080*720P in 640*360P, therefore,
User in compared with the prior art is only capable of the image for watching resolution ratio to be 640*360P, and the application can enable a user to
High resolution is watched to be above the image of 1080*720P in the image or even resolution ratio of 640*360P, so as to improve figure
The clarity of picture.
In this application, the eyeglass of panoramic apparatus is often convex lens, and the image presented by convex lens often distorts
Image, although user sees distortion in this way, improving the resolution ratio for the image that user watches by previous embodiment
Image still suffer from bad viewing experience.Therefore, in order to avoid such case generation, in another embodiment of the application,
Referring to fig. 2, step S104 includes:
In step s 201, the anti-distortion parameter of panoramic apparatus is obtained；
In this application, different convex lenses is different the distortion effects of picture, however, for any one panorama
For equipment, including convex lens panoramic apparatus dispatch from the factory when secured, the manufacturer of panoramic apparatus is in panoramic apparatus
Before factory, the anti-distortion parameter of the convex lens in panoramic apparatus can be determined, and the anti-distortion parameter is stored in panorama and is set
In standby.In this way, panoramic apparatus can be directly in the anti-distortion parameter for locally obtaining storage.
In step S202, anti-distortion processing is carried out to third texture feature based on anti-distortion parameter, obtains the 4th texture
Feature；
It, can be right based on anti-distortion parameter using any one anti-distortion processing method in the prior art in the application
Third texture feature carries out anti-distortion processing, obtains the 4th textural characteristics, and the application does not do specific anti-distortion processing method
It limits.
In step S203, the 4th textural characteristics are rendered on the display interface of panoramic apparatus.
In another embodiment of the application, the second textural characteristics loaded on the surface of panorama model are being obtained, in order to
System resource is saved, the first textural characteristics of part loaded on the surface of panorama model can also be obtained according to actual needs,
Referring to Fig. 3, detailed process includes:
In step S301, the current visual angle of panoramic apparatus is determined；
In this application, gyroscope is provided in panoramic apparatus, gyroscope can detecte the rotational angle etc. of panoramic apparatus
Parameter, in this way, the parameter of the gyroscope in available panoramic apparatus, then determines panoramic apparatus according to the parameter of gyroscope
Current visual angle, current visual angle are the direction of the camera lens of panoramic apparatus.
In step s 302, view field of the current visual angle on the surface of panorama model is determined；
In this application, panorama model can be sphere, and panoramic apparatus can be located at the center of sphere, panoramic apparatus and complete
The distance of scape model surface can be determined, that is, being the radius of sphere, secondly, the field angle of panoramic apparatus is also solid
It is fixed constant, or perhaps be determining after panoramic apparatus factory, and be stored in panoramic apparatus, therefore, panorama is set
The standby field angle that stored panoramic apparatus can be directly obtained from local, further according to field angle, current visual angle and sphere
Radius is assured that out view field of the current visual angle on the surface of panorama model.
It is of course also possible to view field of the current visual angle on the surface of panorama model is determined by other methods, this
Apply to specific method without limitation.
In step S303, on the surface of panorama model, the textural characteristics being located in the view field, and conduct are obtained
Second textural characteristics.
In this application, before when the first textural characteristics of original image are loaded on the surface of panorama model, the
Each of one textural characteristics element all has respective positions on the surface of panorama model, in the first textural characteristics
Any one element, panoramic apparatus can determine the element the location of on the surface of panorama model, then by the position
In the corresponding relationship set and form corresponding table item with the element, and be stored in position between the element that loads in position, for
Other each elements in first textural characteristics, equally execution aforesaid operations.
In this way, in this step, each position in the view field can be determined, then in the view field
Any one position in the corresponding relationship between element loaded in position and in position, is searched corresponding with the position
Element, for other each positions in the view field, equally execution aforesaid operations, in this way, multiple members can be found
Element generates a textural characteristics, and conduct according to its relationship between the position in the view field according to by each element
Second textural characteristics.
It should be noted that for simple description, therefore, it is stated as a series of action groups for embodiment of the method
It closes, but those skilled in the art should understand that, the embodiment of the present application is not limited by the described action sequence, because according to
According to the embodiment of the present application, some steps may be performed in other sequences or simultaneously.Secondly, those skilled in the art also should
Know, the embodiments described in the specification are all preferred embodiments, and related movement not necessarily the application is implemented
Necessary to example.
Referring to Fig. 4, a kind of structural block diagram of image processing apparatus embodiment of the application is shown, which specifically can wrap
Include following module:
Loading module 11 is loaded onto the surface of panorama model for the first textural characteristics by original image；
Module 12 is obtained, for obtaining the second textural characteristics loaded on the surface of the panorama model, described first
Textural characteristics include second textural characteristics；
Enhance processing module 13, for carrying out super-resolution enhancing processing to second textural characteristics, obtains third line
Feature is managed, the resolution ratio of the corresponding image of the third texture feature is greater than the resolution of the corresponding image of second textural characteristics
Rate；
Rendering module 14, for rendering the third texture feature on the display interface of panoramic apparatus.
In an optional implementation, the acquisition module 12 includes:
First determination unit, for determining the current visual angle of the panoramic apparatus；
Second determination unit, for determining view field of the current visual angle on the surface of the panorama model；
First acquisition unit, for obtaining the texture being located in the view field on the surface of the panorama model
Feature, and as second textural characteristics.
In an optional implementation, first determination unit includes:
Subelement is obtained, for obtaining the parameter of the gyroscope in the panoramic apparatus；
Subelement is determined, for determining the current visual angle of the panoramic apparatus according to the parameter of the gyroscope.
In an optional implementation, the enhancing processing module 13 includes:
Input unit obtains the oversubscription for second textural characteristics to be input in super-resolution enhancing model
Resolution enhances the textural characteristics of model output, and as third texture feature.
In an optional implementation, the enhancing processing module 13 further include:
Second acquisition unit, for obtaining multiple sample graph image sets, it includes multiple identical fields that each sample image, which is concentrated,
The sample image of scape, and include at least one high-resolution sample image and at least one in the sample image of multiple same scenes
The sample image of a low resolution；
Training unit, sample image for being concentrated using the multiple sample image to preset neural network model into
Row training obtains the super-resolution enhancing model until the weight in the preset neural network model restrains.
In an optional implementation, the rendering module 14 includes:
Third acquiring unit, for obtaining the anti-distortion parameter of the panoramic apparatus；
Anti- distortion processing unit, for being carried out at anti-distortion based on the anti-distortion parameter to the third texture feature
Reason, obtains the 4th textural characteristics；
Rendering unit, for rendering the 4th textural characteristics on the display interface of panoramic apparatus.
In this application, the first textural characteristics of original image can be loaded on the surface of panorama model, is then obtained
The second textural characteristics loaded on the surface of panorama model are taken, the first textural characteristics include the second textural characteristics, then to second
Textural characteristics carry out super-resolution enhancing processing, obtain third texture feature, render on the display interface of panoramic apparatus later
Third texture feature.
It in the prior art, is that the second textural characteristics are directly rendered on the display interface of panoramic apparatus.
And in this application, due to carrying out super-resolution enhancing processing to the second textural characteristics, so that obtain the
The resolution ratio of the corresponding image of three textural characteristics is greater than the resolution ratio of the corresponding image of the second textural characteristics, third texture feature pair
The high resolution for the image answered is in the corresponding image of the second textural characteristics in the prior art, it is thus possible to improve image is clear
Clear degree.
For example, it is assumed that original image is the resolution ratio of 1080*720P, the resolution ratio of all surfaces of panorama model is also
The textural characteristics of original image can only be mapped to panorama mould if the field angle of headset equipment is 90 ° by 1080*720P
On 1/4 surface of type, the resolution ratio on 1/4 surface of panorama model is only 640*360P, therefore, is mapped to panorama model
The resolution ratio of the corresponding image of textural characteristics on 1/4 surface is 640*360P, finally to render in the display interface
The resolution ratio of image out is only 640*360P.
And in this application, super-resolution enhancing processing is carried out to the textural characteristics on the surface for being mapped to panorama model,
The high resolution of treated the corresponding image of textural characteristics is possibly even higher than 1080*720P in 640*360P, therefore,
User in compared with the prior art is only capable of the image for watching resolution ratio to be 640*360P, and the application can enable a user to
High resolution is watched to be above the image of 1080*720P in the image or even resolution ratio of 640*360P, so as to improve figure
The clarity of picture.
For device embodiment, since it is basically similar to the method embodiment, related so being described relatively simple
Place illustrates referring to the part of embodiment of the method.
Optionally, the application also provides a kind of mobile terminal, comprising: memory, processor and is stored in the memory
Computer program that is upper and can running on the processor, the computer program is realized above-mentioned when being executed by the processor
Each process of image processing method embodiment, and identical technical effect can be reached, to avoid repeating, which is not described herein again.
Optionally, the application also provides a kind of computer readable storage medium, deposits on the computer readable storage medium
Computer program is contained, the step of computer program realizes image processing method as described above when being executed by processor.
Wherein, the computer readable storage medium, such as read-only memory (Read-Only Memory, abbreviation ROM), arbitrary access
Memory (Random Access Memory, abbreviation RAM), magnetic or disk etc..
All the embodiments in this specification are described in a progressive manner, the highlights of each of the examples are with
The difference of other embodiments, the same or similar parts between the embodiments can be referred to each other.
It should be understood by those skilled in the art that, the embodiments of the present application may be provided as method, apparatus or calculating
Machine program product.Therefore, the embodiment of the present application can be used complete hardware embodiment, complete software embodiment or combine software and
The form of the embodiment of hardware aspect.Moreover, the embodiment of the present application can be used one or more wherein include computer can
With in the computer-usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) of program code
The form of the computer program product of implementation.
The embodiment of the present application is referring to according to the method for the embodiment of the present application, terminal device (system) and computer program
The flowchart and/or the block diagram of product describes.It should be understood that flowchart and/or the block diagram can be realized by computer program instructions
In each flow and/or block and flowchart and/or the block diagram in process and/or box combination.It can provide these
Computer program instructions are set to general purpose computer, special purpose computer, Embedded Processor or other programmable data processing terminals
Standby processor is to generate a machine, so that being held by the processor of computer or other programmable data processing terminal devices
Capable instruction generates for realizing in one or more flows of the flowchart and/or one or more blocks of the block diagram
The device of specified function.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing terminal devices
In computer-readable memory operate in a specific manner, so that instruction stored in the computer readable memory generates packet
The manufacture of command device is included, which realizes in one side of one or more flows of the flowchart and/or block diagram
The function of being specified in frame or multiple boxes.
These computer program instructions can also be loaded into computer or other programmable data processing terminal devices, so that
Series of operation steps are executed on computer or other programmable terminal equipments to generate computer implemented processing, thus
The instruction executed on computer or other programmable terminal equipments is provided for realizing in one or more flows of the flowchart
And/or in one or more blocks of the block diagram specify function the step of.
Although preferred embodiments of the embodiments of the present application have been described, once a person skilled in the art knows bases
This creative concept, then additional changes and modifications can be made to these embodiments.So the following claims are intended to be interpreted as
Including preferred embodiment and all change and modification within the scope of the embodiments of the present application.
Finally, it is to be noted that, herein, relational terms such as first and second and the like be used merely to by
One entity or operation are distinguished with another entity or operation, without necessarily requiring or implying these entities or operation
Between there are any actual relationship or orders.Moreover, the terms "include", "comprise" or its any other variant meaning
Covering non-exclusive inclusion, so that process, method, article or terminal device including a series of elements not only wrap
Those elements are included, but also including other elements that are not explicitly listed, or further includes for this process, method, article
Or the element that terminal device is intrinsic.In the absence of more restrictions, being wanted by what sentence "including a ..." limited
Element, it is not excluded that there is also other identical elements in process, method, article or the terminal device for including the element.
Above to a kind of image processing method, device, terminal device and storage medium provided herein, carry out in detail
Thin to introduce, specific examples are used herein to illustrate the principle and implementation manner of the present application, and above embodiments are said
It is bright to be merely used to help understand the present processes and its core concept；At the same time, for those skilled in the art, foundation
The thought of the application, there will be changes in the specific implementation manner and application range, in conclusion the content of the present specification is not
It is interpreted as the limitation to the application.
Claims (14)
1. a kind of image processing method, which is characterized in that the described method includes:
First textural characteristics of original image are loaded on the surface of panorama model；
The second textural characteristics loaded on the surface of the panorama model are obtained, first textural characteristics include described second
Textural characteristics；
Super-resolution enhancing processing is carried out to second textural characteristics, obtains third texture feature, the third texture feature
The resolution ratio of corresponding image is greater than the resolution ratio of the corresponding image of second textural characteristics；
The third texture feature is rendered on the display interface of panoramic apparatus.
2. the method according to claim 1, wherein what the acquisition loaded on the surface of the panorama model
Second textural characteristics, comprising:
Determine the current visual angle of the panoramic apparatus；
Determine view field of the current visual angle on the surface of the panorama model；
On the surface of the panorama model, the textural characteristics being located in the view field are obtained, and as second line
Manage feature.
3. according to the method described in claim 2, it is characterized in that, the current visual angle of the determination panoramic apparatus, comprising:
Obtain the parameter of the gyroscope in the panoramic apparatus；
The current visual angle of the panoramic apparatus is determined according to the parameter of the gyroscope.
4. the method according to claim 1, wherein described carry out super-resolution increasing to second textural characteristics
Strength reason, obtains third texture feature, comprising:
Second textural characteristics are input in super-resolution enhancing model, the super-resolution enhancing model output is obtained
Textural characteristics, and as third texture feature.
5. according to the method described in claim 4, it is characterized in that, the method also includes:
Multiple sample graph image sets are obtained, each sample image concentrates the sample image including multiple same scenes, and multiple phases
With the sample image in the sample image of scene including at least one high-resolution sample image and at least one low resolution；
The sample image concentrated using the multiple sample image is trained preset neural network model, until described pre-
If neural network model in weight restrain, obtain super-resolution enhancing model.
6. the method according to claim 1, wherein described render described on the display interface of panoramic apparatus
Three textural characteristics, comprising:
Obtain the anti-distortion parameter of the panoramic apparatus；
Anti- distortion processing is carried out to the third texture feature based on the anti-distortion parameter, obtains the 4th textural characteristics；
The 4th textural characteristics are rendered on the display interface of panoramic apparatus.
7. a kind of image processing apparatus, which is characterized in that described device includes:
Loading module is loaded onto the surface of panorama model for the first textural characteristics by original image；
Module is obtained, for obtaining the second textural characteristics loaded on the surface of the panorama model, first texture is special
Sign includes second textural characteristics；
Enhance processing module, for obtaining third texture feature to second textural characteristics progress super-resolution enhancing processing,
The resolution ratio of the corresponding image of the third texture feature is greater than the resolution ratio of the corresponding image of second textural characteristics；
Rendering module, for rendering the third texture feature on the display interface of panoramic apparatus.
8. device according to claim 7, which is characterized in that the acquisition module includes:
First determination unit, for determining the current visual angle of the panoramic apparatus；
Second determination unit, for determining view field of the current visual angle on the surface of the panorama model；
First acquisition unit, for obtaining the textural characteristics being located in the view field on the surface of the panorama model,
And as second textural characteristics.
9. device according to claim 8, which is characterized in that first determination unit includes:
Subelement is obtained, for obtaining the parameter of the gyroscope in the panoramic apparatus；
Subelement is determined, for determining the current visual angle of the panoramic apparatus according to the parameter of the gyroscope.
10. device according to claim 7, which is characterized in that the enhancing processing module includes:
Input unit obtains the super-resolution for second textural characteristics to be input in super-resolution enhancing model
Enhance the textural characteristics of model output, and as third texture feature.
11. device according to claim 10, which is characterized in that the enhancing processing module further include:
Second acquisition unit, for obtaining multiple sample graph image sets, it includes multiple same scenes that each sample image, which is concentrated,
Sample image, and include that at least one high-resolution sample image is low at least one in the sample image of multiple same scenes
The sample image of resolution ratio；
Training unit, the sample image for being concentrated using the multiple sample image instruct preset neural network model
Practice, until the weight in the preset neural network model restrains, obtains the super-resolution enhancing model.
12. device according to claim 7, which is characterized in that the rendering module includes:
Third acquiring unit, for obtaining the anti-distortion parameter of the panoramic apparatus；
Anti- distortion processing unit is obtained for carrying out anti-distortion processing to the third texture feature based on the anti-distortion parameter
To the 4th textural characteristics；
Rendering unit, for rendering the 4th textural characteristics on the display interface of panoramic apparatus.
13. a kind of terminal device characterized by comprising memory, processor and be stored on the memory and can be in institute
The computer program run on processor is stated, such as claim 1 to 6 is realized when the computer program is executed by the processor
Any one of described in image processing method.
14. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium
Program realizes such as image processing method described in any one of claims 1 to 6 when the computer program is executed by processor
Method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910407795.0A CN110223228A (en) | 2019-05-16 | 2019-05-16 | Image processing method, device, terminal device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910407795.0A CN110223228A (en) | 2019-05-16 | 2019-05-16 | Image processing method, device, terminal device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110223228A true CN110223228A (en) | 2019-09-10 |
Family
ID=67821215
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910407795.0A Pending CN110223228A (en) | 2019-05-16 | 2019-05-16 | Image processing method, device, terminal device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110223228A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110599564A (en) * | 2019-09-19 | 2019-12-20 | 浙江大搜车软件技术有限公司 | Image display method and device, computer equipment and storage medium |
| WO2021052040A1 (en) * | 2019-09-20 | 2021-03-25 | 华为技术有限公司 | Video image enhancement method, device, apparatus, chip, and storage medium |
| CN112905293A (en) * | 2021-03-26 | 2021-06-04 | 北京房江湖科技有限公司 | Graph loading method and system and graph rendering method and system |
| WO2024001923A1 (en) * | 2022-06-30 | 2024-01-04 | 先临三维科技股份有限公司 | Mapping method and apparatus, device, and storage medium |
| WO2024120031A1 (en) * | 2022-12-09 | 2024-06-13 | 腾讯科技（深圳）有限公司 | Video data processing method and apparatus, computer device, and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030068098A1 (en) * | 2001-09-27 | 2003-04-10 | Michael Rondinelli | System and method for panoramic imaging |
| US20030095338A1 (en) * | 2001-10-29 | 2003-05-22 | Sanjiv Singh | System and method for panoramic imaging |
| CN109461213A (en) * | 2018-11-16 | 2019-03-12 | 京东方科技集团股份有限公司 | Image processing method, device, equipment and storage medium based on virtual reality |
| US20190096035A1 (en) * | 2017-09-28 | 2019-03-28 | Zhengmin Li | Super-resolution apparatus and method for virtual and mixed reality |
-
2019
- 2019-05-16 CN CN201910407795.0A patent/CN110223228A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030068098A1 (en) * | 2001-09-27 | 2003-04-10 | Michael Rondinelli | System and method for panoramic imaging |
| US20030095338A1 (en) * | 2001-10-29 | 2003-05-22 | Sanjiv Singh | System and method for panoramic imaging |
| US20190096035A1 (en) * | 2017-09-28 | 2019-03-28 | Zhengmin Li | Super-resolution apparatus and method for virtual and mixed reality |
| CN109461213A (en) * | 2018-11-16 | 2019-03-12 | 京东方科技集团股份有限公司 | Image processing method, device, equipment and storage medium based on virtual reality |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110599564A (en) * | 2019-09-19 | 2019-12-20 | 浙江大搜车软件技术有限公司 | Image display method and device, computer equipment and storage medium |
| WO2021052040A1 (en) * | 2019-09-20 | 2021-03-25 | 华为技术有限公司 | Video image enhancement method, device, apparatus, chip, and storage medium |
| CN112905293A (en) * | 2021-03-26 | 2021-06-04 | 北京房江湖科技有限公司 | Graph loading method and system and graph rendering method and system |
| CN112905293B (en) * | 2021-03-26 | 2023-07-07 | 贝壳找房(北京)科技有限公司 | Graphics loading method and system and graphics rendering method and system |
| WO2024001923A1 (en) * | 2022-06-30 | 2024-01-04 | 先临三维科技股份有限公司 | Mapping method and apparatus, device, and storage medium |
| WO2024120031A1 (en) * | 2022-12-09 | 2024-06-13 | 腾讯科技（深圳）有限公司 | Video data processing method and apparatus, computer device, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110223228A (en) | Image processing method, device, terminal device and storage medium | |
| CN112989904B (en) | Method for generating style image, method, device, equipment and medium for training model | |
| CN111899192B (en) | Interaction method, interaction device, electronic equipment and computer-readable storage medium | |
| CN111243049B (en) | Face image processing method and device, readable medium and electronic equipment | |
| CN108665415B (en) | Image quality improving method and device based on deep learning | |
| CN113989173A (en) | Video fusion method and device, electronic equipment and storage medium | |
| CN109102484B (en) | Method and apparatus for processing image | |
| US20130250060A1 (en) | Depth Buffer Compression for Stochastic Motion Blur Rasterization | |
| US12106493B2 (en) | Accurate optical flow estimation in stereo pairs of equirectangular images | |
| WO2024016923A1 (en) | Method and apparatus for generating special effect graph, and device and storage medium | |
| Yang et al. | Dynamic 3D scene depth reconstruction via optical flow field rectification | |
| CN114630057B (en) | Method and device for determining special effect video, electronic equipment and storage medium | |
| CN111818265B (en) | Interaction method and device based on augmented reality model, electronic equipment and medium | |
| CN114339447B (en) | Method, device and equipment for converting picture into video and storage medium | |
| CN111669476B (en) | Watermark processing method, device, electronic equipment and medium | |
| CN112802206A (en) | Roaming view generation method, device, equipment and storage medium | |
| US20220207848A1 (en) | Method and apparatus for generating three dimensional images | |
| CN110636331B (en) | Method and apparatus for processing video | |
| CN111833459A (en) | Image processing method and device, electronic equipment and storage medium | |
| CN107491934B (en) | 3D interview system based on virtual reality | |
| CN111727455A (en) | Enhancing image data with appearance controls | |
| CN116363239A (en) | Method, device, equipment and storage medium for generating special effect diagram | |
| CN110941413B (en) | Display screen generation method and related device | |
| CN114170341A (en) | Image processing method, device, equipment and medium | |
| CN116527993A (en) | Video processing method, apparatus, electronic device, storage medium and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190910 |