CN112153465B - Image loading method and device - Google Patents

Image loading method and device Download PDF

Info

Publication number
CN112153465B
CN112153465B CN201910578189.5A CN201910578189A CN112153465B CN 112153465 B CN112153465 B CN 112153465B CN 201910578189 A CN201910578189 A CN 201910578189A CN 112153465 B CN112153465 B CN 112153465B
Authority
CN
China
Prior art keywords
image
resolution
app
super
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910578189.5A
Other languages
Chinese (zh)
Other versions
CN112153465A (en
Inventor
胡泊
陆凯
王孝满
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Century Trading Co Ltd
Beijing Jingdong Shangke Information Technology Co Ltd
Original Assignee
Beijing Jingdong Century Trading Co Ltd
Beijing Jingdong Shangke Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Century Trading Co Ltd, Beijing Jingdong Shangke Information Technology Co Ltd filed Critical Beijing Jingdong Century Trading Co Ltd
Priority to CN201910578189.5A priority Critical patent/CN112153465B/en
Publication of CN112153465A publication Critical patent/CN112153465A/en
Application granted granted Critical
Publication of CN112153465B publication Critical patent/CN112153465B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440263Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234363Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41Structure of client; Structure of client peripherals
    • H04N21/414Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
    • H04N21/41407Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop

Abstract

The embodiment of the invention provides an image loading method and device, when a user uses an APP through a data network, when an electronic device detects that the network condition meets a preset condition, an image acquisition request for requesting low resolution is sent to a server, so that the server sends the low resolution image to the electronic device, the electronic device converts the low resolution image into a high resolution image by utilizing a locally deployed super resolution model and loads the high resolution image through the APP, and for the user, the low resolution image is smaller, so that the data flow of the user can be saved. Meanwhile, under the condition of bad network conditions, the loading speed of the image can be accelerated. In addition, for the server, issuing a low resolution image can save a lot of server bandwidth and data space, thereby saving costs.

Description

Image loading method and device
Technical Field
The embodiment of the invention relates to the technical field of image processing, in particular to an image loading method and device.
Background
Along with the high-speed expansion of mobile internet technology, electronic devices such as mobile phones and the like gradually become indispensable tools in life of people, various application programs (APP) are run on the tools, different APP bear different functions, and users perform social, shopping, learning or entertainment by using various APP.
In the process that the user uses the APP, the electronic equipment acquires characters, images and the like from the server, and the acquired content is presented to the user through a screen of the electronic equipment. Typically, the APP process is essentially a process in which the electronic device continuously acquires and loads images from a server, since the images are more intuitive and more visual than text. In this process, if the user uses a data network, the data traffic of the user is severely consumed when the resolution of the image is high. To avoid this drawback, some images commonly used for accessing the APP are usually built in an installation package of the APP, and the user downloads and follows the installation package, thereby saving the commonly used images locally of the electronic device. Thereafter, the APP runs, loading the common images locally.
In the process of loading the image, the image needs to be built in an installation package of the APP. However, to avoid swelling of the APP's installation package, only a limited number of images can be built into the installation package. If the APP is running, it is necessary to access other images, and then the terminal device is required to acquire the images from the server and load the images. At this time, if the user uses the data traffic, the data traffic of the user is also severely consumed.
Disclosure of Invention
The embodiment of the invention provides an image loading method and device, electronic equipment acquires a low-resolution image from a server, converts the low-resolution image into a high-resolution image, loads and displays the high-resolution image, and achieves the aim of reducing the consumption data flow.
In a first aspect, an embodiment of the present invention provides an image loading method, including:
detecting the current network condition of electronic equipment, and running an application program APP which is required to load high-definition images currently on the electronic equipment;
if the network condition meets a preset condition, sending an image acquisition request to a server, wherein the image acquisition request is used for acquiring an image with resolution lower than a first threshold;
receiving an image with resolution lower than a first threshold sent by the server;
and converting the image into a high-definition image by using a super-resolution model and loading the high-definition image through the APP.
In one possible design, before the detecting the current network condition of the electronic device, the method further includes:
sending a downloading request to the server, wherein the downloading request is used for requesting to download the APP;
and receiving an installation package sent by the server, wherein the installation package carries the super-resolution model.
In one possible design, before sending the image acquisition request to the server if the network condition meets a preset condition, the method further includes:
if the network condition meets a preset condition, detecting whether the electronic equipment caches the image with the resolution lower than a first threshold;
and if the electronic equipment caches the image with the resolution lower than the first threshold value, converting the image into a high-definition image by using a super-resolution model and loading the high-definition image through the APP.
In a possible design, the super-resolution model includes any one of the following models: the super-resolution generation is performed on an antagonism network model, a random forest model, an interpolation model and a sparse model.
In a possible design, the converting the image into a high definition image using a super resolution model and loading the image through the APP includes:
memory is allocated for the super-resolution model;
operating the super-resolution model on the memory, and inputting the image with the resolution lower than a first threshold value into the super-resolution model to obtain the high-definition image;
and loading the high-definition image through the APP.
In a second aspect, an embodiment of the present invention provides an image loading method, including:
Receiving an image acquisition request sent by electronic equipment when detecting that the current network condition meets a preset condition, running an application program APP which is currently required to load high-definition images on the electronic equipment, wherein the image acquisition request is used for acquiring images with resolution lower than a first threshold;
acquiring an image with resolution lower than a first threshold according to the image acquisition request;
and sending the image with the resolution lower than a first threshold to the electronic equipment.
In one possible design, before detecting that the current network condition meets the preset condition, the receiving electronic device further includes:
receiving a downloading request sent by the electronic equipment, wherein the downloading request is used for requesting to download the APP;
and sending an installation package of the APP to the electronic equipment, wherein the installation package carries the super-resolution model.
In a possible design, the super-resolution model includes any one of the following models: the super-resolution generation is performed on an antagonism network model, a random forest model, an interpolation model and a sparse model.
In a third aspect, an embodiment of the present invention provides an image loading apparatus, including:
the processing unit is used for detecting the current network condition of the electronic equipment, and an application program APP needing to be loaded with the high-definition image currently runs on the electronic equipment;
The sending unit is used for sending an image acquisition request to a server if the network condition of the water treatment unit meets a preset condition, wherein the image acquisition request is used for acquiring an image with resolution lower than a first threshold;
a receiving unit, configured to receive an image with a resolution lower than a first threshold sent by the server;
the processing unit is also used for converting the image into a high-definition image by utilizing a super-resolution model and loading the high-definition image through the APP.
In a possible design, the sending unit is further configured to send a download request to the server before the processing unit detects the current network condition of the electronic device, where the download request is used to request downloading of the APP;
the receiving unit is further configured to receive an installation packet sent by the server, where the installation packet carries the super-resolution model.
In a feasible design, before the network condition meets a preset condition, the sending unit is further configured to detect whether the electronic device caches an image with a resolution lower than a first threshold before sending an image acquisition request to a server, and if the electronic device caches an image with a resolution lower than the first threshold, convert the image into a high-definition image by using a super-resolution model and load the image through the APP.
In a possible design, the super-resolution model includes any one of the following models: the super-resolution generation is performed on an antagonism network model, a random forest model, an interpolation model and a sparse model.
In a feasible design, the processing unit is configured to allocate a memory for the super-resolution model, input the image with the resolution lower than a first threshold to the super-resolution model, run the super-resolution model on the memory, input the image with the resolution lower than the first threshold to the super-resolution model, obtain the high-definition image, and load the high-definition image through the APP.
In a fourth aspect, an embodiment of the present invention provides an image loading method, including:
the receiving unit is used for receiving an image acquisition request sent by the electronic equipment when the current network condition is detected to meet the preset condition, an application program APP needing to load high-definition images currently runs on the electronic equipment, and the image acquisition request is used for acquiring images with resolution lower than a first threshold value;
a processing unit, configured to acquire an image with a resolution lower than a first threshold according to the image acquisition request;
and the sending unit is used for sending the image with the resolution lower than a first threshold value to the electronic equipment.
In a possible design, the receiving unit is further configured to receive a download request sent by the electronic device, where the download request is used for requesting to download the APP, before receiving an image acquisition request sent by the electronic device when detecting that a current network condition meets a preset condition;
the sending unit is further configured to send an installation package of the APP to the electronic device, where the installation package carries the super-resolution model.
In a possible design, the super-resolution model includes any one of the following models: the super-resolution generation is performed on an antagonism network model, a random forest model, an interpolation model and a sparse model.
In a fifth aspect, an embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored on the memory and executable on the processor, where the processor implements the method as described above in the first aspect or the various possible implementations of the first aspect when the program is executed by the processor.
In a sixth aspect, embodiments of the present invention provide a server comprising a processor, a memory and a computer program stored on the memory and executable on the processor, the processor implementing the method according to the above first aspect or the various possible implementations of the first aspect when executing the program.
In a seventh aspect, embodiments of the present invention provide a storage medium having stored therein instructions that, when executed on an electronic device, cause the electronic device to perform the method as described above in the first aspect or in the various possible implementations of the first aspect.
In an eighth aspect, embodiments of the present invention provide a storage medium having stored therein instructions that, when executed on a server, cause the server to perform the method as described above in the second aspect or in the various possible implementations of the second aspect.
In a ninth aspect, embodiments of the present invention provide a computer program product which, when run on an electronic device, causes the electronic device to perform the method as described above in the first aspect or in the various possible implementations of the first aspect.
In a tenth aspect, embodiments of the present invention provide a computer program product which, when run on a server, causes the server to perform the method as described above in the second aspect or in the various possible implementations of the second aspect.
According to the image loading method and device provided by the embodiment of the invention, when a user uses the APP through a data network, when the electronic equipment detects that the network condition meets the preset condition, an image acquisition request for requesting low resolution is sent to the server, so that the server sends the low resolution image to the electronic equipment, the electronic equipment converts the low resolution image into the high resolution image by utilizing a locally deployed super resolution model and loads the high resolution image through the APP, and for the user, the low resolution image is smaller, so that the data flow of the user can be saved. Meanwhile, under the condition of bad network conditions, the loading speed of the image can be accelerated. In addition, for the server, issuing a low resolution image can save a lot of server bandwidth and data space, thereby saving costs.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings that are needed in the embodiments or the description of the prior art will be briefly described below, it will be obvious that the drawings in the following description are some embodiments of the present invention, and that other drawings can be obtained according to these drawings without inventive effort to a person skilled in the art.
FIG. 1 is a schematic diagram of an operating environment of an image loading method according to an embodiment of the present invention;
FIG. 2 is a flowchart of an image loading method according to an embodiment of the present invention;
FIG. 3 is another flow chart of an image loading method provided by an embodiment of the present invention;
FIG. 4 is a flowchart of training a super-resolution model in an image loading method according to an embodiment of the present invention;
FIG. 5 is a flowchart of a super resolution process in an image loading method according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an image loading device according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of another image loading device according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of another image loading device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The user uses the APP process, which is essentially the process that the electronic device carrying the APP continuously acquires images from the server and loads them. In this process, if the user uses a data network, the data traffic of the user is severely consumed when the resolution of the image is high. To avoid this drawback, some images commonly used for accessing the APP are usually built in an installation package of the APP, and the user downloads and follows the installation package, thereby saving the commonly used images locally of the electronic device. Thereafter, the APP runs, loading the common images locally.
In the process of loading the image, the image needs to be built in an installation package of the APP. However, to avoid swelling of the APP's installation package, only a limited number of images can be built into the installation package. If the APP is running, it is necessary to access other images, and then the terminal device is required to acquire the images from the server and load the images. At this time, if the user uses the data traffic, the data traffic of the user is also severely consumed. Moreover, this approach is very dead, and when the APP common images change, it is necessary to develop an APP installation package that reissues the APP carrying the new image, and the user downloads the installation package and updates the locally stored common images. In addition, if the APP is running, it is necessary to access other images, and then the terminal device is required to acquire the images from the server and load the images. At this time, if the user uses the data network, the data traffic of the user is also severely consumed.
There are other ways to solve the problem that the image loading is slow or the loading fails when the network condition is not good. For example, depending on the WiFi environment, in this way, only if the electronic device is connected to WiFi, the APP is running, and the server sends a high-resolution image to the APP, so as to avoid consuming the data traffic of the user. However, this approach relies heavily on WiFi networks, limiting the user's environment of using the APP, and is very inflexible. For another example, the server compresses the image by webp and then sends the compressed image to the electronic device, and the size of the compressed image is about 2/3 of the original image. In another example, in the spam scheme, if the electronic device fails to load the image, a thumbnail of the image is obtained from the server and loaded, but the thumbnail is enlarged and has an unclear problem.
In view of this, an embodiment of the present invention provides an image loading method and apparatus, where an electronic device obtains a low-resolution image from a server, converts the low-resolution image into a high-resolution image, and loads and displays the high-resolution image, so as to achieve the purpose of reducing the data traffic consumption.
Fig. 1 is a schematic diagram of an operation environment of an image loading method according to an embodiment of the present invention. Referring to fig. 1, an electronic device establishes a network connection with a server, an APP is run on the electronic device, and the server is a server that provides a business service for the APP. The electronic device has a function of detecting a network condition, and when the current network condition is not good, the electronic device transmits an image acquisition request for acquiring an image with a resolution lower than a first threshold to the server to acquire an image with a low resolution. After receiving the image acquisition request, the server does not transmit the high-resolution image to the electronic device, but transmits the image with the resolution lower than the first threshold value to the electronic device. After the electronic equipment receives the image with the resolution lower than the first threshold value, the image with the resolution lower than the first threshold value is converted into a high-definition image by utilizing a super-resolution model deployed on the electronic equipment, namely, the image with the low resolution is converted into the image with the high resolution by utilizing a super-resolution technology. The electronic device may be, for example, a computer, a notebook, a mobile phone, etc. of the user, and the embodiment of the invention is not limited. The server may be, for example, a server to which the APP corresponds.
Next, on the basis of fig. 1, an image loading method according to an embodiment of the present invention will be described in detail. For example, please refer to fig. 2.
Fig. 2 is a flowchart of an image loading method according to an embodiment of the present invention. The embodiment describes the image loading method in detail from the interaction point of the server and the terminal device, and the embodiment includes:
101. detecting whether the current network condition of the electronic equipment meets a preset condition, running an application program APP which is required to load high-definition images currently on the electronic equipment, and executing step 102 if the network condition meets the preset condition; if the network condition does not meet the preset condition, step 106 is performed.
For example, when a user clicks an icon of an APP on a touch screen of an electronic device, and starts the APP, the APP needs to load a main page of the APP, some advertisements, and the like, and there is a need to load a high-definition image. At this time, the electronic device detects its own current network condition. For example, the preset condition is that the electronic device is connected to the server through the data network.
102. And if the network condition meets the preset condition, sending an image acquisition request to a server, wherein the image acquisition request is used for acquiring an image with the resolution lower than a first threshold value.
For example, if the electronic device detects that the network condition meets the preset condition, for example, the electronic device is connected to the server through the data network, the electronic device sends an image for acquiring the resolution lower than the first threshold to the server, so as to request the server to send the image with low resolution to the electronic device.
103. And acquiring an image with resolution lower than a first threshold according to the image acquisition request.
Illustratively, after receiving the image acquisition request, the server acquires an image with a resolution lower than the first threshold, i.e. a low resolution image, from a local or other place according to the image acquisition request, where the low resolution image has a corresponding relationship with the high resolution image required by the APP.
104. And sending the image with the resolution lower than a first threshold to the electronic equipment.
The server sends the low resolution image determined in step 103 to the electronic device through the data network, so that the data traffic of the user can be saved due to the small low resolution image.
105. And converting the image into a high-definition image by using a super-resolution model and loading the high-definition image through the APP.
The electronic device receives the low-resolution image, converts the low-resolution image into a high-definition image by using a locally deployed super-resolution model, that is, converts the low-resolution image into a high-resolution image by using a super-resolution technology, and then loads the high-resolution image through the APP and displays the high-resolution image to a user.
106. Requesting to acquire a high definition image from a server.
According to the image loading method provided by the embodiment, when a user uses the APP through a data network, when the electronic equipment detects that the network condition meets the preset condition, an image acquisition request for requesting low resolution is sent to the server, so that the server sends the low resolution image to the electronic equipment, the electronic equipment converts the low resolution image into a high resolution image by utilizing a locally deployed super resolution model, and the APP is used for loading, and for the user, the data flow of the user can be saved because the low resolution image is smaller. Meanwhile, under the condition of bad network conditions, the loading speed of the image can be accelerated. In addition, for the server, issuing a low resolution image can save a lot of server bandwidth and data space, thereby saving costs.
In the above embodiments, the network conditions include, but are not limited to, a network connection manner between the electronic device and the server. For example, the network condition refers to the degree of network connection, and at this time, a correspondence relation between the packet loss rate and the degree of network condition is preset, and when the packet loss rate is lower than a certain threshold value a, the network condition is considered to be good, and when the packet loss rate is higher than a threshold value b, the network condition is considered to be bad, and when the packet loss rate is between the threshold value a and the threshold value b, the network condition is considered to be good. As another example, the network condition refers to a network delay of the current network, the number of users accessing the server, and the like.
In the above embodiment, the super-resolution model is used to convert the low-resolution image into the high-resolution image by using the super-resolution technique, which is an inversion technique because the low-resolution image information is missing. In the embodiment of the invention, the super-resolution model comprises, but is not limited to, a super-resolution generation countermeasure network model, a random forest model, an interpolation model, a sparse model and the like. The interpolation model restores the low-resolution image into the high-resolution image by adding some priori information into the low-resolution image; the random forest model is trained based on a regression method; the sparse model is obtained based on sparse learning; the convolutional neural network (Convolutional Neural Networks, CNN) model has good effect in the Super-resolution technology, and the Super-resolution generation countermeasure network (Super-Resolution Generative Adversarial Network, SGRAN) model in the CNN model is obtained by performing Gaussian noise processing, motion blur adding and the like on the image.
In the embodiment of the application, the super-resolution model is pre-deployed inside the electronic device. Prior to deployment, a super-resolution model needs to be trained. Since training the super-resolution model by means of machine learning and the like requires strong performance and computational power, model training is performed by a server, which may be a server providing business services for the APP or other servers. After the server trains the super-resolution model, the super-resolution model is packed into an installation Package of the APP, such as in an Android installation Package (APK) of the APP, and the installation Package of the APP is uploaded to an APP application store, such as an APP store, a server providing services for the APP, and the like, so that the APP is downloaded by a user. Therefore, in the downloading process, the electronic device sends a downloading request to the server, wherein the downloading request is used for requesting to download the APP. After receiving the downloading request, the server sends the APP installation package to the electronic equipment; correspondingly, the electronic equipment receives an installation package sent by the server, wherein the installation package carries the super-resolution model. And then, the electronic equipment loads the installation package, and installs the APP and the resolution model at the same time. After the installation is successful, the super-resolution model is started at the same time of starting the APP each time.
In the following, a detailed description will be given of how to quickly load an image when an APP requiring a high-definition image is run by an electronic device. For example, referring to fig. 3, fig. 3 is another flowchart of an image loading method provided by an embodiment of the present invention, where the embodiment includes:
201. and the electronic equipment detects that the APP is required to load the high-definition image.
Illustratively, a trained super-resolution model is pre-deployed on the electronic device. When a user clicks an APP, the APP needs to load high-definition images, such as advertisement images issued by a server when the APP is started, images displayed on a front page of the APP, and the like.
202. The electronic device detects whether the electronic device caches the image with the resolution lower than the first threshold, and if yes, step 203 is executed; if there is no image with resolution lower than the first threshold in the cache of the electronic device, step 204 is executed;
for example, when the high-definition image needs to be loaded through the APP, the electronic device first determines whether a low-resolution image exists in the cache, and if so, executes step 203 to process the image; if there is no low resolution image, step 204 is executed to request the server to issue a low resolution image, where the low resolution image is an image with a resolution lower than the first threshold.
203. The electronic device reads the image in the buffer with a resolution below the first threshold.
204. The electronic device sends an image acquisition request to the server, the image acquisition request being for acquiring an image having a resolution below a first threshold.
205. The server acquires an image with a resolution lower than a first threshold value according to the image acquisition request.
206. The server sends the image with the resolution lower than a first threshold to the electronic device.
207. The electronic device converts the image into a high definition image using a super resolution model.
208. And the electronic equipment loads the high-definition image through the APP.
Illustratively, in steps 207-208, the electronic device processes the low resolution image using a super resolution model such as an SRGAN model, generates a high resolution image, and communicates the high resolution image to the APP for display by the APP's image display frame.
In the above example, because the server transmits the low resolution image to the electronic device, the user's traffic consumption can be saved when the user uses the data network. In addition, regardless of the data network or WiFi used by the user, the time for image loading can be shortened in case the network conditions are not good.
In the above embodiment, the super-resolution model is obtained by training the sample image by the server, and because the super-resolution model obtained by training the server is a model generated by curing, the model cannot be directly deployed to the electronic device, and the model can be deployed to the electronic device after being converted. In the following, taking the super-resolution model as an SRGAN model as an example, how the server trains to obtain the SRGAN model and converts the model will be described in detail. For example, see fig. 4.
Fig. 4 is a flowchart of training a super-resolution model in the image loading method according to the embodiment of the present invention, where the embodiment includes:
301. the server trains the SRGAN model.
Illustratively, the server trains the sample image by using TensorFlow and the like to obtain an SRGAN model. During the training process, graphic symbols (Graphdef files) and checkpoint (checkpoint) files are generated.
TensorFlow is an artificial intelligence framework of Google (Google) open source, and is also a mainstream network building and model training tool. The SRGAN model is a super-resolution model based on a convolutional neural network and is trained based on a generation type antagonism network (Generative Adversarial Networks, GAN) method, and comprises a generator and a discriminator, wherein the generator is also called a G network, the discriminator is also called a D network, and a main body of the discriminator uses a visual geometry group (Visual Geometry Group, VGG); the generator is a chain of residual block (residual block) connections. The generator generates a high resolution image from the low resolution image, and the arbiter determines whether the high resolution image was generated by the generator or the original image in the database. When the discriminator can discriminate the image generated by the generator as the original image in the data, namely the generator is successfully deceived the discriminator, the model training is considered to be finished.
In the model training process, a sub-pixel (sub-pixel) module is added into the SRGAN model, so that the generator can increase the resolution of the image only at the last network layer, and the calculation resource consumption is reduced while the resolution is improved.
302. And performing network curing operation on the image symbol file and the check point file to obtain a cured image symbol file.
Illustratively, the server performs network curing operation on the Graphdef file and the checkpoint generated in step 301 to obtain a cured Graphdef file, where the cured Graphdef file is also called a frozen image symbol file, i.e. a frozen Graphdef file.
303. And performing model conversion on the solidified image symbol file to obtain an SRGAN model capable of running on the electronic equipment.
The server performs model conversion, illustratively using binary tools provided by TensorFlow or by Python programming, and post-conversion reviews model files prefixed with tflite, which can be run directly on the electronic device. The model file with the suffix of tflite corresponds to a TensorFlow Life scheme, the TensorFlow Life scheme is a lightweight solution of TensorFlow, and is converted into an electronic device design, so that the model file can realize low-delay end-side machine learning reasoning by optimizing a kernel, presetting an activation function and the like.
304. And packaging the SRGAN model into an installation package of the APP, so that the SRGAN model is deployed into the electronic equipment after the electronic equipment downloads the installation package.
In the following, taking the super-resolution model as an example of the SRGAN model, how the super-resolution model performs super-resolution processing on the low-resolution picture in the above embodiment will be described in detail. For example, see fig. 5.
Fig. 5 is a flowchart of super resolution processing in an image loading method according to an embodiment of the present invention, where the embodiment includes:
401. the electronic device initializes the SRGAN model.
Illustratively, the process of converting a low resolution image to a high resolution image by an electronic device using the SRGAN model is accomplished by scheduling a TensorFlow Life.
402. The electronic device allocates memory for the SRGAN model.
Illustratively, memory needs to be allocated for the SRGAN model prior to running the SRGAN model.
403. And the electronic equipment runs the SRGAN model on the memory, and inputs the image with the resolution lower than a first threshold value into the super-resolution model to obtain the high-definition image.
404. And the electronic equipment loads the high-definition image through the APP.
Illustratively, in steps 403 and 404, the electronic device inputs the low resolution image in the cache or the low resolution image received from the server as an input of the model to the SRGAN model, so that the SRGAN model starts the reasoning process, i.e. the process of converting the low resolution image into a high resolution image. In the process, the SRGAN model can improve the resolution of the low-resolution image by a factor of more or less, and the SRGAN model supports a superdivision effect of 4 times at most depending on the capacity of the SRGAN model.
405. And the electronic equipment releases the memory.
Illustratively, after converting the low resolution image to a high resolution image, the electronic device releases memory allocated to the SRGAN model.
The following are examples of the apparatus of the present invention that may be used to perform the method embodiments of the present invention. For details not disclosed in the embodiments of the apparatus of the present invention, please refer to the embodiments of the method of the present invention.
Fig. 6 is a schematic structural diagram of an image loading device according to an embodiment of the present invention. The image loading device according to the embodiment may be an electronic device or a chip applied to the electronic device. The image loading apparatus may be used to perform the functions of the electronic device in the above-described embodiments. As shown in fig. 6, the image loading apparatus 100 may include:
the processing unit 11 is used for detecting the current network condition of the electronic equipment, and the electronic equipment runs an application program APP which is required to load the high-definition image currently;
a sending unit 12, configured to send an image acquisition request to a server if the water treatment unit 11 outputs that the network condition meets a preset condition, where the image acquisition request is used to acquire an image with a resolution lower than a first threshold;
a receiving unit 13, configured to receive an image with a resolution lower than a first threshold value sent by the server;
The processing unit 11 is further configured to convert the image into a high-definition image by using a super-resolution model and load the high-definition image through the APP.
In a possible design, the sending unit 12 is further configured to send a download request to the server, before the processing unit 11 detects the current network condition of the electronic device, where the download request is used to request downloading of the APP;
the receiving unit 13 is further configured to receive an installation packet sent by the server, where the installation packet carries the super-resolution model.
In a possible design, before the network condition meets a preset condition, the sending unit 12 is further configured to detect whether the electronic device caches an image with a resolution lower than a first threshold before sending an image acquisition request to a server, and if the electronic device caches an image with a resolution lower than the first threshold, convert the image into a high-definition image by using a super-resolution model and load the image through the APP.
In a possible design, the super-resolution model includes any one of the following models: the super-resolution generation is performed on an antagonism network model, a random forest model, an interpolation model and a sparse model.
In a feasible design, the processing unit 11 is configured to allocate a memory for the super-resolution model, run the super-resolution model on the memory, and input the image with the resolution lower than the first threshold to the super-resolution model to obtain the high-definition image, and load the high-definition image through the APP.
The image loading device provided in the embodiment of the present application may perform the actions of the electronic device in the above embodiment, and its implementation principle and technical effects are similar, and are not described herein again.
Fig. 7 is a schematic structural diagram of another image loading device according to an embodiment of the present invention. The image loading device according to the present embodiment may be a server or a chip applied to the server. The image loading device may be used to perform the functions of the server in the above-described embodiments. As shown in fig. 7, the image loading apparatus 200 may include:
a receiving unit 21, configured to receive an image acquisition request sent by an electronic device when detecting that a current network condition meets a preset condition, where the electronic device runs an application APP that needs to load a high-definition image currently, where the image acquisition request is used to acquire an image with a resolution lower than a first threshold;
A processing unit 22 for acquiring an image with a resolution lower than a first threshold value according to the image acquisition request;
a transmitting unit 23, configured to transmit the image with the resolution lower than the first threshold to the electronic device.
In a possible design, the receiving unit 21 is further configured to receive a download request sent by the electronic device, where the download request is used for requesting to download the APP, before receiving an image acquisition request sent by the electronic device when detecting that the current network condition meets a preset condition;
the sending unit 23 is further configured to send an installation package of the APP to the electronic device, where the installation package carries the super-resolution model.
In a possible design, the super-resolution model includes any one of the following models: the super-resolution generation is performed on an antagonism network model, a random forest model, an interpolation model and a sparse model.
The login verification device provided by the embodiment of the invention can execute the action of the server in the embodiment, and the implementation principle and the technical effect are similar, and are not repeated here.
It should be understood that the above receiving unit may be actually implemented as a receiver, and the transmitting unit may be actually implemented as a transmitter. And the processing unit can be realized in the form of software called by the processing element; or in hardware. For example, the processing unit may be a processing element that is set up separately, may be implemented as integrated in a chip of the above-mentioned apparatus, or may be stored in a memory of the above-mentioned apparatus in the form of program codes, and may be called by a processing element of the above-mentioned apparatus to execute the functions of the above-mentioned processing unit. Furthermore, all or part of these units may be integrated together or may be implemented independently. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each unit above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in the form of software.
For example, the above units may be one or more integrated circuits configured to implement the above methods, such as: one or more application specific integrated circuits (application specific integrated circuit, ASIC), or one or more microprocessors (digital signal processor, DSP), or one or more field programmable gate arrays (field programmable gate array, FPGA), or the like. For another example, when some of the above elements are implemented in the form of processing element scheduler code, the processing element may be a general purpose processor, such as a central processing unit (central processing unit, CPU) or other processor that may invoke the program code. For another example, the units may be integrated together and implemented in the form of a system-on-a-chip (SOC).
Fig. 8 is a schematic structural diagram of another image loading device according to an embodiment of the present invention. As shown in fig. 8, the image loading apparatus 300 includes:
at least one processor 31 and a memory 32;
the memory 32 stores computer-executable instructions;
the at least one processor 31 executes computer-executable instructions stored in the memory 32 such that the at least one processor 31 performs an image loading method as performed by an electronic device or an image loading method as performed by a server.
The specific implementation process of the processor 31 may be referred to the above method embodiment, and its implementation principle and technical effects are similar, and this embodiment will not be described herein again.
Optionally, the image loading device 300 further comprises a communication part 33. The processor 31, the memory 32, and the communication unit 33 may be connected via a bus 34.
The embodiment of the invention also provides a storage medium, wherein computer-executable instructions are stored in the storage medium, and the computer-executable instructions are used for realizing the image loading method when being executed by a processor.
Embodiments of the present invention also provide a computer program product which, when run on a computer, causes the computer to perform an image loading method as described above.
In the above embodiments, it should be understood that the described apparatus and method may be implemented in other ways. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be additional divisions when actually implemented, for example, multiple modules may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or modules, which may be in electrical, mechanical, or other forms.
The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional module in the embodiments of the present invention may be integrated in one processing unit, or each module may exist alone physically, or two or more modules may be integrated in one unit. The units formed by the modules can be realized in a form of hardware or a form of hardware and software functional units.
The integrated modules, which are implemented in the form of software functional modules, may be stored in a computer readable storage medium. The software functional module is stored in a storage medium, and includes several instructions for causing an electronic device (which may be a personal computer, a server, or a network device, etc.) or a processor (english: processor) to perform some of the steps of the methods according to the embodiments of the invention.
It is understood that the processor may be a central processing unit (central processing unit, CPU), but may also be other general purpose processors, digital signal processors (digital signal processor, DSP), application specific integrated circuits (application specific integrated circuit, ASIC), etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor for execution, or in a combination of hardware and software modules in a processor for execution.
The memory may comprise a high-speed RAM memory, and may further comprise a non-volatile memory NVM, such as at least one magnetic disk memory, and may also be a U-disk, a removable hard disk, a read-only memory, a magnetic disk or optical disk, etc.
The bus may be an industry standard architecture (industry standard architecture, ISA) bus, an external device interconnect (peripheral component, PCI) bus, or an extended industry standard architecture (extended Industry standard architecture, EISA) bus, among others. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, the buses in the drawings of the present invention are not limited to only one bus or to one type of bus.
The storage medium may be implemented by any type or combination of volatile or nonvolatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an application specific integrated circuit (application specific integrated circuits, ASIC). The processor and the storage medium may reside as discrete components in a terminal or server.
Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the method embodiments described above may be performed by hardware associated with program instructions. The foregoing program may be stored in a computer readable storage medium. The program, when executed, performs steps including the method embodiments described above; and the aforementioned storage medium includes: various media that can store program code, such as ROM, RAM, magnetic or optical disks.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.

Claims (8)

1. An image loading method, comprising:
detecting the current network condition of electronic equipment, and running an application program APP which is required to load high-definition images currently on the electronic equipment;
if the network condition meets a preset condition, sending an image acquisition request to a server, wherein the image acquisition request is used for acquiring an image with resolution lower than a first threshold;
receiving an image with resolution lower than a first threshold sent by the server;
converting the image into a high-definition image by using a super-resolution model and loading the high-definition image through the APP;
before the detecting the current network condition of the electronic device, the method further comprises: sending a downloading request to the server, wherein the downloading request is used for requesting to download the APP;
Receiving an installation package sent by the server, wherein the installation package carries the super-resolution model;
loading the installation package, and installing the APP and the super-resolution model at the same time; the super-resolution model is started when the APP is started.
2. The method according to claim 1, wherein before sending the image acquisition request to the server if the network condition satisfies a preset condition, further comprising:
if the network condition meets a preset condition, detecting whether the electronic equipment caches the image with the resolution lower than a first threshold;
and if the electronic equipment caches the image with the resolution lower than the first threshold value, converting the image into a high-definition image by using a super-resolution model and loading the high-definition image through the APP.
3. The method of claim 1, wherein the super-resolution model comprises any one of the following models: the super-resolution generation is performed on an antagonism network model, a random forest model, an interpolation model and a sparse model.
4. The method of claim 1, wherein said converting said image into a high definition image using a super resolution model and loading via said APP comprises:
Memory is allocated for the super-resolution model;
operating the super-resolution model on the memory, and inputting the image with the resolution lower than a first threshold value into the super-resolution model to obtain the high-definition image;
and loading the high-definition image through the APP.
5. An image loading method, comprising:
receiving an image acquisition request sent by electronic equipment when detecting that the current network condition meets a preset condition, running an application program APP which is currently required to load high-definition images on the electronic equipment, wherein the image acquisition request is used for acquiring images with resolution lower than a first threshold;
acquiring an image with resolution lower than a first threshold according to the image acquisition request;
transmitting the image with the resolution lower than a first threshold to the electronic device;
before detecting that the current network condition meets the preset condition, the receiving electronic device further comprises:
receiving a downloading request sent by the electronic equipment, wherein the downloading request is used for requesting to download the APP;
sending an installation package of the APP to the electronic equipment, wherein the installation package carries a super-resolution model, so that the electronic equipment loads the installation package and installs the super-resolution model while installing the APP; wherein the super-resolution model is started when the APP is started.
6. The method of claim 5, wherein the super-resolution model comprises any one of the following models: the super-resolution generation is performed on an antagonism network model, a random forest model, an interpolation model and a sparse model.
7. An image loading apparatus, comprising:
the processing unit is used for detecting the current network condition of the electronic equipment, and an application program APP needing to be loaded with the high-definition image currently runs on the electronic equipment;
the sending unit is used for sending an image acquisition request to a server if the processing unit outputs that the network condition meets a preset condition, wherein the image acquisition request is used for acquiring an image with resolution lower than a first threshold;
a receiving unit, configured to receive an image with a resolution lower than a first threshold sent by the server;
the processing unit is also used for converting the image into a high-definition image by utilizing a super-resolution model and loading the high-definition image through the APP;
the sending unit is further configured to send a download request to the server before the processing unit detects the current network condition of the electronic device, where the download request is used to request downloading of the APP;
the receiving unit is further configured to receive an installation packet sent by the server, where the installation packet carries the super-resolution model; loading the installation package, and installing the APP and the super-resolution model at the same time; the super-resolution model is started when the APP is started.
8. An image loading apparatus, comprising:
the receiving unit is used for receiving an image acquisition request sent by the electronic equipment when the current network condition is detected to meet the preset condition, an application program APP needing to load high-definition images currently runs on the electronic equipment, and the image acquisition request is used for acquiring images with resolution lower than a first threshold value;
a processing unit, configured to acquire an image with a resolution lower than a first threshold according to the image acquisition request;
a transmitting unit configured to transmit the image with the resolution lower than a first threshold to the electronic device;
the receiving unit is further configured to receive a download request sent by the electronic device before receiving an image acquisition request sent by the electronic device when the current network condition is detected to meet a preset condition, where the download request is used to request downloading of the APP;
the sending unit is further configured to send an installation package of the APP to the electronic device, where the installation package carries the super-resolution model, so that the electronic device loads the installation package and installs the super-resolution model while installing the APP; wherein the super-resolution model is started when the APP is started.
CN201910578189.5A 2019-06-28 2019-06-28 Image loading method and device Active CN112153465B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910578189.5A CN112153465B (en) 2019-06-28 2019-06-28 Image loading method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910578189.5A CN112153465B (en) 2019-06-28 2019-06-28 Image loading method and device

Publications (2)

Publication Number Publication Date
CN112153465A CN112153465A (en) 2020-12-29
CN112153465B true CN112153465B (en) 2024-01-16

Family

ID=73891143

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910578189.5A Active CN112153465B (en) 2019-06-28 2019-06-28 Image loading method and device

Country Status (1)

Country Link
CN (1) CN112153465B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111142751A (en) * 2019-12-26 2020-05-12 腾讯科技(深圳)有限公司 Image processing method and device, intelligent terminal and storage medium
CN113055742B (en) * 2021-03-05 2023-06-09 Oppo广东移动通信有限公司 Video display method, device, terminal and storage medium

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101594203A (en) * 2008-05-30 2009-12-02 索尼株式会社 Dispensing device, sending method and receiving system
CN102047287A (en) * 2008-06-17 2011-05-04 株式会社Ntt都科摩 Image/video quality enhancement and super-resolution using sparse transformations
CN104144313A (en) * 2013-05-10 2014-11-12 中国电信股份有限公司 Video communication method, video communication system and cloud image server
CN204903013U (en) * 2015-06-19 2015-12-23 杭州电子科技大学 Supplementary cell -phone carries out external device of infrared thermal imaging
CN107105338A (en) * 2017-03-17 2017-08-29 深圳市先河系统技术有限公司 The method and terminal asked for display image, video and response to network
CN107784686A (en) * 2016-08-25 2018-03-09 上海交通大学 Various visual angles media displaying method based on multiresolution
CN107959668A (en) * 2016-10-14 2018-04-24 南宁富桂精密工业有限公司 Streaming media optimization method and buffer storage
WO2018133597A1 (en) * 2017-01-19 2018-07-26 优视科技有限公司 Method and system for loading web page images, electronic device and readable storage medium

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10489887B2 (en) * 2017-04-10 2019-11-26 Samsung Electronics Co., Ltd. System and method for deep learning image super resolution

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101594203A (en) * 2008-05-30 2009-12-02 索尼株式会社 Dispensing device, sending method and receiving system
CN102047287A (en) * 2008-06-17 2011-05-04 株式会社Ntt都科摩 Image/video quality enhancement and super-resolution using sparse transformations
CN104144313A (en) * 2013-05-10 2014-11-12 中国电信股份有限公司 Video communication method, video communication system and cloud image server
CN204903013U (en) * 2015-06-19 2015-12-23 杭州电子科技大学 Supplementary cell -phone carries out external device of infrared thermal imaging
CN107784686A (en) * 2016-08-25 2018-03-09 上海交通大学 Various visual angles media displaying method based on multiresolution
CN107959668A (en) * 2016-10-14 2018-04-24 南宁富桂精密工业有限公司 Streaming media optimization method and buffer storage
WO2018133597A1 (en) * 2017-01-19 2018-07-26 优视科技有限公司 Method and system for loading web page images, electronic device and readable storage medium
CN107105338A (en) * 2017-03-17 2017-08-29 深圳市先河系统技术有限公司 The method and terminal asked for display image, video and response to network

Also Published As

Publication number Publication date
CN112153465A (en) 2020-12-29

Similar Documents

Publication Publication Date Title
EP3625708A1 (en) Two-dimensional code generation method, two-dimensional code processing method, apparatus, device, and two-dimensional code
CN103914205B (en) A kind of file thumbnail methods of exhibiting of intelligent terminal and device
CN107040574B (en) Screenshot and data processing method and device
CN112153465B (en) Image loading method and device
CN111666497A (en) Application program loading method and device, electronic equipment and readable storage medium
CN107479868B (en) Interface loading method, device and equipment
US20190114989A1 (en) Systems and methods for image optimization
CN111338623A (en) Method, device, medium and electronic equipment for developing user interface
CN111581555A (en) Document loading method, device, equipment and storage medium
CN105487640A (en) Image pre-loading and caching method of Android watch
CN113411404A (en) File downloading method, device, server and storage medium
CN113127361A (en) Application program development method and device, electronic equipment and storage medium
CN110874298B (en) Request data storage method and terminal equipment
CN110750744A (en) Page presentation method and device
CN102377793B (en) Flash interaction method and device based on mobile communication equipment terminal
CN108549557A (en) Using storing method, terminal and storage medium
CN110505289B (en) File downloading method and device, computer readable medium and wireless communication equipment
CN112416303A (en) Software development kit thermal restoration method and device and electronic equipment
CN107580033B (en) Load information processing method and device, message processing method and device, and server
CN116596748A (en) Image stylization processing method, apparatus, device, storage medium, and program product
JP5643449B1 (en) Server and method for displaying display screen on terminal device
CN114692053A (en) Lossless compression method for webpage picture, electronic equipment and storage medium
CN111414383B (en) Data request method, data processing system and computing device
CN114218175A (en) Resource cross-platform sharing method and device, terminal equipment and storage medium
CN110611817B (en) Image compression based on text image content

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant