CN112153465A - Image loading method and device - Google Patents

Image loading method and device Download PDF

Info

Publication number
CN112153465A
CN112153465A CN201910578189.5A CN201910578189A CN112153465A CN 112153465 A CN112153465 A CN 112153465A CN 201910578189 A CN201910578189 A CN 201910578189A CN 112153465 A CN112153465 A CN 112153465A
Authority
CN
China
Prior art keywords
image
resolution
model
app
super
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910578189.5A
Other languages
Chinese (zh)
Other versions
CN112153465B (en
Inventor
胡泊
陆凯
王孝满
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Century Trading Co Ltd
Beijing Jingdong Shangke Information Technology Co Ltd
Original Assignee
Beijing Jingdong Century Trading Co Ltd
Beijing Jingdong Shangke Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Century Trading Co Ltd, Beijing Jingdong Shangke Information Technology Co Ltd filed Critical Beijing Jingdong Century Trading Co Ltd
Priority to CN201910578189.5A priority Critical patent/CN112153465B/en
Publication of CN112153465A publication Critical patent/CN112153465A/en
Application granted granted Critical
Publication of CN112153465B publication Critical patent/CN112153465B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440263Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234363Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41Structure of client; Structure of client peripherals
    • H04N21/414Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
    • H04N21/41407Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The embodiment of the invention provides an image loading method and device, when a user uses an APP through a data network and when an electronic device detects that a network condition meets a preset condition, an image acquisition request for requesting low resolution is sent to a server, so that the server sends a low resolution image to the electronic device, the electronic device converts the low resolution image into a high resolution image by using a locally deployed super resolution model and loads the high resolution image through the APP, and for the user, the low resolution image is smaller, so that the data traffic of the user can be saved. Meanwhile, under the condition that the network condition is not good, the loading speed of the image can be accelerated. In addition, for the server, the issuing of the low-resolution image can save a large amount of server bandwidth and data space, thereby saving the cost.

Description

Image loading method and device
Technical Field
The embodiment of the invention relates to the technical field of image processing, in particular to an image loading method and device.
Background
With the high-speed development of the mobile internet technology, electronic devices such as mobile phones gradually become essential tools in people's lives, various Applications (APPs) run on the tools, different APPs have different functions, and users perform social contact, shopping, learning or entertainment and the like by using the various APPs.
In the process that the user uses the APP, the electronic equipment acquires characters, images and the like from the server and presents the acquired contents to the user through a screen of the electronic equipment. Generally, since images are more intuitive and more visual than characters, using the APP process is essentially a process in which an electronic device continuously obtains images from a server and loads them. In the process, if the user uses a data network, when the resolution of the image is high, the data traffic of the user is seriously consumed. In order to avoid the disadvantage, some commonly used images for accessing the APP are usually built in an installation package of the APP, and a user downloads and follows the installation package, so that the commonly used images are stored locally in the electronic device. Then, the APP runs, and the common images are loaded from the local.
In the process of loading the image, the image needs to be built in an installation package of the APP. However, to avoid the installation package volume expansion of APP, only a limited number of images can be built in the installation package. If the APP runs and other images need to be accessed, the terminal equipment needs to acquire the images from the server and load the images. In this case, if the user uses the data traffic, the data traffic of the user is also consumed seriously.
Disclosure of Invention
The embodiment of the invention provides an image loading method and device.
In a first aspect, an embodiment of the present invention provides an image loading method, including:
detecting the current network condition of electronic equipment, wherein an application program APP which needs to be loaded with a high-definition image currently runs on the electronic equipment;
if the network condition meets a preset condition, sending an image acquisition request to a server, wherein the image acquisition request is used for acquiring an image with the resolution lower than a first threshold;
receiving an image which is sent by the server and has the resolution lower than a first threshold value;
and converting the image into a high-definition image by using a super-resolution model and loading the high-definition image through the APP.
In a possible design, before detecting the current network condition of the electronic device, the method further includes:
sending a downloading request to the server, wherein the downloading request is used for requesting downloading of the APP;
and receiving an installation package sent by the server, wherein the super-resolution model is carried by the installation package.
In a possible design, before sending the image acquisition request to the server if the network condition satisfies a preset condition, the method further includes:
if the network condition meets a preset condition, detecting whether the electronic equipment caches the image with the resolution ratio lower than a first threshold value;
if the electronic equipment caches the image with the resolution lower than the first threshold, the image is converted into a high-definition image by using a super-resolution model and is loaded through the APP.
In one possible design, the super-resolution model includes any one of the following models: the super-resolution generation method comprises a confrontation network model, a random forest model, an interpolation model and a sparse model.
In one possible design, the converting the image into a high definition image using a super resolution model and loading the high definition image through the APP includes:
allocating a memory for the super-resolution model;
running the super-resolution model on the memory, and inputting the image with the resolution lower than a first threshold value into the super-resolution model to obtain the high-definition image;
and loading the high-definition image through the APP.
In a second aspect, an embodiment of the present invention provides an image loading method, including:
receiving an image acquisition request sent by electronic equipment when the current network condition is detected to meet a preset condition, wherein the electronic equipment runs an application program APP which needs to load a high-definition image at present, and the image acquisition request is used for acquiring an image with resolution lower than a first threshold;
acquiring an image with resolution lower than a first threshold according to the image acquisition request;
and sending the image with the resolution lower than a first threshold value to the electronic equipment.
In a possible design, before the receiving electronic device detects that the current network condition satisfies the preset condition, the method further includes:
receiving a downloading request sent by the electronic equipment, wherein the downloading request is used for requesting to download the APP;
and sending the installation package of the APP to the electronic equipment, wherein the super-resolution model is carried by the installation package.
In one possible design, the super-resolution model includes any one of the following models: the super-resolution generation method comprises a confrontation network model, a random forest model, an interpolation model and a sparse model.
In a third aspect, an embodiment of the present invention provides an image loading apparatus, including:
the processing unit is used for detecting the current network condition of the electronic equipment, and an application program APP which needs to be loaded with a high-definition image currently runs on the electronic equipment;
the sending unit is used for sending an image acquisition request to a server if the network condition of the water processing unit meets a preset condition, wherein the image acquisition request is used for acquiring an image with the resolution ratio lower than a first threshold value;
the receiving unit is used for receiving the image which is sent by the server and has the resolution lower than a first threshold value;
and the processing unit is also used for converting the image into a high-definition image by using a super-resolution model and loading the high-definition image through the APP.
In a possible design, before the processing unit detects a current network condition of the electronic device, the sending unit is further configured to send a download request to the server, where the download request is used to request to download the APP;
the receiving unit is further configured to receive an installation package sent by the server, where the installation package carries the super-resolution model.
In a feasible design, the processing unit, when the network condition satisfies a preset condition, before the sending unit sends an image acquisition request to a server, is further configured to detect whether the electronic device caches an image with a resolution lower than a first threshold, and if the electronic device caches an image with a resolution lower than the first threshold, convert the image into a high-definition image by using a super-resolution model and load the high-definition image through the APP.
In one possible design, the super-resolution model includes any one of the following models: the super-resolution generation method comprises a confrontation network model, a random forest model, an interpolation model and a sparse model.
In a feasible design, the processing unit is configured to allocate a memory for the super-resolution model, input the image with the resolution lower than the first threshold to the super-resolution model, run the super-resolution model on the memory, input the image with the resolution lower than the first threshold to the super-resolution model, obtain the high-definition image, and load the high-definition image through the APP.
In a fourth aspect, an embodiment of the present invention provides an image loading method, including:
the device comprises a receiving unit, a processing unit and a processing unit, wherein the receiving unit is used for receiving an image acquisition request sent by electronic equipment when the current network condition is detected to meet a preset condition, the electronic equipment runs an application program APP which needs to load a high-definition image at present, and the image acquisition request is used for acquiring an image with the resolution lower than a first threshold;
the processing unit is used for acquiring an image with the resolution ratio lower than a first threshold value according to the image acquisition request;
a sending unit, configured to send the image with the resolution lower than a first threshold to the electronic device.
In a feasible design, the receiving unit is further configured to receive a download request sent by the electronic device before receiving an image acquisition request sent by the electronic device when it is detected that a current network condition meets a preset condition, where the download request is used to request to download the APP;
the sending unit is further configured to send an installation package of the APP to the electronic device, where the installation package carries the super-resolution model.
In one possible design, the super-resolution model includes any one of the following models: the super-resolution generation method comprises a confrontation network model, a random forest model, an interpolation model and a sparse model.
In a fifth aspect, an embodiment of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the method according to the first aspect or the various possible implementations of the first aspect.
In a sixth aspect, an embodiment of the present invention provides a server, including a processor, a memory, and a computer program stored on the memory and executable on the processor, where the processor executes the program to implement the method according to the first aspect or the various possible implementations of the first aspect.
In a seventh aspect, an embodiment of the present invention provides a storage medium, where instructions are stored, and when the instructions are executed on an electronic device, the electronic device is caused to perform the method according to the first aspect or the various possible implementations of the first aspect.
In an eighth aspect, embodiments of the present invention provide a storage medium having stored therein instructions that, when executed on a server, cause the server to perform a method as set forth in the second aspect or various possible implementations of the second aspect.
In a ninth aspect, embodiments of the present invention provide a computer program product, which, when run on an electronic device, causes the electronic device to perform the method according to the first aspect or the various possible implementations of the first aspect.
In a tenth aspect, embodiments of the present invention provide a computer program product, which when run on a server, causes the server to perform the method according to the second aspect or the various possible implementations of the second aspect.
According to the image loading method and device provided by the embodiment of the invention, when a user uses an APP through a data network and when the electronic equipment detects that the network condition meets the preset condition, the electronic equipment sends an image acquisition request for requesting low resolution to the server, so that the server sends a low resolution image to the electronic equipment, the electronic equipment converts the low resolution image into a high resolution image by using a locally deployed super resolution model, and the high resolution image is loaded through the APP. Meanwhile, under the condition that the network condition is not good, the loading speed of the image can be accelerated. In addition, for the server, the issuing of the low-resolution image can save a large amount of server bandwidth and data space, thereby saving the cost.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic diagram of an operating environment of an image loading method according to an embodiment of the present invention;
FIG. 2 is a flowchart of an image loading method according to an embodiment of the present invention;
FIG. 3 is another flowchart of an image loading method according to an embodiment of the present invention;
FIG. 4 is a flowchart of training a super-resolution model in the image loading method according to the embodiment of the present invention;
fig. 5 is a flowchart of super-resolution processing in the image loading method provided by the embodiment of the present invention;
fig. 6 is a schematic structural diagram of an image loading apparatus according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of another image loading apparatus according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of another image loading apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The user uses the APP process, which is substantially a process in which the electronic device bearing the APP continuously obtains images from the server and loads the images. In the process, if the user uses a data network, when the resolution of the image is high, the data traffic of the user is seriously consumed. In order to avoid the disadvantage, some commonly used images for accessing the APP are usually built in an installation package of the APP, and a user downloads and follows the installation package, so that the commonly used images are stored locally in the electronic device. Then, the APP runs, and the common images are loaded from the local.
In the process of loading the image, the image needs to be built in an installation package of the APP. However, to avoid the installation package volume expansion of APP, only a limited number of images can be built in the installation package. If the APP runs and other images need to be accessed, the terminal equipment needs to acquire the images from the server and load the images. In this case, if the user uses the data traffic, the data traffic of the user is also consumed seriously. Moreover, this kind of mode is very rigid, and when the image that APP was used often changed, need research and development reissue the installation package of the APP that carries new image, the user downloads this installation package and updates the commonly used image of local storage. In addition, if the APP runs and needs to access other images, the terminal device needs to acquire and load the images from the server. In this case, if the user uses the data network, the data traffic of the user is also consumed seriously.
At present, other modes are provided for solving the problem that image loading is slow or loading fails when the network condition is not good. For example, the image updating method depends on a WiFi environment, and in the method, only when the electronic device is connected to WiFi, when the APP runs, the server sends a high-resolution image to the APP, so that data traffic of a user is prevented from being consumed. However, this approach relies heavily on WiFi networks, limiting the environment in which users use APP, and is very inflexible. For another example, the server compresses the image by webp and transmits the compressed image to the electronic device, and the size of the compressed image is about 2/3 of the original image. For another example, in the bottom-pop scheme, if the electronic device fails to load an image, a thumbnail of the image is acquired from the server and loaded, but the thumbnail is not clear after being enlarged.
In view of this, embodiments of the present invention provide an image loading method and apparatus, in which an electronic device obtains a low-resolution image from a server, converts the low-resolution image into a high-resolution image, and then loads and displays the high-resolution image, thereby achieving the purpose of reducing data consumption.
Fig. 1 is a schematic operating environment diagram of an image loading method according to an embodiment of the present invention. Referring to fig. 1, an electronic device establishes a network connection with a server, and runs an APP on the electronic device, where the server is a server providing a service for the APP. The electronic device has a function of detecting a network condition, and when the current network condition is not good, the electronic device sends an image acquisition request for acquiring an image with a resolution lower than a first threshold to the server to acquire an image with a low resolution. After receiving the image acquisition request, the server does not send the high-resolution image to the electronic device, but sends the image with the resolution lower than the first threshold value to the electronic device. After receiving the image with the resolution lower than the first threshold, the electronic device converts the image with the resolution lower than the first threshold into a high-definition image by using a super-resolution model deployed on the electronic device, that is, converts the image with the low resolution into the image with the high resolution by using a super-resolution technology. The electronic device may be, for example, a computer, a notebook, a mobile phone, and the like of a user, and the embodiment of the present invention is not limited. The server may be, for example, a server corresponding to the APP.
Next, on the basis of fig. 1, an image loading method according to an embodiment of the present invention is described in detail. For example, please refer to fig. 2.
Fig. 2 is a flowchart of an image loading method according to an embodiment of the present invention. The embodiment describes the image loading method in detail from the perspective of interaction between a server and a terminal device, and includes:
101. detecting whether the current network condition of the electronic equipment meets a preset condition, running an application program APP which needs to be loaded with a high-definition image currently on the electronic equipment, and executing step 102 if the network condition meets the preset condition; if the network condition does not satisfy the preset condition, go to step 106.
Illustratively, when a user clicks an icon of an APP on a touch screen on an electronic device to start the APP, the APP needs to load a main page of the APP, some advertisements, and the like, and needs to load a high-definition image. At this time, the electronic device detects its current network status. For example, the preset condition is that the electronic device is connected to a server through a data network.
102. And if the network condition meets a preset condition, sending an image acquisition request to a server, wherein the image acquisition request is used for acquiring an image with the resolution lower than a first threshold value.
For example, if the electronic device detects that the network condition satisfies a preset condition, for example, the electronic device is connected to a server through a data network, the electronic device sends an image for acquiring a resolution lower than a first threshold to the server to request the server to send a low-resolution image to the electronic device.
103. And acquiring the image with the resolution lower than a first threshold according to the image acquisition request.
Illustratively, after receiving the image acquisition request, the server acquires, from the local or other place, an image with a resolution lower than the first threshold, that is, a low-resolution image, according to the image acquisition request, where the low-resolution image has a corresponding relationship with the high-resolution image required by the APP.
104. And sending the image with the resolution lower than a first threshold value to the electronic equipment.
Illustratively, the server sends the low-resolution image determined in step 103 to the electronic device through a data network, and since the low-resolution image is small, data traffic of the user can be saved.
105. And converting the image into a high-definition image by using a super-resolution model and loading the high-definition image through the APP.
Illustratively, after receiving a low-resolution image, the electronic device converts the low-resolution image into a high-definition image by using a super-resolution model deployed locally, that is, converts the low-resolution image into a high-resolution image by using a super-resolution technology, and then loads and displays the high-resolution image to a user through an APP.
106. And requesting to acquire a high-definition image from a server.
According to the image loading method provided by the embodiment, when a user uses an APP through a data network and when the electronic device detects that the network condition meets the preset condition, an image acquisition request for requesting low resolution is sent to the server, so that the server sends a low resolution image to the electronic device, the electronic device converts the low resolution image into a high resolution image by using a locally deployed super-resolution model, and the high resolution image is loaded through the APP. Meanwhile, under the condition that the network condition is not good, the loading speed of the image can be accelerated. In addition, for the server, the issuing of the low-resolution image can save a large amount of server bandwidth and data space, thereby saving the cost.
In the above embodiments, the network condition includes, but is not limited to, a network connection manner between the electronic device and the server. For example, the network condition refers to the degree of quality of network connection, and at this time, a corresponding relationship between the packet loss rate and the degree of quality of the network condition is preset, and when the packet loss rate is lower than a certain threshold a, the network condition is considered to be good, when the packet loss rate is higher than a threshold b, the network condition is considered to be poor, and when the packet loss rate is between the threshold a and the threshold b, the network condition is considered to be good. As another example, the network condition refers to a network delay of a current network, the number of users accessing the server, and the like.
In the above embodiments, the super-resolution model is used to convert an image with a low resolution into an image with a high resolution by using a super-resolution technique, which is an inversion technique because of the lack of image information of the low resolution. In the embodiment of the invention, the super-resolution model includes, but is not limited to, a super-resolution generation countermeasure network model, a random forest model, an interpolation model, a sparse model and the like. The interpolation model restores the low-resolution image into a high-resolution image by adding some prior information into the low-resolution image; the random forest model is obtained by training based on a regression method; the sparse model is obtained based on sparse learning; a Convolutional Neural Network (CNN) model has a very good effect in a Super-Resolution technology, and a Super-Resolution generation countermeasure Network (SGRAN) model in the CNN model is obtained by performing gaussian noise processing, motion blur addition, and the like on an image.
In the embodiment of the application, the super-resolution model is pre-deployed inside the electronic device. Before deployment, a super-resolution model needs to be trained. Because strong performance and computing power are needed for training the super-resolution model through machine learning and other manners, the model training is performed by a server, which may be a server providing business services for the APP or other servers. After the server trains the super-resolution model, the super-resolution model is packaged into an installation Package of the APP, for example, the super-resolution model is packaged into an Android installation Package (Android Package, APK) of the APP, and the installation Package of the APP is uploaded to an APP application store, for example, an APP store, a server providing services for the APP, and the like, for a user to download. Therefore, in the downloading process, the electronic device sends a downloading request to the server, and the downloading request is used for requesting to download the APP. After receiving the downloading request, the server sends the installation package of the APP to the electronic equipment; correspondingly, the electronic equipment receives an installation package sent by the server, and the installation package carries the super-resolution model. And then, the electronic equipment loads the installation package, and installs the resolution model while installing the APP. After the successful installation, the super-resolution model is started at the same time of APP starting every time.
In the following, an example is used to describe how to quickly load an image when the electronic device runs an APP that requires a high-definition image. For example, referring to fig. 3, fig. 3 is another flowchart of an image loading method according to an embodiment of the present invention, where the embodiment includes:
201. the electronic equipment detects that the APP needs to load the high-definition images.
Illustratively, the trained super-resolution model is pre-deployed on the electronic device. When a user clicks an APP, the APP needs to load a high-definition image, for example, an advertisement image issued by a server when the APP is started, an image displayed on a home page of the APP, and the like.
202. The electronic device detects whether the electronic device caches the image with the resolution lower than the first threshold, and if the electronic device caches the image with the resolution lower than the first threshold, step 203 is executed; if there is no image with the resolution lower than the first threshold in the cache of the electronic device, executing step 204;
illustratively, when a high-definition image needs to be loaded through an APP, the electronic device first determines whether a low-resolution image exists in a cache, and if so, executes step 203 to process the image; if there is no low-resolution image, step 204 is executed to request the server to issue a low-resolution image, where the low-resolution image is an image with a resolution lower than the first threshold.
203. The electronic device reads the image with the resolution lower than the first threshold value in the buffer.
204. The electronic device sends an image acquisition request to the server, wherein the image acquisition request is used for acquiring images with the resolution lower than a first threshold value.
205. And the server acquires the image with the resolution lower than the first threshold according to the image acquisition request.
206. The server sends the image with the resolution lower than the first threshold value to the electronic equipment.
207. The electronic device converts the image into a high-definition image using a super-resolution model.
208. The electronic equipment loads the high-definition image through the APP.
Illustratively, in step 207-.
In the above example, because of the low resolution image sent by the server to the electronic device, traffic consumption of the user can be saved when the user uses the data network. In addition, regardless of the data network or WiFi used by the user, if the network condition is not good, the image loading time can be shortened.
In the above embodiment, the super-resolution model is obtained by training the sample image by the server, and since the super-resolution model obtained by training by the server is a model generated through curing, the model cannot be directly deployed to the electronic device, and can be deployed to the electronic device only after being converted. In the following, taking the super-resolution model as the SRGAN model as an example, how the server trains to obtain the SRGAN model and converts the SRGAN model will be described in detail. For example, see fig. 4.
Fig. 4 is a flowchart of training a super-resolution model in an image loading method provided in the embodiment of the present invention, where the embodiment includes:
301. the server trains the SRGAN model.
Illustratively, the server trains the sample image using TensorFlow or the like, resulting in the SRGAN model. In the training process, a graphic symbol (Graphdef file) and a check point (checkpoint) file are generated.
TensorFlow is an artificial intelligence framework of Google (Google) open source and is also a mainstream network building and model training tool. The SRGAN model is a super-resolution model based on a Convolutional Neural Network (CNN), and is trained based on a Generative Adaptive Network (GAN) method, the SRGAN model comprises a generator and a discriminator, the generator is also called G network, the discriminator is also called D network, and the main body of the discriminator uses Visual Geometry Group (VGG); the generator is a chain of residual block connections. The generator generates a high-resolution image from the low-resolution image, and the discriminator determines whether the high-resolution image is generated by the generator or an original image in the database. When the discriminator is able to discriminate the image generated by the generator as the original image in the data, i.e., the discriminator was successfully fooled by the generator, the model training is considered to be over.
In the model training process, a sub-pixel (sub) module is added in the SRGAN model, so that the resolution of the generator is increased on the image in the last network layer, the resolution is improved, and meanwhile, the consumption of computing resources is reduced.
302. And carrying out network curing operation on the image symbol file and the check point file to obtain a cured image symbol file.
Illustratively, the server performs a network solidification operation on the Graphdef file and the checkpoint generated in step 301 to obtain a solidified Graphdef file, where the solidified Graphdef file is also referred to as a frozen image symbol file, that is, a frozen Graphdef file.
303. And performing model conversion on the solidified image symbol file to obtain an SRGAN model which can run on the electronic equipment.
Illustratively, the server performs model transformation using a binary tool provided by TensorFlow or by Python programming, the transformation being examined after the transformation. The model file of tflite corresponds to a TensorFlow Life scheme, the TensorFlow Life scheme is a lightweight solution of TensorFlow, the TensorFlow Life scheme is converted into an electronic device design, and you can realize low-delay end-side machine learning inference by optimizing a kernel, presetting an activation function and the like.
304. And packaging the SRGAN model into an installation package of the APP, so that the SRGAN model is deployed into the electronic equipment after the electronic equipment downloads the installation package.
Next, taking the super-resolution model as the SRGAN model as an example, how the super-resolution model performs super-resolution processing on a low-resolution picture in the above embodiment is described in detail. For example, see fig. 5.
Fig. 5 is a flowchart of super-resolution processing in an image loading method according to an embodiment of the present invention, where the embodiment includes:
401. the electronic device initializes the SRGAN model.
Illustratively, the process of converting a low resolution image to a high resolution image by an electronic device using the SRGAN model is accomplished by scheduling TensorFlow Life.
402. The electronic device allocates memory for the SRGAN model.
For example, before the SRGAN model is run, memory needs to be allocated for the SRGAN model.
403. And the electronic equipment runs an SRGAN model on a memory, and inputs the image with the resolution lower than a first threshold value into the super-resolution model to obtain the high-definition image.
404. The electronic equipment loads the high-definition image through the APP.
Illustratively, in steps 403 and 404, the electronic device inputs the low resolution image in the cache or the low resolution image received from the server as an input of the model to the SRGAN model, so that the SRGAN model starts an inference process, i.e., a process of converting the low resolution image into a high resolution image. In the process, the resolution of the low-resolution image can be improved by a factor of about 4 by the SRGAN model, and the SRGAN model supports a hyper-resolution effect of about 4 times at most depending on the capability of the SRGAN model.
405. The electronic device releases the memory.
Illustratively, after converting a low resolution image to a high resolution image, the electronic device frees memory allocated to the SRGAN model.
The following are embodiments of the apparatus of the present invention that may be used to perform embodiments of the method of the present invention. For details which are not disclosed in the embodiments of the apparatus of the present invention, reference is made to the embodiments of the method of the present invention.
Fig. 6 is a schematic structural diagram of an image loading apparatus according to an embodiment of the present invention. The image loading apparatus according to the present embodiment may be an electronic device, or may be a chip applied to an electronic device. The image loading device may be used to perform the functions of the electronic device in the above embodiments. As shown in fig. 6, the image loading apparatus 100 may include:
the processing unit 11 is configured to detect a current network condition of an electronic device, where an application APP that needs to load a high-definition image is currently running on the electronic device;
a sending unit 12, configured to send an image acquisition request to a server if the network condition of the water processing unit 11 meets a preset condition, where the image acquisition request is used to acquire an image with a resolution lower than a first threshold;
a receiving unit 13, configured to receive an image sent by the server and having a resolution lower than a first threshold;
and the processing unit 11 is further configured to convert the image into a high-definition image by using a super-resolution model and load the high-definition image through the APP.
In a possible design, the sending unit 12, before the processing unit 11 detects the current network condition of the electronic device, is further configured to send a download request to the server, where the download request is used to request to download the APP;
the receiving unit 13 is further configured to receive an installation package sent by the server, where the installation package carries the super-resolution model.
In a possible design, before the network condition satisfies a preset condition and the sending unit 12 sends an image acquisition request to a server, the processing unit 11 is further configured to detect whether the electronic device caches an image with a resolution lower than a first threshold, and if the electronic device caches an image with a resolution lower than the first threshold, convert the image into a high-definition image by using a super-resolution model and load the high-definition image through the APP.
In one possible design, the super-resolution model includes any one of the following models: the super-resolution generation method comprises a confrontation network model, a random forest model, an interpolation model and a sparse model.
In a feasible design, the processing unit 11 is configured to allocate a memory for the super-resolution model, run the super-resolution model on the memory, input the image with the resolution lower than the first threshold to the super-resolution model, obtain the high-definition image, and load the high-definition image through the APP.
The image loading device provided in the embodiment of the present application can execute the actions of the electronic device in the above embodiments, and the implementation principle and the technical effect are similar, which are not described herein again.
Fig. 7 is a schematic structural diagram of another image loading apparatus according to an embodiment of the present invention. The image loading apparatus according to the present embodiment may be a server, or may be a chip applied to a server. The image loading apparatus may be configured to perform the functions of the server in the above embodiments. As shown in fig. 7, the image loading apparatus 200 may include:
the device comprises a receiving unit 21, a processing unit and a processing unit, wherein the receiving unit 21 is used for receiving an image acquisition request sent by an electronic device when detecting that a current network condition meets a preset condition, the electronic device runs an application program APP which needs to load a high-definition image currently, and the image acquisition request is used for acquiring an image with a resolution lower than a first threshold;
a processing unit 22, configured to acquire, according to the image acquisition request, an image with a resolution lower than a first threshold;
a sending unit 23, configured to send the image with the resolution lower than the first threshold to the electronic device.
In a feasible design, the receiving unit 21 is further configured to receive a download request sent by the electronic device before receiving an image acquisition request sent by the electronic device when it is detected that a current network condition meets a preset condition, where the download request is used to request to download the APP;
the sending unit 23 is further configured to send an installation package of the APP to the electronic device, where the installation package carries the super-resolution model.
In one possible design, the super-resolution model includes any one of the following models: the super-resolution generation method comprises a confrontation network model, a random forest model, an interpolation model and a sparse model.
The login verification device provided by the embodiment of the invention can execute the action of the server in the embodiment, the implementation principle and the technical effect are similar, and the detailed description is omitted.
It should be noted that the above receiving unit may be actually implemented as a receiver, and the transmitting unit may be actually implemented as a transmitter. The processing unit can be realized in the form of software called by the processing element; or may be implemented in hardware. For example, the processing unit may be a processing element separately set up, or may be implemented by being integrated into a chip of the apparatus, or may be stored in a memory of the apparatus in the form of program code, and a function of the processing unit may be called and executed by a processing element of the apparatus. In addition, all or part of the units can be integrated together or can be independently realized. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, the steps of the method or the units above may be implemented by hardware integrated logic circuits in a processor element or instructions in software.
For example, the above units may be one or more integrated circuits configured to implement the above methods, such as: one or more Application Specific Integrated Circuits (ASICs), or one or more microprocessors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs), etc. For another example, when some of the above units are implemented in the form of a processing element scheduler code, the processing element may be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor that can call the program code. As another example, these units may be integrated together and implemented in the form of a system-on-a-chip (SOC).
Fig. 8 is a schematic structural diagram of another image loading apparatus according to an embodiment of the present invention. As shown in fig. 8, the image loading apparatus 300 includes:
at least one processor 31 and memory 32;
the memory 32 stores computer-executable instructions;
the at least one processor 31 executes computer-executable instructions stored by the memory 32, causing the at least one processor 31 to perform an image loading method as performed by an electronic device or a server.
For a specific implementation process of the processor 31, reference may be made to the above method embodiments, which implement the principle and the technical effect similarly, and details of this embodiment are not described herein again.
Optionally, the image loading apparatus 300 further includes a communication section 33. The processor 31, the memory 32, and the communication unit 33 may be connected by a bus 34.
The embodiment of the present invention further provides a storage medium, where the storage medium stores computer execution instructions, and the computer execution instructions are executed by a processor to implement the image loading method described above.
Embodiments of the present invention further provide a computer program product, which, when running on a computer, causes the computer to execute the above image loading method.
In the above embodiments, it should be understood that the described apparatus and method may be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the modules is only one logical division, and other divisions may be realized in practice, for example, a plurality of modules may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each module may exist alone physically, or two or more modules are integrated into one unit. The unit formed by the modules can be realized in a hardware form, and can also be realized in a form of hardware and a software functional unit.
The integrated module implemented in the form of a software functional module may be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions to enable an electronic device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the method according to various embodiments of the present invention.
It should be understood that the processor may be a Central Processing Unit (CPU), other general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor, or in a combination of the hardware and software modules within the processor.
The memory may comprise a high-speed RAM memory, and may further comprise a non-volatile storage NVM, such as at least one disk memory, and may also be a usb disk, a removable hard disk, a read-only memory, a magnetic or optical disk, etc.
The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, the buses in the figures of the present invention are not limited to only one bus or one type of bus.
The storage medium may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an Application Specific Integrated Circuits (ASIC). Of course, the processor and the storage medium may reside as discrete components in a terminal or server.
Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. An image loading method, comprising:
detecting the current network condition of electronic equipment, wherein an application program APP which needs to be loaded with a high-definition image currently runs on the electronic equipment;
if the network condition meets a preset condition, sending an image acquisition request to a server, wherein the image acquisition request is used for acquiring an image with the resolution lower than a first threshold;
receiving an image which is sent by the server and has the resolution lower than a first threshold value;
and converting the image into a high-definition image by using a super-resolution model and loading the high-definition image through the APP.
2. The method of claim 1, wherein before detecting the current network condition of the electronic device, further comprising:
sending a downloading request to the server, wherein the downloading request is used for requesting downloading of the APP;
and receiving an installation package sent by the server, wherein the super-resolution model is carried by the installation package.
3. The method according to claim 1 or 2, wherein before sending the image acquisition request to the server if the network condition satisfies a preset condition, the method further comprises:
if the network condition meets a preset condition, detecting whether the electronic equipment caches the image with the resolution ratio lower than a first threshold value;
if the electronic equipment caches the image with the resolution lower than the first threshold, the image is converted into a high-definition image by using a super-resolution model and is loaded through the APP.
4. The method of claim 1 or 2, wherein the super-resolution model comprises any one of the following models: the super-resolution generation method comprises a confrontation network model, a random forest model, an interpolation model and a sparse model.
5. The method according to claim 1 or 2, wherein the converting the image into a high-definition image by using a super-resolution model and loading the high-definition image by the APP comprises:
allocating a memory for the super-resolution model;
running the super-resolution model on the memory, and inputting the image with the resolution lower than a first threshold value into the super-resolution model to obtain the high-definition image;
and loading the high-definition image through the APP.
6. An image loading method, comprising:
receiving an image acquisition request sent by electronic equipment when the current network condition is detected to meet a preset condition, wherein the electronic equipment runs an application program APP which needs to load a high-definition image at present, and the image acquisition request is used for acquiring an image with resolution lower than a first threshold;
acquiring an image with resolution lower than a first threshold according to the image acquisition request;
and sending the image with the resolution lower than a first threshold value to the electronic equipment.
7. The method of claim 6, wherein before the receiving the image acquisition request sent by the electronic device when the current network condition is detected to satisfy the preset condition, the method further comprises:
receiving a downloading request sent by the electronic equipment, wherein the downloading request is used for requesting to download the APP;
and sending the installation package of the APP to the electronic equipment, wherein the installation package carries a super-resolution model.
8. The method of claim 7, wherein the super-resolution model comprises any one of the following models: the super-resolution generation method comprises a confrontation network model, a random forest model, an interpolation model and a sparse model.
9. An image loading apparatus characterized by comprising:
the processing unit is used for detecting the current network condition of the electronic equipment, and an application program APP which needs to be loaded with a high-definition image currently runs on the electronic equipment;
the sending unit is used for sending an image acquisition request to a server if the network condition of the processing unit meets a preset condition, wherein the image acquisition request is used for acquiring an image with the resolution lower than a first threshold value;
the receiving unit is used for receiving the image which is sent by the server and has the resolution lower than a first threshold value;
and the processing unit is also used for converting the image into a high-definition image by using a super-resolution model and loading the high-definition image through the APP.
10. An image loading apparatus characterized by comprising:
the device comprises a receiving unit, a processing unit and a processing unit, wherein the receiving unit is used for receiving an image acquisition request sent by electronic equipment when the current network condition is detected to meet a preset condition, the electronic equipment runs an application program APP which needs to load a high-definition image at present, and the image acquisition request is used for acquiring an image with the resolution lower than a first threshold;
the processing unit is used for acquiring an image with the resolution ratio lower than a first threshold value according to the image acquisition request;
a sending unit, configured to send the image with the resolution lower than a first threshold to the electronic device.
CN201910578189.5A 2019-06-28 2019-06-28 Image loading method and device Active CN112153465B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910578189.5A CN112153465B (en) 2019-06-28 2019-06-28 Image loading method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910578189.5A CN112153465B (en) 2019-06-28 2019-06-28 Image loading method and device

Publications (2)

Publication Number Publication Date
CN112153465A true CN112153465A (en) 2020-12-29
CN112153465B CN112153465B (en) 2024-01-16

Family

ID=73891143

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910578189.5A Active CN112153465B (en) 2019-06-28 2019-06-28 Image loading method and device

Country Status (1)

Country Link
CN (1) CN112153465B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111142751A (en) * 2019-12-26 2020-05-12 腾讯科技(深圳)有限公司 Image processing method and device, intelligent terminal and storage medium
CN113055742A (en) * 2021-03-05 2021-06-29 Oppo广东移动通信有限公司 Video display method, device, terminal and storage medium
CN118035585A (en) * 2024-04-11 2024-05-14 深圳麦风科技有限公司 Webpage resource loading method and device, terminal equipment and storage medium

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101594203A (en) * 2008-05-30 2009-12-02 索尼株式会社 Dispensing device, sending method and receiving system
CN102047287A (en) * 2008-06-17 2011-05-04 株式会社Ntt都科摩 Image/video quality enhancement and super-resolution using sparse transformations
CN104144313A (en) * 2013-05-10 2014-11-12 中国电信股份有限公司 Video communication method, video communication system and cloud image server
CN204903013U (en) * 2015-06-19 2015-12-23 杭州电子科技大学 Supplementary cell -phone carries out external device of infrared thermal imaging
CN107105338A (en) * 2017-03-17 2017-08-29 深圳市先河系统技术有限公司 The method and terminal asked for display image, video and response to network
CN107784686A (en) * 2016-08-25 2018-03-09 上海交通大学 Various visual angles media displaying method based on multiresolution
CN107959668A (en) * 2016-10-14 2018-04-24 南宁富桂精密工业有限公司 Streaming media optimization method and buffer storage
WO2018133597A1 (en) * 2017-01-19 2018-07-26 优视科技有限公司 Method and system for loading web page images, electronic device and readable storage medium
US20180293707A1 (en) * 2017-04-10 2018-10-11 Samsung Electronics Co., Ltd. System and method for deep learning image super resolution

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101594203A (en) * 2008-05-30 2009-12-02 索尼株式会社 Dispensing device, sending method and receiving system
CN102047287A (en) * 2008-06-17 2011-05-04 株式会社Ntt都科摩 Image/video quality enhancement and super-resolution using sparse transformations
CN104144313A (en) * 2013-05-10 2014-11-12 中国电信股份有限公司 Video communication method, video communication system and cloud image server
CN204903013U (en) * 2015-06-19 2015-12-23 杭州电子科技大学 Supplementary cell -phone carries out external device of infrared thermal imaging
CN107784686A (en) * 2016-08-25 2018-03-09 上海交通大学 Various visual angles media displaying method based on multiresolution
CN107959668A (en) * 2016-10-14 2018-04-24 南宁富桂精密工业有限公司 Streaming media optimization method and buffer storage
WO2018133597A1 (en) * 2017-01-19 2018-07-26 优视科技有限公司 Method and system for loading web page images, electronic device and readable storage medium
CN107105338A (en) * 2017-03-17 2017-08-29 深圳市先河系统技术有限公司 The method and terminal asked for display image, video and response to network
US20180293707A1 (en) * 2017-04-10 2018-10-11 Samsung Electronics Co., Ltd. System and method for deep learning image super resolution

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111142751A (en) * 2019-12-26 2020-05-12 腾讯科技(深圳)有限公司 Image processing method and device, intelligent terminal and storage medium
CN113055742A (en) * 2021-03-05 2021-06-29 Oppo广东移动通信有限公司 Video display method, device, terminal and storage medium
CN113055742B (en) * 2021-03-05 2023-06-09 Oppo广东移动通信有限公司 Video display method, device, terminal and storage medium
CN118035585A (en) * 2024-04-11 2024-05-14 深圳麦风科技有限公司 Webpage resource loading method and device, terminal equipment and storage medium

Also Published As

Publication number Publication date
CN112153465B (en) 2024-01-16

Similar Documents

Publication Publication Date Title
CN112153465B (en) Image loading method and device
CN111666497B (en) Application program loading method and device, electronic equipment and readable storage medium
US11301564B2 (en) Decision forest compilation
CN112114892B (en) Deep learning model acquisition method, loading method and selection method
CN108496198B (en) Image processing method and device
CN103914205A (en) File thumbnail display method and device for intelligent terminal
US20220058772A1 (en) Image Processing Method and Device
US20190114989A1 (en) Systems and methods for image optimization
CN105487640A (en) Image pre-loading and caching method of Android watch
CN110874298B (en) Request data storage method and terminal equipment
CN113127361A (en) Application program development method and device, electronic equipment and storage medium
CN107479868B (en) Interface loading method, device and equipment
CN104050001A (en) Resource processing method, device and equipment based on Android system
CN102377793B (en) Flash interaction method and device based on mobile communication equipment terminal
CN112882826B (en) Resource cooperative scheduling method and device
CN108549557A (en) Using storing method, terminal and storage medium
CN107580033B (en) Load information processing method and device, message processing method and device, and server
CN111158907A (en) Data processing method and device, electronic equipment and storage medium
CN114692053A (en) Lossless compression method for webpage picture, electronic equipment and storage medium
CN114218175A (en) Resource cross-platform sharing method and device, terminal equipment and storage medium
CN113849195A (en) Application program loading method and device, storage medium and terminal
CN108733401B (en) Method and device for realizing browser compatibility
CN115220803A (en) Data processing method and equipment
CN107643951B (en) Process processing method and device and electronic equipment
CN112991172A (en) Image processing method, image processing device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant