CN109804409A - The method and apparatus of image procossing - Google Patents

The method and apparatus of image procossing Download PDF

Info

Publication number
CN109804409A
CN109804409A CN201780062040.0A CN201780062040A CN109804409A CN 109804409 A CN109804409 A CN 109804409A CN 201780062040 A CN201780062040 A CN 201780062040A CN 109804409 A CN109804409 A CN 109804409A
Authority
CN
China
Prior art keywords
region
resolution
area
video image
target video
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201780062040.0A
Other languages
Chinese (zh)
Inventor
王鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Honor Device Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Publication of CN109804409A publication Critical patent/CN109804409A/en
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234345—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/4728—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region

Abstract

The embodiment of the invention discloses a kind of method and apparatus of image procossing.A kind of method of image procossing, comprising: handled with first area of the first resolution to the target video image, include the hot spot region of the target video image in the first area；It is handled with second area of the resolution ratio of gradual change to the target video image, the resolution ratio of the gradual change is less than the first resolution, wherein, the corresponding resolution ratio in the closer picture position in hot spot region described in distance is higher in the second area, the second area is located at outside the first area, and the second area and the first area constitute the target video image；By treated, target video image is sent to playback terminal.It, can be by more useful video data transmission to the playback terminal of user using the embodiment of the present invention.

Description

Image processing method and device Technical Field
The present invention relates to the field of image processing, and in particular, to a method and apparatus for image processing.
Background
The CDN is a Network constructed on the existing transmission Network, and by means of edge servers deployed in various places, through functional modules of load balancing, Content distribution, scheduling and the like of a central platform, a user can obtain required Content nearby, Network congestion is reduced, and the access response speed and hit rate of the user are improved. As shown in fig. 1, a network server uploads a video file to a file server, the file server distributes the video file to a plurality of content distribution networks, and finally, a user obtains or watches the video from the corresponding distribution network through a playing terminal.
Although the CDN approach solves the problem of video transmission and distribution, when the data size of a video is too large, and the transmission bandwidth is small or limited, the user may experience problems such as video jamming during the process of watching the video, and the user cannot obtain better watching experience.
Disclosure of Invention
The embodiment of the invention provides an image processing method and device, which can process video images of a target video under the condition of limited transmission bandwidth, reduce the data volume of the target video while providing better watching experience for a user, and transmit more useful video data to a playing terminal of the user.
A first aspect of an embodiment of the present invention provides an image processing method, including:
processing a first region of a target video image at a first resolution, wherein the first region comprises a hot spot region of the target video image;
processing a second area of the target video image with gradient resolution, wherein the gradient resolution is smaller than the first resolution, the higher the resolution corresponding to an image at a position closer to the hot spot area in the second area is, the second area is located outside the first area, and the second area and the first area constitute the target video image;
and sending the processed target video image to a playing terminal.
Specifically, the hot spot area of the target video image is a focus area of the sight line of the user, that is, a certain area that the user pays attention to or is interested in, and the hot spot area may be, for example, a ball in a ball game, specifically, a football, a volleyball, a basketball, or the like, or may be a singer in a concert, specifically, a face of the singer, or a participant in a conference, or the like.
In the embodiment of the invention, a target video image area can be divided into a first area and a second area according to a hot spot area, so that the hot spot area is in the first area, then the first area is processed with a first resolution, the second area is processed with a gradient resolution, and finally the processed target video image is sent to a playing terminal of a user, wherein the first resolution is greater than all resolutions of the second area, the resolution of the second area is reduced along with the increase of the resolution with the distance from the hot spot area, under the condition of the same bandwidth, the resolution of the hot spot area is higher and the resolutions of other areas are lower through the processing of the video image, and meanwhile, the data volume of the processed video image is smaller than the data volume of an original video image, so that useful video data, namely video data which the user wants to see, is transmitted to the playing terminal of the user, the watching experience of the user is guaranteed, and the bandwidth cost is saved.
In a first possible implementation manner of the first aspect, before the processing the first region of the target video image at the first resolution, the method further includes: determining a hot spot area of the target video image; and determining a first area and a second area of the target video image according to the hot spot area.
With reference to the first possible implementation manner of the first aspect, in a second possible implementation manner, the determining a first region of the target video image according to the hot spot region includes: determining a center of the hotspot region; and determining a region which takes the center as a circle center and a first preset distance as a radius as a first region of the target video image.
With reference to the first possible implementation manner of the first aspect, in a third possible implementation manner, the determining a first region of the target video image according to the hot spot region includes: and determining the minimum regular graph area containing the hot spot area as a first area of the target video image. Specifically, the minimum regular pattern area includes, but is not limited to, a circular area, a rectangular area, a diamond area.
In a fourth possible implementation manner of the first aspect, before the processing the first region of the target video image at the first resolution, the method further includes: determining a transmission bandwidth between the terminal and the playing terminal; determining the first resolution and the graduated resolution according to the region size of the first region, the region size of the second region, and the transmission bandwidth. Specifically, the transmission bandwidth with the playing terminal may be determined according to a target image acquisition request sent by a user through the playing terminal, for example, the target image acquisition request carries bandwidth data; the transmission bandwidth with the playing terminal can also be determined according to the access bandwidth of the playing terminal, wherein the access bandwidth of the terminal is the actual network bandwidth of the user; the transmission bandwidth with the playing terminal can also be determined according to the effective bandwidth which can be provided by the transmission network.
With reference to the fourth possible implementation manner of the first aspect, in a fifth possible implementation manner, the determining the resolution of the gradual change according to the area size of the first area, the area size of the second area, and the transmission bandwidth includes: and determining the resolution of the target image position in the second area according to the distance between the target image position in the second area and the center of the hot spot area.
With reference to the fifth possible implementation manner of the first aspect, in a sixth possible implementation manner, the determining, according to the distance between the target image position in the second area and the center of the hotspot area, the resolution of the target image position in the second area includes: acquiring the distance between the position of the target image in the second area and the center of the hot spot area; and determining the resolution of the target image position in the second area according to a gradient function, wherein the gradient function is a function representing the corresponding relation between the resolution and the distance. Specifically, the distance may be in units of centimeters, inches, pixels, and the like.
With reference to the fifth possible implementation manner of the first aspect, in a seventh possible implementation manner, the second area includes a third area and a fourth area, where a shortest distance between the third area and a center of the hot spot area is smaller than a shortest distance between the fourth area and the center of the hot spot area; the determining the resolution of the target image position in the second region according to the distance between the target image position in the second region and the center of the hotspot region comprises: acquiring the distance between the position of the target image in the third area and the center of the hot spot area; determining the resolution of the target image position in the third region according to a gradient function, wherein the gradient function is a function representing the corresponding relation between the resolution and the distance; determining a second resolution as a resolution of the target image position in the fourth region, wherein the second resolution is less than or equal to a minimum resolution of the target image position in the third region, and the second resolution is a fixed resolution.
With reference to the seventh possible implementation manner of the first aspect, in an eighth possible implementation manner, the second resolution is 1/N of the first resolution, and N is an integer greater than 1; or the second resolution is 1/P of the minimum resolution of the target image position in the third area, and P is an integer greater than or equal to 1.
In a ninth possible implementation manner of the first aspect, the gradual change function may be a gradual change function that satisfies a change rule of human vision, such as a parabolic function, an elliptic function, or a first order decreasing function.
With reference to the fifth possible implementation manner of the first aspect, in a tenth possible implementation manner, the second region includes K sub-regions, the gradually-changed resolution is a discretely-changed resolution, and the resolution corresponding to the sub-regions is a fixed resolution, where a shortest distance between a (K-1) th sub-region and a center of the hot-spot region is smaller than a shortest distance between the K-th sub-region and the center of the hot-spot region, and K is a positive integer greater than or equal to 2; the determining the resolution of the target image position in the second region according to the distance between the target image position in the second region and the center of the hotspot region comprises: acquiring the distance between the position of the target image in the second area and the center of the hot spot area; determining a sub-region where the target image position is located according to the distance; and determining the resolution of the position of the target image according to the corresponding relation between the resolution and the sub-region. Specifically, the shape of the sub-region may be a regular shape such as a circle, a rectangle, a regular triangle, or the like.
With reference to the tenth possible implementation manner of the first aspect, in an eleventh possible implementation manner, the correspondence between the resolution and the sub-region includes: the resolution corresponding to the 1 st sub-area of the target video image is 1/L of the first resolution or a difference value between the first resolution and a first preset value, wherein L is an integer greater than 1.
With reference to the eleventh possible implementation manner of the first aspect, in a twelfth possible implementation manner, the correspondence between the resolution and the sub-region further includes: the resolution corresponding to the Kth sub-area of the target video image is 1/M of the resolution corresponding to the (K-1) th sub-area, and M is an integer greater than 1; or the resolution corresponding to the Kth sub-area of the target video image is the difference value between the resolution corresponding to the (K-1) th sub-area of the target video image and a second preset value.
In a thirteenth possible implementation manner of the first aspect, the method further includes: determining a target bandwidth required by transmitting the target video image and a transmission bandwidth between the target video image and the playing terminal; the step of processing a first region of a target video image at a first resolution is performed if the target bandwidth is greater than the transmission bandwidth. Specifically, the target video image is an original target video image, that is, a target video image before processing.
In a fourteenth possible implementation manner of the first aspect, the gradual change of resolution includes a continuously changing resolution or a discretely changing resolution.
A second aspect of the embodiments of the present invention provides an image processing apparatus, including:
the first region processing module is used for processing a first region of the target video image at a first resolution, wherein the first region comprises a hot spot region of the target video image;
a second region processing module, configured to process a second region of the target video image with a gradient resolution, where the gradient resolution is smaller than the first resolution, where a resolution corresponding to an image at a position closer to the hot spot region in the second region is higher, the second region is located outside the first region, and the second region and the first region form the target video image;
and the sending module is used for sending the processed target video image to the playing terminal.
The apparatus for image processing provided in the second aspect of the embodiment of the present invention further includes other program modules for executing the method for image processing provided in the first aspect of the embodiment of the present invention, which are not described herein again.
A third aspect of embodiments of the present invention provides an image processing apparatus, which includes a processor, a memory, and a communication interface, where the processor, the memory, and the communication interface are connected to each other, where the communication interface is configured to receive and transmit data, the memory is configured to store application program codes for an apparatus supporting image processing to perform the foregoing methods, and the processor is configured to perform the various methods of the first aspect.
A fourth aspect of embodiments of the present invention provides a computer storage medium storing computer program instructions for an apparatus for image processing, the computer program instructions comprising instructions for executing a program according to the first aspect.
A fifth aspect of embodiments of the present invention provides a computer program for executing the methods provided in the first aspect.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments or the background art of the present invention, the drawings required to be used in the embodiments or the background art of the present invention will be described below.
Fig. 1 is a technical architecture diagram of content delivery using a CDN in the prior art;
fig. 2 is a schematic structural diagram of a system for performing live video broadcast according to an embodiment of the present invention;
FIG. 3 is a flow chart of a method for image processing according to an embodiment of the present invention;
fig. 4a is a target video image without distinguishing hot spot areas and non-hot spot areas provided by the embodiment of the present invention;
fig. 4b is a target video image after distinguishing the hot spot regions according to the embodiment of the present invention;
fig. 5a is a schematic diagram illustrating a relationship among a hot spot area, a first area, and a second area according to an embodiment of the present invention;
fig. 5b is a schematic diagram illustrating a relationship among a hot spot region, a first region, and a second region according to another embodiment of the present invention;
fig. 5c is a schematic diagram illustrating a relationship among a hot spot area, a first area, and a second area according to another embodiment of the present invention;
FIG. 6 is a layout diagram of a first region and a second region provided by an embodiment of the present invention;
FIG. 7a is a further layout of the first region and the second region provided by the embodiment of the present invention;
FIG. 7b is a further layout of the first region and the second region provided by the embodiment of the present invention;
FIG. 8a is a further layout of the first region and the second region provided by the embodiment of the present invention;
FIG. 8b is a further layout of the first region and the second region provided by the embodiment of the present invention;
FIG. 9a is a target video image before processing provided by an embodiment of the invention;
FIG. 9b is a processed target video image provided by an embodiment of the present invention;
FIG. 10 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention;
fig. 11 is a schematic structural diagram of another image processing apparatus according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention.
The scheme of the embodiment of the invention is suitable for a system structure for video acquisition, production and transmission, for example, as shown in fig. 2, a system structure schematic diagram for video live broadcast includes a video acquisition end 101, a video splicing server 102, a network server 103, a transmission and distribution network 104 and a playing terminal 105. In the system structure diagram shown in fig. 2, during a live video broadcast process, a video capture terminal 101 is configured to capture various initial video images, such as live video images, and send the captured video images to a video splicing server 102, where the video capture terminal 101 may be, for example, a camera, a video camera, or other devices; the video splicing server 102 is configured to perform editing processing such as clipping and encoding on a video image acquired by the video acquisition terminal 101 to generate a target video, and send the target video to the network server 103; the network server 103 is a background server of the video website, which may also be called a cloud server, and the network server 103 distributes the target video to the transmission and distribution network 104; the transmission and distribution network 104 includes a plurality of cache servers/streaming media servers, the transmission and distribution network 104 distributes the target video to each playing terminal 105, the transmission and distribution network 104 may be, for example, a CDN, and the playing terminal may be, for example, a mobile phone, a Virtual Reality (VR) device, a tablet computer, and other terminals that can be used for playing the video.
Specifically, the video may be a VR video or a normal video.
The system structure shown in fig. 2 can transmit an original target video to a playing terminal of a user, and based on the system structure shown in fig. 2, the method for processing an image according to the embodiment of the present invention can be implemented, wherein the step of processing the original target video image can be performed by a video splicing server, where the original target video image can be any one frame video image of the target video, the step specifically includes processing a first region of the target video image at a first resolution, processing a second region of the target video image at a gradual resolution, or performing the step of processing the target video image by a cache server/streaming server in a transmission and distribution network, or adding an image processing node in the system structure to perform the step of processing the target video image, for example, an image processing server is added in the transmission and distribution network to execute the step of processing the target video image; and then, the transmission and distribution network sends the processed target video image to a playing terminal of a user, so that the data volume of each frame of video image in the target video is reduced, the transmission bandwidth is saved, and useful video data are transmitted to the playing terminal of the user.
It should be noted that the method for processing an image provided by the embodiment of the present invention can also be executed by two or more servers in the above system structure, which is not listed here.
In a possible implementation manner, the system structure shown in fig. 2 may also be a schematic diagram of a system structure for performing video on demand, where the video splicing server 102 sends the target video to the network server 103, after the network server 103 distributes the target video to the multiple transmission distribution networks 104, the target video may be cached or stored in the cache server/streaming server within a period of time, when a user needs to watch the target video, a target video acquisition request is initiated to the network server through the playing terminal, then the network server returns the address of the cache server/streaming server closest to the playing terminal, the playing terminal accesses the cache server/streaming server closest to the playing terminal according to the address, and then can watch the target video, the step of processing the target video image may be executed by a cache server/streaming server in the transmission and distribution network, or the step of processing the target video image may be executed by adding an image processing server in the transmission and distribution network.
It should be understood that the method for processing images provided by the embodiment of the present invention is applicable to other system configurations related to video image transmission besides the system configuration shown in fig. 2.
Referring to fig. 3, fig. 3 is a schematic flowchart of an image processing method according to an embodiment of the present invention, where the method of the present invention can be implemented on the aforementioned server, where the server specifically includes at least one of a video splicing server, a cache server/streaming server, and an image processing server, and the method includes, but is not limited to, the following steps:
step S201: processing a first region of a target video image at a first resolution, wherein the first region comprises a hot spot region of the target video image.
The hot spot area is a focus area of the sight of the user, that is, the hot spot area is an area that is more concerned or interested by the user than other areas of the target video image, and the user wants to see the area more clearly.
Optionally, before step S201 is executed, a hot spot region of the target video image may be further determined, and the first region and the second region of the target video image may be determined according to the hot spot region.
Specifically, for example, if the target video image is a frame of video image of a live video of a football match, an area where the football is located may be determined as a hot spot area or a face area of a certain player may be determined as a hot spot area; if the target video image is a frame of video image of a live video of a concert, the region where the face of the singer is located can be determined as a hot spot region or the region where the body of the singer is located can be determined as a hot spot region; if the target video image is a large conference, specifically, if the target video image is a frame of video image of a national conference, the region where the face or the body of the speaker is located in the conference may be determined as a hot spot region.
For example, referring to fig. 4a and 4b, fig. 4a is a target video image without distinguishing a hot spot region and a non-hot spot region, and fig. 4b is a target video image after distinguishing a hot spot region, where fig. 4a is a frame of video image of a basketball game video, and there are image frames of a player, a court, a basketball, etc. in the image, an image frame enclosed by a white region in fig. 4b, that is, an image frame corresponding to the basketball is the hot spot region, and an image frame enclosed by a black region, that is, an image frame corresponding to a region outside the basketball is the non-hot spot region.
Optionally, the hot spot region of the target video image may be determined according to an image-based target identification method, where the image-based target identification method may be a class feature-based target detection method, and may also be a texture and boundary feature-based target detection method.
Alternatively, when a target detection method based on class features is adopted, the class-Haar features, which are simple rectangular features that can reflect the gray level change of the local features of the detection object, are known as being similar to Haar wavelets (English: Haar wavelet). For example, the method comprises the steps of extracting Haar-like features of a region to be identified of a target video image, obtaining feature values of the region to be identified, and classifying the feature values through an AdaBoost (English: Adaptive Boosting) algorithm, so as to determine the hotspot region.
Optionally, when the target detection method based on the texture and the boundary features is adopted, the features of the pixel points can be extracted by using Harris operators, Forstner operators, Moravec operators and other point feature extraction operators, and then the features of different categories are formed through clustering operation.
In an alternative manner, when the first area and the second area are determined according to the hot spot area, the center of the first area may coincide with the center of the hot spot area, so that the hot spot area is located in the middle of the first area.
In one possible implementation, in the case where the shape of the hot spot region is irregular, the first region and the second region may be determined in at least two ways: 1) the center of the hot spot region can be determined, and a region with the center as a circle center and a first preset distance as a radius is determined as a first region of the target video image, wherein the radius of the first preset distance is larger than the maximum distance between the center of the hot spot region and the edge of the hot spot region, and correspondingly, the region outside the first region is a second region; 2) the minimum regular graph area including the hot spot area may be determined as the first area of the target video image, where the minimum regular graph may be a circular, rectangular, triangular, or rhombic area, and accordingly, the area outside the first area is the second area.
Specifically, the unit of the first preset distance may be an image distance unit such as centimeter, inch, pixel, etc., and the first preset distance may be set according to the length and width of the total area of the target video image, for example, 1/2, 1/3, etc. of the length of the total area; it may also be set according to the longest distance between the center of the hot spot region and the edge of the hot spot region, for example, the sum of the longest distance and a preset threshold, specifically, the preset threshold is, for example, a distance of 1 unit.
Optionally, the size of the first area may be fixed, or may be set by the user, for example, if the user sets the first preset distance to 2 centimeters, the first area is determined to be a circular area with the center of the hot spot area as a center and with 2 centimeters as a radius, and the area size of the first area is 4 pi square centimeters.
In another possible implementation manner, when the shape of the hot spot region is a regular pattern, in addition to determining the first region and the second region by using the two manners, the hot spot region may be directly determined as the first region, and accordingly, the region outside the first region is the second region.
The following description will be given by way of example of how to determine the first region and the second region according to the hot spot region, referring to fig. 5a-5c, assuming that the target video image with the hot spot region determined is as shown in fig. 4 b. Fig. 5a is a schematic diagram illustrating a relationship between a hot spot area, a first area, and a second area, where in fig. 5a, the first area is a circular area with a center of the hot spot area as a center and a first preset distance as a radius, the hot spot area is within the first area, and the area outside the first area is the second area; fig. 5b is a schematic diagram illustrating a relationship between a hot spot region, a first region and a second region according to another embodiment of the present invention, in fig. 5b, the first region is a minimum square region including the hot spot region, a center of the first region coincides with a center of the hot spot region, and a region outside the first region is the second region; fig. 5c is a schematic diagram illustrating a relationship among a hot spot area, a first area, and a second area according to another embodiment of the present invention; in fig. 5c, the first area coincides with the hot spot area, and the area outside the hot spot area is the second area.
Optionally, before performing step S201, a transmission bandwidth with the playing terminal and a target bandwidth required for transmitting the target video image may also be determined, and in case that the target bandwidth is greater than the transmission bandwidth, step S201 is performed; and under the condition that the target bandwidth is smaller than the transmission bandwidth, the unprocessed/original target video image is directly sent to the playing terminal without processing the target video image.
Alternatively, the transmission bandwidth may be determined in at least several ways:
1) under the condition that the bandwidth resources of the server are not limited, determining the access bandwidth of the playing terminal as a transmission bandwidth, namely determining the actual bandwidth of a user as the transmission bandwidth;
2) under the condition that the bandwidth resources of the server are not limited, receiving a bandwidth indication sent by a user through a playing terminal, and determining a transmission bandwidth according to the bandwidth indication, namely determining the bandwidth specified by the user as the transmission bandwidth; for example, assuming that the current access bandwidth of the user is 20M, but the user does not want all the bandwidth to be used for transmitting the target video image, and the user only wants to use the bandwidth of 10M for transmitting the target video image, the user sets the playback bandwidth to 10M on the available playback terminal, the playback terminal sets the bandwidth indication to 10M, and the server determines 10M as the transmission bandwidth after receiving the bandwidth indication.
3) Under the condition that the bandwidth resource of the server is limited, determining the current residual idle bandwidth of the server according to the bandwidth resource allocation condition of the server, and then determining part of the idle bandwidth as the transmission bandwidth between the idle bandwidth and the playing terminal according to a certain bandwidth allocation rule, wherein the transmission bandwidth is less than or equal to the access bandwidth of the playing terminal under the condition.
Step S202: processing a second area of the target video image with gradient resolution, wherein the gradient resolution is smaller than the first resolution, the higher the resolution corresponding to an image at a position closer to the hot spot area in the second area is, the second area is located outside the first area, and the second area and the first area constitute the target video image.
Specifically, the first resolution and the resolution of the gradation may be determined according to a transmission bandwidth with the play terminal.
In a possible implementation manner, the first resolution may be determined according to the transmission bandwidth, the area size of the first area, and the area size of the second area, and the resolution of the target image position may be determined according to a correspondence between the distance between the target image position in the second area and the center of the hotspot area and the resolution.
Specifically, the sizes of the first region and the second region may be measured by areas, or may be measured by the total amount of pixels in the regions.
Wherein, the resolution in the second area satisfies the conditional relationship that the closer the image position is to the hot spot area, the higher the corresponding resolution is, and then at least the following resolution change rules can be set for the second area: 1) the resolution of the second area continuously changes, and a function corresponding relation exists between the resolution and a distance, wherein the distance refers to the distance between the target image position of the second area and the center of the hot spot area, and the larger the distance is, the smaller the function value is; 2) the resolution of the second area is discretely changed, wherein the second area is divided into a plurality of sub-areas, the resolution in the sub-areas is fixed, different sub-areas correspond to different resolutions, and the farther the sub-area is from the center of the hot spot area, the lower the resolution is; 3) the resolution of the second region changes continuously in a part of the second region and discretely in another part of the second region.
Specifically, the gradual change function between the resolution and the distance may be a parabolic function, an elliptic function, a first order decreasing function, or the like, which conforms to the function relationship of the visual change of human eyes.
Several variation rules of the resolution of the fade in the second area are specifically described below, while the method of determining the first resolution and the resolution of the fade is described in connection with the variation rules of the resolution of the second area.
It should be noted that the first resolution and the gradient resolution determined in the following scenarios are both resolutions per unit area, and if the total resolution of a certain region of the target video image needs to be determined, the total resolution of the certain region can be obtained by multiplying the resolution per unit area by the corresponding region area.
It should be noted that, in several scenarios described below, the center of the first area is the center of the hot spot area, and in an alternative, the center of the first area may not coincide with the center of the hot spot area.
The implementation scenario one: the resolution of the second region varies continuously, wherein a functional correspondence exists between the resolution and the distance.
In this implementation scenario, the distance may be used as an independent variable of the gradual change function, the resolution may be used as a dependent variable of the gradual change function, and the dependent variable and the independent variable satisfy a decreasing correspondence relationship, that is, the dependent variable decreases with an increase of the independent variable, and then the decreasing function may be used as the gradual change function between the resolution and the distance.
Specifically, the size of the first region may be determined, the maximum radius of the second region may be determined, where the maximum radius refers to a maximum distance between a center of the hot spot region and an edge of the second region, then a resolution corresponding to a transmission bandwidth is determined according to the transmission bandwidth, the first resolution is determined according to a principle that a total amount of pixels is consistent and a gradual change function, and resolutions at each distance are determined according to gradual change functions of the distances and the resolutions, where the principle that the total amount of pixels is consistent refers to a target video image with the same size of the region, and a total amount of pixels included in the target video image before the resolution change is equal to a total amount of pixels included in the target video image after the resolution change.
The specific process is described below by taking the function as a first decreasing function.
For example, the distribution of the first region and the second region is as shown in fig. 6, where fig. 6 is a distribution of the first region and the second region, the shape of the first region is a circle, the radius is d1, the first resolution is R1, the second region is a gray region outside the first region, the resolution function of the second region is R2 ═ R1- (x-d1), and the resolution corresponding to the transmission bandwidth is R0, it should be noted that the resolution R0 in the embodiment of the present invention refers to the resolution corresponding to the transmission bandwidth, that is, the transmission bandwidth can support the resolution of the transmitted target video image on a unit area under the condition of no pause in playing, and meanwhile, under the condition that the resolution is not explicitly stated, R1, R0, and R2 refer to the resolution on a unit area.
First, the second area is placed in a circular ring formed by two concentric circles using the center of the hot spot area as the center of a circle, as shown in fig. 6, in this case, the radius of the circle 1 is the minimum distance between the center of the hot spot area and the boundary of the second area, the circle 1 coincides with the first area, the radius of the circle 1 is d1, the circle 2 is the minimum circle containing the second area, the radius d2 of the circle 2 is the maximum distance between the center of the hot spot area and the boundary of the second area, and according to the principle that the total amount of pixels is not changed, in this case, the total amount of pixels is the sum of the total amount of pixels of the circle 1 and the total amount of pixels of the circular ring is equal to the total amount of pixels of the circle 2, so as to obtain the formula:
wherein (pi d1)2) R1 is the total number of pixels of circle 1, R0 (pi d 2)2) The total number of pixels of circle 2.
Then, the reverse is carried out to obtain R1, R1 ═ R0 × d22+1/3d13-2/3d23-d22d1)/d22
According to the above formula, the first resolution R1 can be determined from R0, d1 and d2, and the resolution function of the second region is R2 ═ (R0 × d 2) from R2 ═ R1- (x-d1)2+1/3d13-2/3d23-d22d1)/d22X + d1, acquiring the distance x between the target position image on the second area and the center of the hot spot area, and bringing the value of x into the resolution which can determine the target position image.
Specifically, d1, d2 may be obtained after determining the first region and the second region.
Specifically, R0 can be obtained according to the transmission bandwidth, and assuming that the area of the target video image is 5 inches and the resolution corresponding to the transmission bandwidth is 1080P, where 1080P includes 1280 × 720 pixels, then R0 is 1280 × 720/5, and the unit of R0 is pixel/inch.
It should be understood that the above examples are only used for explaining the embodiments of the present invention, and should not be construed as limiting, and the resolution function of the second region may also be another function, where when the resolution function is a linear function (R2 ═ kx + b, k and b are constants), the value of k is not limited to 1, and the shape of the first region may also be another shape.
Implementation scenario two: the resolution of the second area is discretely changed, the second area has K sub-areas, wherein the shortest distance between the (K-1) th sub-area and the center of the hot spot area is smaller than the shortest distance between the K th sub-area and the center of the hot spot area, namely, the K th sub-area is outside the (K-1) th sub-area or surrounds the (K-1) th sub-area in visual.
In this embodiment scenario, in a possible implementation manner, in order to determine the resolution of each sub-region, the resolutions of each sub-region may be associated, so that the resolution between each sub-region is changed regularly, for example, the resolution of the kth sub-region is 1/M of the (K-1) th sub-region, where M is an integer greater than 1, and for example, the resolution of the kth sub-region is obtained by subtracting a second preset value from the resolution of the (K-1) th sub-region; then, the resolution of the sub-region closest to the center of the hot spot region in the K sub-regions, that is, the resolution of the 1 st sub-region is determined.
In an alternative scheme, the resolution of the 1 st sub-area is 1/L of the first resolution or the first resolution minus a first preset value, wherein L is an integer greater than 1.
Optionally, L and M may be set equal or unequal; the first preset value and the second preset value can be set to be equal or unequal.
In another possible implementation manner, the resolutions of the sub-regions may not be associated, that is, the resolutions of the sub-regions are irregularly changed, at this time, the sub-regions may be respectively associated with the first resolution, for example, the resolution of the 1 st sub-region is obtained by subtracting a first preset value from the first resolution, the resolution of the 2 nd sub-region is obtained by subtracting a second preset value from the second resolution, and the resolution of the 3 rd sub-region is 1/L … K th sub-region of the first resolution is 1/M of the first resolution, which is only taken as an example, and in addition, a condition that "the farther the hot spot is from the hot spot region, the lower the corresponding resolution is" is also required to be satisfied between the sub-regions.
Alternatively, the first resolution may be replaced with a predetermined fixed resolution smaller than the first resolution.
The resolution relationship between the two sub-regions and the resolution of each region, i.e. each sub-region of the first region and the second region, are specifically described below.
For example, the distribution of the first region and the second region is shown in FIG. 7a, and FIG. 7a is another distribution of the first region and the second region, wherein the first region has a circular shape and a radius d 1; the first resolution is R1, the second area is a gray area outside the first area, the second area has 3 sub-areas, the distance between the 1 st sub-area and the first area is the nearest and is a circular ring, the radius of the inner ring is d1, and the radius of the outer ring is d 2; the 2 nd sub-area is adjacent to the 1 st sub-area and is also a circular ring, the radius of the inner ring is d2, and the radius of the outer ring is d 3; the 3 rd sub-area is outside the 2 nd sub-area, the area of the target video image is S, and the resolution corresponding to the transmission bandwidth is R0.
Under the condition that the resolution ratio of the 1 st subregion is 1/L of the first resolution ratio, and the resolution ratio of the K-th subregion is 1/M of the resolution ratio of the (K-1) th subregion, assuming that L and M both take on the value of 2, according to the principle that the total amount of pixels is unchanged, under the condition, the principle that the total amount of pixels is the sum of the total amounts of pixels of the first region and the second region and is equal to the total amount of pixels of the whole video image, and the sum of the total amounts of pixels of each subregion is equal to the total amount of pixels of the second region, the formula is obtained:
(πd12)*R1+1/2(πd22-πd12)*R1+1/4(πd32-πd22)*R1+1/8(S-πd32)*R1＝R0*S
wherein (pi d1)2) R1 is the total number of pixels in the first region, 1/2 (pi d 2)2-πd12) R1 is the total number of pixels in sub-region 1, 1/4 (pi d 2)2-πd12) R1 is the total number of pixels in sub-region 2, 1/8 (S-pi d 3)2) R1 is the total number of pixels in the 3 rd sub-area, and R0S is the total number of pixels in the entire target video image.
Then, the reverse is carried out to obtain R1, R1 ═ R0S)/(1/2 π d12+1/4πd22+1/8πd32+1/8S)
According to the formula, the first resolution R1 can be determined according to R0, d1, d2, d3 and S, and then the resolution of each sub-region can be determined according to the relation between each sub-region and R1, wherein the resolution of the 1 st sub-region is 1/2R1, the resolution of the 2 nd sub-region is 1/4R1, and the resolution of the 3 rd sub-region is 1/8R 1.
Specifically, d1, d2, d3 may be default distances or may be set by a user, and d1, d2, and d3 may change regularly, for example, d1 is 1/2d2, d2 is 1/2d3, and d3 is half of the width of the target video image.
Under the condition that the resolution of the 1 st sub-area is the first resolution minus the first preset value, and the resolution of the K sub-area is the resolution of the (K-1) th sub-area minus the second preset value, assuming that the first preset value and the second preset value are r, according to the principle that the total pixel amount is unchanged, under the condition, the principle that the total pixel amount is unchanged is that the sum of the total pixel amounts of the first area and the second area is equal to the total pixel amount of the whole video image, and the sum of the total pixel amounts of each sub-area is equal to the total pixel amount of the second area, obtaining a formula:
(πd12)*R1+(πd22-πd12)*(R1-r)+(πd32-πd22)*(R1-2r)+(S-πd32)*(R1-3r)＝R0*S
wherein (pi d1)2) R1 is the total number of pixels in the first region (pi d 2)2-πd12) (R1-R) is the total number of pixels in the sub-region 1 (pi d 3)2-πd22) (R1-2R) is the total number of pixels in the sub-region 2, (S-pi d 3)2) And R1-3R is the total pixel amount of the 3 rd sub-area, and R0S is the total pixel amount of the whole target video image.
Then, the reverse is carried out to obtain R1, R1 ═ (R0 × S +3R × S- π d12*r-πd22*r-πd32*r)/S。
According to the formula, the first resolution R1 can be determined according to R0, d1, d2, d3, S and R, and then the resolution of each sub-region can be determined according to the relation between each sub-region and R1, wherein the resolution of the 1 st sub-region is R1-R, the resolution of the 2 nd sub-region is R1-2R, and the resolution of the 3 rd sub-region is R1-3R.
For another example, the distribution of the first region and the second region is shown in fig. 7b, where fig. 7b is another distribution of the first region and the second region, where the first region has a rectangular shape and an area of S1, the first resolution is R1, the second region is a gray region outside the first region, the second region has 3 sub-regions, the 1 st sub-region is closest to the second region and has a rectangular shape and an area of S2, the 2 nd sub-region is adjacent to the 1 st sub-region, the second region also has a rectangular shape and an area of S3, the 3 rd sub-region is outside the 2 nd sub-region, the area of the target video image is S, and the resolution corresponding to the transmission bandwidth is R0.
Assuming that the resolution of the 1 st sub-region is 1/2 of the first resolution, the resolution of the 2 nd sub-region is 1/3 of the first resolution, and the resolution of the 3 rd sub-region is 1/4 of the first resolution, according to the principle that the total amount of pixels is not changed, under this condition, the principle that the total amount of pixels is not changed is that the sum of the total amounts of pixels of the first region and the second region is equal to the total amount of pixels of the whole video image, and the sum of the total amounts of pixels of each sub-region is equal to the total amount of pixels of the second region, a formula is obtained:
S1*R1+S2*1/2R1+S3*1/3R1+(S-S3-S2-S1)*1/4R1＝R0*S。
wherein S1 × R1 is the total number of pixels in the first region, S2 × 1/2R1 is the total number of pixels in the 1 st sub-region, S3 × 1/3R1 is the total number of pixels in the 2 nd sub-region, (S-S3-S2-S1) × 1/4R1 is the total number of pixels in the 3 rd sub-region, and R0 × S is the total number of pixels in the entire target video image.
Then backward thrust is given to R1, R1 ═ (R0 × S)/(3/4S1+1/4S2+1/12S3+1/4S)
According to the above formula, the first resolution R1, and further the resolution 1/2R1 of the 1 st sub-region, the resolution 1/3R1 of the 2 nd sub-region, and the resolution 1/4R1 of the 3 rd sub-region can be determined according to R0, S1, S2, S3, S4, and S.
Specifically, S1, S2, S3 and S4 may be default settings or may be set by a user, and are not discussed here.
It should be understood that the above examples are only used for explaining the embodiments of the present invention, and should not be construed as limiting, the second region may further include a plurality of sub-regions, for example, the second region includes 4 sub-regions, 5 sub-regions, and the like, the shape of each sub-region is not limited to the above circular shape and rectangular shape, and the resolution of each sub-region may also be varied in other ways.
In this implementation scenario, a distance between a target image position in the second area china and a center of the hot spot area may be first obtained, a sub-area where the target image position is located is determined according to the distance, and then a resolution of the target image position is determined according to a corresponding relationship between the resolution and the sub-area, for example, it is determined that the target image position is located in the 1 st sub-area, and the resolution of the 1 st sub-area is the resolution of the target image position.
The implementation scene three: the resolution of the second region changes continuously in a partial region of the target video image and changes discretely in another partial region of the target video image.
In this implementation scenario, for the continuously changing area, the resolution of each target image position may be determined by referring to the scheme in the first implementation scenario, and for the discretely changing area, the resolution of each sub-area may be determined by referring to the scheme in the second implementation scenario, so as to determine the resolution of the target image position, specifically, in which sub-area the target image position is located, the resolution of the target image position is the resolution of the current sub-area.
Referring to fig. 8a, by way of example, fig. 8a is a diagram illustrating another distribution of a first region and a second region, wherein the first region has a circular shape and a radius d1, the first resolution is R1, the second region is a gray region outside the first region, the second region has two regions, the second region includes a third region and a fourth region, the third region is a circular ring, the inner ring has a radius d1, the outer ring has a radius d2, the resolution of the third region changes continuously, the resolution function of the third region is R2 ═ R1- (x-d1), the fourth region is outside the third region, the fourth region includes two sub-regions, i.e., a region and B region, the a region is adjacent to the third region, the inner ring radius d2 of the a region, the outer ring has a radius d3, the B region is outside the a region, and the resolution of the a region is 1/2 which is the minimum resolution of the third region, the resolution of the B area is 1/3 of the minimum resolution of the third area, the area of the target video image is S, and the resolution corresponding to the transmission bandwidth is R0.
According to the principle that the total amount of the pixels is not changed, under the condition, the principle that the total amount of the pixels is not changed is that the sum of the total amounts of the pixels in the first region, the third region and the fourth region is equal to the total amount of the pixels in the whole video image, and the sum of the total amounts of the pixels in the region A and the region B is equal to the total amount of the pixels in the fourth region, so that a formula is obtained:
wherein (pi d1)2) R1 is the total number of pixels in the first area, and is the total number of pixels in the third area (pi d 3)2-πd22) 1/2(R1-d2+ d1) is the total number of pixels in the A region, (S-pi d 3)2) 1/3(R1-d2+ d1) is the total number of pixels in the B region, and R0 is the total number of pixels in the entire target video image.
Then, the reverse is carried out to obtain R1, R1 ═ R0S +1/6 π d23+1/3πd13-1/2πd22d1-(1/6πd32+1/3S)(d1-d2))/(1/2πd22+1/6πd33+1/3S-πd12)
According to the above formula, the first resolution R1, and thus the resolution function of the third region, the resolution of the a region, and the resolution of the B region can be determined according to R0, d1, d2, d3, and S.
In an alternative embodiment, the distribution diagram of fig. 8B can be further modified as shown in fig. 8B, where fig. 8B is a further distribution diagram of the first region and the second region, where the first region has a circular shape and a radius d1, the first resolution is R1, the second region is a gray region outside the first region, and the second region has two regions, and the second region includes a fifth region and a sixth region, where the fifth region includes two sub-regions, respectively, a region a and a region B, the inner ring radius of the region a is d1, the outer ring radius is d2, the inner ring radius of the region B is d2, the outer ring radius is d3, the resolution of the fifth region varies discretely, the resolution of the region a is 1/2, and the resolution of the region B is 1/3, which is the minimum resolution of the fifth region; the sixth area is outside the fifth area, the resolution function of the sixth area is R2 ═ 1/3 × R1- (x-d1), the area of the target video image is S, and the resolution corresponding to the transmission bandwidth is R0.
The way of calculating R1 can refer to the above mentioned scheme, and R2 can be further obtained after R1 is obtained by calculation, which is not described herein again.
It should be understood that the above examples are only used for explaining the embodiments of the present invention, and should not be construed as limiting, the resolution function of the second region may be other functions, the shape of the first region may be other shapes, and the second region may be divided in other ways.
It should be noted that the above-described several implementation scenarios are only partial embodiments of the resolution change rule of the second region, and in the alternative, there may be other resolution change rules, for example, sub-regions in the third implementation scenario may be further refined to make the resolution change continuously or discretely, and are not listed here.
According to the scheme, the corresponding resolution ratio can be determined through the transmission bandwidth, the corresponding relation between the first resolution ratio and the gradual-change resolution ratio and the distance can be obtained according to the principle that the total amount of the pixels is unchanged, and then the resolution ratio of the image position in the second area at each distance is determined.
After the resolution ratios at all the distances are determined, the distance between the target image position and the center of the hot spot area can be obtained, the target resolution ratio at the distance is determined, and the image picture of the target image position is processed according to the target resolution ratio.
Step S203: and sending the processed target video image to a playing terminal.
Specifically, the processed target video image is visually different from the original target video image, the image frames of the respective areas of the original target video image have the same definition, the definition of the image frame of the first area of the processed target video image is higher than that of the image frame of the second area, and the image frame of the second area farther from the first area has the lower definition.
Specifically, according to different resolution change rules adopted in the processing process, the definition change of the target video image is different, the target video image is also different in visual presentation, and the image picture closer to the center of the hot spot area has higher definition.
The following describes an example of a target video image before processing and a target video image after processing.
See fig. 9 a-9 b. Fig. 9a shows the target video image before processing, and the sharpness of the image screen at each position of the target video image before processing is the same. Fig. 9b is the processed target video image, and if the adopted resolution change rule corresponds to the implementation scene two in the step S203, for example, the processed target video image has 4 layers, where the image picture of the innermost layer, i.e., the first region, is clearest, the image picture of the second layer, i.e., the 1 st sub-region, is less sharp than the image picture of the innermost layer, the image picture of the third layer, i.e., the 2 nd sub-region, is less sharp than the image picture of the second layer, and the image picture of the outermost layer, i.e., the 3 rd sub-region, is less sharp than the image picture of the third layer.
Specifically, for the target video, under the condition of limited transmission bandwidth, the above processing can be performed on each frame of the target video image, so that the image frame concerned or interested by the user can be ensured to be clear enough and meet the resolution requirement of the transmission bandwidth.
In the method described in fig. 3, the target video image area is divided into a first area and a second area according to the hot spot area of the target video image, so that the hot spot area is in the first area, then the first area is processed with a first resolution, the second area is processed with a gradient resolution, and finally the processed target video image is sent to the user's playing terminal, where the first resolution is greater than all resolutions of the second area, the resolution of the second area decreases as the distance from the hot spot area increases, the resolution of the hot spot area is higher and the resolution of the other areas is lower by processing the video image under the same transmission bandwidth, and the data volume of the processed video image is smaller than the data volume of the original video image, so that useful video data, i.e. video data desired by the user, can be transmitted to the user's playing terminal, the watching experience of the user is guaranteed, and the bandwidth cost is saved.
The method of embodiments of the present invention is set forth above in detail and the apparatus of embodiments of the present invention is provided below.
Referring to fig. 10, fig. 10 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention, the apparatus at least includes a first area processing module 310, a second area processing module 320, and a sending module 330, wherein each module is described in detail as follows:
a first region processing module 310, configured to process a first region of a target video image at a first resolution, where the first region includes a hot spot region of the target video image;
a second region processing module 320, configured to process a second region of the target video image with a gradient resolution, where the gradient resolution is smaller than the first resolution, where a resolution corresponding to an image position closer to the hot spot region in the second region is higher, the second region is located outside the first region, and the second region and the first region form the target video image;
a sending module 330, configured to send the processed target video image to a playing terminal.
Optionally, the apparatus further comprises:
the hot spot region determining module 340 is configured to determine a hot spot region of the target video image;
and an area dividing module 350, configured to determine the first area and the second area of the target video image according to the hot spot area.
Optionally, the region dividing module 350 includes:
a center determining submodule 351, configured to determine a center of the hot spot region;
the first region determining sub-module 352 is configured to determine a region with the center as a center and a first preset distance as a radius as the first region of the target video image.
Optionally, the region dividing module 350 is specifically configured to:
and determining the minimum regular graph area containing the hot spot area as a first area of the target video image.
Optionally, the apparatus further comprises:
a transmission bandwidth determining module 360, configured to determine a transmission bandwidth with the playback terminal;
a resolution determination module 370, configured to determine the first resolution and the gradient resolution according to the area size of the first area, the area size of the second area, and the transmission bandwidth.
Optionally, the resolution determination module 370 includes:
a first resolution determination sub-module 370 for determining the first resolution based on the region size of the first region, the region size of the second region, and the transmission bandwidth;
the gradient resolution determining submodule 371 is configured to determine the resolution of the target image position in the second area according to the distance between the target image position in the second area and the center of the hot spot area.
Optionally, the fade resolution determination sub-module 371 includes:
a first distance obtaining unit 3711, configured to obtain a distance between a target image position in the second area and a center of the hotspot area;
a first resolution determining unit 3712, configured to determine the resolution of the target image location in the second region according to a gradient function, where the gradient function is a function representing a correspondence between the resolution and the distance.
Optionally, the second area includes a third area and a fourth area, wherein a shortest distance between the third area and the center of the hot spot area is smaller than a shortest distance between the fourth area and the center of the hot spot area; the fade resolution determination sub-module 371 includes:
a second distance obtaining unit 3713, configured to obtain a distance between the target image position in the third area and the center of the hot spot area;
a second resolution determining unit 3714, configured to determine a resolution of the target image position in the third region according to a gradient function, where the gradient function is a function representing a correspondence between the resolution and the distance;
a third resolution determining unit 3715, configured to determine the second resolution as the resolution of the target image position in the fourth region, where the second resolution is less than or equal to the minimum resolution of the target image position in the third region, and the second resolution is a fixed resolution.
Optionally, the second resolution is 1/N of the first resolution, and N is an integer greater than 1; or the second resolution is 1/P of the minimum resolution of the target image position in the third area, and P is an integer greater than or equal to 1.
Optionally, the tapering function comprises: parabolic functions, elliptic functions, decreasing functions of the first order.
Optionally, the second region includes K sub-regions, the resolution of the gradual change is a discretely-changed resolution, and the resolution corresponding to the sub-regions is a fixed resolution, where a shortest distance between a (K-1) th sub-region and a center of the hot spot region is smaller than a shortest distance between the K-th sub-region and the center of the hot spot region, and K is a positive integer greater than or equal to 2; the fade resolution determination sub-module 371 includes:
a third distance obtaining unit 3716, configured to obtain a distance between the target image position in the second area and the center of the hot spot area;
a sub-region determining unit 3717, configured to determine a sub-region where the target image is located according to the distance;
a fourth resolution determining unit 3718, configured to determine the resolution of the target image position according to the correspondence between the resolution and the sub-region.
Optionally, the correspondence between the resolution and the sub-region includes: the resolution corresponding to the 1 st sub-area of the target video image is 1/L of the first resolution or the difference between the first resolution and a first preset value, wherein L is an integer greater than or equal to 1.
Optionally, the correspondence between the resolution and the sub-region further includes: the resolution corresponding to the Kth sub-area of the target video image is 1/M of the resolution corresponding to the (K-1) th sub-area, and M is an integer greater than 1; or the resolution corresponding to the Kth sub-area of the target video image is the difference value between the resolution corresponding to the (K-1) th sub-area of the target video image and a second preset value.
It should be noted that the implementation of each module may also correspond to the corresponding description of the method embodiment shown in fig. 3.
In the embodiment illustrated in fig. 10, the image processing server may divide the target video image area into a first area and a second area according to a hot spot area of the target video image, so that the hot spot area is in the first area, then process the first area with a first resolution, process the second area with a gradually changed resolution, and finally send the processed target video image to the user's play terminal, where the first resolution is greater than all resolutions of the second area, the resolution of the second area decreases as the distance from the hot spot area increases, and in the case of the same bandwidth, the resolution of the hot spot area is higher by processing the video image, and the resolutions of the other areas are lower, so that the data amount of the processed video image is smaller than the data amount of the original video image, and useful video data, that is, video data that the user wants to see, is transmitted to the user's play terminal, the watching experience of the user is guaranteed, and the bandwidth cost is saved.
Referring to fig. 11, fig. 11 is a schematic structural diagram of another image processing apparatus according to an embodiment of the present invention, where the apparatus includes a processor 41, a memory 42, and a communication interface 43. The processor 41 is connected to the memory 42 and the communication interface 43, for example, the processor 41 may be connected to the memory 42 and the communication interface 43 through a bus.
The processor 41 is configured to support the image processing apparatus to perform the corresponding functions in the method of image processing described in fig. 3. The processor 41 may be a Central Processing Unit (CPU), a Network Processor (NP), a hardware chip, or any combination thereof. The hardware chip may be an application-specific integrated circuit (ASIC), a Programmable Logic Device (PLD), or a combination thereof. The PLD may be a Complex Programmable Logic Device (CPLD), a field-programmable gate array (FPGA), a General Array Logic (GAL), or any combination thereof.
The memory 42 is used to store program codes and the like. The memory 42 may include a volatile memory (RAM), such as a Random Access Memory (RAM); the memory 72 may also include a non-volatile memory (ROM), such as a read-only memory (ROM), a flash memory (HDD), a hard disk (HDD) or a solid-state drive (SSD); the memory 42 may also comprise a combination of the above-mentioned kinds of memories.
The communication interface 43 is used for receiving and transmitting data.
Processor 41 may call the program code to perform the following operations:
processing a first region of a target video image at a first resolution, wherein the first region comprises a hot spot region of the target video image;
processing a second area of the target video image with gradient resolution, wherein the gradient resolution is smaller than the first resolution, the resolution corresponding to an image position closer to the hot spot area in the second area is higher, the second area is located outside the first area, and the second area and the first area form the target video image;
the processed target video image is transmitted to the playback terminal through the communication interface 43.
Optionally, before processing the first region of the target video image at the first resolution, the processor 41 is further configured to:
determining a hot spot area of the target video image;
and determining a first area and a second area of the target video image according to the hot spot area.
Optionally, the determining, by the processor 41, a first region of the target video image according to the hot spot region specifically includes:
determining a center of the hotspot region;
and determining a region which takes the center as a circle center and a first preset distance as a radius as a first region of the target video image.
Optionally, the determining, by the processor 41, a first region of the target video image according to the hot spot region specifically includes:
and determining the minimum regular graph area containing the hot spot area as a first area of the target video image.
Optionally, before the processor 41 processes the first region of the target video image at the first resolution, it is further configured to:
determining a transmission bandwidth between the terminal and the playing terminal;
determining the first resolution and the graduated resolution according to the region size of the first region, the region size of the second region, and the transmission bandwidth.
Optionally, the processor 41 determines the first resolution and the resolution of the gradual change according to the size of the first region, the size of the second region, and the transmission bandwidth, and specifically includes:
determining the first resolution according to a region size of the first region, a region size of the second region, and the transmission bandwidth;
and determining the resolution of the target image position in the second area according to the distance between the target image position in the second area and the center of the hot spot area.
Optionally, the determining, by the processor 41, the resolution of the target image position in the second region according to the distance between the target image position in the second region and the center of the hot spot region specifically includes:
acquiring the distance between the position of the target image in the second area and the center of the hot spot area;
and determining the resolution of the target image position in the second area according to a gradient function, wherein the gradient function is a function representing the corresponding relation between the resolution and the distance.
Optionally, the second area includes a third area and a fourth area, wherein a shortest distance between the third area and the center of the hot spot area is smaller than a shortest distance between the fourth area and the center of the hot spot area; the processor 41 determines the resolution of the target image position in the second region according to the distance between the target image position in the second region and the center of the hot spot region, and specifically includes:
acquiring the distance between the position of the target image in the third area and the center of the hot spot area;
determining the resolution of the target image position in the third region according to a gradient function, wherein the gradient function is a function representing the corresponding relation between the resolution and the distance;
determining a second resolution as a resolution of the target image position in the fourth region, wherein the second resolution is less than or equal to a minimum resolution of the target image position in the third region, and the second resolution is a fixed resolution.
Optionally, the second resolution is 1/N of the first resolution, and N is an integer greater than 1; or the second resolution is 1/P of the minimum resolution of the target image position in the third area, and P is an integer greater than or equal to 1.
Optionally, the tapering function comprises: parabolic functions, elliptic functions, decreasing functions of the first order.
Optionally, the second region includes K sub-regions, the resolution of the gradual change is a discretely-changed resolution, and the resolution corresponding to the sub-regions is a fixed resolution, where a shortest distance between a (K-1) th sub-region and a center of the hot spot region is smaller than a shortest distance between the K-th sub-region and the center of the hot spot region, and K is a positive integer greater than or equal to 2; the processor 41 determines the resolution of the target image position in the second region according to the distance between the target image position in the second region and the center of the hot spot region, and specifically includes:
acquiring the distance between the position of the target image in the second area and the center of the hot spot area;
determining a sub-region where the target image position is located according to the distance;
and determining the resolution of the position of the target image according to the corresponding relation between the resolution and the sub-region.
Optionally, the correspondence between the resolution and the sub-region includes: the resolution corresponding to the 1 st sub-area of the target video image is 1/L of the first resolution or a difference value between the first resolution and a first preset value, wherein L is an integer greater than 1.
Optionally, the correspondence between the resolution and the sub-region further includes: the resolution corresponding to the Kth sub-area of the target video image is 1/M of the resolution corresponding to the (K-1) th sub-area, and M is an integer greater than 1; or the resolution corresponding to the Kth sub-area of the target video image is the difference value between the resolution corresponding to the (K-1) th sub-area of the target video image and a second preset value.
Optionally, the processor 41 is further configured to:
determining a target bandwidth required by transmitting the target video image and a transmission bandwidth between the target video image and the playing terminal;
the step of processing a first region of a target video image at a first resolution is performed if the target bandwidth is greater than the transmission bandwidth.
It should be noted that the implementation of each operation may also correspond to the corresponding description of the method embodiment shown in fig. 3.
Embodiments of the present invention also provide a computer storage medium storing a computer program, the computer program comprising program instructions that, when executed by an image processing apparatus, cause the image processing apparatus to perform the method according to the foregoing embodiments.
Embodiments of the present invention also provide a computer program, which includes program instructions, when executed by an image processing apparatus, for executing the method according to the foregoing embodiments.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.

Claims (19)

  1. A method of image processing, comprising:
    processing a first region of a target video image at a first resolution, wherein the first region comprises a hot spot region of the target video image;
    processing a second area of the target video image with gradient resolution, wherein the gradient resolution is smaller than the first resolution, the higher the resolution corresponding to an image at a position closer to the hot spot area in the second area is, the second area is located outside the first area, and the second area and the first area constitute the target video image;
    and sending the processed target video image to a playing terminal.
  2. The method of claim 1, wherein said processing said first region at said first resolution further comprises, prior to:
    determining a hot spot area of the target video image;
    and determining a first area and a second area of the target video image according to the hot spot area.
  3. The method of claim 2, wherein said determining a first region of the target video image from the hotspot region comprises:
    determining a center of the hotspot region;
    and determining a region which takes the center as a circle center and a first preset distance as a radius as a first region of the target video image.
  4. The method of claim 2, wherein said determining a first region of the target video image from the hotspot region comprises:
    and determining the minimum regular graph area containing the hot spot area as a first area of the target video image.
  5. The method of any of claims 1-4, wherein the graduated resolution comprises a continuously varying resolution or a discretely varying resolution.
  6. The method of any of claims 1-5, wherein processing the first region of the target video image at the first resolution further comprises:
    determining a transmission bandwidth between the terminal and the playing terminal;
    determining the first resolution and the graduated resolution according to the region size of the first region, the region size of the second region, and the transmission bandwidth.
  7. The method of claim 6, wherein the determining the resolution of the fade in accordance with the region size of the first region, the region size of the second region, and the transmission bandwidth comprises:
    and determining the resolution of the target image position in the second area according to the distance between the target image position in the second area and the center of the hot spot area.
  8. The method of claim 7, wherein determining the resolution of the target image location in the second region from the distance between the target image location in the second region and the center of the hotspot region comprises:
    acquiring the distance between the position of the target image in the second area and the center of the hot spot area;
    and determining the resolution of the target image position in the second area according to a gradient function, wherein the gradient function is a function representing the corresponding relation between the resolution and the distance.
  9. The method of claim 7, wherein the second area comprises a third area and a fourth area, wherein a shortest distance between the third area and a center of the hot spot area is less than a shortest distance between the fourth area and the center of the hot spot area;
    the determining the resolution of the target image position in the second region according to the distance between the target image position in the second region and the center of the hotspot region comprises:
    acquiring the distance between the position of the target image in the third area and the center of the hot spot area;
    determining the resolution of the target image position in the third region according to a gradient function, wherein the gradient function is a function representing the corresponding relation between the resolution and the distance;
    determining a second resolution as a resolution of the target image position in the fourth region, wherein the second resolution is less than or equal to a minimum resolution of the target image position in the third region, and the second resolution is a fixed resolution.
  10. The method of claim 9, wherein the second resolution is 1/N of the first resolution, N being an integer greater than 1; or
    The second resolution is 1/P of the minimum resolution of the target image position in the third region, and P is an integer greater than or equal to 1.
  11. The method of any of claims 8-10, wherein the tapering function comprises:
    a parabolic function, an elliptical function, or a decreasing function.
  12. The method of claim 7, wherein the second region comprises K sub-regions, the gradual change resolution is discretely variable resolution, the sub-regions correspond to a fixed resolution, wherein a shortest distance between a (K-1) th sub-region and a center of the hot spot region is smaller than a shortest distance between a kth sub-region and the center of the hot spot region, and K is a positive integer greater than or equal to 2;
    the determining the resolution of the target image position in the second region according to the distance between the target image position in the second region and the center of the hotspot region comprises:
    acquiring the distance between the position of the target image in the second area and the center of the hot spot area;
    determining a sub-region where the target image position is located according to the distance;
    and determining the resolution of the position of the target image according to the corresponding relation between the resolution and the sub-region.
  13. The method of claim 12, wherein the correspondence between the resolution and the sub-region comprises:
    the resolution corresponding to the 1 st sub-area of the target video image is 1/L of the first resolution or a difference value between the first resolution and a first preset value, wherein L is an integer greater than 1.
  14. The method of claim 13, wherein the correspondence between the resolution and the sub-region further comprises:
    the resolution corresponding to the Kth sub-area of the target video image is 1/M of the resolution corresponding to the (K-1) th sub-area, and M is an integer greater than 1; or
    And the resolution corresponding to the Kth sub-area of the target video image is the difference value between the resolution corresponding to the (K-1) th sub-area of the target video image and a second preset value.
  15. The method of any one of claims 1-14, further comprising:
    determining a target bandwidth required by transmitting the target video image and a transmission bandwidth between the target video image and the playing terminal;
    the step of processing a first region of a target video image at a first resolution is performed if the target bandwidth is greater than the transmission bandwidth.
  16. An apparatus for image processing, comprising:
    the first region processing module is used for processing a first region of a target video image at a first resolution, wherein the first region comprises a hot spot region of the target video image;
    a second region processing module, configured to process a second region of the target video image with a gradient resolution, where the gradient resolution is smaller than the first resolution, where a resolution corresponding to an image at a position closer to the hot spot region in the second region is higher, the second region is located outside the first region, and the second region and the first region form the target video image;
    and the sending module is used for sending the processed target video image to the playing terminal.
  17. An image processing apparatus, comprising a processor, a memory and a communication interface, the processor, the memory and the communication interface being connected to each other, wherein the communication interface is configured to receive and transmit data, the memory is configured to store program code, and the processor is configured to call the program code to perform the following operations:
    processing a first region of a target video image at a first resolution, wherein the first region comprises a hot spot region of the target video image;
    processing a second area of the target video image with gradient resolution, wherein the gradient resolution is smaller than the first resolution, the higher the resolution corresponding to an image at a position closer to the hot spot area in the second area is, the second area is located outside the first area, and the second area and the first area constitute the target video image;
    and sending the processed target video image to a playing terminal.
  18. A computer storage medium, characterized in that the computer storage medium stores a computer program comprising program instructions that, when executed by an apparatus for image processing, cause the apparatus for image processing to perform the method of any one of claims 1-15.
  19. A computer program comprising program instructions for performing the method of any one of claims 1-15 when executed by an image processing apparatus.
CN201780062040.0A 2016-12-26 2017-04-14 The method and apparatus of image procossing Pending CN109804409A (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
CN2016112209939 2016-12-26
CN201611220993 2016-12-26
PCT/CN2017/080645 WO2018120519A1 (en) 2016-12-26 2017-04-14 Image processing method and device

Publications (1)

Publication Number Publication Date
CN109804409A true CN109804409A (en) 2019-05-24

Family

ID=62706644

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201780062040.0A Pending CN109804409A (en) 2016-12-26 2017-04-14 The method and apparatus of image procossing

Country Status (2)

Country Link
CN (1) CN109804409A (en)
WO (1) WO2018120519A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111667411A (en) * 2020-06-12 2020-09-15 深圳天度物联信息技术有限公司 Image transmission method and device, electronic equipment and storage medium
CN112911191A (en) * 2021-01-28 2021-06-04 联想(北京)有限公司 Video call quality adjusting method and device, electronic equipment and storage medium

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111951343A (en) * 2019-05-16 2020-11-17 阿里巴巴集团控股有限公司 Image generation method and device and image display method and device
US11477426B2 (en) * 2020-10-26 2022-10-18 Avaya Management L.P. Selective image broadcasting in a video conference
CN113408440A (en) * 2021-06-24 2021-09-17 展讯通信（上海）有限公司 Video data jam detection method, device, equipment and storage medium

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020064314A1 (en) * 2000-09-08 2002-05-30 Dorin Comaniciu Adaptive resolution system and method for providing efficient low bit rate transmission of image data for distributed applications
CN101655984A (en) * 2008-05-12 2010-02-24 美国西门子医疗解决公司 System for adaptively processing medical image data
US20100128983A1 (en) * 2008-11-25 2010-05-27 Canon Kabushiki Kaisha Imaging system and imaging method
CN102915521A (en) * 2012-08-30 2013-02-06 中兴通讯股份有限公司 Method and device for processing mobile terminal images
TW201419846A (en) * 2012-11-09 2014-05-16 Inst Information Industry Dynamic resolution regulating system and dynamic resolution regulating method
CN104137050A (en) * 2012-02-22 2014-11-05 西里克斯系统公司 Remote display with hierarchical updating
CN104159129A (en) * 2014-08-08 2014-11-19 北京大学 Method for progressively transmitting partial splitting blocks under remote sensing data limited bandwidth
CN105635624A (en) * 2014-10-27 2016-06-01 华为技术有限公司 Video image processing method, device and system

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020064314A1 (en) * 2000-09-08 2002-05-30 Dorin Comaniciu Adaptive resolution system and method for providing efficient low bit rate transmission of image data for distributed applications
CN101655984A (en) * 2008-05-12 2010-02-24 美国西门子医疗解决公司 System for adaptively processing medical image data
US20100128983A1 (en) * 2008-11-25 2010-05-27 Canon Kabushiki Kaisha Imaging system and imaging method
CN104137050A (en) * 2012-02-22 2014-11-05 西里克斯系统公司 Remote display with hierarchical updating
CN102915521A (en) * 2012-08-30 2013-02-06 中兴通讯股份有限公司 Method and device for processing mobile terminal images
TW201419846A (en) * 2012-11-09 2014-05-16 Inst Information Industry Dynamic resolution regulating system and dynamic resolution regulating method
CN104159129A (en) * 2014-08-08 2014-11-19 北京大学 Method for progressively transmitting partial splitting blocks under remote sensing data limited bandwidth
CN105635624A (en) * 2014-10-27 2016-06-01 华为技术有限公司 Video image processing method, device and system

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111667411A (en) * 2020-06-12 2020-09-15 深圳天度物联信息技术有限公司 Image transmission method and device, electronic equipment and storage medium
CN112911191A (en) * 2021-01-28 2021-06-04 联想(北京)有限公司 Video call quality adjusting method and device, electronic equipment and storage medium

Also Published As

Publication number Publication date
WO2018120519A1 (en) 2018-07-05

Similar Documents

Publication Publication Date Title
CN109804409A (en) The method and apparatus of image procossing
US11601699B2 (en) Predictive content delivery for video streaming services
US11653065B2 (en) Content based stream splitting of video data
US20190327470A1 (en) Video enhancement method and device
US11589110B2 (en) Digital media system
EP3734980A1 (en) Video playback method and terminal, server and storage medium
CN106162232A (en) video playing control method and device
US8983188B1 (en) Edge-aware smoothing in images
US10529122B2 (en) Information processing apparatus, information processing method, and storage medium
CN110852334B (en) System and method for adaptive pixel filtering
CN107610045B (en) Brightness compensation method, device and equipment in fisheye picture splicing and storage medium
CN114513506A (en) Service processing method, access edge cloud server and service processing system
CN114007059A (en) Video compression method, decompression method, device, electronic equipment and storage medium
CN107818553B (en) Image gray value adjusting method and device
CN112261408B (en) Image processing method and device for head-mounted display equipment and electronic equipment
CN113810755B (en) Panoramic video preview method and device, electronic equipment and storage medium
CN110941413B (en) Display screen generation method and related device
CN116567247A (en) Video encoding method, real-time communication method, device, equipment and storage medium
CN114205359A (en) Video rendering coordination method, device and equipment
CN110941596A (en) Disk space release method and device, computing equipment and computer storage medium
CN112565819B (en) Video data processing method and device, electronic equipment and storage medium
Seeling et al. I. want. pixels.(Entering the age of 4K)
CN115914503A (en) Image synthesis method, video conference method, device and storage medium
CN117440160A (en) Video processing method, device, equipment and storage medium
CN115398880A (en) Image processing apparatus, method of operating image processing apparatus, and non-volatile storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right

Effective date of registration: 20210425

Address after: Unit 3401, unit a, building 6, Shenye Zhongcheng, No. 8089, Hongli West Road, Donghai community, Xiangmihu street, Futian District, Shenzhen, Guangdong 518040

Applicant after: Honor Device Co.,Ltd.

Address before: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen

Applicant before: HUAWEI TECHNOLOGIES Co.,Ltd.

TA01 Transfer of patent application right