CN108063946B

CN108063946B - Image encoding method and apparatus, storage medium, and electronic apparatus

Info

Publication number: CN108063946B
Application number: CN201711141665.4A
Authority: CN
Inventors: 涂远东; 曾新海
Original assignee: Tencent Technology Chengdu Co Ltd
Current assignee: Tencent Technology Chengdu Co Ltd
Priority date: 2017-11-16
Filing date: 2017-11-16
Publication date: 2021-09-24
Anticipated expiration: 2037-11-16
Also published as: CN108063946A

Abstract

The invention discloses an image coding method and device, a storage medium and an electronic device. Wherein, the method comprises the following steps: acquiring an image picture matched with a current playing visual angle from a panoramic video image frame in video stream data to be played; determining a first area and a second area contained in the image picture, wherein the first area is an interested area in the image picture, and the second area is an area except the interested area in the image picture; acquiring the coding weight of a first macro block contained in a first area and the coding weight of a second macro block contained in a second area, wherein the coding weight of the first macro block is greater than that of the second macro block; and coding the first area according to the coding weight of the first macro block, and coding the second area according to the coding weight of the second macro block. The invention solves the technical problem of low coding efficiency when the video image frame is coded in the related technology.

Description

Image encoding method and apparatus, storage medium, and electronic apparatus

Technical Field

The present invention relates to the field of computers, and in particular, to an image encoding method and apparatus, a storage medium, and an electronic apparatus.

Background

Currently, in a Virtual Reality (VR) live video broadcasting process, a 360-degree spherical panoramic image is used to form each frame of video image frame in a video. The 360-degree spherical panoramic image can be divided into different image frames according to a Field of View (FOV for short), so that when the change of the View angle of the player is detected, the image frames displayed in the display interface of the device are adjusted, and the purpose of enabling the player to experience the feeling of being personally on the scene is achieved.

In the VR live video broadcasting process, all picture contents in a panoramic image corresponding to a video image frame are usually required to be encoded before transmitting the video image frame in video stream data to each playing client, and this encoding manner will occupy a large amount of encoding time, thereby causing a technical problem of low encoding efficiency.

In view of the above problems, no effective solution has been proposed.

Disclosure of Invention

Embodiments of the present invention provide an image encoding method and apparatus, a storage medium, and an electronic apparatus, so as to at least solve the technical problem of low encoding efficiency when encoding a video image frame in the related art.

According to an aspect of an embodiment of the present invention, there is provided an image encoding method including: acquiring an image picture matched with a current playing visual angle from a panoramic video image frame in video stream data to be played; determining a first region and a second region included in the image picture, wherein the first region is an interested region in the image picture, and the second region is a region except the interested region in the image picture; acquiring the coding weight of a first macro block contained in the first area and the coding weight of a second macro block contained in the second area, wherein the coding weight of the first macro block is greater than the coding weight of the second macro block; and encoding the first region according to the encoding weight of the first macro block, and encoding the second region according to the encoding weight of the second macro block.

According to another aspect of the embodiments of the present invention, there is also provided an image encoding apparatus including: the first acquisition unit is used for acquiring an image picture matched with a current playing visual angle from a panoramic video image frame in video stream data to be played; a determining unit configured to determine a first region and a second region included in the image screen, wherein the first region is a region of interest in the image screen, and the second region is a region other than the region of interest in the image screen; a second obtaining unit, configured to obtain a coding weight of a first macroblock included in the first region and a coding weight of a second macroblock included in the second region, where the coding weight of the first macroblock is greater than the coding weight of the second macroblock; and an encoding unit configured to encode the first region according to the encoding weight of the first macroblock and encode the second region according to the encoding weight of the second macroblock.

According to another aspect of the embodiments of the present invention, there is also provided a storage medium including a stored program, wherein the program executes the method.

According to another aspect of the embodiments of the present invention, there is also provided an electronic apparatus, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the method through the computer program.

In the embodiment of the invention, an image picture matched with a current playing visual angle is acquired from a panoramic video image frame in video stream data to be played, and a first area and a second area contained in the image picture are determined, wherein the first area is an interested area in the image picture, and the second area is an area except the interested area; and then acquiring the coding weight of a first macro block contained in the first area and the coding weight of a second macro block contained in the second area so as to realize the coding of the first area according to the coding weight of the first macro block and the coding of the second area according to the coding weight of the second macro block. And the coding weight of the first macro block is greater than that of the second macro block. That is to say, for a panoramic video image frame in video stream data to be played, a macro block in an interested region in an image picture matched with a current playing view angle is encoded by using a higher encoding weight, and a macro block in a non-interested region is encoded by using a lower encoding weight, so that different regions in the image picture corresponding to the current playing view angle are encoded by using different encoding weights, thereby improving encoding flexibility, overcoming the problems of occupying a large amount of encoding time and having lower encoding efficiency caused by encoding all picture contents by using the same encoding mode in the related art, and further realizing the effect of improving the image encoding efficiency.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:

FIG. 1 is a schematic diagram of an application environment of an alternative image encoding method according to an embodiment of the present invention;

FIG. 2 is a flow chart of an alternative image encoding method according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of an alternative image encoding method according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of an alternative image encoding method according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of yet another alternative image encoding method according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of yet another alternative image encoding method according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of yet another alternative image encoding method according to an embodiment of the present invention;

FIG. 8 is a schematic diagram of an alternative image encoding apparatus according to an embodiment of the present invention;

FIG. 9 is a schematic diagram of an alternative image encoding apparatus according to an embodiment of the present invention;

FIG. 10 is a schematic diagram of an alternative electronic device according to an embodiment of the invention.

Detailed Description

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

According to an aspect of the embodiments of the present invention, there is provided an embodiment of the above-described image encoding method. As an alternative embodiment, the image encoding method may be, but is not limited to, applied in the application environment shown in fig. 1, where the terminal 102 records panoramic video content in video stream data to be live broadcast, and transmits panoramic video image frames in the video stream data to the playing device 106 (such as VR glasses) through the network 104, where the playing device 106 encodes the obtained panoramic video image frames through the encoder 108. Acquiring an image picture matched with a current playing visual angle from the panoramic video image frame, and determining a first area and a second area contained in the image picture, wherein the first area is an interested area in the image picture, and the second area is an area except the interested area; and then acquiring the coding weight of a first macro block contained in the first area and the coding weight of a second macro block contained in the second area so as to realize the coding of the first area according to the coding weight of the first macro block and the coding of the second area according to the coding weight of the second macro block. And the coding weight of the first macro block is greater than that of the second macro block.

In this embodiment, for a panoramic video image frame in video stream data to be played, a macro block in an interested area in an image picture matched with a current playing view angle is encoded by using a higher encoding weight, and a macro block in a non-interested area is encoded by using a lower encoding weight, so that different areas in the image picture corresponding to the current playing view angle are encoded by using different encoding weights, thereby improving encoding flexibility, overcoming the problems of occupying a large amount of encoding time and having lower encoding efficiency caused by encoding all picture contents by using the same encoding mode in the related art, further realizing the effect of improving the image encoding efficiency, and further ensuring the real-time performance of video live broadcast.

Optionally, in this embodiment, the playing device may include, but is not limited to, at least one of the following: the system comprises a mobile phone, a tablet computer, a notebook computer, a desktop PC, a digital television, VR glasses and other hardware equipment for displaying panoramic video images in live video stream data. The network may include, but is not limited to, at least one of: wide area networks, metropolitan area networks, and local area networks. The above is only an example, and the present embodiment is not limited to this.

In an embodiment of the present invention, there is provided an image encoding method, as shown in fig. 2, including:

s202, acquiring an image picture matched with a current playing visual angle from a panoramic video image frame in video stream data to be played;

s204, determining a first area and a second area contained in the image picture, wherein the first area is an interested area in the image picture, and the second area is an area except the interested area in the image picture;

s206, acquiring the coding weight of a first macro block contained in the first area and the coding weight of a second macro block contained in the second area, wherein the coding weight of the first macro block is greater than that of the second macro block;

and S208, coding the first area according to the coding weight of the first macro block, and coding the second area according to the coding weight of the second macro block.

Optionally, in this embodiment, the image encoding method may be applied to, but not limited to, a scene in which a panoramic video image is played, such as in a Virtual Reality (VR) video playing process and a panoramic video playing process, where the playing may include, but is not limited to: video on demand and video live broadcast. Wherein, the above video can be applied to, but not limited to: video playing applications, gaming applications, shopping applications, etc. The above is only an example, and this is not limited in this embodiment.

It should be noted that, for a panoramic video image frame in video stream data to be played, a macro block in an interested region in an image picture matched with a current playing view angle is encoded by using a higher encoding weight, and a macro block in a non-interested region is encoded by using a lower encoding weight, so that a ROI region encoding mode is adopted for different regions in the image picture corresponding to the current playing view angle, and different encoding weights are used for encoding for different regions, so as to improve encoding flexibility, overcome the problems of a large amount of encoding time occupied and low encoding efficiency caused by encoding all picture contents by using the same encoding mode in the related art, further realize the effect of improving the image encoding efficiency, and further ensure the real-time of video live broadcast.

Optionally, in this embodiment, the panoramic video image frame may be, but is not limited to, a 360-degree spherical panoramic image, where the 360-degree spherical panoramic image may be, but is not limited to, divided into a plurality of views (fields of views, FOV for short), and an image frame corresponding to each View generates a path of View video stream data to be played by a playing device. In this embodiment, the image frames may be, but not limited to, obtained by using pyramid model mapping projection for each view angle.

For example, as shown in fig. 3, the playback device is illustrated by taking VR glasses as an example, the left side of fig. 3 shows that 360-degree spherical panoramic images in a video can be viewed by using VR glasses, and the right side of fig. 3 shows that a built-in pyramid model (each vertex is located on a spherical surface) performs mapping projection in the 360-degree spherical panoramic images. Further, each view angle will obtain a corresponding image picture (including the projection of the pyramid model on each surface), as shown in fig. 4, the left side of fig. 4 shows the view angle image after the projection of the pyramid model on each surface is expanded, and after image processing such as stretching, an image picture matched with the view angle shown on the right side of fig. 4 will be obtained, wherein the shaded part represents the picture corresponding to the bottom of the model.

In this embodiment, the model for performing the projection process on the panoramic video image frame is not limited to the pyramid model, and may include other models such as a trapezoid model.

Optionally, in this embodiment, the image encoding method may be but is not limited to be applied to the encoding system shown in fig. 5, a panoramic video image frame is obtained, and is divided into n view angles according to configuration, where the view angles are from 1 to n, pyramid model mapping projections corresponding to the n view angles are respectively obtained, and then image frames corresponding to different view angles are obtained. Further, the image picture is input into an encoder to be encoded, so as to obtain video stream data for playing, wherein the video stream data comprises view video stream data corresponding to a plurality of views respectively.

Optionally, in this embodiment, the first Region is a Region of Interest (ROI) in the image, for example, as shown in the right side of fig. 3, the first Region may be a bottom Region of a pyramid model corresponding to the current playing view, and other regions (e.g., side regions) of the pyramid model correspond to second regions. The above is only an optional region determining manner, and in this embodiment, the first region and the second region may be determined in different manners according to different models, but not limited to this.

It should be noted that, in this embodiment, when different regions in an image picture are coded, the regions include unit regions for coding, such as macroblocks. The first macroblock in the first area and the second macroblock in the second area may be but not limited to macroblocks of the same size, such as macroblocks in units of 16 × 16 pixels, or macroblocks of different sizes; in addition, the sizes of the first macro blocks in the first area may be the same size or different sizes, and the sizes of the second macro blocks in the second area may be the same size or different sizes. This is not limited in this embodiment.

Optionally, in this embodiment, after the first region and the second region are determined, the encoding weight of the first macroblock in the first region and the encoding weight of the second macroblock in the second region may be obtained, but are not limited to this, where the encoding weight of the first macroblock in the first region is greater than the encoding weight of the second macroblock in the second region.

In addition, in this embodiment, the first region may include, but is not limited to, a plurality of first macroblocks, and the encoding weights of the plurality of first macroblocks are the same; the second region may include, but is not limited to, a plurality of second macro blocks, and the encoding weights of the plurality of second macro blocks may be determined, but is not limited to, according to a target distance between the second macro blocks and the first region. Optionally, in this embodiment, the smaller the weight is, the smaller the encoding weight of the second macroblock is.

Optionally, in this embodiment, the encoding weight may be, but is not limited to, used for determining the encoding length, that is, the larger the encoding weight of the macroblock is, the longer the corresponding encoding length is, so as to ensure that the encoding distortion of the macroblock is smaller. In this embodiment, for a panoramic video image frame, coding is implemented by combining a ROI region coding mode and using different coding weights for different macro blocks in different regions, so as to achieve flexibility of coding, and coding is not required to be implemented by using the same coding mode for all picture contents, so that coding time and bandwidth are saved, and coding efficiency is further improved.

According to the embodiment provided by the application, for a panoramic video image frame in video stream data to be played, a macro block in an interested area in an image picture matched with a current playing visual angle is encoded by using a higher encoding weight, and a macro block in a non-interested area is encoded by using a lower encoding weight, so that different areas in the image picture corresponding to the current playing visual angle are encoded by using different encoding weights, the encoding flexibility is improved, the problems that in the related technology, the occupied encoding time is large and the encoding efficiency is low due to the fact that all picture contents are encoded by using the same encoding mode, the effect of improving the image encoding efficiency is realized, and the real-time performance of live video is ensured.

As an alternative, encoding the first region according to the encoding weight of the first macroblock, and encoding the second region according to the encoding weight of the second macroblock includes:

s1, configuring a first coding length for the first macroblock in the first region according to the coding weight of the first macroblock; encoding a first macroblock in a first region using a first encoding length;

s2, configuring a second coding length for the second macroblock in the second region according to the coding weight of the second macroblock; encoding a second macroblock in the second region using a second encoding length;

wherein the first code length is greater than the second code length.

Specifically, the description will be made with reference to fig. 6, where the square shown in fig. 6 is an image screen, a shadow area in the image screen is a first area, and a blank area outside the shadow area in the image screen is a second area. The first macro block in the first area comprises macro block C and macro block D, and the second macro block in the second area comprises macro block A and macro block B. In addition, the coding weights of the macroblock C and the macroblock D in the first region may be configured with the same value or with different values, and the coding weights of the macroblock a and the macroblock B in the second region may be configured with the same value or with different values. Correspondingly, in the present embodiment, different coding weights will correspond to different coding lengths.

For example, the coding weights of the macro block C and the macro block D in the first region are both configured as a, and the coding weights of the macro block a and the macro block B in the second region are both configured as B, where a > B. Correspondingly, the coding length of the macro block C and the macro block D is greater than the coding length of the macro block a and the macro block B, for example, the coding weight a corresponds to a coding length of 16 bits, and the coding weight B corresponds to a coding length of 8 bits.

For another example, the coding weights of the macro block C and the macro block D in the first region are both configured as a, and the coding weights of the macro block a and the macro block B in the second region may be configured as B and C according to the target distance between the macro block and the first region, but not limited thereto, where as shown in fig. 6, the target distance between the macro block B and the first region is closer, the corresponding relationship between the coding weights is: a > c > b. Correspondingly, the coding length of the macro block C and the macro block D is greater than that of the macro block B, and the coding length of the macro block B is greater than that of the macro block a, for example, the coding length is 16 bits for the coding weight a, 8 bits for the coding weight C, and 4 bits for the coding weight B.

It should be noted that the shorter the coding length is, the less image information is carried by the coded macroblock, and the higher the distortion factor is. Therefore, in the present embodiment, by configuring a higher encoding weight for the first region (i.e. the region of interest), the first region in the image picture is encoded by using a longer encoding byte, so as to ensure that the first region can retain the original image information during encoding, and reduce the distortion caused by compression encoding; and the second area (namely the non-interested area) is configured with lower coding weight to reduce the coding bytes, so that the coding cost of the non-concerned area is reduced, the coding time is shortened, the bandwidth occupation is reduced, and the effect of improving the coding efficiency is realized.

According to the embodiment provided by the application, different coding weights are configured for different regions, so that the different regions are coded by using the coding lengths corresponding to the different coding weights, the coding flexibility is improved, the coding utilization rate of limited coding bytes is improved, and the coding efficiency is ensured.

As an alternative, the obtaining of the coding weight of the second macroblock included in the second region includes:

s1, acquiring the target distance from the second macro block to the first area;

and S2, determining the coding weight of the second macro block according to the target distance, wherein the coding weight of the second macro block is smaller when the target distance is larger.

Optionally, in this embodiment, the obtaining the target distance from the second macro block to the first area includes:

s21, acquiring the coordinate position of the second macro block;

s22, calculating the distance from the second macro block to each boundary in the first area according to the coordinate position of the second macro block;

and S23, taking the minimum value of the distances as the target distance.

Optionally, in this embodiment, the first region includes a plurality of first macroblocks, and the coding weights of the plurality of first macroblocks are the same; the second region includes a plurality of second macroblocks, and the smaller the target distance, the smaller the second coding length of the second macroblock.

Optionally, in this embodiment, the target distance may be, but is not limited to, a point-to-straight distance. That is, after acquiring the coordinate position of any one second macro block in the second area, the distance from the second macro block to each boundary in the first area may be acquired, and the calculation formula of the target distance d may be as follows:

where, (x, y) is the coordinate position of the second macro block, Ax + By + C is the expression of a straight line where any one of the boundaries is located, and A, B, C is the coefficient of the straight line.

Specifically, referring to fig. 6-7, the square shown in fig. 6 is an image frame, the shadow area in the image frame is a first area, and the blank area outside the shadow area in the image frame is a second area. The first macro block in the first area comprises macro block C and macro block D, and the second macro block in the second area comprises macro block A and macro block B.

For example, as shown in fig. 7, taking the macro block a (0, y) as an example, the distance from the macro block a to each boundary of the first area is calculated by the above formula (1) as follows:

the macro block A intersects the first boundary at an intersection point X1, and the corresponding distance is d 1;

the macro block A intersects the second boundary at an intersection point X2, and the corresponding distance is d 2;

the macro block A intersects with the third boundary at an intersection point X3, and the corresponding distance is d 3;

the macro block A intersects with the fourth boundary at an intersection point X4, and the corresponding distance is d 4;

as can be seen from fig. 7, the distance d1 from the macroblock a to the first slice boundary is the minimum value of the above distances, and the distance d1 is taken as the target distance from the macroblock a to the first area.

According to the embodiment provided by the application, the target distance from the second macro block to the boundary of the first area in the second area is obtained, so that the coding weight of the second macro block is determined according to the distance from the target distance to the first area, the macro block close to the region of interest is configured with higher coding weight, more image information is reserved, the macro block of the principle region of interest is configured with lower coding weight, and after compression coding, the coding cost is saved, the bandwidth is reduced, and the effect of improving the coding efficiency is further realized.

As an alternative, the determining the first region and the second region included in the image frame includes:

s1, determining the middle point of each boundary of the image picture;

s2, a closed region obtained by connecting the midpoints of the respective boundaries is determined as a first region, and a region other than the first region in the image screen is determined as a second region.

Specifically, as described with reference to fig. 4, after the projection of the current playing view angle on each surface of the pyramid model is obtained, image processing such as stretching may be performed on the view angle image (as shown in the left side of fig. 4, the shadow is a bottom region, and is also a binocular attention region) after the projection is expanded, for example, four vertex angles are stretched into right angles, an image picture corresponding to the current playing view angle shown in the right side of fig. 4 is obtained, then, midpoints of each edge boundary of the image picture are connected to obtain a closed region, which is used as a first region (an interest region) in the image picture, and the blank region is used as a second region.

According to the embodiment provided by the application, the pyramid model is used for mapping and projecting, the image picture matched with the current playing view angle is obtained, and the first region and the second region in the image picture are accurately determined, so that the determination step of the region of interest is simplified, the encoding time is shortened, and the encoding efficiency is improved.

As an optional scheme, acquiring an image picture matched with a current playing view angle from a panoramic video image frame in video stream data to be played includes:

s1, acquiring panoramic video image frames;

s2, determining a view angle image matched with the current playing view angle through a pyramid model;

and S3, stretching the view angle image to obtain an image picture matched with the current playing view angle.

Specifically, as described with reference to fig. 3-4, after a panoramic video image frame is acquired, as shown in the right side of fig. 3, a pyramid model built in a sphere is used to implement mapping projection, wherein each vertex of the pyramid model is on the spherical surface, and the bottom area is a binocular attention area, after the projection is expanded, an angle-of-view image matched with a current playing angle of view is accurately determined, as shown in the left side of fig. 4, and then stretching processing is performed to obtain an image picture matched with the current playing angle of view, as shown in the right side of fig. 4, where a shaded area shown in the right side of fig. 4 is a first area.

According to the embodiment provided by the application, the pyramid model is used for carrying out mapping projection, the view angle image matched with the current playing view angle is obtained, and the image picture to be coded is obtained through image processing, so that the first region in the image can be rapidly and accurately positioned, and high-efficiency coding is realized.

It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.

Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.

According to another aspect of the embodiments of the present invention, there is also provided an image encoding apparatus for implementing the above-described image encoding method, as shown in fig. 8, the apparatus including:

1) a first obtaining unit 802, configured to obtain an image frame matched with a current playing view angle from a panoramic video image frame in video stream data to be played;

2) a determining unit 804, configured to determine a first region and a second region included in the image frame, where the first region is an interested region in the image frame, and the second region is a region other than the interested region in the image frame;

3) a second obtaining unit 806, configured to obtain a coding weight of a first macroblock included in the first region and a coding weight of a second macroblock included in the second region, where the coding weight of the first macroblock is greater than the coding weight of the second macroblock;

4) the encoding unit 808 is configured to encode the first region according to the encoding weight of the first macroblock, and encode the second region according to the encoding weight of the second macroblock.

Optionally, in this embodiment, the image encoding apparatus may be applied to, but not limited to, a scene in which a panoramic video image is played, such as during a Virtual Reality (VR) video playing process and during a panoramic video playing process, where the playing may include, but is not limited to: video on demand and video live broadcast. Wherein, the above video can be applied to, but not limited to: video playing applications, gaming applications, shopping applications, etc. The above is only an example, and this is not limited in this embodiment.

Optionally, in this embodiment, the image encoding apparatus may be, but is not limited to, applied to the encoding system shown in fig. 5, obtain one panoramic video image frame, divide the panoramic video image frame into n view angles according to configuration, and obtain pyramid model mapping projections corresponding to the n view angles respectively from view angle 1 to view angle n, so as to obtain image frames corresponding to different view angles. Further, the image picture is input into an encoder to be encoded, so as to obtain video stream data for playing, wherein the video stream data comprises view video stream data corresponding to a plurality of views respectively.

As an alternative, as shown in fig. 9, the encoding unit 808 includes:

1) a first configuring module 902, configured to configure a first coding length for a first macroblock in a first region according to a coding weight of the first macroblock;

2) a first encoding module 904 for encoding a first macroblock in a first region using a first encoding length;

3) a second configuring module 906, configured to configure a second coding length for the second macroblock in the second region according to the coding weight of the second macroblock;

4) a second encoding module 908 for encoding a second macroblock in the second region using a second encoding length;

wherein the first code length is greater than the second code length.

As an optional solution, the second obtaining unit 806 includes:

1) the first acquisition module is used for acquiring the target distance from the second macro block to the first area;

2) and the first determining module is used for determining the coding weight of the second macro block according to the target distance, wherein the larger the target distance is, the smaller the coding weight of the second macro block is.

Optionally, in this embodiment, the first obtaining module includes:

(1) the obtaining submodule is used for obtaining the coordinate position of the second macro block;

(2) the calculating submodule is used for calculating the distance from the second macro block to each boundary in the first area according to the coordinate position of the second macro block;

(3) and the determining submodule is used for taking the minimum value in the distances as the target distance.

Optionally, in this embodiment, the first region includes a plurality of first macro blocks, and the encoding weights of the plurality of first macro blocks are the same; the second region includes a plurality of second macroblocks, and the smaller the target distance, the smaller the second coding length of the second macroblock.

For example, as shown in fig. 7, taking the macro block a (0, y) as an example, the distance from the macro block a to each boundary of the first area is calculated by the above formula (2) as follows:

As an alternative, the determining unit includes:

1) the second determining module is used for determining the middle point of each boundary of the image picture;

2) and the third determining module is used for determining a closed area obtained by connecting the middle points of the boundaries as a first area and determining an area except the first area in the image picture as a second area.

As an optional solution, the first obtaining unit includes:

1) the second acquisition module is used for acquiring a panoramic video image frame;

2) the fourth determining module is used for determining a visual angle image matched with the current playing visual angle through the pyramid model;

3) and the processing module is used for stretching the view angle image to obtain an image picture matched with the currently played view angle.

According to still another aspect of the embodiments of the present invention, there is also provided an electronic device for implementing the above-described image encoding method, as shown in fig. 10, the electronic device may include: one or more processors 1002 (only one of which is shown), memory 1004, sensors 1006, display 1008, encoder 1010, and transmission 1012.

The memory 1004 can be used for storing software programs and modules, such as the image encoding method and apparatus in the embodiment of the present invention

The processor 1002 executes various functional applications and data processing, i.e., an image encoding method, by executing software programs and modules stored in the memory 1004 in accordance with program instructions/modules. The memory 1004 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 1004 may further include memory located remotely from the processor 1002, which may be connected to the terminal through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmitting device 1012 is used for receiving or transmitting data via a network. Examples of the network may include a wired network and a wireless network. In one example, the transmission device 1012 includes a Network adapter (NIC) that can be connected to a router via a Network cable and other Network devices to communicate with the internet or a local area Network. In one example, the transmitting device 1012 is a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.

Specifically, the memory 1004 is used for storing preset action conditions, information of preset authorized users and application programs.

Optionally, the specific examples in this embodiment may refer to the examples described in embodiment 1 and embodiment 2, and this embodiment is not described herein again.

It can be understood by those skilled in the art that the structure shown in fig. 10 is only an illustration, and the electronic device may also be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palmtop computer, a Mobile Internet Device (MID), a PAD, and the like. Fig. 10 is a diagram illustrating a structure of the electronic device. For example, the electronic device may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in FIG. 10, or have a different configuration than shown in FIG. 10.

Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.

In yet another aspect of embodiments of the present invention, a storage medium is also provided. Optionally, in this embodiment, the storage medium may be located in at least one of a plurality of network devices in a network.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps:

s1, acquiring an image picture matched with the current playing visual angle from a panoramic video image frame in the video stream data to be played;

s2, determining a first area and a second area contained in the image picture, wherein the first area is an interested area in the image picture, and the second area is an area except the interested area in the image picture;

s3, obtaining the coding weight of the first macro block contained in the first region and the coding weight of the second macro block contained in the second region, wherein the coding weight of the first macro block is greater than the coding weight of the second macro block;

s4, the first region is encoded according to the encoding weight of the first macroblock, and the second region is encoded according to the encoding weight of the second macroblock.

Optionally, the storage medium is further arranged to store program code for performing the steps of:

wherein the first code length is greater than the second code length.

Optionally, in this embodiment, the storage medium may include, but is not limited to: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.

The integrated unit in the above embodiments, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing one or more computer devices (which may be personal computers, servers, network devices, etc.) to execute all or part of the steps of the method according to the embodiments of the present invention.

In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the several embodiments provided in the present application, it should be understood that the disclosed client may be implemented in other manners. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims

1. An image encoding method, comprising:

acquiring an image picture matched with a current playing visual angle from a panoramic video image frame in video stream data to be played;

determining a first area and a second area contained in the image picture, wherein the first area is an interested area in the image picture, and the second area is an area except the interested area in the image picture;

acquiring the coding weight of a first macro block contained in the first area and the coding weight of a second macro block contained in the second area, wherein the coding weight of the first macro block is greater than the coding weight of the second macro block, and the coding weight of the second macro block is determined according to the distance from the second macro block to the first area;

and coding the first region according to the coding length corresponding to the coding weight of the first macro block, and coding the second region according to the coding length corresponding to the coding weight of the second macro block.

2. The method according to claim 1, wherein the encoding the first region according to the encoding length corresponding to the encoding weight of the first macroblock, and the encoding the second region according to the encoding length corresponding to the encoding weight of the second macroblock comprises:

configuring a first coding length for the first macro block in the first region according to the coding weight of the first macro block; encoding the first macroblock in the first region using the first encoding length;

configuring a second coding length for the second macro block in the second region according to the coding weight of the second macro block; encoding the second macroblock in the second region using the second encoding length;

wherein the first code length is greater than the second code length.

3. The method according to claim 1, wherein said obtaining the coding weight of the second macroblock contained in the second region comprises:

acquiring a target distance from the second macro block to the first area;

and determining the coding weight of the second macro block according to the target distance, wherein the larger the target distance is, the smaller the coding weight of the second macro block is.

4. The method of claim 3, wherein the obtaining the target distance of the second macroblock to the first region comprises:

acquiring the coordinate position of the second macro block;

calculating the distance from the second macro block to each boundary in the first area according to the coordinate position of the second macro block;

and taking the minimum value of the distances as the target distance.

5. The method according to any one of claims 1 to 4, wherein the first region comprises a plurality of the first macro blocks, and the encoding weights of the first macro blocks are the same; the second region includes a plurality of second macroblocks, and the smaller the target distance, the smaller the second coding length of the second macroblock.

6. The method of claim 1, wherein the determining the first region and the second region included in the image frame comprises:

determining the middle point of each boundary of the image picture;

and determining a closed area obtained by connecting the middle points of the boundaries as the first area, and determining an area except the first area in the image picture as the second area.

7. The method according to claim 1, wherein the obtaining of the image picture matching the current playing view from the panoramic video image frame in the video stream data to be played comprises:

acquiring the panoramic video image frame;

determining a view angle image matched with the current playing view angle through a pyramid model;

and stretching the view angle image to obtain the image picture matched with the current playing view angle.

8. An image encoding device characterized by comprising:

the first acquisition unit is used for acquiring an image picture matched with a current playing visual angle from a panoramic video image frame in video stream data to be played;

a determining unit, configured to determine a first region and a second region included in the image frame, where the first region is a region of interest in the image frame, and the second region is a region of the image frame other than the region of interest;

a second obtaining unit, configured to obtain a coding weight of a first macroblock included in the first region and a coding weight of a second macroblock included in the second region, where the coding weight of the first macroblock is greater than the coding weight of the second macroblock, and the coding weight of the second macroblock is determined according to a distance from the second macroblock to the first region;

and the coding unit is used for coding the first area according to the coding length corresponding to the coding weight of the first macro block and coding the second area according to the coding length corresponding to the coding weight of the second macro block.

9. The apparatus of claim 8, wherein the encoding unit comprises:

a first configuration module, configured to configure a first coding length for the first macroblock in the first region according to the coding weight of the first macroblock; a first encoding module for encoding the first macroblock in the first region using the first encoding length;

a second configuration module, configured to configure a second coding length for the second macroblock in the second region according to the coding weight of the second macroblock; a second encoding module for encoding the second macroblock in the second region using the second encoding length;

wherein the first code length is greater than the second code length.

10. The apparatus of claim 8, wherein the second obtaining unit comprises:

a first obtaining module, configured to obtain a target distance from the second macro block to the first area;

a first determining module, configured to determine the coding weight of the second macroblock according to the target distance, where the larger the target distance is, the smaller the coding weight of the second macroblock is.

11. The apparatus of claim 10, wherein the first obtaining module comprises:

the obtaining submodule is used for obtaining the coordinate position of the second macro block;

the calculating submodule is used for calculating the distance from the second macro block to each boundary in the first area according to the coordinate position of the second macro block;

a determination submodule configured to take a minimum value of the distances as the target distance.

12. The apparatus according to any one of claims 8 to 11, wherein the first region comprises a plurality of the first macro blocks, and the encoding weights of the first macro blocks are the same; the second region includes a plurality of second macroblocks, and the smaller the target distance, the smaller the second coding length of the second macroblock.

13. The apparatus of claim 8, wherein,

the determination unit includes:

the second determining module is used for determining the middle point of each boundary of the image picture;

a third determining module, configured to determine a closed region obtained by connecting the middle points of the boundaries as the first region, and determine a region other than the first region in the image as the second region;

the first acquisition unit includes:

the second acquisition module is used for acquiring the panoramic video image frame;

the fourth determining module is used for determining a view angle image matched with the current playing view angle through a pyramid model;

and the processing module is used for stretching the view angle image to obtain the image picture matched with the current playing view angle.

14. A storage medium, characterized in that the storage medium comprises a stored program, wherein the program when executed performs the method of any one of claims 1 to 7.

15. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the method of any one of claims 1 to 7 by means of the computer program.