CN106657972B - A kind of control method and device of video playing - Google Patents

Abstract

The present invention provides a video playback control method and device, wherein the control method applied on the server side includes: receiving the user's viewing angle direction data; according to the viewing angle direction data, using a predetermined three-dimensional mapping model, to obtain the three-dimensional mapping model and The coordinate range of the region irrelevant to the user's visual angle; according to the coordinate range determined in the three-dimensional mapping model, determine the discarded image area corresponding to the coordinate range in the panoramic video image to be output; remove the discarded image from the panoramic video image to be output The image data of the remaining area outside the area is transmitted to the video image output terminal. The embodiments of the present invention solve the problems in the prior art that there is either a large consumption or a serious video freeze when playing a panoramic video.

Description

Method and device for controlling video playback

technical field

The present invention relates to the field of multimedia technology, in particular to a method and device for controlling video playback.

Background technique

When performing virtual reality (VR, Virtual Reality) or 360-degree panoramic video live broadcast, the live broadcast picture can be changed correspondingly according to the user's perspective. However, usually, the picture viewed by the user at one moment has only a certain angle, and the angle is generally within the range of 80 to 110 degrees between the horizontal field of view and the vertical field of view, that is to say, the picture viewed by the user at one moment Only a small part of the 360-degree panoramic image is occupied, and the occupied part is about 1/8 to 1/6 of the 360-degree panoramic image, while other parts of the 360-degree image are invisible at the same time.

At present, the following two methods are usually used in the prior art for VR or 360-degree panoramic live broadcast:

First, the server splices the scene into a 360-degree panoramic video image according to the algorithm, and transmits the video data to the client through video encoding. After decoding, the client performs spherical mapping on the 360-degree image, maps the image to the ball, and then Data (user's viewing angle direction data) to control the display of video images, that is, to control the display of images within the range of viewing angles. However, this method consumes a lot of energy, and there are problems such as high resolution of 360-degree panoramic video images, high encoding power consumption, large number of rendering cameras, large amount of calculation for splicing algorithms, and high transmission bit rate.

Second, the client sends the gyroscope data to the server, and the server only renders the image within the field of view according to the gyroscope data, and sends the image code to the client, and the client directly displays the image within the field of view. However, in this method, it takes a long time for the gyroscope data to go from the client to the server, and after the image is encoded, from the server to the client, so it is easy to cause serious video freezes when the field of view changes.

To sum up, in the prior art, when playing a panoramic video image, there are problems of high consumption or serious video freeze.

Contents of the invention

The purpose of the present invention is to provide a method and device for controlling video playback, so as to solve the problems of large consumption and serious video freeze in the prior art when performing panoramic video playback.

In the first aspect, the embodiment of the present invention provides a control method for video playback, which is applied to the server side, including:

Receive the user's view direction data;

According to the viewing angle direction data, using a predetermined three-dimensional mapping model, acquiring a coordinate range of an area in the three-dimensional mapping model that is not related to the viewing angle of the user;

According to the coordinate range determined in the three-dimensional mapping model, determine a discarded image area corresponding to the coordinate range in the panoramic video image to be output;

The image data of the remaining area in the panoramic video image to be output except the discarded image area is transmitted to the video image output terminal.

Optionally, the step of obtaining the coordinate range of an area in the three-dimensional mapping model that is not related to the user's viewing angle by using a predetermined three-dimensional mapping model according to the viewing angle direction data includes: according to the viewing angle Direction data, determining the position coordinates of the user's current viewpoint on the three-dimensional mapping model; according to the position coordinates of the current viewpoint on the three-dimensional mapping model, determining the position of the lowest relevant point of the image on the three-dimensional mapping model Coordinates, wherein the least relevant point of the image is the intersection of the extension line of the line connecting the current viewpoint and the central point of the three-dimensional mapping model with the three-dimensional mapping model; according to the position coordinates of the lowest relevant point of the image, Acquire the coordinate range of an area in the three-dimensional mapping model that is not related to the viewing angle of the user.

Optionally, the step of obtaining the coordinate range of an area in the 3D mapping model that is not related to the user's viewing angle according to the position coordinates of the lowest correlation point of the image includes: The visible angle of view above and the position coordinates of the lowest correlation point of the image, determine the coordinate range of the area in the three-dimensional mapping model that is not related to the user's visual angle; or, according to the position coordinates of the lowest correlation point of the image and the preset The specified offset range is used to determine the coordinate range of an area in the three-dimensional mapping model that is not related to the viewing angle of the user.

Optionally, the three-dimensional mapping model includes a three-dimensional sphere model; the step of determining the discarded image area corresponding to the coordinate range in the panoramic video image to be output according to the coordinate range determined in the three-dimensional mapping model includes : According to the mapping relationship between the panoramic video image and the three-dimensional spherical model and the coordinate range, determine a discarded image area corresponding to the coordinate range in the panoramic video image to be output.

Optionally, the position coordinates of the lowest correlation point of the image include the latitude and longitude of the lowest correlation point of the image; the coordinate range includes: a range of latitude coordinates and a range of longitude coordinates centered on the lowest correlation point of the image; The mapping relationship between the panoramic video image and the three-dimensional spherical model and the coordinate range, the step of determining the discarded image area corresponding to the coordinate range in the panoramic video image to be output includes: according to the panoramic video image and The mapping relationship of the three-dimensional sphere model, the latitude of the lowest correlation point of the image, and the latitude coordinate range are obtained to obtain the height range of the discarded image area corresponding to the coordinate range in the panoramic video image to be output; according to the The mapping relationship between the panoramic video image and the three-dimensional spherical model, the longitude of the least relevant point of the image, and the longitude coordinate range to obtain the discarded image area corresponding to the coordinate range in the panoramic video image to be output width range.

Optionally, the panoramic video image is a rectangular image or an elliptical image; if the mapping relationship between the panoramic video image and the three-dimensional spherical model includes: the height of the panoramic video image is mapped to the diameter of the three-dimensional spherical model , then: according to the mapping relationship between the panoramic video image and the three-dimensional spherical model, the latitude of the lowest correlation point of the image, and the latitude coordinate range, the coordinates in the panoramic video image to be output are obtained The step of discarding the height range of the image area corresponding to the range includes:

When α+β/2<=π/2 and α-β/2>=-π/2:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

When α+β/2>π/2:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

T11=0;

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

When α-β/2<-π/2:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

B12=H

and / or,

If the mapping relationship between the panoramic video image and the three-dimensional spherical model includes: the height of the panoramic video image is mapped to the half circumference of the longitude circle of the three-dimensional spherical model, then:

According to the mapping relationship between the panoramic video image and the three-dimensional sphere model, the latitude of the lowest correlation point of the image, and the latitude coordinate range, the panoramic video image to be output corresponding to the coordinate range is obtained. Steps for discarding height ranges of image regions, including:

When α+β/2<=π/2 and α-β/2>=-π/2:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

When α+β/2>π/2:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

T11=0;

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

When α-β/2<-π/2:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

B12=H

Wherein, α represents the latitude of the lowest correlation point of the image, β represents the angle value corresponding to the latitude coordinate range, H represents the height of the panoramic video image, T1 represents the upper boundary value of the height range of the discarded image area, B1 represents the lower boundary value of the height range of the discarded image region, T11 represents the upper boundary value of the height range of the first region in the discarded image region, and B11 represents the lower boundary value of the height range of the first region in the discarded image region Boundary value, T12 represents the upper boundary value of the height range of the second area in the discarded image area, and B12 represents the lower boundary value of the height range of the second area in the discarded image area.

Optionally, the panoramic video image is a rectangular image; the mapping relationship between the panoramic video image and the three-dimensional spherical model includes: mapping the width of the panoramic video image to the equatorial perimeter of the three-dimensional spherical model, then: According to the mapping relationship between the panoramic video image and the three-dimensional sphere model, the longitude of the lowest correlation point of the image, and the longitude coordinate range, the panoramic video image to be output corresponding to the coordinate range is obtained. Steps for discarding width ranges of image regions, including:

When δ-γ/2>=0 and δ+γ/2<=2π:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

The right boundary value of the width range of the discarded image area is obtained by the following formula:

When δ-γ/2<0:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

L11=0

The right boundary value of the width range of the discarded image area is obtained by the following formula:

R12=W

When δ+γ/2>2π:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

L11=0

The right boundary value of the width range of the discarded image area is obtained by the following formula:

R12=W

Among them, δ represents the longitude of the lowest correlation point of the image, γ represents the angle value corresponding to the longitude coordinate range, W represents the width of the panoramic video image, L1 represents the left boundary value of the width range of the discarded image area, and R1 represents The right boundary value of the width range of the discarded image area, L11 represents the left boundary value of the width range of the first area in the discarded image area, and R11 represents the right boundary value of the width range of the first area in the discarded image area , L12 represents the left boundary value of the width range of the second area in the discarded image area, and R12 represents the right boundary value of the width range of the second area in the discarded image area.

Optionally, the panoramic video image is an elliptical image; the mapping relationship between the panoramic video image and the three-dimensional spherical model includes: the width of the panoramic video image is mapped to the equatorial perimeter of the three-dimensional spherical model, then : According to the mapping relationship between the panoramic video image and the three-dimensional spherical model, the longitude of the lowest correlation point of the image, and the longitude coordinate range, obtain the corresponding coordinate range in the panoramic video image to be output The step of discarding the width range of the image region includes:

When δ-γ/2>=0 and δ+γ/2<=2π:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

The right boundary value of the width range of the discarded image area is obtained by the following formula:

When δ-γ/2<0:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

L11=s

The right boundary value of the width range of the discarded image area is obtained by the following formula:

R12=s+w

When δ+γ/2>2π:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

L11=s

The right boundary value of the width range of the discarded image area is obtained by the following formula:

R12=s+w

Among them, δ represents the longitude of the lowest correlation point of the image, γ represents the angle value corresponding to the longitude coordinate range, s represents the starting point abscissa of each row of images in the panoramic video image, and w represents the width of each row of images in the panoramic video image , L1 represents the left boundary value of the width range of the discarded image region, R1 represents the right boundary value of the width range of the discarded image region, L11 represents the left boundary value of the width range of the first region in the discarded image region, R11 represents the right boundary value of the width range of the first area in the discarded image area, L12 represents the left boundary value of the width range of the second area in the discarded image area, and R12 represents the width of the second area in the discarded image area The right boundary value of the width range.

Optionally, the step of transmitting the image data of the remaining area in the panoramic video image to be output except the discarded image area to the video image output terminal includes: obtaining the Render the remaining area in the panoramic video image except the discarded image area; transmit the rendered image data to a video image output terminal; or render the panoramic video image; Setting the data corresponding to the discarded image area in the rendered image data to a specific value, and encoding the set image data; transmitting the encoded image data to a video image output terminal.

Optionally, the step of transmitting the image data of the remaining area in the panoramic video image to be output except the discarded image area to the video image output terminal includes: transmitting the image data of the remaining area and the The viewing angle direction data is transmitted to the video image output terminal, so that the video image output terminal determines the discarded image area of the panoramic video image according to the viewing angle direction data.

In a second aspect, an embodiment of the present invention provides a method for controlling video playback, which is applied to a video image output terminal, including:

Obtaining the viewing direction data of the user in real time, and sending the viewing direction data to the server;

Receive the image data of the remaining area except the discarded image area in the panoramic video image transmitted by the server;

mapping the image data of the remaining area into a predetermined three-dimensional mapping model;

The image part corresponding to the user's viewing angle is acquired from the mapped three-dimensional mapping model, and the image part corresponding to the user's viewing angle is played.

Optionally, the step of receiving the image data of the remaining area of the panoramic video image transmitted from the server side except the discarded image area includes: receiving the image data of the panoramic video image transmitted from the server side except the discarded image area The image data of the remaining areas outside and the viewing angle direction data corresponding to the image data of the remaining areas; after playing the image part, the control method further includes: determining the mapped three-dimensional mapping model according to the viewing angle direction data Discard the coordinate range of the image area as described in the above; obtain the user's current viewing angle direction data in real time, and determine whether the image part corresponding to the user's viewing angle in the mapped three-dimensional mapping model falls into the The coordinate range of the discarded image area; wherein, if the image part corresponding to the user's viewing angle in the mapped three-dimensional mapping model falls within the coordinate range of the discarded image area, then from the previous mapped, the latest In the three-dimensional mapping model including the image part corresponding to the discarded image area, the image part of the discarded image area corresponding to the viewing angle of the user is obtained, and the acquired image part of the discarded image area is used for display; or, from the mapping In the following three-dimensional mapping model, the image part of the adjacent area of the discarded image area related to the user's current viewing angle is obtained, and the image part of the adjacent area of the discarded image area is used as the image part of the adjacent area of the discarded image area as the The part of the image corresponding to the current viewing angle is displayed.

In the third aspect, the embodiment of the present invention provides a video playback control device, which is applied to the server end, including:

The first receiving module is used to receive the viewing direction data of the user;

An acquisition module, configured to acquire a coordinate range of an area in the three-dimensional mapping model that is not related to the user's viewing angle by using a predetermined three-dimensional mapping model according to the viewing angle direction data;

A determining module, configured to determine a discarded image area corresponding to the coordinate range in the panoramic video image to be output according to the coordinate range determined in the three-dimensional mapping model;

The output module is used to transmit the image data of the remaining area in the panoramic video image to be output except the discarded image area to the video image output terminal.

Optionally, the acquiring module includes: a first determining unit, configured to determine the position coordinates of the user's current viewpoint on the three-dimensional mapping model according to the viewing angle direction data; The position coordinates of the current viewpoint on the three-dimensional mapping model, determining the position coordinates of the lowest correlation point of the image on the three-dimensional mapping model, wherein the lowest correlation point of the image is the distance between the current viewpoint and the three-dimensional mapping model The intersection point of the extension line of the connecting line of the central point and the three-dimensional mapping model; the acquisition unit is used to acquire the position coordinates of the lowest relevant point of the image in the three-dimensional mapping model that is not related to the current viewing angle of the user The coordinate extent of the region.

Optionally, the acquiring unit is configured to, according to the user's viewing angle on the 3D mapping model and the position coordinates of the lowest correlation point of the image, determine an area in the 3D mapping model that is not related to the user's viewing angle or, according to the position coordinates of the lowest relevant point of the image and a preset offset range, determine the coordinate range of an area in the three-dimensional mapping model that is not related to the viewing angle of the user.

Optionally, the three-dimensional mapping model includes a three-dimensional spherical model; the determining module is configured to, according to the mapping relationship between the panoramic video image and the three-dimensional spherical model and the coordinate range, determine The discarded image region corresponding to the coordinate range.

Optionally, the position coordinates of the lowest correlation point of the image include the latitude and longitude of the lowest correlation point of the image; the coordinate range includes: a range of latitude coordinates and a range of longitude coordinates centered on the lowest correlation point of the image; the determining The module includes: a third determining unit, configured to obtain the panoramic video image to be output according to the mapping relationship between the panoramic video image and the three-dimensional spherical model, the latitude of the lowest correlation point of the image, and the latitude coordinate range The height range of the discarded image area corresponding to the coordinate range; the fourth determining unit is used for according to the mapping relationship between the panoramic video image and the three-dimensional spherical model, the longitude of the lowest correlation point of the image, and the longitude The coordinate range is used to obtain the width range of the discarded image area corresponding to the coordinate range in the to-be-output panoramic video image.

Optionally, the panoramic video image is a rectangular image or an elliptical image; if the mapping relationship between the panoramic video image and the three-dimensional spherical model includes: the height of the panoramic video image is mapped to the diameter of the three-dimensional spherical model , then the third determining unit is used for:

When α+β/2<=π/2 and α-β/2>=-π/2:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

When α+β/2>π/2:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

T11=0;

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

When α-β/2<-π/2:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

B12=H

and / or,

If the mapping relationship between the panoramic video image and the three-dimensional spherical model includes: the height of the panoramic video image is mapped to the half circumference of the longitude circle of the three-dimensional spherical model, then the third determination unit is used for:

When α+β/2<=π/2 and α-β/2>=-π/2:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

When α+β/2>π/2:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

T11=0;

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

When α-β/2<-π/2:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

B12=H

Wherein, α represents the latitude of the lowest correlation point of the image, β represents the angle value corresponding to the latitude coordinate range, H represents the height of the panoramic video image, T1 represents the upper boundary value of the height range of the discarded image area, B1 represents the lower boundary value of the height range of the discarded image region, T11 represents the upper boundary value of the height range of the first region in the discarded image region, and B11 represents the lower boundary value of the height range of the first region in the discarded image region Boundary value, T12 represents the upper boundary value of the height range of the second area in the discarded image area, and B12 represents the lower boundary value of the height range of the second area in the discarded image area.

Optionally, the panoramic video image is a rectangular image; the mapping relationship between the panoramic video image and the three-dimensional spherical model includes: the width of the panoramic video image is mapped to the equatorial perimeter of the three-dimensional spherical model, then the The fourth determining unit is used for:

When δ-γ/2>=0 and δ+γ/2<=2π:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

The right boundary value of the width range of the discarded image area is obtained by the following formula:

When δ-γ/2<0:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

L11=0

The right boundary value of the width range of the discarded image area is obtained by the following formula:

R12=W

When δ+γ/2>2π:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

L11=0

The right boundary value of the width range of the discarded image area is obtained by the following formula:

R12=W

Wherein, δ represents the longitude of the lowest correlation point of the image, γ represents the angle value corresponding to the longitude coordinate range, W represents the width of the panoramic video image, L1 represents the left boundary value of the width range of the discarded image region, R1 represents the right boundary value of the width range of the discarded image region, L11 represents the left boundary value of the width range of the first region in the discarded image region, and R11 represents the right boundary value of the width range of the first region in the discarded image region Boundary value, L12 represents the left boundary value of the width range of the second area in the discarded image area, and R12 represents the right boundary value of the width range of the second area in the discarded image area.

Optionally, the panoramic video image is an elliptical image; the mapping relationship between the panoramic video image and the three-dimensional spherical model includes: the width of the panoramic video image is mapped to the equatorial perimeter of the three-dimensional spherical model, then The fourth determining unit is used for:

When δ-γ/2>=0 and δ+γ/2<=2π:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

The right boundary value of the width range of the discarded image area is obtained by the following formula:

When δ-γ/2<0:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

L11=s

The right boundary value of the width range of the discarded image area is obtained by the following formula:

R12=s+w

When δ+γ/2>2π:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

L11=s

The right boundary value of the width range of the discarded image area is obtained by the following formula:

R12=s+w

Among them, δ represents the longitude of the lowest correlation point of the image, γ represents the angle value corresponding to the longitude coordinate range, s represents the starting point abscissa of each row of images in the panoramic video image, and w represents the width of each row of images in the panoramic video image , L1 represents the left boundary value of the width range of the discarded image region, R1 represents the right boundary value of the width range of the discarded image region, L11 represents the left boundary value of the width range of the first region in the discarded image region, R11 represents the right boundary value of the width range of the first area in the discarded image area, L12 represents the left boundary value of the width range of the second area in the discarded image area, and R12 represents the width of the second area in the discarded image area The right boundary value of the width range.

Optionally, the output module includes: a first output unit, configured to obtain, according to the discarded image region, remaining regions in the panoramic video image other than the discarded image region, and perform processing on the remaining regions rendering; transmitting the image data obtained after rendering to the video image output terminal; or, a second output unit for rendering the panoramic video image; combining the image data obtained after rendering with the discarded image area The corresponding data is set to a specific value, and the set image data is encoded; the encoded image data is transmitted to the video image output terminal.

Optionally, the output module is configured to transmit the image data of the remaining area and the viewing angle direction data to a video image output terminal, so that the video image output terminal determines the viewing angle direction data according to the viewing angle direction data. The discarded image area of the panoramic video image.

In a fourth aspect, an embodiment of the present invention provides a video playback control device, which is applied to a video image output terminal, including:

A sending module, configured to acquire the user's viewing angle direction data in real time, and send the viewing angle direction data to the server;

The second receiving module is used to receive the image data of the remaining area except the discarded image area in the panoramic video image transmitted by the server;

a mapping module, configured to map the image data of the remaining area into a predetermined three-dimensional mapping model;

A playing module, configured to acquire the image part corresponding to the user's viewing angle from the mapped three-dimensional mapping model, and play the image part corresponding to the user's viewing angle.

Optionally, the second receiving module is configured to receive image data of remaining areas in the panoramic video image transmitted from the server except the discarded image area and viewing angle direction data corresponding to the image data of the remaining areas; The control device further includes: a coordinate range determining module, configured to determine the coordinate range of the discarded image area in the mapped three-dimensional mapping model according to the viewing angle direction data after the playing module plays the image part; a judging module , used to obtain the user's current viewing angle direction data in real time, and determine whether the image part corresponding to the user's viewing angle in the mapped three-dimensional mapping model falls into the coordinates of the discarded image area according to the current viewing angle direction data obtained in real time Range; Wherein, if the image part corresponding to the user's visual angle in the mapped three-dimensional mapping model falls within the coordinate range of the discarded image area, then from the previous mapping, the most recent one contains the image corresponding to the discarded image area In the three-dimensional mapping model of the image part of the user, the image part of the discarded image area corresponding to the user's viewing angle is obtained, and the image part of the discarded image area obtained is displayed; or, from the mapped three-dimensional mapping model, Acquire the image part of the adjacent area of the discarded image area related to the user's current viewing angle, and use the image part of the adjacent area of the discarded image area as the image part corresponding to the user's current viewing angle to display.

In the fifth aspect, the embodiment of the present invention also provides a computer-readable storage medium for storing a computer program for video playback, and the computer program can be executed by a processor as described in any one of the first aspect. A method for controlling video playback on the server side.

In the sixth aspect, the embodiment of the present invention also provides a computer-readable storage medium for storing a computer program for video playback, and the computer program can be executed by a processor as described in any one of the second aspect. A method for controlling video playback of a video image output terminal.

In a seventh aspect, an embodiment of the present invention provides an electronic device, the electronic device includes one or more processors, and the processors are configured to execute the following control method:

Receive the user's view direction data;

According to the viewing angle direction data, using a predetermined three-dimensional mapping model, acquiring a coordinate range of an area in the three-dimensional mapping model that is not related to the viewing angle of the user;

According to the coordinate range determined in the three-dimensional mapping model, determine a discarded image area corresponding to the coordinate range in the panoramic video image to be output;

The image data of the remaining area in the panoramic video image to be output except the discarded image area is transmitted to the video image output terminal.

In an eighth aspect, an embodiment of the present invention further provides an electronic device, the electronic device includes one or more processors, and the processors are configured to execute the following control method:

Obtaining the viewing direction data of the user in real time, and sending the viewing direction data to the server;

Receive the image data of the remaining area except the discarded image area in the panoramic video image transmitted by the server;

mapping the image data of the remaining area into a predetermined three-dimensional mapping model;

The image part corresponding to the user's viewing angle is acquired from the mapped three-dimensional mapping model, and the image part corresponding to the user's viewing angle is played.

The beneficial effects of the present invention are:

In the embodiment of the present invention, by receiving the viewing direction data of the user, and according to the viewing direction data, using a predetermined three-dimensional mapping model, the coordinate range of the area unrelated to the user's viewing angle in the three-dimensional mapping model is obtained, and according to the three-dimensional Map the coordinate range determined in the model, determine the discarded image area corresponding to the coordinate range in the panoramic video image to be output, and finally transmit the image data of the remaining area in the panoramic video image to be output except the discarded image area to the video Image output terminal. In this way, the embodiment of the present invention only transmits the image data of the remaining area in the panoramic video image to be output except the discarded image area to the video image output terminal, which reduces the overall image data compared to the transmission of all the image data of the 360-degree panoramic video image. The number of splicing 360-degree panoramic video image cameras reduces the number of splicing algorithm processing, reduces the processing time of video image encoding, and reduces the bit rate under the same parameters, thereby reducing the consumption of video transmission; in addition, due to The image data of the remaining area in the panoramic video image except the discarded image area is transmitted to the video image output terminal, so compared with the method of only transmitting the image data within the user's visual angle range, it reduces the jam caused by the image data transmission delay This solves the problems of large consumption and serious video freeze in the prior art when playing panoramic video.

Description of drawings

Fig. 1 represents the flow chart of the steps of the control method of video playback in the first embodiment of the present invention;

Fig. 2 represents the flow chart of the steps of the control method of video playback in the second embodiment of the present invention;

Fig. 3 shows the schematic diagram of the position between the current viewpoint and the lowest correlation point of the image in the three-dimensional spherical model;

Fig. 4 shows the schematic diagram of the latitude section of the three-dimensional spherical model;

Fig. 5 represents the display diagram of the height range of the discarded image area in the rectangular image;

Fig. 6 represents the display diagram of the height range of the discarded image area in the oval image;

Fig. 7 represents the display diagram of the height range of the first area and the second area of the discarded image area in the oval image;

Fig. 8 shows the schematic diagram of the longitude section of the three-dimensional spherical model;

Fig. 9 represents the display diagram of the width range of the first region and the second region of the discarded image region in the rectangular image;

Fig. 10 represents the display diagram of the width range of the first region and the second region of the discarded image region in the oval image;

Fig. 11 shows the flow chart of the steps of the control method of video playback in the third embodiment of the present invention;

Fig. 12 shows the structural block diagram of the control device of video playback in the fourth embodiment of the present invention;

FIG. 13 shows a structural block diagram of a device for controlling video playback in a fifth embodiment of the present invention.

Detailed ways

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

First embodiment:

As shown in Figure 1, it is a flow chart of the steps of the control method for video playback in the first embodiment of the present invention, the control method is applied to the server side, including:

Step 101, receiving viewing direction data of a user.

In this step, the server end may receive viewing angle direction data transmitted in real time by the video image output terminal. Specifically, the view direction data may be gyroscope data, and the gyroscope data may be used to indicate the user's viewing posture, that is, to indicate the user's view direction. In this way, when the server receives the gyroscope data transmitted by the video image output terminal in real time, it can obtain the user's view direction data in real time.

Step 102 , according to the view direction data, using a predetermined 3D mapping model, to obtain a coordinate range of an area in the 3D mapping model that is not related to the user's viewing angle.

In this step, specifically, after receiving the viewing direction data of the user, the server may use a predetermined 3D mapping model to obtain the coordinate range of an area not related to the user's viewing angle in the 3D mapping model. Specifically, the three-dimensional mapping model may include a three-dimensional sphere model, a three-dimensional solid model, and the like. In addition, specifically, since the viewing angle direction data can be used to indicate the user's viewing posture, that is, indicate the user's current viewpoint, the viewing angle of the user can also be obtained according to the viewing angle direction data. Specifically, the viewing angle is centered on the current viewpoint.

Step 103, according to the coordinate range determined in the three-dimensional mapping model, determine the discarded image area corresponding to the coordinate range in the panoramic video image to be output.

In this step, specifically, after obtaining the coordinate range of an area not related to the user's viewing angle in the 3D mapping model, the coordinate range in the panoramic video image to be output can be determined according to the coordinate range determined in the 3D mapping model. The range corresponds to the discarded image area.

Step 104, transmit the image data of the remaining area except the discarded image area in the panoramic video image to be output to the video image output terminal.

In this step, specifically, after determining the discarded image area corresponding to the coordinate range in the panoramic video image to be output, the image data of the remaining area in the panoramic video image except the discarded image area can be obtained, and the remaining area The image data is transmitted to the video image output terminal.

In this way, this embodiment uses the predetermined 3D mapping model to obtain the coordinate range of an area not related to the user's viewing angle in the 3D mapping model according to the user's viewing angle direction data, and according to the coordinate range determined in the 3D mapping model, Determine the discarded image area corresponding to the coordinate range in the panoramic video image to be output, and finally transmit the image data of the remaining area except the discarded image area in the panoramic video image to be output to the video image output terminal. Compared with the way of transmitting all the data of the 360-degree panoramic video image in the prior art, this can reduce the amount of video image transmission data without affecting the user's normal viewing of the video image within the viewing angle range, thereby reducing splicing. The number of 360-degree panoramic video image cameras reduces the number of splicing algorithm processing, reduces the processing time of video image encoding, and reduces the bit rate under the same parameters, thereby reducing the consumption of video transmission; in addition, because the panoramic video The image data of the remaining area in the image except the discarded image area is transmitted to the video image output terminal, so compared with the method of only transmitting image data within the user's visual angle range, the problem of stuttering caused by image data transmission delay is reduced .

Second embodiment:

As shown in Figure 2, it is a flow chart of the steps of the control method for video playback in the second embodiment of the present invention, the control method is applied to the server side, including:

Step 201, receiving viewing direction data of a user.

In this step, the server end may receive viewing angle direction data transmitted in real time by the video image output terminal. Specifically, the view direction data may be gyroscope data, and the gyroscope data may be used to indicate the user's viewing posture, that is, to indicate the user's view direction. In this way, when the server receives the gyroscope data transmitted by the video image output terminal in real time, it can obtain the user's view direction data in real time.

Step 202: Determine the position coordinates of the user's current viewpoint on the three-dimensional mapping model according to the view direction data.

In this step, specifically, the user's view direction data is data indicating the user's current view point. After receiving the viewing direction data of the user, the server may utilize a predetermined 3D mapping model to map the current viewpoint onto the 3D mapping model, thereby determining the position coordinates of the user's current viewing point on the 3D mapping model. Specifically, the position coordinates of the current viewpoint on the three-dimensional mapping model include latitude and longitude coordinates.

Step 203, according to the position coordinates of the current viewpoint on the 3D mapping model, determine the position coordinates of the lowest relevant point of the image on the 3D mapping model.

In this step, specifically, the lowest relevant point of the image is the intersection of the extension line of the line connecting the current viewpoint and the center point of the three-dimensional mapping model and the three-dimensional mapping model. In this way, after the position coordinates of the current viewpoint are determined, the position coordinates of the lowest relevant point of the image on the three-dimensional mapping model can be determined according to the position coordinates of the current viewpoint. Specifically, the position coordinates of the lowest relevant point of the image include longitude coordinates and latitude coordinates of the lowest relevant point of the image.

Specifically, the three-dimensional mapping model may be used as a three-dimensional sphere model for specific description herein. As shown in FIG. 3 , it is a schematic diagram of the position between the current viewpoint and the lowest correlation point of the image in the three-dimensional spherical model. In Fig. 3, point A is the current viewpoint, and point B is the intersection point of the extension line connecting the current viewpoint and the center point of the three-dimensional mapping model with the three-dimensional mapping model, that is, the lowest correlation point of the image.

Step 204, according to the position coordinates of the lowest relevant point of the image, obtain the coordinate range of the area not related to the user's visual angle in the 3D mapping model.

In this step, specifically, after determining the position coordinates of the current viewpoint on the three-dimensional mapping model, the user's viewing angle range can be obtained according to the position coordinates of the current viewpoint on the three-dimensional mapping model, wherein the viewing angle range is represented by the current Viewpoint is the center.

Specifically, when obtaining the coordinate range of an area not related to the user's viewing angle in the 3D mapping model according to the position coordinates of the lowest correlation point of the image, the user's viewing angle on the 3D mapping model and the minimum correlation point of the image can be Position coordinates, which determine the coordinate range of an area in the 3D mapping model that is not related to the user's viewing angle. Specifically, the coordinate range of the area not related to the user's viewing angle may be the coordinate range of the area corresponding to the angle range of the viewing angle with the lowest relevant point of the image as the center. Specifically referring to FIG. 3 , assuming that a is a viewing angle, b is a coordinate range of an area not related to the user's viewing angle.

In addition, when obtaining the coordinate range of an area not related to the user's viewing angle in the 3D mapping model according to the position coordinates of the lowest relevant point of the image, it can also be based on the position coordinates of the lowest relevant point of the image and the preset offset range , to determine the coordinate range of the area in the 3D mapping model that is not related to the viewing angle of the user. Specifically, after the offset range is set in advance, the offset can be performed centering on the lowest relevant point of the image, so as to obtain the coordinate range of an area that is not related to the user's viewing angle. In addition, specifically, the offset range may be determined according to the movement law of the human head. For example, when a person's head turns horizontally, the video image output terminal will send gyroscope data (viewing direction data) to the server in real time. At this time, as long as the head is turned at the maximum speed, the server will return the corresponding viewing angle data at the current moment. Before the image data of the previous moment, the head has not turned to the discarded image area corresponding to the angle of view direction data at the previous moment. For example, in VR experience, it takes about 150ms for the human head to turn from the front to the horizontal rotation of 90 degrees. At this time, the return time of the video image is controlled within 100ms, so that the head has not turned to the angle of view direction data at the previous moment. When within the corresponding discarded image area, the image data corresponding to the viewing angle direction data at the current moment has been transmitted to the video image output terminal.

Step 205, according to the mapping relationship between the panoramic video image and the three-dimensional spherical model and the coordinate range, determine the discarded image area corresponding to the coordinate range in the panoramic video image to be output.

In this step, specifically, the three-dimensional mapping model includes a three-dimensional sphere model. At this time, according to the coordinate range determined in the three-dimensional mapping model, when determining the discarded image area corresponding to the coordinate range in the panoramic video image to be output, it can be based on The mapping relationship between the panoramic video image and the three-dimensional spherical model and the coordinate range determine the discarded image area corresponding to the coordinate range in the panoramic video image to be output.

Wherein, specifically, the mapping relationship between the panoramic video image and the three-dimensional spherical model may include that the height of the panoramic video image is mapped to the diameter of the three-dimensional spherical model, the height of the panoramic video image is mapped to the half circumference of the longitude circle of the three-dimensional spherical model, and the panoramic video image The width of is mapped to the equatorial perimeter of the 3D sphere model. In addition, during mapping, the upper left of the panoramic video image can be used as a starting point for the height and width of the panoramic video image.

In addition, the position coordinates of the lowest correlation point of the image include the latitude and longitude of the lowest correlation point of the image; the coordinate range includes: the range of latitude coordinates and the range of longitude coordinates centered on the lowest correlation point of the image; When determining the discarded image area corresponding to the coordinate range in the panoramic video image to be output, the mapping relationship and the coordinate range can be obtained according to the mapping relationship between the panoramic video image and the three-dimensional spherical model, the latitude of the lowest correlation point of the image, and the latitude coordinate range. In the output panoramic video image, the height range of the discarded image area corresponding to the coordinate range; The width range of the discarded image region corresponding to the coordinate range.

The following describes in detail the obtained height range and width range of the discarded image region corresponding to the coordinate range in the panoramic video image to be output.

One, when the panoramic video image is a rectangular image or an elliptical image, and the mapping relationship between the panoramic video image and the three-dimensional spherical model includes: when the height of the panoramic video image is mapped to the diameter of the three-dimensional spherical model, according to the panoramic video image and the three-dimensional spherical model The mapping relationship, the latitude and the latitude coordinate range of the lowest correlation point of the image, the height range of the discarded image area corresponding to the coordinate range in the panoramic video image to be output can include the following situations:

When α+β/2<=π/2 and α-β/2>=-π/2, discarding the image area at this time does not include the south pole and the north pole of the three-dimensional spherical model, then:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

When α+β/2>π/2, the discarded image area includes the North Pole of the three-dimensional sphere model, and the discarded image area includes the first area and the second area, and when the panoramic video image is a rectangle During image, the width of the first area is the width of rectangular image, and when panoramic video image is oval image, the width of first area is the width of every line image in panoramic video image, then:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

T11=0;

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

When α-β/2<-π/2, discarding the image area now includes the south pole of the three-dimensional spherical model, and discarding the image area now includes the first area and the second area, and when the panoramic video image is During the rectangular image, the width of the second area is the width of the rectangular image, and when the panoramic video image is an ellipse image, the width of the second area is the width of every row of images in the panoramic video image, then:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

B12=H

and / or,

If the mapping relationship between the panoramic video image and the three-dimensional spherical model includes: the height of the panoramic video image is mapped to the semicircumference of the longitude circle of the three-dimensional spherical model, then: according to the mapping relationship between the panoramic video image and the three-dimensional spherical model, the latitude of the lowest correlation point of the image And the latitude coordinate range, when obtaining the height range of the discarded image area corresponding to the coordinate range in the panoramic video image to be output, can include the following several situations:

When α+β/2<=π/2 and α-β/2>=-π/2, discarding the image area at this time does not include the south pole and the north pole of the three-dimensional spherical model, then:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

When α+β/2>π/2, the discarded image area includes the North Pole of the three-dimensional sphere model, and the discarded image area includes the first area and the second area, and when the panoramic video image is a rectangle During the image, the width of the first area is the width of the rectangular image, and when the panoramic video image is an ellipse image, the width of the first area is the width of every row of images in the panoramic video image, then:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

T11=0;

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

When α-β/2<-π/2, discarding the image area now includes the south pole of the three-dimensional spherical model, and discarding the image area now includes the first area and the second area, and when the panoramic video image is In the case of a rectangular image, the width of the second area is the width of the rectangular image, and when the panoramic video image is an elliptical image, the width of the second area is the width of each row of images in the panoramic video image, then:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

B12=H

Among them, α represents the latitude of the lowest correlation point of the image, β represents the angle value corresponding to the latitude coordinate range, H represents the height of the panoramic video image, T1 represents the upper boundary value of the height range of the discarded image region, and B1 represents the height range of the discarded image region T11 indicates the upper boundary value of the height range of the first area in the discarded image area, B11 indicates the lower boundary value of the height range of the first area in the discarded image area, and T12 indicates the height of the second area in the discarded image area The upper boundary value of the range, B12 indicates the lower boundary value of the height range of the second area in the discarded image area.

Specifically, as shown in FIG. 4, point B in FIG. 4 represents the lowest correlation point of the image, α represents the latitude of the lowest correlation point of the image, and β represents the angle value corresponding to the latitude coordinate range. In addition, after obtaining the upper boundary value and the lower boundary value of the height range of the discarded image area, if the panoramic video image is a rectangular image, the display diagram of the height range of the discarded image area as shown in Figure 5 can be obtained, specifically , in Fig. 5, the vertical region between T1 to B1 represents the height range of the discarded image region; if the panoramic video image is an elliptical image, then the display diagram of the height range of the discarded image region as shown in Fig. 6 can be obtained Specifically, in FIG. 6 , the vertical area between T1 and B1 represents the height range of the discarded image area. In addition, if the discarded image area includes the first area and the second area, the upper boundary value and the lower boundary value of the height range of the first area in the discarded image area and the upper boundary value and the height range of the second area can be obtained respectively. After the lower boundary value, if the panoramic video image is an elliptical image, you can obtain the display diagram of the height range of the first area and the second area of the discarded image area as shown in Figure 7, specifically, in Figure 7, The vertical area between B11 and the upper boundary line (ie, the W axis) indicates the height range of the first area, and the vertical area between T12 and B12 indicates the height range of the second area.

Second, when the panoramic video image is a rectangular image, and the mapping relationship between the panoramic video image and the three-dimensional spherical model includes: when the width of the panoramic video image is mapped to the equatorial perimeter of the three-dimensional spherical model, according to the mapping relationship between the panoramic video image and the three-dimensional spherical model, The longitude and longitude coordinate range of the lowest correlation point of the image are obtained to obtain the width range of the discarded image area corresponding to the coordinate range in the panoramic video image to be output, which may include the following methods:

When δ-γ/2>=0 and δ+γ/2<=2π:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

The right boundary value of the width range of the discarded image area is obtained by the following formula:

When δ-γ/2<0:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

L11=0

The right boundary value of the width range of the discarded image area is obtained by the following formula:

R12=W

When δ+γ/2>2π:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

L11=0

The right boundary value of the width range of the discarded image area is obtained by the following formula:

R12=W

Among them, δ represents the longitude of the lowest correlation point of the image, γ represents the angle value corresponding to the longitude coordinate range, W represents the width of the panoramic video image, L1 represents the left boundary value of the width range of the discarded image area, and R1 represents the width range of the discarded image area L11 represents the left boundary value of the width range of the first region in the discarded image region, R11 represents the right border value of the width range of the first region in the discarded image region, L12 represents the width of the second region in the discarded image region The left boundary value of the range, R12 indicates the right boundary value of the width range of the second area in the discarded image area.

Specifically, as shown in FIG. 8, point B in FIG. 8 represents the lowest correlation point of the image, δ represents the longitude of the lowest correlation point of the image, and γ represents the angle value corresponding to the longitude coordinate range. In addition, if the discarded image area includes the first area and the second area, then the left boundary value and the right boundary value of the width range of the first area in the discarded image area and the left boundary value and the right boundary value of the width range of the second area can be obtained respectively. After the right boundary value, if the panoramic video image is a rectangular image, then the display diagram of the width range of the first area and the second area of the discarded image area as shown in Figure 9 can be obtained, specifically, in Figure 9, H The horizontal area between the axis and R11 indicates the width range of the first area, and the horizontal area between L12 and the right border of the entire area indicates the width range of the second area.

Third, when the panoramic video image is an elliptical image, and the mapping relationship between the panoramic video image and the three-dimensional spherical model includes: when the width of the panoramic video image is mapped to the equatorial perimeter of the three-dimensional spherical model, according to the mapping relationship between the panoramic video image and the three-dimensional spherical model , the longitude of the lowest correlation point of the image and the longitude coordinate range to obtain the width range of the discarded image area corresponding to the coordinate range in the panoramic video image to be output, which may include the following methods:

When δ-γ/2>=0 and δ+γ/2<=2π:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

The right boundary value of the width range of the discarded image area is obtained by the following formula:

When δ-γ/2<0:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

L11=s

The right boundary value of the width range of the discarded image area is obtained by the following formula:

R12=s+w

When δ+γ/2>2π:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

L11=s

The right boundary value of the width range of the discarded image area is obtained by the following formula:

R12=s+w

Among them, δ represents the longitude of the lowest correlation point of the image, γ represents the angle value corresponding to the longitude coordinate range, s represents the abscissa of the starting point of each row of images in the panoramic video image, w represents the width of each row of images in the panoramic video image, and L1 represents discarding The left boundary value of the width range of the image region, R1 represents the right boundary value of the width range of the discarded image region, L11 represents the left boundary value of the width range of the first region in the discarded image region, R11 represents the value of the first region in the discarded image region The right boundary value of the width range, L12 represents the left boundary value of the width range of the second area in the discarded image area, and R12 represents the right boundary value of the width range of the second area in the discarded image area.

Specifically, as shown in FIG. 10, if the discarded image area includes the first area and the second area, then the left boundary value and right boundary value of the width range of the first area in the discarded image area and the right boundary value of the second area can be respectively obtained. After the left boundary value and the right boundary value of the width range, if the panoramic video image is an elliptical image, the display diagram of the width range of the first area and the second area of the discarded image area as shown in Figure 10 can be obtained, specifically Yes, in FIG. 10 , the horizontal area between the H axis and R11 represents the width range of the first region, and the horizontal region between L12 and the right border of the entire region represents the width range of the second region.

Step 206, transmit the image data of the remaining area and the viewing angle direction data to the video image output terminal.

In this step, specifically, when transmitting the image data of the remaining area except the discarded image data in the panoramic video image to be output to the video image output terminal, the image data of the remaining area and the viewing direction data may be The video image output terminal is transmitted to the video image output terminal, so that the video image output terminal can determine the discarded image area of the panoramic video image according to the viewing angle direction data after receiving the viewing angle direction data.

In addition, specifically, when the image data of the remaining area in the panoramic video image to be output except for the discarded image data is transmitted to the video image output terminal, the discarded image area in the panoramic video image can be obtained according to the discarded image area. and render the remaining area, and then transmit the rendered image data to the video image output terminal; or render the panoramic video image, and then combine the rendered image data with the discarded image area The corresponding data is set to a specific value, and the set image data is encoded, and finally the encoded image data is transmitted to the video image output terminal.

In this way, the embodiment of the present invention determines the discarded image area corresponding to the coordinate range in the panoramic video image to be output through the mapping relationship between the panoramic video image and the three-dimensional spherical model and the coordinate range, and removes the discarded image area from the panoramic video image to be output. The image data of the remaining area outside the area is transmitted to the video image output terminal. Compared with the transmission of all image data of the 360-degree panoramic video image, the number of splicing 360-degree panoramic video image cameras is reduced, and the number of splicing algorithm processing is reduced. , which reduces the processing time of video image encoding, and reduces the bit rate under the same parameters, thereby reducing the consumption of video transmission; in addition, compared with the method of only transmitting image data within the viewing angle range of the user, it reduces due to image data The problem of freezing caused by transmission delay solves the problems of large consumption and serious video freezing in the prior art when performing panoramic video playback.

Third embodiment:

As shown in FIG. 11 , it is a flow chart of the steps of the video playback control method in the third embodiment of the present invention. The control method is applied to a video image output terminal, including:

Step 1101, acquire the viewing angle direction data of the user in real time, and send the viewing angle direction data to the server.

In this step, specifically, the viewing direction data of the user may be gyroscope data, and the viewing direction data may be used to indicate the viewing posture of the user, that is, to indicate the viewing direction of the user. Therefore, the video image output terminal can obtain the user's viewing angle direction data through the gyroscope, and after the video image output terminal obtains the user's viewing angle direction data in real time, it can send the viewing angle direction data to the server.

Step 1102, receiving the image data of the remaining area except the discarded image area in the panoramic video image transmitted from the server.

In this step, specifically, after receiving the viewing direction data of the user, the server side can determine the coordinates of the area not related to the user's viewing angle when mapping the panoramic video image into the 3D mapping model according to the viewing direction data The discarded image area corresponding to the range, and the image data of the remaining area in the panoramic video image except the discarded image area is transmitted to the video image output terminal. At this time, the video image output terminal receives the image data of the remaining area in the panoramic video image transmitted from the server end except the discarded image area.

Step 1103, map the image data of the remaining area into a predetermined three-dimensional mapping model.

In this step, specifically, after receiving the image data of the remaining area, the video image output terminal can map the image data of the remaining area into a predetermined three-dimensional mapping model, so as to facilitate playing the video corresponding to the viewing angle of the user. Image section.

Step 1104, acquire the image part corresponding to the user's viewing angle from the mapped three-dimensional mapping model, and play the image part corresponding to the user's viewing angle.

In this step, specifically, after mapping the image data of the remaining regions into a predetermined three-dimensional mapping model, the user's viewing angle direction data can be obtained from the mapped three-dimensional mapping model corresponding to the user's viewing angle. image part, and play the image part corresponding to the user's viewing angle.

In this way, this embodiment sends the viewing angle direction data acquired in real time to the server end, and receives the image data of the remaining areas in the panoramic video image transmitted by the server end except the discarded image area, and then maps the image data of the remaining areas to In the predetermined three-dimensional mapping model, finally, the image part corresponding to the user's viewing angle is obtained from the mapped three-dimensional mapping model, and the image part is played. In this way, since the image data mapped into the three-dimensional mapping model is the image data of the remaining area, compared with only receiving the image data corresponding to the viewing angle, the user can watch a relatively smooth panorama when watching the panoramic video image and changing the viewing angle. The video image solves the problem of serious video freeze in the prior art when playing panoramic video, and improves user experience.

In addition, specifically, when receiving the image data of the remaining area in the panoramic video image transmitted from the server side except the discarded image area, the data of the remaining area in the panoramic video image transmitted from the server side except the discarded image area can be received at the same time. The image data and the view direction data corresponding to the image data of the remaining area. In this way, after playing the image part corresponding to the user's viewing angle, the coordinate range of the discarded image area in the mapped 3D mapping model can be determined according to the received viewing angle direction data, so that the video image output terminal can obtain the remaining At the same time as the image data of the region, the coordinate range of the discarded image region corresponding to the view direction data is determined.

In addition, because the data in the discarded image area in the panoramic video image is discarded, it will cause the panoramic video image to be incomplete. , after playing the image part corresponding to the user's viewing angle, the user's current viewing angle direction data can be obtained in real time, and the image part corresponding to the user's viewing angle in the mapped 3D mapping model can be determined according to the current viewing angle direction data acquired in real time Whether to fall within the coordinate range of the discarded image region.

Specifically, if the image part corresponding to the user's visual angle in the mapped three-dimensional mapping model falls within the coordinate range of the discarded image area, then the image part corresponding to the discarded image area can be obtained from the last mapped image part that contains the image part corresponding to the discarded image area. In the three-dimensional mapping model, the image part of the discarded image area corresponding to the user's viewing angle is acquired, and the acquired image part of the discarded image area is used for display; or, from the mapped three-dimensional mapping model, the user's current visible Regarding the viewing angle, the image part of the adjacent area of the discarded image area is discarded, and the image part of the adjacent area of the discarded image area is displayed as the image part corresponding to the user's current viewing angle. In this way, even if the user's viewing angle changes greatly, the user will not see the images in the discarded image area in the panoramic video image.

In this way, this embodiment sends the viewing angle direction data acquired in real time to the server end, and receives the image data and viewing angle direction data of the remaining areas in the panoramic video image transmitted by the server end except the discarded image area, and then sends the viewing angle direction data of the remaining areas The image data is mapped to a predetermined three-dimensional mapping model, and finally the image part corresponding to the user's viewing angle is obtained from the mapped three-dimensional mapping model, and the image part is played. In this way, since the image data mapped into the three-dimensional mapping model is the image data of the remaining area, compared with only receiving the image data corresponding to the viewing angle, the user can watch a relatively smooth panorama when watching the panoramic video image and changing the viewing angle. The video image solves the problem of serious video freeze in the prior art when playing panoramic video, and improves user experience.

Fourth embodiment:

As shown in FIG. 12, it is a structural block diagram of a control device for video playback in the fourth embodiment of the present invention. The control device is applied to the server side and includes:

The first receiving module 1201 is configured to receive the viewing direction data of the user;

An acquisition module 1202, configured to acquire the coordinate range of an area in the three-dimensional mapping model that is not related to the viewing angle of the user by using a predetermined three-dimensional mapping model according to the viewing angle direction data;

A determining module 1203, configured to determine, according to the coordinate range determined in the three-dimensional mapping model, a discarded image area corresponding to the coordinate range in the panoramic video image to be output;

The output module 1204 is configured to transmit the image data of the remaining area in the panoramic video image to be output except the discarded image area to the video image output terminal.

Optionally, the obtaining module 1202 includes: a first determining unit, configured to determine the position coordinates of the user's current viewpoint on the three-dimensional mapping model according to the view direction data; The position coordinates of the current viewpoint on the three-dimensional mapping model, and determine the position coordinates of the lowest correlation point of the image on the three-dimensional mapping model, wherein the lowest correlation point of the image is the current viewpoint and the three-dimensional mapping model The intersection point of the extension line of the line connecting the center points of the center point and the three-dimensional mapping model; the acquisition unit is used to obtain the current visual angle of the three-dimensional mapping model that is not related to the user according to the position coordinates of the lowest correlation point of the image The coordinate range of the area.

Optionally, the acquiring unit is configured to, according to the user's viewing angle on the 3D mapping model and the position coordinates of the lowest correlation point of the image, determine an area in the 3D mapping model that is not related to the user's viewing angle or, according to the position coordinates of the lowest relevant point of the image and a preset offset range, determine the coordinate range of an area in the three-dimensional mapping model that is not related to the viewing angle of the user.

Optionally, the three-dimensional mapping model includes a three-dimensional spherical model; the determining module 1203 is configured to determine the panoramic video image to be output according to the mapping relationship between the panoramic video image and the three-dimensional spherical model and the coordinate range The discarded image region corresponding to the coordinate range in .

Optionally, the position coordinates of the lowest correlation point of the image include the latitude and longitude of the lowest correlation point of the image; the coordinate range includes: a range of latitude coordinates and a range of longitude coordinates centered on the lowest correlation point of the image; the determining The module includes: a third determining unit, configured to obtain the panoramic video image to be output according to the mapping relationship between the panoramic video image and the three-dimensional spherical model, the latitude of the lowest correlation point of the image, and the latitude coordinate range The height range of the discarded image area corresponding to the coordinate range; the fourth determining unit is used for according to the mapping relationship between the panoramic video image and the three-dimensional spherical model, the longitude of the lowest correlation point of the image, and the longitude The coordinate range is used to obtain the width range of the discarded image area corresponding to the coordinate range in the to-be-output panoramic video image.

Optionally, the panoramic video image is a rectangular image or an elliptical image; if the mapping relationship between the panoramic video image and the three-dimensional spherical model includes: the height of the panoramic video image is mapped to the diameter of the three-dimensional spherical model , then the third determining unit is used for:

When α+β/2<=π/2 and α-β/2>=-π/2:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

When α+β/2>π/2:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

T11=0;

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

When α-β/2<-π/2:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

B12=H

and / or,

If the mapping relationship between the panoramic video image and the three-dimensional spherical model includes: the height of the panoramic video image is mapped to the half circumference of the longitude circle of the three-dimensional spherical model, then the third determination unit is used for:

When α+β/2<=π/2 and α-β/2>=-π/2:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

When α+β/2>π/2:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

T11=0;

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

When α-β/2<-π/2:

The upper boundary value of the height range of the discarded image area is obtained by the following formula:

The lower boundary value of the height range of the discarded image area is obtained by the following formula:

B12=H

Wherein, α represents the latitude of the lowest correlation point of the image, β represents the angle value corresponding to the latitude coordinate range, H represents the height of the panoramic video image, T1 represents the upper boundary value of the height range of the discarded image area, B1 represents the lower boundary value of the height range of the discarded image region, T11 represents the upper boundary value of the height range of the first region in the discarded image region, and B11 represents the lower boundary value of the height range of the first region in the discarded image region Boundary value, T12 represents the upper boundary value of the height range of the second area in the discarded image area, and B12 represents the lower boundary value of the height range of the second area in the discarded image area.

Optionally, the panoramic video image is a rectangular image; the mapping relationship between the panoramic video image and the three-dimensional spherical model includes: the width of the panoramic video image is mapped to the equatorial perimeter of the three-dimensional spherical model, then the The fourth determining unit is used for:

When δ-γ/2>=0 and δ+γ/2<=2π:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

The right boundary value of the width range of the discarded image area is obtained by the following formula:

When δ-γ/2<0:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

L11=0

The right boundary value of the width range of the discarded image area is obtained by the following formula:

R12=W

When δ+γ/2>2π:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

L11=0

The right boundary value of the width range of the discarded image area is obtained by the following formula:

R12=W

Among them, δ represents the longitude of the lowest correlation point of the image, γ represents the angle value corresponding to the longitude coordinate range, W represents the width of the panoramic video image, L1 represents the left boundary value of the width range of the discarded image area, and R1 represents The right boundary value of the width range of the discarded image area, L11 represents the left boundary value of the width range of the first area in the discarded image area, and R11 represents the right boundary value of the width range of the first area in the discarded image area , L12 represents the left boundary value of the width range of the second area in the discarded image area, and R12 represents the right boundary value of the width range of the second area in the discarded image area.

Optionally, the panoramic video image is an elliptical image; the mapping relationship between the panoramic video image and the three-dimensional spherical model includes: the width of the panoramic video image is mapped to the equatorial perimeter of the three-dimensional spherical model, then The fourth determining unit is used for:

When δ-γ/2>=0 and δ+γ/2<=2π:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

The right boundary value of the width range of the discarded image area is obtained by the following formula:

When δ-γ/2<0:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

L11=s

The right boundary value of the width range of the discarded image area is obtained by the following formula:

R12=s+w

When δ+γ/2>2π:

The left boundary value of the width range of the discarded image area is obtained by the following formula:

L11=s

The right boundary value of the width range of the discarded image area is obtained by the following formula:

R12=s+w

Among them, δ represents the longitude of the lowest correlation point of the image, γ represents the angle value corresponding to the longitude coordinate range, s represents the starting point abscissa of each row of images in the panoramic video image, and w represents the width of each row of images in the panoramic video image , L1 represents the left boundary value of the width range of the discarded image region, R1 represents the right boundary value of the width range of the discarded image region, L11 represents the left boundary value of the width range of the first region in the discarded image region, R11 represents the right boundary value of the width range of the first area in the discarded image area, L12 represents the left boundary value of the width range of the second area in the discarded image area, and R12 represents the width of the second area in the discarded image area The right boundary value of the width range.

Optionally, the output module 1204 includes: a first output unit, configured to acquire, according to the discarded image region, remaining regions in the panoramic video image other than the discarded image region, and analyze the remaining regions Perform rendering; transmit the image data obtained after rendering to the video image output terminal; or, a second output unit for rendering the panoramic video image; combine the image data obtained after rendering with the discarded image The data corresponding to the area is set to a specific value, and the set image data is encoded; the encoded image data is transmitted to the video image output terminal.

Optionally, the output module 1204 is configured to transmit the image data of the remaining area and the viewing angle direction data to a video image output terminal, so that the video image output terminal determines according to the viewing angle direction data The discarded image area of the panoramic video image.

In this way, the embodiment of the present invention only transmits the image data of the remaining area in the panoramic video image to be output except the discarded image area to the video image output terminal, which reduces the overall image data compared to the transmission of all the image data of the 360-degree panoramic video image. The number of splicing 360-degree panoramic video image cameras reduces the number of splicing algorithm processing, reduces the processing time of video image encoding, and reduces the bit rate under the same parameters, thereby reducing the consumption of video transmission; in addition, due to The image data of the remaining area in the panoramic video image except the discarded image area is transmitted to the video image output terminal, so compared with the method of only transmitting the image data within the user's visual angle range, it reduces the jam caused by the image data transmission delay This solves the problems of large consumption and serious video freeze in the prior art when playing panoramic video.

Fifth embodiment:

As shown in FIG. 13 , it is a structural block diagram of a video playback control device in a fifth embodiment of the present invention. The control device is applied to a video image output terminal, including:

A sending module 1301, configured to acquire the viewing angle direction data of the user in real time, and send the viewing angle direction data to the server;

The second receiving module 1302 is configured to receive the image data of the remaining area except the discarded image area in the panoramic video image transmitted by the server;

A mapping module 1303, configured to map the image data of the remaining area into a predetermined three-dimensional mapping model;

The playing module 1304 is configured to acquire the image part corresponding to the user's viewing angle from the mapped three-dimensional mapping model, and play the image part corresponding to the user's viewing angle.

Optionally, the second receiving module 1302 is configured to receive the image data of the remaining area except the discarded image area in the panoramic video image transmitted from the server and the viewing angle direction data corresponding to the image data of the remaining area The control device also includes: a coordinate range determination module, used to determine the coordinate range of the discarded image area in the mapped three-dimensional mapping model according to the viewing angle direction data after the playing module plays the image part The judging module is used to obtain the user's current viewing angle direction data in real time, and determine whether the image part corresponding to the user's viewing angle in the mapped three-dimensional mapping model falls into the discarded image according to the current viewing angle direction data obtained in real time The coordinate range of the area; wherein, if the image part corresponding to the user's viewing angle in the mapped three-dimensional mapping model falls within the coordinate range of the discarded image area, then from the previous mapped, the latest included and discarded In the three-dimensional mapping model of the image part corresponding to the image area, the image part of the discarded image area corresponding to the viewing angle of the user is obtained, and the acquired image part of the discarded image area is used for display; or, from the mapped three-dimensional mapping In the model, the image part of the adjacent area of the discarded image area related to the user's current viewing angle is acquired, and the image part of the adjacent area of the discarded image area is regarded as corresponding to the user's current viewing angle part of the image is displayed.

In this way, this embodiment sends the viewing angle direction data acquired in real time to the server end, and receives the image data and viewing angle direction data of the remaining areas in the panoramic video image transmitted by the server end except the discarded image area, and then sends the viewing angle direction data of the remaining areas The image data is mapped to a predetermined three-dimensional mapping model, and finally the image part corresponding to the user's viewing angle is obtained from the mapped three-dimensional mapping model, and the image part is played. In this way, since the image data mapped into the three-dimensional mapping model is the image data of the remaining area, compared with only receiving the image data corresponding to the viewing angle, the user can watch a relatively smooth panorama when watching the panoramic video image and changing the viewing angle. The video image solves the problem of serious video freeze in the prior art when playing panoramic video, and improves user experience.

Sixth embodiment:

This embodiment also provides a computer-readable storage medium for storing a computer program for video playback, and the computer program can be used by a processor to execute the video playback control method applied to the server in the foregoing embodiments.

Seventh embodiment:

This embodiment also provides a computer-readable storage medium for storing a computer program for video playback, and the computer program can be executed by a processor to control the video playback method applied to a video image output terminal in the foregoing embodiments.

Eighth embodiment:

This embodiment provides an electronic device, where the electronic device includes one or more processors, and the processors are configured to execute the following control method:

Receive the user's view direction data;

According to the viewing angle direction data, using a predetermined three-dimensional mapping model, acquiring a coordinate range of an area in the three-dimensional mapping model that is not related to the viewing angle of the user;

According to the coordinate range determined in the three-dimensional mapping model, determine a discarded image area corresponding to the coordinate range in the panoramic video image to be output;

The image data of the remaining area in the panoramic video image to be output except the discarded image area is transmitted to the video image output terminal.

Ninth embodiment:

This embodiment also provides an electronic device, where the electronic device includes one or more processors, and the processors are configured to execute the following control method:

Obtaining the viewing direction data of the user in real time, and sending the viewing direction data to the server;

Receive the image data of the remaining area except the discarded image area in the panoramic video image transmitted by the server;

mapping the image data of the remaining area into a predetermined three-dimensional mapping model;

The image part corresponding to the user's viewing angle is acquired from the mapped three-dimensional mapping model, and the image part corresponding to the user's viewing angle is played.

What has been described above is a preferred embodiment of the present invention. It should be pointed out that for those skilled in the art, some improvements and modifications can also be made without departing from the principles described in the present invention. within the scope of protection of the invention.

Claims (28)

1. A control method for video playing is applied to a server side, and is characterized by comprising the following steps:

receiving visual angle direction data of a user, wherein the visual angle direction data is used for indicating the watching gesture of the user;

acquiring a coordinate range of an area irrelevant to the visual angle of the user in a three-dimensional mapping model by utilizing a predetermined three-dimensional mapping model according to the visual angle direction data;

determining a discarded image area corresponding to the coordinate range in the panoramic video image to be output according to the coordinate range determined in the three-dimensional mapping model;

and transmitting the image data of the remaining area except the discarded image area in the panoramic video image to be output to a video image output terminal.

2. The control method according to claim 1, wherein the step of obtaining a coordinate range of an area unrelated to the viewing angle of the user in a three-dimensional mapping model using a predetermined three-dimensional mapping model according to the viewing angle direction data comprises:

determining the position coordinates of the current viewpoint of the user on the three-dimensional mapping model according to the view direction data;

determining the position coordinates of the lowest relevant point of the image on the three-dimensional mapping model according to the position coordinates of the current viewpoint on the three-dimensional mapping model, wherein the lowest relevant point of the image is the intersection point of the extension line of the connecting line of the current viewpoint and the central point of the three-dimensional mapping model and the three-dimensional mapping model;

and acquiring the coordinate range of the area irrelevant to the visual angle of the user in the three-dimensional mapping model according to the position coordinate of the lowest relevant point of the image.

3. The control method according to claim 2, wherein the step of obtaining a coordinate range of a region irrelevant to the viewing angle of the user in the three-dimensional mapping model from the position coordinates of the lowest relevant point of the image comprises:

determining the coordinate range of a region irrelevant to the visual angle of the user in the three-dimensional mapping model according to the visual angle of the user on the three-dimensional mapping model and the position coordinates of the lowest relevant point of the image; or,

and determining the coordinate range of the area irrelevant to the visual angle of the user in the three-dimensional mapping model according to the position coordinate of the lowest relevant point of the image and a preset offset range.

4. The control method according to claim 2 or 3,

the three-dimensional mapping model comprises a three-dimensional sphere model;

the step of determining a discarded image area corresponding to the coordinate range in the panoramic video image to be output according to the coordinate range determined in the three-dimensional mapping model includes:

and determining a discarded image area corresponding to the coordinate range in the panoramic video image to be output according to the mapping relation between the panoramic video image and the three-dimensional sphere model and the coordinate range.

5. The control method according to claim 4,

the position coordinates of the lowest relevant point of the image comprise the latitude and longitude of the lowest relevant point of the image;

the coordinate range includes: latitude coordinate range and longitude coordinate range with the lowest relevant point of the image as the center;

the step of determining a discarded image area corresponding to the coordinate range in the panoramic video image to be output according to the mapping relation between the panoramic video image and the three-dimensional sphere model and the coordinate range comprises the following steps:

obtaining a height range of a discarded image area corresponding to the coordinate range in the panoramic video image to be output according to the mapping relation between the panoramic video image and the three-dimensional sphere model, the latitude of the lowest relevant point of the image and the latitude coordinate range;

and obtaining the width range of a discarded image area corresponding to the coordinate range in the panoramic video image to be output according to the mapping relation between the panoramic video image and the three-dimensional sphere model, the longitude of the lowest relevant point of the image and the longitude coordinate range.

6. The control method according to claim 5,

the panoramic video image is a rectangular image or an elliptical image;

if the mapping relationship between the panoramic video image and the three-dimensional sphere model comprises: mapping the height of the panoramic video image to the diameter of the three-dimensional sphere model, and then:

the step of obtaining a height range of a discarded image area corresponding to the coordinate range in the panoramic video image to be output according to the mapping relationship between the panoramic video image and the three-dimensional sphere model, the latitude of the lowest relevant point of the image and the latitude coordinate range includes:

when α + β/2< ═ pi/2 and α - β/2> -pi/2:

the upper boundary value of the height range of the discarded image area is obtained by the following formula:

the lower boundary value of the height range of the discarded image area is obtained by the following formula:

when α + β/2> π/2:

the upper boundary value of the height range of the discarded image area is obtained by the following formula:

T11＝0；

the lower boundary value of the height range of the discarded image area is obtained by the following formula:

when α - β/2< -pi/2:

the upper boundary value of the height range of the discarded image area is obtained by the following formula:

the lower boundary value of the height range of the discarded image area is obtained by the following formula:

B12＝H

and/or the presence of a gas in the gas,

if the mapping relationship between the panoramic video image and the three-dimensional sphere model comprises: and mapping the height of the panoramic video image to be the semi-perimeter of the longitude circumference of the three-dimensional sphere model, and then:

the step of obtaining a height range of a discarded image area corresponding to the coordinate range in the panoramic video image to be output according to the mapping relationship between the panoramic video image and the three-dimensional sphere model, the latitude of the lowest relevant point of the image and the latitude coordinate range includes:

when α + β/2< ═ pi/2 and α - β/2> -pi/2:

the upper boundary value of the height range of the discarded image area is obtained by the following formula:

the lower boundary value of the height range of the discarded image area is obtained by the following formula:

when α + β/2> π/2:

the upper boundary value of the height range of the discarded image area is obtained by the following formula:

T11＝0；

the lower boundary value of the height range of the discarded image area is obtained by the following formula:

when α - β/2< -pi/2:

the upper boundary value of the height range of the discarded image area is obtained by the following formula:

the lower boundary value of the height range of the discarded image area is obtained by the following formula:

B12＝H

where α denotes the latitude of the image lowest relevant point, β denotes the angle value corresponding to the latitude coordinate range, H denotes the height of the panoramic video image, T1 denotes the upper boundary value of the height range of the discarded image region, B1 denotes the lower boundary value of the height range of the discarded image region, T11 denotes the upper boundary value of the height range of the first region in the discarded image region, B11 denotes the lower boundary value of the height range of the first region in the discarded image region, T12 denotes the upper boundary value of the height range of the second region in the discarded image region, and B12 denotes the lower boundary value of the height range of the second region in the discarded image region.

7. The control method according to claim 5,

the panoramic video image is a rectangular image;

the mapping relation between the panoramic video image and the three-dimensional sphere model comprises: and mapping the width of the panoramic video image to the equator circumference of the three-dimensional sphere model, and then:

the step of obtaining a width range of a discarded image area corresponding to the coordinate range in the panoramic video image to be output according to the mapping relationship between the panoramic video image and the three-dimensional sphere model, the longitude of the lowest relevant point of the image and the longitude coordinate range includes:

when δ - γ/2> -0 and δ + γ/2< ═ 2 pi:

the left boundary value of the width range of the discarded image area is obtained by the following formula:

the right boundary value of the width range of the discarded image area is obtained by the following formula:

when δ - γ/2< 0:

the left boundary value of the width range of the discarded image area is obtained by the following formula:

L11＝0

the right boundary value of the width range of the discarded image area is obtained by the following formula:

R12＝W

when δ + γ/2>2 π:

the left boundary value of the width range of the discarded image area is obtained by the following formula:

L11＝0

the right boundary value of the width range of the discarded image area is obtained by the following formula:

R12＝W

where δ denotes a longitude of the image lowest correlation point, γ denotes an angle value corresponding to the longitude coordinate range, W denotes a width of the panoramic video image, L1 denotes a left boundary value of the width range of the discarded image region, R1 denotes a right boundary value of the width range of the discarded image region, L11 denotes a left boundary value of the width range of the first region in the discarded image region, R11 denotes a right boundary value of the width range of the first region in the discarded image region, L12 denotes a left boundary value of the width range of the second region in the discarded image region, and R12 denotes a right boundary value of the width range of the second region in the discarded image region.

8. The control method according to claim 5,

the panoramic video image is an elliptical image;

the mapping relation between the panoramic video image and the three-dimensional sphere model comprises: and mapping the width of the panoramic video image to the equator circumference of the three-dimensional sphere model, and then:

the step of obtaining a width range of a discarded image area corresponding to the coordinate range in the panoramic video image to be output according to the mapping relationship between the panoramic video image and the three-dimensional sphere model, the longitude of the lowest relevant point of the image and the longitude coordinate range includes:

when δ - γ/2> -0 and δ + γ/2< ═ 2 pi:

the left boundary value of the width range of the discarded image area is obtained by the following formula:

the right boundary value of the width range of the discarded image area is obtained by the following formula:

when δ - γ/2< 0:

the left boundary value of the width range of the discarded image area is obtained by the following formula:

L11＝s

the right boundary value of the width range of the discarded image area is obtained by the following formula:

R12＝s+w

when δ + γ/2>2 π:

the left boundary value of the width range of the discarded image area is obtained by the following formula:

L11＝s

the right boundary value of the width range of the discarded image area is obtained by the following formula:

R12＝s+w

where δ denotes a longitude of the lowest relevant point of the image, γ denotes an angle value corresponding to the longitude coordinate range, s denotes a start abscissa of each line of image in the panoramic video image, w denotes a width of each line of image in the panoramic video image, L1 denotes a left boundary value of the width range of the discarded image region, R1 denotes a right boundary value of the width range of the discarded image region, L11 denotes a left boundary value of the width range of the first region in the discarded image region, R11 denotes a right boundary value of the width range of the first region in the discarded image region, L12 denotes a left boundary value of the width range of the second region in the discarded image region, and R12 denotes a right boundary value of the width range of the second region in the discarded image region.

9. The method according to claim 1, wherein the step of transmitting image data of a remaining area of the panoramic video image to be output, excluding the discarded image area, to a video image output terminal comprises:

according to the discarded image area, acquiring a residual area in the panoramic video image except the discarded image area, and rendering the residual area;

transmitting the rendered image data to a video image output terminal;

or,

rendering the panoramic video image;

setting data corresponding to the discarded image area in the rendered image data as a specific value, and encoding the set image data;

and transmitting the encoded image data to a video image output terminal.

10. The method according to claim 1, wherein the step of transmitting image data of a remaining area of the panoramic video image to be output, excluding the discarded image area, to a video image output terminal comprises:

and transmitting the image data of the residual area and the view angle direction data to a video image output terminal so that the video image output terminal determines a discarded image area of the panoramic video image according to the view angle direction data.

11. A control method for video playing is applied to a video image output terminal, and is characterized by comprising the following steps:

the method comprises the steps of acquiring visual angle direction data of a user in real time, and sending the visual angle direction data to a server side, wherein the visual angle direction data is used for indicating the watching posture of the user;

receiving image data of the remaining area except the discarded image area in the panoramic video image transmitted by the server;

mapping the image data of the remaining region into a predetermined three-dimensional mapping model;

and acquiring an image part corresponding to the visual angle of the user from the mapped three-dimensional mapping model, and playing the image part corresponding to the visual angle of the user.

12. The control method according to claim 11, wherein the step of receiving image data of a remaining area of the panoramic video image transmitted by the server, excluding the discarded image area, comprises:

receiving image data of a residual area except for a discarded image area in a panoramic video image transmitted by the server and view direction data corresponding to the image data of the residual area; after playing the image portion, the control method further includes:

determining the coordinate range of the discarded image area in the mapped three-dimensional mapping model according to the view direction data;

acquiring current visual angle direction data of a user in real time, and determining whether an image part corresponding to the visual angle of the user in the mapped three-dimensional mapping model falls into the coordinate range of the discarded image area or not according to the current visual angle direction data acquired in real time; wherein,

if the image part corresponding to the visual angle of the user in the mapped three-dimensional mapping model falls into the coordinate range of the discarded image area, acquiring the image part of the discarded image area corresponding to the visual angle of the user from the three-dimensional mapping model which is mapped before and contains the image part corresponding to the discarded image area at the latest time, and displaying by using the acquired image part of the discarded image area; or,

and acquiring the image part of the adjacent region of the discarded image region related to the current visual angle of the user from the mapped three-dimensional mapping model, and displaying the image part of the adjacent region of the discarded image region as the image part corresponding to the current visual angle of the user.

13. A control device for video playing is applied to a server side, and is characterized by comprising:

the device comprises a first receiving module, a second receiving module and a display module, wherein the first receiving module is used for receiving visual angle direction data of a user, and the visual angle direction data is used for indicating the watching gesture of the user;

the acquisition module is used for acquiring the coordinate range of the area irrelevant to the visual angle of the user in the three-dimensional mapping model by utilizing a predetermined three-dimensional mapping model according to the visual angle direction data;

the determining module is used for determining a discarded image area corresponding to the coordinate range in the panoramic video image to be output according to the coordinate range determined in the three-dimensional mapping model;

and the output module is used for transmitting the image data of the remaining area except the discarded image area in the panoramic video image to be output to the video image output terminal.

14. The control device of claim 13, wherein the obtaining module comprises:

the first determining unit is used for determining the position coordinates of the current viewpoint of the user on the three-dimensional mapping model according to the view direction data;

a second determining unit, configured to determine, according to a position coordinate of the current viewpoint on the three-dimensional mapping model, a position coordinate of a lowest relevant point of an image on the three-dimensional mapping model, where the lowest relevant point of the image is an intersection point of an extension line of a connection line between the current viewpoint and a center point of the three-dimensional mapping model and the three-dimensional mapping model;

and the acquisition unit is used for acquiring the coordinate range of the area irrelevant to the current visual angle of the user in the three-dimensional mapping model according to the position coordinate of the lowest relevant point of the image.

15. The control device according to claim 14, wherein the acquiring unit is configured to determine a coordinate range of a region in the three-dimensional mapping model that is not related to the viewing angle of the user, based on the viewing angle of the user on the three-dimensional mapping model and the position coordinates of the lowest relevant point of the image; or determining the coordinate range of the area irrelevant to the visual angle of the user in the three-dimensional mapping model according to the position coordinate of the lowest relevant point of the image and a preset offset range.

16. The control device according to claim 14 or 15, characterized in that the three-dimensional mapping model comprises a three-dimensional sphere model; the determining module is used for determining a discarded image area corresponding to the coordinate range in the panoramic video image to be output according to the mapping relation between the panoramic video image and the three-dimensional sphere model and the coordinate range.

17. The control device of claim 16, wherein the position coordinates of the lowest image-related point include a latitude and a longitude of the lowest image-related point; the coordinate range includes: latitude coordinate range and longitude coordinate range with the lowest relevant point of the image as the center; the determining module comprises:

a third determining unit, configured to obtain, according to a mapping relationship between the panoramic video image and the three-dimensional sphere model, a latitude of a lowest relevant point of the image, and the latitude coordinate range, a height range of a discarded image area corresponding to the coordinate range in the panoramic video image to be output;

and the fourth determining unit is used for obtaining the width range of a discarded image area corresponding to the coordinate range in the panoramic video image to be output according to the mapping relation between the panoramic video image and the three-dimensional sphere model, the longitude of the lowest relevant point of the image and the longitude coordinate range.

18. The control device according to claim 17, wherein the panoramic video image is a rectangular image or an elliptical image; if the mapping relationship between the panoramic video image and the three-dimensional sphere model comprises: the height of the panoramic video image is mapped to the diameter of the three-dimensional sphere model, and the third determining unit is configured to:

when α + β/2< ═ pi/2 and α - β/2> -pi/2:

the upper boundary value of the height range of the discarded image area is obtained by the following formula:

the lower boundary value of the height range of the discarded image area is obtained by the following formula:

when α + β/2> π/2:

the upper boundary value of the height range of the discarded image area is obtained by the following formula:

T11＝0；

the lower boundary value of the height range of the discarded image area is obtained by the following formula:

when α - β/2< -pi/2:

the upper boundary value of the height range of the discarded image area is obtained by the following formula:

the lower boundary value of the height range of the discarded image area is obtained by the following formula:

B12＝H

and/or the presence of a gas in the gas,

if the mapping relationship between the panoramic video image and the three-dimensional sphere model comprises: the height of the panoramic video image is mapped to a half-perimeter of a longitudinal circumference of the three-dimensional sphere model, and the third determining unit is configured to:

when α + β/2< ═ pi/2 and α - β/2> -pi/2:

the upper boundary value of the height range of the discarded image area is obtained by the following formula:

the lower boundary value of the height range of the discarded image area is obtained by the following formula:

when α + β/2> π/2:

the upper boundary value of the height range of the discarded image area is obtained by the following formula:

T11＝0；

the lower boundary value of the height range of the discarded image area is obtained by the following formula:

when α - β/2< -pi/2:

the upper boundary value of the height range of the discarded image area is obtained by the following formula:

the lower boundary value of the height range of the discarded image area is obtained by the following formula:

B12＝H

where α denotes the latitude of the image lowest relevant point, β denotes the angle value corresponding to the latitude coordinate range, H denotes the height of the panoramic video image, T1 denotes the upper boundary value of the height range of the discarded image region, B1 denotes the lower boundary value of the height range of the discarded image region, T11 denotes the upper boundary value of the height range of the first region in the discarded image region, B11 denotes the lower boundary value of the height range of the first region in the discarded image region, T12 denotes the upper boundary value of the height range of the second region in the discarded image region, and B12 denotes the lower boundary value of the height range of the second region in the discarded image region.

19. The control device according to claim 17, wherein the panoramic video image is a rectangular image; the mapping relation between the panoramic video image and the three-dimensional sphere model comprises: the width of the panoramic video image is mapped to the equatorial circumference of the three-dimensional sphere model, and the fourth determining unit is configured to:

when δ - γ/2> -0 and δ + γ/2< ═ 2 pi:

the left boundary value of the width range of the discarded image area is obtained by the following formula:

the right boundary value of the width range of the discarded image area is obtained by the following formula:

when δ - γ/2< 0:

the left boundary value of the width range of the discarded image area is obtained by the following formula:

L11＝0

the right boundary value of the width range of the discarded image area is obtained by the following formula:

R12＝W

when δ + γ/2>2 π:

the left boundary value of the width range of the discarded image area is obtained by the following formula:

L11＝0

the right boundary value of the width range of the discarded image area is obtained by the following formula:

R12＝W

where δ denotes a longitude of the image lowest correlation point, γ denotes an angle value corresponding to the longitude coordinate range, W denotes a width of the panoramic video image, L1 denotes a left boundary value of the width range of the discarded image region, R1 denotes a right boundary value of the width range of the discarded image region, L11 denotes a left boundary value of the width range of the first region in the discarded image region, R11 denotes a right boundary value of the width range of the first region in the discarded image region, L12 denotes a left boundary value of the width range of the second region in the discarded image region, and R12 denotes a right boundary value of the width range of the second region in the discarded image region.

20. The control device according to claim 17, wherein the panoramic video image is an elliptical image; the mapping relation between the panoramic video image and the three-dimensional sphere model comprises: the width of the panoramic video image is mapped to the equatorial circumference of the three-dimensional sphere model, and the fourth determining unit is configured to:

when δ - γ/2> -0 and δ + γ/2< ═ 2 pi:

the left boundary value of the width range of the discarded image area is obtained by the following formula:

the right boundary value of the width range of the discarded image area is obtained by the following formula:

when δ - γ/2< 0:

the left boundary value of the width range of the discarded image area is obtained by the following formula:

L11＝s

the right boundary value of the width range of the discarded image area is obtained by the following formula:

R12＝s+w

when δ + γ/2>2 π:

the left boundary value of the width range of the discarded image area is obtained by the following formula:

L11＝s

the right boundary value of the width range of the discarded image area is obtained by the following formula:

R12＝s+w

where δ denotes a longitude of the lowest relevant point of the image, γ denotes an angle value corresponding to the longitude coordinate range, s denotes a start abscissa of each line of image in the panoramic video image, w denotes a width of each line of image in the panoramic video image, L1 denotes a left boundary value of the width range of the discarded image region, R1 denotes a right boundary value of the width range of the discarded image region, L11 denotes a left boundary value of the width range of the first region in the discarded image region, R11 denotes a right boundary value of the width range of the first region in the discarded image region, L12 denotes a left boundary value of the width range of the second region in the discarded image region, and R12 denotes a right boundary value of the width range of the second region in the discarded image region.

21. The control device of claim 13, wherein the output module comprises:

a first output unit, configured to obtain a remaining region in the panoramic video image, excluding the discarded image region, according to the discarded image region, and render the remaining region; transmitting the rendered image data to a video image output terminal; or,

a second output unit, configured to render the panoramic video image; setting data corresponding to the discarded image area in the rendered image data as a specific value, and encoding the set image data; and transmitting the encoded image data to a video image output terminal.

22. The control device of claim 13, wherein the output module is configured to transmit the image data of the remaining area and the view direction data to a video image output terminal, so that the video image output terminal determines a discarded image area of the panoramic video image according to the view direction data.

23. A control device for video playing is applied to a video image output terminal, and is characterized by comprising:

the system comprises a sending module, a receiving module and a display module, wherein the sending module is used for acquiring the visual angle direction data of a user in real time and sending the visual angle direction data to a server side, and the visual angle direction data is used for indicating the watching posture of the user;

the second receiving module is used for receiving image data of the remaining area except the discarded image area in the panoramic video image transmitted by the server;

a mapping module for mapping the image data of the remaining region into a predetermined three-dimensional mapping model;

and the playing module is used for acquiring the image part corresponding to the visual angle of the user from the mapped three-dimensional mapping model and playing the image part corresponding to the visual angle of the user.

24. The control device according to claim 23, wherein the second receiving module is configured to receive image data of a remaining area of the panoramic video image transmitted by the server, except for a discarded image area, and view direction data corresponding to the image data of the remaining area;

the control device further includes:

a coordinate range determining module, configured to determine, according to the view direction data, a coordinate range of the discarded image region in the mapped three-dimensional mapping model after the playing module plays the image portion;

the judging module is used for acquiring the current visual angle direction data of the user in real time and determining whether an image part corresponding to the visual angle of the user in the mapped three-dimensional mapping model falls into the coordinate range of the discarded image area or not according to the current visual angle direction data acquired in real time; if the image part corresponding to the visual angle of the user in the mapped three-dimensional mapping model falls into the coordinate range of the discarded image area, acquiring the image part of the discarded image area corresponding to the visual angle of the user from the three-dimensional mapping model which is mapped before and contains the image part corresponding to the discarded image area at the latest time, and displaying by using the acquired image part of the discarded image area; or acquiring the image part of the adjacent area of the discarded image area, which is related to the current visual angle of the user, from the mapped three-dimensional mapping model, and displaying the image part of the adjacent area of the discarded image area as the image part corresponding to the current visual angle of the user.

25. A computer-readable storage medium for storing a computer program for video playback, the computer program being executable by a processor to perform the method for controlling video playback applied to a server according to any one of claims 1 to 10.

26. A computer-readable storage medium for storing a computer program for video playback, the computer program being executable by a processor to perform the method for controlling video playback applied to a video image output terminal according to any one of claims 11 to 12.

27. An electronic device, comprising one or more processors configured to perform a control method comprising:

receiving visual angle direction data of a user, wherein the visual angle direction data is used for indicating the watching gesture of the user;

acquiring a coordinate range of an area irrelevant to the visual angle of the user in a three-dimensional mapping model by utilizing a predetermined three-dimensional mapping model according to the visual angle direction data;

determining a discarded image area corresponding to the coordinate range in the panoramic video image to be output according to the coordinate range determined in the three-dimensional mapping model;

and transmitting the image data of the remaining area except the discarded image area in the panoramic video image to be output to a video image output terminal.

28. An electronic device, comprising one or more processors configured to perform a control method comprising:

the method comprises the steps of acquiring visual angle direction data of a user in real time, and sending the visual angle direction data to a server side, wherein the visual angle direction data is used for indicating the watching posture of the user;

receiving image data of the remaining area except the discarded image area in the panoramic video image transmitted by the server;

mapping the image data of the remaining region into a predetermined three-dimensional mapping model;

and acquiring an image part corresponding to the visual angle of the user from the mapped three-dimensional mapping model, and playing the image part corresponding to the visual angle of the user.