Detailed Description
in order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be described in further detail with reference to the accompanying drawings, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, an embodiment of the present invention provides a flow diagram of a panoramic video playing method, and a specific implementation method includes:
step S101, receiving a panoramic video playing request of a player, wherein the panoramic video playing request comprises viewpoint position information of a user.
And step S102, determining a video unit to be played according to the viewpoint position information of the user, wherein the video unit is obtained by pre-dividing the video frame of the panoramic video to be played.
step S103, sending the determined video unit to be played to the player in a video stream mode.
because a one-second panoramic video usually has 25 panoramic pictures, i.e. 25 video frames, each panoramic picture is an image with 360 degrees or 720 degrees, when a server side uniformly compresses a segment of panoramic video into a video code stream, the video stream in the code stream having a large angle range is not watched by a user but occupies a transmission bandwidth, so the server side actively acquires viewpoint position information of the user in a player, wherein the viewpoint position information may include a coordinate position of the user, an observation direction of the user, and the like, and the server may determine a maximum angle range which can be currently seen by the user according to the viewpoint position information, so that only the video stream of the portion may be compressed and then transmitted to the player, obviously, a portion of the transmission bandwidth may be saved by doing so.
before step S101 is executed, the server needs to pre-divide the current panoramic video, that is, the server divides the video frame into video units corresponding to a plurality of sub-coordinate intervals in one direction according to a coordinate interval covered by the video frame in the direction, where the sub-coordinate intervals are subsets of the coordinate intervals, and a total set of all the sub-coordinate intervals is the same as the coordinate interval covered by the direction.
That is, the server side needs to complete the cutting of the panoramic video, because the view angle range of a general user is 120 degrees at most, the server side can perform the cutting by comprehensively considering the view range of the user and the complexity of coding after the cutting, for example, as shown in fig. 2, an original video is 360 degrees, and the server determines that a coordinate interval corresponding to the current viewpoint position information of the user is approximately (10 °, 120 °), then the server side, in order to satisfy the current view range of the user as much as possible, will select to cut each video frame in the panoramic video by using multiple cutting angles, so that the cut video streams obtained by various cutting methods can satisfy the viewing requirements of the user as much as possible, for example, the server side can copy the video frames corresponding to (0, 360 °) at the current time into 3 parts at the same time, then the first part obtains the cut (0, a 240 deg. first video unit, a second cut of (120 deg., 360 deg.) second video unit, and a third cut of (0, 120 deg.) and (240 deg., 360 deg.) third video unit and fourth video unit, so that the server side can obtain a cut first video stream composed of the first video units, a cut second video stream composed of the second video units, and a cut third video stream composed of the third video units and the fourth video units. Obviously, any one of the three video streams is sent to the player to satisfy the current visual field range of the user.
Certainly, considering that the third video stream is actually composed of the third video unit and the fourth video unit, further, the server side may perform splicing according to the coordinate attributes of the third video unit and the fourth video unit of each frame, then obtain a spliced image, further compress the spliced images of the plurality of video frames within a set time duration, obtain the third video stream, and send the third video stream to the player side so that the player can play the third video stream.
it can be understood that the server side may also divide each video frame in the panoramic video into smaller segments, for example, fig. 2 may be divided into one sub-coordinate interval for each degree in the horizontal direction, and then since the coordinate interval corresponding to the current viewpoint position information of the user is approximately (10 ° and 120 °), a video unit corresponding to each sub-coordinate interval within (0 and 120 °) may be compressed into one video stream and sent to the player side, and the player receives the compressed video stream, decompresses it, and then stitches the video units corresponding to each sub-coordinate interval.
Considering that the video stream obtained after being cut by the server is usually stored in the server in the form of segments, the playing request sent by the server side received by the player side further includes an identifier of the panoramic video to be played, and before determining the video unit to be played, the method further includes: determining at least one frame of video frame to be played according to the identification of the panoramic video to be played;
or the panoramic video playing request is viewpoint position updating information; before the determining the video unit to be played, the method further includes: and determining at least one frame of video frame to be played from the panoramic video being played according to the viewpoint position updating information.
specifically, the playing request sent by the player to the server includes that the panoramic video to be played is a certain movie, so the server receives the request and then correspondingly sends the cut video stream corresponding to the movie to the player, and of course, the playing request sent by the player may also include the start time of the video stream, so when the server receives the playing request including the start time, the panoramic video of the set period after the start time can be correspondingly cut, the cut video stream is formed, for example, the panoramic video within five seconds after 12:00 is cut, and the cut video stream within the time segment is obtained (12:00, 12: 05).
of course, the broadcast request that is received at this time may be a change of the viewpoint position information of the user, for example, from (10 °, 120 °) to (60 °, 180 °), so that the server side needs to transmit the video unit combination video stream corresponding to each sub-coordinate interval within (60 °, 180 °) to the player side when receiving the broadcast request, and usually, the server side compresses the determined video units to obtain the video stream.
Because the viewpoint position information of the user has non-fixity, in order to ensure that the video stream to be played sent by the server side can still meet the watching requirement of the user even if the viewpoint of the user temporarily changes in a small range, the visual field range of the video stream sent by the server side to the player is usually larger than the coordinate interval determined by the player side, namely, the angle range of the user visual field is determined according to the observation point position of the user in the viewpoint position information and the observation direction of the user; determining a coordinate interval of each frame of video to be played according to the angle range of the user visual field, wherein the coordinate interval of each frame of video to be played is larger than the angle range of the user visual field; and determining a video unit to be played according to the coordinate interval of each frame of video frame to be played.
That is, when the angle range of the field of view of the user is (10 °, 120 °), the server side determines that the range of the coordinate interval of each frame of video frame is (0, 240 °), and then compresses the video unit corresponding to the coordinate interval of each frame of video frame within a set time length, for example, five seconds, which is (0, 240 °) into one video stream to be transmitted to the player, so that the viewing direction of the user is changed from (10 °, 120 °) to (60 °, 180 °) within the five seconds in time, and then the 5 seconds of video stream previously provided by the server can still meet the viewing requirements of the user.
in addition, if the user performs the pause processing in the playing process of the video, and it is considered that the user may watch the panoramic video at each angle in the pause process at this time, when the player performs the pause, the player sends a request to the server side, where the request includes a stop time point of the paused video, and when the server side receives the request, the server side simultaneously sends the video frame corresponding to the time point to the player side, and the player performs playing after receiving the video frame, so that the user can browse the panoramic picture at the time point.
Because the motion information in the player can be timely sent to the server side, the server side in the embodiment of the invention can further predict the track of the viewpoint position information of the user according to the operation information transmitted by the player, namely predict the viewpoint position information of the user in the next time period; determining a video unit in a video frame to be played in the next time interval from the panoramic video according to the predicted viewpoint position information;
and when the viewpoint position information of the next time interval of the user is consistent with the predicted viewpoint position information, sending a video stream of the next time interval to the player, wherein the video stream of the next time interval is composed of video units in the video frame to be played of the next time interval.
for example, the server side predicts that the coordinate interval corresponding to the viewpoint position information of the user is likely to be changed to (120 ° and 240 °) within the next five seconds, the server side can compress the video unit of the sub-coordinate interval corresponding to (100 ° and 280 °) of each frame of video frame within the five seconds to obtain a video stream, the server side stores the video stream as a standby video stream at the server side, and the server side timely transmits the standby video stream to the player when the coordinate interval corresponding to the playing request sent by the player at the next time to the server side is indeed consistent with the prediction.
Based on the same inventive concept, an embodiment of the present invention provides a panoramic video playing method from a player perspective, as shown in fig. 3, the method includes:
step S201, a player sends a panoramic video playing request to a server side, wherein the panoramic video playing request comprises viewpoint position information of a user;
Step S202, the player receives a video stream sent by the server side, wherein the video stream comprises a video unit to be played, which is determined by the server side according to the viewpoint position information of the user; the video unit is obtained by pre-dividing a video frame of a panoramic video to be played at the server side.
wherein the player may be a virtual reality headset comprising first to sixth sides (based on which it may be determined that the first to sixth sides of the headset are respectively front, rear, upper, lower, left, right with respect to the camera, assuming that the virtual reality headset is worn on the head of a user in normal use, as shown in fig. 4). The infrared lamps (also called infrared points) are arranged on each side surface, wherein the infrared lamps on each side surface are arranged according to a preset arrangement rule.
Further, after the player receives a video stream composed of video units to be played and sent by the server side, the method further includes:
the player splices video units corresponding to the sub-coordinate intervals belonging to the same video frame according to the sub-coordinate intervals of each video unit in the received video stream to obtain spliced video frames;
And the player plays the spliced video frames according to the time sequence.
Specifically, the player splices the sub-coordinate intervals of each video unit in the received video stream according to the coordinate attributes to obtain an image spliced by the video units of each frame of video frame; and the player plays the images spliced by each frame of video frame according to the time sequence.
That is to say, after receiving the video stream sent by the server side, the player performs splicing according to the sub-coordinate interval of each video unit in the video stream, for example, it is mentioned above that a video frame of (0, 360 °) is cut into video units with each degree as one sub-coordinate interval, so if the coordinate interval corresponding to the current viewpoint position information of the user is approximately (10 °, 120 °), the player needs to splice the video units of each sub-coordinate interval in (0, 240 °) in the video stream according to the coordinate attribute and the sequence of time, and after the splicing is completed, normal playing can be performed.
Based on the same technical concept, the embodiment of the invention also provides a server and a player, and the server and the player can execute the method embodiment. As shown in fig. 5, the server provided in the embodiment of the present invention includes: receiving unit 401, determining unit 402, and transmitting unit 403, where:
A receiving unit 401, configured to receive a panoramic video playing request of a player, where the panoramic video playing request includes viewpoint position information of a user;
a determining unit 402, configured to determine a video unit to be played according to the viewpoint position information of the user, where the video unit is obtained by pre-dividing a video frame of a panoramic video to be played;
A sending unit 403, configured to send the determined video unit to be played to the player in a video stream manner.
Further, the panoramic video playing request further includes an identifier of a panoramic video to be played, and the determining unit 402 is further configured to: determining at least one frame of video frame to be played according to the identification of the panoramic video to be played;
or the panoramic video playing request is viewpoint position updating information; the determining unit 402 is further configured to: and determining at least one frame of video frame to be played from the panoramic video being played according to the viewpoint position updating information.
Further, still include: a dividing unit 404, configured to divide the video frame into video units corresponding to a plurality of sub-coordinate intervals in a direction according to a coordinate interval covered by the video frame in the direction, where the sub-coordinate intervals are subsets of the coordinate intervals, and a total set of all sub-coordinate intervals is the same as the coordinate interval covered by the direction.
wherein the determining unit 402 is specifically configured to: determining the angle range of the user field of vision according to the observation point position of the user in the viewpoint position information and the observation direction of the user;
Determining a coordinate interval of each frame of video to be played according to the angle range of the user visual field, wherein the coordinate interval of each frame of video to be played is larger than the angle range of the user visual field;
And determining a video unit to be played according to the coordinate interval of each frame of video frame to be played.
Further, still include: a prediction unit 405 for predicting viewpoint position information of a user for a next period;
the determining unit 402 is further configured to: determining a video unit in a video frame to be played in the next time interval from the panoramic video according to the predicted viewpoint position information;
the sending unit 403 is further configured to: and when the viewpoint position information of the next time interval of the user is consistent with the predicted viewpoint position information, sending a video stream of the next time interval to the player, wherein the video stream of the next time interval is composed of video units in the video frame to be played of the next time interval.
based on the same inventive concept, an embodiment of the present invention further provides a panoramic video player, as shown in fig. 5, including: a sending unit 501 and a receiving unit 502, wherein:
a sending unit 501, configured to send a panoramic video playing request to a server side, where the panoramic video playing request includes viewpoint position information of a user;
a receiving unit 502, configured to receive a video stream sent by the server side, where the video stream includes a video unit to be played that is determined by the server side according to the viewpoint position information of the user; the video unit is obtained by pre-dividing a video frame of a panoramic video to be played at the server side.
Further, the video splicing device further includes a splicing unit 503, configured to splice video units corresponding to sub-coordinate intervals belonging to the same video frame according to the sub-coordinate interval of each video unit in the received video stream, so as to obtain a spliced video frame; and playing the spliced images of each frame of video frame according to the time sequence.
In summary, in the embodiments of the present invention, a plurality of video frames are obtained by scanning a panoramic video, and then the video frames are segmented according to the coordinate attributes of the video frames to obtain a plurality of video units, that is, the panoramic video is decomposed in advance; when a playing request of the panoramic video is received, the video unit of the panoramic video corresponding to the viewpoint position information is determined according to the viewpoint position information of the user carried in the playing request, and the video stream formed by the video unit is sent to the player.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
it will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.