CN106604041B - Panoramic video distribution method and system based on visual continuity - Google Patents

Panoramic video distribution method and system based on visual continuity Download PDF

Info

Publication number
CN106604041B
CN106604041B CN201611087528.2A CN201611087528A CN106604041B CN 106604041 B CN106604041 B CN 106604041B CN 201611087528 A CN201611087528 A CN 201611087528A CN 106604041 B CN106604041 B CN 106604041B
Authority
CN
China
Prior art keywords
sequence
code stream
angle
stream
code
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201611087528.2A
Other languages
Chinese (zh)
Other versions
CN106604041A (en
Inventor
马国强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Mengwang Video Co Ltd
Original Assignee
Shenzhen Mengwang Video Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Mengwang Video Co Ltd filed Critical Shenzhen Mengwang Video Co Ltd
Priority to CN201611087528.2A priority Critical patent/CN106604041B/en
Publication of CN106604041A publication Critical patent/CN106604041A/en
Application granted granted Critical
Publication of CN106604041B publication Critical patent/CN106604041B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists
    • H04N21/26208—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists the scheduling operation being performed under constraints
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
    • H04N21/64723—Monitoring of network processes or resources, e.g. monitoring of network load
    • H04N21/64738—Monitoring network characteristics, e.g. bandwidth, congestion level

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer Security & Cryptography (AREA)
  • Television Signal Processing For Recording (AREA)
  • Studio Devices (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The invention discloses a panoramic video distribution method and system based on visual continuity. The method of the invention utilizes the continuity of human vision to carry out self-adaptive distribution on the multi-path code stream spliced by the panoramic video. Therefore, the panoramic video can distribute all paths of code streams according to the network congestion state, and the smoothness of playing the panoramic video is improved without influencing the watching effect of audiences.

Description

Panoramic video distribution method and system based on visual continuity
Technical Field
The invention relates to the field of video distribution, in particular to a panoramic video distribution method and system based on visual continuity.
Background
The panoramic video can provide dynamic real-time video information while providing omnidirectional visual information, so that people can see the whole dynamic scene surrounding the camera. Panoramas are typically large resolution images, the generation of which requires a significant amount of computational effort. Compared with a panoramic image, the panoramic video adds a new time dimension. Therefore, the amount of computation and data will be tens of times larger than that of the panoramic view, which is a great challenge to the processing power of the hardware system and the working efficiency of the software system. When the code stream of the panoramic video is distributed and transmitted, great pressure is exerted on the bandwidth of the network, and the mobile network with extremely limited bandwidth is extremely challenging.
Disclosure of Invention
The embodiment of the invention aims to provide a panoramic video distribution method based on visual continuity, and aims to solve the problems of large calculation amount, low efficiency and unsmooth panoramic video distribution in the prior art.
The embodiment of the invention is realized in such a way that a panoramic video distribution method based on visual continuity comprises the following steps:
step A: one code stream in multiple paths of optional panoramic video code streams is taken as a reference code stream and recorded as a streamcurStream ofcurThe corresponding camera collects the view angle as a reference angle, and the reference angle is recorded as thetacur；
And B: calculating the camera collecting visual angle theta corresponding to each path of code stream of the panoramic videonAnd thetacurAngle difference d theta ofn；θnRepresents streamnA corresponding camera collects a visual angle; streamnThe nth spliced code stream representing the panoramic video,n is 1,2, the number of spliced code streams is represented by N;
and C: arranging multiple paths of code streams of the panoramic video into two code stream queues;
step D: sending const second panoramic video multi-path code streamnN is 1,2,. cndot.n; const represents a time window duration parameter;
step E: in the const second time window, sending the visual angle information in the time window
Figure GDA0002166832000000021
Figure GDA0002166832000000022
Representing the upper and lower visual angle limits of the nth time window;
step F: if the panoramic video multi-path code streamnIf the code stream in the nt +1 time window exists, entering the step G; otherwise, ending the distribution of the multi-path code stream of the panoramic video; nt represents a time window index, and the initial value is 1;
step G: distributing multi-path code streams of the panoramic video according to the visual angle information;
step H: let nt be nt +1, determine const,
Figure GDA0002166832000000023
then re-enter step D.
Wherein d θn＝θn-θcur，n＝1,2,...,N；
Figure GDA0002166832000000024
t0Representing the initial buffering time of the transmitted code stream, and taking 3-6 seconds; t is tcRepresenting the buffering time of the sending code stream;
Figure GDA0002166832000000025
min (variable | condition) and max (variable | condition) respectively represent minimum and maximum values of variables satisfying the condition;
Figure GDA0002166832000000026
respectively representing the upper time limit and the lower time limit of the nth time window,
Figure GDA0002166832000000027
another object of an embodiment of the present invention is to provide a panoramic video distribution system based on visual continuity, the system including:
a reference code stream and reference angle setting module for selecting one code stream in multiple paths of code streams of the panoramic video as a reference code stream to be recorded as a streamcurStream ofcurThe corresponding camera collects the view angle as a reference angle, and the reference angle is recorded as thetacur(ii) a Wherein, streamnRepresenting the nth spliced code stream of the panoramic video, wherein N is 1,2, and N represent the number of the spliced code streams;
an angle difference calculation module for calculating the camera collection view angle and theta corresponding to each path of code stream of the panoramic videocurAngle difference d theta ofn，dθn＝θn-θcur，n＝1,2,...,N；θnRepresents streamnA corresponding camera collects a visual angle;
the code stream queue sequencing device is used for arranging the multi-path code streams of the panoramic video into two code stream queues;
a first multi-path code stream sending module for sending const second panoramic video multi-path code streamnN is 1,2,. cndot.n; const represents a time window duration parameter;
a visual angle information sending module for sending the visual angle information in the time window in the const second time window
Figure GDA0002166832000000031
Wherein the content of the first and second substances,
Figure GDA0002166832000000032
representing the upper and lower visual angle limits of the nth time window; a first judgment processing module for judging if the panoramic video multi-path code stream is streamnIf the code stream in the nt +1 time window exists, the multi-path is enteredA code stream distribution module; otherwise, ending the distribution of the multi-path code stream of the panoramic video;
the multi-path code stream distribution device is used for distributing multi-path code streams of the panoramic video according to the visual angle information;
a setting module, configured to make nt equal to nt +1, determine const,
Figure GDA0002166832000000033
then, the first multi-path code stream sending module is re-entered.
Wherein the content of the first and second substances,
Figure GDA0002166832000000034
t0representing the initial buffering time of the transmitted code stream, and taking 3-6 seconds; t is tcRepresenting the buffering time of the sending code stream; nt represents a time window index, and the initial value is 1;
Figure GDA0002166832000000035
min (variable | condition) and max (variable | condition) respectively represent minimum and maximum values of variables satisfying the condition;
Figure GDA0002166832000000036
the invention has the advantages of
The invention provides a panoramic video distribution method based on visual continuity. The method of the invention utilizes the continuity of human vision to carry out self-adaptive distribution on the multi-path code stream spliced by the panoramic video. Therefore, the panoramic video can distribute all paths of code streams according to the network congestion state, and the smoothness of playing the panoramic video is improved without influencing the watching effect of audiences.
Drawings
Fig. 1 is a flowchart of a panoramic video distribution method based on visual continuity according to a preferred embodiment of the present invention;
FIG. 2 is a flowchart of the detailed method of Step2 in FIG. 1;
FIG. 3 is a flowchart of the detailed method of Step6 in FIG. 1;
fig. 4 is a block diagram of a panoramic video distribution system based on visual continuity according to a preferred embodiment of the present invention;
FIG. 5 is a detailed structure diagram of the code stream queue sorting apparatus in FIG. 4;
FIG. 6 is a detailed block diagram of the multi-path code stream distribution apparatus in FIG. 4;
FIG. 7 is a detailed configuration view of the first processing device of FIG. 6;
fig. 8 is a detailed configuration diagram of the second processing device in fig. 6.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and examples, and for convenience of description, only parts related to the examples of the present invention are shown. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The embodiment of the invention provides a panoramic video distribution method based on visual continuity. The method of the embodiment of the invention utilizes the continuity of human vision to carry out self-adaptive distribution on the multi-path code stream spliced by the panoramic video. Therefore, the panoramic video can distribute all paths of code streams according to the network congestion state, and the smoothness of playing the panoramic video is improved without influencing the watching effect of audiences.
Example one
Fig. 1 is a flowchart of a panoramic video distribution method based on visual continuity according to a preferred embodiment of the present invention; the method comprises the following steps:
step 0: one code stream in multiple paths of optional panoramic video code streams is taken as a reference code stream and recorded as a streamcurStream ofcurThe corresponding camera collects the view angle as a reference angle, and the reference angle is recorded as thetacur。
Wherein, streamnRepresenting the nth spliced code stream of the panoramic video, wherein N is 1,2, and N represent the number of the spliced code streams; thetanRepresents streamnThe corresponding camera captures the view angle.
Step 1: calculating the collecting visual angle and theta of camera corresponding to each path of code stream of panoramic videocurAngle difference d theta ofn，dθn＝θn-θcur，n＝1,2,...,N。
Step 2: and arranging the multi-path code streams of the panoramic video into two code stream queues.
FIG. 2 is a flowchart of the detailed method of Step2 in FIG. 1; the method comprises the following steps:
step21, arranging all the angle differences less than or equal to 0 in ascending order, and recording the angle differences as
Figure GDA0002166832000000051
Wherein k1 represents the number of all angular differences equal to or less than 0;
step22, arranging the corresponding camera collection visual angles into an angle sequence according to the arrangement sequence of all the angle differences less than or equal to 0
Figure GDA0002166832000000052
Referred to as a first angular sequence; arranging multiple paths of code streams of the corresponding panoramic video into a code stream sequence and recording the code stream sequence
Figure GDA0002166832000000053
Called a first code stream sequence;
step23, arranging all the angle differences larger than 0 in descending order and recording the angle differences as
Figure GDA0002166832000000054
Step24, arranging the corresponding camera collection visual angles into an angle sequence according to the arrangement sequence of all the angle differences larger than 0, and recording the angle sequence as
Figure GDA0002166832000000055
Referred to as a second angular sequence; arranging multiple paths of code streams of the corresponding panoramic video into a code stream sequence and recording the code stream sequence
Figure GDA0002166832000000056
Referred to as a second code stream sequence.
Step 3: sending const second panoramic video multi-path code stream streamn，n＝1,2,...,N。
Wherein the content of the first and second substances,
Figure GDA0002166832000000057
const represents a time window duration parameter; t is t0The initial buffering time of the sending code stream is shown, and can be 3-6 seconds generally; t is tcRepresenting the buffering time of the sending code stream; nt denotes a time window index and has an initial value of 1.
Step 4: in the const second time window, sending the visual angle information in the time window
Figure GDA0002166832000000058
Wherein the visual angle information comprises
Figure GDA0002166832000000059
Indicating the upper and lower visual angle limits for the nth time window,
Figure GDA00021668320000000510
min (variable | condition) and max (variable | condition) respectively represent minimum and maximum values of variables satisfying the condition;
Figure GDA00021668320000000511
step 5: if the panoramic video multi-path code streamnIf the code stream in the nt +1 time window exists, entering Step 6; otherwise, ending the distribution of the panoramic video multi-path code stream.
Step 6: and distributing multi-path code streams of the panoramic video according to the visual angle information.
FIG. 3 is a flowchart of the detailed method of Step6 in FIG. 1; the method comprises the following steps:
step 61: computing
Figure GDA00021668320000000512
Referred to as the sign view angle.
Step 62: first, calculate
Figure GDA0002166832000000061
Then, finding out the camera collecting angle corresponding to the minimum value, and recording the camera collecting angle as thetanowAnd the corresponding code stream is marked as streamnow(ii) a abs (variable) means taking the absolute value of the variable;
step 63: if streamnowIf the first code stream sequence is located, a first processing mode is adopted; otherwise, the second processing mode is adopted.
A first processing mode:
step J1: find streamnowA position in the first code stream sequence.
Step J2: and rearranging the second code stream sequence and the second angle sequence.
Step J21: all the first code stream sequences are positioned in the streamnowAnd the former code streams are put into the second code stream sequence one by one according to the reverse order, the code streams in the original second code stream sequence are sequentially moved backwards, the code streams exceeding the sequence length are sequentially moved out and put into the tail part of the first code stream sequence, and the updated second code stream sequence is obtained.
Step J22: all are positioned at theta in the first angle sequencenowAnd putting the previous angles into the second angle sequence one by one according to a reverse order, sequentially moving the angles in the original second angle sequence backwards, sequentially moving the angles exceeding the length of the sequence out of the tail part of the first angle sequence, and acquiring an updated second angle sequence.
Step J3: rearranging the first code stream sequence and the first angle sequence:
will streamnowMoving to the first bit of the first code stream sequence, and simultaneously locating all the bits in the streamnowThe latter code stream moves forwards sequentially to obtain an updated first code stream sequence; then, theta is measurednowMove to the first of the first angular sequence while all will be at θnowAnd the later code streams sequentially move forwards to obtain an updated first angle sequence.
A second processing mode:
step L1: find streamnowA position in the second sequence of code streams.
Step L2: and rearranging the first code stream sequence and the first angle sequence.
Step L21: stream of the second code stream sequencenowAnd all are located in streamnowThe former code streams are put into the first code stream sequence one by one according to the reverse order, the code streams in the original first code stream sequence are sequentially moved backwards, the code streams exceeding the sequence length are sequentially moved out and put into the tail part of the second code stream sequence, and the updated first code stream sequence is obtained.
Step L22: in the second angle sequence, thetanowAnd all lie at θnowThe previous angles are put into the first angle sequence one by one according to the reverse order, the angles in the original first angle sequence are sequentially moved backwards, the angles exceeding the length of the sequence are sequentially moved out to be put into the tail part of the second angle sequence, and the updated first angle sequence is obtained.
Step L3: rearranging the second code stream sequence and the second angle sequence: all are located in streamnowThe latter code streams move forward in sequence until one code stream moves to the first position of the second code stream sequence, and an updated second code stream sequence is obtained; will all be located at thetanowAnd the subsequent code streams are sequentially moved forwards until an angle is moved to the first position of the second angle sequence, and the updated second angle sequence is obtained.
Step 64: constructing a panoramic video multi-path code stream distribution sequence table:
Figure GDA0002166832000000071
Figure GDA0002166832000000072
step 65: and sequentially transmitting the code streams in nt +1 time windows corresponding to the multi-path code streams of the panoramic video according to the sequence of the multi-path code stream distribution sequence table of the panoramic video.
Step 7: let nt be nt +1, determine const,
Figure GDA0002166832000000073
then Step3 is re-entered.
Example two
Fig. 4 is a block diagram of a panoramic video distribution system based on visual continuity according to a preferred embodiment of the present invention; the system comprises:
a reference code stream and reference angle setting module for selecting one code stream in multiple paths of code streams of the panoramic video as a reference code stream to be recorded as a streamcurStream ofcurThe corresponding camera collects the view angle as a reference angle, and the reference angle is recorded as thetacur。
Wherein, streamnRepresenting the nth spliced code stream of the panoramic video, wherein N is 1,2, and N represent the number of the spliced code streams; thetanRepresents streamnThe corresponding camera captures the view angle.
An angle difference calculation module for calculating the camera collection view angle and theta corresponding to each path of code stream of the panoramic videocurAngle difference d theta ofn，dθn＝θn-θcur，n＝1,2,...,N。
And the code stream queue sequencing device is used for arranging the multi-path code streams of the panoramic video into two code stream queues.
A first multi-path code stream sending module for sending const second panoramic video multi-path code streamn，n＝1,2,...,N。
Wherein the content of the first and second substances,
Figure GDA0002166832000000081
const represents a time window duration parameter; t is t0The initial buffering time of the sending code stream is shown, and can be 3-6 seconds generally; t is tcRepresenting the buffering time of the sending code stream; nt denotes a time window index and has an initial value of 1.
A visual angle information sending module for sending the visual angle information in the time window in the const second time window
Figure GDA0002166832000000082
Wherein the visual angle information comprises
Figure GDA0002166832000000083
Indicating the upper and lower visual angle limits for the nth time window,
Figure GDA0002166832000000084
min (variable | condition) and max (variable | condition) respectively represent minimum and maximum values of variables satisfying the condition;
Figure GDA0002166832000000085
a first judgment processing module for judging if the panoramic video multi-path code stream is streamnIf the code stream in the nt +1 time window exists, entering a multi-path code stream distribution module; otherwise, ending the distribution of the panoramic video multi-path code stream.
And the multi-path code stream distribution device is used for distributing the multi-path code streams of the panoramic video according to the visual angle information.
A setting module, configured to make nt equal to nt +1, determine const,
Figure GDA0002166832000000086
then, the first multi-path code stream sending module is re-entered.
Further, fig. 5 is a detailed structure diagram of the code stream queue sorting apparatus in fig. 4; the code stream queue sequencing device comprises:
the ascending order arrangement module of the angle difference is used for ascending order arrangement of all the angle differences less than or equal to 0 and recording the angle differences as
Figure GDA0002166832000000087
Wherein k1 represents the number of all angular differences equal to or less than 0;
a first angle sequence and a first code stream sequence setting module, configured to arrange corresponding camera acquisition views into an angle sequence according to the arrangement sequence of all angle differences smaller than or equal to 0
Figure GDA0002166832000000088
Referred to as a first angular sequence; arranging multiple paths of code streams of the corresponding panoramic video into a code stream sequence and recording the code stream sequence
Figure GDA0002166832000000089
Called a first code stream sequence;
the angle difference descending order arrangement module is used for descending order arrangement of all the angle differences larger than 0 and recording the angle differences as
Figure GDA00021668320000000810
A second angle sequence and second code stream sequence setting module, configured to arrange corresponding camera acquisition views into an angle sequence according to the arrangement sequence of all angle differences greater than 0, and record the angle sequence as
Figure GDA0002166832000000091
Referred to as a second angular sequence; arranging multiple paths of code streams of the corresponding panoramic video into a code stream sequence and recording the code stream sequence
Figure GDA0002166832000000092
Referred to as a second code stream sequence.
Further, fig. 6 is a detailed structure diagram of the multi-path code stream distribution apparatus in fig. 4; the multi-path code stream distribution device comprises:
a sign view angle calculation module for calculating
Figure GDA0002166832000000093
Referred to as the sign view angle.
A third code stream acquisition module for first calculating
Figure GDA0002166832000000094
Then, finding out the camera collecting angle corresponding to the minimum value, and recording the camera collecting angle as thetanowAnd the corresponding code stream is marked as streamnow(ii) a abs (variable) means taking the absolute value of the variable;
a second judgment processing module for if streamnowIf the first code stream sequence is located, entering a first processing device; otherwise, entering a second processing device.
The distribution sequence table building module is used for building a panoramic video multi-path code stream distribution sequence table:
Figure GDA0002166832000000095
Figure GDA0002166832000000096
and the second multi-path code stream sending module is used for sequentially sending the code streams in nt +1 time windows corresponding to the multi-path code streams of the panoramic video according to the sequence of the multi-path code stream distribution sequence table of the panoramic video.
Further, fig. 7 is a detailed structural view of the first processing device in fig. 6; the first processing device includes:
a first position finding module for finding streamnowA position in the first code stream sequence.
The first reordering module is used for reordering the second code stream sequence and the second angle sequence.
Step J21: all the first code stream sequences are positioned in the streamnowAnd the former code streams are put into the second code stream sequence one by one according to the reverse order, the code streams in the original second code stream sequence are sequentially moved backwards, the code streams exceeding the sequence length are sequentially moved out and put into the tail part of the first code stream sequence, and the updated second code stream sequence is obtained.
Step J22: all are positioned at theta in the first angle sequencenowAnd putting the previous angles into the second angle sequence one by one according to a reverse order, sequentially moving the angles in the original second angle sequence backwards, sequentially moving the angles exceeding the length of the sequence out of the tail part of the first angle sequence, and acquiring an updated second angle sequence.
The second reordering module is used for reordering the first code stream sequence and the first angle sequence:
will streamnowMoving to the first bit of the first code stream sequence, and simultaneously locating all the bits in the streamnowThe latter code stream moves forwards sequentially to obtain an updated first code stream sequence; then, theta is measurednowMove to the first of the first angular sequence while all will be at θnowAnd the later code streams sequentially move forwards to obtain an updated first angle sequence.
Further, fig. 8 is a detailed configuration diagram of the second processing device in fig. 6. The second processing device includes:
a second position finding module for finding streamnowA position in the second sequence of code streams.
And the third reordering module is used for reordering the first code stream sequence and the first angle sequence.
Step L21: stream of the second code stream sequencenowAnd all are located in streamnowThe former code streams are put into the first code stream sequence one by one according to the reverse order, the code streams in the original first code stream sequence are sequentially moved backwards, the code streams exceeding the sequence length are sequentially moved out and put into the tail part of the second code stream sequence, and the updated first code stream sequence is obtained.
Step L22: in the second angle sequence, thetanowAnd all lie at θnowThe previous angles are put into the first angle sequence one by one according to the reverse order, the angles in the original first angle sequence are sequentially moved backwards, the angles exceeding the length of the sequence are sequentially moved out to be put into the tail part of the second angle sequence, and the updated first angle sequence is obtained.
A fourth reordering module, configured to reorder the second code stream sequence and the second angle sequence:
all are located in streamnowThe latter code streams move forward in sequence until one code stream moves to the first position of the second code stream sequence, and an updated second code stream sequence is obtained; will all be located at thetanowAnd the subsequent code streams are sequentially moved forwards until an angle is moved to the first position of the second angle sequence, and the updated second angle sequence is obtained.
It will be understood by those skilled in the art that all or part of the steps in the method according to the above embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, such as ROM, RAM, magnetic disk, optical disk, etc.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.

Claims (4)

1. A panoramic video distribution method based on visual continuity is characterized by comprising the following steps:
step A: one code stream in multiple paths of optional panoramic video code streams is taken as a reference code stream and recorded as a streamcurStream ofcurThe corresponding camera collects the view angle as a reference angle, and the reference angle is recorded as thetacur；
And B: calculating the camera collecting visual angle theta corresponding to each path of code stream of the panoramic videonAnd thetacurAngle difference d theta ofn；θnRepresents streamnA corresponding camera collects a visual angle; streamnRepresenting the nth spliced code stream of the panoramic video, wherein N is 1,2, and N represent the number of the spliced code streams;
and C: arranging multiple paths of code streams of the panoramic video into two code stream sequences;
step D: sending const second panoramic video multi-path code streamnN is 1,2,. cndot.n; const represents a time window duration parameter;
step E: in the const second time window, sending the visual angle information in the time window
Figure FDA0002372524650000011
Figure FDA0002372524650000012
Representing the upper and lower visual angle limits of the nth time window;
step F: if the panoramic video multi-path code streamnIf the code stream in the nt +1 time window exists, entering the step G; otherwise, ending the distribution of the multi-path code stream of the panoramic video; nt represents a time window index, and the initial value is 1;
step G: distributing multi-path code streams of the panoramic video according to the visual angle information;
step H: let nt be nt +1, determine const,
Figure FDA0002372524650000013
then re-entering step D;
Figure FDA0002372524650000014
respectively representing the upper time limit and the lower time limit of the nth time window;
the distributing of the multi-path code streams of the panoramic video according to the visual angle information specifically comprises the following steps:
computing
Figure FDA0002372524650000015
Referred to as the sign view;
computing
Figure FDA0002372524650000016
Then, finding out the camera collecting angle corresponding to the minimum value, and recording the camera collecting angle as thetanowAnd the corresponding code stream is marked as streamnow(ii) a abs (variable) means taking the absolute value of the variable;
if streamnowIf the first code stream sequence is located, a first processing mode is adopted; otherwise, adopting a second processing mode;
constructing a panoramic video multi-path code stream distribution sequence table:
Figure FDA0002372524650000021
(if N-k1 ≧ k1),
Figure FDA0002372524650000022
(if N-k1<k1)，
Sequentially sending code streams in nt +1 time windows corresponding to the multi-path code streams of the panoramic video according to the sequence of the multi-path code stream distribution sequence table of the panoramic video;
the first processing mode is as follows:
find streamnowA position in the first code stream sequence;
rearranging the second code stream sequence and the second angle sequence; the method specifically comprises the following steps:
all the first code stream sequences are positioned in the streamnowThe former code stream is put one by one according to the reverse orderEntering a second code stream sequence, sequentially moving back code streams in the original second code stream sequence, sequentially moving out code streams exceeding the sequence length and placing the code streams in the tail part of the first code stream sequence to obtain an updated second code stream sequence; all are positioned at theta in the first angle sequencenowThe previous angles are put into the second angle sequence one by one according to the reverse order, the angles in the original second angle sequence are sequentially moved backwards, the angles exceeding the length of the sequence are sequentially moved out of the tail part of the first angle sequence, and an updated second angle sequence is obtained;
rearranging the first code stream sequence and the first angle sequence; the method specifically comprises the following steps:
will streamnowMoving to the first bit of the first code stream sequence, and simultaneously locating all the bits in the streamnowThe latter code stream moves forwards sequentially to obtain an updated first code stream sequence; then, theta is measurednowMove to the first of the first angular sequence while all will be at θnowThe later code stream sequentially moves forwards to obtain an updated first angle sequence;
the second processing mode is as follows:
find streamnowA position in the second sequence of code streams;
rearranging the first code stream sequence and the first angle sequence; the method specifically comprises the following steps:
stream of the second code stream sequencenowAnd all are located in streamnowThe former code streams are put into the first code stream sequence one by one according to the reverse order, the code streams in the original first code stream sequence are sequentially moved backwards, the code streams exceeding the sequence length are sequentially moved out and put into the tail part of the second code stream sequence, and the updated first code stream sequence is obtained; in the second angle sequence, thetanowAnd all lie at θnowThe previous angles are put into the first angle sequence one by one according to a reverse order, the angles in the original first angle sequence are sequentially moved backwards, the angles exceeding the length of the sequence are sequentially moved out of the tail part of the second angle sequence, and an updated first angle sequence is obtained;
rearranging the second code stream sequence and the second angle sequence; the method specifically comprises the following steps:
all are located in streamnowThe latter code streams move forward in sequence until one code stream moves to a second code stream sequenceObtaining an updated second code stream sequence; will all be located at thetanowThe latter code stream moves forwards in sequence until an angle moves to the first position of the second angle sequence, and an updated second angle sequence is obtained;
the arranging of the multiple panoramic video code streams into two code stream sequences specifically comprises:
all the angle differences less than or equal to 0 are arranged in ascending order and are recorded as
Figure FDA0002372524650000031
Wherein k1 represents the number of all angular differences equal to or less than 0;
respectively arranging the corresponding camera acquisition visual angles into an angle sequence according to the arrangement sequence of all the angle differences less than or equal to 0
Figure FDA0002372524650000032
Referred to as a first angular sequence; arranging multiple paths of code streams of the corresponding panoramic video into a code stream sequence and recording the code stream sequence
Figure FDA0002372524650000033
Called a first code stream sequence;
all angle differences greater than 0 are sorted in descending order and are recorded as
Figure FDA0002372524650000034
Respectively arranging the corresponding camera acquisition visual angles into an angle sequence according to the arrangement sequence of all the angle differences larger than 0, and recording the angle sequence as
Figure FDA0002372524650000035
Referred to as a second angular sequence; arranging multiple paths of code streams of the corresponding panoramic video into a code stream sequence and recording the code stream sequence
Figure FDA0002372524650000036
Referred to as a second code stream sequence.
2. The visual continuity-based panoramic video distribution method according to claim 1,
dθn＝θn-θcur，n＝1,2,...,N；
Figure FDA0002372524650000037
t0representing the initial buffering time of the transmitted code stream, and taking 3-6 seconds; t is tcRepresenting the buffering time of the sending code stream;
Figure FDA0002372524650000038
min (variable | condition) and max (variable | condition) respectively represent minimum and maximum values of variables satisfying the condition;
Figure FDA0002372524650000039
3. a panoramic video distribution system based on visual continuity, the system comprising:
a reference code stream and reference angle setting module for selecting one code stream in multiple paths of code streams of the panoramic video as a reference code stream to be recorded as a streamcurStream ofcurThe corresponding camera collects the view angle as a reference angle, and the reference angle is recorded as thetacur(ii) a Wherein, streamnRepresenting the nth spliced code stream of the panoramic video, wherein N is 1,2, and N represent the number of the spliced code streams;
an angle difference calculation module for calculating the camera collection view angle and theta corresponding to each path of code stream of the panoramic videocurAngle difference d theta ofn，dθn＝θn-θcur，n＝1,2,...,N；θnRepresents streamnA corresponding camera collects a visual angle;
the code stream sequence ordering device is used for arranging the multi-path code streams of the panoramic video into two code stream sequences;
first multi-path code stream sending moduleAnd is used for transmitting const second panoramic video multi-path code streamnN is 1,2,. cndot.n; const represents a time window duration parameter;
a visual angle information sending module for sending the visual angle information in the time window in the const second time window
Figure FDA0002372524650000041
Wherein the content of the first and second substances,
Figure FDA0002372524650000042
representing the upper and lower visual angle limits of the nth time window; a first judgment processing module for judging if the panoramic video multi-path code stream is streamnIf the code stream in the nt +1 time window exists, entering a multi-path code stream distribution module; otherwise, ending the distribution of the multi-path code stream of the panoramic video;
the multi-path code stream distribution device is used for distributing multi-path code streams of the panoramic video according to the visual angle information;
a setting module, configured to make nt equal to nt +1, determine const,
Figure FDA0002372524650000043
then, re-entering the first multi-path code stream sending module;
Figure FDA0002372524650000044
respectively representing the upper time limit and the lower time limit of the nth time window;
wherein, the multichannel code stream distribution device includes:
a sign view angle calculation module for calculating
Figure FDA0002372524650000045
Referred to as the sign view;
a third code stream acquisition module for calculating
Figure FDA0002372524650000046
Then, finding the camera collecting angle corresponding to the minimum value, and recording asθnowAnd the corresponding code stream is marked as streamnow(ii) a abs (variable) means taking the absolute value of the variable;
a second judgment processing module for if streamnowIf the first code stream sequence is located, entering a first processing device; otherwise, entering a second processing device;
the distribution sequence table building module is used for building a panoramic video multi-path code stream distribution sequence table:
Figure FDA0002372524650000047
(if N-k1 ≧ k1),
Figure FDA0002372524650000051
(if N-k1<k1)，
The second multi-path code stream sending module is used for sequentially sending code streams in nt +1 time windows corresponding to the multi-path code streams of the panoramic video according to the sequence of the multi-path code stream distribution sequence table of the panoramic video;
the first processing device includes:
a first position finding module for finding streamnowA position in the first code stream sequence;
the first reordering module is used for reordering the second code stream sequence and the second angle sequence; the method specifically comprises the following steps:
all the first code stream sequences are positioned in the streamnowThe former code streams are put into the second code stream sequence one by one according to the reverse order, the code streams in the original second code stream sequence are sequentially moved backwards, the code streams exceeding the sequence length are sequentially moved out to be put into the tail part of the first code stream sequence, and the updated second code stream sequence is obtained;
all are positioned at theta in the first angle sequencenowThe previous angles are put into the second angle sequence one by one according to the reverse order, the angles in the original second angle sequence are sequentially moved backwards, the angles exceeding the length of the sequence are sequentially moved out of the tail part of the first angle sequence, and an updated second angle sequence is obtained;
the second reordering module is used for reordering the first code stream sequence and the first angle sequence; the method specifically comprises the following steps:
will streamnowMoving to the first bit of the first code stream sequence, and simultaneously locating all the bits in the streamnowThe latter code stream moves forwards sequentially to obtain an updated first code stream sequence; then, theta is measurednowMove to the first of the first angular sequence while all will be at θnowThe later code stream sequentially moves forwards to obtain an updated first angle sequence;
the second processing device includes:
a second position finding module for finding streamnowA position in the second sequence of code streams;
the third reordering module is used for reordering the first code stream sequence and the first angle sequence; the method specifically comprises the following steps:
stream of the second code stream sequencenowAnd all are located in streamnowThe former code streams are put into the first code stream sequence one by one according to the reverse order, the code streams in the original first code stream sequence are sequentially moved backwards, the code streams exceeding the sequence length are sequentially moved out and put into the tail part of the second code stream sequence, and the updated first code stream sequence is obtained; in the second angle sequence, thetanowAnd all lie at θnowThe previous angles are put into the first angle sequence one by one according to a reverse order, the angles in the original first angle sequence are sequentially moved backwards, the angles exceeding the length of the sequence are sequentially moved out of the tail part of the second angle sequence, and an updated first angle sequence is obtained;
a fourth reordering module, configured to reorder the second code stream sequence and the second angle sequence; the method specifically comprises the following steps:
all are located in streamnowThe latter code streams move forward in sequence until one code stream moves to the first position of the second code stream sequence, and an updated second code stream sequence is obtained; will all be located at thetanowThe latter code stream moves forwards in sequence until an angle moves to the first position of the second angle sequence, and an updated second angle sequence is obtained;
the code stream sequence sequencing device comprises:
the ascending order arrangement module of the angle difference is used for ascending order arrangement of all the angle differences less than or equal to 0 and recording the angle differences as
Figure FDA0002372524650000061
Wherein k1 represents the number of all angular differences equal to or less than 0;
a first angle sequence and a first code stream sequence setting module, configured to arrange corresponding camera acquisition views into an angle sequence according to the arrangement sequence of all angle differences smaller than or equal to 0
Figure FDA0002372524650000062
Referred to as a first angular sequence; arranging multiple paths of code streams of the corresponding panoramic video into a code stream sequence and recording the code stream sequence
Figure FDA0002372524650000063
Called a first code stream sequence;
the angle difference descending order arrangement module is used for descending order arrangement of all the angle differences larger than 0 and recording the angle differences as
Figure FDA0002372524650000064
A second angle sequence and second code stream sequence setting module, configured to arrange corresponding camera acquisition views into an angle sequence according to the arrangement sequence of all angle differences greater than 0, and record the angle sequence as
Figure FDA0002372524650000065
Referred to as a second angular sequence; arranging multiple paths of code streams of the corresponding panoramic video into a code stream sequence and recording the code stream sequence
Figure FDA0002372524650000066
Referred to as a second code stream sequence.
4. The visual continuity-based panoramic video distribution system of claim 3,
Figure FDA0002372524650000067
t0representing the initial buffering time of the transmitted code stream, and taking 3-6 seconds; t is tcRepresenting the buffering time of the sending code stream; nt represents a time window index, and the initial value is 1;
Figure FDA0002372524650000068
min (variable | condition) and max (variable | condition) respectively represent minimum and maximum values of variables satisfying the condition;
Figure FDA0002372524650000069
CN201611087528.2A 2016-12-01 2016-12-01 Panoramic video distribution method and system based on visual continuity Active CN106604041B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611087528.2A CN106604041B (en) 2016-12-01 2016-12-01 Panoramic video distribution method and system based on visual continuity

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611087528.2A CN106604041B (en) 2016-12-01 2016-12-01 Panoramic video distribution method and system based on visual continuity

Publications (2)

Publication Number Publication Date
CN106604041A CN106604041A (en) 2017-04-26
CN106604041B true CN106604041B (en) 2020-05-22

Family

ID=58594616

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611087528.2A Active CN106604041B (en) 2016-12-01 2016-12-01 Panoramic video distribution method and system based on visual continuity

Country Status (1)

Country Link
CN (1) CN106604041B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109348291B (en) * 2018-11-16 2021-05-18 深圳市梦网视讯有限公司 Panoramic video code stream distribution method and system based on motion analysis

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101600099A (en) * 2009-04-09 2009-12-09 上海交通大学 The real-time transmission synchronous control method of multi-angle video Flow
CN105791882A (en) * 2016-03-22 2016-07-20 腾讯科技（深圳）有限公司 Video coding method and device
CN105933343A (en) * 2016-06-29 2016-09-07 深圳市优象计算技术有限公司 Stream cache mechanism used for network play of 720-degree panoramic video
CN106060513A (en) * 2016-06-29 2016-10-26 深圳市优象计算技术有限公司 Code stream caching mechanism for cylindrical surface stereoscopic panoramic video network playing

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9996895B2 (en) * 2015-05-01 2018-06-12 Ricoh Company, Ltd. Image display system, information processing apparatus, and image display method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101600099A (en) * 2009-04-09 2009-12-09 上海交通大学 The real-time transmission synchronous control method of multi-angle video Flow
CN105791882A (en) * 2016-03-22 2016-07-20 腾讯科技（深圳）有限公司 Video coding method and device
CN105933343A (en) * 2016-06-29 2016-09-07 深圳市优象计算技术有限公司 Stream cache mechanism used for network play of 720-degree panoramic video
CN106060513A (en) * 2016-06-29 2016-10-26 深圳市优象计算技术有限公司 Code stream caching mechanism for cylindrical surface stereoscopic panoramic video network playing

Also Published As

Publication number Publication date
CN106604041A (en) 2017-04-26

Similar Documents

Publication Publication Date Title
US10482925B1 (en) Frame selection of video data
US20210158008A1 (en) UAV Video Aesthetic Quality Evaluation Method Based On Multi-Modal Deep Learning
WO2021109952A1 (en) Video editing method, apparatus and server, and computer readable storage medium
CN108366216A (en) TV news recording, record and transmission method, device and server
CN108010037B (en) Image processing method, device and storage medium
US8903130B1 (en) Virtual camera operator
JP2017507557A (en) Process for improving the quality of experience for users who view high-definition video streams on their devices
US20030202120A1 (en) Virtual lighting system
EP3038056A1 (en) Method and system for processing video content
US8824823B1 (en) Increased quality of image objects based on depth in scene
WO2018102880A1 (en) Systems and methods for replacing faces in videos
CN112132797B (en) Short video quality screening method
US20180129917A1 (en) Neural network training
US10250803B2 (en) Video generating system and method thereof
CN113365156B (en) Panoramic video multicast stream view angle prediction method based on limited view field feedback
CN111382647B (en) Picture processing method, device, equipment and storage medium
CN111510735A (en) Encoding transmission method and device for multi-channel video in weak network environment and unmanned vehicle
CN114071015B (en) Method, device, medium and equipment for determining linkage snapshot path
CN106604041B (en) Panoramic video distribution method and system based on visual continuity
WO2016192467A1 (en) Method and device for playing videos
CN109815861B (en) User behavior information statistical method based on face recognition
CN114095780A (en) Panoramic video editing method, device, storage medium and equipment
CN107707830A (en) Panoramic video based on one-way communication plays camera system
JP2022522579A (en) Equipment, systems, and methods for wearable head-mounted displays
CN106851212B (en) Panoramic video distribution method and system based on user characteristics

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
CB02 Change of applicant information

Address after: 518000 Guangdong city of Shenzhen province Nanshan District Guangdong streets high in the four Longtaili Technology Building Room 325 No. 30

Applicant after: Shenzhen Monternet encyclopedia Information Technology Co. Ltd.

Address before: The central Shenzhen city of Guangdong Province, 518057 Keyuan Road, Nanshan District science and Technology Park No. 15 Science Park Sinovac A Building 1 unit 403, No. 405 unit

Applicant before: BAC Information Technology Co., Ltd.

CB02 Change of applicant information
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information

Address after: 518000 Guangdong city of Shenzhen province Nanshan District Guangdong streets high in the four Longtaili Technology Building Room 325 No. 30

Applicant after: Shenzhen mengwang video Co., Ltd

Address before: 518000 Guangdong city of Shenzhen province Nanshan District Guangdong streets high in the four Longtaili Technology Building Room 325 No. 30

Applicant before: SHENZHEN MONTNETS ENCYCLOPEDIA INFORMATION TECHNOLOGY Co.,Ltd.

CB02 Change of applicant information
GR01 Patent grant
GR01 Patent grant