CN113256491A - Free visual angle data processing method, device, equipment and storage medium - Google Patents
Free visual angle data processing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN113256491A CN113256491A CN202110514671.XA CN202110514671A CN113256491A CN 113256491 A CN113256491 A CN 113256491A CN 202110514671 A CN202110514671 A CN 202110514671A CN 113256491 A CN113256491 A CN 113256491A
- Authority
- CN
- China
- Prior art keywords
- frame
- view
- images
- target
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000000007 visual effect Effects 0.000 title claims abstract description 169
- 238000003672 processing method Methods 0.000 title claims abstract description 13
- 230000004927 fusion Effects 0.000 claims abstract description 166
- 238000000034 method Methods 0.000 claims abstract description 59
- 230000008569 process Effects 0.000 claims abstract description 41
- 238000012545 processing Methods 0.000 claims abstract description 20
- 230000007704 transition Effects 0.000 claims description 50
- 238000007781 pre-processing Methods 0.000 claims description 6
- 238000010586 diagram Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 230000008901 benefit Effects 0.000 description 5
- 230000009471 action Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000000750 progressive effect Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- WYTGDNHDOZPMIW-RCBQFDQVSA-N alstonine Natural products C1=CC2=C3C=CC=CC3=NC2=C2N1C[C@H]1[C@H](C)OC=C(C(=O)OC)[C@H]1C2 WYTGDNHDOZPMIW-RCBQFDQVSA-N 0.000 description 1
- 238000005267 amalgamation Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/32—Indexing scheme for image data processing or generation, in general involving image mosaicing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Television Signal Processing For Recording (AREA)
Abstract
The application discloses a free visual angle data processing method, a free visual angle data processing device, free visual angle data processing equipment and a storage medium. The method comprises the following steps: receiving a visual angle switching instruction in the process of playing video data corresponding to a first visual angle; determining a second view according to the view switching indication; pausing playing of video data corresponding to the first visual angle, and loading a pre-constructed fusion data set; determining M frames of fused images starting from the a +1 th frame in the fused data set as target fused images; playing the target pictures in the M frames of target fusion images one by one; and then playing the data starting from the a + M +1 th frame in the video data corresponding to the second visual angle. Based on the scheme disclosed by the application, the frequency of switching and playing the video data can be obviously reduced, so that the visual angle switching process is greatly simplified. In addition, because only two times of data source switching are required to be executed, the frequency of the Kanton phenomenon is greatly reduced, and the fluency of video watching can be obviously improved.
Description
Technical Field
The present application belongs to the field of data processing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for processing free view angle data.
Background
Currently, the free view technology is increasingly applied to the field of video playing. In the video playing process based on the free visual angle, the user can select the favorite visual angle to watch the shot scene. For example, 18 stands are arranged at a shooting site, and a user can select any one of the stands to view.
When the user needs to switch the view angles, in order to realize the progressive viewing experience, the video data of each transition view angle are loaded and played one by one according to the sequence. For example, if the user currently views at the 2 nd position, and if the user instructs to switch to the 7 th position for viewing, where the 3 rd position, the 4 th position, the 5 th position, and the 6 th position are sequentially arranged between the 2 nd position and the 7 th position, the video data corresponding to the 3 rd position, the 4 th position, the 5 th position, the 6 th position, and the 7 th position need to be loaded and played one by one.
It can be seen that the existing view angle switching scheme has the problems of complex switching process and obvious pause phenomenon in the video playing process.
Disclosure of Invention
In view of the above, an object of the present application is to provide a method, an apparatus, a device and a storage medium for processing free view angle data, which can reduce the number of times of switching playing video data in a view angle switching process, thereby simplifying the view angle switching process and improving a pause phenomenon in the video playing process.
In order to achieve the above purpose, the present application provides the following technical solutions:
in one aspect, the present application provides a method for processing freeview data, including:
receiving a visual angle switching instruction in the process of playing video data corresponding to a first visual angle;
determining a second view according to the view switching indication;
pausing playing of the video data corresponding to the first visual angle, and loading a pre-constructed fusion data set, wherein the fusion data set comprises a plurality of frames of fusion images, the i-th frame of fusion image is formed by splicing the i-th frame of images in the video data corresponding to N visual angles, the video data corresponding to N visual angles keep frame synchronization, and N is greater than or equal to 3;
determining M frames of fused images starting from the a +1 th frame in the fused data set as target fused images, wherein a is a frame number of images played by video data corresponding to the first view at a pause playing moment, the value of M is greater than or equal to the number P of transition views, and the transition views are views spatially located between the first view and the second view in the N views;
target pictures in the M frames of target fusion images are played one by one, wherein the M frames of target fusion images are sequentially divided into P subsets according to the frame number, each subset comprises at least one frame of target fusion image, and the target pictures in the target fusion image in the jth subset are as follows: a j ═ 1,2, …, P of the image corresponding to the jth transition view in the target fusion image, the jth transition view being closer to the first view than the j +1 th transition view;
and playing data starting from the a + M +1 th frame in the video data corresponding to the second visual angle.
Optionally, the receiving a view switching instruction during the playing of the video data corresponding to the first view includes:
and detecting the touch operation of an operation body on the touch screen in the process of playing the video data corresponding to the first visual angle.
Optionally, the determining a second view according to the view switching indication includes:
and determining the second visual angle according to the starting point and the end point of the touch operation and the moving direction of the operation body.
Optionally, the process of determining the number of target fusion images includes:
determining a switching speed matched with a user;
and determining the number M of the target fusion images according to the switching speed, wherein the number M of the target fusion images and the switching speed are in a negative correlation relationship.
Optionally, the process of constructing the fused data set includes:
respectively extracting image frames in the video data corresponding to the N visual angles;
splicing the image frames with the same frame number into a fused image;
and merging the fused images into a fused data set according to the frame number of the fused images.
In another aspect, the present application provides a freeview data processing apparatus, including:
a switching indication receiving unit, configured to receive a view switching indication during playing of video data corresponding to a first view;
a visual angle determining unit, configured to determine a second visual angle according to the visual angle switching instruction;
the data loading unit is used for pausing playing of the video data corresponding to the first visual angle and loading a pre-constructed fusion data set, wherein the fusion data set comprises a plurality of frames of fusion images, the i-th frame of fusion image is formed by splicing the i-th frame of images in the video data corresponding to N visual angles, the video data corresponding to the N visual angles keep frame synchronization, and N is greater than or equal to 3;
a target fused image determining unit, configured to determine that M frames of fused images starting from an a +1 th frame in the fused data set are target fused images, where a is a frame number of an image played by video data corresponding to the first view at a pause playing time, a value of M is greater than or equal to a number P of transition views, and the transition views are views spatially located between the first view and the second view in the N views;
the first playing control unit is used for playing the target pictures in the M frames of target fusion images one by one, wherein the M frames of target fusion images are sequentially divided into P subsets according to the frame number, each subset comprises at least one frame of target fusion image, and the target pictures in the target fusion image in the jth subset are: a j ═ 1,2, …, P of the image corresponding to the jth transition view in the target fusion image, the jth transition view being closer to the first view than the j +1 th transition view;
and the second playing control unit is used for playing the data starting from the a + M +1 th frame in the video data corresponding to the second visual angle after the first playing control unit plays the target picture in the Mth frame target fusion image.
Optionally, the target fusion image determining unit is specifically configured to:
determining a switching speed matched with a user; determining the number M of the target fusion images according to the switching speed, wherein the number M of the target fusion images and the switching speed are in a negative correlation relationship; and determining M frames of fused images starting from the a +1 th frame in the fused data set as target fused images.
Optionally, the freeview data processing apparatus further includes:
and the preprocessing unit is used for respectively extracting the image frames in the video data corresponding to the N visual angles, splicing the image frames with the same frame number into a fused image, and merging the fused image into a fused data set according to the frame number of the fused image.
In another aspect, the present application provides an electronic device comprising a processor and a memory;
the memory is used for storing programs;
the processor is used for calling and executing the program stored in the memory so as to execute any one of the free perspective data processing methods.
In another aspect, the present application provides a storage medium, in which a program is stored, and when the program is loaded and executed by a processor in an electronic device, the method for processing the free perspective data is implemented as any one of the above-mentioned methods.
Therefore, the beneficial effects of the application are as follows:
in the process of playing video data corresponding to a first visual angle, when a visual angle switching instruction is received, a second visual angle (namely a target visual angle) is determined according to the visual angle switching instruction, the playing of the video data corresponding to the first visual angle is paused, a pre-constructed fusion data set is loaded, an ith frame fusion image in the fusion data set is formed by splicing ith frame images in the video data corresponding to N visual angles, target pictures in M frame fusion images from an a +1 frame in the fusion data set are played one by one, wherein a is a frame number of the image played by the video data corresponding to the first visual angle at the time of pausing, M is greater than or equal to the number P of transition visual angles between the first visual angle and the second visual angle, and the M frame target fusion images are sequentially divided into P subsets according to the frame number, the target picture in the target fusion image in the jth subset is: and playing the image corresponding to the jth transition visual angle in the target fusion image, and then playing the data beginning from the a + M +1 th frame in the video data corresponding to the second visual angle. According to the scheme, the video data corresponding to the multiple visual angles are spliced to form the fusion data set, when the visual angles are required to be switched, only the fusion data set which is constructed in advance is required to be switched to play, and then the video data corresponding to the playing target visual angles are switched, so that the frequency of switching and playing the video data is obviously reduced, the visual angle switching process is greatly simplified, and the technical advantage is more obvious along with the increase of the number of the visual angles between the current visual angle and the target visual angles. In addition, because only two times of data source switching are required to be executed, the frequency of the Kanton phenomenon is greatly reduced, and the fluency of video watching can be obviously improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a flow chart of a freeview data processing method disclosed herein;
FIG. 2 is a schematic diagram of an arrangement of an acquisition device as disclosed herein;
FIG. 3 is a schematic illustration of one stitching approach for fusing images as disclosed herein;
FIG. 4 is a schematic illustration of another stitching approach for fused images as disclosed herein;
FIG. 5 is a schematic diagram of a view switching scheme disclosed herein;
FIG. 6 is a schematic structural diagram of a freeview data processing apparatus according to the present disclosure;
fig. 7 is a hardware configuration diagram of an electronic device disclosed in the present application.
Detailed Description
The application provides a free visual angle data processing method, a device, equipment and a storage medium, which can reduce the times of switching and playing video data in the visual angle switching process, thereby simplifying the visual angle switching process and improving the pause phenomenon in the video playing process.
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1, fig. 1 is a flowchart illustrating a method for processing freeview data disclosed in the present application. The method is executed by a terminal and comprises the following steps:
s101: and receiving a visual angle switching instruction in the process of playing the video data corresponding to the first visual angle.
S102: and determining a second view according to the view switching indication.
For convenience of description, the current viewing angle is referred to as a first viewing angle. And the terminal receives the visual angle switching instruction in the process of playing the video data corresponding to the first visual angle, and determines a second visual angle according to the visual angle switching instruction, namely determines a target visual angle.
In one possible implementation, the view switching indication is an indication that includes a view identifier. The user inputs an identification of a viewing angle desired to view through the input unit. For example, the terminal displays a selection control (such as a button) of an alternative view angle, the user selects the selection control of a certain alternative view angle according to requirements, and the terminal determines the alternative view angle selected by the user as the second view angle. For example, the user speaks an identifier of a viewing angle that the user wants to watch, and the terminal performs recognition analysis on the user voice to determine a second viewing angle. Of course, the scheme of the indication of the angle switching and the determination of the second angle according to the indication of the angle switching is not limited thereto.
S103: and pausing the playing of the video data corresponding to the first visual angle, and loading the pre-constructed fusion data set.
The fusion data set comprises a plurality of frames of fusion images, the ith frame of fusion image in the plurality of frames of fusion images is formed by splicing the ith frame of image in the video data corresponding to the N visual angles, and the video data corresponding to the N visual angles keep frame synchronization.
N acquisition devices are arranged in a shooting field, and the value of N is at least 3. In specific implementation, the number of the acquisition devices is different according to different shooting scenes. Typically, the angle between adjacent acquisition devices is between 10 ° and 20 °. The angle between the two acquisition devices can be understood as: the angle between the two acquisition devices and the line connecting the same target point (such as the center of the field to be photographed). In implementation, the N acquisition devices can be arranged along a straight line or an arc line, and the specific arrangement mode of the acquisition devices is not limited by the scheme.
For example, in an event scenario, 18 capture devices may be arranged around the playing field, with an angle of 20 ° between adjacent capture devices, thereby ensuring that the user can view the game in all directions. For example, in a literary performance scene, 10 acquisition devices may be arranged opposite the stage, with an angle of 20 ° between adjacent acquisition devices. Fig. 2 of the present application shows one arrangement of acquisition devices.
And the video data corresponding to the N visual angles keep frame synchronization. In addition, frame synchronization means that images having the same frame number in video data corresponding to N views correspond to the same time, but errors and deviations can be tolerated.
In a possible implementation manner, the video data corresponding to the N views are video data generated by the N acquisition devices, and the N acquisition devices can synchronously perform shooting. For example, the N acquisition devices can perform synchronous shooting through a hardware synchronization line.
In another possible implementation manner, the video data corresponding to the N views are video data obtained by preprocessing the video data generated by the N acquisition devices. It is understood that if the N acquisition devices cannot guarantee synchronous shooting, the aforementioned preprocessing at least includes a frame synchronization process so that the processed video data maintains frame synchronization. Of course, the foregoing preprocessing may also include image processing to improve the picture rendering effect, for example, performing filter processing.
In implementation, a fusion data set is constructed by using video data corresponding to the N visual angles. Specifically, the number of the fusion images included in the fusion data set is denoted as Q, and the value of Q is the same as the number of the image frames included in the video data corresponding to the N views. For example, the length of the video data corresponding to each view is 10 seconds, and the frame rate is 25, then the number of image frames included in the video data corresponding to each view is 250, and Q also takes a value of 250. The fusion image of the ith frame in the fusion data set is formed by splicing the images of the ith frame in the video data corresponding to the N visual angles. That is to say, the ith frame image in the video data corresponding to the N views is spliced to form the ith frame fused image in the fused data set.
It should be noted that the image stitching scheme described above needs to follow a preset stitching rule.
For example, the fused image may be divided into sub-regions of a row and B columns, the product of a and B needs to be greater than or equal to N, and the i-th frame images in the video data corresponding to the N views are sequentially arranged in the a × B sub-regions to form the i-th frame fused image. As shown in fig. 3, the i-th frame of the video data corresponding to the 1 st view to the 4 th view is located on the 1 st line, the i-th frame of the video data corresponding to the 5 th view to the 8 th view is located on the 2 nd line, the i-th frame of the video data corresponding to the 9 th view to the 12 th view is located on the 3 rd line, and the i-th frame of the video data corresponding to the 13 th view to the 16 th view is located on the 4 th line.
For example, the fused image may be divided into sub-regions in rows a and columns B, the product of a and B is greater than or equal to N, and the ith frame image in the video data corresponding to N views is arranged in a serpentine manner in the sub-regions a × B to form the ith frame fused image, as shown in fig. 4.
Of course, the above is only an example of the splicing rule, and does not form a limitation on the splicing rule, as long as it is ensured that the fused image obtained by splicing includes images in the video data corresponding to the N view angles, and the positions of the images in the fused image are known.
S104: and determining M frames of fused images starting from the a +1 th frame in the fused data set as target fused images.
S105: and playing the target pictures in the M frames of target fusion images one by one.
S106: and playing data starting from the a + M +1 th frame in the video data corresponding to the second visual angle.
The target fusion image is an M frame fusion image starting from the (a + 1) th frame in the fusion data set, a is the frame number of the image played by the video data corresponding to the first visual angle at the time of pause playing, the value of M is greater than or equal to the number P of transition visual angles, and the transition visual angle is a visual angle spatially located between the first visual angle and the second visual angle in the N visual angles.
The M frames of target fusion images are sequentially divided into P subsets according to the frame number, each subset comprises at least one frame of target fusion image, and the target pictures in the target fusion images in the jth subset are as follows: in the target fusion image, a corresponding image to the jth transition view, j being 1,2, …, P, the jth transition view is closer to the first view than the j +1 th transition view, and it is understood that the jth transition view is farther from the second view than the j +1 th transition view. That is, the P transitional viewing angles are the 1 st transitional viewing angle, the 2 nd transitional viewing angle, …, and the P transitional viewing angle in order from the first viewing angle to the second viewing angle.
It should be noted that the number M of target fusion images is greater than or equal to the number P of transition views. When M equals P, each subset includes a target fusion image; when M is greater than P, there is at least one subset including at least two target fusion images.
If the first viewing angle is viewing angle D1, the second viewing angle determined according to the switching indication is viewing angle D9, and sequentially viewing angle D2, viewing angle D3, viewing angle D4, viewing angle D5, viewing angle D6, viewing angle D7 and viewing angle D8 from viewing angle D1 to viewing angle D9 (these 7 viewing angles are transitional viewing angles). If a visual angle switching instruction is received when the 3 rd frame image of the video data corresponding to the visual angle D1 is played, the video data corresponding to the visual angle D1 is immediately paused to load a pre-constructed fusion data set, and a target picture in M frames of fusion images from the 4 th frame in the fusion data set is played one by one, where M is an integer greater than or equal to 7, the M frames of fusion images are sequentially divided into 7 subsets according to the frame number, the target picture in the fusion image in the 1 st subset is an image corresponding to the visual angle D2 in the fusion image, the target picture in the fusion image in the 2 nd subset is an image corresponding to the visual angle D3 in the fusion image, and so on, the target picture in the fusion image in the 7 th subset is an image corresponding to the visual angle D8 in the fusion image. It should be noted that the fused image includes images corresponding to N view angles, and in step S105, it is only necessary to render an image corresponding to a specific transition view angle in the leaked target fused image.
If the first viewing angle is D9, the second viewing angle determined according to the viewing angle switching indication is viewing angle D1, and between viewing angle D9 and viewing angle D1 are viewing angle D8, viewing angle D7, viewing angle D6, viewing angle D5, viewing angle D4, viewing angle D3 and viewing angle D2 in sequence (these 7 viewing angles are transition viewing angles). If a visual angle switching instruction is received when the 5 th frame of the video data corresponding to the visual angle D9 is played, immediately pausing the playing of the video data corresponding to the visual angle D9, loading a pre-constructed fusion data set, playing target pictures in M fusion images from the 6 th frame in the fusion data set one by one, wherein M is an integer greater than or equal to 7, sequentially dividing the M fusion images into 7 subsets according to the frame number, wherein the target pictures in the fusion images in the 1 st subset are the images corresponding to the visual angle D8 in the fusion images, the target pictures in the fusion images in the 2 nd subset are the images corresponding to the visual angle D7 in the fusion images, and so on, the target pictures in the fusion images in the 7 th subset are the images corresponding to the visual angle D2 in the fusion images.
In one possible implementation, the value of M is the same as the number of transition views P. The description here continues with the above example: when receiving a view switching instruction when playing the 3 rd frame image of the video data corresponding to the view D1, immediately pausing to play the video data corresponding to the view D1, loading the pre-constructed fusion data set, and playing the target pictures in the fusion images from the 4 th frame to the 10 th frame one by one, that is, the target fusion images are the fusion images from the 4 th frame to the 10 th frame in the fusion data set.
Wherein the target picture in the 4 th frame fused image is an image corresponding to a viewing angle D2 in the 4 th frame fused image (being the 4 th frame in the video data corresponding to the viewing angle D2), the target picture in the 5 th frame fused image is an image corresponding to a viewing angle D3 in the 5 th frame fused image (being the 5 th frame in the video data corresponding to the viewing angle D3), the target picture in the 6 th frame fused image is a picture corresponding to a viewing angle D4 in the 6 th frame fused image (being the 6 th frame in the video data corresponding to the viewing angle D4), the target picture in the 7 th frame fused image is a picture corresponding to a viewing angle D5 in the 7 th frame fused image (being the 7 th frame in the video data corresponding to the viewing angle D5), the target picture in the 8 th frame fused image is a picture corresponding to a viewing angle D6 in the 8 th frame fused image (being the 8 th frame in the video data corresponding to the viewing angle D6), and the target picture in the 9 th frame fused image is a picture corresponding to a viewing angle D7D 7 in the 8 th frame fused image (being the viewing angle D7) Corresponding to the 9 th frame in the video data), the target picture in the 10 th frame fused image is the picture corresponding to the view angle D8 in the 10 th frame fused image (being the 10 th frame in the video data corresponding to the view angle D8). After the target picture in the fused image of the 10 th frame is played, playing the video data corresponding to the view angle D9, specifically playing the data starting from the a + M +1 frame (i.e. the 11 th frame), and completing the view angle switching.
In another possible implementation, the value of M is greater than the number of transition views P. Then at least one subset contains at least two target fusion images. Correspondingly, in the M frames of target fusion images, at least one group of target pictures in the target fusion images adjacent in time sequence is: and images corresponding to the same transition visual angle in the target fusion image. For example, one subset includes the 4 th frame fused image and the 5 th frame fused image, the target picture in the 4 th frame fused image is the picture corresponding to the view angle D2 in the 4 th frame fused image, and the target picture in the 5 th frame fused image is the picture corresponding to the view angle D2 in the 5 th frame fused image.
Optionally, M is an integer multiple of the number P of transition views, for example, M is twice the number P of transition views. The description here continues with the above example: when receiving a view switching instruction when playing the 3 rd frame image of the video data corresponding to the view D1, immediately pausing to play the video data corresponding to the view D1, loading the pre-constructed fusion data set, and playing the target pictures in the 4 th to 17 th frame fusion images in the fusion data set one by one, that is, the target fusion images are the 4 th to 17 th frame fusion images in the fusion data set.
The 14 frames of target fusion images are sequentially divided into 7 subsets each including 2 frames of target fusion images according to the frame number. Specifically, the 1 st subset includes the 4 th frame and the 5 th frame of fused image, the 2 nd subset includes the 6 th frame and the 7 th frame of fused image, the 3 rd subset includes the 8 th frame and the 9 th frame of fused image, and so on, and the 7 th subset includes the 16 th frame and the 17 th frame of fused image.
Wherein the target pictures in the 4 th frame and the 5 th frame fused images are respectively the images corresponding to the view angle D2 in the 4 th frame and the 5 th frame fused images (respectively the 4 th frame and the 5 th frame in the video data corresponding to the view angle D2), the target pictures in the 6 th frame and the 7 th frame fused images are respectively the images corresponding to the view angle D3 in the 6 th frame and the 7 th frame fused images (respectively the 6 th frame and the 7 th frame in the video data corresponding to the view angle D3), the target pictures in the 8 th frame and the 9 th frame fused images are respectively the pictures corresponding to the view angle D4 in the 8 th frame and the 9 th frame fused images (respectively the 8 th frame and the 9 th frame in the video data corresponding to the view angle D4), the target pictures in the 10 th frame and the 11 th frame fused images are respectively the pictures corresponding to the view angle D5 in the 10 th frame and the 11 th frame fused images (respectively the 10 th frame and the 11 th frame in the video data corresponding to the view angle D5), the target pictures in the 12 th frame and the 13 th frame fused images are respectively the pictures corresponding to the view angle D6 in the 12 th frame and the 13 th frame fused images (respectively the 12 th frame and the 13 th frame in the video data corresponding to the view angle D6), the target pictures in the 14 th frame and the 15 th frame fused images are respectively the pictures corresponding to the view angle D7 in the 14 th frame and the 15 th frame fused images (respectively the 14 th frame and the 15 th frame in the video data corresponding to the view angle D7), and the target pictures in the 16 th frame and the 17 th frame fused images are respectively the pictures corresponding to the view angle D8 in the 16 th frame and the 17 th frame fused images (respectively the 16 th frame and the 17 th frame in the video data corresponding to the view angle D8). After the target picture in the fused image of the 17 th frame is played, the video data corresponding to the view angle D9, specifically the data from the 18 th frame in the video data corresponding to the view angle D9, is played, and the view angle switching is completed.
Optionally, playing the target pictures in the M frames of target fusion images one by one may be: and playing the target pictures in the M frames of target fusion images one by one according to a preset time interval. The preset time interval is determined according to the frame rate of the video. For example, if the frame rate is 25, the preset time interval may be set to 40 ms.
It can be seen that, based on the existing view switching scheme, video data of multiple transition views need to be sequentially loaded and played one by one, and for example, switching from view D1 to view D9 requires that video data of view D2, view D3, view D4, view D5, view D6, view D7, view D8, and view D9 need to be loaded and played one by one, so that view switching can be completed. The more viewing angles are spaced between the current viewing angle and the target viewing angle, the more complicated the switching process. Furthermore, before each time video data is loaded, the playing of the previous video data needs to be paused, that is, each time a view switching is performed, a pause occurs, which results in a frequent pause phenomenon in the video playing process.
In the scheme disclosed by the application, the fusion data set is formed by splicing video data corresponding to all visual angles, when the visual angles need to be switched, only the fusion data set which is constructed in advance needs to be switched to play, and then the video data corresponding to the target visual angles needs to be switched to play, so that the frequency of switching and playing the video data is obviously reduced, the visual angle switching process is greatly simplified, and the technical advantage is more obvious along with the increase of the number of the visual angles between the current visual angle and the target visual angle. In addition, because only two times of data source switching are required to be executed, the frequency of the Kanton phenomenon is greatly reduced, and the fluency of video watching can be obviously improved.
It should be noted that the number M of target fusion images should be greater than or equal to the number of transition views P. When the number M of the target fusion images is equal to the number of the transition visual angles P, visual angle switching can be completed as soon as possible on the premise of ensuring the watching experience of a user. Considering that some users have the requirement of smoothly switching the visual angles, the number M of the target fusion images can be set to be larger than the number P of the transition visual angles, so that the visual angle switching can be more smoothly completed in a slightly longer time.
In implementation, the number M of target fusion images may be set by a user. For example, the number of target fusion images is directly set by the user.
In another implementation, the process of determining the number of target fusion images includes: determining a switching speed matched with a user; and determining the number M of the target fusion images according to the switching speed, wherein the number M of the target fusion images is in a negative correlation relation with the switching speed.
Optionally, a corresponding relationship between the switching speed and the number of the target fusion images is preset, and of course, the corresponding relationship needs to follow a rule that the switching speed and the number of the target fusion images are in a negative correlation relationship, and the number of the target fusion images cannot be smaller than the number of the transition viewing angles. After the switching speed matched with the user is determined, the number of the target fusion images is determined based on the corresponding relation.
In the implementation, the user is provided with an option of switching speed, and the number of the target fusion images is set according to the switching speed selected by the user, it can be understood that the larger the switching speed selected by the user is, the smaller the number of the target fusion images is, the smaller the switching speed selected by the user is, the larger the number of the target fusion images is, and the negative correlation relationship is formed between the target fusion images and the target fusion images. For example, three switching speed options are provided to the user, and the number of target fusion images corresponding to the three switching speed options is set.
In the free view angle data processing method disclosed by the application, when receiving the view angle switching instruction in the process of playing the video data corresponding to the first view angle, determining a second visual angle (namely a target visual angle) according to the visual angle switching indication, pausing playing of video data corresponding to the first visual angle, loading a pre-constructed fusion data set, the i frame fused image in the fused data set is formed by splicing the i frame images in the video data corresponding to N visual angles, target pictures in the M frame fused images from the a +1 frame in the fused data set are played one by one, wherein a is a frame number of an image played by video data corresponding to a first view at a pause playing moment, M is greater than or equal to the number P of transition views between the first view and a second view, M target fusion images are sequentially divided into P subsets according to the frame number, and target pictures in a target fusion image in a jth subset are: and playing the image corresponding to the jth transition visual angle in the target fusion image, and then playing the data beginning from the a + M +1 th frame in the video data corresponding to the second visual angle. According to the scheme, the video data corresponding to the multiple visual angles are spliced to form the fusion data set, when the visual angles are required to be switched, only the fusion data set which is constructed in advance is required to be switched to play, and then the video data corresponding to the playing target visual angles are switched, so that the frequency of switching and playing the video data is obviously reduced, the visual angle switching process is greatly simplified, and the technical advantage is more obvious along with the increase of the number of the visual angles between the current visual angle and the target visual angles. In addition, because only two times of data source switching are required to be executed, the frequency of the Kanton phenomenon is greatly reduced, and the fluency of video watching can be obviously improved.
Some schemes for receiving the indication of switching the viewing angle and determining the second viewing angle according to the indication of switching the viewing angle are described above. In practice, other implementations are also possible, as described herein.
Receiving a view switching instruction in a process of playing video data corresponding to a first view, comprising: and detecting the touch operation of the operation body on the touch screen in the process of playing the video data corresponding to the first visual angle. The operation body includes, but is not limited to, a body (e.g., a finger) of the user, and a stylus.
In one possible implementation, determining the second view according to the view switching indication includes: and determining a second visual angle according to the starting point and the end point of the touch operation and the moving direction of the operation body.
N collection system is certain direction distribution at the scene of shooing, and corresponding N visual angle also is certain law. In an optional implementation scheme, a direction pointing from a first viewing angle to a second viewing angle (that is, a direction of the second viewing angle relative to the first viewing angle) is determined according to a moving direction of the operation body, a number of intervals between the first viewing angle and the second viewing angle is determined according to a distance between a start point and an end point of the touch operation, and then the second viewing angle is determined according to the direction pointing from the first viewing angle to the second viewing angle and the number of intervals between the first viewing angle and the second viewing angle. It can be understood that the larger the distance between the starting point and the end point of the touch operation is, the larger the number of intervals between the first viewing angle and the second viewing angle is, the smaller the distance between the starting point and the end point of the touch operation is, and the smaller the number of intervals between the first viewing angle and the second viewing angle is, the positive correlation relationship between the first viewing angle and the second viewing angle is formed.
Taking the arrangement of the capturing devices shown in fig. 2 as an example, if the current viewing angle is the viewing angle D7, the user performs a touch operation of sliding left on the touch screen, the direction from the first viewing angle to the second viewing angle is determined to be from the viewing angle D7 to the left according to the moving direction of the operating body, the number of intervals between the first viewing angle and the second viewing angle is determined to be 4 according to the distance between the starting point and the ending point of the touch operation, and accordingly, the second viewing angle is determined to be D3.
In another possible implementation manner, the determining the second view according to the view switching indication includes: and determining the candidate visual angle closest to the position of the specific touch operation in the plurality of candidate visual angles, and determining the candidate visual angle as a second visual angle.
Taking the arrangement of the capturing devices shown in fig. 2 as an example, if the current viewing angle is the viewing angle D7, the user performs a specific touch operation on the touch screen, and determines an alternative viewing angle closest to the specific touch operation, for example, the closest viewing angle is the viewing angle D13, then the viewing angle D13 is determined as the second viewing angle.
It should be noted that the specific touch operation should be distinguished from the touch operation for controlling the video playing process (such as pause playing, resume playing after pause, fast forward playing, and fast backward playing). For example, the specific touch operation is set as a circling operation.
The following describes a construction process of the fused data set, which specifically includes:
respectively extracting image frames in the video data corresponding to the N visual angles; splicing the image frames with the same frame number into a fused image; and merging the fused images into a fused data set according to the frame number of the fused images.
In implementation, for video data corresponding to N views, an FFmpeg tool is used to extract image frames in each video data, then a PIL library of Python is used to splice the image frames with the same frame number into a fused image, the frame number of the fused image is the same as the frame number of the image frames spliced to form the fused image, and then the FFmpeg tool is used to merge the fused images into a fused data set according to the frame numbers. In practice, the fused data set may be processed as video data, e.g. MP4 data, AVI data, for data distribution.
It should be noted that the converged data set may be constructed by a terminal or a server.
The scheme disclosed in the present application will be described in more detail below with reference to the schematic diagram of the arrangement of the acquisition devices shown in fig. 2.
16 acquisition devices are arranged on a shooting site, namely 16 visual angles are provided in total, the 16 visual angles are sequentially recorded as a visual angle D1, a visual angle D2, visual angles D3, …, a visual angle D14, a visual angle D15 and a visual angle D16 from left to right, and the difference between the adjacent visual angles is a preset angle.
Processing data generated by 16 acquisition devices to obtain video data corresponding to 16 viewing angles, sequentially recording the video data corresponding to the viewing angles D1, D2, D3, …, D14, D15 and D16 as video data S1, S2, S3, S …, S14, S15 and S16, and respectively storing the video data corresponding to different viewing angles into different folders. If the video length is 10 seconds and the frame rate is 25, the video data corresponding to N views contains 250 image frames.
The fused data set is constructed using video data corresponding to 16 views S17. The fused data set S17 contains 250 fused images. Here, the process of constructing the 1 st frame of fused image in the fused data set is similar to the process of obtaining other fused images by stitching.
The 1 st frame of image in the video data corresponding to the 16 views is stitched by using a Python PIL library, and the 1 st frame of fused image in the obtained fused data set can be as shown in fig. 3 or fig. 4. Wherein, S1-P1 is the 1 st frame image in the video data S1, S2-P1 is the 1 st frame image in the video data S2, S3-P1 is the 1 st frame image in the video data S3, S4-P1 is the 1 st frame image in the video data S4, S5-P1 is the 1 st frame image in the video data S5, S6-P6 is the 1 st frame image in the video data S6, S6-P6 is the first frame image in the video data S6, S6 is the S6, S6-P6 is the first frame image in the S6, S15-P1 are the 1 st frame images in the video data S15, and S16-P1 are the 1 st frame images in the video data S16.
After the stitching of all the images is completed, the fused images are merged into a fused data set according to the frame numbers thereof using the FFmpeg tool S17.
If a view switching instruction is received while the 3 rd frame image in the video data S2 of the view D2 is played, the video data S2 is paused, the fused data set S17 is loaded, and if the second view is determined to be D9 according to the view switching instruction, the target pictures in the 4 th to 10 th frame fused images in the fused data set S17 are played at an interval of 40 milliseconds, that is, the target fused images are the 4 th to 10 th frame fused images in the fused data set S17, and the playing time of the target pictures in each frame of the target fused images is also 40 milliseconds.
Among them, the target picture in the 4 th frame fused image (S17-P4) is the image corresponding to the view angle D2 in the 4 th frame fused image (the 4 th frame in the video data S2 corresponding to the view angle D2), the target picture in the 5 th frame fused image (S17-P5) is the image corresponding to the view angle D3 in the 5 th frame fused image (the 5 th frame in the video data S3 corresponding to the view angle D3), the target picture in the 6 th frame fused image (S17-P6) is the picture corresponding to the view angle D4 in the 6 th frame fused image (the 6 th frame in the video data S4 corresponding to the view angle D4), the target picture in the 7 th frame fused image (S17-P7) is the picture corresponding to the view angle D5 in the 7 th frame fused image (the 7 th frame in the video data S5953 corresponding to the view angle D828456), and the target picture in the 7 th frame fused image (S17-P7) is the target picture corresponding to the view angle D8658 in the fused image (S8672) and the view angle D6 8 frames), the target picture in the 9 th frame fused image (S17-P9) is the picture corresponding to the view angle D7 in the 9 th frame fused image (is the 9 th frame in the video data S7 corresponding to the view angle D7), and the target picture in the 10 th frame fused image (S17-P10) is the picture corresponding to the view angle D8 in the 10 th frame fused image (is the 10 th frame in the video data S8 corresponding to the view angle D8). After the target picture in the fused image of the 10 th frame is played, the video data S9 corresponding to the view angle D9, specifically the data from the 11 th frame in the video data S9, is played, and the view angle switching is completed. The handover procedure is shown in fig. 5.
The application discloses a free visual angle data processing method and correspondingly, a free visual angle data processing device. The descriptions of the two in the specification can be mutually referred.
Referring to fig. 6, fig. 6 is a schematic structural diagram of a freeview data processing apparatus disclosed in the present application. The apparatus includes a switching instruction receiving unit 601, a viewing angle determining unit 602, a data loading unit 603, a target fusion image determining unit 604, a first playback control unit 605, and a second playback control unit 606.
Wherein:
a switching indication receiving unit 601, configured to receive a view switching indication during playing of video data corresponding to a first view.
A view determining unit 602, configured to determine a second view according to the view switching indication.
The data loading unit 603 is configured to pause playing of the video data corresponding to the first view, and load a pre-constructed fusion data set. The fusion data set comprises a plurality of frames of fusion images, wherein the fusion image of the ith frame is formed by splicing the images of the ith frame in the video data corresponding to the N visual angles, and the video data corresponding to the N visual angles keep frame synchronization.
A target fusion image determining unit 604 for determining the M frames of fusion images in the fusion data set from the a +1 th frame as target fusion images. Wherein a is a frame number of an image played by the video data corresponding to the first view at the time of pausing the playing, a value of M is greater than or equal to the number P of transition views, and a transition view is a view spatially located between the first view and the second view in the N views.
A first play control unit 605, configured to play the target pictures in the M frames of target fusion images one by one. The M frames of target fusion images are sequentially divided into P subsets according to the frame number, each subset comprises at least one frame of target fusion image, and the target pictures in the target fusion image in the jth subset are as follows: and j is 1,2, …, and P is the image corresponding to the j-th transition visual angle in the target fusion image, wherein the j-th transition visual angle is closer to the first visual angle than the j + 1-th transition visual angle.
The second playing control unit 606 is configured to play data starting from the a + M +1 th frame in the video data corresponding to the second view after the first playing control unit 605 plays the target picture in the M-th frame target fusion image.
The application discloses free visual angle data processing apparatus utilizes the video data concatenation that a plurality of visual angles correspond to constitute and fuses the data set, when needs carry out the visual angle and switch over, only need switch over to the amalgamation data set that the broadcast was constructed in advance, switch over again to the video data that the broadcast target visual angle corresponds can, show the number of times that has reduced the broadcast video data of switching over, greatly simplified visual angle switching process, and this technical advantage is showing more along with the increase of visual angle quantity between current visual angle and the target visual angle. In addition, because only two times of data source switching are required to be executed, the frequency of the Kanton phenomenon is greatly reduced, and the fluency of video watching can be obviously improved.
Optionally, the switching instruction receiving unit 601 receives the view switching instruction in the process of playing the video data corresponding to the first view, specifically: and detecting the touch operation of the operation body on the touch screen in the process of playing the video data corresponding to the first visual angle.
Optionally, the view determining unit determines the second view according to the view switching instruction, specifically: and determining a second visual angle according to the starting point and the end point of the touch operation and the moving direction of the operation body.
In an optional implementation scheme, a direction pointing from a first viewing angle to a second viewing angle (that is, a direction of the second viewing angle relative to the first viewing angle) is determined according to a moving direction of the operation body, a number of intervals between the first viewing angle and the second viewing angle is determined according to a distance between a start point and an end point of the touch operation, and then the second viewing angle is determined according to the direction pointing from the first viewing angle to the second viewing angle and the number of intervals between the first viewing angle and the second viewing angle. It can be understood that the larger the distance between the starting point and the end point of the touch operation is, the larger the number of intervals between the first viewing angle and the second viewing angle is, the smaller the distance between the starting point and the end point of the touch operation is, and the smaller the number of intervals between the first viewing angle and the second viewing angle is, the negative correlation relationship is formed between the first viewing angle and the second viewing angle.
Optionally, the view determining unit determines the second view according to the view switching instruction, specifically: and determining the candidate visual angle closest to the position of the specific touch operation in the plurality of candidate visual angles, and determining the candidate visual angle as a second visual angle.
Optionally, the target fusion image determining unit 604 is specifically configured to:
determining a switching speed matched with a user; determining the number M of target fusion images according to the switching speed, wherein the number M of the target fusion images and the switching speed are in a negative correlation relationship; and determining the M frames of fused images from the a +1 th frame in the fused data set as target fused images.
Optionally, on the basis of the free perspective data processing apparatus, further setting:
and the preprocessing unit is used for respectively extracting the image frames in the video data corresponding to the N visual angles, splicing the image frames with the same frame number into a fused image, and merging the fused image into a fused data set according to the frame number of the fused image.
The application also discloses an electronic device.
Referring to fig. 7, fig. 7 is a hardware structure diagram of an electronic device disclosed in the present application. The electronic device comprises a processor 701 and a memory 702.
Optionally, the terminal may further include: a communication interface 703, an input unit 704, a display 705, and a communication bus 706. The processor 701, the memory 702, the communication interface 703, the input unit 704, and the display 705 all communicate with each other via the communication bus 706.
In the claimed embodiment, the processor 701 may be a Central Processing Unit (CPU), an application specific integrated circuit, a digital signal processor, an off-the-shelf programmable gate array or other programmable logic device, etc.
The processor 701 may call a program stored in the memory 702.
The memory 702 is used to store one or more programs, which may include program code comprising computer operating instructions. In an embodiment of the present application, the memory stores at least a program code for implementing any one of the above-mentioned free perspective data processing methods.
In one possible implementation, the memory 702 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, the above-mentioned programs, and the like; the storage data area may store data created during use of the computer device, and the like.
In addition, the memory 702 may include high speed random access memory, and may also include non-volatile memory.
The communication interface 703 may be an interface of a communication module.
The input unit 704 may include a touch sensing unit sensing a touch event on the touch display panel, a keyboard, and the like.
The display 705 includes a display panel, such as a touch display panel or the like.
Of course, the structure of the electronic device shown in fig. 7 does not limit the electronic device in the embodiment of the present application, and in practical applications, the electronic device may include more or less components than those shown in fig. 7, or some components may be combined.
The application also discloses a storage medium executable by the electronic device, wherein the storage medium stores a program, and the program is loaded and executed by a processor of the electronic device, so that the electronic device realizes any one of the free perspective data processing methods disclosed in the application.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device, the electronic device and the storage medium disclosed by the embodiment correspond to the method disclosed by the embodiment, so that the description is relatively simple, and the relevant points can be referred to the method part for description.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A freeview data processing method, comprising:
receiving a visual angle switching instruction in the process of playing video data corresponding to a first visual angle;
determining a second view according to the view switching indication;
pausing playing of the video data corresponding to the first visual angle, and loading a pre-constructed fusion data set, wherein the fusion data set comprises a plurality of frames of fusion images, the i-th frame of fusion image is formed by splicing the i-th frame of images in the video data corresponding to N visual angles, the video data corresponding to N visual angles keep frame synchronization, and N is greater than or equal to 3;
determining M frames of fused images starting from the a +1 th frame in the fused data set as target fused images, wherein a is a frame number of images played by video data corresponding to the first view at a pause playing moment, the value of M is greater than or equal to the number P of transition views, and the transition views are views spatially located between the first view and the second view in the N views;
target pictures in the M frames of target fusion images are played one by one, wherein the M frames of target fusion images are sequentially divided into P subsets according to the frame number, each subset comprises at least one frame of target fusion image, and the target pictures in the target fusion image in the jth subset are as follows: a j ═ 1,2, …, P of the image corresponding to the jth transition view in the target fusion image, the jth transition view being closer to the first view than the j +1 th transition view;
and playing data starting from the a + M +1 th frame in the video data corresponding to the second visual angle.
2. The method according to claim 1, wherein receiving the view switching indication during the playing of the video data corresponding to the first view comprises:
and detecting the touch operation of an operation body on the touch screen in the process of playing the video data corresponding to the first visual angle.
3. The method of claim 2, wherein the determining a second view according to the view switching indication comprises:
and determining the second visual angle according to the starting point and the end point of the touch operation and the moving direction of the operation body.
4. The method of claim 1, wherein determining the number of target fusion images comprises:
determining a switching speed matched with a user;
and determining the number M of the target fusion images according to the switching speed, wherein the number M of the target fusion images and the switching speed are in a negative correlation relationship.
5. The method of claim 1, wherein the fused data set building process comprises:
respectively extracting image frames in the video data corresponding to the N visual angles;
splicing the image frames with the same frame number into a fused image;
and merging the fused images into a fused data set according to the frame number of the fused images.
6. A freeview data processing apparatus, comprising:
a switching indication receiving unit, configured to receive a view switching indication during playing of video data corresponding to a first view;
a visual angle determining unit, configured to determine a second visual angle according to the visual angle switching instruction;
the data loading unit is used for pausing playing of the video data corresponding to the first visual angle and loading a pre-constructed fusion data set, wherein the fusion data set comprises a plurality of frames of fusion images, the i-th frame of fusion image is formed by splicing the i-th frame of images in the video data corresponding to N visual angles, the video data corresponding to the N visual angles keep frame synchronization, and N is greater than or equal to 3;
a target fused image determining unit, configured to determine that M frames of fused images starting from an a +1 th frame in the fused data set are target fused images, where a is a frame number of an image played by video data corresponding to the first view at a pause playing time, a value of M is greater than or equal to a number P of transition views, and the transition views are views spatially located between the first view and the second view in the N views;
the first playing control unit is used for playing the target pictures in the M frames of target fusion images one by one, wherein the M frames of target fusion images are sequentially divided into P subsets according to the frame number, each subset comprises at least one frame of target fusion image, and the target pictures in the target fusion image in the jth subset are: a j ═ 1,2, …, P of the image corresponding to the jth transition view in the target fusion image, the jth transition view being closer to the first view than the j +1 th transition view;
and the second playing control unit is used for playing the data starting from the a + M +1 th frame in the video data corresponding to the second visual angle after the first playing control unit plays the target picture in the Mth frame target fusion image.
7. The apparatus according to claim 6, wherein the target fusion image determination unit is specifically configured to:
determining a switching speed matched with a user; determining the number M of the target fusion images according to the switching speed, wherein the number M of the target fusion images and the switching speed are in a negative correlation relationship; and determining M frames of fused images starting from the a +1 th frame in the fused data set as target fused images.
8. The apparatus of claim 6, further comprising:
and the preprocessing unit is used for respectively extracting the image frames in the video data corresponding to the N visual angles, splicing the image frames with the same frame number into a fused image, and merging the fused image into a fused data set according to the frame number of the fused image.
9. An electronic device comprising a processor and a memory;
the memory is used for storing programs;
the processor is configured to call and execute a program stored in the memory to perform the freeview data processing method according to any one of claims 1 to 5.
10. A storage medium, characterized in that the storage medium stores therein a program which, when loaded and executed by a processor in an electronic device, implements the freeview data processing method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110514671.XA CN113256491A (en) | 2021-05-11 | 2021-05-11 | Free visual angle data processing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110514671.XA CN113256491A (en) | 2021-05-11 | 2021-05-11 | Free visual angle data processing method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113256491A true CN113256491A (en) | 2021-08-13 |
Family
ID=77223082
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110514671.XA Pending CN113256491A (en) | 2021-05-11 | 2021-05-11 | Free visual angle data processing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113256491A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113938711A (en) * | 2021-10-13 | 2022-01-14 | 北京奇艺世纪科技有限公司 | Visual angle switching method and device, user side, server and storage medium |
| CN114245190A (en) * | 2021-12-17 | 2022-03-25 | 天翼视讯传媒有限公司 | Double-video-stream free-viewpoint video playing system |
| CN115174942A (en) * | 2022-07-08 | 2022-10-11 | 叠境数字科技（上海）有限公司 | Free visual angle switching method and interactive free visual angle playing system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106612426A (en) * | 2015-10-26 | 2017-05-03 | 华为技术有限公司 | Method and device for transmitting multi-view video |
| CN111447462A (en) * | 2020-05-20 | 2020-07-24 | 上海科技大学 | Video live broadcast method, system, storage medium and terminal based on visual angle switching |
| CN111669569A (en) * | 2019-03-07 | 2020-09-15 | 阿里巴巴集团控股有限公司 | Video generation method and device, medium and terminal |
| CN111669518A (en) * | 2019-03-07 | 2020-09-15 | 阿里巴巴集团控股有限公司 | Multi-angle free visual angle interaction method and device, medium, terminal and equipment |
| CN111669568A (en) * | 2019-03-07 | 2020-09-15 | 阿里巴巴集团控股有限公司 | Multi-angle free visual angle interaction method and device, medium, terminal and equipment |
| CN111866525A (en) * | 2020-09-23 | 2020-10-30 | 腾讯科技（深圳）有限公司 | Multi-view video playing control method and device, electronic equipment and storage medium |
| CN112738534A (en) * | 2019-10-28 | 2021-04-30 | 阿里巴巴集团控股有限公司 | Data processing method and system, server and storage medium |
-
2021
- 2021-05-11 CN CN202110514671.XA patent/CN113256491A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106612426A (en) * | 2015-10-26 | 2017-05-03 | 华为技术有限公司 | Method and device for transmitting multi-view video |
| CN111669569A (en) * | 2019-03-07 | 2020-09-15 | 阿里巴巴集团控股有限公司 | Video generation method and device, medium and terminal |
| CN111669518A (en) * | 2019-03-07 | 2020-09-15 | 阿里巴巴集团控股有限公司 | Multi-angle free visual angle interaction method and device, medium, terminal and equipment |
| CN111669568A (en) * | 2019-03-07 | 2020-09-15 | 阿里巴巴集团控股有限公司 | Multi-angle free visual angle interaction method and device, medium, terminal and equipment |
| CN112738534A (en) * | 2019-10-28 | 2021-04-30 | 阿里巴巴集团控股有限公司 | Data processing method and system, server and storage medium |
| CN111447462A (en) * | 2020-05-20 | 2020-07-24 | 上海科技大学 | Video live broadcast method, system, storage medium and terminal based on visual angle switching |
| CN111866525A (en) * | 2020-09-23 | 2020-10-30 | 腾讯科技（深圳）有限公司 | Multi-view video playing control method and device, electronic equipment and storage medium |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113938711A (en) * | 2021-10-13 | 2022-01-14 | 北京奇艺世纪科技有限公司 | Visual angle switching method and device, user side, server and storage medium |
| CN114245190A (en) * | 2021-12-17 | 2022-03-25 | 天翼视讯传媒有限公司 | Double-video-stream free-viewpoint video playing system |
| CN115174942A (en) * | 2022-07-08 | 2022-10-11 | 叠境数字科技（上海）有限公司 | Free visual angle switching method and interactive free visual angle playing system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113256491A (en) | Free visual angle data processing method, device, equipment and storage medium | |
| CN108616696B (en) | Video shooting method and device, terminal equipment and storage medium | |
| AU2019216671B2 (en) | Method and apparatus for playing video content from any location and any time | |
| US10812868B2 (en) | Video content switching and synchronization system and method for switching between multiple video formats | |
| CN108900902B (en) | Method, device, terminal equipment and storage medium for determining video background music | |
| CN108900771B (en) | Video processing method and device, terminal equipment and storage medium | |
| CN108965705B (en) | Video processing method and device, terminal equipment and storage medium | |
| CN105847636B (en) | A kind of video capture method and mobile terminal | |
| CN112887609B (en) | Shooting method and device, electronic equipment and storage medium | |
| CN106792147A (en) | A kind of image replacement method and device | |
| US9392190B2 (en) | Method and device for acquiring image | |
| EP2866431A1 (en) | Image outputting device | |
| US20200304887A1 (en) | Display overlays for prioritization of video subjects | |
| CN110225405B (en) | Panoramic video playing method and device | |
| CN108616769A (en) | A kind of method and apparatus of video on demand | |
| EP3070942B1 (en) | Method and apparatus for displaying light field video data | |
| CN106385613A (en) | Method and device for controlling playing of bullet screens | |
| US20230308752A1 (en) | Video recording method and electronic device | |
| CN112367487B (en) | Video recording method and electronic equipment | |
| CN116156077A (en) | Method, device, equipment and storage medium for multimedia resource clipping scene | |
| CN113411659A (en) | Video playing system, method, device and medium for video application | |
| CN114025237A (en) | Video generation method and device and electronic equipment | |
| CN114745597B (en) | Video processing method and apparatus, electronic device, and computer-readable storage medium | |
| CN112672059B (en) | Shooting method and shooting device | |
| CN114979746B (en) | Video processing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |