CN114222092A

CN114222092A - Multi-view video transmission method, device, processing equipment, display equipment and medium

Info

Publication number: CN114222092A
Application number: CN202111531568.2A
Authority: CN
Inventors: 张萍; 陈云海; 刘晓玲
Original assignee: China Telecom Corp Ltd
Current assignee: China Telecom Corp Ltd
Priority date: 2021-12-14
Filing date: 2021-12-14
Publication date: 2022-03-22

Abstract

The disclosure provides a multi-view video transmission method, a multi-view video transmission device, a processing device, a display device and a medium, and relates to the technical field of network transmission. The multi-view video transmission method comprises the following steps: acquiring a plurality of groups of video images acquired by a camera group arranged in a multi-angle array, wherein each group of video images corresponds to a camera shooting angle; selecting a plurality of consecutive views in the multi-angle array based on a number of views of a sliding window; splicing the video images of the multiple continuous visual angles based on downsampling operation to generate a multi-grid video image; and carrying out compression coding processing on the multi-grid video image so as to send the processed multi-grid video image to a playing device. Through the technical scheme of this disclosure, through the visual angle quantity in the reasonable setting sliding window, when promoting the video image's of single visual angle of down-sampling quality, guaranteed that the video of the whole angles of the multiunit of gathering all can be selected and transmitted.

Description

Multi-view video transmission method, device, processing equipment, display equipment and medium

Technical Field

The present disclosure relates to the field of network transmission technologies, and in particular, to a multi-view video transmission method, apparatus, processing device, display device, and storage medium.

Background

In order to improve the immersive experience effect of the user, a multi-view technology is developed, the technology supports the user to freely change the viewing angle, the problem that the viewing angle of the traditional video is limited and cannot be interacted is solved, and the technology is applied to head sports and comprehensive art.

In the related art, the implementation process of multi-view video transmission mainly includes: the video images of all visual angles at the same frame time are collected and spliced into a large multi-visual-angle video image, and then the large multi-visual-angle video image is compressed and encoded and then transmitted. When a multi-view video is spliced by 4K, 9 views are arranged, and a video image of a single view needs to be down-sampled to 720P (4K/9), or 16 views need to be down-sampled to 540P (4K/16) as shown in fig. 1.

The above scheme has the following defects: on one hand, due to the limitation of bandwidth and decoding capability of equipment, the splicing resolution has certain upper limit requirements, the more the arranged views are, the fewer pixels which can be distributed by a single view in the spliced image are, and the lower the resolution of the single view is; on the other hand, if the video quality of a single view is to be guaranteed, only the number of views, the range of views or the difference of views can be limited, and this way affects the display effect of the single view video image.

It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present disclosure, and thus may include information that does not constitute prior art known to those of ordinary skill in the art.

Disclosure of Invention

An object of the present disclosure is to provide a multi-view video transmission method, apparatus, processing device, display device, and storage medium, which overcome, at least to some extent, the problem that the number of views and the quality of a single-view video image cannot be considered in the related art.

Additional features and advantages of the disclosure will be set forth in the detailed description which follows, or in part will be obvious from the description, or may be learned by practice of the disclosure.

According to an aspect of the present disclosure, there is provided a multi-view video transmission method applied to a processing device, including: acquiring a plurality of groups of video images acquired by a camera group arranged in a multi-angle array, wherein each group of video images corresponds to a camera shooting angle; selecting a plurality of consecutive views in the multi-angle array based on a number of views of a sliding window, the sliding window configured to slide in an array region corresponding to the multi-angle array, the number of views of the sliding window being less than a total number of views of the multi-angle array; splicing the video images of the multiple continuous visual angles based on downsampling operation to generate a multi-grid video image; and carrying out compression coding processing on the multi-grid video image so as to send the processed multi-grid video image to a playing device.

In one embodiment, the number of views of the sliding window is less than or equal to a window threshold, and the window threshold is configured based on at least one of a playback resolution of a playback device, a transmission resolution of the processing device, and a stitching resolution of the multi-grid video image.

In an embodiment, the selecting the plurality of consecutive views in the multi-angle array based on the number of views of the sliding window includes: and selecting an adjacent view of the reference view from the multi-angle array based on the number of views of the sliding window and a selection rule to obtain the plurality of continuous views.

In one embodiment, before selecting an adjacent view of the reference view in the multi-angle array based on the number of views of the sliding window and a selection rule, the method further comprises: receiving a visual angle setting instruction sent by a playing device, and determining the corresponding reference visual angle based on the visual angle setting instruction; or determining one of the median of the number of the visual angles of the sliding window as the serial number of the reference visual angle.

In an embodiment, the selecting, based on the number of views of the sliding window and a selection rule, an adjacent view of the reference view from the multi-angle array specifically includes: when the selection rule is suitable for selecting two side visual angles adjacent to the reference visual angle, determining a first parameter and a second parameter based on the visual angle number of the sliding window; determining the contiguous views based on a relationship between the corresponding sequence numbers of the reference views in the multi-angle and the first parameter and/or the second parameter.

In an embodiment, the determining the first parameter and the second parameter based on the number of viewing angles of the sliding window specifically includes: calculating the first parameter based on a first calculation formula; calculating the second parameter based on a second calculation formula, wherein the first calculation formula is used for taking the value less than or equal to

The second calculation expression is used for taking the maximum integer greater than or equal to

S is the number of views of the sliding window.

In an embodiment, the determining the adjacent view based on a relationship between the corresponding number of the reference view in the multiple angles and the first parameter and/or the second parameter specifically includes: when the difference value between the corresponding sequence number and the first parameter is less than 1, configuring the sequence number of the adjacent view angle based on the number of the view angles of the sliding window; when the sum of the corresponding sequence number and the second parameter is greater than the total number of the visual angles of the multi-angle array, configuring the sequence number of the adjacent visual angle in a first numerical range, wherein the first numerical range is greater than the difference value between the total number of the visual angles and the number of the visual angles and is less than or equal to the total number of the visual angles; and when the corresponding sequence number is in other numerical value ranges, configuring the sequence number of the adjacent view angle in a second numerical value range, wherein the second numerical value range is greater than or equal to the difference value between the corresponding sequence number and the first parameter and is less than or equal to the sum of the corresponding sequence number and the second parameter.

In an embodiment, the selecting, based on the number of views of the sliding window and a selection rule, an adjacent view of the reference view from the multi-angle array specifically includes: and when the selection rule is suitable for selecting a side view adjacent to the reference view, determining the adjacent view based on the preset position of the adjacent view relative to the reference view and the number of views.

In an embodiment, the determining the adjacent viewing angles based on the preset positions of the adjacent viewing angles relative to the reference viewing angle and the number of viewing angles specifically includes: when the adjacent visual angle is preset on the left side relative to the reference visual angle, detecting whether the left visual angle quantity of the reference visual angle meets the visual angle quantity of the sliding window; if the number of the visual angles of the sliding window is met, selecting a plurality of continuous visual angles from the left visual angle as the adjacent visual angle; if the number of the visual angles of the sliding window is not satisfied, determining the number of the remaining visual angles based on the number of the visual angles of the sliding window and the number of the left visual angles, so as to select all the right visual angles adjacent to the left visual angles and the number of the remaining visual angles.

In an embodiment, the determining the adjacent viewing angles based on the preset positions of the adjacent viewing angles relative to the reference viewing angle and the number of viewing angles specifically includes: when the adjacent visual angle is preset on the right side relative to the reference visual angle, detecting whether the right visual angle quantity of the reference visual angle meets the visual angle quantity of the sliding window; if the number of the visual angles of the sliding window is met, selecting a plurality of continuous visual angles from the right visual angle as the adjacent visual angle; if the number of the visual angles of the sliding window is not satisfied, determining the number of the remaining visual angles based on the number of the visual angles of the sliding window and the number of the right visual angles, so as to select all the right visual angles and the left visual angles adjacent to the number of the remaining visual angles.

In one embodiment, further comprising: when an adjusting instruction for changing the reference visual angle is received, the adjusted reference visual angle is determined based on the adjusting instruction, and the visual angle of the video image in the sliding window is reconfigured.

According to another aspect of the present disclosure, there is provided a multi-view video transmission method applied to a display device, including: receiving a multi-grid video image sent by a processing device; and determining a playing visual angle in the multi-grid video image based on visual angle selection operation of a user so as to play based on the playing visual angle.

In one embodiment, before receiving the multi-grid video image sent by the processing device, the method further includes: receiving a visual angle selection operation of a user; generating a corresponding visual angle setting instruction based on the visual angle selection operation; and sending the visual angle setting instruction to the processing equipment so as to directly determine the playing visual angle based on the visual angle setting instruction when the multi-grid video image is received.

In an embodiment, the determining a playing view angle in the multi-grid video image based on a view angle selection operation of a user specifically includes: displaying the received multi-grid video image; receiving the visual angle selection operation of the multi-grid video image; determining the playing view based on the view selection operation.

In one embodiment, further comprising: generating a visual angle adjusting area, wherein the visual angle adjusting area comprises a multi-angle array and a sliding window sliding on the multi-angle array; receiving adjustment operation of a user on the sliding window to generate a corresponding adjustment instruction based on the adjustment operation; sending the adjustment instruction to the processing device to adjust, by the processing device, a reference perspective based on the adjustment instruction.

In an embodiment, the playing based on the playing perspective specifically includes: extracting a video image to be played corresponding to the playing visual angle from the multi-grid video images; and performing up-sampling operation on the video image to be played based on the playing resolution ratio to generate a playing video, and displaying the playing video.

According to still another aspect of the present disclosure, there is provided a multi-view video transmission apparatus including: the system comprises an acquisition module, a display module and a control module, wherein the acquisition module is used for acquiring a plurality of groups of video images acquired by a camera group arranged in a multi-angle array, and each group of video images corresponds to a camera shooting angle; a selection module, configured to select a plurality of consecutive views in the multi-angle array based on a number of views of a sliding window, where the sliding window is configured to slide in an array region corresponding to the multi-angle array, and the number of views of the sliding window is less than a total number of views of the multi-angle array; the splicing module is used for splicing the video images of the multiple continuous visual angles based on downsampling operation to generate a multi-grid video image; and the sending module is used for carrying out compression coding processing on the multi-grid video image so as to send the processed multi-grid video image to a playing device.

According to still another aspect of the present disclosure, there is provided a multi-view video transmission apparatus including: the receiving module is used for receiving the multi-grid video image sent by the processing equipment; the determining module is used for determining a playing visual angle in the multi-grid video image based on visual angle selection operation of a user; the extraction module is used for extracting a video image to be played corresponding to the playing visual angle from the multi-grid video images; and the display module is used for performing up-sampling operation on the video image to be played based on the playing resolution ratio to generate a playing video and displaying the playing video.

According to yet another aspect of the present disclosure, there is provided a processing apparatus comprising: a processor; and a memory for storing executable instructions for the processor; the processor is configured to perform the multi-view video transmission method of the first aspect described above via execution of the executable instructions.

According to still another aspect of the present disclosure, there is provided a display apparatus including: a processor; and a memory for storing executable instructions for the processor; the processor is configured to perform the multi-view video transmission method of the second aspect described above via execution of the executable instructions.

According to yet another aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the multi-view video transmission method described above.

The multi-view video transmission method and the transmission device provided by the embodiment of the disclosure select a plurality of continuous views in a multi-angle array based on the number of views set by a sliding window with a certain number of views when acquiring a plurality of groups of video images synchronously acquired by a camera group arranged in the multi-angle array, so as to splice the video images of the plurality of continuous views to obtain a multi-grid video image, and transmit the multi-grid video image to a playing device after further compression coding processing, thereby realizing that at least a part of all the groups of video images are spliced, and the sliding window is set, on one hand, the quality of the video image of a single down-sampling view can be ensured by reasonably setting the number of views in the sliding window on the premise that the resolution of the multi-grid video image is limited, and on the other hand, by sliding the sliding window in an array area corresponding to the multi-angle array, the multi-angle video with the appointed view angle number can be selected at will in the multi-group video images, so that the number of the angles arranged in the multi-angle array is not too small, and the number of the shot view angles and the view angle range can be considered.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure. It is to be understood that the drawings in the following description are merely exemplary of the disclosure, and that other drawings may be derived from those drawings by one of ordinary skill in the art without the exercise of inventive faculty.

Fig. 1 is a flow chart illustrating a multi-view video transmission method in the related art;

fig. 2 shows a flow chart of a multi-view video transmission method in an embodiment of the present disclosure;

fig. 3 shows a flow diagram of another multi-view video transmission scheme in an embodiment of the disclosure;

fig. 4 is a flow chart illustrating a multi-view video transmission method according to another embodiment of the disclosure;

fig. 5 is a flow chart illustrating a multi-view video transmission method according to another embodiment of the disclosure;

fig. 6 is a flow chart of a multi-view video transmission method according to another embodiment of the disclosure;

fig. 7 is a flow chart of a multi-view video transmission method according to another embodiment of the disclosure;

fig. 8 is a flow chart of a multi-view video transmission method according to another embodiment of the disclosure;

fig. 9 is a flow chart of a multi-view video transmission method according to another embodiment of the disclosure;

fig. 10 is a flow chart of a multi-view video transmission method according to another embodiment of the disclosure;

fig. 11 is a flow chart illustrating a multi-view video transmission method according to another embodiment of the disclosure;

fig. 12 is a flow chart of a multi-view video transmission method according to another embodiment of the disclosure;

fig. 13 is a flow chart of a multi-view video transmission method according to another embodiment of the disclosure;

fig. 14 is a schematic diagram of a multi-view video transmission apparatus according to an embodiment of the disclosure;

fig. 15 is a schematic diagram of another multi-view video transmission apparatus in an embodiment of the disclosure;

FIG. 16 is a block diagram of a computer device according to an embodiment of the present disclosure; and

fig. 17 shows a block diagram of a program product in an embodiment of the disclosure.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.

According to the scheme provided by the application, by arranging the sliding window with a certain visual angle number, when a plurality of groups of video images synchronously acquired by a camera group arranged in a multi-angle array are acquired, a plurality of continuous visual angles are selected in the multi-angle array based on the visual angle number set by the sliding window to splice the video images of the plurality of continuous visual angles to obtain a multi-lattice video image, the multi-lattice video image is further subjected to compression coding processing and then is sent to playing equipment, so that at least one part of all the groups of video images is selected to be spliced, and the sliding window is arranged, on one hand, the quality of the video image of a single down-sampling visual angle can be ensured on the premise that the resolution ratio of the multi-lattice video image is limited by reasonably setting the visual angle number in the sliding window, on the other hand, the sliding window is arranged in an array area corresponding to the multi-angle array, the multi-angle video with the appointed view angle number can be selected at will in the multi-group video images, so that the number of the angles arranged in the multi-angle array is not too small, and the number of the shot view angles and the view angle range can be considered.

For ease of understanding, the following first explains nouns (abbreviations) referred to in the present application.

Down-sampling, i.e. reducing the image, for an image I with size M × N, s-times down-sampling is performed to obtain a resolution image with size (M/s) × (N/s), where s is a common divisor of M and N, and if an image in matrix form is considered, the image in the window of s × s of the original image is changed into one pixel.

Upsampling, i.e. amplifying an image, almost all image amplification adopts an interpolation method, i.e. a proper interpolation algorithm is adopted to insert new elements between pixel points on the basis of original image pixels.

In order to solve the above problems, the present disclosure proposes a method of achieving super-frequency fusion based on flexible spectrum fusion.

Hereinafter, the steps of the multiview video transmission method in the present exemplary embodiment will be described in more detail with reference to the accompanying drawings and examples.

Fig. 2 shows a flowchart of a multi-view video transmission method in an embodiment of the present disclosure.

As shown in fig. 2, the multi-view video transmission method according to one embodiment of the present disclosure, applied to a processing device, includes the steps of:

step S202, a plurality of groups of video images collected by a camera group arranged in a multi-angle array are obtained, wherein each group of video images corresponds to a camera shooting angle.

The synchronous acquisition of the video images shot by the camera group is realized by acquiring a plurality of groups of video images acquired by the camera group.

Step S204, selecting a plurality of continuous visual angles in the multi-angle array based on the visual angle number of the sliding window, wherein the sliding window is configured to slide in an array area corresponding to the multi-angle array, and the visual angle number of the sliding window is smaller than the total visual angle number of the multi-angle array.

The reference view may be a view determined based on the received view setting instruction, or a view at an intermediate position in the sliding window may be used as the reference view.

In addition, the sliding window is configured to slide in an array region corresponding to the multi-angle array, that is, a plurality of views within the sliding view are a plurality of consecutive views in the multi-angle array.

Those skilled in the art can also understand that the number of viewing angles of the sliding window can be flexibly set.

In one embodiment, the selecting the plurality of consecutive views in the multi-angle array based on the number of views of the sliding window comprises: based on the number of views of the sliding window and a selection rule, selecting an adjacent view of a reference view in the multi-angle array to obtain a plurality of continuous views.

The position of the reference viewing angle may be at the leftmost end, the rightmost end or at the middle position of the sliding window.

In this embodiment, by setting the reference view angle, the reference view angle may correspond to a selected view angle of a user on the video display device, and the reference view angle may also be a view angle initially set on the processing device, and a plurality of continuous view angles within the sliding window including the reference view angle are determined to realize the selection of the continuous view angle.

Specifically, in one embodiment, before selecting an adjacent view of a reference view in the multi-angle array based on the number of views of the sliding window and a selection rule, the method further includes: receiving a visual angle setting instruction sent by the playing equipment, and determining a corresponding reference visual angle based on the visual angle setting instruction; or determining one of the median of the number of views of the sliding window as the serial number of the reference view.

The view setting instruction sent by the playing device can be directly generated for the selected video image view through the user.

As shown in fig. 3, a camera array is determined based on information of a camera group arranged in a multi-angle array, and a video image of a target scene or an object is acquired, and a sliding buffer window is configured as a sliding window corresponding to the camera array, wherein the number of visual angles of the sliding window is S, the number of angles of the camera array is N, S is less than N, and preferably, S is less than or equal to 9. And determining the visual angle in the sliding window by sliding the sliding window back and forth in the display area corresponding to the camera array.

And step S206, splicing the video images of the multiple continuous visual angles based on the downsampling operation to generate a multi-grid video image.

The method comprises the steps of conducting down-sampling processing on a video image, conducting down-sampling operation on a collected video of an original pixel to reduce the original video image to 1/S, wherein S is the number of visual angles of a sliding window, and further conducting splicing to obtain a multi-grid video image.

And step S208, performing compression coding processing on the multi-grid video image so as to send the processed multi-grid video image to the playing device.

The multi-grid video image is compressed and coded, so that reliable transmission of the multi-grid video image is guaranteed.

In the present disclosure, compression encoding is specifically a reversible compression encoding process.

In the embodiment, by setting a sliding window with a certain number of visual angles, when a plurality of groups of video images synchronously acquired by a camera group arranged in a multi-angle array are acquired, a plurality of continuous visual angles are selected in the multi-angle array based on the number of visual angles set by the sliding window to splice the video images of the plurality of continuous visual angles to obtain a multi-grid video image, and the multi-grid video image is further sent to a playing device after compression coding processing, so that at least one part of all the groups of video images is selected to be spliced, and the sliding window is set, on one hand, by reasonably setting the number of visual angles in the sliding window, on the premise that the resolution of the multi-grid video image is limited, the quality of the video image of a single downsampling visual angle can be improved, on the other hand, by sliding the sliding window in an array area corresponding to the multi-angle array, can be in the multi-angle video of appointed visual angle quantity of arbitrary selection in multiunit video image, the video image of all angles all can be selected, and splice the transmission, thereby need not set up the angle quantity that multi-angle array arranged too little, thereby can compromise visual angle quantity and the visual angle scope of shooting, on the one hand again, originally gather multiunit video image data volume big, if whole synchronous transmission to display device, the handling capacity is big, calculation and transmission cost are high, adopt the sliding window that is less than original quantity in this disclosure, carry out the image concatenation, can deploy at 5G edge calculation node, be favorable to promoting the real-time of transmission.

Specifically, as an optimal implementation manner, the reference view angle is determined based on the view angle selected by the user, the stitching resolution of the multi-grid video image is 4K, the video data to be transmitted is constructed into an adaptive sliding window, the data of a plurality of continuous view angles adjacent to the current view angle is adaptively selected according to the current view angle of the user and stored in the sliding window, and then the data of the plurality of view angles in the window are stitched to generate the multi-grid 4K video.

Furthermore, the reference view angle can be determined based on a view angle setting instruction sent by the playing device, that is, the sliding window can respond to a view angle setting instruction generated based on user operation sent by the display device in real time, so that the global view angle number of the user is not affected, and based on the adjustment instruction, the sliding window can be selected by taking any view angle in the global view angle as the reference view angle, so that the problem of less collection view angle number caused by limited 4K resolution in the related art is effectively solved on the premise of determining the playing resolution or the transmission resolution.

In one embodiment, the number of views of the sliding window is less than or equal to a window threshold, and the window threshold is configured based on at least one of a playback resolution of the playback device, a transmission resolution of the processing device, and a stitching resolution of the multi-grid video image.

The playing resolution of the playing device may be sent to the processing device in advance, or may be determined by the processing device based on the industry-defined playing resolution.

As shown in fig. 4, in an embodiment, a specific implementation manner of selecting an adjacent view of a reference view in a multi-angle array based on the number of views of the sliding window and the selection rule in step S204 includes:

step S402, when the selection rule is suitable for selecting two side visual angles adjacent to the reference visual angle, determining a first parameter and a second parameter based on the visual angle number of the sliding window.

The selection rule is suitable for selecting two side visual angles adjacent to the reference visual angle, and particularly, the adjacent visual angles are not on two sides of the reference visual angle by taking the reference visual angle as a center.

In step S404, an adjacent view is determined based on a relationship between the corresponding number in the multi-angle of the reference view and the first parameter and/or the second parameter.

Wherein, S is the number of viewing angles of the sliding window, d is the serial number of the reference viewing angle, N is the total number of all the collected viewing angles, Δ l is a first parameter, and Δ r is a second parameter.

In this embodiment, the first parameter and the second parameter are determined based on the number of views of the sliding window, so that after the serial number of the reference view is determined, other adjacent views in the sliding window can be determined based on the relationship between the serial number of the reference view and the first parameter and/or the second parameter, so as to ensure reasonable layout of different views in the sliding window.

As shown in fig. 5, in an embodiment, in step S402, determining the first parameter and the second parameter based on the number of viewing angles of the sliding window specifically includes:

in step S502, a first parameter is calculated based on a first calculation formula.

In step S504, a second parameter is calculated based on the second calculation formula.

Wherein the first calculation formula is used for taking the value less than or equal to

The second calculation expression is used to take the integer greater than or equal to

S is the number of views of the sliding window.

Specifically, the first calculationIs of the formula

I.e. taking the largest integer less than or equal to the number in the symbol, the second formula being

I.e. the smallest integer greater than or equal to the number within the symbol.

In one embodiment, determining the adjacent views based on the relationship between the corresponding number in the multi-angle with reference to the views and the first parameter and/or the second parameter includes:

in step S506, when the difference between the corresponding sequence number and the first parameter is less than 1, the sequence numbers of the adjacent views are configured based on the number of views of the sliding window.

Step S508, when the sum of the corresponding serial number and the second parameter is greater than the total number of views of the multi-angle array, configuring the serial number of the adjacent view in a first numerical range, where the first numerical range is greater than the difference between the total number of views and the number of views and is less than or equal to the total number of views.

Step S510, when the corresponding sequence number is in another value range, configuring a sequence number of the adjacent view angle in a second value range, where the second value range is greater than or equal to the difference between the corresponding sequence number and the first parameter, and is less than or equal to the sum of the corresponding sequence number and the second parameter.

Specifically, the number of views within the sliding window is determined based on the following equation.

Wherein D is_iAnd i is the serial number of the corresponding visual angle.

In this embodiment, after the reference view angle is determined, based on the relationship between the number of the reference view angle and the first parameter, and the second parameter, other view angles adjacent to the reference view angle are determined, on one hand, video images of a plurality of consecutive view angles in the sliding window can be ensured, so that a user can conveniently select a view angle desired to be watched, on the other hand, by fully considering the range of the number of the reference view angle, reliability of view angle selection of the sliding window can be ensured, and particularly, selection abnormality can be prevented from occurring when the reference view angle is located at both ends.

In one embodiment, in step S204, another specific implementation manner of selecting an adjacent view of a reference view in a multi-angle array based on the number of views of the sliding window and a selection rule includes:

when the selection rule is suitable for selecting a side view adjacent to the reference view, the adjacent view is determined based on the preset position of the adjacent view relative to the reference view and the number of views.

As shown in fig. 6, in an embodiment, the determining the adjacent viewing angles based on the preset positions and the number of viewing angles of the adjacent viewing angles relative to the reference viewing angle specifically includes:

in step S602, when the adjacent view angle is preset on the left side with respect to the reference view angle, it is detected whether the number of left view angles of the reference view angle satisfies the number of view angles of the sliding window.

In step S604, if the number of views of the sliding window is satisfied, a plurality of consecutive views are selected as neighboring views from the left view.

In step S606, if the number of views of the sliding window is not satisfied, the remaining number of views is determined based on the number of views of the sliding window and the number of left views, so as to select all right views adjacent to the left views and the remaining number of views.

As shown in fig. 7, in an embodiment, the determining the adjacent viewing angles based on the preset positions and the number of viewing angles of the adjacent viewing angles relative to the reference viewing angle specifically includes:

in step S702, when the adjacent viewing angle is preset on the right side with respect to the reference viewing angle, it is detected whether the right viewing angle number of the reference viewing angle satisfies the viewing angle number of the sliding window.

In step S704, if the number of viewing angles of the sliding window is satisfied, a plurality of consecutive viewing angles are selected as the adjacent viewing angles in the right side viewing angle.

In step S706, if the number of views of the sliding window is not satisfied, the remaining number of views is determined based on the number of views of the sliding window and the number of right views, so as to select all right views and left views adjacent to the remaining number of views.

In this embodiment, the reference view angle may be used as a reference view angle at the end of the sliding window, that is, the reference view angle is placed at the leftmost end or the rightmost end of the sliding window, and if the reference view angle is placed at the leftmost end, if the number of right-side adjacent view angles satisfies the number of remaining view angles, it is determined that the reference view angle is placed at the leftmost end, if the number of right-side adjacent view angles does not satisfy the number of remaining view angles, the reference view angle is shifted to the right at the portion where the right-side adjacent view angles are not sufficient, and if the number of right-side adjacent view angles is placed at the rightmost end, the manner is opposite to the manner of placing the leftmost end, and this manner can also obtain a plurality of continuous and reliable view angles.

In one embodiment, further comprising: and when an adjusting instruction for changing the reference visual angle is received, determining the adjusted reference visual angle based on the adjusting instruction, and reconfiguring the visual angle of the video image in the sliding window.

In this embodiment, the instruction sent by the display device can be analyzed through interaction with the display device, when the instruction is an adjustment instruction, the corresponding play angle is determined based on the adjustment instruction and is used as a reference angle, and an angle closest to the adjusted reference angle is selected to the sliding window based on the selection mode of the adjacent angles, wherein the number of angles and the angle range of the sliding window can be dynamically adjusted.

Furthermore, the video images corresponding to the visual angles in the sliding window are spliced to generate the multi-grid video image, on one hand, the video images of all the visual angles do not need to be spliced, and on the other hand, if the user needs to check the video images of other visual angles, the user only needs to send an adjusting instruction through the display equipment.

And after the processing equipment receives the adjusting instruction, adjusting the reference visual angle by adjusting the parameter of the reference visual angle, and adjusting the adjacent visual angle of the sliding window after the reference visual angle is adjusted.

Still further, at the display device side, the view switching range of a single operation of the user may be limited within the current sliding window, but the sliding window is dynamically updated, and the global view range of the user is not affected.

As shown in fig. 8, a multi-view video transmission method according to another embodiment of the present disclosure is applied to a display device, including:

and step S802, receiving the multi-grid video image sent by the processing equipment.

Step S804, determining the playing visual angle in the multi-grid video image based on the visual angle selection operation of the user.

The user can perform the operation of selecting the view angle in the multi-grid video image before receiving the multi-grid video image, or after receiving the multi-grid video image, and when the operation is performed before receiving the multi-grid video image, the user correspondingly sends the generated view angle setting instruction to the processing device.

Step 806, extracting a video image to be played corresponding to the playing view angle from the multi-grid video images.

And step S808, performing up-sampling operation on the video image to be played based on the playing resolution, generating a playing video, and displaying the playing video.

In the embodiment, a sliding window with a certain visual angle number is arranged at a processing equipment end, when multiple groups of video images synchronously acquired by a camera group arranged in a multi-angle array are acquired, a plurality of continuous visual angles are selected in the multi-angle array based on the visual angle number set by the sliding window, so that the video images of the plurality of continuous visual angles are spliced to obtain a multi-grid video image, a display device receives the multi-grid video image, and through visual angle selection operation of a user, the playing visual angle can be determined based on the selection of the user to play video.

According to the multi-view video transmission scheme disclosed by the invention, on one hand, according to user operation, data of a plurality of continuous views which are most adjacent to the current view are adaptively selected and stored in a sliding window, and then the data of the plurality of views in the window are spliced to form a multi-grid 4K video without splicing video images of all views; on the other hand, the saved pixels can improve the resolution of the visual angle within the window range, and the video quality of the visual angle can be improved under the limited transmission resolution; on the other hand, the sliding window responds to the user operation in real time, does not influence the overall visual angle range of the user, does not limit the total visual angle quantity, and is beneficial to expanding the visual angle range and reducing the visual angle difference.

As shown in fig. 9, a multi-view video transmission method according to still another embodiment of the present disclosure is applied to a display device, including:

step S902, receiving a view angle selection operation by a user.

In step S904, a corresponding view angle setting instruction is generated based on the view angle selection operation.

In step S906, the angle-of-view setting instruction is sent to the processing apparatus.

Step S908, when the multi-grid video image is received, determining a playing angle of view directly based on the angle of view setting instruction.

In the embodiment, before the display device receives the multi-grid video image, the visual angle selection operation of the user can be received, the corresponding visual angle setting instruction is generated, the visual angle setting instruction is sent to the processing device, the processing device can set the reference visual angle according to the visual angle setting instruction, further determines the adjacent visual angle based on the reference visual angle, and splices the adjacent visual angle into the multi-grid video image, when the multi-grid video image is received, the playing visual angle can be automatically determined based on the visual angle setting instruction generated before, so that the video is played based on the playing visual angle, and therefore the operation steps of the user can be simplified.

As shown in fig. 10, a multi-view video transmission method according to still another embodiment of the present disclosure is applied to a display device, including:

and step S1002, receiving the multi-grid video image sent by the processing equipment.

The generation of the multi-grid video image can be based on a preset reference visual angle setting instruction or a reference visual angle automatically selected by the processing equipment.

And step S1004, displaying the received multi-grid video image.

Step S1006, receiving a view angle selection operation for the multi-grid video image.

In step S1008, a play angle of view is determined based on the angle of view selection operation.

In this embodiment, after receiving the multi-grid video image, the method makes the selection reliability of the playing view higher after receiving the view selection operation of the user on the multi-grid video image.

As shown in fig. 11, in one embodiment, the method further includes:

step S1102 is to generate a view adjustment area, where the view adjustment area includes a multi-angle array and a sliding window sliding in the multi-angle array.

In step S1104, an adjustment operation of the sliding window by the user is received, so as to generate a corresponding adjustment instruction based on the adjustment operation.

In step S1106, an adjustment instruction is transmitted to the processing device to adjust the reference angle of view by the processing device based on the adjustment instruction.

In the embodiment, by generating the adjustment operation on the sliding window, if the user needs to view the video images at other visual angles, the user only needs to send the adjustment instruction, so that the video images at all visual angles do not need to be spliced at the processing equipment end, only the multi-grid video image generated at a part of visual angles which are willing to be viewed needs to be selected, and the multi-grid video image can be switched at any time.

In addition, as can be understood by those skilled in the art, the multi-view video transmission scheme of the present disclosure is applied to a system in which a camera, a processing device and a display device interact, and the system can realize real-time live broadcast of multi-angle video images.

Specifically, in the application of the free visual angle technology, a visual angle adjusting area is generated at a display equipment end, a user is supported to freely drag and change a visual angle, the video content can be watched at multiple angles, the problems that the traditional video watching visual angle is limited and lacks operability are solved, and the method and the device can be used for scenes such as live competition and live comprehensive art. Therefore, the video quality of the free visual angle technology can be improved under the condition of limited bandwidth and equipment decoding capacity; and on the premise of not limiting the number of the collected visual angles, the visual angle range of the free visual angle technology is favorably expanded, and the visual angle difference is reduced.

As shown in fig. 12, the multi-angle video transmission system according to the present disclosure includes an acquisition module 1202, a splicing module 1204, and an interaction and playing module 1206, wherein the acquisition module is specifically a camera group, the splicing module is specifically a processing device, and the interaction and playing module is specifically a display device.

The acquisition module 1202 is configured to acquire multi-angle video images of a camera array and perform synchronization;

the splicing module 1204 is specifically configured to:

(1) the instruction sent by the interaction and play module 1206 is analyzed to determine the current reference view angle and use the current reference view angle as the play view angle.

(2) The number and range of view angles of the window may be dynamically adjusted based on the reference view angle to determine other adjacent view angles within the sliding window.

(3) Splicing the visual angles in the sliding window into a video image with multiple grids; as shown in fig. 3, the current view angle D7 and the window size 9 are all the 9 view angles that are the most adjacent to the current view angle, no matter how many view angles are arranged, to be selected into the sliding window and spliced.

The interaction and play module 1206 is specifically configured to:

(1) and receiving the spliced multi-view video, sending a view angle adjusting instruction to the splicing module, and capturing and playing the video image of the playing view angle.

(2) And acquiring operation information of a user for adjusting the visual angle, and sending an adjusting instruction with the adjusted visual angle parameter to the splicing module after detecting that the visual angle is changed.

(3) The visual angle switching range of the single operation of the user is limited in the current sliding window, and the global visual angle range of the user is not influenced due to the dynamic update of the sliding window.

As shown in fig. 13, in conjunction with fig. 12, the multi-view video transmission scheme according to the present disclosure specifically includes:

in step S1302, the splicing module sets a sliding window.

The window size, i.e., the number of views S in the sliding window, is set to be N (numbers 1,2, …, N).

In step S1304, the acquisition module acquires a plurality of groups of video images at N viewing angles of the camera array and synchronously transmits the video images to the splicing module.

Step S1306, the splicing module receives multiple groups of video images of multiple viewing angles synchronously acquired by the acquisition module, and determines multiple continuous viewing angles in the current sliding window.

Specifically, let d be the serial number of the reference viewing angle, and select S-1 viewing angles most adjacent to d to be placed in the sliding window.

The method comprises the following steps that firstly, a sliding window is used for taking a visual angle [1, S ], secondly, a median of S can be taken, the initial visual angle range of the sliding window can be determined according to the median visual angle, and if the visual angle is updated, the values of the visual angles Di, delta l and delta r in the sliding window are determined according to the current visual angle parameter d and are related to the median of the window, and the two are interchangeable.

In addition, the serial number d of the reference view angle can also be determined by the interactive and playing module.

Wherein D is_iI is the number of the corresponding view angle,

step S1308, the stitching module stitches the video images of the multiple viewing angles in the sliding window through the downsampling operation, so as to generate a multi-grid video image.

Specifically, the video images of each view in the sliding window are down-sampled to a specified resolution (the value is determined by S and transmission resolution, such as 4K and 9 view sliding windows, corresponding to 720P), and the down-sampled video images are spliced to form a large multi-grid video image, compressed and encoded, and transmitted to the interaction and playing module for display.

In step S1310, the interaction and playing module receives the multi-view video stream and the view range parameter of the splicing module, and determines the current playing view.

In step S1312, the interaction and playing module captures a video image of the current playing view angle from the multi-grid video image, and converts the captured video image to a resolution set by the device through upsampling for playing.

Step S1314, the interaction and playing module monitors the user operation, generates an adjustment instruction if the user drags to switch the current playing angle of view, and sends the adjustment instruction to the splicing module.

Specifically, the user can drag left and right to change the viewing angle, and the viewing angle range is limited in the current sliding window.

Step S1316, the splicing module monitors the instruction sent by the interaction and playing module, receives the adjustment instruction, returns to step S1306, and re-determines a plurality of continuous viewing angles in the current sliding window.

It is to be noted that the above-mentioned figures are only schematic illustrations of the processes involved in the method according to an exemplary embodiment of the invention, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.

A multiview video transmission apparatus 1400 according to an embodiment of the present invention is described below with reference to fig. 14. The multi-view video transmission apparatus 1400 shown in fig. 14 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.

The multi-view video transmission apparatus 1400 is represented in the form of a hardware module. The components of the multiview video transmission apparatus 1400 may include, but are not limited to: an obtaining module 1402, configured to obtain multiple sets of video images collected by a camera group arranged in a multi-angle array, where each set of video images corresponds to a camera shooting angle; a selecting module 1404 configured to select a plurality of consecutive views in the multi-angle array based on a number of views of a sliding window, the sliding window configured to slide in an array region corresponding to the multi-angle array, the number of views of the sliding window being less than a total number of views of the multi-angle array; a stitching module 1406 for stitching the video images of the multiple continuous viewing angles based on the downsampling operation to generate a multi-grid video image; a sending module 1408, configured to perform compression coding processing on the multi-grid video image, so as to send the processed multi-grid video image to a playing device.

In one embodiment, the plurality of consecutive views comprises a reference view, and the selection module 1404 is further configured to: based on the number of views of the sliding window and a selection rule, selecting an adjacent view of a reference view in the multi-angle array to obtain a plurality of continuous views.

In one embodiment, further comprising: the receiving module 1410 is configured to receive a view setting instruction sent by the playback device, and determine a corresponding reference view based on the view setting instruction; or determining module 1412, configured to determine one of the median of the number of views of the sliding window as the serial number of the reference view.

In one embodiment, the selection module 1404 is further configured to: determining a first parameter and a second parameter based on the number of viewing angles of the sliding window when the selection rule is suitable for selecting two viewing angles adjacent to the reference viewing angle; the contiguous views are determined based on a relationship between the first parameter and/or the second parameter with reference to corresponding numbers of views in multiple angles.

In one embodiment, the selection module 1404 is further configured to: calculating a first parameter based on a first calculation formula; calculating the second parameter based on the second calculation formula, wherein the first calculation formula is used for taking the value less than or equal to

S is the number of views of the sliding window.

In one embodiment, the selection module 1404 is further configured to: when the difference value between the corresponding sequence number and the first parameter is less than 1, configuring the sequence number of the adjacent visual angle based on the visual angle number of the sliding window; when the sum of the corresponding serial number and the second parameter is larger than the total number of the visual angles of the multi-angle array, configuring the serial number of the adjacent visual angle in a first numerical range, wherein the first numerical range is larger than the difference value between the total number of the visual angles and the number of the visual angles and is smaller than or equal to the total number of the visual angles; and when the corresponding serial number is in other numerical value ranges, configuring the serial number of the adjacent visual angle in a second numerical value range, wherein the second numerical value range is larger than or equal to the difference value between the corresponding serial number and the first parameter and is smaller than or equal to the sum of the corresponding serial number and the second parameter.

In one embodiment, the selection module 1404 is further configured to: when the selection rule is suitable for selecting a side view adjacent to the reference view, the adjacent view is determined based on the preset position of the adjacent view relative to the reference view and the number of views.

In one embodiment, the selection module 1404 is further configured to: when the adjacent visual angles are preset on the left side relative to the reference visual angle, detecting whether the quantity of the left visual angles of the reference visual angle meets the quantity of the visual angles of the sliding window; if the number of the visual angles of the sliding window is met, selecting a plurality of continuous visual angles from the left visual angle as adjacent visual angles; and if the number of the visual angles of the sliding window is not satisfied, determining the number of the remaining visual angles based on the number of the visual angles of the sliding window and the number of the left visual angles so as to select all the left visual angles and the adjacent right visual angles of the number of the remaining visual angles.

In one embodiment, the selection module 1404 is further configured to: when the adjacent visual angles are preset on the right side relative to the reference visual angle, detecting whether the right visual angle number of the reference visual angle meets the visual angle number of the sliding window; if the number of the visual angles of the sliding window is met, selecting a plurality of continuous visual angles from the right side visual angles as adjacent visual angles; and if the number of the visual angles of the sliding window is not satisfied, determining the number of the residual visual angles based on the number of the visual angles of the sliding window and the number of the right visual angles so as to select all the right visual angles and the adjacent left visual angles of the number of the residual visual angles.

In one embodiment, further comprising: the adjusting module 1414 is configured to, when receiving an adjusting instruction for changing the reference view angle, determine an adjusted reference view angle based on the adjusting instruction, and reconfigure the video image view angle in the sliding window.

A multiview video transmission apparatus 1500 according to an embodiment of the present invention is described below with reference to fig. 15. The multi-view video transmission apparatus 1500 shown in fig. 15 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.

The multi-view video transmission apparatus 1500 is represented in the form of a hardware module. The components of the multiview video transmission apparatus 1500 may include, but are not limited to: a receiving module 1502, configured to receive a multi-grid video image sent by a processing device; a determining module 1504, configured to determine a playing view in the multi-grid video image based on a view selection operation of the user, so as to play based on the playing view.

In one embodiment, before receiving the multi-grid video image sent by the processing device, the method further includes: the receiving module 1502 is further configured to: receiving a visual angle selection operation of a user; a first generating module 1506, configured to generate a corresponding view setting instruction based on the view selecting operation; a sending module 1508, configured to send the view angle setting instruction to the processing device, so as to determine, when the multi-grid video image is received, a playing view angle directly based on the view angle setting instruction.

In one embodiment, the determining module 1504 is further configured to: displaying the received multi-grid video image; receiving visual angle selection operation of the multi-grid video image; and determining a playing visual angle based on the visual angle selection operation.

In one embodiment, further comprising: a second generating module 1510, configured to generate a view adjustment area, where the view adjustment area includes a multi-angle array and a sliding window sliding in the multi-angle array; the receiving module 1506 is further configured to: receiving adjustment operation of a user on the sliding window to generate a corresponding adjustment instruction based on the adjustment operation; the sending module 1508 is further configured to: the adjustment instruction is sent to the processing device to adjust, by the processing device, the reference perspective based on the adjustment instruction.

In one embodiment, further comprising: the extracting module 1512 is configured to extract a to-be-played video image corresponding to a playing view in the multi-grid video image; a playing module 1514, configured to perform upsampling operation on the video image to be played based on the playing resolution, generate a playing video, and display the playing video.

As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.

A computer apparatus 1600 according to this embodiment of the invention is described below with reference to fig. 16. The computer device 1600 shown in FIG. 16 is only one example, and should not bring any limitations to the functionality or scope of use of embodiments of the present invention.

As shown in fig. 16, computer device 1600 is in the form of a general purpose computing device. Components of computer device 1600 may include, but are not limited to: the at least one processing unit 1610, the at least one memory unit 1620, and a bus 1630 that couples various system components including the memory unit 1620 and the processing unit 1610.

Wherein the memory unit stores program code that may be executed by the processing unit 1610 to cause the processing unit 1610 to perform steps according to various exemplary embodiments of the present invention as described in the above section "exemplary method" of the present specification. For example, the processing unit 1610 may execute the scheme described in step S202 to step S208 shown in fig. 2.

The memory unit 1620 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM)16201 and/or a cache memory unit 16202, and may further include a read only memory unit (ROM) 16203.

The storage unit 1620 may also include a program/utility 16204 having a set (at least one) of program modules 16205, such program modules 16205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.

Bus 1630 may be any of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

Computer device 1600 may also communicate with one or more external devices 1670 (e.g., keyboard, pointing device, bluetooth device, etc.), and may also communicate with one or more devices that enable a user to interact with computer device 1600, and/or any devices (e.g., router, modem, etc.) that enable computer device 1600 to communicate with one or more other computing devices. Such communication may occur through input/output (I/O) interface 1650. Also, computer device 1600 can communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the internet) through network adapter 1660. As shown, network adapter 1660 communicates with the other modules of computer device 1600 via bus 1630. It should be appreciated that although not shown, other hardware and/or software modules may be used in conjunction with computer device 1600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, aspects of the invention may also be implemented in the form of a program product comprising program code means for causing a terminal device to carry out the steps according to various exemplary embodiments of the invention described in the above section "exemplary methods" of the present description, when said program product is run on the terminal device.

Referring to fig. 17, a program product 1700 for implementing the above method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited in this regard and, in the present document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).

It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.

Moreover, although the steps of the methods of the present disclosure are depicted in the drawings in a particular order, this does not require or imply that the steps must be performed in this particular order, or that all of the depicted steps must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions, etc.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

Claims

1. A multi-view video transmission method applied to a processing device includes:

acquiring a plurality of groups of video images acquired by a camera group arranged in a multi-angle array, wherein each group of video images corresponds to a camera shooting angle;

selecting a plurality of consecutive views in the multi-angle array based on a number of views of a sliding window, the sliding window configured to slide in an array region corresponding to the multi-angle array, the number of views of the sliding window being less than a total number of views of the multi-angle array;

splicing the video images of the multiple continuous visual angles based on downsampling operation to generate a multi-grid video image;

and carrying out compression coding processing on the multi-grid video image so as to send the processed multi-grid video image to a playing device.

2. The multiview video transmission method of claim 1, wherein,

the number of the visual angles of the sliding window is smaller than or equal to a window threshold value, and the window threshold value is configured based on at least one of the playing resolution of the playing device, the transmission resolution of the processing device and the splicing resolution of the multi-grid video image.

3. The method according to claim 1, wherein the plurality of consecutive views comprise a reference view, and wherein selecting the plurality of consecutive views in the multi-angle array based on the number of views of the sliding window comprises:

and selecting an adjacent view of the reference view from the multi-angle array based on the number of views of the sliding window and a selection rule to obtain the plurality of continuous views.

4. The multi-view video transmission method according to claim 3, further comprising, before selecting an adjacent view of said reference views in said multi-angle array based on a number of views of a sliding window and a selection rule:

receiving a visual angle setting instruction sent by a playing device, and determining the corresponding reference visual angle based on the visual angle setting instruction; or

And determining one of the median of the number of the visual angles of the sliding window as the serial number of the reference visual angle.

5. The method according to claim 3, wherein the selecting neighboring views of the reference view in the multi-angle array based on the number of views of the sliding window and a selection rule comprises:

when the selection rule is suitable for selecting two side visual angles adjacent to the reference visual angle, determining a first parameter and a second parameter based on the visual angle number of the sliding window;

determining the contiguous views based on a relationship between the corresponding sequence numbers of the reference views in the multi-angle and the first parameter and/or the second parameter.

6. The method for multiview video transmission according to claim 5, wherein the determining the first parameter and the second parameter based on the number of views of the sliding window specifically comprises:

calculating the first parameter based on a first calculation formula;

calculating the second parameter based on a second calculation formula,

S is the number of views of the sliding window.

7. The method for multiview video transmission according to claim 6, wherein the determining the neighboring views based on the relationship between the corresponding sequence number of the reference view in the multiple angles and the first parameter and/or the second parameter comprises:

when the difference value between the corresponding sequence number and the first parameter is less than 1, configuring the sequence number of the adjacent view angle based on the number of the view angles of the sliding window;

when the sum of the corresponding sequence number and the second parameter is greater than the total number of the visual angles of the multi-angle array, configuring the sequence number of the adjacent visual angle in a first numerical range, wherein the first numerical range is greater than the difference value between the total number of the visual angles and the number of the visual angles and is less than or equal to the total number of the visual angles;

and when the corresponding sequence number is in other numerical value ranges, configuring the sequence number of the adjacent view angle in a second numerical value range, wherein the second numerical value range is greater than or equal to the difference value between the corresponding sequence number and the first parameter and is less than or equal to the sum of the corresponding sequence number and the second parameter.

8. The method according to claim 3, wherein the selecting neighboring views of the reference view in the multi-angle array based on the number of views of the sliding window and a selection rule comprises:

and when the selection rule is suitable for selecting a side view adjacent to the reference view, determining the adjacent view based on the preset position of the adjacent view relative to the reference view and the number of views.

9. The method for multiview video transmission according to claim 8, wherein the determining the neighboring views based on the preset positions of the neighboring views relative to the reference views and the number of views specifically comprises:

when the adjacent visual angle is preset on the left side relative to the reference visual angle, detecting whether the left visual angle quantity of the reference visual angle meets the visual angle quantity of the sliding window;

if the number of the visual angles of the sliding window is met, selecting a plurality of continuous visual angles from the left visual angle as the adjacent visual angle;

if the number of the visual angles of the sliding window is not satisfied, determining the number of the remaining visual angles based on the number of the visual angles of the sliding window and the number of the left visual angles, so as to select all the right visual angles adjacent to the left visual angles and the number of the remaining visual angles.

10. The method for multiview video transmission according to claim 8, wherein the determining the neighboring views based on the preset positions of the neighboring views relative to the reference views and the number of views specifically comprises:

when the adjacent visual angle is preset on the right side relative to the reference visual angle, detecting whether the right visual angle quantity of the reference visual angle meets the visual angle quantity of the sliding window;

if the number of the visual angles of the sliding window is met, selecting a plurality of continuous visual angles from the right visual angle as the adjacent visual angle;

if the number of the visual angles of the sliding window is not satisfied, determining the number of the remaining visual angles based on the number of the visual angles of the sliding window and the number of the right visual angles, so as to select all the right visual angles and the left visual angles adjacent to the number of the remaining visual angles.

11. The multiview video transmission method of any one of claims 3 to 10, further comprising:

when an adjusting instruction for changing the reference visual angle is received, the adjusted reference visual angle is determined based on the adjusting instruction, and the visual angle of the video image in the sliding window is reconfigured.

12. A multi-view video transmission method applied to a display device includes:

receiving a multi-grid video image sent by a processing device;

and determining a playing visual angle in the multi-grid video image based on visual angle selection operation of a user so as to play based on the playing visual angle.

13. The multiview video transmission method of claim 12, further comprising, prior to receiving the multi-grid video image sent by the processing device:

receiving the view angle selection operation of a user;

generating a corresponding visual angle setting instruction based on the visual angle selection operation;

and sending the visual angle setting instruction to the processing equipment so as to directly determine the playing visual angle based on the visual angle setting instruction when the multi-grid video image is received.

14. The method for transmitting multi-view video according to claim 12, wherein the determining the playing view in the multi-grid video image based on the view selection operation of the user specifically comprises:

displaying the received multi-grid video image;

receiving the visual angle selection operation of the multi-grid video image;

determining the playing view based on the view selection operation.

15. The multiview video transmission method of claim 12, further comprising:

generating a visual angle adjusting area, wherein the visual angle adjusting area comprises a multi-angle array and a sliding window sliding on the multi-angle array;

receiving adjustment operation of a user on the sliding window to generate a corresponding adjustment instruction based on the adjustment operation;

sending the adjustment instruction to the processing device to adjust, by the processing device, a reference perspective based on the adjustment instruction.

16. The method for transmitting multiview video according to any of claims 12 to 15, wherein the playing based on the playing view specifically comprises:

extracting a video image to be played corresponding to the playing visual angle from the multi-grid video images;

and performing up-sampling operation on the video image to be played based on the playing resolution ratio to generate a playing video, and displaying the playing video.

17. A multi-view video transmission apparatus applied to a processing device, comprising:

the system comprises an acquisition module, a display module and a control module, wherein the acquisition module is used for acquiring a plurality of groups of video images acquired by a camera group arranged in a multi-angle array, and each group of video images corresponds to a camera shooting angle;

a selection module, configured to select a plurality of consecutive views in the multi-angle array based on a number of views of a sliding window, where the sliding window is configured to slide in an array region corresponding to the multi-angle array, and the number of views of the sliding window is less than a total number of views of the multi-angle array;

the splicing module is used for splicing the video images of the multiple continuous visual angles based on downsampling operation to generate a multi-grid video image;

and the sending module is used for carrying out compression coding processing on the multi-grid video image so as to send the processed multi-grid video image to a playing device.

18. A multi-view video transmission apparatus applied to a display device, comprising:

the receiving module is used for receiving the multi-grid video image sent by the processing equipment;

and the determining module is used for determining a playing visual angle in the multi-grid video image based on visual angle selection operation of a user so as to play based on the playing visual angle.

19. A processing device, a display device, comprising:

a processor; and

a memory for storing executable instructions of the processor;

wherein the processor is configured to perform the multi-view video transmission method of any of claims 1-11 via execution of the executable instructions.

20. A display device, comprising:

a processor; and

a memory for storing executable instructions of the processor;

wherein the processor is configured to perform the multi-view video transmission method of any of claims 12 to 16 via execution of the executable instructions.

21. A computer-readable storage medium, on which a computer program is stored, the computer program, when being executed by a processor, implementing the multiview video transmission method of any one of claims 1 to 16.