CN116957913A - Panoramic video mapping method, device, electronic equipment and storage medium - Google Patents
Panoramic video mapping method, device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN116957913A CN116957913A CN202211085497.2A CN202211085497A CN116957913A CN 116957913 A CN116957913 A CN 116957913A CN 202211085497 A CN202211085497 A CN 202211085497A CN 116957913 A CN116957913 A CN 116957913A
- Authority
- CN
- China
- Prior art keywords
- pictures
- picture
- panoramic video
- model
- triangular prism
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013507 mapping Methods 0.000 title claims abstract description 96
- 238000000034 method Methods 0.000 title claims abstract description 59
- 238000004590 computer program Methods 0.000 claims description 10
- 238000010586 diagram Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 5
- 238000011161 development Methods 0.000 description 4
- 230000018109 developmental process Effects 0.000 description 4
- 238000012544 monitoring process Methods 0.000 description 4
- 230000003993 interaction Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Abstract
The invention provides a panoramic video mapping method, a panoramic video mapping device, electronic equipment and a storage medium. Acquiring a spherical panoramic video, mapping the spherical panoramic video to a triangular prism-like model to acquire a plurality of pictures, namely mapping all the content of the spherical panoramic video to the triangular prism-like model, so as to cover all the video pictures after the spherical panoramic video is unfolded, ensure that the video pictures are complete and avoid losing the content, and combining the pictures to acquire a planar rectangular video picture so as to completely present all the content of the spherical panoramic video; the plurality of pictures comprise three horizontal pictures corresponding to three side faces of the triangular prism-like model, a first bottom picture corresponding to a first bottom face of the triangular prism-like model and a second bottom picture corresponding to a second bottom face of the triangular prism-like model, namely, each face of the triangular prism-like model is mapped with at least one picture, and the content of each picture is free from picture distortion, so that the combined plane rectangular video picture is undistorted, and the picture is guaranteed to be undistorted.
Description
Technical Field
The present invention relates to the field of virtual reality technologies, and in particular, to a panoramic video mapping method, a device, an electronic device, and a storage medium.
Background
With the rapid development of communication technology, the experience requirement of people on video live broadcast is continuously improved. Compared with the traditional common 2D video live broadcast, the virtual reality video with higher resolution and immersive panoramic interaction becomes a new direction of video live broadcast technology development. Therefore, there is a need to present spherical panoramic video to a flat panel display.
At present, spherical panoramic video is mostly projected to a plane through a mapping mode such as longitude and latitude map mapping, hexahedron mapping and the like, so that panoramic video mapping is completed. However, in the mapping process, there may be a case where the content is lost, so that a complete picture cannot be comprehensively presented; and mapping all the content of the spherical panoramic video into a smaller plane will result in picture distortion. Therefore, how to completely present all the content of the spherical panoramic video and ensure that the picture is not distorted is a problem that needs to be solved at present.
Disclosure of Invention
The invention provides a panoramic video mapping method, a device, electronic equipment and a storage medium, which are used for solving the defects of content loss and picture distortion in spherical panoramic video mapping in the prior art.
The invention provides a panoramic video mapping method, which comprises the following steps:
acquiring a spherical panoramic video;
mapping the spherical panoramic video to a triangular prism-like model to obtain a plurality of pictures, wherein the pictures comprise three horizontal pictures corresponding to three sides of the triangular prism-like model, a first bottom picture corresponding to a first bottom surface of the triangular prism-like model and a second bottom picture corresponding to a second bottom surface of the triangular prism-like model;
and combining the multiple pictures to obtain a planar rectangular video picture.
According to the panoramic video mapping method provided by the invention, the mapping of the spherical panoramic video to the triangular prism-like model to obtain a plurality of pictures comprises the following steps:
mapping the spherical panoramic video to a triangular prism-like model based on a plurality of virtual cameras to obtain a plurality of pictures;
the plurality of virtual cameras are arranged in the central area of the triangular prism-like model, the plurality of virtual cameras comprise three horizontal cameras, a first bottom camera and a second bottom camera, the three horizontal cameras are used for shooting and acquiring three horizontal pictures in the directions of the three side faces respectively, the first bottom camera is used for shooting and acquiring the first bottom picture in the direction of the first bottom face, and the second bottom camera is used for shooting and acquiring the second bottom picture in the direction of the second bottom face.
According to the panoramic video mapping method provided by the invention, the number of the first bottom cameras is a plurality of the second bottom cameras;
the number of the first bottom end pictures is a plurality of the second bottom end pictures.
According to the panoramic video mapping method provided by the invention, the number of the first bottom cameras is 3, the number of the second bottom cameras is 3, the number of the first bottom pictures is 3, and the number of the second bottom pictures is 3;
the horizontal view field angle of the first bottom camera is 120 degrees, and the horizontal view field angle of the second bottom camera is 120 degrees.
According to the panoramic video mapping method provided by the invention, the first bottom end picture is a plane rectangular picture, and the second bottom end picture is a plane rectangular picture;
and the first bottom end pictures have a common picture, and the second bottom end pictures have a common picture.
According to the panoramic video mapping method provided by the invention, the horizontal view field angle of the horizontal camera is 120 degrees.
According to the panoramic video mapping method provided by the invention, the vertical view field angle of the horizontal camera is 120 degrees;
The vertical view field angle of the first bottom camera is 30 degrees, and the vertical view field angle of the second bottom camera is 30 degrees.
According to the panoramic video mapping method provided by the invention, the combining of the multiple pictures to obtain a planar rectangular video picture comprises the following steps:
determining the joint of the pictures based on the edges of the triangular prism-like model;
and merging the multiple pictures based on the joint to obtain a planar rectangular video picture.
The invention also provides a panoramic video mapping device, which comprises:
the acquisition module is used for acquiring spherical panoramic video;
the mapping module is used for mapping the spherical panoramic video to a triangular prism-like model to obtain a plurality of pictures, wherein the pictures comprise three horizontal pictures corresponding to three sides of the triangular prism-like model, a first bottom picture corresponding to a first bottom surface of the triangular prism-like model and a second bottom picture corresponding to a second bottom surface of the triangular prism-like model;
and the merging module is used for merging the plurality of pictures to obtain a plane rectangular video picture.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the panoramic video mapping method as described above when executing the program.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a panoramic video mapping method as described in any of the above.
According to the panoramic video mapping method, the device, the electronic equipment and the storage medium, the spherical panoramic video is obtained, the spherical panoramic video is mapped to the triangular prism-like model, and a plurality of pictures are obtained, namely, all the content of the spherical panoramic video is mapped to the triangular prism-like model, so that all video pictures after the spherical panoramic video is unfolded are covered, the completeness of the video pictures is ensured, the content is prevented from being lost, and then the pictures are combined to obtain a planar rectangular video picture, and all the content of the spherical panoramic video is completely presented; the plurality of pictures comprise three horizontal pictures corresponding to three side faces of the triangular prism-like model, a first bottom picture corresponding to a first bottom face of the triangular prism-like model and a second bottom picture corresponding to a second bottom face of the triangular prism-like model, namely, each face of the triangular prism-like model is mapped with at least one picture, and the content of each picture is free from picture distortion, so that the combined plane rectangular video picture is undistorted, and the picture is guaranteed to be undistorted.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a panoramic video mapping method provided by the invention;
FIG. 2 is a schematic diagram of mapping of a triangular prism-like model according to the present invention;
FIG. 3 is a schematic diagram showing the combination of the images of the triangular prism-like models according to the present invention;
FIG. 4 is a second schematic diagram of a triangular prism-like model according to the present invention;
FIG. 5 is a third schematic diagram of a triangular prism-like model according to the present invention;
fig. 6 is a schematic structural diagram of a panoramic video mapping device provided by the present invention;
fig. 7 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
With the rapid development of communication technology, namely with the popularization of kilomega-to-home and 5G (5 th generation mobile networks, fifth generation mobile communication network) mobile networks, and the wide application of a large number of high-performance STBs (Set Top boxes) and mobile terminal devices, the experience requirements of people on video live broadcast are continuously improved. Compared with the traditional common 2D video live broadcast, the virtual reality video with higher resolution and immersive panoramic interaction becomes a new direction of video live broadcast technology development. In the live video broadcast process, real-time monitoring is a core requirement. Currently, virtual reality video is mainly spherical panoramic video, and thus, it is necessary to present spherical panoramic video to a flat panel display.
At present, spherical panoramic video is mostly projected to a plane through a mapping mode such as longitude and latitude map mapping, hexahedron mapping and the like, so that panoramic video mapping is completed. However, in the mapping process, there may be a case where the content is lost, so that a complete picture cannot be comprehensively presented; and mapping all the content of the spherical panoramic video into a smaller plane will result in picture distortion. Therefore, how to completely present all the content of the spherical panoramic video and ensure that the picture is not distorted is a problem that needs to be solved at present.
For example, in the prior art, spherical panoramic video is acquired; mapping the panoramic video onto a plane by adopting pseudo-cylindrical projection to obtain a panoramic image; cutting the panoramic image along the weft by adopting a belt segmentation method; dividing the segmented panoramic image into a non-central area and a central area, splitting the non-central area, rearranging and combining the non-central area into a similar central area, and combining the central area and the similar central area into a mapping image of the panoramic video. However, with the projection method based on band division, the method has problems of data loss or discontinuous final spliced pictures in upper and lower areas (generally, 45 degrees each) after mapping. In addition, on the basis of segmentation, a special mapping model is adopted to map all contents of the spherical panoramic video in a smaller plane, but the problem of local picture distortion or inversion with a real picture occurs. In live video monitoring, displaying full pictures in a controllable range and picture undistorted is a strong requirement, so the prior art does not meet the live panoramic video monitoring.
In view of the above problems, the present invention proposes the following embodiments. Fig. 1 is a schematic flow chart of a panoramic video mapping method provided by the present invention, as shown in fig. 1, the panoramic video mapping method includes:
Step 110, acquiring spherical panoramic video.
Here, the spherical panoramic video may be constructed as one 3D scene. The spherical panoramic video can be acquired by the execution main body of the method provided by the embodiment of the invention, or can be acquired by other acquisition equipment, and then the spherical panoramic video is sent to the execution main body by other acquisition equipment.
And 120, mapping the spherical panoramic video to a triangular prism-like model to obtain a plurality of pictures, wherein the pictures comprise three horizontal pictures corresponding to three sides of the triangular prism-like model, a first bottom picture corresponding to a first bottom surface of the triangular prism-like model and a second bottom picture corresponding to a second bottom surface of the triangular prism-like model.
Here, the triangular prism-like model may include, but is not limited to: the embodiment of the present invention is not limited to a straight prism, a regular prism, an oblique prism, or the like.
Here, since the triangular prism-like model includes 5 faces, the plurality of pictures includes at least 5 pictures, that is, any one face of the triangular prism-like model corresponds to at least one picture.
Here, the number of the first bottom frames may be one, two or more. The number of second bottom end pictures may be one, two or more.
Specifically, the spherical panoramic video is mapped to three sides of the triangular prism-like model respectively, namely, the spherical panoramic video is mapped to three sides of the horizontal position of the triangular prism-like model respectively, and three horizontal pictures corresponding to the three sides of the triangular prism-like model are obtained; mapping the spherical panoramic video to the first bottom surface of the triangular prism-like model, namely mapping the spherical panoramic video to the bottom position of the triangular prism-like model, and obtaining a first bottom picture corresponding to the first bottom surface of the triangular prism-like model; and mapping the spherical panoramic video to the second bottom surface of the similar triangular prism model, namely mapping the spherical panoramic video to the other bottom position of the similar triangular prism model, and obtaining a second bottom picture corresponding to the second bottom surface of the similar triangular prism model.
In a specific embodiment, three horizontal pictures are mapped to the directions of three sides of the similar triangular prism model; mapping to the direction of the first bottom surface of the triangular prism-like model to obtain a first bottom end picture; and mapping the second bottom end picture to the direction of the second bottom surface of the triangular prism-like model.
In an embodiment, the number of the first bottom frames is 3, so that all video frames on the first bottom surface are covered, the integrity of the video frames is ensured, and the content is prevented from being lost.
In an embodiment, the number of the second bottom end pictures is 3, so that all video pictures on the second bottom surface are covered, the integrity of the video pictures is ensured, and the content is prevented from being lost.
In an embodiment, the plurality of first bottom end pictures can have common pictures, so that all video pictures on the first bottom surface are covered, the integrity of the video pictures is ensured, and the loss of content is avoided; and/or, the plurality of second bottom end pictures can have common pictures, so that all video pictures of the second bottom surface are covered, the integrity of the video pictures is ensured, and the content is prevented from being lost.
And step 130, combining the multiple pictures to obtain a planar rectangular video picture.
Here, the planar rectangular video picture is a video picture expanded after spherical panorama video mapping.
In a specific embodiment, determining a joint between any two pictures based on an expanded view of the triangular prism-like model; and combining the multiple pictures based on the joint between any two pictures to obtain a planar rectangular video picture. For example, the triangular prism-like model includes a first side, a second side, and a third side, and if the edges along the intersection of the first side and the third side are spread, one side edge of the second side is a joint between the horizontal frame corresponding to the second side and the horizontal frame corresponding to the first side, and the other side edge of the second side is a joint between the horizontal frame corresponding to the second side and the horizontal frame corresponding to the third side; one bottom edge of the first side surface is a joint between the first bottom end picture and the horizontal picture corresponding to the first side surface, and the other bottom edge of the first side surface is a joint between the second bottom end picture and the horizontal picture corresponding to the first side surface; one bottom edge of the second side surface is a joint between the first bottom end picture and the horizontal picture corresponding to the second side surface, and the other bottom edge of the second side surface is a joint between the second bottom end picture and the horizontal picture corresponding to the second side surface; one bottom edge of the third side surface is a joint between the first bottom end picture and the horizontal picture corresponding to the third side surface, and the other bottom edge of the third side surface is a joint between the second bottom end picture and the horizontal picture corresponding to the third side surface. Based on the above, the joint of each frame combined into the plane rectangular video frame is formed by unfolding the similar triangular prism, and the joint is continuous, so that the plane rectangular video frame is continuous, and further the frame distortion is avoided.
In an embodiment, the number of the first bottom frames is 3, the number of the second bottom frames is 3, the triangular prism-like model includes a first side, a second side and a third side, if the edges connected along the first side and the third side are unfolded, one bottom edge of the first side is a joint between the first bottom frame and a horizontal frame corresponding to the first side, and the other bottom edge of the first side is a joint between the first second bottom frame and a horizontal frame corresponding to the first side; one bottom edge of the second side surface is a joint between the second first bottom end picture and the horizontal picture corresponding to the second side surface, and the other bottom edge of the second side surface is a joint between the second bottom end picture and the horizontal picture corresponding to the second side surface; one bottom edge of the third side surface is a joint between the third first bottom end picture and the horizontal picture corresponding to the third side surface, and the other bottom edge of the third side surface is a joint between the third second bottom end picture and the horizontal picture corresponding to the third side surface. Based on the method, 9 frames can be combined into a plane rectangular video frame, so that all video frames after the spherical panoramic video is unfolded are covered, the integrity of the video frames is ensured, the loss of content is avoided, and the complete frames are comprehensively presented.
According to the panoramic video mapping method provided by the embodiment of the invention, the spherical panoramic video is obtained, the spherical panoramic video is mapped to the triangular prism-like model, a plurality of pictures are obtained, namely, all the contents of the spherical panoramic video are mapped to the triangular prism-like model, so that all the video pictures after the spherical panoramic video is unfolded are covered, the integrity of the video pictures is ensured, the content is prevented from being lost, and then the pictures are combined to obtain a planar rectangular video picture, and all the contents of the spherical panoramic video are completely presented; the plurality of pictures comprise three horizontal pictures corresponding to three side faces of the triangular prism-like model, a first bottom picture corresponding to a first bottom face of the triangular prism-like model and a second bottom picture corresponding to a second bottom face of the triangular prism-like model, namely, each face of the triangular prism-like model is mapped with at least one picture, and the content of each picture is free from picture distortion, so that the combined plane rectangular video picture is undistorted, and the picture is guaranteed to be undistorted.
Based on the above embodiment, in the method, the step 120 includes:
mapping the spherical panoramic video to a triangular prism-like model based on a plurality of virtual cameras to obtain a plurality of pictures;
The plurality of virtual cameras are arranged in the central area of the triangular prism-like model, the plurality of virtual cameras comprise three horizontal cameras, a first bottom camera and a second bottom camera, the three horizontal cameras are used for shooting and acquiring three horizontal pictures in the directions of the three side faces respectively, the first bottom camera is used for shooting and acquiring the first bottom picture in the direction of the first bottom face, and the second bottom camera is used for shooting and acquiring the second bottom picture in the direction of the second bottom face.
Specifically, three horizontal pictures are obtained by shooting in the directions of three sides of the triangular prism-like model through three horizontal cameras respectively; shooting towards the direction of the first bottom surface of the triangular prism-like model through a first bottom end camera to obtain a first bottom end picture; and shooting the second bottom end image to the direction of the second bottom surface of the triangular prism-like model through a second bottom end camera to obtain a second bottom end image.
The plurality of virtual cameras are arranged in the center area of the triangular prism-like model, so that the spherical panoramic video can be mapped to the triangular prism-like model. Since the triangular prism-like model includes 5 faces, at least 5 pictures can be mapped, one virtual camera can take a picture, and one virtual camera can generate one viewing angle.
Here, the number of the first bottom-end cameras may be 1 or more. The number of second bottom end cameras may be 1 or more.
In some embodiments, the number of the first bottom cameras is 3, and one first bottom camera shoots and obtains one first bottom picture, so the number of the first bottom pictures is 3, thereby covering all video pictures on the first bottom surface, ensuring that the video pictures are complete and avoiding losing content.
The sum of the horizontal view angles of the 3 first bottom cameras is greater than or equal to 360 degrees, so that the view angles of the 3 first bottom cameras cover all video pictures on the first bottom surface.
In one embodiment, the sum of the horizontal field of view angles of the 3 first bottom cameras is equal to 360 degrees. The horizontal field angles of the 3 first bottom cameras may be 120 degrees, respectively; the distribution can be performed according to actual demands, and the sum of the horizontal view field angles of the 3 first bottom cameras is only required to be 360 degrees, for example, the horizontal view field angle of the first bottom camera is 110 degrees, the horizontal view field angle of the second first bottom camera is 120 degrees, and the horizontal view field angle of the third first bottom camera is 130 degrees.
In some embodiments, the number of the second bottom cameras is 3, and one second bottom camera shoots and obtains one second bottom picture, so the number of the second bottom pictures is 3, thereby covering all video pictures on the second bottom surface, ensuring the integrity of the video pictures and avoiding losing content.
The sum of the horizontal view angles of the 3 second bottom cameras is greater than or equal to 360 degrees, so that the view angles of the 3 second bottom cameras cover all video pictures on the second bottom surface.
In an embodiment, the sum of the horizontal field angles of 3 second bottom cameras is equal to 360 degrees. The horizontal field angles of the 3 second bottom cameras may be 120 degrees, respectively; the distribution can be performed according to actual requirements, and the sum of the horizontal view field angles of the 3 second bottom cameras is only required to be 360 degrees, for example, the horizontal view field angle of the first second bottom camera is 110 degrees, the horizontal view field angle of the second bottom camera is 120 degrees, and the horizontal view field angle of the third second bottom camera is 130 degrees.
In some embodiments, the sum of the horizontal view angles of the three horizontal cameras is greater than or equal to 360 degrees, so that the view angles of the three horizontal cameras cover all video pictures on three sides, the integrity of the video pictures is ensured, and the content is prevented from being lost.
In one embodiment, the sum of the horizontal field angles of the three horizontal cameras is equal to 360 degrees. The horizontal view field angles of the three horizontal cameras can be 120 degrees respectively; the distribution can be carried out according to actual demands, and the sum of the horizontal view field angles of the three horizontal cameras is only required to be 360 degrees, for example, the horizontal view field angle of the first horizontal camera is 110 degrees, the horizontal view field angle of the second horizontal camera is 120 degrees, and the horizontal view field angle of the third horizontal camera is 130 degrees.
In some embodiments, the sum of the vertical field angle of the horizontal camera, the vertical field angle of the first bottom camera, and the vertical field angle of the second bottom camera is greater than or equal to 180 degrees, so that the viewing angle of the virtual camera covers all video pictures at the vertical position, ensuring that the video pictures are complete and avoiding losing content.
In an embodiment, the sum of the vertical field angle of the horizontal camera, the vertical field angle of the first bottom camera, and the vertical field angle of the second bottom camera is equal to 180 degrees. For example, the vertical field angle of the horizontal camera is 120 degrees, the vertical field angle of the first bottom camera is 30 degrees, and the vertical field angle of the second bottom camera is 30 degrees; alternatively, the vertical field angle of the horizontal camera is 100 degrees, the vertical field angle of the first bottom camera is 40 degrees, and the vertical field angle of the second bottom camera is 40 degrees.
In an embodiment, the three horizontal cameras are main cameras, based on which the vertical field angle of the horizontal cameras is larger, i.e. the vertical field angle of the horizontal camera is larger than the vertical field angle of the first bottom camera, and the vertical field angle of the horizontal camera is larger than the vertical field angle of the second bottom camera. For example, referring to fig. 3, three horizontal pictures 1, 2, 3 are larger than the first bottom pictures 4, 5, 6 and larger than the second bottom pictures 7, 8, 9, which are not equally large due to the different vertical field angles.
According to the panoramic video mapping method provided by the embodiment of the invention, the plurality of virtual cameras are arranged in the central area of the triangular prism-like model, so that support is provided for mapping the spherical panoramic video to the triangular prism-like model, and therefore, the whole content of the spherical panoramic video is mapped to the triangular prism-like model, all video pictures after the spherical panoramic video is unfolded are covered, the integrity of the video pictures is ensured, and the content is prevented from being lost; and the plurality of virtual cameras comprise three horizontal cameras for shooting three horizontal pictures in the directions of three side surfaces, a first bottom camera for shooting the direction of the first bottom surface to obtain a first bottom picture, and a second bottom camera for shooting the direction of the second bottom surface to obtain a second bottom picture, namely, each surface of the triangular prism-like model is shot with at least one picture, and the content of each picture is free from picture distortion, so that the combined plane rectangular video picture is free from distortion, and the picture is free from distortion.
Based on any of the above embodiments, the edge positions of the planar rectangular video frames obtained after the merging are considered to be severely deformed. Based on the above, the number of the first bottom cameras is a plurality, and the number of the second bottom cameras is a plurality; the number of the first bottom end pictures is a plurality of the second bottom end pictures.
In some embodiments, the sum of the horizontal field of view angles of the plurality of first bottom cameras is greater than or equal to 360 degrees such that the view angles of the plurality of first bottom cameras cover all of the video frames of the first bottom surface.
In an embodiment, the sum of the horizontal field of view angles of the plurality of first bottom cameras is equal to 360 degrees. The horizontal field angle of view of the plurality of first bottom cameras may be 360 degrees/N, N being the number of first bottom cameras; the distribution can be carried out according to actual demands, and the sum of the horizontal view field angles of the plurality of first bottom cameras is only required to be 360 degrees.
In some embodiments, the sum of the horizontal field of view angles of the plurality of second bottom cameras is greater than or equal to 360 degrees such that the view angles of the plurality of second bottom cameras cover all of the video frames of the second bottom surface.
In an embodiment, the sum of the horizontal field of view angles of the plurality of second bottom end cameras is equal to 360 degrees. The horizontal field angle of view of the plurality of second bottom cameras may be 360 degrees/N, N being the number of second bottom cameras; the distribution can be carried out according to actual demands, and the sum of the horizontal view field angles of the second bottom cameras is only required to be 360 degrees.
According to the panoramic video mapping method provided by the embodiment of the invention, the number of the first bottom cameras is multiple, the number of the second bottom cameras is multiple, so that the number of the first bottom pictures obtained by shooting is multiple, the number of the second bottom pictures is multiple, the serious deformation of the edge positions of the planar rectangular video pictures obtained after combination is avoided, all the video pictures after the spherical panoramic video is unfolded are covered, the integrity of the video pictures is ensured, and the content is prevented from being lost; the number of the first bottom cameras is in one-to-one correspondence with the number of the first bottom pictures, and the number of the second bottom cameras is in one-to-one correspondence with the number of the second bottom pictures, namely, each picture is photographed by a virtual camera alone, so that picture distortion does not exist in the content of each picture, and the combined plane rectangular video pictures are not distorted, so that the picture is ensured to be undistorted.
Based on any one of the above embodiments, in the method, the number of the first bottom cameras is 3, the number of the second bottom cameras is 3, the number of the first bottom pictures is 3, and the number of the second bottom pictures is 3; the horizontal view field angle of the first bottom camera is 120 degrees, and the horizontal view field angle of the second bottom camera is 120 degrees.
In one embodiment, the first bottom frame is a planar rectangular frame. Further, there is a common picture among the plurality of first bottom-end pictures, i.e., there is redundancy in the viewing angles of the three first bottom-end cameras.
In one embodiment, the second bottom frame is a planar rectangular frame. Further, there is a common picture for the plurality of second bottom-end pictures, i.e., there is redundancy in the viewing angles of the three second bottom-end cameras.
For easy understanding, referring to fig. 2, three first bottom end frames are mapped on a first bottom surface (upper bottom surface) of the triangular prism-like model, and the three first bottom end frames are all plane rectangular frames, and there is a common frame for the three first bottom end frames, wherein each first bottom end frame has an edge which coincides with an edge of the first bottom surface; and three second bottom end pictures are mapped on the second bottom surface (lower bottom surface) of the triangular prism-like model, are plane rectangular pictures, and have common pictures, wherein each second bottom end picture has an edge which coincides with the edge of the first bottom surface.
According to the panoramic video mapping method provided by the embodiment of the invention, the number of the first bottom cameras is 3, the number of the second bottom cameras is 3, so that the number of the first bottom pictures obtained by shooting is 3, and the number of the second bottom pictures is 3, and the first bottom pictures are matched with three horizontal pictures one by one, so that serious deformation of the edge positions of the planar rectangular video pictures obtained after combination is avoided, all video pictures after spherical panoramic video expansion are covered, the integrity of the video pictures is ensured, and the content loss is avoided; the number of the first bottom cameras is in one-to-one correspondence with the number of the first bottom pictures, and the number of the second bottom cameras is in one-to-one correspondence with the number of the second bottom pictures, namely, each picture is photographed by a virtual camera alone, so that picture distortion does not exist in the content of each picture, and the combined plane rectangular video pictures are not distorted, so that the picture is ensured not to be distorted; and the horizontal view field angle of the first bottom camera is 120 degrees, and the horizontal view field angle of the second bottom camera is 120 degrees, so that the sum of the horizontal view field angles of the three first bottom cameras is equal to 360 degrees, all video pictures of the first bottom surface are covered, the integrity of the video pictures is ensured, the loss of content is avoided, redundancy is avoided, invalid information is reduced, the sum of the horizontal view field angles of the three second bottom cameras is equal to 360 degrees, all video pictures of the second bottom surface are covered, the integrity of the video pictures is ensured, the loss of content is avoided, redundancy is avoided, and invalid information is reduced.
Based on any one of the above embodiments, in the method, the first bottom frame is a planar rectangular frame, and the second bottom frame is a planar rectangular frame; the plurality of first bottom end pictures have common pictures, and the plurality of second bottom end pictures have common pictures, so that the edge positions of the planar rectangular video pictures obtained after combination are prevented from being seriously deformed, all the video pictures after the first bottom surface and the second bottom surface are unfolded are covered, and the completeness of the video pictures is ensured.
Based on any embodiment, in the method, the horizontal view field angle of the horizontal camera is 120 degrees, so that the sum of the horizontal view field angles of the three horizontal cameras is equal to 360 degrees, all video pictures at the horizontal position are covered, the integrity of the video pictures is ensured, the loss of content is avoided, redundancy is avoided, and invalid information is reduced.
Based on any of the above embodiments, in the method, a vertical field angle of the horizontal camera is 120 degrees; the vertical view field angle of the first bottom camera is 30 degrees, and the vertical view field angle of the second bottom camera is 30 degrees, so that the sum of the vertical view field angle of the horizontal camera, the vertical view field angle of the first bottom camera and the vertical view field angle of the second bottom camera is 360 degrees, all video pictures at the vertical position are covered, the integrity of the video pictures is ensured, the content is prevented from being lost, redundancy is avoided, and invalid information is reduced.
Based on any of the above embodiments, the method in step 130 includes:
determining the joint of the pictures based on the edges of the triangular prism-like model;
and merging the multiple pictures based on the joint to obtain a planar rectangular video picture.
Since the plurality of images each have an edge overlapping with the edge of the triangular prism-like model, one overlapping edge exists on either side of the triangular prism-like model. Based on the edge of the triangular prism-like model, the joint of a plurality of pictures can be determined, and then the pictures are combined to obtain the planar rectangular video picture based on the joint of the pictures.
Specifically, determining a joint between any two pictures based on edges of the triangular prism-like model; and combining the multiple pictures based on the joint between any two pictures to obtain a planar rectangular video picture.
In an embodiment, referring to fig. 3, the number of the first bottom frames is 3, the number of the second bottom frames is 3, the triangular prism-like model includes a first side, a second side, and a third side, if the edges along the intersection of the first side and the third side are unfolded, one bottom edge of the first side is a joint between the first bottom frame 4 and the horizontal frame 1 corresponding to the first side, and the other bottom edge of the first side is a joint between the first second bottom frame 7 and the horizontal frame 1 corresponding to the first side; one bottom edge of the second side is the joint between the second first bottom end picture 5 and the horizontal picture 2 corresponding to the second side, and the other bottom edge of the second side is the joint between the second bottom end picture 8 and the horizontal picture 2 corresponding to the second side; one bottom edge of the third side is the joint between the third first bottom end picture 6 and the horizontal picture 3 corresponding to the third side, and the other bottom edge of the third side is the joint between the third second bottom end picture 9 and the horizontal picture 3 corresponding to the third side. Based on the method, 9 frames can be combined into a plane rectangular video frame, so that all video frames after the spherical panoramic video is unfolded are covered, the integrity of the video frames is ensured, the loss of content is avoided, and the complete frames are comprehensively presented.
According to the panoramic video mapping method provided by the embodiment of the invention, the joint of each picture combined into the planar rectangular video picture is determined based on the edge of the triangular prism-like model, so that the joint is continuous, the planar rectangular video picture is continuous, and further picture distortion is avoided; and each picture combined into the plane rectangular video picture is mapped respectively, so that the content of each picture is complete, and the content of each picture has no picture distortion, so that the combined plane rectangular video picture is not distorted, and the picture is ensured not to be distorted; the combined multiple frames are frames obtained by panoramic mapping video mapping, and the combined plane rectangular video frames are all video frames and do not contain invalid information, so that the invalid information is reduced; and the aspect ratio of the combined planar rectangular video picture is suitable for display on a mainstream flat panel display.
In order to facilitate understanding of the embodiments described above, a specific embodiment is described herein as an example. Referring to fig. 4 and 5, a plurality of virtual cameras are disposed in a central region of a triangular prism-like model, a horizontal view angle of a horizontal camera is 120 degrees, a vertical view angle of the horizontal camera is 120 degrees, and a horizontal picture is a planar rectangular picture; the number of the first bottom cameras is 3, the number of the second bottom cameras is 3, the number of the first bottom pictures is 3, and the number of the second bottom pictures is 3; the horizontal view field angle of the first bottom camera is 120 degrees, the vertical view field angle of the first bottom camera is 30 degrees, the horizontal view field angle of the second bottom camera is 120 degrees, the vertical view field angle of the second bottom camera is 30 degrees, the first bottom picture is a plane rectangular picture, and the second bottom picture is a plane rectangular picture. It can be understood that the spherical panoramic video is mapped to the mitsubishi-like model to obtain 9 pictures, the 9 pictures are spliced and combined and displayed on a plane, the content of the video picture is not lost, each picture is not distorted, invalid information is not contained, and the requirement of live broadcast real-time monitoring of the spherical panoramic video is met.
The panoramic video mapping apparatus provided by the invention is described below, and the panoramic video mapping apparatus described below and the panoramic video mapping method described above can be referred to correspondingly.
Fig. 6 is a schematic structural diagram of a panoramic video mapping device according to the present invention, as shown in fig. 6, the panoramic video mapping device includes:
an acquisition module 610, configured to acquire a spherical panoramic video;
the mapping module 620 is configured to map the spherical panoramic video to a triangular prism-like model, and obtain a plurality of pictures, where the plurality of pictures include three horizontal pictures corresponding to three sides of the triangular prism-like model, a first bottom picture corresponding to a first bottom surface of the triangular prism-like model, and a second bottom picture corresponding to a second bottom surface of the triangular prism-like model;
and the merging module 630 is configured to merge the multiple frames to obtain a planar rectangular video frame.
According to the panoramic video mapping device provided by the embodiment of the invention, the spherical panoramic video is obtained, the spherical panoramic video is mapped to the triangular prism-like model, a plurality of pictures are obtained, namely, all the contents of the spherical panoramic video are mapped to the triangular prism-like model, so that all the video pictures after the spherical panoramic video is unfolded are covered, the integrity of the video pictures is ensured, the content is prevented from being lost, and then the pictures are combined to obtain a planar rectangular video picture, and all the contents of the spherical panoramic video are completely presented; the plurality of pictures comprise three horizontal pictures corresponding to three side faces of the triangular prism-like model, a first bottom picture corresponding to a first bottom face of the triangular prism-like model and a second bottom picture corresponding to a second bottom face of the triangular prism-like model, namely, each face of the triangular prism-like model is mapped with at least one picture, and the content of each picture is free from picture distortion, so that the combined plane rectangular video picture is undistorted, and the picture is guaranteed to be undistorted.
Based on any of the above embodiments, the mapping module 620 includes:
the mapping unit is used for mapping the spherical panoramic video to a triangular prism-like model based on a plurality of virtual cameras to obtain a plurality of pictures;
the plurality of virtual cameras are arranged in the central area of the triangular prism-like model, the plurality of virtual cameras comprise three horizontal cameras, a first bottom camera and a second bottom camera, the three horizontal cameras are used for shooting and acquiring three horizontal pictures in the directions of the three side faces respectively, the first bottom camera is used for shooting and acquiring the first bottom picture in the direction of the first bottom face, and the second bottom camera is used for shooting and acquiring the second bottom picture in the direction of the second bottom face.
Based on any of the above embodiments, the number of the first bottom-end cameras is plural, and the number of the second bottom-end cameras is plural;
the number of the first bottom end pictures is a plurality of the second bottom end pictures.
Based on any of the above embodiments, the number of the first bottom cameras is 3, the number of the second bottom cameras is 3, the number of the first bottom pictures is 3, and the number of the second bottom pictures is 3;
The horizontal view field angle of the first bottom camera is 120 degrees, and the horizontal view field angle of the second bottom camera is 120 degrees.
Based on any one of the above embodiments, the first bottom frame is a planar rectangular frame, and the second bottom frame is a planar rectangular frame;
and the first bottom end pictures have a common picture, and the second bottom end pictures have a common picture.
Based on any of the above embodiments, the horizontal camera may have a horizontal field angle of 120 degrees.
Based on any of the above embodiments, the vertical field angle of the horizontal camera is 120 degrees;
the vertical view field angle of the first bottom camera is 30 degrees, and the vertical view field angle of the second bottom camera is 30 degrees.
Based on any of the above embodiments, the merging module 630 includes:
a joint determining unit, configured to determine joints of the multiple pictures based on edges of the triangular prism-like model;
and the picture merging unit is used for merging the pictures based on the joint to obtain a plane rectangular video picture.
Fig. 7 illustrates a physical schematic diagram of an electronic device, as shown in fig. 7, which may include: processor 710, communication interface (Communications Interface) 720, memory 730, and communication bus 740, wherein processor 710, communication interface 720, memory 730 communicate with each other via communication bus 740. Processor 710 may invoke logic instructions in memory 730 to perform a panoramic video mapping method comprising: acquiring a spherical panoramic video; mapping the spherical panoramic video to a triangular prism-like model to obtain a plurality of pictures, wherein the pictures comprise three horizontal pictures corresponding to three sides of the triangular prism-like model, a first bottom picture corresponding to a first bottom surface of the triangular prism-like model and a second bottom picture corresponding to a second bottom surface of the triangular prism-like model; and combining the multiple pictures to obtain a planar rectangular video picture.
Further, the logic instructions in the memory 730 described above may be implemented in the form of software functional units and may be stored in a computer readable storage medium when sold or used as a stand alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being storable on a non-transitory computer readable storage medium, the computer program, when executed by a processor, being capable of performing the panoramic video mapping method provided by the above methods, the method comprising: acquiring a spherical panoramic video; mapping the spherical panoramic video to a triangular prism-like model to obtain a plurality of pictures, wherein the pictures comprise three horizontal pictures corresponding to three sides of the triangular prism-like model, a first bottom picture corresponding to a first bottom surface of the triangular prism-like model and a second bottom picture corresponding to a second bottom surface of the triangular prism-like model; and combining the multiple pictures to obtain a planar rectangular video picture.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the panoramic video mapping method provided by the above methods, the method comprising: acquiring a spherical panoramic video; mapping the spherical panoramic video to a triangular prism-like model to obtain a plurality of pictures, wherein the pictures comprise three horizontal pictures corresponding to three sides of the triangular prism-like model, a first bottom picture corresponding to a first bottom surface of the triangular prism-like model and a second bottom picture corresponding to a second bottom surface of the triangular prism-like model; and combining the multiple pictures to obtain a planar rectangular video picture.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (11)
1. A panoramic video mapping method, comprising:
acquiring a spherical panoramic video;
mapping the spherical panoramic video to a triangular prism-like model to obtain a plurality of pictures, wherein the pictures comprise three horizontal pictures corresponding to three sides of the triangular prism-like model, a first bottom picture corresponding to a first bottom surface of the triangular prism-like model and a second bottom picture corresponding to a second bottom surface of the triangular prism-like model;
and combining the multiple pictures to obtain a planar rectangular video picture.
2. The panoramic video mapping method according to claim 1, wherein said mapping said spherical panoramic video to a triangular prism-like model to obtain a plurality of pictures comprises:
mapping the spherical panoramic video to a triangular prism-like model based on a plurality of virtual cameras to obtain a plurality of pictures;
the plurality of virtual cameras are arranged in the central area of the triangular prism-like model, the plurality of virtual cameras comprise three horizontal cameras, a first bottom camera and a second bottom camera, the three horizontal cameras are used for shooting and acquiring three horizontal pictures in the directions of the three side faces respectively, the first bottom camera is used for shooting and acquiring the first bottom picture in the direction of the first bottom face, and the second bottom camera is used for shooting and acquiring the second bottom picture in the direction of the second bottom face.
3. The panoramic video mapping method of claim 2, wherein the number of first bottom cameras is a plurality and the number of second bottom cameras is a plurality;
the number of the first bottom end pictures is a plurality of the second bottom end pictures.
4. The panoramic video mapping method of claim 3, wherein the number of first bottom cameras is 3, the number of second bottom cameras is 3, the number of first bottom pictures is 3, and the number of second bottom pictures is 3;
the horizontal view field angle of the first bottom camera is 120 degrees, and the horizontal view field angle of the second bottom camera is 120 degrees.
5. The panoramic video mapping method of claim 3, wherein said first bottom frame is a planar rectangular frame and said second bottom frame is a planar rectangular frame;
and the first bottom end pictures have a common picture, and the second bottom end pictures have a common picture.
6. The panoramic video mapping method of claim 2, wherein the horizontal camera has a horizontal field angle of 120 degrees.
7. The panoramic video mapping method of claim 2, wherein the vertical field of view angle of the horizontal camera is 120 degrees;
The vertical view field angle of the first bottom camera is 30 degrees, and the vertical view field angle of the second bottom camera is 30 degrees.
8. The panoramic video mapping method according to any one of claims 1-7, wherein said merging said plurality of pictures to obtain a planar rectangular video picture comprises:
determining the joint of the pictures based on the edges of the triangular prism-like model;
and merging the multiple pictures based on the joint to obtain a planar rectangular video picture.
9. A panoramic video mapping apparatus, comprising:
the acquisition module is used for acquiring spherical panoramic video;
the mapping module is used for mapping the spherical panoramic video to a triangular prism-like model to obtain a plurality of pictures, wherein the pictures comprise three horizontal pictures corresponding to three sides of the triangular prism-like model, a first bottom picture corresponding to a first bottom surface of the triangular prism-like model and a second bottom picture corresponding to a second bottom surface of the triangular prism-like model;
and the merging module is used for merging the plurality of pictures to obtain a plane rectangular video picture.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the panoramic video mapping method of any one of claims 1 to 8 when the program is executed by the processor.
11. A non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor implements the panoramic video mapping method of any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211085497.2A CN116957913A (en) | 2022-09-06 | 2022-09-06 | Panoramic video mapping method, device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211085497.2A CN116957913A (en) | 2022-09-06 | 2022-09-06 | Panoramic video mapping method, device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116957913A true CN116957913A (en) | 2023-10-27 |
Family
ID=88444962
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211085497.2A Pending CN116957913A (en) | 2022-09-06 | 2022-09-06 | Panoramic video mapping method, device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116957913A (en) |
-
2022
- 2022-09-06 CN CN202211085497.2A patent/CN116957913A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11055901B2 (en) | Method, apparatus, medium, and server for generating multi-angle free-perspective video data | |
| US11330172B2 (en) | Panoramic image generating method and apparatus | |
| CN109547766B (en) | Panoramic image generation method and device | |
| US20200213620A1 (en) | Image splicing method and apparatus, and storage medium | |
| CN101689292B (en) | Banana codec | |
| CN108648257B (en) | Panoramic picture acquisition method and device, storage medium and electronic device | |
| CN106604003B (en) | Method and system for realizing curved-surface curtain projection through short-focus projection | |
| CN112351266B (en) | Three-dimensional visual processing method, device, equipment, display system and medium | |
| WO2023207963A1 (en) | Image processing method and apparatus, electronic device, and storage medium | |
| CN108765582B (en) | Panoramic picture display method and device | |
| WO2024002023A1 (en) | Method and apparatus for generating panoramic stereoscopic image, and electronic device | |
| CN111731190B (en) | Display method and system of vehicle-mounted AVM, electronic equipment and storage medium | |
| CN116957913A (en) | Panoramic video mapping method, device, electronic equipment and storage medium | |
| CN114513646B (en) | Method and device for generating panoramic video in three-dimensional virtual scene | |
| CN113873259A (en) | Panoramic video frame insertion method and device and computer readable storage medium | |
| CN111988650B (en) | Video playing method, device, equipment and storage medium of three-dimensional screen | |
| CN117560578B (en) | Multi-channel video fusion method and system based on three-dimensional scene rendering and irrelevant to view points | |
| CN115002442B (en) | Image display method and device, electronic equipment and storage medium | |
| CN116860112B (en) | Combined scene experience generation method, system and medium based on XR technology | |
| CN111698513A (en) | Image acquisition method, display method, device, electronic equipment and storage medium | |
| CN117830085A (en) | Video conversion method and device | |
| CN117459694A (en) | Image generation method, device, electronic equipment and storage medium | |
| CN116962652A (en) | XR virtual shooting method and device and XR media server | |
| CN116016959A (en) | Video stream processing method, device, server and storage medium | |
| CN116405782A (en) | Panoramic image generation method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |