Method and device for recording panoramic video
Technical Field
The embodiment of the invention relates to the technical field of video recording, in particular to a method and a device for recording a panoramic video.
Background
The traditional video recording and playing system can realize the recording and playing of the video only by the steps of video acquisition, manufacturing, playing and the like, and thus, a long time is needed. And the traditional video recording and playing equipment records the video in one direction, and cannot acquire the videos in other directions at the same time, if the videos in other directions are to be acquired, the camera equipment needs to be rotated, or a plurality of cameras are adopted to record simultaneously, and the videos in all directions shot by the camera equipment are spliced by a computer in the later stage. At present, some panoramic shooting devices with multiple cameras are too large in size and not suitable for being carried about, and most videos shot by the panoramic shooting devices can only be checked in a later period, so that applications such as real-time sharing and live broadcasting cannot be realized.
At present, a plurality of cameras are mostly adopted for shooting videos in all directions, and then the videos are spliced by an upper computer. The disadvantages are as follows: the processing of the upper computer is needed, and the processing time is long. The existing panoramic shooting equipment integrating a plurality of cameras on single equipment is large in size, is not suitable for carrying, and cannot realize real-time sharing and live broadcast application.
Disclosure of Invention
The embodiment of the invention aims to provide a method and a device for recording a panoramic video, and aims to solve the problems that the traditional panoramic video needs to be shot by a plurality of cameras and needs to be subjected to post-processing by a computer, and the existing multi-camera panoramic shooting equipment is large in size, not suitable for being carried about, incapable of live broadcasting and sharing in real time and the like.
To achieve the purpose, the embodiment of the invention adopts the following technical scheme:
a method of recording panoramic video, the method comprising:
acquiring 360-degree panoramic video data through N cameras, wherein N is more than or equal to 1, the horizontal angle of the video data shot by the cameras is a horizontal angle of 360 degrees/N, and the vertical angle of the video data shot by the cameras is a vertical angle of 360 degrees/N;
processing the video data acquired by each camera through a preset algorithm;
and sending the processed panoramic video data to a mobile terminal in a local storage mode or a wireless mode or a wired mode.
Preferably, the processing the video data acquired by each camera through a preset algorithm includes:
inputting the video data to an FPGA;
processing the video data through the FPGA, and outputting the processed video data to an APU, wherein the processing comprises distortion correction, projection and graphic alignment, or the processing comprises projection and graphic alignment, and the projection comprises cylindrical projection and spherical projection;
and H.264 coding the input panoramic video data through the APU.
Preferably, the inputting the video data to the FPGA includes:
transmitting the video data to the FPGA through a parallel YUV data interface; or,
the camera video is output through an MIPI data interface and is input into the FPGA through an MIPI-to-parallel YUV data format interface chip.
The processing the video data by the FPGA, outputting the processed video data to the APU, wherein the processing includes distortion correction, projection and graphic alignment, or the processing includes projection and graphic alignment, and the projection includes cylindrical projection and spherical projection, including:
receiving input video data in a YUV data format;
detecting the format of the video data through a signal format detection module, wherein the format comprises resolution, pixel clock, line frequency and field frequency information;
inputting the detected video data into a DDR memory unit, processing the video data, and inputting the processed video data into the APU, wherein the processing comprises distortion correction, projection and graphic alignment, or the processing comprises projection and graphic alignment, and the projection comprises cylindrical projection and spherical projection.
Preferably, the sending the processed panoramic video data to the mobile terminal in a local storage mode or a wireless mode or a wired mode includes:
and storing the processed video data into a local SDcard, or sending the processed video data to wireless routing equipment and/or intelligent equipment through WIFI (wireless fidelity), or outputting the processed video data to the intelligent equipment through HDMI (high-definition multimedia interface), or outputting the processed video data to a mobile terminal through USB (universal serial bus).
An apparatus for recording panoramic video, the apparatus comprising:
the device comprises an acquisition module, a display module and a control module, wherein the acquisition module is used for acquiring 360-degree panoramic video data through N cameras, N is more than or equal to 1, the horizontal angle of the video data shot by the cameras is a horizontal angle of 360 degrees/N, and the vertical angle of the video data shot by the cameras is a vertical angle of 360 degrees/N;
the processing module is used for processing the video data acquired by each camera through a preset algorithm;
and the sending module is used for sending the processed panoramic video data to the mobile terminal in a local storage mode or a wireless mode or a wired mode.
Preferably, the processing module comprises:
an input unit for inputting the video data to the FPGA;
the processing unit is used for processing the video data through the FPGA and outputting the processed video data to the APU, wherein the processing comprises distortion correction, projection and graphic alignment, or the processing comprises projection and graphic alignment, and the projection comprises cylindrical projection and spherical projection;
an encoding unit for H.264 encoding the input panoramic video data by the APU.
Preferably, the input unit is configured to:
transmitting the video data to the FPGA through a parallel YUV data interface; or,
the camera video is output through an MIPI data interface and is input into the FPGA through an MIPI-to-parallel YUV data format interface chip.
Preferably, the processing unit is configured to:
receiving input video data in a YUV data format;
detecting the format of the video data through a signal format detection module, wherein the format comprises resolution, pixel clock, line frequency and field frequency information;
inputting the detected video data into a DDR memory unit, processing the video data, and inputting the processed video data into the APU, wherein the processing comprises distortion correction, projection and graphic alignment, or the processing comprises projection and graphic alignment, and the projection comprises cylindrical projection and spherical projection.
Preferably, the sending module includes:
the storage unit is used for storing the processed panoramic video data into a local SDcard;
the wireless sending unit is used for sending the information to the wireless routing equipment and/or the intelligent equipment through WIFI;
the first wired sending unit is used for outputting the signals to the intelligent equipment through the HDMI;
and the second wired sending unit is used for outputting the data to the mobile terminal through the USB.
The embodiment of the invention obtains 360-degree panoramic video data through N cameras, wherein N is more than or equal to 1, the horizontal angle of the video data shot by the cameras is a horizontal angle of 360 degrees/N, and the vertical angle of the video data shot by the cameras is a vertical angle of 360 degrees/N; processing the video data acquired by each camera through a preset algorithm; and sending the processed panoramic video data to a mobile terminal in a local storage or wireless mode or wired mode, thereby realizing the purpose of conveniently recording and playing the 360-degree panoramic video.
Drawings
Fig. 1 is a schematic flowchart of a first embodiment of a method for recording a panoramic video according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a three-view handheld 360-panoramic video real-time recording and playing device according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of an apparatus for recording panoramic video data according to an embodiment of the present invention;
FIG. 4 is a functional block diagram of an apparatus for recording panoramic video according to an embodiment of the present invention;
FIG. 5 is a functional block diagram of a processing module 402 according to an embodiment of the present invention;
fig. 6 is a schematic diagram of functional modules of the sending module 403 according to an embodiment of the present invention.
Detailed Description
The embodiments of the present invention will be described in further detail with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of and not restrictive on the broad invention. It should be further noted that, for convenience of description, only some structures, not all structures, relating to the embodiments of the present invention are shown in the drawings.
Example one
Referring to fig. 1, fig. 1 is a schematic flowchart of a method for recording a panoramic video according to a first embodiment of the present invention.
In an embodiment, the method for recording the panoramic video includes:
101, acquiring 360-degree panoramic video data through N cameras, wherein N is more than or equal to 1, the horizontal angle of the video data shot by the cameras is a horizontal angle of 360 degrees/N, and the vertical angle of the video data shot by the cameras is a vertical angle of 360 degrees/N;
wherein, when N >6, the camera may be wide-angle. When N >8, the camera is a normal camera. When N < ═ 6, the camera needs to be fisheye.
Specifically, assuming that 3 fisheye cameras are used to acquire 360-degree panoramic video data, the invention adopts 3 fisheyes, the horizontal viewing angle is greater than 180 degrees, the vertical viewing angle is greater than 180 degrees, and the cameras, the FPGA, the APU and a plurality of peripheral chips are implemented as shown in fig. 2, and fig. 2 is a schematic structural diagram of a three-eye handheld 360-degree panoramic video real-time recording and playing device provided by an embodiment of the invention. The structure of the handheld 360-degree panoramic video real-time recording and playing system is shown in fig. 2, wherein 01, 02 and 03 in fig. 2 respectively represent different cameras, the handheld 360-degree panoramic video real-time recording and playing device generates a 360-degree video, and the 360-degree video is coded by H.264 and then is sent to a cloud end or an intelligent device end through wifi or is stored on a local SDcard. The system can realize that: 1) directly send the video live broadcast to the smart machine through wifi. 2) The video is sent to the cloud end through the wireless router, and cloud end data storage and sharing are achieved. The video of high in the clouds can be live at smart machine and web end. 3) And transmitting the panoramic video to a computer for storage and playing through the usb 3.0. 4) And transmitting the data to the intelligent equipment through the HDMI for playing. 5) And storing the coded video into a local SDcard.
102, processing video data acquired by each camera through a preset algorithm;
preferably, the processing the video data acquired by each camera through a preset algorithm includes:
inputting the video data to an FPGA;
processing the video data through the FPGA, and outputting the processed video data to an APU, wherein the processing comprises distortion correction, projection and graphic alignment, or the processing comprises projection and graphic alignment, and the projection comprises cylindrical projection and spherical projection;
and H.264 coding the input panoramic video data through the APU.
Preferably, the inputting the video data to the FPGA includes:
transmitting the video data to the FPGA through a parallel YUV data interface; or,
the camera video is output through an MIPI data interface and is input into the FPGA through an MIPI-to-parallel YUV data format interface chip.
Preferably, the processing the video data by the FPGA to output the processed video data to the APU includes distortion correction, projection and graphic alignment, or the processing includes projection and graphic alignment, and the projection includes cylindrical projection and spherical projection, and includes:
receiving input video data in a YUV data format;
detecting the format of the video data through a signal format detection module, wherein the format comprises resolution, pixel clock, line frequency and field frequency information;
inputting the detected video data into a DDR memory unit, processing the video data, and inputting the processed video data into the APU, wherein the processing comprises distortion correction, projection and graphic alignment, or the processing comprises projection and graphic alignment, and the projection comprises cylindrical projection and spherical projection.
When the number N of cameras is >8, the adopted cameras are ordinary cameras, and the photographed image has no distortion, so that distortion correction is not required. When the number N of the cameras is less than 8, the adopted cameras are common cameras, and the shot images have distortion and need to be subjected to distortion correction. The projection may be a cylindrical projection or a spherical projection.
Specifically, referring to fig. 3, fig. 3 is a schematic structural diagram of an apparatus for recording a panoramic video according to an embodiment of the present invention.
The invention is realized by adopting an FPGA, an APU and a plurality of peripheral chips, a system block diagram is shown as 3, three paths of wide-angle camera data can be input into the FPGA in two modes, the mode 1 is as follows: parallel YUV data FPGA, mode 2: the three wide-angle cameras output MIPI format data, and the MIPI-parallel YUV data format data is input into the FPGA through an interface chip of MIPI-parallel YUV data format. Three paths of video data are output to the APU after distortion correction, cylindrical projection and graphic alignment splicing in the FPGA, and are output in four modes after the APU is subjected to H.264 coding: 1) stored to the local SDCard. 2) And sending through wifi. 3) And outputting through the HDMI. 4) Output through USB 3.0.
And 103, sending the processed panoramic video data to the mobile terminal in a local storage mode or a wireless mode or a wired mode.
Preferably, the sending the processed panoramic video data to the mobile terminal in a local storage mode or a wireless mode or a wired mode includes:
and storing the processed panoramic video data into a local SDcard, or sending the processed panoramic video data to wireless routing equipment and/or intelligent equipment through WIFI (wireless fidelity), or outputting the processed panoramic video data to the intelligent equipment through HDMI (high-definition multimedia interface), or outputting the processed panoramic video data to a mobile terminal through USB (universal serial bus).
The embodiment of the invention obtains 360-degree panoramic video data through N cameras, wherein N is more than or equal to 1, the horizontal angle of the video data shot by the cameras is a horizontal angle of 360 degrees/N, and the vertical angle of the video data shot by the cameras is a vertical angle of 360 degrees/N; processing the video data acquired by each camera through a preset algorithm; and sending the processed panoramic video data to a mobile terminal in a local storage or wireless mode or wired mode, thereby realizing the purpose of conveniently recording and playing the 360-degree panoramic video.
Example two
Referring to fig. 4, fig. 4 is a functional block diagram of an apparatus for recording a panoramic video according to an embodiment of the present invention.
In a second embodiment, the apparatus for recording a panoramic video includes:
the acquiring module 401 is configured to acquire 360-degree panoramic video data through N cameras, where N is greater than or equal to 1, a horizontal angle of video data shot by the cameras is a horizontal angle of 360 °/N, and a vertical angle of video data shot by the cameras is a vertical angle of 360 °/N;
specifically, assuming that 3 fisheye cameras are used to acquire 360-degree panoramic video data, the invention adopts 3 fisheyes, the horizontal viewing angle is greater than 180 degrees, the vertical viewing angle is greater than 180 degrees, and the cameras, the FPGA, the APU and a plurality of peripheral chips are implemented as shown in fig. 2, and fig. 2 is a schematic structural diagram of a three-eye handheld 360-degree panoramic video real-time recording and playing device provided by an embodiment of the invention. The structure of the handheld 360-degree panoramic video real-time recording and playing system is shown in fig. 2, the handheld 360-degree panoramic video real-time recording and playing device generates a 360-degree video, and the 360-degree video is coded by H.264 and then is sent to a cloud end or an intelligent device end through wifi or is stored on a local SDcard. The system can realize that: 1) directly send the video live broadcast to the smart machine through wifi. 2) The video is sent to the cloud end through the wireless router, and cloud end data storage and sharing are achieved. The video of high in the clouds can be live at smart machine and web end. 3) And transmitting the panoramic video to a computer for storage and playing through the usb 3.0. 4) And transmitting the data to the intelligent equipment through the HDMI for playing. 5) And storing the coded video into a local SDcard.
A processing module 402, configured to process video data acquired by each camera through a preset algorithm;
preferably, referring to fig. 5, fig. 5 is a functional block diagram of the processing module 402 according to the embodiment of the present invention. The processing module 402 comprises:
an input unit 501, configured to input the video data to an FPGA;
preferably, the input unit 301 is configured to:
transmitting the video data to the FPGA through a parallel YUV data interface; or,
the camera video is output through an MIPI data interface and is input into the FPGA through an MIPI-to-parallel YUV data format interface chip.
A processing unit 502, configured to process the video data through the FPGA and output the processed video data to the APU, where the processing includes distortion correction, projection and graph alignment, or the processing includes projection and graph alignment, and the projection includes cylindrical projection and spherical projection;
preferably, the processing unit 502 is configured to:
receiving input video data in a YUV data format;
detecting the format of the video data through a signal format detection module, wherein the format comprises resolution, pixel clock, line frequency and field frequency information;
inputting the detected video data into a DDR memory unit, processing the video data, and inputting the processed video data into the APU, wherein the processing comprises distortion correction, projection and graphic alignment, or the processing comprises projection and graphic alignment, and the projection comprises cylindrical projection and spherical projection.
An encoding unit 503 for performing h.264 encoding on the input panoramic video data by the APU.
Specifically, referring to fig. 3, fig. 3 is a schematic structural diagram of an apparatus for recording a panoramic video according to an embodiment of the present invention.
The invention is realized by adopting an FPGA, an APU and a plurality of peripheral chips, a system block diagram is shown as 3, three paths of wide-angle camera data can be input into the FPGA in two modes, the mode 1 is as follows: parallel YUV data FPGA, mode 2: the three wide-angle cameras output MIPI format data, and the MIPI-parallel YUV data format data is input into the FPGA through an interface chip of MIPI-parallel YUV data format. Three paths of video data are output to the APU after distortion correction, cylindrical projection and graphic alignment splicing in the FPGA, and are output in four modes after the APU is subjected to H.264 coding: 1) stored to the local SDCard. 2) And sending through wifi. 3) And outputting through the HDMI. 4) Output through USB 3.0.
A sending module 403, configured to send the processed panoramic video data to the mobile terminal in a local storage manner, or in a wireless manner or in a wired manner.
Preferably, referring to fig. 6, fig. 6 is a schematic diagram of functional modules of the sending module 403 according to the embodiment of the present invention. The sending module 403 includes:
a storage unit 601, configured to store the processed panoramic video data in a local SDcard;
a wireless sending unit 602, configured to send the information to a wireless routing device and/or an intelligent device through WIFI;
a first wired transmitting unit 603 for outputting to the smart device through HDMI;
and a second wired transmitting unit 604 for outputting to the mobile terminal through USB.
The embodiment of the invention obtains 360-degree panoramic video data through N cameras, wherein N is more than or equal to 1, the horizontal angle of the video data shot by the cameras is a horizontal angle of 360 degrees/N, and the vertical angle of the video data shot by the cameras is a vertical angle of 360 degrees/N; processing the video data acquired by each camera through a preset algorithm; and sending the processed panoramic video data to a mobile terminal in a local storage or wireless mode or wired mode, thereby realizing the purpose of conveniently recording and playing the 360-degree panoramic video.
The technical principle of the embodiment of the present invention is described above in conjunction with the specific embodiments. The description is only intended to explain the principles of embodiments of the invention and should not be taken in any way as limiting the scope of the embodiments of the invention. Based on the explanations herein, those skilled in the art will be able to conceive of other embodiments of the present invention without inventive step, and these embodiments will fall within the scope of the present invention.