WO2022133782A1 - 视频传输方法及系统、视频处理方法及装置、播放终端、可移动平台 - Google Patents

视频传输方法及系统、视频处理方法及装置、播放终端、可移动平台 Download PDF

Info

Publication number
WO2022133782A1
WO2022133782A1 PCT/CN2020/138557 CN2020138557W WO2022133782A1 WO 2022133782 A1 WO2022133782 A1 WO 2022133782A1 CN 2020138557 W CN2020138557 W CN 2020138557W WO 2022133782 A1 WO2022133782 A1 WO 2022133782A1
Authority
WO
WIPO (PCT)
Prior art keywords
image transmission
frame rate
video
setting information
recording
Prior art date
Application number
PCT/CN2020/138557
Other languages
English (en)
French (fr)
Inventor
杨效
曹子晟
张超
王新伟
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2020/138557 priority Critical patent/WO2022133782A1/zh
Priority to CN202080071094.5A priority patent/CN114556958A/zh
Publication of WO2022133782A1 publication Critical patent/WO2022133782A1/zh

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234381Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4334Recording operations
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440281Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/637Control signals issued by the client directed to the server or network components
    • H04N21/6373Control signals issued by the client directed to the server or network components for rate control, e.g. request to the server to modify its transmission rate
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/647Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules

Definitions

  • the present invention generally relates to the field of video technology, and more particularly to a video transmission method and system, a video processing method and device, a playback terminal, and a movable platform.
  • the time-travel experience machine with glasses pursues the ultimate image transmission experience (low latency, high image quality) and supports high frame rate recording.
  • the camera recording format of the time-travel machine and the image transmission format are bound one by one, and the frame rate must be the same.
  • the glasses need to insert an SD (Secure Digital) card (the SD card memory has aircraft-side video) playback requirements, but the processing capacity of the video-transmission processing chip on the glasses side is limited, and sometimes some video files cannot be played back. affect the user experience.
  • SD Secure Digital
  • a video transmission method and system a video data processing method and device, a playback terminal, and a movable platform are provided.
  • a first aspect of the present invention provides a video transmission method, which is applied to a playback terminal, where the playback terminal is communicatively connected to a movable platform, and the movable platform includes an image transmission device and a photographing device, and the photographing device is used for Shooting video data, the image transmission device is configured to acquire the video data and send the video data to the playback terminal, the method includes: acquiring image transmission setting information and recording specification setting information, wherein the image transmission The setting information includes one mode setting information in the first image transmission mode and the second image transmission mode, and the recording specification setting information includes the recording resolution and the recording frame rate.
  • the video transmission frame rate is the first frame rate
  • the video recording frame rate is greater than or equal to the first frame rate.
  • the video transmission setting information is the second video transmission mode
  • the video transmission frame rate is the second frame rate.
  • the recording frame rate is less than or equal to the second frame rate
  • the first frame rate is less than the second frame rate;
  • a control signal is sent, and the The control signal is used to control the shooting device to shoot video data, to control the image transmission device to send the video data to the playback terminal based on the image transmission setting information, and to control the shooting device to set based on the video recording Information records video data.
  • a second aspect of the present invention provides a video transmission method, which is applied to a movable platform.
  • the playback terminal is communicatively connected to the movable platform.
  • the movable platform includes a video transmission device and a shooting device, and the shooting device is used for shooting video.
  • the playback terminal is used to acquire and play the video data
  • the image transmission device is used to acquire the video data and send the video data to the playback terminal
  • the method includes: acquiring the playback terminal The sent control signal, the control signal is used to control the image transmission device to send the video data to the playback terminal based on the image transmission setting information when the shooting device is shooting video data, and to control the shooting device
  • Video data is recorded based on recording setting information, wherein the image transmission setting information includes one of a first image transmission mode and a second image transmission mode, and the recording specification setting information includes a recording resolution and a recording frame rate , when the image transmission setting information is the first image transmission mode, the image transmission frame rate is the first frame rate, and the video recording frame rate is greater than or equal to the first frame rate.
  • the image transmission setting information is the first frame rate In the two-image transmission mode, the image transmission frame rate is the second frame rate, the video recording frame rate is less than or equal to the second frame rate, and the first frame rate is less than the second frame rate; in response to the control signal, control the shooting device to shoot video data, control the image transmission device to send the video data to the playback terminal based on the image transmission setting information, and control the shooting device to record the video data based on the video recording setting information.
  • a third aspect of the present application provides a playback terminal, where the playback terminal includes:
  • a communication interface for making the playback terminal communicate with a movable platform wherein the movable platform includes an image transmission device and a photographing device, the photographing device is used for shooting video data, and the image transmission device is used for acquiring Sending the video data to the playback terminal;
  • One or more processors configured to execute the program instructions stored in the memory, so that the processor performs the following steps: acquiring image transmission setting information and video recording specification setting information, wherein the image transmission setting information includes Mode setting information in the first image transmission mode and the second image transmission mode, the recording specification setting information includes the recording resolution and the recording frame rate, when the image transmission setting information is the first image transmission mode , the image transmission frame rate is the first frame rate, the video frame rate is greater than or equal to the first frame rate, and when the image transmission setting information is the second image transmission mode, the image transmission frame rate is the second frame rate, so The recording frame rate is less than or equal to the second frame rate, and the first frame rate is less than the second frame rate; based on the image transmission setting information and the recording specification setting information, a control signal is sent, and the control signal uses Controlling the shooting device to shoot video data, and controlling the image transmission device to send the video data to the playback terminal based on the image transmission setting information, and controlling the shooting device to record video based on the video recording setting information. data.
  • a fourth aspect of the present application provides a movable platform, the movable platform includes
  • a photographing device which is used for photographing video data
  • An image transmission device configured to acquire the video data and send the video data to the playback terminal
  • One or more processors configured to execute the program instructions stored in the memory, so that the processors perform the following steps: acquiring a control signal sent by the playback terminal, where the control signal is used to control the shooting
  • the device controls the image transmission device to send the video data to the playback terminal based on the image transmission setting information
  • the shooting device controls the shooting device to record the video data based on the video recording setting information
  • the image transmission setting information includes a mode setting information in the first image transmission mode and the second image transmission mode
  • the recording specification setting information includes the recording resolution and the recording frame rate.
  • the image transmission frame rate is the second frame rate
  • the recording frame rate is less than or equal to the second frame rate
  • the first frame rate is less than the second frame rate
  • the camera in response to the control signal, the camera is controlled to shoot video data, and the The image transmission device sends the video data to the playback terminal based on the image transmission setting information, and controls the shooting device to record the video data based on the video recording setting information.
  • a fifth aspect of the present application provides a video transmission system, the video transmission system includes: the aforementioned playback terminal; and the aforementioned movable platform.
  • a sixth aspect of the present application provides a video processing method, the method includes: acquiring a first video code stream, a second video code stream and an audio code stream, wherein the first video code stream is used for playing on a first type of playback terminal , the encoding format of the first video stream corresponds to the encoding format that can be played by the first type of playback terminal, the second video stream is used for playback by the second type of playback terminal, and the second video stream
  • the encoding format corresponding to the encoding format that can be played by the second type of playback terminal, the video specification of the second video stream is less than or equal to the threshold specification; the first video stream and the second video stream and the audio code stream is packaged as a video file in a preset file format; the video file is stored in the first memory for playback of the video file by the first type of playback terminal and/or the second type of playback terminal .
  • a seventh aspect of the present application provides a video processing apparatus, the apparatus includes: a memory for storing executable program instructions; one or more processors for executing the program instructions stored in the memory, so that The processor executes the aforementioned video processing method; the memory includes a first memory, the first memory is used for storing the video file, and the memory is pluggably arranged in the video processing device.
  • An eighth aspect of the present application provides a movable platform, where the movable platform includes: a photographing device for photographing video data; a memory for storing executable program instructions; one or more processors, for executing the program instructions stored in the memory, so that the processor executes the aforementioned video processing method; the memory includes a first memory for storing the video file, and the The first memory is pluggably arranged in the video processing device.
  • a ninth aspect of the present application provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the aforementioned video transmission method, or implements the aforementioned video processing method.
  • the video transmission device can be controlled to transmit video data to the playback terminal in one of the first video transmission mode and the second video transmission mode, so that the user can experience low latency or high image quality. and the video transmission method of the present application can make the recording frame rate equal to or lower than the first frame rate in the first image transmission mode, so as to meet the user's demand for low frame rate recording, thereby saving recording
  • the video frame rate can be set to be greater than or equal to the second frame rate, so as to meet the needs of users with various video frame rates and provide users with more videos Experience and selection to improve user experience.
  • the video processing method of the present application by encapsulating the first video code stream, the second video code stream and the audio code stream in one video file, enables the video file to be played on various types of playback terminals, overcoming the need for
  • the capability limitation of the second type of playback terminal chip improves the video playback effect of the first type of playback terminal, and compared with the solution of encapsulating the first video stream and the second video stream independently, the present application reduces one audio stream , reducing the processing load of the processor, reducing the total code rate of the storage, and improving the user experience.
  • FIG. 1 shows a schematic block diagram of a video transmission system in an embodiment of the present application
  • FIG. 2 shows a schematic flowchart of a video transmission method in an embodiment of the present application
  • FIG. 3 shows a schematic flowchart of a video transmission method in another embodiment of the present application
  • FIG. 4 shows a schematic block diagram of a playback terminal in an embodiment of the present application
  • FIG. 5 shows a schematic block diagram of a movable platform in an embodiment of the present application
  • FIG. 6 shows a schematic flowchart of a video processing method in an embodiment of the present application
  • FIG. 7 shows a schematic diagram of a packaging structure of a video file in an embodiment of the present application.
  • FIG. 8 shows a schematic block diagram of a video processing apparatus in an embodiment of the present application.
  • the present application provides a video transmission system, which may include a movable platform, a playback terminal, a control terminal, etc.
  • the movable platform may include an aircraft such as an unmanned aerial vehicle, a vehicle, a ship, a robot, and the like.
  • the playback terminal may include a terminal device for playing image transmission data, and the playback terminal may be a handheld or wearable terminal device.
  • the playback terminal may include a smart phone, a tablet computer, a desktop computer, a computer, glasses, gloves, a helmet, and a microphone. or any combination of them.
  • the control terminal can include, for example, a mobile phone, a remote control, a tablet computer, a notebook, etc.
  • the control terminal can also be a part or all of the control system of the mobile platform, and can also be movable through software, hardware, or a combination of software and hardware. Part or all of the computer equipment of the control method of the platform.
  • the movable platform may include a photographing device and an image transmission device, where the photographing device is used to photograph video or image data; the image transmission device is used to acquire the video data and send the video data to the playback terminal.
  • the photographing device may be a device with an image sensor, such as a camera, a video camera, a camera, or other devices.
  • An image sensor (also referred to herein as a camera sensor) is a device for capturing image data, which may be, for example, a CMOS image sensor, a CCD image sensor, or the like.
  • the photographing device may also be disposed on the body of a movable platform (eg, an aircraft, a handheld gimbal) in the form of an embedded imaging system.
  • the photographing device can be installed on the body of the movable platform as a load, and the user can control the photographing device through a playback terminal or a control terminal.
  • An application program may be installed on the playback terminal and/or the control terminal, and the application program is used to receive the operation instructions input by the user.
  • One or more of the stage, carrier, and payload provide control signals, the control signals from the terminal equipment may result in control of the payload, such as controlling the operation of a camera such as a camera or other camera (capturing still or moving video or images, zooming, turn on or off, switch imaging modes, change video or image resolution, change focus, change depth of field, change exposure time, change viewing angle or field of view).
  • a camera such as a camera or other camera (capturing still or moving video or images, zooming, turn on or off, switch imaging modes, change video or image resolution, change focus, change depth of field, change exposure time, change viewing angle or field of view).
  • the video transmission system may include an airplane 101 , glasses 102 , and a remote control device 103 .
  • the aircraft 101 is, for example, an unmanned aerial vehicle, and the unmanned aerial vehicle may be a rotary-wing unmanned aerial vehicle, such as a quad-rotor unmanned aerial vehicle, a hexa-rotor unmanned aerial vehicle, an eight-rotor unmanned aerial vehicle, or a fixed-wing unmanned aerial vehicle.
  • the aircraft 101 may include a power system, a gimbal, a camera, a video transmission device such as a digital video transmission transceiver, a mobile device, and the like.
  • the power system is used to provide moving power for the aircraft 101, and the power system includes one or more of a propeller, a motor, and an ESC.
  • the photographing device can be mounted on the main body of the movable platform through the pan/tilt, and is used for capturing images during the movement of the movable platform, and also for recording video during the movement of the movable platform.
  • Shooting devices include, but are not limited to, multispectral imagers, hyperspectral imagers, visible light cameras, and infrared cameras.
  • the gimbal is a multi-axis transmission and stabilization system.
  • a digital image transmission transceiver can refer to a device that transmits information (such as image data) through a digital communication link, that is, a digital image transmission transceiver can be used for a mobile platform to exchange information with other terminals, and other terminals can include playback terminals and control terminals. one or more of.
  • a digital image transmission transceiver may be used to send a data packet including image data captured by a photographing device to a playback terminal.
  • the mobile device is used to control the movement of the movable platform, that is, to control the movement parameters of the movable platform, and the movement parameters include movement speed, movement posture, movement direction, movement trajectory, and the like.
  • the playback terminal may refer to a glasses device on the flying hand side or other devices.
  • the glasses device may refer to a virtual reality (Virtual Reality, VR) or an augmented reality (Augmented Reality, AR) glasses device that can play videos.
  • the glasses equipment on the pilot side may refer to the glasses equipment controlled by the pilot (ie, the contestant).
  • the playback terminal may include an image transmission device such as a digital image transmission transceiver and a display, and the image transmission device may be used to interact information between the playback terminal and the movable platform.
  • the image transmission device of the playback terminal may be configured to receive a data packet sent by the movable platform through the image transmission device and play according to the data packet, so as to present a picture captured by the movable platform.
  • the aircraft 101 is equipped with a photographing device such as a camera to record video and send real-time image transmission data to the ground-side glasses 102
  • the remote control device 103 is used to control the flight, video, etc. of the aircraft 101
  • the glasses 102 are used to receive And display the plane-side image transmission data such as images or videos, set the format, control the plane-side video, etc.
  • a crossover experience machine with glasses which pursues the ultimate image transmission experience (such as low latency and high image quality), and supports high frame rate video recording.
  • image transmission high image quality mode: Since the channel width of the air end (such as the aircraft end) and the ground end (such as the glasses end) is fixed (such as 40Mbps), the higher the transmission frame rate, the more each frame can be allocated. The smaller the average bandwidth received, the worse the picture quality. Therefore, the image transmission low frame rate of 60fps will be better than the image transmission high frame rate of 120fps, so it can be called the image transmission low frame rate of 60fps as the high image quality mode.
  • Image transmission Low-latency mode: Because the higher the frame rate, the faster the screen refresh rate, the smaller the delay that the human eye can perceive, so the image transmission high frame rate of 120fps is called the low-latency mode.
  • the recording format of the camera, such as the camera, and the image transmission format are bound one by one, and the frame rate must be the same.
  • This method has the following determinations: a) Image transmission low-latency mode: camera settings can only choose 1080P120 video recording, such a high frame rate High-speed video recording is sometimes not what the user wants. For example, sometimes in order to save SD card storage space, users prefer to use 1080P60, but users cannot select this mode unless they switch to high-quality mode; b) Image transmission high-quality mode: camera settings can only choose 4KP60 or 1080P60 video recording, The slow motion recording mode of 1080P120 (such as 4x slow motion) cannot be selected, which limits the user's performance.
  • Camera video format settings and image transmission format settings are placed under different menus. Users can set high image quality/low latency for image transmission, and they cannot see the changes in the corresponding camera video format; users set the camera video format, they cannot view the changes. Changes to image transmission with high quality/low latency. Unless you switch to another menu to view, the operation is too complicated.
  • the present application provides a video transmission method, which is applied to a playback terminal, and the playback terminal is communicatively connected to a movable platform, and the movable platform includes an image transmission device and a shooting device, and the shooting device for shooting video data, and the image transmission device is used for acquiring the video data and sending the video data to the playback terminal, the method includes: acquiring image transmission setting information and recording specification setting information, wherein the The image transmission setting information includes one of the first image transmission mode and the second image transmission mode.
  • the recording specification setting information includes the recording resolution and the recording frame rate.
  • the image transmission setting information is the first image transmission mode
  • the image transmission frame rate is the first frame rate
  • the video recording frame rate is greater than or equal to the first frame rate.
  • the image transmission setting information is the second image transmission mode
  • the image transmission frame rate is the first frame rate.
  • Two frame rates the recording frame rate is less than or equal to the second frame rate
  • the first frame rate is less than the second frame rate;
  • a control signal is sent, The control signal is used to control the shooting device to shoot video data, control the image transmission device to send the video data to the playback terminal based on the image transmission setting information, and control the shooting device to send the video data based on the video recording.
  • the video transmission device can be controlled to transmit video data to the playback terminal in one of the first video transmission mode and the second video transmission mode, so that the user can experience low latency or high image quality. and the video transmission method of the present application can make the recording frame rate equal to or lower than the first frame rate in the first image transmission mode, so as to meet the user's demand for low frame rate recording, thereby saving recording
  • the video frame rate can be set to be greater than or equal to the second frame rate, so as to meet the needs of users with various video frame rates and provide users with more videos Experience and selection to improve user experience.
  • the video transmission method 200 of the present application will be described based on the perspective of a playback terminal with reference to FIG. 2 .
  • the playback terminal is connected to a movable platform in communication, the movable platform includes an image transmission device and a shooting device, the shooting device is used for shooting video data, and the image transmission device is used to obtain the video. data and send the video data to the playback terminal,
  • the video transmission method 200 of the present application includes: in step S201, acquiring image transmission setting information and video recording specification setting information, wherein the image transmission setting information includes a first image transmission mode and a second image transmission mode A mode setting information in the image transmission mode, the recording specification setting information includes the recording resolution and the recording frame rate.
  • the image transmission setting information is the first image transmission mode
  • the image transmission frame rate is the first Frame rate
  • the recording frame rate is greater than or equal to the first frame rate
  • the image transmission setting information is the second image transmission mode
  • the image transmission frame rate is the second frame rate
  • the recording frame rate is less than or equal to the the second frame rate
  • the first frame rate is smaller than the second frame rate
  • a control signal is sent, and the control signal is used to control the
  • the shooting device is shooting video data, controls the image transmission device to send the video data to the playback terminal based on the image transmission setting information, and controls the shooting device to record the video data based on the video recording setting information.
  • the first image transmission mode may be a high-quality image transmission mode.
  • the image transmission frame rate is the first frame rate.
  • the first frame rate is less than or equal to 60fps, for example, the first frame rate is equal to 60fps, as shown in Table 1;
  • the second image transmission mode may be a low-latency mode, when the image transmission setting information is the first In the two-image transmission mode, the image transmission frame rate is the second frame rate.
  • the second frame rate is greater than 60fps and less than or equal to 120fps.
  • the second frame rate may be equal to 120fps, as shown in Table 1. .
  • acquiring the image transmission setting information and video recording specification setting information of the photographing device includes: first, in response to a first user instruction, controlling the display of the playback terminal to display a setting interface, where the setting interface is configured to display: The options corresponding to the first image transmission mode and the options corresponding to the second image transmission mode; optionally, the setting interface may also be used to display the setting information of other shooting devices, such as camera parameters, etc., and the playback terminal There may be multiple function keys such as physical keys (eg menu keys, etc.) or virtual function keys, etc., and the display of the playback terminal may be controlled to display a setting interface based on the first user instruction input by the user through the function keys.
  • one of the first image transmission mode and the second image transmission mode is determined as the image transmission setting information
  • the mode selection instruction input by the user through, for example, a function key is obtained
  • one of the first image transmission mode and the second image transmission mode is determined as the image transmission setting information
  • the setting interface may also display multiple options, such as Image transmission specifications, camera parameters, video specifications, etc., you can select one of multiple options by obtaining the instructions entered by the user through function keys such as direction keys, volume keys, etc., and then perform multiple sub-options set under the option. Select, such as low-latency mode, high-definition mode, etc. under the image transmission specification option.
  • the recording specification setting information is determined based on the determined image transmission setting information.
  • the linkage of the video recording of the camera and the setting of the image transmission frame rate is realized, which can ensure that the user can obtain the desired video effect such as low latency or high image quality. It is possible to observe the pictures that meet the subjective needs in the playback terminal.
  • the camera video format setting, image transmission high-quality and low-latency switching options under the camera's menu it is convenient for users to quickly check the status changes.
  • the playable terminal may automatically determine the recording specification setting information based on the determined image transmission setting information, or may also manually determine the recording specification setting information based on a user instruction.
  • the video recording specification setting information includes video recording resolution and video recording frame rate, that is, video recording specifications (also referred to as video recording format herein) may include video recording resolution and video recording frame rate.
  • video recording format 4KP60 means that the video resolution is 4KP and the video frame rate is 60fps.
  • the video format is 1080P60
  • the video format is 1080P120
  • the video resolution is 1080P.
  • the video frame rate is 120fps.
  • the video recording resolution includes but is not limited to 4KP, 720P or 1080P, or other resolutions.
  • determining the recording specification setting information based on the determined image transmission setting information includes: when the image transmission setting information is the first image transmission mode, responding to a recording specification setting instruction input by a user , controlling the display of the playback terminal to display multiple first-type recording specifications on the setting interface, and the recording frame rates of the multiple first-type recording specifications are greater than or equal to the first frame rate;
  • the specification selection instruction determines one of the multiple first-type recording specifications as the recording specification setting information.
  • the first-type recording specification may include but not limited to 4KP60, 1080P60, 1080P120, or other recording specifications.
  • the camera can be controlled to perform a slow motion recording mode such as quadruple slow motion, so that the user can obtain more video effect experience.
  • the image transmission setting information and the recording specification setting information are linked.
  • the recording frame rate is greater than or equal to the first image transmission mode.
  • a frame rate for example, as shown in Table 1, for example, when the image transmission setting information is the first image transmission mode, such as high image quality, the first frame rate can be 60fps, and the recording frame rate is greater than or equal to The first frame rate, for example, the recording frame rate can be 60fps or 120fps. Therefore, in the high-quality mode, the recording format can be selected from 4KP60, 1080P60, and 1080P120.
  • determining the recording specification setting information based on the determined image transmission setting information includes: when the image transmission setting information is the second image transmission mode, setting an instruction based on a recording specification input by a user , controlling the display of the playback terminal to display multiple second-type recording specifications on the setting interface, and the recording frame rate of the multiple second-type recording specifications is less than or equal to the second frame rate; based on the specifications input by the user A selection instruction is used to determine one of the plurality of second-type recording specifications as the recording specification setting information.
  • the second type of video recording specifications may include but not limited to 1080P60, 1080P120, or other video recording specifications.
  • the storage space required for recording video can be saved by reducing the recording frame rate under the premise of obtaining low latency, such as SD card storage space .
  • the image transmission setting information and the recording specification setting information are linked.
  • the recording frame rate is less than or equal to the first image transmission mode.
  • Two frame rates for example, as shown in Table 1, for example, when the image transmission setting information is the second image transmission mode, such as low latency, the second frame rate can be 120fps, and the recording frame rate is less than or equal to The first frame rate, for example, the video frame rate can be 60fps or 120fps. Therefore, in the low-latency mode, the video recording format can be selected from 1080P60 and 1080P120.
  • the video recording format There can also be more than 2 listed in the table.
  • Both video recording and video transmission of the camera can actively drop frames. For example, there is a situation where the frame rate of video transmission is 120fps and the frame is dropped to 60fps. Another example is the situation where the frame rate of video transmission is 120fps and the video of the camera is actively dropped to 60fps.
  • the image transmission setting takes precedence over the recording specification setting.
  • the image transmission preference that is, the image transmission mode
  • the recording format when the recording format is set, only the first type of recording may be displayed.
  • Specifications such as 4KP60, 1080P60, 1080P120 shown in the first 3 rows of Table 1
  • the video transmission preference that is, the video transmission mode
  • only the second type can be displayed
  • Recording specifications such as 1080P60 and 1080P120 as shown in the last two rows of Table 1.
  • the switching of the video format of the camera will not lead to the passive switching of the image transmission mode.
  • the method of the present application further includes: based on the image transmission setting information and the recording specification setting information, determining a capture frame rate of the image sensor of the photographing device, wherein the capture frame rate is greater than or equal to the The larger of the video frame rate and the image transmission frame rate.
  • Capture frame rate refers to the number of image frames captured by the image sensor per second.
  • the image sensor only outputs a high frame rate when necessary, such as a second frame rate, and the second frame rate is, for example, 120 fps, for example, when the image transmission setting information is the first image transmission mode, and the recording frame rate
  • the capture frame rate is equal to the second frame rate, for example, 120fps; for another example, when the image transmission setting information is the second image transmission mode, the capture frame rate is equal to the second frame rate.
  • the frame rate is 120fps
  • in the other video specification settings and image transmission format settings of the first image mode for example, when the video frame rate and image transmission frame rate are both lower than the second frame rate, for example, both are the first frame rate (for example, 60fps).
  • the image sensor may not output a high frame rate such as 120fps, thereby reducing the system load and power consumption.
  • the method further includes: acquiring a mode switching instruction input by the user, the mode switching The instruction is used to instruct to switch the currently running first image transmission mode to the second image transmission mode or to switch the currently running second image transmission mode to the first image transmission mode; the mode switching instruction Send to the movable platform to switch the currently running first image transmission mode to the second image transmission mode or switch the currently running second image transmission mode to the first image transmission mode; according to In the switched image transmission mode, the video specification setting information is updated, and the updated video recording specification setting information is sent to the movable platform, so as to control the shooting device to perform video recording with the updated video recording specification setting information, wherein the updated video specification setting information
  • the subsequent video recording specification setting information includes a second resolution and the first video recording frame rate, and the second resolution is less than or equal to the first resolution.
  • the playback terminal The recording specification setting information can be updated automatically, or the recording
  • the video resolution or video frame rate of the camera should remain unchanged as much as possible. If this is not possible, reduce the video resolution, even if the updated video resolution is lower than or equal to the pre-update video resolution.
  • the video specifications of the shooting device can be kept from changing with the image transmission mode as much as possible, avoiding the need for users to learn and explore the corresponding relationship between the recording format and the image transmission format due to the change, simplifying the user operation. Save user time and improve user experience.
  • 1080P60 (120->60) also means that the capture frame rate is 120fps and the frame is dropped to 60fps to obtain the video frame rate.
  • the interval extraction method may also adopt an image fusion method, etc., which is not specifically limited here.
  • the video specification of the camera can be passively switched.
  • the method further includes: when switching the first image transmission mode to the second image transmission mode mode, or, after switching the second image transmission mode to the first image transmission mode, send the updated recording specification setting information to the movable platform to control the shooting device to update the The recording specification setting information is used for recording, and the updated recording specification setting information includes the second resolution and the second recording frame rate; when the second resolution is different from the first resolution and/or when the When the second video recording frame rate is different from the first video recording frame rate, prompt information is output to prompt the user that the video recording specification has changed.
  • the prompt information can be text information or other types of prompt information
  • the display of the playback terminal can differentiate and display the prompt information in any suitable preset manner.
  • the preset differentiated display includes at least one of the following display methods: high Bright display, flashing display, additional symbol display, differentiated shading color display, or differentiated font color display, for example, displaying prompt information in red shading color, or any other video that enables users to quickly identify changes Specification.
  • the video transmission method 300 is applied to a movable platform, the playback terminal is communicatively connected to the movable platform, the movable platform includes an image transmission device and a shooting device, the shooting device is used for shooting video data, and the playback terminal is used for The video data is acquired and played, and the image transmission apparatus is configured to acquire the video data and send the video data to the playback terminal.
  • the video transmission method 300 includes the following steps: in step S301, acquiring a control signal sent by the playback terminal, where the control signal is used to control the image transmission device to control the image transmission device when capturing video data Send the video data to the playback terminal based on the image transmission setting information, and control the shooting device to record the video data based on the video recording setting information, wherein the image transmission setting information includes a first image transmission mode and a second image transmission mode A mode setting information in the video recording specification setting information including video recording resolution and video recording frame rate, when the video transmission setting information is the first video transmission mode, the video transmission frame rate is the first frame rate, The video frame rate is greater than or equal to the first frame rate.
  • the video transmission setting information is the second video transmission mode
  • the video transmission frame rate is the second frame rate
  • the video recording frame rate is less than or equal to the second video transmission mode.
  • frame rate the first frame rate is lower than the second frame rate; in step S302, in response to the control signal, the camera is controlled to shoot video data, and the image transmission device is controlled based on the image transmission
  • the setting information sends the video data to the playback terminal, and controls the shooting device to record the video data based on the recording setting information.
  • the first image transmission mode may be a high-quality image transmission mode.
  • the image transmission frame rate is the first frame rate.
  • the first frame rate is less than or equal to 60fps, for example, the first frame rate is equal to 60fps, as shown in Table 1;
  • the second image transmission mode may be a low-latency mode, when the image transmission setting information is the first In the two-image transmission mode, the image transmission frame rate is the second frame rate.
  • the second frame rate is greater than 60fps and less than or equal to 120fps.
  • the second frame rate may be equal to 120fps, as shown in Table 1. .
  • the recording specification setting information includes one of a plurality of first-type recording specifications, and the plurality of first-type recordings.
  • the recording frame rate of the specification option is greater than or equal to the first frame rate.
  • the first type of video recording specifications may include, but not limited to, 4KP60, 1080P60, 1080P120, or other video recording specifications.
  • the camera can be controlled to perform a slow motion recording mode such as quadruple slow motion, so that the user can obtain more video effect experience.
  • the image transmission setting information and the recording specification setting information are linked.
  • the recording frame rate is greater than or equal to the first image transmission mode.
  • a frame rate for example, as shown in Table 1, for example, when the image transmission setting information is the first image transmission mode, such as high image quality, the first frame rate can be 60fps, and the recording frame rate is greater than or equal to The first frame rate, for example, the recording frame rate can be 60fps or 120fps. Therefore, in the high-quality mode, the recording format can be selected from 4KP60, 1080P60, and 1080P120.
  • the recording specification setting information includes one of the plurality of second-type recording specifications, and the plurality of second-type recording specifications
  • the recording frame rate is less than or equal to the second frame rate.
  • the second frame rate may be 120fps
  • the recording frame rate is less than or equal to the first image transmission mode.
  • Frame rate for example, the video frame rate can be 60fps or 120fps. Therefore, in the low-latency mode, the video format can be selected from 1080P60 and 1080P120. Of course, when there are other frame rates and resolutions, the video format can also be more than 2 listed in the table.
  • Both video recording and video transmission of the camera can actively drop frames. For example, there is a situation where the frame rate of video transmission is 120fps and the frame is dropped to 60fps. Another example is the situation where the frame rate of video transmission is 120fps and the video of the camera is actively dropped to 60fps.
  • the image transmission setting takes precedence over the recording specification setting.
  • the image transmission preference that is, the image transmission mode
  • the recording format when the recording format is set, only the first type of recording may be displayed.
  • Specifications such as 4KP60, 1080P60, 1080P120 shown in the first 3 rows of Table 1
  • the video transmission preference that is, the video transmission mode
  • only the second type can be displayed
  • Recording specifications such as 1080P60 and 1080P120 as shown in the last two rows of Table 1.
  • the switching of the video format of the camera will not lead to the passive switching of the image transmission mode.
  • the video transmission method 300 further includes: based on the image transmission setting information and the recording specification setting information, determining a capture frame rate of the image sensor of the photographing device, wherein the capture frame rate is greater than or equal to the The larger of the video frame rate and the image transmission frame rate.
  • the movable platform can automatically determine the capture frame rate of the image sensor of the photographing device after obtaining the image transmission setting information and the recording specification setting information, or, the playback terminal can also be based on the image transmission setting information and the recording specification setting information. , determining the capture frame rate of the image sensor of the shooting device, the movable platform obtains the capture frame rate from the playback terminal, and controls the image sensor of the capture device to capture image data based on the capture frame rate.
  • the image sensor only outputs a high frame rate when necessary, such as a second frame rate, and the second frame rate is, for example, 120 fps, for example, when the image transmission setting information is the first image transmission mode, and the recording frame rate
  • the capture frame rate is equal to the second frame rate, for example, 120fps; for another example, when the image transmission setting information is the second image transmission mode, the capture frame rate is equal to the second frame rate.
  • the frame rate is 120fps
  • in the other video specification settings and image transmission format settings of the first image mode for example, when the video frame rate and image transmission frame rate are both lower than the second frame rate, for example, both are the first frame rate (for example, 60fps).
  • the image sensor may not output a high frame rate such as 120fps, thereby reducing the system load and power consumption.
  • the video transmission method 300 further includes: acquiring a mode switching instruction sent by the playback terminal; responding to the mode switching instruction , switch the currently running first image transmission mode to the second image transmission mode or switch the currently running second image transmission mode to the first image transmission mode; obtain the updated video specification setting information, and control the shooting device to record with the updated recording specification setting information, wherein the updated recording specification setting information includes a second resolution and the first recording frame rate, and the second resolution is smaller than or equal to the first resolution.
  • the updated video recording specification setting information may be obtained from the playback terminal, or may also be the updated video recording specification setting information determined by the movable platform according to the switched image transmission mode.
  • the video resolution or video frame rate of the camera should remain unchanged as much as possible. If this is not possible, reduce the video resolution, even if the updated video resolution is lower than or equal to the pre-update video resolution.
  • the video specifications of the shooting device can be kept from changing with the image transmission mode as much as possible, avoiding the need for users to learn and explore the corresponding relationship between the recording format and the image transmission format due to the change, simplifying the user operation. The user time is saved, the learning cost of the user is reduced, and the user experience is improved.
  • the playback terminal 400 of the present application will be described with reference to FIG. 4 , wherein the playback terminal can be used as the execution body of the foregoing video transmission method 200 .
  • the playback terminal 400 includes one or more processors 401 , a memory 402 , a communication interface 403 , a display 404 and the like. These components are interconnected by a bus system and/or other form of connection mechanism (not shown).
  • the memory 402 is used for storing executable instructions, for example, for storing corresponding steps and program instructions for implementing the video transmission method 200 according to the embodiment of the present application.
  • the memory 402 can store an operating system (including programs for processing various basic system services and for performing hardware-related tasks), an application program required for at least one function (such as a control function of a camera, a sound playback function, an image playback function) and many more. Some of these applications can be downloaded from external servers via wireless communication. Other applications may be installed in the memory 402 at the time of manufacture or from the factory, eg, for basic functions of the glasses-type playback terminal 400 (eg, answering a phone call, making a phone call, receiving a message, sending a message, etc.).
  • One or more computer program products may be included, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory.
  • the volatile memory may include, for example, random access memory (RAM) and/or cache memory, or the like.
  • the non-volatile memory may include, for example, a read only memory (ROM), a hard disk, a flash memory (eg, an SD card), and the like.
  • the communication interface 403 can be used for communication between the playback terminal 400 and the movable platform or other devices, including wired or wireless communication.
  • the communication interface 403 may be for accessing a wireless network based on a communication standard, such as WiFi (eg WiFi6), software-defined radio (SDR for short), 2G, 3G, 4G, 5G or a combination thereof.
  • the communication interface 403 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel.
  • the communication interface 403 also includes a near field communication (NFC) module to facilitate short-range communication.
  • the NFC module may be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
  • RFID radio frequency identification
  • IrDA infrared data association
  • UWB ultra-wideband
  • Bluetooth Bluetooth
  • the playback terminal 400 further includes a display 404, a device for displaying images or videos generated by the processor component.
  • the display may include, for example, a liquid crystal display (LCD) device or an organic light emitting diode (OLED) device.
  • the display may display various images, such as menus, selected operating parameters, a capture mode selection interface, images captured by an image sensor, received by the processor, etc. and/or received from a user interface of the device other information, etc.
  • the playback terminal 400 further includes an input device (not shown), which may be a device used by the user to input various operations, for example, may include function keys (for example, touch keys, keys (for example, volume control keys, switch keys, etc.) ), mechanical keys, soft keys, etc., one or more of joysticks, microphones, touch screens, etc., or other user input devices for enabling a user to enter information.
  • Data eg, audio, video, images, etc. is entered by device to obtain.
  • Play terminal 400 such as glasses may also include a power supply unit (not shown) that can be configured to receive external power or provide internal power to supply appropriate power required to operate the elements and components included in play terminal 400 .
  • the power supply unit may include a battery, and the battery may be configured to be embedded in the terminal body, or configured to be detachable from the terminal body.
  • the playback terminal 400 such as glasses according to an embodiment of the present application may include an optical unit that forms a virtual image of visual information, and may display the virtual image having various sizes in various positions based on the optical unit.
  • the processor 401 may be a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other form of processing with data processing capabilities and/or instruction execution capabilities unit, and can control other components in the camera to perform desired functions.
  • the processor 401 is capable of executing the instructions stored in the memory 402 to perform corresponding steps of the video transmission method described herein.
  • processor 401 can include one or more embedded processors, processor cores, microprocessors, logic circuits, hardware finite state machines (FSMs), digital signal processors (DSPs), ISPs, encoders, or combinations thereof combination.
  • FSMs hardware finite state machines
  • DSPs digital signal processors
  • ISPs encoders, or combinations thereof combination.
  • processor 401 is configured to execute program instructions stored in the memory, causing one or more processors to perform the following steps:
  • the image transmission setting information includes one mode setting information in the first image transmission mode and the second image transmission mode
  • the recording specification setting information includes the recording resolution and Video frame rate
  • the video transmission frame rate is the first frame rate
  • the video recording frame rate is greater than or equal to the first frame rate.
  • the image transmission frame rate is the second frame rate
  • the video recording frame rate is less than or equal to the second frame rate
  • the first frame rate is less than the second frame rate
  • a control signal is sent, and the control signal is used to control the image transmission device to send the image transmission device to the image transmission device based on the image transmission setting information when shooting video data.
  • the playback terminal sends the video data, and controls the shooting device to record the video data based on the video recording setting information.
  • the first frame rate is less than or equal to 60 fps
  • the second frame rate is greater than 60 fps and less than or equal to 120 fps.
  • the video recording resolution includes 4KP, 720P or 1080P or other suitable resolutions.
  • the processor 401 is configured to execute program instructions stored in the memory, so that one or more processors execute acquiring the image transmission setting information and the video recording specification setting information of the photographing device, including: responding to the first user instruction , controlling the display of the playback terminal to display a setting interface, where the setting interface is used to display: options corresponding to the first image transmission mode and options corresponding to the second image transmission mode; a mode selection instruction in response to user input , determining one of the first image transmission mode and the second image mode as the image transmission setting information; and determining the recording specification setting information based on the determined image transmission setting information.
  • the processor 401 is configured to execute program instructions stored in the memory, so that one or more processors execute the determination of the video recording specification setting information based on the determined image transmission setting information, including: when When the image transmission setting information is the first image transmission mode, in response to the recording specification setting instruction input by the user, the display of the playback terminal is controlled to display a plurality of first type recording specifications on the setting interface, the plurality of first type recording specifications.
  • the recording frame rate of one type of recording specification is greater than or equal to the first frame rate; in response to a specification selection instruction input by the user, one of the plurality of first type of recording specifications is determined as the recording specification setting information.
  • the processor 401 is configured to execute program instructions stored in the memory, so that one or more processors execute the determination of the video recording specification setting information based on the determined image transmission setting information, including: when When the image transmission setting information is the second image transmission mode, based on the recording specification setting instruction input by the user, the display of the playback terminal is controlled to display a plurality of second type recording specifications on the setting interface, and the plurality of second The recording frame rate of the recording-like specification is less than or equal to the second frame rate; based on the specification selection instruction input by the user, one of the plurality of second-type recording specifications is determined as the recording specification setting information.
  • the processor 401 is configured to execute program instructions stored in the memory, so that one or more processors execute: based on the image transmission setting information and the recording specification setting information, determine the image sensor of the photographing device The acquisition frame rate, wherein the acquisition frame rate is greater than or equal to the larger of the video recording frame rate and the image transmission frame rate.
  • the capture frame rate is equal to the second frame rate, or when The image transmission setting information is the second image transmission mode, and the capture frame rate is equal to the second frame rate.
  • the processor 401 when the photographing device records video at a first resolution and a first video recording frame rate, the processor 401 is configured to execute program instructions stored in the memory, so that one or more processors execute the The processor is further configured to: obtain a mode switching instruction input by the user, where the mode switching instruction is used to instruct to switch the currently running first image transmission mode to the second image transmission mode or to switch the currently running second image transmission mode The transmission mode is switched to the first image transmission mode; the mode switching instruction is sent to the movable platform, so as to switch the currently running first image transmission mode to the second image transmission mode or to switch the currently running image transmission mode.
  • the second image transmission mode is switched to the first image transmission mode; according to the switched image transmission mode, the recording specification setting information is updated, and the updated recording specification setting information is sent to the movable platform to control the
  • the shooting device performs video recording with the updated video recording specification setting information, wherein the updated video recording specification setting information includes a second resolution and the first video recording frame rate, and the second resolution is less than or equal to the first video recording frame rate. a resolution.
  • the processor 401 is configured to execute the program instructions stored in the memory, so that one or more processors execute: when After the first image transmission mode is switched to the second image transmission mode, or, after the second image transmission mode is switched to the first image transmission mode, the camera is controlled to use the second resolution and recording at the second recording frame rate; when the second resolution is different from the first resolution and/or when the second recording frame rate is different from the first recording frame rate, output prompt information ;
  • the playback terminal further includes a display, which is used for acquiring and displaying the prompt information, so that when the video recording format changes, the user can obtain the prompt information in time.
  • the first frame rate is less than or equal to 60fps, for example, the first frame rate is equal to 60fps
  • the second frame rate is greater than 60fps and less than or equal to 120fps, for example, the second frame rate is equal to 120fps
  • the video recording resolution includes 4KP, 720P or 1080P, or other suitable resolutions.
  • the movable platform of the present application will be described with reference to FIG. 5 , wherein the movable platform 500 can be used to execute the relevant steps in the aforementioned video transmission method 300 .
  • the movable platform 500 includes one or more processors 501 , a camera 505 , a memory 502 , an image transmission device 503 , a camera 504 , a communication interface (not shown), and the like. These components are interconnected by a bus system and/or other form of connection mechanism (not shown).
  • the memory 502 is used for storing executable instructions, for example, for storing corresponding steps and program instructions for implementing the video transmission method 300 according to the embodiment of the present application.
  • the memory 502 may store an operating system (including programs for handling various basic system services and for performing hardware-related tasks), application programs required for at least one function, and the like.
  • One or more computer program products may be included, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory.
  • the volatile memory may include, for example, random access memory (RAM) and/or cache memory, or the like.
  • the non-volatile memory may include, for example, a read only memory (ROM), a hard disk, a flash memory (eg, an SD card), and the like.
  • a communication interface may be used for communication between the movable platform 500 and a terminal device (eg, a control terminal, a playback terminal such as glasses) or other devices, including wired or wireless communication.
  • the communication interface 1004 may be for accessing a wireless network based on a communication standard, such as WiFi (eg WiFi6), software-defined radio (SDR for short), 2G, 3G, 4G, 5G or a combination thereof.
  • the removable platform 500 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel.
  • 1004 also includes a near field communication (NFC) module to facilitate short-range communication.
  • the NFC module may be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
  • RFID radio frequency identification
  • IrDA infrared data association
  • UWB ultra-wideband
  • Bluetooth Bluetooth
  • the processor 501 may be a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other form of processing with data processing capabilities and/or instruction execution capabilities unit, and can control other components in the camera to perform desired functions.
  • the processor 501 is capable of executing the instructions stored in the memory 502 to perform the corresponding steps of the video transmission method 300 described herein.
  • processor 1001 can include one or more embedded processors, processor cores, microprocessors, logic circuits, hardware finite state machines (FSMs), digital signal processors (DSPs), ISPs, encoders, or combinations thereof combination.
  • FSMs hardware finite state machines
  • DSPs digital signal processors
  • ISPs encoders, or combinations thereof combination.
  • the shooting device 504 is used for shooting video or image data, and the image transmission device 503 is used for acquiring the video data and sending the video data to the playback terminal.
  • the photographing device 504 may be a device having an image sensor, such as a camera, a video camera, a camera, or other devices.
  • An image sensor (also referred to herein as a camera sensor) is a device for capturing image data, which may be, for example, a CMOS image sensor, a CCD image sensor, or the like.
  • the photographing device 504 may also be disposed on the body of a movable platform (eg, an aircraft, a handheld gimbal) in the form of an embedded imaging system.
  • the photographing device 504 can be set on the body of the movable platform as a load, and the user can control the photographing device through a playback terminal or a control terminal.
  • the image transmission device 503, such as a digital image transmission transceiver may refer to a device that transmits information (such as image data) through a digital communication link, that is, the digital image transmission transceiver can be used for a mobile platform to exchange information with other terminals.
  • the photographing device 504 may be disposed on the movable platform body.
  • the movable platform includes an aircraft (eg, an unmanned aerial vehicle), a robot, a vehicle, a gimbal, or a boat.
  • the platform body is the fuselage of the unmanned aerial vehicle, wherein the photographing device can be arranged on the movable platform body in the form of an embedded imaging system.
  • the platform body is the body of the vehicle.
  • the vehicle may be a self-driving car or a semi-autonomous driving car, which is not limited here.
  • the photographing device is applied to the robot, the platform body is the robot.
  • the platform body is the hull.
  • the processor 1001 is configured to execute the program instructions stored in the memory, so that one or more processors execute the relevant steps of the video transmission method 300 described above.
  • the specific video transmission method 300 reference may be made to the foregoing description. , the description will not be repeated here.
  • the present application also provides a video transmission system, where the video transmission system includes the playback terminal described above, a movable platform, a control terminal, and the like.
  • the video transmission system includes the playback terminal described above, a movable platform, a control terminal, and the like.
  • the video transmission system includes the playback terminal described above, a movable platform, a control terminal, and the like.
  • the video transmission method, playback terminal, movable platform and video transmission system of the present application it is possible to control the image transmission device to transmit to the playback terminal in one of the first image transmission mode and the second image transmission mode.
  • the video data is transmitted, so that the user can experience a low-latency or high-quality image transmission experience
  • the recording frame rate can be equal to or lower than the first frame rate in the first image transmission mode , so as to meet the user's demand for low frame rate recording, thereby saving the storage space occupied by the recorded video data
  • the recording frame rate in the second image transmission mode the recording frame rate can be made greater than or equal to the second frame rate, so as to meet the Provide users with more recording experience and choices to improve user experience.
  • a time-travel experience machine with glasses the glasses need to be inserted into an SD card (for example, the card memory contains aircraft video) playback requirements.
  • SD card for example, the card memory contains aircraft video
  • the image transmission processing chip on the glasses side can only decode the H.264 encoding format, but cannot decode the H.265 encoding format (for example, the H.265 encoding format video recorded on the aircraft side cannot be broadcast, and the H.265 encoding format cannot be broadcast.
  • the encoding format has a lower bit rate and better picture quality), and the processing capacity is insufficient, only 4Kp30 processing capacity, about 265420800pixel/s (for example, 4Kp60, 4Kp50, 1080p240 and other video broadcasts recorded on the aircraft side No.
  • the usual methods are: 1. There are no restrictions on the video encoding format and video specifications on the aircraft side. In this way, the receiving glasses end cannot play back some video files, and the experience is not very good; 2. Restrict the video encoding format and video specifications of the aircraft end, and only record the video that can be played back by the glasses end. This limits the means for users to record videos, affects the production of films, and the experience is not good. 3. Record two files at the same time and place them in two video files, one file is used for playback on computer terminals, and the other is used for glasses card playback.
  • the present application also provides a video processing method, the method includes: acquiring a first video code stream, a second video code stream and an audio code stream, wherein the first video code stream is used for The first type of playback terminal plays, the encoding format of the first video stream corresponds to the encoding format that can be played by the first type of playback terminal, and the second video stream is used for the second type of playback terminal to play, so The encoding format of the second video code stream corresponds to the encoding format that can be played by the second type of playback terminal, and the video specification of the second video code stream is less than or equal to the threshold specification;
  • the second video code stream and the audio code stream are encapsulated into a video file in a preset file format; the video file is stored in the first memory for playback by the first type of playback terminal or the second type of playback terminal the video file.
  • the video processing method of the present application by encapsulating the first video code stream, the second video code stream and the audio code stream in one video file, enables the video file to be played on various types of playback terminals, and overcomes the problem of the second video code stream.
  • the capability limitation of the playback terminal chip improves the video playback effect of the playback terminal of the first type, and compared with the solution of encapsulating the first video code stream and the second video code stream independently, the present application reduces one audio code stream and reduces the The processing load of the processor is reduced, the total code rate of the storage is reduced, and the user experience is improved.
  • FIG. 6 and FIG. 6 are schematic flowcharts of the video processing method in an embodiment of the present invention.
  • the video processing method 600 of the present application includes the following steps: in step S601 , acquiring a first video code stream, a second video code stream, and an audio code stream, wherein the first video code stream and audio code stream are obtained.
  • the video code stream is used for playback by the first type of playback terminal
  • the encoding format of the first video code stream corresponds to the encoding format that can be played by the first type of playback terminal
  • the second video code stream is used for the second type of playback.
  • the encoding format of the second video stream corresponds to the encoding format that can be played by the second type of playback terminal, and the video specification of the second video stream is less than or equal to the threshold specification; in step S602, the The first video code stream, the second video code stream and the audio code stream are encapsulated into video files in a preset file format; in step S603, the video files are stored in the first memory for the purpose of the The first type of playback terminal and/or the second type of playback terminal plays back the video file.
  • the first video stream may be referred to as the main video stream.
  • the encoding format of the first video stream is: H.264 encoding format or H.265 encoding format, or other encoding formats.
  • the mobile terminal may include, but is not limited to, a mobile phone, a tablet computer, and the like.
  • the encoding format of the video stream used for the first type of playback terminal is not limited here, it can be more selective, and when it uses the H.265 encoding format, it can be compared with the H.264 encoding format. It has a lower bit rate and better picture quality.
  • the video specification of the first video stream is greater than or equal to the threshold specification
  • the threshold specification can be reasonably set according to actual needs
  • the threshold specification can be 4Kp30
  • the video specification of the first video stream It can exceed 4Kp30, such as 4Kp60, 4Kp50, 1080p60, 1080p120, 1080p240 or other video specifications, so as to obtain better video playback effects and improve user experience.
  • the second video stream may also be called an auxiliary video stream, and the encoding format of the second video stream corresponds to the encoding format that can be played by the second type of playback terminal.
  • the encoding format of the second video stream is the H.264 encoding format
  • the video specification of the second video stream is less than or equal to the threshold specification, such as not exceeding 4Kp30, for example, it can be 4Kp30, 1080P30, etc.
  • the first video code stream, the second video code stream and the audio code stream are encapsulated into video files in a preset file format, such as MP4 or MOV or other file formats.
  • the encapsulation process can be any scheme known to those skilled in the art that can encapsulate the three in one video file.
  • the track data in the video file can also include: Thumbnail, data header, creation date, source data, stream data, etc.
  • the first video code stream and the second video code stream are encapsulated in one file, and the same audio code stream can be used, and the same thumbnail image can also be used. Therefore, reducing the number of One channel of audio stream is saved, and the storage of one track of thumbnails and other information is reduced, which reduces the CPU load during the recording process and the total bit rate of the storage and writing card.
  • the auxiliary channel video stream is produced for playback on the glasses side, which overcomes the capability limitation of the glasses chip, and retains the main channel code stream's support for H.265 encoding and using 4Kp30 or higher specifications.
  • the track type corresponding to the second video stream is a type that cannot be recognized by the first type of playback terminal, such as a non-standard private type, and the glasses-side program can recognize the second video stream and It can be played back correctly, but it can achieve the effect that cannot be recognized by the computer (PC) terminal, the video playback on the mobile phone, the video post-editing software, etc., then when the SD card is inserted into the computer (PC) terminal or mobile terminal for playback, the user Then, the existence of the second video stream will not be perceived, which optimizes the user experience.
  • the first type of playback terminal such as a non-standard private type
  • An embodiment of the present application further provides a video processing apparatus, and the video processing apparatus can be used as an execution body of the foregoing video processing method.
  • the video processing apparatus 800 may include one or more processors 801, a memory 802, a communication interface, and the like. These components are interconnected by a bus system and/or other form of connection mechanism (not shown).
  • the memory 802 is used for storing executable instructions, for example, for storing corresponding steps and program instructions for implementing the video processing method 600 according to the embodiment of the present application.
  • One or more computer program products may be included, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory.
  • the volatile memory may include, for example, random access memory (RAM) and/or cache memory, or the like.
  • the non-volatile memory may include, for example, a read only memory (ROM), a hard disk, a flash memory (eg, an SD card), and the like.
  • the memory 802 includes a first memory (such as an SD card), the first memory is used to store the video file, and the first memory is pluggable and provided in the video processing apparatus 800, when other terminal equipment is needed When playing a video file, the first memory can be taken out from the video processing device and inserted into a playing terminal such as glasses to play back the video.
  • a first memory such as an SD card
  • the video processing apparatus 800 may have a communication interface for communicating with other devices, including wired or wireless communication. I won't go into details here.
  • the video processing device 800 may be any device having an image data processing function, such as a photographing device, a movable platform mounted with a photographing device, etc., which are not specifically limited herein.
  • the present application also provides a movable platform.
  • the movable platform 500 includes one or more processors 501, a photographing device 505, a memory 502, an image transmission device 503, a photographing device 504, a communication interface (not shown) etc. These components are interconnected by a bus system and/or other form of connection mechanism (not shown).
  • One or more processors 501 are configured to execute the program instructions stored in the memory 502, so that the processors execute the aforementioned video processing method 600, the structure description of the movable platform 500 and the related video processing method 600. For details of the steps, reference may be made to the foregoing description, which will not be repeated here.
  • the memory 502 is used for storing executable instructions, for example, for storing corresponding steps and program instructions for implementing the video processing method 600 according to the embodiment of the present application.
  • One or more computer program products may be included, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory.
  • the volatile memory may include, for example, random access memory (RAM) and/or cache memory, or the like.
  • the non-volatile memory may include, for example, a read only memory (ROM), a hard disk, a flash memory (eg, an SD card), and the like.
  • the memory 502 includes a first memory (such as an SD card), the first memory is used to store the video file, and the first memory is pluggable and provided in the removable platform 500, when other terminal devices are needed.
  • the first memory can be taken out from the video processing device and inserted into a playing terminal such as glasses to play back the video.
  • the video processing method, device and movable platform of the present application encapsulate the first video code stream, the second video code stream and the audio code stream in one video file, so that the video file can be used in a variety of type of playback terminal for playback, overcoming the capability limitation of the second type of playback terminal chip, improving the video playback effect of the first type of playback terminal, and compared with the solution of encapsulating the first video stream and the second video stream independently , the application reduces one audio code stream, reduces the processing load of the processor, reduces the total code rate of storage, and improves the user experience.
  • an embodiment of the present invention further provides a computer storage medium, on which a computer program is stored.
  • One or more computer program instructions may be stored on the computer-readable storage medium, and the processor may execute the program instructions stored in the storage device to implement (implemented by the processor) the embodiments of the present invention described herein. function and/or other desired functions, for example, to execute the corresponding steps of the video transmission methods 200 and 300 according to the embodiments of the present application, or to execute the corresponding steps of the video processing method 600 according to the embodiments of the present application, the computer can Various application programs and various data, such as various data used and/or generated by the application program, may also be stored in the read storage medium.
  • the computer storage medium may include, for example, a memory card for a smartphone, a storage unit for a tablet computer, a hard disk for a personal computer, read only memory (ROM), erasable programmable read only memory (EPROM), portable compact disk Read only memory (CD-ROM), USB memory, or any combination of the above storage media.
  • the computer-readable storage medium can be any combination of one or more computer-readable storage media.
  • the disclosed apparatus and method may be implemented in other manners.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components may be combined or May be integrated into another device, or some features may be omitted, or not implemented.
  • Various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof.
  • a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all of the functions of some modules according to the embodiments of the present invention.
  • DSP digital signal processor
  • the present invention may also be implemented as apparatus programs (eg, computer programs and computer program products) for performing part or all of the methods described herein.
  • Such a program implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such signals may be downloaded from Internet sites, or provided on carrier signals, or in any other form.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer Security & Cryptography (AREA)
  • Studio Devices (AREA)
  • Television Signal Processing For Recording (AREA)

Abstract

一种视频传输方法及系统、视频处理方法及装置、播放终端、可移动平台。视频传输方法(200)应用于播放终端,播放终端与可移动平台通信连接,可移动平台包括图传装置和拍摄装置,拍摄装置用于拍摄视频数据,图传装置用于获取视频数据并向播放终端发送视频数据,包括:获取图传设置信息和录像规格设置信息(S201),录像规格设置信息包括录像分辨率和录像帧率,当图传设置信息为第一图传模式时,图传帧率为第一帧率,录像帧率大于或等于第一帧率,当图传设置信息为第二图传模式时,图传帧率为第二帧率,录像帧率小于或等于第二帧率,第一帧率小于第二帧率;基于图传设置信息和录像规格设置信息,发送控制信号(S202)。

Description

视频传输方法及系统、视频处理方法及装置、播放终端、可移动平台
说明书
技术领域
本发明总地涉及视频技术领域,更具体地涉及一种视频传输方法及系统、视频处理方法及装置、播放终端、可移动平台。
背景技术
带眼镜的穿越机体验机,追求极致图传体验(低延时、高画质),支持高帧率录像,目前穿越机的相机录像格式和图传格式一一绑定,帧率必须相同,限制用户对录像格式的可选择性。另外,带眼镜的穿越机体验机,眼镜有插SD(Secure Digital)卡(SD卡内存有飞机端录像)回放需求,但眼镜端图传处理芯片处理能力受限,有时无法回放部分视频文件,影响用户使用体验。
因此,鉴于上述问题的存在,一种视频传输方法及系统、视频数据处理方法及装置、播放终端、可移动平台。
发明内容
为了解决上述问题中的至少一个而提出了本发明。具体地,本发明第一方面提供一种视频传输方法,应用于播放终端,所述播放终端与可移动平台通信连接,所述可移动平台包括图传装置和拍摄装置,所述拍摄装置用于拍摄视频数据,所述图传装置用于获取所述视频数据并向所述播放终端发送所述视频数据,所述方法包括:获取图传设置信息和录像规格设置信息,其中,所述图传设置信息包括第一图传模式和第二图传模式中的一种模式设置信息,所述录像规格设置信息包括录像分辨率和录像帧率,当所述图传设置信息为所述第一图传模式时,图传帧率为第一帧率,所述录像帧率大于或等于所述第一帧率,当图传设置信息为第二图传模式时,图传帧率为第二帧率,所述录像帧率小于或等于所述第二帧率,所述第一帧率小于所述第二帧率;基于所述图传设置信息和录像规格设置信息,发送控制信号,所述控制信号用于控制所述拍摄装置在拍摄视频数据,并控制所述图传装置基于所述图传设置信息向所述播放终端发送所述视频数据,以及控制所述拍摄装置基于所述录像设置信息录制视频数据。
本发明第二方面提供一种视频传输方法,应用于可移动平台,所述播放终端与可移动平台通信连接,所述可移动平台包括图传装置和拍摄装置,所述拍摄装置用于拍摄视频数据,所述播放终端用于获取所述视频数据并播放,所述图传装置用于获取所述视频数据并向所述播放终端发送所述视频数据,所述方法包括:获取所述播放终端发送的控制信号,所述控制信号用于控制所述拍摄装置在拍摄视频数据时,控制所述图传装置基于图传设置 信息向所述播放终端发送所述视频数据,以及控制所述拍摄装置基于录像设置信息录制视频数据,其中,所述图传设置信息包括第一图传模式和第二图传模式中的一种模式设置信息,所述录像规格设置信息包括录像分辨率和录像帧率,当所述图传设置信息为所述第一图传模式时,图传帧率为第一帧率,所述录像帧率大于或等于所述第一帧率,当图传设置信息为第二图传模式时,图传帧率为第二帧率,所述录像帧率小于或等于所述第二帧率,所述第一帧率小于所述第二帧率;响应于所述控制信号,控制所述拍摄装置拍摄视频数据,并控制所述图传装置基于图传设置信息向所述播放终端发送所述视频数据,以及控制所述拍摄装置基于录像设置信息录制视频数据。
本申请第三方面提供一种播放终端,所述播放终端包括:
存储器,用于存储可执行的程序指令;
通信接口,用于使所述播放终端和可移动平台进行通信,其中,所述可移动平台包括图传装置和拍摄装置,所述拍摄装置用于拍摄视频数据,所述图传装置用于获取所述视频数据并向所述播放终端发送所述视频数据;
一个或多个处理器,用于执行所述存储器中存储的所述程序指令,使得所述处理器执行以下步骤:获取图传设置信息和录像规格设置信息,其中,所述图传设置信息包括第一图传模式和第二图传模式中的一种模式设置信息,所述录像规格设置信息包括录像分辨率和录像帧率,当所述图传设置信息为所述第一图传模式时,图传帧率为第一帧率,所述录像帧率大于或等于所述第一帧率,当图传设置信息为第二图传模式时,图传帧率为第二帧率,所述录像帧率小于或等于所述第二帧率,所述第一帧率小于所述第二帧率;基于所述图传设置信息和录像规格设置信息,发送控制信号,所述控制信号用于控制所述拍摄装置在拍摄视频数据,并控制所述图传装置基于所述图传设置信息向所述播放终端发送所述视频数据,以及控制所述拍摄装置基于所述录像设置信息录制视频数据。
本申请第四方面提供一种可移动平台,所述可移动平台包括
拍摄装置,所述拍摄装置用于拍摄视频数据;
图传装置,用于获取所述视频数据并向所述播放终端发送所述视频数据,
存储器,用于存储可执行的程序指令;
一个或多个处理器,用于执行所述存储器中存储的所述程序指令,使得所述处理器执行以下步骤:获取所述播放终端发送的控制信号,所述控制信号用于控制所述拍摄装置在拍摄视频数据时,控制所述图传装置基于图传设置信息向所述播放终端发送所述视频数据,以及控制所述拍摄装置基于录像设置信息录制视频数据,其中,所述图传设置信息包括第一图传模式和第二图传模式中的一种模式设置信息,所述录像规格设置信息包括录像分辨率和录像帧率,当所述图传设置信息为所述第一图传模式时,图传帧率为第一帧率,所述 录像帧率大于或等于所述第一帧率,当图传设置信息为第二图传模式时,图传帧率为第二帧率,所述录像帧率小于或等于所述第二帧率,所述第一帧率小于所述第二帧率;响应于所述控制信号,控制所述拍摄装置拍摄视频数据,并控制所述图传装置基于图传设置信息向所述播放终端发送所述视频数据,以及控制所述拍摄装置基于录像设置信息录制视频数据。
本申请第五方面提供一种视频传输系统,所述视频传输系统包括:前述的播放终端;以及前述的可移动平台。
本申请第六方面提供一种视频处理方法,所述方法包括:获取第一视频码流、第二视频码流和音频码流,其中,所述第一视频码流用于第一类播放终端播放,所述第一视频码流的编码格式与所述第一类播放终端所能播放的编码格式相对应,所述第二视频码流用于第二类播放终端播放,所述第二视频码流的编码格式与所述第二类播放终端所能播放的编码格式相对应,第二视频码流的视频规格小于或等于阈值规格;将所述第一视频码流和所述第二视频码流和音频码流封装为预设文件格式的视频文件;将所述视频文件存储至第一存储器中,以供所述第一类播放终端和/或所述第二类播放终端回放所述视频文件。
本申请第七方面提供一种视频处理装置,所述装置包括:存储器,用于存储可执行的程序指令;一个或多个处理器,用于执行所述存储器中存储的所述程序指令,使得所述处理器执行前述的视频处理方法;所述存储器包括第一存储器,所述第一存储器用于存储所述视频文件,并且所述存储器可插拔地设置于视频处理装置内。
本申请第八方面提供一种可移动平台,所述可移动平台包括:拍摄装置,所述拍摄装置用于拍摄视频数据;存储器,用于存储可执行的程序指令;一个或多个处理器,用于执行所述存储器中存储的所述程序指令,使得所述处理器执行前述的视频处理方法;所述存储器包括第一存储器,所述第一存储器用于存储所述视频文件,并且所述第一存储器可插拔地设置于视频处理装置内。
本申请第九方面提供一种计算机存储介质,其上存储有计算机程序,所述计算机程序被处理器执行时实现前述的视频传输方法,或者,实现前述的视频处理方法。
根据本申请的视频传输方法,可以控制图传装置可以在第一图传模式和第二图传模式中的一个模式下向播放终端传输视频数据,从而使用户体验到低延时或高画质的图传体验,并且本申请的视频传输方法中可以在第一图传模式下,使录像帧率等于或低于第一帧率,从而可以满足用户对于低帧率录像的需求,进而节省录像的视频数据所占用的存储空间,还可以在第二图传模式下,使录像帧率大于或等于第二帧率,从而可以满足用户多种录像帧率的需求,给用户提供更多的录像体验和选择,提升用户的使用体验。
另外,本申请的视频处理方法,通过将第一视频码流、第二视频码流和音频码流封装在一个视频文件中,使得该视频文件可以在多种类型的播放终端进行播放,克服了第二类播放终端芯片的能力限制,提升了第一类播放终端的视频播放效果,并且相比将第一视频码流和第二视频码流独立封装的方案,本申请减少了一路音频码流,降低了处理器的处理负载,降低了存储总码率,提升了用户体验。
附图说明
为了更清楚地说明本发明实施例中的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本发明的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动性的前提下,还可以根据这些附图获得其他的附图。
图1示出了本申请一个实施例中的视频传输系统的示意性框图;
图2示出了本申请一个实施例中的视频传输方法的示意性流程图;
图3示出了本申请另一个实施例中的视频传输方法的示意性流程图;
图4示出了本申请一个实施例中的播放终端的示意性框图;
图5示出了本申请一个实施例中的可移动平台的示意性框图;
图6示出了本申请一个实施例中的视频处理方法的示意性流程图;
图7示出了本申请一个实施例中的视频文件的封装结构的示意图;
图8示出了本申请一个实施例中的视频处理装置的示意性框图。
具体实施方式
为了使得本发明的目的、技术方案和优点更为明显,下面将参照附图详细描述根据本发明的示例实施例。显然,所描述的实施例仅仅是本发明的一部分实施例,而不是本发明的全部实施例,应理解,本发明不受这里描述的示例实施例的限制。基于本发明中描述的本发明实施例,本领域技术人员在没有付出创造性劳动的情况下所得到的所有其它实施例都应落入本发明的保护范围之内。
在下文的描述中,给出了大量具体的细节以便提供对本发明更为彻底的理解。然而,对于本领域技术人员而言显而易见的是,本发明可以无需一个或多个这些细节而得以实施。在其他的例子中,为了避免与本发明发生混淆,对于本领域公知的一些技术特征未进行描述。
应当理解的是,本发明能够以不同形式实施,而不应当解释为局限于这里提出的实 施例。相反地,提供这些实施例将使公开彻底和完全,并且将本发明的范围完全地传递给本领域技术人员。
在此使用的术语的目的仅在于描述具体实施例并且不作为本发明的限制。在此使用时,单数形式的“一”、“一个”和“所述/该”也意图包括复数形式,除非上下文清楚指出另外的方式。还应明白术语“组成”和/或“包括”,当在该说明书中使用时,确定所述特征、整数、步骤、操作、元件和/或部件的存在,但不排除一个或更多其它的特征、整数、步骤、操作、元件、部件和/或组的存在或添加。在此使用时,术语“和/或”包括相关所列项目的任何及所有组合。
为了彻底理解本发明,将在下列的描述中提出详细的结构,以便阐释本发明提出的技术方案。本发明的可选实施例详细描述如下,然而除了这些详细描述外,本发明还可以具有其他实施方式。
下面结合附图,对本发明的一些实施方式作详细说明。在不冲突的情况下,下述的实施例及实施例中的特征可以相互组合。
首先,本申请提供一种视频传输系统,该视频传输系统可以包括可移动平台、播放终端和控制终端等,其中,可移动平台可以包括飞机例如无人机、车辆、船、机器人等。播放终端可以包括用于播放图传数据,播放终端可以是手持的或者穿戴式的终端设备,例如,播放终端可以包括智能手机、平板电脑、桌上型电脑、计算机、眼镜、手套、头盔、麦克风或者其中任意的结合。控制终端可以包括例如手机、遥控器、平板电脑、笔记本等,该控制终端还可以是可移动平台的控制系统的一部分或者全部,还可以是可以通过软件、硬件或者软硬件结合的方式实现可移动平台的控制方法的计算机设备的部分或者全部。
可移动平台可以包括拍摄装置和图传装置,拍摄装置用于拍摄视频或图像数据;所述图传装置用于获取所述视频数据并向所述播放终端发送所述视频数据。拍摄装置可以是具有图像传感器的装置,例如相机、摄像机、摄像头或者还可以是其他装置。图像传感器(本文也称相机传感器)是用于捕获图像数据的器件,其例如可以为CMOS图像传感器、CCD图像传感器等。在一个示例中,拍摄装置还可以是以嵌入式影像系统的方式设置于可移动平台(例如飞行器、手持云台)的本体上。可选地,拍摄装置可以作为负载设置于可移动平台的本体上,用户可以通过播放终端或控制终端控制拍摄装置。
播放终端和/或控制终端上可以安装有应用程序(Application,简称APP),该应用程序用于接收用户输入的操作指令,播放终端和/或控制终端根据用户输入的操作指令,向飞行器或云台、承载体及负载中的一个或者多个提供控制信号,终端设备的控制信号可以导致负载控制,如控制拍摄装置例如相机或者其它拍摄装置的操作(捕获静止或者运动的视频或图像、变焦、开启或关闭、切换成像模式、改变视频或图像分辨率、改变焦距、改 变景深、改变曝光时间、改变可视角度或者视场)。
在一个具体示例中,如图1所示,视频传输系统可以包括飞机101、眼镜102、遥控装置103。
飞机101例如为无人机,无人机可以是旋翼型无人机,例如四旋翼无人机、六旋翼无人机、八旋翼无人机,也可以是固定翼无人机。飞机101可以包括动力系统、云台、拍摄装置、图传装置例如数字图传收发机、移动装置等。
动力系统用于为飞机101提供移动动力,动力系统包括螺旋桨、电机、电调中的一种或多种。拍摄装置可以通过云台搭载于可移动平台的主体上,并用于在可移动平台的移动过程中进行图像拍摄,还用于在可移动平台的移动过程中进行录像。拍摄装置包括但不限于多光谱成像仪、高光谱成像仪、可见光相机及红外相机等。云台为多轴传动及增稳系统,云台电机通过调整转动轴的转动角度来对图像拍摄装置的拍摄角度进行补偿,并通过设置适当的缓冲机构来防止或减小成像设备的抖动。数字图传收发机可以是指以数字通信链路传输信息(如图像数据)的装置,即数字图传收发机可用于可移动平台与其他终端进行交互信息,其他终端可以包括播放终端和控制终端中的一个或多个。例如,数字图传收发机可用于向播放终端发送拍摄装置拍摄的包括图像数据的数据包。移动装置用于控制可移动平台移动,即控制可移动平台的移动参数,移动参数包括移动速度、移动姿态、移动方向及移动轨迹等等。
本申请实施例中,播放终端可以指飞手端的眼镜设备或者为其他设备。该眼镜设备可以是指虚拟现实(Virtual Reality,VR)或增强现实(Augmented Reality,AR)等可以播放视频的眼镜设备。在穿越机竞赛中,飞手端的眼镜设备可以是指由飞手(即参赛选手)控制的眼镜设备。播放终端可以包括图传装置例如数字图传收发机及显示器等,该图传装置可以用于播放终端与可移动平台交互信息。例如,播放终端的图传装置可以用于接收可移动平台通过图传装置发送的数据包并根据该数据包进行播放,以呈现可移动平台拍摄的画面。
在如图1所示的系统中,飞机101搭载例如相机的拍摄装置进行录像、发送实时图传数据给地面端眼镜102,遥控装置103用于控制飞机101飞行、录像等,眼镜102用于接收并显示飞机端的图传数据例如图像或视频、设置格式、控制飞机端录像等。
带眼镜的穿越机体验机,追求极致图传体验(例如低延时、高画质),支持高帧率录像。本文中,(图传)高画质模式:由于天空端(例如飞机端)和地面端(例如眼镜端)的信道宽度是固定的(比如40Mbps),传输的帧率越高,每帧能分摊到的平均带宽就越小,画质就越差。所以图传低帧率60fps会比图传高帧率120fps画质好,因此可以称图传低帧率60fps为高画质模式。(图传)低延时模式:由于帧率越高,屏幕刷新速度越快, 人眼能够感受到的延时就越小,所以称图传高帧率120fps为低延时模式。
目前常规的方案中,主要存在以下几处缺点:
1、拍摄装置例如相机的录像格式和图传格式一一绑定,帧率必须相同,这样的方式具有以下确定:a)图传低延时模式:相机设置只能选1080P120录像,这么高帧率的录像有时不是用户想要的。比如有时为了节省SD卡存储空间,用户更倾向于使用1080P60,但用户无法选中这个模式,除非切换到高画质模式;b)图传高画质模式:相机设置只能选4KP60或1080P60录像,无法选中1080P120(例如4倍慢动作)这个慢动作录像模式,限制了用户的发挥。
2、相机录像格式、图传格式一方发生变化时导致另一方跟着变化,会让用户觉得奇怪,用户需要学习、摸索,才能明白相机录像格式和图传格式的对应关系;
3、相机录像格式设置、图传格式设置分别放在不同的菜单下,用户设置图传高画质/低延时,无法看到对应的相机录像格式的变化;用户设置相机录像格式,无法看到图传高画质/低延时的变化。除非切换到另外的菜单里去查看,操作起来太复杂。
因此,鉴于上述问题的存在,本申请提供一种视频传输方法,应用于播放终端,所述播放终端与可移动平台通信连接,所述可移动平台包括图传装置和拍摄装置,所述拍摄装置用于拍摄视频数据,所述图传装置用于获取所述视频数据并向所述播放终端发送所述视频数据,所述方法包括:获取图传设置信息和录像规格设置信息,其中,所述图传设置信息包括第一图传模式和第二图传模式中的一种模式设置信息,所述录像规格设置信息包括录像分辨率和录像帧率,当所述图传设置信息为所述第一图传模式时,图传帧率为第一帧率,所述录像帧率大于或等于所述第一帧率,当图传设置信息为第二图传模式时,图传帧率为第二帧率,所述录像帧率小于或等于所述第二帧率,所述第一帧率小于所述第二帧率;基于所述图传设置信息和录像规格设置信息,发送控制信号,所述控制信号用于控制所述拍摄装置在拍摄视频数据,控制所述图传装置基于所述图传设置信息向所述播放终端发送所述视频数据,以及控制所述拍摄装置基于所述录像设置信息录制视频数据。
根据本申请的视频传输方法,可以控制图传装置可以在第一图传模式和第二图传模式中的一个模式下向播放终端传输视频数据,从而使用户体验到低延时或高画质的图传体验,并且本申请的视频传输方法中可以在第一图传模式下,使录像帧率等于或低于第一帧率,从而可以满足用户对于低帧率录像的需求,进而节省录像的视频数据所占用的存储空间,还可以在第二图传模式下,使录像帧率大于或等于第二帧率,从而可以满足用户多种录像帧率的需求,给用户提供更多的录像体验和选择,提升用户的使用体验。
下面,参考图2基于播放终端的角度对本申请的视频传输方法200进行描述。应用于播放终端,所述播放终端与可移动平台通信连接,所述可移动平台包括图传装置和拍摄装置,所述拍摄装置用于拍摄视频数据,所述图传装置用于获取所述视频数据并向所述播放终端发送所述视频数据,
首先,如图2所示,本申请的视频传输方法200包括:在步骤S201中,获取图传设置信息和录像规格设置信息,其中,所述图传设置信息包括第一图传模式和第二图传模式中的一种模式设置信息,所述录像规格设置信息包括录像分辨率和录像帧率,当所述图传设置信息为所述第一图传模式时,图传帧率为第一帧率,所述录像帧率大于或等于所述第一帧率,当图传设置信息为第二图传模式时,图传帧率为第二帧率,所述录像帧率小于或等于所述第二帧率,所述第一帧率小于所述第二帧率;在步骤S202中,基于所述图传设置信息和录像规格设置信息,发送控制信号,所述控制信号用于控制所述拍摄装置在拍摄视频数据,控制所述图传装置基于所述图传设置信息向所述播放终端发送所述视频数据,以及控制所述拍摄装置基于所述录像设置信息录制视频数据。
在一个示例中,第一图传模式可以为图传高画质模式,当所述图传设置信息为所述第一图传模式时,图传帧率为第一帧率,可选地,所述第一帧率小于或等于60fps,例如,第一帧率等于60fps,如表格1中所示;示例性地,第二图传模式可以为低延时模式,当图传设置信息为第二图传模式时,图传帧率为第二帧率,可选地,所述第二帧率大于60fps且小于或等于120fps,例如,第二帧率可以等于120fps,如表格1中所示。
在一个示例中,获取拍摄装置的图传设置信息和录像规格设置信息,包括:首先,响应于第一用户指令,控制所述播放终端的显示器显示设置界面,所述设置界面用于显示:所述第一图传模式对应的选项和所述第二图传模式对应的选项;可选地,所述设置界面还可以是用于显示其他的拍摄装置的设置信息,例如相机参数等,播放终端可以设置有多个功能按键例如实体键(例如菜单键等)或者虚拟功能键等,可以基于用户通过功能按键输入的第一用户指令,控制播放终端的显示器显示设置界面。接着,响应于用户输入的模式选择指令,确定所述第一图传模式和所述第二图像模式中的一个作为所述图传设置信息,例如获取用户通过例如功能按键输入的模式选择指令,响应于用户输入的模式选择指令,确定所述第一图传模式和所述第二图像模式中的一个作为所述图传设置信息;再一个示例中,设置界面还可以显示多个选项,例如图传规格、相机参数、视频规格等,可以通过获取用户通过功能按键例如方向键、音量键等输入的指令,在多个选项中选定一个,并进而对该选项下设置的多个子选项进行选择,例如图传规格选项下的低延时模式、高画质模式等。最后,基于已确定的所述图传设置信息,确定所述录像规格设置信息。通过将图传设置信息和录像规格设置信息的设置联动,实现了拍摄装置视频录制和图传帧率的设置联动, 可以保证用户获得期望的视频效果例如低延时或高画质,并且,用户可以在播放终端实施观察到符合主观需求的画面。并且,通过将相机录像格式设置、图传高画质和低延时切换选项均设置于相机的菜单下,方便用户快速查看到状态变化。
可以播放终端可以基于已确定的所述图传设置信息,自动确定所述录像规格设置信息,或者,还可以基于用户指令,手动确定录像规格设置信息。在本申请中,所述录像规格设置信息包括录像分辨率和录像帧率,也即录像规格(本文也称录像格式)可以包括录像分辨率和录像帧率,如表格1中所示,录像格式4KP60,则表示录像分辨率为4KP,录像帧率为60fps,再例如录像格式1080P60,则表示录像分辨率为1080P,录像帧率为60fps;又例如录像格式1080P120,则表示录像分辨率为1080P,录像帧率为120fps。可选地,录像分辨率包括但不限于4KP、720P或1080P,或者其他的分辨率。
在一个示例中,基于已确定的所述图传设置信息,确定所述录像规格设置信息,包括:当所述图传设置信息为第一图传模式时,响应于用户输入的录像规格设置指令,控制所述播放终端的显示器在所述设置界面显示多个第一类录像规格,所述多个第一类录像规格的录像帧率大于或等于所述第一帧率;响应于用户输入的规格选择指令,确定所述多个第一类录像规格中的一个作为所述录像规格设置信息,例如第一类录像规格可以包括但不限于4KP60、1080P60、1080P120,或者其他的录像规格。例如,当用户确定录像规格设置信息为1080P120时,则可以控制拍摄装置进行慢动作录像模式例如四倍慢动作,使得用户获得更多的视频效果体验。
本申请实施例中,图传设置信息和录像规格设置信息联动,例如当所述图传设置信息为所述第一图传模式例如高画质时,所述录像帧率大于或等于所述第一帧率,例如,如表格1中所示,例如当所述图传设置信息为所述第一图传模式例如高画质时,则第一帧率可以为60fps,录像帧率大于或等于所述第一帧率,例如录像帧率可以为60fps、120fps,因此,在高画质模式下,录像格式可以在4KP60、1080P60、1080P120中选择,当然在具有其他的帧率和分辨率时,录像格式还可以多于表格中列出的3种。
在另一个示例中,基于已确定的所述图传设置信息,确定所述录像规格设置信息,包括:当所述图传设置信息为第二图传模式时,基于用户输入的录像规格设置指令,控制所述播放终端的显示器在所述设置界面显示多个第二类录像规格,所述多个第二类录像规格的录像帧率小于或等于所述第二帧率;基于用户输入的规格选择指令,确定所述多个第二类录像规格中的一个作为所述录像规格设置信息。例如第二类录像规格可以包括但不限于1080P60、1080P120,或者其他的录像规格。例如,在第二图像模式下,当用户确定录像规格设置信息为1080P60时,可以在获得低延时的前提下,通过降低录像帧率节省录像视频所需占用的存储空间,例如SD卡存储空间。
本申请实施例中,图传设置信息和录像规格设置信息联动,例如当所述图传设置信息为所述第二图传模式例如低延时时,所述录像帧率小于或等于所述第二帧率,例如,如表格1中所示,例如当所述图传设置信息为所述第二图传模式例如低延时时,则第二帧率可以为120fps,录像帧率小于或等于所述第一帧率,例如录像帧率可以为60fps、120fps,因此,在低延时模式下,录像格式可以在1080P60、1080P120中选择,当然在具有其他的帧率和分辨率时,录像格式还可以多于表格中列出的2种。
拍摄装置录像和图传都可以进行主动丢帧,例如存在图传帧率120fps丢帧到图传60fps的情况,再例如存在图传帧率120fps,拍摄装置录像主动丢帧到60fps录像的情况。
在本申请实施例中,图传设置优先于录像规格设置,例如,图传偏好(也即图传模式)为高画质模式时,当对录像格式进行设置时,可以仅显示第一类录像规格,例如如表格1前3行所示的4KP60、1080P60、1080P120,当图传偏好(也即图传模式)为低延时模式时,当对录像格式进行设置时,可以仅显示第二类录像规格,例如如表格1后两行所示的1080P60、1080P120。通常拍摄装置录像规格切换不会导致图传模式的被动切换。
表格1:
Figure PCTCN2020138557-appb-000001
在一个示例中,本申请的方法还包括:基于所述图传设置信息和录像规格设置信息,确定所述拍摄装置的图像传感器的采集帧率,其中,所述采集帧率大于或等于所述录像帧率和所述图传帧率中的较大者。采集帧率是指图像传感器每秒采集的图像帧数。
图像传感器仅在必要时才输出高帧率例如第二帧率,第二帧率例如为120fps,例如,当所述图传设置信息为所述第一图传模式时,且所述录像帧率等于所述第二帧率时,所述采集帧率等于第二帧率例如为120fps;又例如,当所述图传设置信息为所述第二图传模式,所述采集帧率等于第二帧率例如为120fps,则在第一图像模式的其他录像规格设置和图传格式设置,例如录像帧率和图传帧率均小于第二帧率时,例如均为第一帧率(例如60fps)时,则图像传感器则可以不输出例如120fps的高帧率,从而可以降低系统的 负载和功耗。
在一个示例中,当所述拍摄装置以第一分辨率和第一录像帧率进行录像时,也即处于录像模式时,所述方法还包括:获取用户输入的模式切换指令,所述模式切换指令用于指示将当前运行的所述第一图传模式切换为第二图传模式或者将当前运行的所述第二图传模式切换为所述第一图传模式;将所述模式切换指令发送给所述可移动平台,以将当前运行的所述第一图传模式切换为第二图传模式或者将当前运行的所述第二图传模式切换为所述第一图传模式;根据切换后的图传模式,更新录像规格设置信息,并将更新后的录像规格设置信息发送给可移动平台,以控制所述拍摄装置以更新后的录像规格设置信息进行录像,其中,所述更新后的录像规格设置信息包括第二分辨率和所述第一录像帧率,所述第二分辨率小于或等于所述第一分辨率,示例性地,根据切换后的图传模式,播放终端可以自动更新录像规格设置信息,或者,还可以根据用户的指令,更新录像规格设置信息。
当图传模式切换时,拍摄装置的录像分辨率或录像帧率尽量保持不变,如果无法做到就降低录像分辨率,也即使得更新后的录像分辨率低于或等于更新前的录像分辨率,通过这样的设置,可以使得拍摄装置的录像规格尽量不随着图传模式发生而变化,避免由于变化而使得用户去需要学习、摸索录像格式和图传格式的对应关系,简化了用户操作,节约了用户时间,提升了用户体验。
在一个具体示例中,在录像模式下,当从低延时模式切换至高画质模式时,尽量保持录像格式不发生变化,例如表格2中所示。
表格2:
切换前(相机格式) 切换后(相机格式)
1080P60(120->60) 1080P60(60-60)
1080P120(120-120) 1080P120(120-120,此时图传帧率120->60
1080P60(120->60)也即是指采集帧率120fps丢帧到60fps,以获得录像帧率,在本申请实施例中,具体的丢帧处理方式可以采用等间隔丢帧方式或者称为等间隔提取方式,也可以采用图像融合方式等,在此不做具体限定。
在另一个具体示例中,在录像模式下,当从高画质模式切换至低延时模式时,尽量保持录像格式不发生变化,例如表格3中所示。
表格3:
切换前(相机格式) 切换后(相机格式)
4KP60(120-60) 1080P60(120->60)
1080P60(60-60或120->60) 1080P60(120->60)
进一步,当图传模式切换(例如高画质和低延时之间切换)时可以导致拍摄装置的录像规格被动切换,此时不管是拍摄装置的录像帧率或录像分辨率发生了变化,交互上都要给出提示,例如:当所述拍摄装置以第一分辨率和第一录像帧率进行录像时,所述方法还包括:当将所述第一图传模式切换为第二图传模式后,或者,当将所述第二图传模式切换为所述第一图传模式后,向所述可移动平台发送更新后的录像规格设置信息,以控制所述拍摄装置以更新后的录像规格设置信息进行录像,所述更新后的录像规格设置信息包括第二分辨率和所述第二录像帧率;当所述第二分辨率与所述第一分辨率不同和/或当所述第二录像帧率和所述第一录像帧率不同时,输出提示信息,以提示用户录像规格发生了变化。该提示信息可以为文字信息或者其他的类型的提示信息,播放终端的显示器可以以任意适合的预设方式区别化显示提示信息,预设方式区别化显示包括以下显示方式中的至少一种:高亮显示、闪烁显示、附加符号显示、区别化的底纹颜色显示或区别化的字体颜色显示,例如,以红色的底纹颜色显示提示信息,或者其他任意能够使用户能够快速识别发生变化的录像规格。
下面,参考图3基于可移动平台的角度对本申请的视频传输方法300进行描述。视频传输方法300应用于可移动平台,所述播放终端与可移动平台通信连接,所述可移动平台包括图传装置和拍摄装置,所述拍摄装置用于拍摄视频数据,所述播放终端用于获取所述视频数据并播放,所述图传装置用于获取所述视频数据并向所述播放终端发送所述视频数据。
参考图3,视频传输方法300包括以下步骤:在步骤S301中,获取所述播放终端发送的控制信号,所述控制信号用于控制所述拍摄装置在拍摄视频数据时,控制所述图传装置基于图传设置信息向所述播放终端发送所述视频数据,以及控制所述拍摄装置基于录像设置信息录制视频数据,其中,所述图传设置信息包括第一图传模式和第二图传模式中的一种模式设置信息,所述录像规格设置信息包括录像分辨率和录像帧率,当所述图传设置信息为所述第一图传模式时,图传帧率为第一帧率,所述录像帧率大于或等于所述第一帧率,当图传设置信息为第二图传模式时,图传帧率为第二帧率,所述录像帧率小于或等于所述第二帧率,所述第一帧率小于所述第二帧率;在步骤S302中,响应于所述控制信号,控制所述拍摄装置拍摄视频数据,并控制所述图传装置基于所述图传设置信息向所述播放终端发送所述视频数据,以及控制所述拍摄装置基于所述录像设置信息录制视频数据。
在一个示例中,第一图传模式可以为图传高画质模式,当所述图传设置信息为所述第一图传模式时,图传帧率为第一帧率,可选地,所述第一帧率小于或等于60fps,例如,第一帧率等于60fps,如表格1中所示;示例性地,第二图传模式可以为低延时模式,当图传设置信息为第二图传模式时,图传帧率为第二帧率,可选地,所述第二帧率大于60fps且小于或等于120fps,例如,第二帧率可以等于120fps,如表格1中所示。
可选地,当所述图传设置信息包括第一图传模式例如高画质模式时,所述录像规格设置信息包括多个第一类录像规格中的一个,所述多个第一类录像规格选项的录像帧率大于或等于所述第一帧率。例如第一类录像规格可以包括但不限于4KP60、1080P60、1080P120,或者其他的录像规格。例如,当用户确定录像规格设置信息为1080P120时,则可以控制拍摄装置进行慢动作录像模式例如四倍慢动作,使得用户获得更多的视频效果体验。
本申请实施例中,图传设置信息和录像规格设置信息联动,例如当所述图传设置信息为所述第一图传模式例如高画质时,所述录像帧率大于或等于所述第一帧率,例如,如表格1中所示,例如当所述图传设置信息为所述第一图传模式例如高画质时,则第一帧率可以为60fps,录像帧率大于或等于所述第一帧率,例如录像帧率可以为60fps、120fps,因此,在高画质模式下,录像格式可以在4KP60、1080P60、1080P120中选择,当然在具有其他的帧率和分辨率时,录像格式还可以多于表格中列出的3种。
在一个示例中,当所述图传设置信息为第二图传模式时,所述录像规格设置信息包括所述多个第二类录像规格中的一个,所述多个第二类录像规格的录像帧率小于或等于所述第二帧率。例如,如表格1中所示,例如当所述图传设置信息为所述第二图传模式例如低延时时,则第二帧率可以为120fps,录像帧率小于或等于所述第一帧率,例如录像帧率可以为60fps、120fps,因此,在低延时模式下,录像格式可以在1080P60、1080P120中选择,当然在具有其他的帧率和分辨率时,录像格式还可以多于表格中列出的2种。
拍摄装置录像和图传都可以进行主动丢帧,例如存在图传帧率120fps丢帧到图传60fps的情况,再例如存在图传帧率120fps,拍摄装置录像主动丢帧到60fps录像的情况。
在本申请实施例中,图传设置优先于录像规格设置,例如,图传偏好(也即图传模式)为高画质模式时,当对录像格式进行设置时,可以仅显示第一类录像规格,例如如表格1前3行所示的4KP60、1080P60、1080P120,当图传偏好(也即图传模式)为低延时模式时,当对录像格式进行设置时,可以仅显示第二类录像规格,例如如表格1后两行所示的1080P60、1080P120。通常拍摄装置录像规格切换不会导致图传模式的被动切换。
在一个示例中,视频传输方法300还包括:基于所述图传设置信息和录像规格设置信息,确定所述拍摄装置的图像传感器的采集帧率,其中,所述采集帧率大于或等于所述录像帧率和所述图传帧率中的较大者。可移动平台可以在获得图传设置信息和录像规 格设置信息后,自动确定所述拍摄装置的图像传感器的采集帧率,或者,还可以是播放终端基于所述图传设置信息和录像规格设置信息,确定所述拍摄装置的图像传感器的采集帧率,可移动平台从播放终端获取该采集帧率,并控制述拍摄装置的图像传感器基于该采集帧率采集图像数据。
图像传感器仅在必要时才输出高帧率例如第二帧率,第二帧率例如为120fps,例如,当所述图传设置信息为所述第一图传模式时,且所述录像帧率等于所述第二帧率时,所述采集帧率等于第二帧率例如为120fps;又例如,当所述图传设置信息为所述第二图传模式,所述采集帧率等于第二帧率例如为120fps,则在第一图像模式的其他录像规格设置和图传格式设置,例如录像帧率和图传帧率均小于第二帧率时,例如均为第一帧率(例如60fps)时,则图像传感器则可以不输出例如120fps的高帧率,从而可以降低系统的负载和功耗。
在一个示例中,当所述拍摄装置以第一分辨率和第一录像帧率进行录像时,视频传输方法300还包括:获取所述播放终端发送的模式切换指令;响应于所述模式切换指令,将当前运行的所述第一图传模式切换为第二图传模式或者将当前运行的所述第二图传模式切换为所述第一图传模式;获取更新后的录像规格设置信息,并控制所述拍摄装置以更新后的录像规格设置信息进行录像,其中,所述更新后的录像规格设置信息包括第二分辨率和所述第一录像帧率,所述第二分辨率小于或等于所述第一分辨率。该更新后的录像规格设置信息可以从播放终端获取的,或者,还可以是可移动平台根据切换后的图传模式,确定的更新后的录像规格设置信息。
当图传模式切换时,拍摄装置的录像分辨率或录像帧率尽量保持不变,如果无法做到就降低录像分辨率,也即使得更新后的录像分辨率低于或等于更新前的录像分辨率,通过这样的设置,可以使得拍摄装置的录像规格尽量不随着图传模式发生而变化,避免由于变化而使得用户去需要学习、摸索录像格式和图传格式的对应关系,简化了用户操作,节约了用户时间,降低了用户的学习成本,提升了用户体验。
关于视频传输方法300的相关步骤的一些细节描述还可以参考前文的视频传输方法200,在此不再赘述。
下面,参考图4对本申请的播放终端400进行描述,其中该播放终端可以作为前文的视频传输方法200执行主体。
在一个示例中,如图4所示,播放终端400包括一个或多个处理器401、存储器402、通信接口403、显示器404等。这些组件通过总线系统和/或其它形式的连接机构(未示出)互连。
存储器402用于存储可执行指令,例如用于存储用于实现根据本申请实施例的视频传 输方法200中的相应步骤和程序指令。存储器402可存储操作系统(包括用于处理各种基本系统服务和用于执行硬件相关任务的程序)、至少一个功能所需的应用程序(比如拍摄装置的控制功能、声音播放功能、图像播放功能等)等。这些应用程序中的一些可以经由无线通信从外部服务器下载。其它应用程序可以在制造或出厂时被安装在存储器402内,例如对于眼镜型播放终端400的基本功能(例如,接电话、打电话、接收消息、发送消息等)。常见的是应用程序被存储在存储器402中,并且由处理器执行以执行眼镜型播放终端400的操作(或功能)。可以包括一个或多个计算机程序产品,所述计算机程序产品可以包括各种形式的计算机可读存储介质,例如易失性存储器和/或非易失性存储器。所述易失性存储器例如可以包括随机存取存储器(RAM)和/或高速缓冲存储器(cache)等。所述非易失性存储器例如可以包括只读存储器(ROM)、硬盘、闪存(例如SD卡)等。
通信接口403可以用于播放终端400和可移动平台或其他设备之间进行通信,包括有线或者无线方式的通信。通信接口403可以为接入基于通信标准的无线网络,如WiFi(例如WiFi6)、软件无线电(Software-defined radio,简称SDR)、2G、3G、4G、5G或它们的组合。在一个示例性实施例中,通信接口403经由广播信道接收来自外部广播管理系统的广播信号或广播相关信息。在一个示例性实施例中,通信接口403还包括近场通信(NFC)模块,以促进短程通信。例如,在NFC模块可基于射频识别(RFID)技术,红外数据协会(IrDA)技术,超宽带(UWB)技术,蓝牙(BT)技术和其他技术来实现。
可选地,播放终端400还包括显示器404,用于显示由处理器组件产生的图像或视频的器件。显示器可以包括,例如,液晶显示(LCD)装置或有机发光二极管(OLED)装置。基于从处理器获取的数据,显示器可以显示各种图像,例如菜单、所选择的操作参数、拍摄模式选择界面、由图像传感器捕获的图像、用处理器等和/或从设备的用户界面接收的其它信息等。
可选地,播放终端400还包括输入装置(未示出)可以是用户用来输入各种操作的装置,例如可以包括功能键(比如例如,触摸键、按键(例如音量控制按键、开关按键等)、机械键、软键等、操作杆、麦克风和触摸屏等中的一个或多个,或者其他用于使得用户能够输入信息的用户输入装置。数据(例如,音频、视频、图像等)通过输入装置来获得。
例如眼镜的播放终端400还可以包括电源单元(未示出)能够被配置为接收外部电力或者提供内部电力,以便供应对包括在播放终端400中的元件和组件进行操作所需的适当电力。电源单元以包括电池,并且该电池可以被配置为嵌入在终端主体中,或者被配置为可从终端主体拆卸。根据本申请的实施方式的例如眼镜的播放终端400可以包括形成可视信息的虚像的光学单元,并且可以基于光学单元在各种位置中显示具有各种大小的虚像。
处理器401可以是中央处理单元(CPU)、图像处理单元(GPU)、专用集成电路(ASIC)、 现场可编程门阵列(FPGA)或者具有数据处理能力和/或指令执行能力的其它形式的处理单元,并且可以控制拍摄装置中的其它组件以执行期望的功能。处理器401能够执行存储器402中存储的所述指令,以执行本文描述的视频传输方法的相应步骤。例如,处理器401能够包括一个或多个嵌入式处理器、处理器核心、微型处理器、逻辑电路、硬件有限状态机(FSM)、数字信号处理器(DSP)、ISP、编码器或它们的组合。
在一个示例中,处理器401用于执行所述存储器中存储的程序指令,使得一个或多个处理器执行以下步骤:
获取图传设置信息和录像规格设置信息,其中,所述图传设置信息包括第一图传模式和第二图传模式中的一种模式设置信息,所述录像规格设置信息包括录像分辨率和录像帧率,当所述图传设置信息为所述第一图传模式时,图传帧率为第一帧率,所述录像帧率大于或等于所述第一帧率,当图传设置信息为第二图传模式时,图传帧率为第二帧率,所述录像帧率小于或等于所述第二帧率,所述第一帧率小于所述第二帧率;
基于所述图传设置信息和录像规格设置信息,发送控制信号,所述控制信号用于控制所述拍摄装置在拍摄视频数据时,控制所述图传装置基于所述图传设置信息向所述播放终端发送所述视频数据,以及控制所述拍摄装置基于所述录像设置信息录制视频数据。
可选地,所述第一帧率小于或等于60fps、所述第二帧率大于60fps且小于或等于120fps。
可选地,所述录像分辨率包括4KP、720P或1080P或者其他适合的分辨率。
在一个示例中,处理器401用于执行所述存储器中存储的程序指令,使得一个或多个处理器执行获取拍摄装置的图传设置信息和录像规格设置信息,包括:响应于第一用户指令,控制所述播放终端的显示器显示设置界面,所述设置界面用于显示:所述第一图传模式对应的选项和所述第二图传模式对应的选项;响应于用户输入的模式选择指令,确定所述第一图传模式和所述第二图像模式中的一个作为所述图传设置信息;基于已确定的所述图传设置信息,确定所述录像规格设置信息。
在一个示例中,处理器401用于执行所述存储器中存储的程序指令,使得一个或多个处理器执行基于已确定的所述图传设置信息,确定所述录像规格设置信息,包括:当所述图传设置信息为第一图传模式时,响应于用户输入的录像规格设置指令,控制所述播放终端的显示器在所述设置界面显示多个第一类录像规格,所述多个第一类录像规格的录像帧率大于或等于所述第一帧率;响应于用户输入的规格选择指令,确定所述多个第一类录像规格中的一个作为所述录像规格设置信息。
在一个示例中,处理器401用于执行所述存储器中存储的程序指令,使得一个或多个处理器执行基于已确定的所述图传设置信息,确定所述录像规格设置信息,包括:当所述图传设置信息为第二图传模式时,基于用户输入的录像规格设置指令,控制所述播放终端 的显示器在所述设置界面显示多个第二类录像规格,所述多个第二类录像规格的录像帧率小于或等于所述第二帧率;基于用户输入的规格选择指令,确定所述多个第二类录像规格中的一个作为所述录像规格设置信息。
在一个示例中,处理器401用于执行所述存储器中存储的程序指令,使得一个或多个处理器执行:基于所述图传设置信息和录像规格设置信息,确定所述拍摄装置的图像传感器的采集帧率,其中,所述采集帧率大于或等于所述录像帧率和所述图传帧率中的较大者。可选地,当所述图传设置信息为所述第一图传模式,且所述录像帧率等于所述第二帧率时,所述采集帧率等于所述第二帧率,或者当所述图传设置信息为所述第二图传模式,且所述采集帧率等于所述第二帧率。
在一个示例中,当所述拍摄装置以第一分辨率和第一录像帧率进行录像时,处理器401用于执行所述存储器中存储的程序指令,使得一个或多个处理器执行所述处理器还用于:获取用户输入的模式切换指令,所述模式切换指令用于指示将当前运行的所述第一图传模式切换为第二图传模式或者将当前运行的所述第二图传模式切换为所述第一图传模式;将所述模式切换指令发送给所述可移动平台,以将当前运行的所述第一图传模式切换为第二图传模式或者将当前运行的所述第二图传模式切换为所述第一图传模式;根据切换后的图传模式,更新录像规格设置信息,并将更新后的录像规格设置信息发送给可移动平台,以控制所述拍摄装置以更新后的录像规格设置信息进行录像,其中,所述更新后的录像规格设置信息包括第二分辨率和所述第一录像帧率,所述第二分辨率小于或等于所述第一分辨率。
当所述拍摄装置以第一分辨率和第一录像帧率进行录像时,在一个示例中,处理器401用于执行所述存储器中存储的程序指令,使得一个或多个处理器执行:当将所述第一图传模式切换为第二图传模式后,或者,当将所述第二图传模式切换为所述第一图传模式后,控制所述拍摄装置以第二分辨率和所述第二录像帧率进行录像;当所述第二分辨率与所述第一分辨率不同和/或当所述第二录像帧率和所述第一录像帧率不同时,输出提示信息;所述播放终端还包括显示器,所述显示器用于获取并显示所述提示信息,以便在录像格式发生变化时,用户能及时获取到提示信息。
在本申请实施例中,所述第一帧率小于或等于60fps,例如第一帧率等于60fps,所述第二帧率大于60fps且小于或等于120fps,例如第二帧率等于120fps。可选地,所述录像分辨率包括4KP、720P或1080P,或者其他适合的分辨率。
下面,参考附图5对本申请的可移动平台进行描述,其中,该可移动平台500可以用于执行前述的视频传输方法300中的相关步骤。
在一个示例中,如图5所示,可移动平台500包括一个或多个处理器501、拍摄装置505、存储器502、图传装置503、拍摄装置504、通信接口(未示出)等。这些组件通过总线系统和/或其它形式的连接机构(未示出)互连。
存储器502用于存储可执行指令,例如用于存储用于实现根据本申请实施例的视频传输方法300中的相应步骤和程序指令。存储器502可存储操作系统(包括用于处理各种基本系统服务和用于执行硬件相关任务的程序)、至少一个功能所需的应用程序等。可以包括一个或多个计算机程序产品,所述计算机程序产品可以包括各种形式的计算机可读存储介质,例如易失性存储器和/或非易失性存储器。所述易失性存储器例如可以包括随机存取存储器(RAM)和/或高速缓冲存储器(cache)等。所述非易失性存储器例如可以包括只读存储器(ROM)、硬盘、闪存(例如SD卡)等。
通信接口(未示出)可以用于可移动平台500和终端设备(例如控制终端、播放终端例如眼镜)或其他设备之间进行通信,包括有线或者无线方式的通信。通信接口1004可以为接入基于通信标准的无线网络,如WiFi(例如WiFi6)、软件无线电(Software-defined radio,简称SDR)、2G、3G、4G、5G或它们的组合。在一个示例性实施例中,可移动平台500经由广播信道接收来自外部广播管理系统的广播信号或广播相关信息。在一个示例性实施例中,1004还包括近场通信(NFC)模块,以促进短程通信。例如,在NFC模块可基于射频识别(RFID)技术,红外数据协会(IrDA)技术,超宽带(UWB)技术,蓝牙(BT)技术和其他技术来实现。
处理器501可以是中央处理单元(CPU)、图像处理单元(GPU)、专用集成电路(ASIC)、现场可编程门阵列(FPGA)或者具有数据处理能力和/或指令执行能力的其它形式的处理单元,并且可以控制拍摄装置中的其它组件以执行期望的功能。处理器501能够执行存储器502中存储的所述指令,以执行本文描述的视频传输方法300的相应步骤。例如,处理器1001能够包括一个或多个嵌入式处理器、处理器核心、微型处理器、逻辑电路、硬件有限状态机(FSM)、数字信号处理器(DSP)、ISP、编码器或它们的组合。
拍摄装置504用于拍摄视频或图像数据,所述图传装置503用于获取所述视频数据并向所述播放终端发送所述视频数据。拍摄装置504可以是具有图像传感器的装置,例如相机、摄像机、摄像头或者还可以是其他装置。图像传感器(本文也称相机传感器)是用于捕获图像数据的器件,其例如可以为CMOS图像传感器、CCD图像传感器等。在一个示例中,拍摄装置504还可以是以嵌入式影像系统的方式设置于可移动平台(例如飞行器、手持云台)的本体上。可选地,拍摄装置504可以作为负载设置于可移动平台的本体上,用户可以通过播放终端或控制终端控制拍摄装置。图传装置503例如数字图传收发机,可 以是指以数字通信链路传输信息(如图像数据)的装置,即数字图传收发机可用于可移动平台与其他终端进行交互信息。
拍摄装置504可以设置于可移动平台本体。在某些实施方式中,可移动平台包括飞行器(例如无人飞行器)、机器人、车辆、云台或船。当拍摄装置应用于无人飞行器时,平台本体为无人飞行器的机身,其中,拍摄装置可以以嵌入式影像系统的方式设置于可移动平台本体。当拍摄装置应用于车辆时,平台本体为车辆的车身。该车辆可以是自动驾驶汽车或者半自动驾驶汽车,在此不做限制。当拍摄装置应用于机器人时,平台本体为机器人。当拍摄装置应用于船时,平台本体为船身。
在一个示例中,处理器1001用于执行所述存储器中存储的程序指令,使得一个或多个处理器执行前文中描述的视频传输方法300的相关步骤,具体视频传输方法300的描述可以参考前文,在此不再重复描述。
另外本申请还提供一种视频传输系统,该视频传输系统包括前文描述的播放终端和可移动平台、控制终端等。具体的视频传输系统的细节可以参考前文图1、图4和图5的描述,在此不再重复。
综上所述,根据本申请的视频传输方法、播放终端、可移动平台及视频传输系统,可以控制图传装置可以在第一图传模式和第二图传模式中的一个模式下向播放终端传输视频数据,从而使用户体验到低延时或高画质的图传体验,并且本申请的视频传输方法中可以在第一图传模式下,使录像帧率等于或低于第一帧率,从而可以满足用户对于低帧率录像的需求,进而节省录像的视频数据所占用的存储空间,还可以在第二图传模式下,使录像帧率大于或等于第二帧率,从而可以满足用户多种录像帧率的需求,给用户提供更多的录像体验和选择,提升用户的使用体验。
带眼镜的穿越机体验机,眼镜有插SD卡(例如,卡内存有飞机端录像)回放需求。
但由于眼镜端图传处理芯片处理能力受限:因此其只能解码H.264编码格式,无法解码H.265编码格式(例如飞机端录制的H.265编码格式视频播不了,而H.265编码格式相对于H.264编码格式码率更低、画质更好),且处理能力不足,只有4Kp30的处理能力,约265420800pixel/s(例如,飞机端录制的4Kp60、4Kp50、1080p240等录像播不了。
面对上述问题,通常的处理方式是:1、不限制飞机端录像编码格式、录像规格。这样就要接收眼镜端无法回放部分视频文件这个缺陷,体验不太好;2、限制飞机端录像编码格式、录像规格,只录制眼镜端能够回放的视频。这样限制了用户录制视频的手段,影响出片,体验也不好。3、同时录制两路文件分别放置在两个视频文件中,一个文件用于 计算机终端等播放,一个文件用于眼镜插卡回放,然而这种方式中当SD卡插在计算机终端时,计算机会识别出眼镜端播放的视频文件,使得显示更加繁琐、多余;且同时录制两个视频文件,CPU负载会上升,同时写入多路码流,也会影响写卡速度。
因此,鉴于上述问题的存在,本申请还提供一种视频处理方法,所述方法包括:获取第一视频码流、第二视频码流和音频码流,其中,所述第一视频码流用于第一类播放终端播放,所述第一视频码流的编码格式与所述第一类播放终端所能播放的编码格式相对应,所述第二视频码流用于第二类播放终端播放,所述第二视频码流的编码格式与所述第二类播放终端所能播放的编码格式相对应,第二视频码流的视频规格小于或等于阈值规格;将所述第一视频码流和所述第二视频码流和音频码流封装为预设文件格式的视频文件;将所述视频文件存储至第一存储器中,以供所述第一类播放终端或所述第二类播放终端回放所述视频文件。本申请的视频处理方法,通过将第一视频码流、第二视频码流和音频码流封装在一个视频文件中,使得该视频文件可以在多种类型的播放终端进行播放,克服了第二类播放终端芯片的能力限制,提升了第一类播放终端的视频播放效果,并且相比将第一视频码流和第二视频码流独立封装的方案,本申请减少了一路音频码流,降低了处理器的处理负载,降低了存储总码率,提升了用户体验。
下面,参考图6对本申请的视频处理方法进行描述,其中,图6图6示出了本发明一个实施例中的视频处理方法的示意性流程图。
在一个示例中,如图6所示,本申请的视频处理方法600包括以下步骤:在步骤S601中,获取第一视频码流、第二视频码流和音频码流,其中,所述第一视频码流用于第一类播放终端播放,所述第一视频码流的编码格式与所述第一类播放终端所能播放的编码格式相对应,所述第二视频码流用于第二类播放终端播放,所述第二视频码流的编码格式与所述第二类播放终端所能播放的编码格式相对应,第二视频码流的视频规格小于或等于阈值规格;在步骤S602中,将所述第一视频码流和所述第二视频码流和音频码流封装为预设文件格式的视频文件;在步骤S603中,将所述视频文件存储至第一存储器中,以供所述第一类播放终端和/或所述第二类播放终端回放所述视频文件。
第一视频码流可以称为主路视频码流,示例性地,当所述第一类播放终端包括计算机终端或移动终端中的至少一种时,所述第一视频码流的编码格式为H.264编码格式或H.265编码格式,或者其他的编码格式。移动终端可以包括但不限于手机、平板电脑等。在此不限制用于第一类播放终端的视频码流的编码格式,可以使其具有更多的选择性,且当其使用例如H.265编码格式时,相比H.264编码格式其可以具有更低的码率、更好的画 质。
在一个示例中,所述第一视频码流的视频规格大于或等于所述阈值规格,该阈值规格可以根据实际需要合理设定,该阈值规格可以为4Kp30,则第一视频码流的视频规格可以超过4Kp30,例如4Kp60、4Kp50、1080p60、1080p120、1080p240或者其他的视频规格,从而获得更好的视频播放效果,提升用户体验。
在一个示例中,第二视频码流也可以称为辅路视频码流,所述第二视频码流的编码格式与所述第二类播放终端所能播放的编码格式相对应,例如,当所述第二类播放终端包括眼镜终端时,所述第二视频码流的编码格式为H.264编码格式,第二视频码流的视频规格小于或等于阈值规格,例如不超过4Kp30,例如可以为4Kp30、1080P30等。
将所述第一视频码流和所述第二视频码流和音频码流封装为预设文件格式的视频文件,例如MP4或MOV或者其他的文件格式。封装过程可以是本领域技术人员熟知的任何能够将三者封装在一个视频文件中的方案,在如图7所示的视频文件的封装方案中,视频文件中的轨道(track)数据还可以包括缩略图、数据头、创建日期、源数据、码流数据等。其中,由于本申请的方案中将第一视频码流和所述第二视频码流封装在一个文件中,且可以使用同一路音频码流,且还可以使用相同的缩略图等,因此,减少了一路音频码流轨道、并且减少了一路缩略图等信息的轨道的存储,降低了录像过程中的CPU负载,以及存储写卡的总码率。
本申请实施例中通过制作辅路视频码流,用于眼镜端的播放,克服了眼镜芯片的能力限制,保留了主路码流对H.265编码、使用4Kp30以上规格的支持。
在一个示例中,所述第二视频码流对应的轨道类型为所述第一类播放终端无法识别的类型,例如为非标准的私有类型,眼镜端程序能能够识别出第二视频码流并能正确地进行回放,但其能达到计算机(PC)终端、手机端的视频播放、视频后期编辑软件等无法识别的效果,那么在将SD卡插入计算机(PC)终端或手机端进行播放时,用户则不会感知到第二视频码流的存在,优化了用户体验。
本申请实施例还提供一种视频处理装置,该视频处理装置可以作为前述视频处理方法的执行主体。
在一个示例中,如图8所示,视频处理装置800可以包括一个或多个处理器801、存储器802、通信接口等。这些组件通过总线系统和/或其它形式的连接机构(未示出)互连。
存储器802用于存储可执行指令,例如用于存储用于实现根据本申请实施例的视频处理方法600中的相应步骤和程序指令。可以包括一个或多个计算机程序产品,所述计算机 程序产品可以包括各种形式的计算机可读存储介质,例如易失性存储器和/或非易失性存储器。所述易失性存储器例如可以包括随机存取存储器(RAM)和/或高速缓冲存储器(cache)等。所述非易失性存储器例如可以包括只读存储器(ROM)、硬盘、闪存(例如SD卡)等。存储器802包括第一存储器(例如SD卡),所述第一存储器用于存储所述视频文件,并且所述第一存储器可插拔地设置于视频处理装置800内,当需要用其他的终端设备播放视频文件时,则可以将该第一存储器从视频处理装置中取出,插入播放终端例如眼镜进行视频的回放。
视频处理装置800可以具有通信接口用于或其他设备之间进行通信,包括有线或者无线方式的通信。在此不做赘述。
视频处理装置800可以是任意的具有图像数据处理功能的装置,例如拍摄装置、搭载有拍摄装置的可移动平台等,在此不对其进行具体限定。
另外,本申请还提供一种可移动平台,如图5所示,可移动平台500包括一个或多个处理器501、拍摄装置505、存储器502、图传装置503、拍摄装置504、通信接口(未示出)等。这些组件通过总线系统和/或其它形式的连接机构(未示出)互连。
一个或多个处理器501用于执行所述存储器502中存储的所述程序指令,使得所述处理器执行前述的视频处理方法600,有关可移动平台500的结构描述以及视频处理方法600的相关步骤的细节可以参考前文的描述,在此不再赘述。
存储器502用于存储可执行指令,例如用于存储用于实现根据本申请实施例的视频处理方法600中的相应步骤和程序指令。可以包括一个或多个计算机程序产品,所述计算机程序产品可以包括各种形式的计算机可读存储介质,例如易失性存储器和/或非易失性存储器。所述易失性存储器例如可以包括随机存取存储器(RAM)和/或高速缓冲存储器(cache)等。所述非易失性存储器例如可以包括只读存储器(ROM)、硬盘、闪存(例如SD卡)等。存储器502包括第一存储器(例如SD卡),所述第一存储器用于存储所述视频文件,并且所述第一存储器可插拔地设置于可移动平台500内,当需要用其他的终端设备播放视频文件时,则可以将该第一存储器从视频处理装置中取出,插入播放终端例如眼镜进行视频的回放。
综上所述,本申请的视频处理方法以及装置和可移动平台,通过将第一视频码流、第二视频码流和音频码流封装在一个视频文件中,使得该视频文件可以在多种类型的播放终端进行播放,克服了第二类播放终端芯片的能力限制,提升了第一类播放终端的视频播放效果,并且相比将第一视频码流和第二视频码流独立封装的方案,本申请减少了一路音 频码流,降低了处理器的处理负载,降低了存储总码率,提升了用户体验。
另外,本发明实施例还提供了一种计算机存储介质,其上存储有计算机程序。在所述计算机可读存储介质上可以存储一个或多个计算机程序指令,处理器可以运行存储装置存储的所述程序指令,以实现本文所述的本发明实施例中(由处理器实现)的功能以及/或者其它期望的功能,例如以执行根据本申请实施例的视频传输方法200、300的相应步骤,或者,以执行根据本申请实施例的视频处理方法600的相应步骤在所述计算机可读存储介质中还可以存储各种应用程序和各种数据,例如所述应用程序使用和/或产生的各种数据等。
例如,所述计算机存储介质例如可以包括智能电话的存储卡、平板电脑的存储部件、个人计算机的硬盘、只读存储器(ROM)、可擦除可编程只读存储器(EPROM)、便携式紧致盘只读存储器(CD-ROM)、USB存储器、或者上述存储介质的任意组合。所述计算机可读存储介质可以是一个或多个计算机可读存储介质的任意组合。
尽管这里已经参考附图描述了示例实施例,应理解上述示例实施例仅仅是示例性的,并且不意图将本发明的范围限制于此。本领域普通技术人员可以在其中进行各种改变和修改,而不偏离本发明的范围和精神。所有这些改变和修改意在被包括在所附权利要求所要求的本发明的范围之内。
本领域普通技术人员可以意识到,结合本文中所公开的实施例描述的各示例的单元及算法步骤,能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行,取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能,但是这种实现不应认为超出本发明的范围。
在本申请所提供的几个实施例中,应该理解到,所揭露的设备和方法,可以通过其它的方式实现。例如,以上所描述的设备实施例仅仅是示意性的,例如,所述单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个设备,或一些特征可以忽略,或不执行。
在此处所提供的说明书中,说明了大量具体细节。然而,能够理解,本发明的实施例可以在没有这些具体细节的情况下实践。在一些实例中,并未详细示出公知的方法、结构和技术,以便不模糊对本说明书的理解。
类似地,应当理解,为了精简本发明并帮助理解各个发明方面中的一个或多个,在对本发明的示例性实施例的描述中,本发明的各个特征有时被一起分组到单个实施例、图、或者对其的描述中。然而,并不应将该本发明的方法解释成反映如下意图:即所要求保护 的本发明要求比在每个权利要求中所明确记载的特征更多的特征。更确切地说,如相应的权利要求书所反映的那样,其发明点在于可以用少于某个公开的单个实施例的所有特征的特征来解决相应的技术问题。因此,遵循具体实施方式的权利要求书由此明确地并入该具体实施方式,其中每个权利要求本身都作为本发明的单独实施例。
本领域的技术人员可以理解,除了特征之间相互排斥之外,可以采用任何组合对本说明书(包括伴随的权利要求、摘要和附图)中公开的所有特征以及如此公开的任何方法或者设备的所有过程或单元进行组合。除非另外明确陈述,本说明书(包括伴随的权利要求、摘要和附图)中公开的每个特征可以由提供相同、等同或相似目的替代特征来代替。
此外,本领域的技术人员能够理解,尽管在此所述的一些实施例包括其它实施例中所包括的某些特征而不是其它特征,但是不同实施例的特征的组合意味着处于本发明的范围之内并且形成不同的实施例。例如,在权利要求书中,所要求保护的实施例的任意之一都可以以任意的组合方式来使用。
本发明的各个部件实施例可以以硬件实现,或者以在一个或者多个处理器上运行的软件模块实现,或者以它们的组合实现。本领域的技术人员应当理解,可以在实践中使用微处理器或者数字信号处理器(DSP)来实现根据本发明实施例的一些模块的一些或者全部功能。本发明还可以实现为用于执行这里所描述的方法的一部分或者全部的装置程序(例如,计算机程序和计算机程序产品)。这样的实现本发明的程序可以存储在计算机可读介质上,或者可以具有一个或者多个信号的形式。这样的信号可以从因特网网站上下载得到,或者在载体信号上提供,或者以任何其他形式提供。
应该注意的是上述实施例对本发明进行说明而不是对本发明进行限制,并且本领域技术人员在不脱离所附权利要求的范围的情况下可设计出替换实施例。在权利要求中,不应将位于括号之间的任何参考符号构造成对权利要求的限制。本发明可以借助于包括有若干不同元件的硬件以及借助于适当编程的计算机来实现。在列举了若干装置的单元权利要求中,这些装置中的若干个可以是通过同一个硬件项来具体体现。单词第一、第二、以及第三等的使用不表示任何顺序。可将这些单词解释为名称。

Claims (48)

  1. 一种视频传输方法,其特征在于,应用于播放终端,所述播放终端与可移动平台通信连接,所述可移动平台包括图传装置和拍摄装置,所述拍摄装置用于拍摄视频数据,所述图传装置用于获取所述视频数据并向所述播放终端发送所述视频数据,所述方法包括:
    获取图传设置信息和录像规格设置信息,其中,所述图传设置信息包括第一图传模式和第二图传模式中的一种模式设置信息,所述录像规格设置信息包括录像分辨率和录像帧率,当所述图传设置信息为所述第一图传模式时,图传帧率为第一帧率,所述录像帧率大于或等于所述第一帧率,当图传设置信息为第二图传模式时,图传帧率为第二帧率,所述录像帧率小于或等于所述第二帧率,所述第一帧率小于所述第二帧率;
    基于所述图传设置信息和录像规格设置信息,发送控制信号,所述控制信号用于控制所述拍摄装置在拍摄视频数据,并控制所述图传装置基于所述图传设置信息向所述播放终端发送所述视频数据,以及控制所述拍摄装置基于所述录像设置信息录制视频数据。
  2. 如权利要求1所述的方法,其特征在于,所述获取拍摄装置的图传设置信息和录像规格设置信息,包括:
    响应于第一用户指令,控制所述播放终端的显示器显示设置界面,所述设置界面用于显示:所述第一图传模式对应的选项和所述第二图传模式对应的选项;
    响应于用户输入的模式选择指令,确定所述第一图传模式和所述第二图像模式中的一个作为所述图传设置信息;
    基于已确定的所述图传设置信息,确定所述录像规格设置信息。
  3. 如权利要求2所述的方法,其特征在于,基于已确定的所述图传设置信息,确定所述录像规格设置信息,包括:
    当所述图传设置信息为第一图传模式时,响应于用户输入的录像规格设置指令,控制所述播放终端的显示器在所述设置界面显示多个第一类录像规格,所述多个第一类录像规格的录像帧率大于或等于所述第一帧率;
    响应于用户输入的规格选择指令,确定所述多个第一类录像规格中的一个作为所述录像规格设置信息。
  4. 如权利要求2所述的方法,其特征在于,基于已确定的所述图传设置信息,确定所述录像规格设置信息,包括:
    当所述图传设置信息为第二图传模式时,基于用户输入的录像规格设置指令,控制所述播放终端的显示器在所述设置界面显示多个第二类录像规格,所述多个第二类录像规格的录像帧率小于或等于所述第二帧率;
    基于用户输入的规格选择指令,确定所述多个第二类录像规格中的一个作为所述录像 规格设置信息。
  5. 如权利要求1所述的方法,其特征在于,所述方法还包括:基于所述图传设置信息和录像规格设置信息,确定所述拍摄装置的图像传感器的采集帧率,其中,所述采集帧率大于或等于所述录像帧率和所述图传帧率中的较大者。
  6. 如权利要求5所述的方法,其特征在于,当所述图传设置信息为所述第一图传模式,且所述录像帧率等于所述第二帧率时,所述采集帧率等于所述第二帧率,或者
    当所述图传设置信息为所述第二图传模式,且所述采集帧率等于所述第二帧率。
  7. 如权利要求1所述的方法,其特征在于,当所述拍摄装置以第一分辨率和第一录像帧率进行录像时,所述方法还包括:
    获取用户输入的模式切换指令,所述模式切换指令用于指示将当前运行的所述第一图传模式切换为第二图传模式或者将当前运行的所述第二图传模式切换为所述第一图传模式;
    将所述模式切换指令发送给所述可移动平台,以将当前运行的所述第一图传模式切换为第二图传模式或者将当前运行的所述第二图传模式切换为所述第一图传模式;
    根据切换后的图传模式,更新录像规格设置信息,并将更新后的录像规格设置信息发送给可移动平台,以控制所述拍摄装置以更新后的录像规格设置信息进行录像,其中,所述更新后的录像规格设置信息包括第二分辨率和所述第一录像帧率,所述第二分辨率小于或等于所述第一分辨率。
  8. 如权利要求1所述的方法,其特征在于,当所述拍摄装置以第一分辨率和第一录像帧率进行录像时,所述方法还包括:
    当将所述第一图传模式切换为第二图传模式后,或者,当将所述第二图传模式切换为所述第一图传模式后,向所述可移动平台发送更新后的录像规格设置信息,以控制所述拍摄装置以更新后的录像规格设置信息进行录像,所述更新后的录像规格设置信息包括第二分辨率和所述第二录像帧率;
    当所述第二分辨率与所述第一分辨率不同和/或当所述第二录像帧率和所述第一录像帧率不同时,输出提示信息。
  9. 如权利要求1至8任一项所述的方法,其特征在于,所述第一帧率小于或等于60fps,所述第二帧率大于60fps且小于或等于120fps。
  10. 如权利要求1至9任一项所述的方法,其特征在于,所述录像分辨率包括4KP、720P或1080P。
  11. 一种视频传输方法,其特征在于,应用于可移动平台,所述播放终端与可移动平台通信连接,所述可移动平台包括图传装置和拍摄装置,所述拍摄装置用于拍摄视频数据, 所述播放终端用于获取所述视频数据并播放,所述图传装置用于获取所述视频数据并向所述播放终端发送所述视频数据,所述方法包括:
    获取所述播放终端发送的控制信号,所述控制信号用于控制所述拍摄装置在拍摄视频数据时,控制所述图传装置基于图传设置信息向所述播放终端发送所述视频数据,以及控制所述拍摄装置基于录像设置信息录制视频数据,其中,所述图传设置信息包括第一图传模式和第二图传模式中的一种模式设置信息,所述录像规格设置信息包括录像分辨率和录像帧率,当所述图传设置信息为所述第一图传模式时,图传帧率为第一帧率,所述录像帧率大于或等于所述第一帧率,当图传设置信息为第二图传模式时,图传帧率为第二帧率,所述录像帧率小于或等于所述第二帧率,所述第一帧率小于所述第二帧率;
    响应于所述控制信号,控制所述拍摄装置拍摄视频数据,并控制所述图传装置基于图传设置信息向所述播放终端发送所述视频数据,以及控制所述拍摄装置基于录像设置信息录制视频数据。
  12. 如权利要求11所述的方法,其特征在于,当所述图传设置信息包括第一图传模式时,所述录像规格设置信息包括多个第一类录像规格中的一个,所述多个第一类录像规格选项的录像帧率大于或等于所述第一帧率。
  13. 如权利要求11所述的方法,其特征在于,当所述图传设置信息为第二图传模式时,所述录像规格设置信息包括所述多个第二类录像规格中的一个,所述多个第二类录像规格的录像帧率小于或等于所述第二帧率。
  14. 如权利要求11所述的方法,其特征在于,所述方法还包括:基于所述图传设置信息和录像规格设置信息,确定所述拍摄装置的图像传感器的采集帧率,其中,所述采集帧率大于或等于所述录像帧率和所述图传帧率中的较大者。
  15. 如权利要求14所述的方法,其特征在于,当所述图传设置信息为所述第一图传模式,且所述录像帧率等于所述第二帧率时,所述采集帧率等于所述第二帧率,或者
    当所述图传设置信息为所述第二图传模式,且所述采集帧率等于所述第二帧率。
  16. 如权利要求11所述的方法,其特征在于,当所述拍摄装置以第一分辨率和第一录像帧率进行录像时,所述方法还包括:
    获取所述播放终端发送的模式切换指令;
    响应于所述模式切换指令,将当前运行的所述第一图传模式切换为第二图传模式或者将当前运行的所述第二图传模式切换为所述第一图传模式;
    获取更新后的录像规格设置信息,并控制所述拍摄装置以更新后的录像规格设置信息进行录像,其中,所述更新后的录像规格设置信息包括第二分辨率和所述第一录像帧率,所述第二分辨率小于或等于所述第一分辨率。
  17. 如权利要求11至16任一项所述的方法,其特征在于,所述第一帧率小于或等于60fps,所述第二帧率大于60fps且小于或等于120fps。
  18. 如权利要求11至17任一项所述的方法,其特征在于,所述录像分辨率包括4KP、720P或1080P。
  19. 一种播放终端,其特征在于,所述播放终端包括:
    存储器,用于存储可执行的程序指令;
    通信接口,用于使所述播放终端和可移动平台进行通信,其中,所述可移动平台包括图传装置和拍摄装置,所述拍摄装置用于拍摄视频数据,所述图传装置用于获取所述视频数据并向所述播放终端发送所述视频数据;
    一个或多个处理器,用于执行所述存储器中存储的所述程序指令,使得所述处理器执行以下步骤:
    获取图传设置信息和录像规格设置信息,其中,所述图传设置信息包括第一图传模式和第二图传模式中的一种模式设置信息,所述录像规格设置信息包括录像分辨率和录像帧率,当所述图传设置信息为所述第一图传模式时,图传帧率为第一帧率,所述录像帧率大于或等于所述第一帧率,当图传设置信息为第二图传模式时,图传帧率为第二帧率,所述录像帧率小于或等于所述第二帧率,所述第一帧率小于所述第二帧率;
    基于所述图传设置信息和录像规格设置信息,发送控制信号,所述控制信号用于控制所述拍摄装置在拍摄视频数据,并控制所述图传装置基于所述图传设置信息向所述播放终端发送所述视频数据,以及控制所述拍摄装置基于所述录像设置信息录制视频数据。
  20. 如权利要求19所述的播放终端,其特征在于,所述播放终端还包括显示器,所述获取拍摄装置的图传设置信息和录像规格设置信息,包括:响应于第一用户指令,控制所述播放终端的显示器显示设置界面,所述设置界面用于显示:所述第一图传模式对应的选项和所述第二图传模式对应的选项;
    响应于用户输入的模式选择指令,确定所述第一图传模式和所述第二图像模式中的一个作为所述图传设置信息;
    基于已确定的所述图传设置信息,确定所述录像规格设置信息。
  21. 如权利要求20所述的播放终端,其特征在于,基于已确定的所述图传设置信息,确定所述录像规格设置信息,包括:
    当所述图传设置信息为第一图传模式时,响应于用户输入的录像规格设置指令,控制所述播放终端的显示器在所述设置界面显示多个第一类录像规格,所述多个第一类录像规格的录像帧率大于或等于所述第一帧率;
    响应于用户输入的规格选择指令,确定所述多个第一类录像规格中的一个作为所述录 像规格设置信息。
  22. 如权利要求20所述的播放终端,其特征在于,基于已确定的所述图传设置信息,确定所述录像规格设置信息,包括:
    当所述图传设置信息为第二图传模式时,基于用户输入的录像规格设置指令,控制所述播放终端的显示器在所述设置界面显示多个第二类录像规格,所述多个第二类录像规格的录像帧率小于或等于所述第二帧率;
    基于用户输入的规格选择指令,确定所述多个第二类录像规格中的一个作为所述录像规格设置信息。
  23. 如权利要求19所述的播放终端,其特征在于,所述处理器还用于:基于所述图传设置信息和录像规格设置信息,确定所述拍摄装置的图像传感器的采集帧率,其中,所述采集帧率大于或等于所述录像帧率和所述图传帧率中的较大者。
  24. 如权利要求23所述的播放终端,其特征在于,当所述图传设置信息为所述第一图传模式,且所述录像帧率等于所述第二帧率时,所述采集帧率等于所述第二帧率,或者
    当所述图传设置信息为所述第二图传模式,且所述采集帧率等于所述第二帧率。
  25. 如权利要求19所述的播放终端,其特征在于,当所述拍摄装置以第一分辨率和第一录像帧率进行录像时,所述处理器还用于:
    获取用户输入的模式切换指令,所述模式切换指令用于指示将当前运行的所述第一图传模式切换为第二图传模式或者将当前运行的所述第二图传模式切换为所述第一图传模式;
    将所述模式切换指令发送给所述可移动平台,以将当前运行的所述第一图传模式切换为第二图传模式或者将当前运行的所述第二图传模式切换为所述第一图传模式;
    根据切换后的图传模式,更新录像规格设置信息,并将更新后的录像规格设置信息发送给可移动平台,以控制所述拍摄装置以更新后的录像规格设置信息进行录像,其中,所述更新后的录像规格设置信息包括第二分辨率和所述第一录像帧率,所述第二分辨率小于或等于所述第一分辨率。
  26. 如权利要求19所述的播放终端,其特征在于,当所述拍摄装置以第一分辨率和第一录像帧率进行录像时,所述处理器还用于:
    当将所述第一图传模式切换为第二图传模式后,或者,当将所述第二图传模式切换为所述第一图传模式后,控制所述拍摄装置以第二分辨率和所述第二录像帧率进行录像;
    当所述第二分辨率与所述第一分辨率不同和/或当所述第二录像帧率和所述第一录像帧率不同时,输出提示信息;
    所述播放终端还包括显示器,所述显示器用于获取并显示所述提示信息。
  27. 如权利要求19至26任一项所述的播放终端,其特征在于,所述第一帧率小于或等于60fps,所述第二帧率大于60fps且小于或等于120fps。
  28. 如权利要求19至27任一项所述的播放终端,其特征在于,所述录像分辨率包括4KP、720P或1080P。
  29. 一种可移动平台,其特征在于,所述可移动平台包括
    拍摄装置,所述拍摄装置用于拍摄视频数据;
    图传装置,用于获取所述视频数据并向所述播放终端发送所述视频数据,
    存储器,用于存储可执行的程序指令;
    一个或多个处理器,用于执行所述存储器中存储的所述程序指令,使得所述处理器执行以下步骤:
    获取所述播放终端发送的控制信号,所述控制信号用于控制所述拍摄装置在拍摄视频数据时,控制所述图传装置基于图传设置信息向所述播放终端发送所述视频数据,以及控制所述拍摄装置基于录像设置信息录制视频数据,其中,所述图传设置信息包括第一图传模式和第二图传模式中的一种模式设置信息,所述录像规格设置信息包括录像分辨率和录像帧率,当所述图传设置信息为所述第一图传模式时,图传帧率为第一帧率,所述录像帧率大于或等于所述第一帧率,当图传设置信息为第二图传模式时,图传帧率为第二帧率,所述录像帧率小于或等于所述第二帧率,所述第一帧率小于所述第二帧率;
    响应于所述控制信号,控制所述拍摄装置拍摄视频数据,并控制所述图传装置基于图传设置信息向所述播放终端发送所述视频数据,以及控制所述拍摄装置基于录像设置信息录制视频数据。
  30. 如权利要求29所述的可移动平台,其特征在于,当所述图传设置信息包括第一图传模式时,所述录像规格设置信息包括多个第一类录像规格中的一个,所述多个第一类录像规格选项的录像帧率大于或等于所述第一帧率。
  31. 如权利要求29所述的可移动平台,其特征在于,当所述图传设置信息为第二图传模式时,所述录像规格设置信息包括所述多个第二类录像规格中的一个,所述多个第二类录像规格的录像帧率小于或等于所述第二帧率。
  32. 如权利要求29所述的可移动平台,其特征在于,所述处理器还用于:基于所述图传设置信息和录像规格设置信息,确定所述拍摄装置的图像传感器的采集帧率,其中,所述采集帧率大于或等于所述录像帧率和所述图传帧率中的较大者。
  33. 如权利要求32所述的可移动平台,其特征在于,当所述图传设置信息为所述第一图传模式,且所述录像帧率等于所述第二帧率时,所述采集帧率等于所述第二帧率, 或者
    当所述图传设置信息为所述第二图传模式,且所述采集帧率等于所述第二帧率。
  34. 如权利要求29所述的可移动平台,其特征在于,当所述拍摄装置以第一分辨率和第一录像帧率进行录像时,所述处理器还用于:
    获取所述播放终端发送的模式切换指令;
    响应于所述模式切换指令,将当前运行的所述第一图传模式切换为第二图传模式或者将当前运行的所述第二图传模式切换为所述第一图传模式;
    获取更新后的录像规格设置信息,并控制所述拍摄装置以更新后的录像规格设置信息进行录像,其中,所述更新后的录像规格设置信息包括第二分辨率和所述第一录像帧率,所述第二分辨率小于或等于所述第一分辨率。
  35. 如权利要求29至34任一项所述的可移动平台,其特征在于,所述第一帧率小于或等于60fps,所述第二帧率大于60fps且小于或等于120fps。
  36. 如权利要求29至35任一项所述的可移动平台,其特征在于,所述录像分辨率包括4KP、720P或1080P。
  37. 一种视频传输系统,其特征在于,所述视频传输系统包括:
    如权利要求19至28任一项所述的播放终端;以及
    如权利要求29至36任一项所述的可移动平台。
  38. 一种视频处理方法,其特征在于,所述方法包括:
    获取第一视频码流、第二视频码流和音频码流,其中,所述第一视频码流用于第一类播放终端播放,所述第一视频码流的编码格式与所述第一类播放终端所能播放的编码格式相对应,所述第二视频码流用于第二类播放终端播放,所述第二视频码流的编码格式与所述第二类播放终端所能播放的编码格式相对应,第二视频码流的视频规格小于或等于阈值规格;
    将所述第一视频码流和所述第二视频码流和音频码流封装为预设文件格式的视频文件;
    将所述视频文件存储至第一存储器中,以供所述第一类播放终端和/或所述第二类播放终端回放所述视频文件。
  39. 如权利要求38所述的视频处理方法,其特征在于,当所述第一类播放终端包括计算机终端或移动终端中的至少一种时,所述第一视频码流的编码格式为H.264或H.265编码格式。
  40. 如权利要求38所述的视频处理方法,其特征在于,所述第一视频码流的视频规格大于或等于所述阈值规格。
  41. 如权利要求38所述的视频处理方法,其特征在于,当所述第二类播放终端包括眼镜终端时,所述第二视频码流的编码格式为H.264编码格式。
  42. 如权利要求38所述的视频处理方法,其特征在于,所述阈值规格包括4Kp30。
  43. 如权利要求38所述的视频处理方法,其特征在于,所述第二视频码流对应的轨道类型为所述第一类播放终端无法识别的类型。
  44. 如权利要求38至43任一项所述的视频处理方法,其特征在于,所述预设文件格式包括MP4或MOV。
  45. 一种视频处理装置,其特征在于,所述装置包括:
    存储器,用于存储可执行的程序指令;
    一个或多个处理器,用于执行所述存储器中存储的所述程序指令,使得所述处理器执行如权利要求38至44任一项所述的视频处理方法;
    所述存储器包括第一存储器,所述第一存储器用于存储所述视频文件,并且所述存储器可插拔地设置于视频处理装置内。
  46. 一种可移动平台,其特征在于,所述可移动平台包括:
    拍摄装置,所述拍摄装置用于拍摄视频数据;
    存储器,用于存储可执行的程序指令;
    一个或多个处理器,用于执行所述存储器中存储的所述程序指令,使得所述处理器执行如权利要求38至44任一项所述的视频处理方法;
    所述存储器包括第一存储器,所述第一存储器用于存储所述视频文件,并且所述第一存储器可插拔地设置于视频处理装置内。
  47. 如权利要求46所述的可移动平台,其特征在于,所述可移动平台包括飞行器、机器人、车辆、云台或船。
  48. 一种计算机存储介质,其上存储有计算机程序,其特征在于,所述计算机程序被处理器执行时实现1至28任一项所述的视频传输方法,或者,实现38至44任一项所述的视频处理方法。
PCT/CN2020/138557 2020-12-23 2020-12-23 视频传输方法及系统、视频处理方法及装置、播放终端、可移动平台 WO2022133782A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2020/138557 WO2022133782A1 (zh) 2020-12-23 2020-12-23 视频传输方法及系统、视频处理方法及装置、播放终端、可移动平台
CN202080071094.5A CN114556958A (zh) 2020-12-23 2020-12-23 视频传输方法及系统、视频处理方法及装置、播放终端、可移动平台

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/138557 WO2022133782A1 (zh) 2020-12-23 2020-12-23 视频传输方法及系统、视频处理方法及装置、播放终端、可移动平台

Publications (1)

Publication Number Publication Date
WO2022133782A1 true WO2022133782A1 (zh) 2022-06-30

Family

ID=81668204

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/138557 WO2022133782A1 (zh) 2020-12-23 2020-12-23 视频传输方法及系统、视频处理方法及装置、播放终端、可移动平台

Country Status (2)

Country Link
CN (1) CN114556958A (zh)
WO (1) WO2022133782A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115515008A (zh) * 2022-09-19 2022-12-23 深圳市天和荣科技有限公司 一种视频处理方法、终端和视频处理系统

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101043618A (zh) * 2006-03-24 2007-09-26 腾讯科技(深圳)有限公司 一种在多路视频通讯中控制帧率的装置和方法
CN102497540A (zh) * 2011-12-21 2012-06-13 北京硅盾安全技术有限公司 一种视频信息自适应调整与传输方法
CN105049918A (zh) * 2015-07-08 2015-11-11 成都西可科技有限公司 一种本地录制视频和网络直播视频的分离方法
US20180357932A1 (en) * 2017-06-13 2018-12-13 At&T Intellectual Property I, L.P. Autonomous multi-pass data acquisition using unmanned aerial vehicles
CN110324585A (zh) * 2019-07-15 2019-10-11 清华大学 基于高速移动平台的slam系统的实现方法
CN110892731A (zh) * 2018-07-25 2020-03-17 深圳市大疆创新科技有限公司 视频播放速度控制方法及系统、控制终端和可移动平台
CN111182268A (zh) * 2016-01-29 2020-05-19 深圳市大疆创新科技有限公司 视频数据传输方法、系统、设备和拍摄装置
CN111566586A (zh) * 2019-06-21 2020-08-21 深圳市大疆创新科技有限公司 存储介质、图传模块、无人飞行器及其控制终端和套件

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101043618A (zh) * 2006-03-24 2007-09-26 腾讯科技(深圳)有限公司 一种在多路视频通讯中控制帧率的装置和方法
CN102497540A (zh) * 2011-12-21 2012-06-13 北京硅盾安全技术有限公司 一种视频信息自适应调整与传输方法
CN105049918A (zh) * 2015-07-08 2015-11-11 成都西可科技有限公司 一种本地录制视频和网络直播视频的分离方法
CN111182268A (zh) * 2016-01-29 2020-05-19 深圳市大疆创新科技有限公司 视频数据传输方法、系统、设备和拍摄装置
US20180357932A1 (en) * 2017-06-13 2018-12-13 At&T Intellectual Property I, L.P. Autonomous multi-pass data acquisition using unmanned aerial vehicles
CN110892731A (zh) * 2018-07-25 2020-03-17 深圳市大疆创新科技有限公司 视频播放速度控制方法及系统、控制终端和可移动平台
CN111566586A (zh) * 2019-06-21 2020-08-21 深圳市大疆创新科技有限公司 存储介质、图传模块、无人飞行器及其控制终端和套件
CN110324585A (zh) * 2019-07-15 2019-10-11 清华大学 基于高速移动平台的slam系统的实现方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115515008A (zh) * 2022-09-19 2022-12-23 深圳市天和荣科技有限公司 一种视频处理方法、终端和视频处理系统
CN115515008B (zh) * 2022-09-19 2024-02-27 深圳市天和荣科技有限公司 一种视频处理方法、终端和视频处理系统

Also Published As

Publication number Publication date
CN114556958A (zh) 2022-05-27

Similar Documents

Publication Publication Date Title
US8988558B2 (en) Image overlay in a mobile device
CN107771395B (zh) 生成和发送用于虚拟现实的元数据的方法和装置
KR102234928B1 (ko) 가상 현실 경험 공유
US20190246104A1 (en) Panoramic video processing method, device and system
US9462218B2 (en) Video display device and method of displaying video
US10771736B2 (en) Compositing and transmitting contextual information during an audio or video call
KR102492565B1 (ko) 가상 현실 미디어 컨텐츠의 패키징 및 스트리밍을 위한 방법 및 장치
US20140111670A1 (en) System and method for enhanced image capture
US9086790B2 (en) Image displaying method, image displaying program, and display
WO2017190351A1 (en) Systems and methods for video processing and display
EP3151548A1 (en) Video recording method and device
WO2020083114A1 (zh) 航拍相机的图像处理方法、图像处理系统及无人机
WO2020093850A1 (zh) 双光图像整合方法、整合设备及无人机
US11258949B1 (en) Electronic image stabilization to improve video analytics accuracy
KR20190055064A (ko) 화각 가상 현실 스트리밍을 위한 세션 제어 지원을 위한 방법 및 장치
WO2018103034A1 (zh) 用于传输图像的方法、设备和无人机
WO2021237619A1 (zh) 视频文件编辑方法、设备、系统及计算机可读存储介质
US20200366962A1 (en) Bitstream merger and extractor
WO2022133782A1 (zh) 视频传输方法及系统、视频处理方法及装置、播放终端、可移动平台
WO2020006650A1 (zh) 影像拍摄器和手持云台、可移动平台
CN109155820B (zh) 球形全景图的合成方法、无人机系统、无人机、终端及其控制方法
WO2022040965A1 (zh) 图像传输方法、装置及云台系统
WO2018010473A1 (zh) 基于智能显示设备的无人机云台转动控制方法
CN114782278A (zh) 图像去噪方法、装置、系统及电子设备
WO2018010472A1 (zh) 控制无人机云台转动的智能显示设备及其控制系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20966362

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20966362

Country of ref document: EP

Kind code of ref document: A1