WO2015070694A1 - 屏幕拼接系统和视频数据流的处理方法 - Google Patents

屏幕拼接系统和视频数据流的处理方法 Download PDF

Info

Publication number
WO2015070694A1
WO2015070694A1 PCT/CN2014/089348 CN2014089348W WO2015070694A1 WO 2015070694 A1 WO2015070694 A1 WO 2015070694A1 CN 2014089348 W CN2014089348 W CN 2014089348W WO 2015070694 A1 WO2015070694 A1 WO 2015070694A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
subsystem
data stream
video data
smooth
Prior art date
Application number
PCT/CN2014/089348
Other languages
English (en)
French (fr)
Inventor
胡春波
乐振晓
沈林杰
黄崇基
Original Assignee
杭州海康威视数字技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 杭州海康威视数字技术股份有限公司 filed Critical 杭州海康威视数字技术股份有限公司
Priority to US15/037,280 priority Critical patent/US9645784B2/en
Publication of WO2015070694A1 publication Critical patent/WO2015070694A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1423—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display
    • G06F3/1446—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display display composed of modules, e.g. video walls
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H04N21/4316—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/2624—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects for obtaining an image which is composed of whole input images, e.g. splitscreen
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/268—Signal distribution or switching
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
    • H04N7/0127—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H04N5/144—Movement detection
    • H04N5/145—Movement estimation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H04N5/21—Circuitry for suppressing or minimising disturbance, e.g. moiré or halo

Definitions

  • the present invention relates to the field of video display, and in particular to a screen splicing system and a method for processing a video data stream.
  • the typical screen splicing system used in the related art can support screen splicing and roaming of analog video signals, digital video signals, network compressed code streams, and the like.
  • display. 1 is a schematic diagram of an implementation principle of a screen splicing system according to the related art.
  • the screen splicing system may include: a video capture subsystem (which may be specifically divided into an encoding subsystem and a decoding subsystem) and a screen splicing subsystem.
  • Each subsystem can exchange video data through a high-speed external device interconnection bus (PCI Express, referred to as PCIe), and perform command parameter interaction through the main control subsystem.
  • PCI Express high-speed external device interconnection bus
  • Video capture subsystem mainly set to achieve video capture, screen cutting and video transmission, etc.
  • Typical video sources can include: analog video signal, optical / digital serial interface (SDI) / digital video interface (DVI ) / Digital video signals such as High Definition Multimedia Interface (HDMI) and network compressed streams.
  • SDI digital serial interface
  • DVI digital video interface
  • HDMI High Definition Multimedia Interface
  • Screen splicing subsystem mainly set to receive the video data sent by the video acquisition subsystem, perform corresponding video processing on the received video image according to the screen splicing related control parameters, and then output to the splicing display wall through the video output interface. display.
  • the screen splicing system pays attention to the display effect of the video image, and mainly focuses on the video clarity, the video fluency and the synchronization between the sub-screens.
  • the video fluency of the screen splicing system used in the related art completely depends on the video source frame rate
  • the current mainstream analog camera, network camera, and digital video recorder (DVR) is a set.
  • a computer system that performs image storage processing has a function of long-term recording, recording, remote monitoring, and control of images/voices, and the video frame rate is lower than the display refresh frame rate (a typical network camera (IP Camera, IPC for short).
  • IP Camera IP Camera, IPC for short.
  • the frame rate is 25HZ/30HZ, and the typical display refresh rate is 50HZ/60HZ), resulting in mainstream video sources.
  • the splicing display fluency on the screen splicing system used did not achieve the desired effect, and even the image jitter and blurring phenomenon, the user viewing experience is not comfortable enough.
  • the invention provides a screen splicing system and a processing method of a video data stream, so as to at least solve the splicing display fluency of the screen splicing system used in the related art, and even the image jitter and blur phenomenon, the user viewing experience Not comfortable enough.
  • a screen splicing system is provided.
  • the screen splicing system comprises: a video capture subsystem configured to collect the original video data stream, and output the collected original video data stream to the smooth display subsystem after the smooth image processing mode is turned on; the smooth display subsystem And connected to the video capture subsystem, configured to receive the original video data stream from the video capture subsystem, perform image processing and screen cut processing on the original video data stream in turn, and send the cut video data stream to The screen splicing subsystem; the screen splicing subsystem is connected with the smooth display subsystem, and is configured to splicing the cut video data stream and sending it to the splicing display wall.
  • the smooth display subsystem comprises: a video smoothing processing module, configured to sequentially perform image processing on each adjacent two original video image frames in the original video data stream to generate one or more smooth video image frames. And inserting each video image frame into two adjacent original video image frames that generate the video frame of the video; the video clearing the transmission module is set to insert one or more video frames of the video
  • the video data stream is screen cut and output to the screen stitching subsystem.
  • the video smoothing processing module is configured to sequentially acquire consecutive two adjacent original video image frames from the original video data stream; and perform a smooth calculation on each of the two adjacent original video image frames respectively.
  • a clear video image frame to be inserted between adjacent two original video image frames is obtained.
  • the video smoothing processing module is configured to perform downsampling image processing on the adjacent first original video image frame and the second original video image frame, respectively, to obtain a first downsampled video image frame and a second downsampled video image. And then performing motion estimation on the first downsampled video image frame and the second downsampled video image frame respectively, estimating a global motion vector, and performing global motion vector on the first original video image frame and the second original video image frame. Motion estimation, obtaining a video frame of the video to be inserted.
  • the video clear display transmission module is configured to perform screen cutting processing on the video data stream inserted into one or more video frames of the video according to the number of the spliced sub-screens in the screen splicing display screen, and cut the video data after cutting The stream is sent to the screen stitching subsystem.
  • the screen splicing system further comprises: a main control subsystem; and the main control subsystem is configured to obtain a display layer parameter set from the screen splicing subsystem and determine that the image processing mode has been turned on
  • the layer parameter set is sent to the smooth display subsystem, and the smooth image processing parameter set is obtained from the smooth display subsystem, and the smooth image processing parameter set is sent to the video capture subsystem.
  • the video capture subsystem is configured to determine whether the smooth image processing mode has been turned on.
  • the original video data stream is output to the smooth display subsystem, and when it is determined that the smooth display system is not turned on, In the image processing mode, the original video data stream is screen cut and output to the screen splicing subsystem.
  • the video capture subsystem is further configured to acquire the original video data stream via the VP, and/or to acquire the video compressed code stream via the network and decode the video compressed code stream to obtain the original video data stream.
  • a method of processing a video data stream is provided.
  • the method is applied to the above screen splicing system, comprising: receiving an original video data stream from a video acquisition subsystem, and sequentially performing image processing on each adjacent two original video image frames in the original video data stream to generate a Or a plurality of video image frames; inserting each video image frame into two adjacent original video image frames for generating the video image frame, and inserting one or more video frames of the video
  • the video data stream is screen cut and output to the screen stitching subsystem.
  • the image processing is performed on each adjacent two original video image frames in the original video data stream in sequence, and generating one or more video images of the video image includes: respectively, adjacent to the first original video image frame and The second original video image frame is subjected to downsampling image processing to obtain a first downsampled video image frame and a second downsampled video image frame; and motion estimation is performed on the first downsampled video image frame and the second downsampled video image frame, respectively, The global motion vector is estimated; the motion estimation of the first original video image frame and the second original video image frame is performed by using the global motion vector to obtain a smooth video image frame to be inserted.
  • the original video data stream from the video acquisition subsystem is received, and each adjacent two original video image frames in the original video data stream are sequentially subjected to image processing to generate one or more smooth displays.
  • a video image frame inserting each video image frame into adjacent two original video image frames that generate the video frame of the video, and performing a screen on the video data stream in which one or more video frames of the video are inserted
  • the cutting process is output to the screen splicing subsystem, that is, based on the screen splicing system used in the related art, the original video data stream collected by the video acquisition subsystem is not directly output to the screen splicing subsystem, but After the image processing of the original video data is processed and outputted to the screen splicing subsystem, the splicing display of the screen splicing system used in the related art is poorly fluent, and even image jitter and blurring may occur.
  • the user's viewing experience is not comfortable enough, which improves the smoothness of the screen splicing and significantly improves the
  • FIG. 1 is a schematic diagram of an implementation principle of a screen splicing system according to the related art
  • FIG. 2 is a structural block diagram of a screen splicing system according to an embodiment of the present invention.
  • FIG. 3 is a block diagram showing the structure of a screen splicing system in accordance with a preferred embodiment of the present invention.
  • FIG. 4 is a schematic diagram of video smoothing calculations in accordance with a preferred embodiment of the present invention.
  • FIG. 5 is a schematic diagram of a smooth video source output sequence and a screen cut transmission process in accordance with a preferred embodiment of the present invention
  • FIG. 6 is a schematic structural diagram of a screen splicing system according to a preferred embodiment of the present invention.
  • FIG. 7 is a flow chart of a method of processing a video data stream in accordance with an embodiment of the present invention.
  • the computer includes various products such as a personal computer, a server, a mobile terminal, and the like, and a device having a processing chip such as a central processing unit (CPU), a single chip microcomputer, or a digital signal processor (DSP) can be referred to as a computer.
  • a processing chip such as a central processing unit (CPU), a single chip microcomputer, or a digital signal processor (DSP)
  • CPU central processing unit
  • DSP digital signal processor
  • This manipulation transforms the data or maintains it in a location in the computer's memory system, which reconfigures or changes the operation of the computer in a manner that is understood by those skilled in the art.
  • the data structure that maintains the data is the physical location of the memory with the specific attributes defined by the format of the data.
  • the following embodiments can be applied to a computer, for example, to a personal computer (PC), or to a mobile terminal currently employed in a smart operating system, and are not limited thereto.
  • a computer for example, to a personal computer (PC), or to a mobile terminal currently employed in a smart operating system, and are not limited thereto.
  • the operating system of a computer or a mobile terminal There is no special requirement for the operating system of a computer or a mobile terminal, as long as it can detect the contact, determine whether the contact conforms to the predetermined rule, and implement the corresponding function according to the attribute of the contact.
  • the screen splicing system may include: a video capture subsystem 10 configured to collect an original video data stream, and output the collected original video data stream to the smooth display subsystem after the smooth image processing mode is turned on.
  • the smooth display subsystem 20 is connected to the video capture subsystem, and is configured to receive the original video data stream from the video capture subsystem, and sequentially perform image processing and screen cut processing on the original video data stream, and after cutting
  • the video data stream is sent to the screen splicing subsystem;
  • the screen splicing subsystem 30 is connected to the smooth display subsystem, and is configured to splicing the cut video data stream and sending it to the splicing display wall.
  • the splicing display of the screen splicing system used has poor fluency, and even image jitter and blurring may occur, and the user viewing experience is not comfortable enough.
  • the video capture subsystem collects the original video data stream, and outputs the original video data stream to the smooth display subsystem after the smooth image processing mode is turned on;
  • the smooth display subsystem receives the data stream from The original video data stream of the video acquisition subsystem sequentially performs image processing and screen cutting processing on the original video data stream, and sends the cut video data stream to the screen splicing subsystem;
  • the screen splicing subsystem pairs the cut video The data stream is spliced and sent to the splicing display wall.
  • the original video data stream collected by the video acquisition subsystem is not directly output to the screen splicing subsystem.
  • the original video data stream is subjected to smooth image processing and then output to the screen splicing subsystem, thereby solving the splicing display smoothness of the screen splicing system used in the related art, and even image jitter and The phenomenon of blurring, the user's viewing experience is not comfortable enough, which improves the smooth display of the screen stitching.
  • Sexuality significantly improves the overall display of the screen splicing system.
  • Motion Flow it can also be called double frame, which is a frame rate conversion technology, that is, a new frame is interpolated by motion compensation on the basis of the original frame. This technique can greatly improve the subjective visual effects of fast moving scenes in the human eye watching video.
  • the smoothing subsystem 20 may include: a video smoothing processing module 200 configured to sequentially perform image processing on each adjacent two original video image frames in the original video data stream to generate One Or a plurality of video image frames are displayed, and each video image frame is inserted into two adjacent original video image frames for generating the video image frame; the video clear display transmission module 202 is set to be inserted The video data stream of one or more video frames of the video is subjected to screen cutting processing and output to the screen splicing subsystem.
  • a video smoothing processing module 200 configured to sequentially perform image processing on each adjacent two original video image frames in the original video data stream to generate One Or a plurality of video image frames are displayed, and each video image frame is inserted into two adjacent original video image frames for generating the video image frame
  • the video clear display transmission module 202 is set to be inserted
  • the video data stream of one or more video frames of the video is subjected to screen cutting processing and output to the screen splicing subsystem.
  • the video smoothing processing module 200 is configured to sequentially acquire consecutive two adjacent original video image frames from the original video data stream; respectively, each of the two adjacent original video acquired.
  • the image frame performs a smooth calculation to obtain a smooth video image frame to be inserted between adjacent two original video image frames.
  • the screen splicing system is capable of supporting both open and closed video processing modes.
  • the original video data can be transmitted to the smooth display subsystem through the PCIe bus for smooth display calculation, and the smooth display subsystem can be original.
  • the video data and the video data that has been smoothly calculated are screen cut according to the smooth output sequence and transmitted out via PCIe.
  • the video smoothing processing module 200 is configured to perform downsampling image processing on the adjacent first original video image frame and the second original video image frame, respectively, to obtain a first downsampled video image frame. And second downsampling the video image frame, and then performing motion estimation on the first downsampled video image frame and the second downsampled video image frame respectively, estimating a global motion vector, and using the global motion vector to the first original video image frame and The second original video image frame is subjected to motion estimation to obtain a video frame of the video to be inserted.
  • the video smoothing calculation can include the following processing steps:
  • Step 1 Obtain two frames of original video image frames consecutively in time, that is, an original video image frame (FrameT1) at time T1 and an original video image frame (FrameT2) at time T2;
  • Step 2 Perform 1/4 on FrameT1 respectively (the 1/4 used here is only a preferred implementation manner, mainly for the purpose of facilitating the motion estimation mentioned in the following step 3, and of course, other ratios may be selected according to actual conditions.
  • 1/9 Downsampling image processing (ie, scaling 1/4 simultaneously in both its horizontal and vertical directions) to obtain a 1/4 downsampled video image frame (D-FrameT1) at time T1, 1/4 of FrameT2 Sampling image processing to obtain a 1/4 downsampled video image frame (D-FrameT2) at time T2;
  • Step 3 Perform forward motion estimation and backward motion estimation on D-FrameT1 and D-FrameT2, and then estimate the global motion vector of the downsampled video image frame.
  • the forward motion estimation uses D-FrameT2 as the current video image frame to D-FrameT1 searches for the reference video image frame
  • the backward motion estimation uses D-FrameT1 as the current video image frame
  • D-FrameT2 as the reference video image frame to search
  • Step 4 The motion estimation and the global motion vector of the sampled video image frame are performed, and the forward video and the backward motion estimation are performed on the original video image frames FrameT1 and FrameT2, and the video display frame FrameT1- is obtained by performing smooth interpolation according to the calculation result. 2.
  • the video clear cut transmission module 202 is configured to perform screen cut processing on the video data stream in which one or more video frames are inserted according to the number of stitched sub-screens in the screen splicing display screen.
  • the cut video data stream is sent to the screen stitching subsystem.
  • FIG. 5 is a schematic diagram of a smooth video source output sequence and a large screen cut transmission process according to a preferred embodiment of the present invention.
  • the number of splicing sub-screens in the splicing display wall is four, and the smooth display subsystem performs the large-screen cutting on the video source image and the smooth display image in the order in which they are displayed.
  • the original video image frame (FrameT1) at time T1 and the original video image frame (FrameT2) at time T2 are image-processed to generate a video frame frame (T1-2), and the frame (T1-2) is inserted.
  • the original video image frame (FrameT2) at time T2 and the original video image frame (FrameT3) at time T3 are image-processed to generate a video frame frame (T2-3), and Frame (T2-3) is inserted between FrameT2 and FrameT3, and so on... generates a smooth video data output sequence.
  • the video data output sequence is screen cut, and finally the cut video data sub-blocks are transmitted outward through the PCIe bus.
  • the above screen splicing system may further include: a main control subsystem 40; the main control subsystem 40 is configured to obtain from the screen splicing subsystem when it is determined that the smooth image processing mode has been turned on. Display a layer parameter set, send the display layer parameter set to the smooth display subsystem, and obtain a smooth image processing parameter set from the smooth display subsystem, and send the smooth image processing parameter set to the video capture subsystem.
  • the main control subsystem needs to control each subsystem through network commands, PCIe host commands, and the like.
  • Video data is transmitted between the PCIe bus to enable the opening and closing of various subsystem functions and the cooperation between the various systems.
  • the main control subsystem can obtain the display layer related parameters of the screen splicing subsystem according to the operation result of the user, for example, display coordinates, layer split parameters, and receiving address of the PCIe transmission. Then, the main control subsystem performs statistical sorting on the display layer related parameters, and sends the display layer related parameters to the smooth display subsystem, and simultaneously starts the smooth display mode.
  • the main control subsystem also receives the smooth video source receiving parameters from the smooth display subsystem (which may include: clear channel, smooth display resource, receiving address of PCIe transmission), and enable smooth PCIe reception.
  • the main control subsystem also sends the smooth source switching parameters to the video acquisition subsystem, and starts the smooth video transmission of the video acquisition subsystem.
  • the video capture subsystem 10 is configured to determine whether the smooth image processing mode has been currently enabled. When it is determined that the smooth image processing mode has been turned on, the original video data stream is output to the smooth display subsystem. When it is determined that the smooth image processing mode is not turned on, the original video data stream is screen cut and output to the screen stitching subsystem.
  • the video capture subsystem 10 is further configured to acquire the original video data stream via the VP, and/or acquire the video compressed code stream via the network, and decode the video compressed code stream to obtain the original video data. flow.
  • the video capture subsystem can acquire the original video data through the VP for video pre-processing, encoding, and PCIe wall display. If the smooth display function is enabled, the original video data can be transmitted to the Changxian via PCIe.
  • the original video data can only be transmitted to the screen splicing subsystem via PCIe after the screen is cut; the video capture subsystem can also obtain the video compressed code stream through the network, and adopt decoding processing to obtain The original video data, and then according to whether the smooth display function is enabled for PCIe transmission, wherein if the smooth display function is enabled, the original video data can be transmitted to the smooth display subsystem through PCIe, and if the smooth display function is turned off, only the The original video data is screen cut and transmitted to the screen stitching subsystem via PCIe.
  • FIG. 6 is a block diagram showing the structure of a screen splicing system in accordance with a preferred embodiment of the present invention.
  • the screen splicing system provided by the preferred embodiment of the present invention may include: a video capture subsystem, a smooth display subsystem, and a screen splicing subsystem.
  • the video data can be transmitted between the various subsystems through the PCIe bus, and the opening and closing of each subsystem function and the mutual cooperation between the various systems can be performed by the main control subsystem (not shown) through the network command, the PCIe host. Commands, etc. are controlled.
  • Video capture subsystem mainly set to achieve video capture, screen cutting and video transmission, etc.
  • Typical video sources can include: analog video signal, optical / digital serial interface (SDI) / digital video interface (DVI ) / Digital video signals such as High Definition Multimedia Interface (HDMI) and network compressed streams.
  • SDI digital serial interface
  • DVI digital video interface
  • HDMI High Definition Multimedia Interface
  • the video capture subsystem may further include: an encoding subsystem and a decoding subsystem.
  • the encoding subsystem can capture the original video data through the video port (VP) for video pre-processing, encoding, and PCIe wall display. If the smooth display function is enabled, the original video data can be transmitted through PCIe. Input to the smooth display subsystem, and if the smooth display function is turned off, the original video data can only be transferred to the screen splicing subsystem via PCIe after screen cutting;
  • VP video port
  • PCIe PCIe wall display
  • the decoding subsystem can obtain the video compressed code stream through the network, and adopt decoding processing to obtain the original video data, and then perform PCIe transmission according to whether the smooth display function is enabled, wherein if the smooth display function is enabled, the original video data can be passed.
  • the PCIe is transmitted to the smooth display subsystem, and if the smooth display function is turned off, the original video data can only be transferred to the screen splicing subsystem via PCIe after the screen is cut.
  • the main setting is used when the screen splicing system is turned on and the splicing function is enabled, which may include: a video smoothing pre-processing module, a video smooth display computing module, and a video smooth display cutting module;
  • the video smoothing pre-processing module is configured to receive the original video data via local transmission or PCIe transmission, and perform video pre-processing such as scaling according to smooth display requirements;
  • the video display module is set to obtain the pre-processed video data, and the smooth calculation library is called to perform video smooth display calculation, and the smooth display image is output;
  • the video clear display transmission module is set to obtain the original video data and the video data after the smooth processing, and the screen data is cut according to the video smooth output sequence and the screen cutting related control parameters, and then the screen is cut.
  • the video segmentation block after the screen cut processing is transmitted to each of the tiled display subscreens via PCIe.
  • Screen splicing subsystem mainly configured to receive video segmentation blocks via PCIe (including: smooth video segmentation block and original video segmentation block), and perform received video segmentation blocks according to screen splicing or screen roaming related display control parameters. Zoom, deinterlace, transparent calculation, layer overlay, etc., and then output to the splicing display wall through the video output interface for display.
  • PCIe including: smooth video segmentation block and original video segmentation block
  • the above-described screen splicing system is capable of supporting two video processing modes of turning on and off.
  • the screen splicing system is turned on, after the video capture subsystem obtains the original video data, the original video data can be transmitted to the smooth display subsystem through the PCIe bus for smooth display calculation, and the smooth display subsystem can be original.
  • the video data and the video data that has been smoothly calculated are screen cut according to the smooth output sequence and transmitted out via PCIe.
  • the screen splicing subsystem can receive each video segmentation block via PCIe and output the display after performing layer overlay processing and the like.
  • Main control subsystem Mainly set to obtain the display layer related parameters of the screen splicing subsystem according to the operation result of the user, for example: display coordinates, layer split parameters, and receiving address of PCIe transmission. Master subsystem Statistics are displayed on the display layer related parameters, and the display layer related parameters are sent to the smooth display subsystem, and the smooth display mode is also enabled.
  • the main control subsystem receives the smooth video source receiving parameters from the smooth display subsystem (which may include: clear channel, smooth display resource, receiving address of PCIe transmission), and open PCIe reception.
  • the main control subsystem sends the smooth source switching parameters to the video acquisition subsystem and starts the smooth video transmission of the video acquisition subsystem. At this point, the screen splicing system works in the smooth stitching mode.
  • FIG. 7 is a flow chart of a method of processing a video data stream in accordance with an embodiment of the present invention. The method is applied to the above screen splicing system. As shown in FIG. 7, the method may include the following processing steps:
  • Step S702 Receive the original video data stream from the video collection subsystem, and perform smooth image processing on each adjacent two original video image frames in the original video data stream to generate one or more smooth video image frames.
  • Step S704 Insert each video image frame into two adjacent original video image frames for generating the video image frame, and perform screen cutting on the video data stream in which one or more video frames are inserted. Processing, output to the screen stitching subsystem.
  • the method shown in FIG. 7 solves the problem that the splicing display of the screen splicing system used in the related art has poor fluency, and even the image jitter and blur phenomenon occur, and the user's viewing experience is not comfortable enough, thereby improving the screen.
  • the splicing display fluency significantly improves the overall display of the screen splicing system.
  • step S702 image processing is performed on each adjacent two original video image frames in the original video data stream in sequence, and generating one or more video display image frames may include the following operations:
  • Step S1 performing down-sampling image processing on the adjacent first original video image frame and the second original video image frame, respectively, to obtain a first downsampled video image frame and a second downsampled video image frame;
  • Step S2 performing motion estimation on the first downsampled video image frame and the second downsampled video image frame, respectively, and estimating a global motion vector;
  • Step S3 Perform motion estimation on the first original video image frame and the second original video image frame by using a global motion vector to obtain a smooth video image frame to be inserted.
  • the embodiment of the present invention provides a video display for screen stitching.
  • a technical solution for smooth splicing and/or roaming the technical solution can realize a set of video by adding a step of calculating the video source between the video capture subsystem and the screen splicing subsystem. Collection, screen splicing, screen roaming, smooth splicing, and smooth display of the screen in one screen.
  • the screen is smooth
  • the system may include: a video capture subsystem, a video display subsystem, a screen cut transmission subsystem, and a screen splicing display subsystem.
  • the screen display system realizes smooth splicing and smooth roaming, and obviously improves the display of the screen splicing. Fluency, which significantly improves the overall display of the screen splicing system.
  • modules or steps of the present invention described above can be implemented by a general-purpose computing device that can be centralized on a single computing device or distributed across a network of multiple computing devices. Alternatively, they may be implemented by program code executable by the computing device such that they may be stored in the storage device by the computing device and, in some cases, may be different from the order herein.
  • the steps shown or described are performed, or they are separately fabricated into individual integrated circuit modules, or a plurality of modules or steps thereof are fabricated as a single integrated circuit module.
  • the invention is not limited to any specific combination of hardware and software.
  • the screen splicing system and the video data stream processing method provided by the embodiments of the present invention have the following beneficial effects: based on the screen splicing system used in the related art, the video acquisition subsystem is no longer collected. The original video data stream is directly output to the screen splicing subsystem, but is outputted to the screen splicing subsystem after performing smooth image processing on the original video data stream, thereby improving the smoothness of the screen splicing display and significantly improving the screen. The overall display effect of the splicing system.

Abstract

本发明公开了一种屏幕拼接系统和视频数据流的处理方法，在上述系统中，视频采集子系统，设置为采集原始视频数据流，并在开启畅显图像处理模式后将采集到的原始视频数据流输出至畅显子系统；畅显子系统，与视频采集子系统相连接，设置为接收来自于视频采集子系统的原始视频数据流，依次对原始视频数据流进行畅显图像处理和屏幕切割处理，并将切割后的视频数据流发送至屏幕拼接子系统；屏幕拼接子系统，与畅显子系统相连接，设置为对切割后的视频数据流进行拼接处理，并发送至拼接显示墙。根据本发明提供的技术方案，提高了屏幕拼接的显示流畅性，显著改善了屏幕拼接系统的整体显示效果。

Description

屏幕拼接系统和视频数据流的处理方法 技术领域
本发明涉及视频显示领域，具体而言，涉及一种屏幕拼接系统和视频数据流的处理方法。
背景技术
目前，相关技术中所采用的典型的屏幕拼接系统(采用多个显示屏拼接的显示方式，实现单屏的显示功能)能够支持模拟视频信号、数字视频信号、网络压缩码流等屏幕拼接和漫游显示。图1是根据相关技术的屏幕拼接系统的实现原理示意图。如图1所示，屏幕拼接系统可以包括：视频采集子系统(具体可以分为编码子系统和解码子系统)以及屏幕拼接子系统。各个子系统相互之间可以通过高速外部设备互联总线(PCI Express，简称为PCIe)进行视频数据交互，通过主控子系统进行命令参数交互。
下面将分别对屏幕拼接系统中的各个子系统的功能加以介绍。
(1)视频采集子系统：主要设置为实现视频采集，屏幕切割和视频传输等功能，其典型的视频源可以包括：模拟视频信号、光纤/数字串行接口(SDI)/数字视频接口(DVI)/高清晰度多媒体接口(HDMI)等数字视频信号和网络压缩码流。
(2)屏幕拼接子系统：主要设置为接收视频采集子系统发送的视频数据，根据屏幕拼接相关控制参数对接收到的视频图像进行相应的视频处理，然后通过视频输出接口输出至拼接显示墙加以显示。
由此可见，屏幕拼接系统注重视频图像的显示效果，其主要关注视频清晰度、视频流畅性以及各个子屏相互之间的同步性。然而，由于相关技术中所采用的屏幕拼接系统的视频流畅性完全依赖于视频源帧率，而时下主流的模拟摄像机、网络摄像机、数字硬盘录像机(Digital Video Recorder，简称为DVR，它是一套进行图像存储处理的计算机系统，具有对图像/语音进行长时间录像、录音、远程监视和控制的功能)等视频帧率均低于显示刷新帧率(典型网络摄像机(IP Camera，简称为IPC，是一种可产生数字视频流，并将视频流通过有线或无线网络进行传输的摄像机)的帧率为25HZ/30HZ，典型的显示刷新帧率为50HZ/60HZ)，从而导致主流视频源在目前所使用的屏幕拼接系统上的拼接显示流畅性并未取得理想效果，甚至会出现图像抖动和模糊的现象，用户观看体验不够舒适。
发明内容
本发明提供了一种屏幕拼接系统和视频数据流的处理方法，以至少解决相关技术中所采用的屏幕拼接系统的拼接显示流畅性较差，甚至会出现图像抖动和模糊的现象，用户观看体验不够舒适的问题。
根据本发明的一个方面，提供了一种屏幕拼接系统。
根据本发明的屏幕拼接系统包括：视频采集子系统，设置为采集原始视频数据流，并在开启畅显图像处理模式后将采集到的原始视频数据流输出至畅显子系统；畅显子系统，与视频采集子系统相连接，设置为接收来自于视频采集子系统的原始视频数据流，依次对原始视频数据流进行畅显图像处理和屏幕切割处理，并将切割后的视频数据流发送至屏幕拼接子系统；屏幕拼接子系统，与畅显子系统相连接，设置为对切割后的视频数据流进行拼接处理，并发送至拼接显示墙。
优选地，畅显子系统包括：视频畅显处理模块，设置为依次对原始视频数据流中的每相邻两个原始视频图像帧进行畅显图像处理，生成一个或多个畅显视频图像帧，并将每个畅显视频图像帧插入至生成该畅显视频图像帧的相邻两个原始视频图像帧中；视频畅显切割传输模块，设置为对插入一个或多个畅显视频图像帧的视频数据流进行屏幕切割处理，输出至屏幕拼接子系统。
优选地，视频畅显处理模块，设置为依次从原始视频数据流中获取时间连续的相邻两个原始视频图像帧；分别对获取到的每相邻两个原始视频图像帧执行畅显计算，得到待插入至相邻两个原始视频图像帧之间的畅显视频图像帧。
优选地，视频畅显处理模块，设置为分别对相邻的第一原始视频图像帧和第二原始视频图像帧进行下采样图像处理，得到第一下采样视频图像帧和第二下采样视频图像帧，然后分别对第一下采样视频图像帧和第二下采样视频图像帧进行运动估计，估算出全局运动矢量，以及采用全局运动矢量对第一原始视频图像帧和第二原始视频图像帧进行运动估计，得到待插入的畅显视频图像帧。
优选地，视频畅显切割传输模块，设置为按照屏幕拼接显示屏中的拼接子屏数量对插入一个或多个畅显视频图像帧的视频数据流进行屏幕切割处理，并将切割后的视频数据流发送至屏幕拼接子系统。
优选地，屏幕拼接系统还包括：主控子系统；主控子系统，设置为在确定已经开启畅显图像处理模式的情况下，从屏幕拼接子系统获取显示图层参数集合，并将显示 图层参数集合发送至畅显子系统，以及从畅显子系统获取畅显图像处理参数集合，并将畅显图像处理参数集合发送至视频采集子系统。
优选地，视频采集子系统，设置为判断当前是否已经开启畅显图像处理模式，当确定已经开启畅显图像处理模式时，将原始视频数据流输出至畅显子系统，当确定未开启畅显图像处理模式时，将原始视频数据流进行屏幕切割后输出至屏幕拼接子系统。
优选地，视频采集子系统，还设置为经由VP采集原始视频数据流，和/或，经由网络获取视频压缩码流，并对视频压缩码流进行解码获取原始视频数据流。
根据本发明的另一方面，提供了一种视频数据流的处理方法。
该方法应用于上述屏幕拼接系统中，包括：接收来自于视频采集子系统的原始视频数据流，依次对原始视频数据流中的每相邻两个原始视频图像帧进行畅显图像处理，生成一个或多个畅显视频图像帧；将每个畅显视频图像帧插入至生成该畅显视频图像帧的相邻两个原始视频图像帧中，并对插入一个或多个畅显视频图像帧的视频数据流进行屏幕切割处理，输出至屏幕拼接子系统。
优选地，依次对原始视频数据流中的每相邻两个原始视频图像帧进行畅显图像处理，生成一个或多个畅显视频图像帧包括：分别对相邻的第一原始视频图像帧和第二原始视频图像帧进行下采样图像处理，得到第一下采样视频图像帧和第二下采样视频图像帧；分别对第一下采样视频图像帧和第二下采样视频图像帧进行运动估计，估算出全局运动矢量；采用全局运动矢量对第一原始视频图像帧和第二原始视频图像帧进行运动估计，得到待插入的畅显视频图像帧。
通过本发明实施例，采用接收来自于视频采集子系统的原始视频数据流，依次对原始视频数据流中的每相邻两个原始视频图像帧进行畅显图像处理，生成一个或多个畅显视频图像帧；将每个畅显视频图像帧插入至生成该畅显视频图像帧的相邻两个原始视频图像帧中，并对插入一个或多个畅显视频图像帧的视频数据流进行屏幕切割处理，输出至屏幕拼接子系统，即在相关技术中所使用的屏幕拼接系统的基础上，不再将视频采集子系统所采集到的原始视频数据流直接输出至屏幕拼接子系统，而是在对原始视频数据流进行畅显图像处理后再输出至屏幕拼接子系统，由此解决了相关技术中所采用的屏幕拼接系统的拼接显示流畅性较差，甚至会出现图像抖动和模糊的现象，用户观看体验不够舒适的问题，进而提高了屏幕拼接的显示流畅性，显著改善了屏幕拼接系统的整体显示效果。
附图说明
此处所说明的附图用来提供对本发明的进一步理解，构成本申请的一部分，本发明的示意性实施例及其说明用于解释本发明，并不构成对本发明的不当限定。在附图中：
图1是根据相关技术的屏幕拼接系统的实现原理示意图；
图2是根据本发明实施例的屏幕拼接系统的结构框图；
图3是根据本发明优选实施例的屏幕拼接系统的结构框图；
图4是根据本发明优选实施例的视频畅显计算的示意图；
图5是根据本发明优选实施例的畅显视频源输出序列和屏幕切割传输处理过程示意图；
图6是根据本发明优选实施例的屏幕拼接系统的结构示意图；
图7是根据本发明实施例的视频数据流的处理方法的流程图。
具体实施方式
下文中将参考附图并结合实施例来详细说明本发明。需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。
在以下描述中，除非另外指明，否则将参考由一个或多个计算机执行的动作和操作的符号表示来描述本申请的各实施例。其中，计算机包括个人计算机、服务器、移动终端等各种产品，使用了中央处理器(CPU)、单片机、数字信号处理器(DSP)等具有处理芯片的设备均可以称为计算机。由此，可以理解，有时被称为计算机执行的这类动作和操作包括计算机的处理单元对以结构化形式表示数据的电信号的操纵。这一操纵转换了数据或在计算机的存储器系统中的位置上维护它，这以本领域的技术人员都理解的方式重配置或改变了计算机的操作。维护数据的数据结构是具有数据的格式所定义的特定属性的存储器的物理位置。然而，尽管在上述上下文中描述本发明，但它并不意味着限制性的，如本领域的技术人员所理解的，后文所描述的动作和操作的各方面也可用硬件来实现。
转向附图，其中相同的参考标号指代相同的元素，本申请的原理被示为在一个合适的计算环境中实现。以下描述基于所述的本申请的实施例，并且不应认为是关于此处未明确描述的替换实施例而限制本申请。
以下实施例可以应用到计算机中，例如：应用到个人计算机(PC)中，也可以应用到目前采用了智能操作系统中的移动终端中，并且并不限于此。对于计算机或移动终端的操作系统并没有特殊要求，只要能够检测接触、确定该接触是否与预定规则相符合，以及根据该接触的属性实现相应功能即可。
图2是根据本发明实施例的屏幕拼接系统的结构框图。如图2所示，该屏幕拼接系统可以包括：视频采集子系统10，设置为采集原始视频数据流，并在开启畅显图像处理模式后将采集到的原始视频数据流输出至畅显子系统；畅显子系统20，与视频采集子系统相连接，设置为接收来自于视频采集子系统的原始视频数据流，依次对原始视频数据流进行畅显图像处理和屏幕切割处理，并将切割后的视频数据流发送至屏幕拼接子系统；屏幕拼接子系统30，与畅显子系统相连接，设置为对切割后的视频数据流进行拼接处理，并发送至拼接显示墙。
相关技术中，所采用的屏幕拼接系统的拼接显示流畅性较差，甚至会出现图像抖动和模糊的现象，用户观看体验不够舒适。采用如图2所示的系统，视频采集子系统采集原始视频数据流，并在开启畅显图像处理模式后将采集到的原始视频数据流输出至畅显子系统；畅显子系统接收来自于视频采集子系统的原始视频数据流，依次对原始视频数据流进行畅显图像处理和屏幕切割处理，并将切割后的视频数据流发送至屏幕拼接子系统；屏幕拼接子系统对切割后的视频数据流进行拼接处理，并发送至拼接显示墙，即在相关技术中所使用的屏幕拼接系统的基础上，不再将视频采集子系统所采集到的原始视频数据流直接输出至屏幕拼接子系统，而是在对原始视频数据流进行畅显图像处理后再输出至屏幕拼接子系统，由此解决了相关技术中所采用的屏幕拼接系统的拼接显示流畅性较差，甚至会出现图像抖动和模糊的现象，用户观看体验不够舒适的问题，进而提高了屏幕拼接的显示流畅性，显著改善了屏幕拼接系统的整体显示效果。
需要说明的是，畅显(Motion Flow)：又可以称为倍帧，是一种帧率转换技术，即在原有帧的基础上，通过运动补偿的方式插值出新的一帧。这种技术可以在很大程度上改善人眼观看视频中快速运动场景的主观视觉效果。
优选地，如图3所示，畅显子系统20可以包括：视频畅显处理模块200，设置为依次对原始视频数据流中的每相邻两个原始视频图像帧进行畅显图像处理，生成一个 或多个畅显视频图像帧，并将每个畅显视频图像帧插入至生成该畅显视频图像帧的相邻两个原始视频图像帧中；视频畅显切割传输模块202，设置为对插入一个或多个畅显视频图像帧的视频数据流进行屏幕切割处理，输出至屏幕拼接子系统。
优选地，如图3所示，视频畅显处理模块200，设置为依次从原始视频数据流中获取时间连续的相邻两个原始视频图像帧；分别对获取到的每相邻两个原始视频图像帧执行畅显计算，得到待插入至相邻两个原始视频图像帧之间的畅显视频图像帧。
在优选实施例中，屏幕拼接系统能够支持开启畅显和关闭畅显两种视频处理模式。在屏幕拼接系统开启畅显模式的情况下，视频采集子系统在获取到原始视频数据后，可以通过PCIe总线将原始视频数据传输至畅显子系统进行畅显计算，畅显子系统能够将原始视频数据和经过畅显计算得到的视频数据按照畅显输出序列进行屏幕切割以及经由PCIe向外传输。
优选地，如图3所示，视频畅显处理模块200，设置为分别对相邻的第一原始视频图像帧和第二原始视频图像帧进行下采样图像处理，得到第一下采样视频图像帧和第二下采样视频图像帧，然后分别对第一下采样视频图像帧和第二下采样视频图像帧进行运动估计，估算出全局运动矢量，以及采用全局运动矢量对第一原始视频图像帧和第二原始视频图像帧进行运动估计，得到待插入的畅显视频图像帧。
图4是根据本发明优选实施例的视频畅显计算的示意图。如图4所示，视频畅显计算可以包括以下处理步骤：
步骤一、获取时间上前后连续的两帧原始视频图像帧，即T1时刻的原始视频图像帧(FrameT1)和T2时刻的原始视频图像帧(FrameT2)；
步骤二、分别对FrameT1进行1/4(此处所采用的1/4仅是一种优选实施方式，主要是为了便于下面步骤三中提到的运动估计，当然还可以根据实际情况选取其他比例，例如：1/9)下采样图像处理(即在其水平方向和垂直方向同时缩放1/4)获得T1时刻的1/4下采样视频图像帧(D-FrameT1)，对FrameT2进行1/4下采样图像处理获得T2时刻的1/4下采样视频图像帧(D-FrameT2)；
步骤三、对D-FrameT1和D-FrameT2进行前向运动估计和后向运动估计，进而估计出下采样视频图像帧的全局运动矢量，前向运动估计以D-FrameT2为当前视频图像帧，以D-FrameT1为参考视频图像帧进行搜索，后向运动估计以D-FrameT1为当前视频图像帧，以D-FrameT2为参考视频图像帧进行搜索；
步骤四、以下采样视频图像帧的运动估计和全局运动矢量，对原始视频图像帧FrameT1和FrameT2进行前向和后向运动估计，并根据计算结果进行畅显插值从而得到畅显视频图像帧FrameT1-2。
优选地，如图3所示，视频畅显切割传输模块202，设置为按照屏幕拼接显示屏中的拼接子屏数量对插入一个或多个畅显视频图像帧的视频数据流进行屏幕切割处理，并将切割后的视频数据流发送至屏幕拼接子系统。
图5是根据本发明优选实施例的畅显视频源输出序列和大屏切割传输处理过程示意图。如图5所示，在该优选实施例中，拼接显示墙中的拼接子屏数量为4个，畅显子系统将视频源图像和畅显计算图像按其先后显示顺序进行大屏切割，首先对T1时刻的原始视频图像帧(FrameT1)和T2时刻的原始视频图像帧(FrameT2)进行畅显图像处理，生成畅显视频图像帧Frame(T1-2)，并将Frame(T1-2)插入FrameT1和FrameT2之间；其次对T2时刻的原始视频图像帧(FrameT2)和T3时刻的原始视频图像帧(FrameT3)进行畅显图像处理，生成畅显视频图像帧Frame(T2-3)，并将Frame(T2-3)插入FrameT2和FrameT3之间，以此类推…生成畅显视频数据输出序列。然后按照拼接显示墙中的拼接子屏数量对畅显视频数据输出序列进行屏幕切割，最后将切割后的各个视频数据子块通过PCIe总线向外传输。
优选地，如图3所示，上述屏幕拼接系统还可以包括：主控子系统40；主控子系统40，设置为在确定已经开启畅显图像处理模式的情况下，从屏幕拼接子系统获取显示图层参数集合，并将显示图层参数集合发送至畅显子系统，以及从畅显子系统获取畅显图像处理参数集合，并将畅显图像处理参数集合发送至视频采集子系统。
在优选实施例中，由于视频采集子系统、畅显子系统和屏幕拼接子系统相互之间无法直接进行信息交互，因此，需要采用主控子系统通过网络命令、PCIe主机命令等控制各个子系统之间通过PCIe总线传输视频数据，实现各个子系统功能的开启与关闭以及各个系统之间的相互配合。首先，主控子系统可以根据用户的操作结果，获取屏幕拼接子系统的显示图层相关参数，例如：显示坐标、图层分割参数、PCIe传输的接收地址。然后，主控子系统对显示图层相关参数进行统计整理，并将显示图层相关参数发送至畅显子系统，同时开启畅显模式。此外，主控子系统还接收来自于畅显子系统的畅显视频源接收参数(可以包括：畅显通道、畅显资源、PCIe传输的接收地址)，并开启畅显PCIe接收。另外，主控子系统还将畅显源切换参数发送至视频采集子系统，并开启视频采集子系统的畅显视频传输。
优选地，如图3所示，视频采集子系统10，设置为判断当前是否已经开启畅显图像处理模式，当确定已经开启畅显图像处理模式时，将原始视频数据流输出至畅显子系统，当确定未开启畅显图像处理模式时，将原始视频数据流进行屏幕切割后输出至屏幕拼接子系统。
优选地，如图3所示，视频采集子系统10，还设置为经由VP采集原始视频数据流，和/或，经由网络获取视频压缩码流，并对视频压缩码流进行解码获取原始视频数据流。
在优选实施例中，视频采集子系统可以通过VP采集原始视频数据进行视频预处理、编码以及PCIe上墙显示，其中，如果开启畅显功能，则可以将原始视频数据通过PCIe传输至畅显子系统，而如果关闭畅显功能，则只能在将原始视频数据进行屏幕切割后经由PCIe传输至屏幕拼接子系统；视频采集子系统还可以通过网络获取视频压缩码流，并采用解码处理以获取原始视频数据，然后根据是否开启畅显功能进行PCIe传输，其中，如果开启畅显功能，则可以将原始视频数据通过PCIe传输至畅显子系统，而如果关闭畅显功能，则只能在将原始视频数据进行屏幕切割后经由PCIe传输至屏幕拼接子系统。
下面将结合图6所示的优选实施方式对上述优选实施过程作进一步的描述。
图6是根据本发明优选实施例的屏幕拼接系统的结构示意图。如图6所示，本发明优选实施例所提供的屏幕拼接系统可以包括：视频采集子系统、畅显子系统和屏幕拼接子系统。各个子系统之间可以通过PCIe总线传输视频数据，并且各个子系统功能的开启与关闭以及各个系统之间的相互配合均可以由主控子系统(图中未示出)通过网络命令、PCIe主机命令等进行控制。
下面将分别对该优选实施例提供的屏幕拼接系统中的各个子系统的功能加以介绍。
(1)视频采集子系统：主要设置为实现视频采集，屏幕切割和视频传输等功能，其典型的视频源可以包括：模拟视频信号、光纤/数字串行接口(SDI)/数字视频接口(DVI)/高清晰度多媒体接口(HDMI)等数字视频信号和网络压缩码流。
该视频采集子系统又可以进一步包括：编码子系统和解码子系统。
1)编码子系统可以通过视频口(VP)采集原始视频数据进行视频预处理、编码以及PCIe上墙显示，其中，如果开启畅显功能，则可以将原始视频数据通过PCIe传 输至畅显子系统，而如果关闭畅显功能，则只能在将原始视频数据进行屏幕切割后经由PCIe传输至屏幕拼接子系统；
2)解码子系统可以通过网络获取视频压缩码流，并采用解码处理以获取原始视频数据，然后根据是否开启畅显功能进行PCIe传输，其中，如果开启畅显功能，则可以将原始视频数据通过PCIe传输至畅显子系统，而如果关闭畅显功能，则只能在将原始视频数据进行屏幕切割后经由PCIe传输至屏幕拼接子系统。
(2)畅显子系统：主要设置为在屏幕拼接系统开启畅显拼接功能时使用，其中，可以包括：视频畅显预处理模块、视频畅显计算模块和视频畅显切割传输模块；
1)视频畅显预处理模块，设置为经由本地传输或PCIe传输接收原始视频数据，并根据畅显需求进行缩放等视频畅显预处理；
2)视频畅显计算模块，设置为获取经过预处理后的视频数据，调用畅显计算库进行视频畅显计算，输出畅显图像；
需要说明的是，视频畅显预处理模块与视频畅显计算模块所能够实现的功能总和相当于上述视频畅显处理模块。
3)视频畅显切割传输模块，设置为获取原始视频数据和经过畅显处理后的视频数据，根据视频畅显输出序列和屏幕切割相关控制参数对获取到的视频数据进行屏幕切割，然后将经过屏幕切割处理后的视频分割块经由PCIe传输至各个拼接显示子屏。
(3)屏幕拼接子系统：主要设置为经由PCIe接收视频分割块(包括：畅显视频分割块和原始视频分割块)，根据屏幕拼接或屏幕漫游相关显示控制参数对接收到的视频分割块进行缩放、去隔行、透明计算、图层叠加等处理，然后通过视频输出接口输出至拼接显示墙加以显示。
在该优选实施例中，上述屏幕拼接系统能够支持开启畅显和关闭畅显两种视频处理模式。在屏幕拼接系统开启畅显模式的情况下，视频采集子系统在获取到原始视频数据后，可以通过PCIe总线将原始视频数据传输至畅显子系统进行畅显计算，畅显子系统能够将原始视频数据和经过畅显计算得到的视频数据按照畅显输出序列进行屏幕切割以及经由PCIe向外传输。屏幕拼接子系统能够经由PCIe接收各个视频分割块，并在进行图层叠加等处理后输出显示。
(4)主控子系统：主要设置为根据用户的操作结果，获取屏幕拼接子系统的显示图层相关参数，例如：显示坐标、图层分割参数、PCIe传输的接收地址。主控子系统 对显示图层相关参数进行统计整理，并将显示图层相关参数发送至畅显子系统，同时开启畅显模式。主控子系统接收来自于畅显子系统的畅显视频源接收参数(可以包括：畅显通道、畅显资源、PCIe传输的接收地址)，并开启畅显PCIe接收。主控子系统将畅显源切换参数发送至视频采集子系统，并开启视频采集子系统的畅显视频传输。至此，屏幕拼接系统在畅显拼接模式下工作。
图7是根据本发明实施例的视频数据流的处理方法的流程图。该方法应用于上述屏幕拼接系统中，如图7所示，该方法可以包括以下处理步骤：
步骤S702：接收来自于视频采集子系统的原始视频数据流，依次对原始视频数据流中的每相邻两个原始视频图像帧进行畅显图像处理，生成一个或多个畅显视频图像帧；
步骤S704：将每个畅显视频图像帧插入至生成该畅显视频图像帧的相邻两个原始视频图像帧中，并对插入一个或多个畅显视频图像帧的视频数据流进行屏幕切割处理，输出至屏幕拼接子系统。
采用如图7所示的方法，解决了相关技术中所采用的屏幕拼接系统的拼接显示流畅性较差，甚至会出现图像抖动和模糊的现象，用户观看体验不够舒适的问题，进而提高了屏幕拼接的显示流畅性，显著改善了屏幕拼接系统的整体显示效果。
优选地，在步骤S702中，依次对原始视频数据流中的每相邻两个原始视频图像帧进行畅显图像处理，生成一个或多个畅显视频图像帧可以包括以下操作：
步骤S1：分别对相邻的第一原始视频图像帧和第二原始视频图像帧进行下采样图像处理，得到第一下采样视频图像帧和第二下采样视频图像帧；
步骤S2：分别对第一下采样视频图像帧和第二下采样视频图像帧进行运动估计，估算出全局运动矢量；
步骤S3：采用全局运动矢量对第一原始视频图像帧和第二原始视频图像帧进行运动估计，得到待插入的畅显视频图像帧。
从以上的描述中，可以看出，上述实施例实现了如下技术效果(需要说明的是这些效果是某些优选实施例可以达到的效果)：本发明实施例提供了一种对屏幕拼接显示视频进行畅显拼接和/或畅显漫游的技术方案，该技术方案可以通过在视频采集子系统和屏幕拼接子系统之间新增对视频源进行畅显计算的步骤，从而实现了一种集视频采集、屏幕拼接、屏幕漫游、畅显拼接、畅显漫游于一体的屏幕畅显系统。该屏幕畅显 系统可以包括：视频采集子系统、视频畅显计算子系统、屏幕切割传输子系统和屏幕拼接显示子系统，该屏幕畅显系统实现了畅显拼接与畅显漫游，明显提高了屏幕拼接的显示流畅性，从而显著提升了屏幕拼接系统的整体显示效果。
显然，本领域的技术人员应该明白，上述的本发明的各模块或各步骤可以用通用的计算装置来实现，它们可以集中在单个的计算装置上，或者分布在多个计算装置所组成的网络上，可选地，它们可以用计算装置可执行的程序代码来实现，从而，可以将它们存储在存储装置中由计算装置来执行，并且在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤，或者将它们分别制作成各个集成电路模块，或者将它们中的多个模块或步骤制作成单个集成电路模块来实现。这样，本发明不限制于任何特定的硬件和软件结合。
以上所述仅为本发明的优选实施例而已，并不用于限制本发明，对于本领域的技术人员来说，本发明可以有各种更改和变化。凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。
工业实用性
如上所述，本发明实施例提供的一种屏幕拼接系统和视频数据流的处理方法具有以下有益效果：在相关技术中所使用的屏幕拼接系统的基础上，不再将视频采集子系统所采集到的原始视频数据流直接输出至屏幕拼接子系统，而是在对原始视频数据流进行畅显图像处理后再输出至屏幕拼接子系统，进而提高了屏幕拼接的显示流畅性，显著改善了屏幕拼接系统的整体显示效果。

Claims (10)

  1. 一种屏幕拼接系统，包括：
    视频采集子系统，设置为采集原始视频数据流，并在开启畅显图像处理模式后将采集到的所述原始视频数据流输出至畅显子系统；
    所述畅显子系统，与所述视频采集子系统相连接，设置为接收来自于所述视频采集子系统的所述原始视频数据流，依次对所述原始视频数据流进行畅显图像处理和屏幕切割处理，并将切割后的视频数据流发送至屏幕拼接子系统；
    所述屏幕拼接子系统，与所述畅显子系统相连接，设置为对所述切割后的视频数据流进行拼接处理，并发送至拼接显示墙。
  2. 根据权利要求1所述的系统，其中，所述畅显子系统包括：
    视频畅显处理模块，设置为依次对所述原始视频数据流中的每相邻两个原始视频图像帧进行畅显图像处理，生成一个或多个畅显视频图像帧，并将每个畅显视频图像帧插入至生成该畅显视频图像帧的相邻两个原始视频图像帧中；
    视频畅显切割传输模块，设置为对插入所述一个或多个畅显视频图像帧的视频数据流进行屏幕切割处理，输出至所述屏幕拼接子系统。
  3. 根据权利要求2所述的系统，其中，所述视频畅显处理模块，设置为依次从所述原始视频数据流中获取时间连续的相邻两个原始视频图像帧；分别对获取到的每相邻两个原始视频图像帧执行畅显计算，得到待插入至所述相邻两个原始视频图像帧之间的畅显视频图像帧。
  4. 根据权利要求3所述的系统，其中，所述视频畅显处理模块，设置为分别对相邻的第一原始视频图像帧和第二原始视频图像帧进行下采样图像处理，得到第一下采样视频图像帧和第二下采样视频图像帧，然后分别对所述第一下采样视频图像帧和所述第二下采样视频图像帧进行运动估计，估算出全局运动矢量，以及采用所述全局运动矢量对所述第一原始视频图像帧和所述第二原始视频图像帧进行运动估计，得到所述待插入的畅显视频图像帧。
  5. 根据权利要求4所述的系统，其中，所述视频畅显切割传输模块，设置为按照所述屏幕拼接显示屏中的拼接子屏数量对插入所述一个或多个畅显视频图像帧 的视频数据流进行屏幕切割处理，并将所述切割后的视频数据流发送至屏幕拼接子系统。
  6. 根据权利要求1所述的系统，其中，所述屏幕拼接系统还包括：主控子系统；
    所述主控子系统，设置为在确定已经开启所述畅显图像处理模式的情况下，从所述屏幕拼接子系统获取显示图层参数集合，并将所述显示图层参数集合发送至所述畅显子系统，以及从所述畅显子系统获取畅显图像处理参数集合，并将所述畅显图像处理参数集合发送至所述视频采集子系统。
  7. 根据权利要求1所述的系统，其中，所述视频采集子系统，设置为判断当前是否已经开启所述畅显图像处理模式，当确定已经开启所述畅显图像处理模式时，将所述原始视频数据流输出至所述畅显子系统，当确定未开启所述畅显图像处理模式时，将所述原始视频数据流进行屏幕切割后输出至所述屏幕拼接子系统。
  8. 根据权利要求1所述的系统，其中，所述视频采集子系统，还设置为经由视频口VP采集所述原始视频数据流，和/或，经由网络获取视频压缩码流，并对所述视频压缩码流进行解码获取所述原始视频数据流。
  9. 一种视频数据流的处理方法，所述方法应用于权利要求1至8中任一项所述的屏幕拼接系统中，所述方法包括：
    接收来自于所述视频采集子系统的所述原始视频数据流，依次对所述原始视频数据流中的每相邻两个原始视频图像帧进行畅显图像处理，生成一个或多个畅显视频图像帧；
    将每个畅显视频图像帧插入至生成该畅显视频图像帧的相邻两个原始视频图像帧中，并对插入所述一个或多个畅显视频图像帧的视频数据流进行屏幕切割处理，输出至所述屏幕拼接子系统。
  10. 根据权利要求9所述的方法，其中，依次对所述原始视频数据流中的每相邻两个原始视频图像帧进行畅显图像处理，生成一个或多个畅显视频图像帧包括：
    分别对相邻的第一原始视频图像帧和第二原始视频图像帧进行下采样图像处理，得到第一下采样视频图像帧和第二下采样视频图像帧；
    分别对所述第一下采样视频图像帧和所述第二下采样视频图像帧进行运动估计，估算出全局运动矢量；
    采用所述全局运动矢量对所述第一原始视频图像帧和所述第二原始视频图像帧进行运动估计，得到所述待插入的畅显视频图像帧。
PCT/CN2014/089348 2013-11-18 2014-10-23 屏幕拼接系统和视频数据流的处理方法 WO2015070694A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US15/037,280 US9645784B2 (en) 2013-11-18 2014-10-23 Screen splicing system and video data stream processing method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201310578376.6A CN104660916B (zh) 2013-11-18 2013-11-18 屏幕拼接系统和视频数据流的处理方法
CN201310578376.6 2013-11-18

Publications (1)

Publication Number Publication Date
WO2015070694A1 true WO2015070694A1 (zh) 2015-05-21

Family

ID=53056747

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2014/089348 WO2015070694A1 (zh) 2013-11-18 2014-10-23 屏幕拼接系统和视频数据流的处理方法

Country Status (3)

Country Link
US (1) US9645784B2 (zh)
CN (1) CN104660916B (zh)
WO (1) WO2015070694A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105872410A (zh) * 2016-05-25 2016-08-17 广东威创视讯科技股份有限公司 一种拼接墙信号控制方法、装置及系统
CN110677600A (zh) * 2019-10-11 2020-01-10 福建星网视易信息系统有限公司 超宽画面的多组显示方法和系统、点播设备和点播系统
CN112639935A (zh) * 2019-07-31 2021-04-09 西安诺瓦星云科技股份有限公司 Led显示屏、led显示系统和显示箱体
CN114996518A (zh) * 2022-08-04 2022-09-02 深圳市稻兴实业有限公司 一种基于云平台的超高清视频数据存储分类管理系统

Families Citing this family (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6104680B2 (ja) * 2013-03-21 2017-03-29 株式会社日立国際電気 画像処理装置、撮像装置、監視システム、符号化装置、画像処理方法
CN105554450B (zh) * 2015-12-14 2018-10-12 深圳大学 分布式视频全景显示系统
CN107205128A (zh) * 2017-05-11 2017-09-26 西安万像电子科技有限公司 文件的显示方法、装置和系统
WO2019084801A1 (zh) * 2017-10-31 2019-05-09 深圳市大疆创新科技有限公司 运动估计方法和装置
WO2019106409A1 (en) * 2017-11-29 2019-06-06 Chan Ka Chun Systems and methods for processing and transmitting image signals over mobile phone telephony networks
CN109275024A (zh) * 2018-09-14 2019-01-25 洪海光电集团有限公司 通过音频信号控制多媒体设备的装置及其方法
CN109491626A (zh) * 2018-09-29 2019-03-19 上海摩奇贝斯展示设计营造有限公司 侧屏拼接演绎系统
CN109240634B (zh) * 2018-11-20 2021-11-26 宁波Gqy视讯股份有限公司 分布式拼接显示系统的输出节点及分布式拼接显示系统
CN109640168B (zh) * 2018-11-27 2020-07-24 Oppo广东移动通信有限公司 视频处理方法、装置、电子设备和计算机可读介质
CN109996013B (zh) * 2019-05-31 2019-08-16 南京巨鲨显示科技有限公司 一种低延时视频拼接方法、装置和系统
CN110769168A (zh) * 2019-09-27 2020-02-07 广州市水一方信息科技有限公司 一种侧屏拼接演绎系统
CN110855908B (zh) * 2019-11-18 2022-09-27 中移（杭州）信息技术有限公司 多方视频混屏方法、装置、网络设备及存储介质
CN111541905B (zh) * 2020-04-17 2022-05-20 广州虎牙科技有限公司 一种直播方法、装置、计算机设备和存储介质
CN113554659B (zh) * 2020-04-23 2023-06-02 杭州海康威视数字技术股份有限公司 图像处理方法、装置、电子设备、存储介质及显示系统
CN112118475A (zh) * 2020-08-13 2020-12-22 视联动力信息技术股份有限公司 一种跨屏窗口显示方法、装置、终端设备和存储介质
CN113286196B (zh) * 2021-05-14 2023-02-17 亿咖通(湖北)技术有限公司 一种车载视频播放系统及视频分屏显示方法及装置
CN113920951A (zh) * 2021-09-15 2022-01-11 上海祺壹智能科技有限公司 一种小电流刷新屏幕的方法
CN114339070B (zh) * 2021-12-28 2024-04-16 苏州长风航空电子有限公司 一种基于MPSoC的多路图像处理显示交互系统
CN114615548B (zh) * 2022-03-29 2023-12-26 湖南国科微电子股份有限公司 一种视频数据的处理方法、装置及计算机设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101778283A (zh) * 2010-01-14 2010-07-14 宁波Gqy视讯股份有限公司 一种在控制端实现实时映射拼接显示屏上视频信号的方法
CN101807389A (zh) * 2010-03-19 2010-08-18 上海博康智能网络科技有限公司 大屏拼接方法及系统
CN102104788A (zh) * 2009-12-17 2011-06-22 美国博通公司 一种用于视频处理的方法和系统
CN102611869A (zh) * 2012-02-10 2012-07-25 江苏清投视讯科技有限公司 一种面向输出的多屏拼接系统网络传输技术

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2183898B1 (en) * 2007-07-05 2016-09-14 Coherent Logix Incorporated Mobile television broadcast system
US20090202227A1 (en) * 2008-02-13 2009-08-13 Samsung Electronics Co., Ltd. Method and apparatus for recording data, method and apparatus for reproducing data, and recording medium for recording data
CN101281650B (zh) * 2008-05-05 2010-05-12 北京航空航天大学 用于视频稳定的快速全局运动估计方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102104788A (zh) * 2009-12-17 2011-06-22 美国博通公司 一种用于视频处理的方法和系统
CN101778283A (zh) * 2010-01-14 2010-07-14 宁波Gqy视讯股份有限公司 一种在控制端实现实时映射拼接显示屏上视频信号的方法
CN101807389A (zh) * 2010-03-19 2010-08-18 上海博康智能网络科技有限公司 大屏拼接方法及系统
CN102611869A (zh) * 2012-02-10 2012-07-25 江苏清投视讯科技有限公司 一种面向输出的多屏拼接系统网络传输技术

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105872410A (zh) * 2016-05-25 2016-08-17 广东威创视讯科技股份有限公司 一种拼接墙信号控制方法、装置及系统
CN105872410B (zh) * 2016-05-25 2019-01-18 广东威创视讯科技股份有限公司 一种拼接墙信号控制方法、装置及系统
CN112639935A (zh) * 2019-07-31 2021-04-09 西安诺瓦星云科技股份有限公司 Led显示屏、led显示系统和显示箱体
CN110677600A (zh) * 2019-10-11 2020-01-10 福建星网视易信息系统有限公司 超宽画面的多组显示方法和系统、点播设备和点播系统
CN110677600B (zh) * 2019-10-11 2021-10-01 福建星网视易信息系统有限公司 超宽画面的多组显示方法和系统、点播设备和点播系统
CN114996518A (zh) * 2022-08-04 2022-09-02 深圳市稻兴实业有限公司 一种基于云平台的超高清视频数据存储分类管理系统
CN114996518B (zh) * 2022-08-04 2022-10-21 深圳市稻兴实业有限公司 一种基于云平台的超高清视频数据存储分类管理系统

Also Published As

Publication number Publication date
CN104660916A (zh) 2015-05-27
CN104660916B (zh) 2018-09-18
US9645784B2 (en) 2017-05-09
US20160291917A1 (en) 2016-10-06

Similar Documents

Publication Publication Date Title
WO2015070694A1 (zh) 屏幕拼接系统和视频数据流的处理方法
US9270941B1 (en) Smart video conferencing system
KR20220130197A (ko) 촬영 방법, 장치, 전자설비 및 저장매체
KR20150006771A (ko) 비디오의 선택된 부분들을 고해상도로 렌더링하는 방법 및 장치
EP2695049A1 (en) Adaptive presentation of content
CN104822077B (zh) 客户端的运行方法及客户端
US20140229542A1 (en) System, Apparatus, and Method for Sharing a Screen Having Multiple Visual Components
US10601889B1 (en) Broadcasting panoramic videos from one server to multiple endpoints
WO2023169297A1 (zh) 动画特效生成方法、装置、设备及介质
WO2019076356A1 (zh) 基于虚拟现实场景的视频处理方法、服务器、虚拟现实设备和系统
CN108156511B (zh) 一种实现双屏或多屏拼接显示的方法和系统
CN111314624A (zh) 预览画面生成方法、装置、硬盘录像机及存储介质
US20140002645A1 (en) Server and video surveillance method of target place
US9996311B2 (en) Efficient communication interface for casting interactively controlled visual content
JP2023519372A (ja) 三次元ビデオの処理方法、装置、可読記憶媒体及び電子機器
KR20190018914A (ko) 서버, 디스플레이장치 및 그 제어방법
JP7471510B2 (ja) ピクチャのビデオへの変換の方法、装置、機器および記憶媒体
CN110727485A (zh) 一种屏幕界面可选择性共享方法及系统
CN107580228B (zh) 一种监控视频处理方法、装置及设备
US10887653B2 (en) Systems and methods for performing distributed playback of 360-degree video in a plurality of viewing windows
CN114374853A (zh) 内容展示方法、装置、计算机设备和存储介质
WO2023088104A1 (zh) 视频的处理方法、装置、电子设备和存储介质
CN105338281A (zh) 一种视频显示方法和装置
CN111045770A (zh) 用于远程展示的方法、第一终端、设备、可读存储介质
US20110285821A1 (en) Information processing apparatus and video content playback method

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 15037280

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14862977

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 14862977

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 21.02.2017)

122 Ep: pct application non-entry in european phase

Ref document number: 14862977

Country of ref document: EP

Kind code of ref document: A1