WO2020135407A1 - 多串流影像处理装置及方法 - Google Patents

多串流影像处理装置及方法 Download PDF

Info

Publication number
WO2020135407A1
WO2020135407A1 PCT/CN2019/127870 CN2019127870W WO2020135407A1 WO 2020135407 A1 WO2020135407 A1 WO 2020135407A1 CN 2019127870 W CN2019127870 W CN 2019127870W WO 2020135407 A1 WO2020135407 A1 WO 2020135407A1
Authority
WO
WIPO (PCT)
Prior art keywords
circuit
image
stream
sub
main image
Prior art date
Application number
PCT/CN2019/127870
Other languages
English (en)
French (fr)
Inventor
孙明勇
汪浩
Original Assignee
厦门星宸科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 厦门星宸科技有限公司 filed Critical 厦门星宸科技有限公司
Publication of WO2020135407A1 publication Critical patent/WO2020135407A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/40—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/60—Memory management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
    • H04N21/23106—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion involving caching operations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H04N21/2365—Multiplexing of several video streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/242—Synchronization processes, e.g. processing of PCR [Program Clock References]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4331—Caching operations, e.g. of an advertisement for later insertion during playback
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
    • H04N21/4347—Demultiplexing of several video streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/04—Synchronising

Definitions

  • the invention relates to multi-stream image processing technology, in particular to a multi-stream image processing device and method.
  • IP surveillance camera IP camera
  • IP camera IP surveillance camera
  • the resolution of some video streams is high, and a high-quality display screen can be displayed on the corresponding monitor.
  • Some video streams have lower resolutions and can be displayed on lower resolution monitors to provide, for example, but not limited to, real-time monitoring.
  • an object of the present invention is to provide a multi-stream image processing device and method to improve the existing technology.
  • Another object of the present invention is to provide a multi-stream image processing device and method, which can process multiple image streams in a single post-stage circuit in a time-sharing manner, avoiding the high installation cost of multiple post-stage circuits .
  • the invention includes a multi-stream image processing method, including: a pre-stage circuit generates a plurality of image streams based on an identical image source, wherein the image stream includes a main image stream and at least one sub-image stream, And the resolution of the main video stream is higher than the resolution of the sub-video stream; in an image frame processing time interval, the Nth sub-image frame of the sub-image stream is temporarily stored in a memory by a pre-stage circuit At least one current sub-image temporary storage block in the module, and a main image temporary storage block temporarily storing the N-th main image frame of the main image stream in the memory module; during the image frame processing time interval In the first sub-interval, a subsequent circuit reads the N-1th sub-image frame of the sub-image stream temporarily stored in the at least one existing sub-image temporary storage block of the memory module for processing; In the second sub-interval after the first sub-interval in the video image processing time interval, the Nth main video frame temporarily stored in the main image temporary storage block is read by the subsequent circuit for
  • the invention also includes a multi-stream image processing device, including: a memory module, a front-end circuit, a back-end circuit, and a processing circuit.
  • the processing circuit is electrically coupled to the memory module, the pre-stage circuit and the post-stage circuit, and is configured to execute software and firmware executable instructions to execute a multi-stream image processing method.
  • the steps include: generating a plurality of video streams from the same video source by the pre-stage circuit, wherein the video stream includes a main video stream and at least one sub video stream, and the resolution of the main video stream is higher than that of the sub video stream Resolution of the image stream; in an image frame processing time interval, the Nth sub-image frame of the sub-image stream is temporarily stored by the pre-stage circuit in at least one current sub-image temporary storage block in the memory module, And temporarily storing the Nth main image frame of the main image stream in the main image temporary storage block in the memory module; in a first sub-interval of the image frame processing time interval, read by the post-stage circuit The N-1th sub-image frame of the sub-image stream temporarily stored in the at least one existing sub-image temporary storage block in the memory module for processing; after the first sub-interval in the image frame processing time interval In a second sub-interval, the post-stage circuit reads the Nth main image frame temporarily stored in the main image temporary storage block for processing, where N is a positive integer greater
  • FIG. 1 shows a block diagram of a multi-stream image processing device according to an embodiment of the invention
  • FIG. 2 shows a flowchart of a multi-stream image processing method in an embodiment of the invention
  • FIG. 3 shows a sequence diagram of image processing performed by a multi-stream image processing device in an embodiment of the invention
  • FIG. 4 shows a block diagram of a synchronization circuit in an embodiment of the invention
  • FIG. 5 shows a flowchart of a synchronization process performed when a synchronization circuit operates in an embodiment of the invention
  • FIG. 6 shows a schematic diagram of the main image screen in an embodiment of the invention
  • FIG. 7 shows a block diagram of a synchronization circuit in another embodiment of the invention.
  • FIG. 8 shows a flowchart of a synchronization process performed when a synchronization circuit operates in another embodiment of the present invention
  • FIG. 9 shows a timing diagram of image processing performed by a multi-stream image processing device in another embodiment of the present invention.
  • FIG. 10 shows a block diagram of a synchronization circuit in still another embodiment of the present invention.
  • FIG. 11 shows a flowchart of a synchronization process performed by a synchronization circuit in another embodiment of the present invention.
  • An object of the present invention is to provide a multi-stream image processing device and method that can process multiple image streams in a single post-stage circuit in a time-sharing manner, avoiding the high installation cost of multiple post-stage circuits.
  • FIG. 1 is a block diagram of a multi-stream image processing device 100 according to an embodiment of the invention.
  • the multi-stream image processing device 100 can take images of the environment to generate multiple image streams MS (main video stream), SS1 (first sub-video stream), SS2 (second Sub-video stream), and subsequent processing on the video streams MS, SS1, SS2 to generate a processed main video stream EMS, a processed first sub-video stream ESS1, and a processed second sub-video stream ESS2.
  • a multi-stream image processing device 100 includes a memory module 110, a pre-stage circuit 120, a post-stage circuit 130, a processing circuit 140, and a synchronization circuit 150.
  • the memory module 110 has different blocks to store different data required for multi-stream image processing.
  • the memory module 110 may be implemented by a faster memory, such as, but not limited to, double data rate synchronous dynamic random access memory (double data synchronous dynamic random access memory; DDR SDRAM).
  • a multi-stream image processing device 100 may further include a memory controller 115.
  • the memory controller 115 may be implemented by, for example, but not limited to, a memory interface unit (MIU).
  • MIU memory interface unit
  • Other circuit modules in the multi-stream image processing device 100 such as the pre-stage circuit 120 and the post-stage circuit 130, can read and write the memory module 110 through the memory controller 115 to store data in the memory module 110 Or read data from the memory module 110.
  • the pre-stage circuit 120 is an image signal processor (ISP) configured to generate a plurality of image streams MS, SS1, SS2 according to an identical image source IS.
  • the image source IS may be, for example, but not limited to a photosensitive element (not shown) of a network surveillance camera. After the photosensitive element senses the image, the front-end circuit 120 generates image streams MS, SS1, and SS2 with different resolutions, and stores it in the memory module 110 through the memory controller 115.
  • the video stream MS is the main video stream, and the video streams SS1 and SS2 are the first and second sub-video streams.
  • the resolution of the main video stream MS is higher than the resolutions of the sub video streams SS1 and SS2.
  • the resolution of the main video stream MS is 4K
  • the resolution of the first sub video stream SS1 is 1280 ⁇ 720
  • the resolution of the second sub video stream SS2 is 720 ⁇ 576. It should be noted that the above-mentioned number of sub-image streams and the resolution value of each image stream are only examples, and the present invention is not limited thereto. In one embodiment, the number of sub-image streams may be one or more.
  • the post-stage circuit 130 is an encoder configured to read the video streams MS, SS1, and SS2 from the memory module 110 through the memory controller 115 to perform the encoding process and generate the main video string after the encoding process Stream EMS, processed first sub video stream ESS1, processed second sub video stream ESS2.
  • the encoding process may be performed according to, for example, but not limited to, H264, H265, or other encoding standards.
  • the synchronization circuit 150 is electrically coupled to the pre-stage circuit 120, the post-stage circuit 130 and the processing circuit 140, and is configured to enable the pre-stage circuit 120 to temporarily store the image stream in the memory module 110 according to the control of the processing circuit 140 and the storage speed and The read speed of the post-stage circuit 130 streaming video from the memory module 110 can be balanced.
  • the processing circuit 140 is electrically coupled to the memory module 110, the pre-stage circuit 120, the post-stage circuit 130, and the synchronization circuit 150.
  • the processing circuit 140 can execute a software executable instruction 141 to execute the functions of the multi-stream image processing device 100.
  • the processing circuit 140 can retrieve a software executable executable instruction 141 from a storage module (not shown) included in a multi-stream image processing device 100, and the software executable executable instruction 141 includes, for example, but not Limited to the firmware of the pre-stage circuit 120, the post-stage circuit 130, and the synchronization circuit 150 and other related instructions for operating and controlling the pre-stage circuit 120, the post-stage circuit 130, and the synchronization circuit 150, to further operate and control the pre-stage circuit 120,
  • the post-stage circuit 130 and the synchronizing circuit 150 achieve the aforementioned temporary storage, reading and encoding of the video streaming video streams MS, SS1 and SS2, and generate the processed main video stream EMS and the processed first sub-video stream ESS1, the effect of the second sub-image stream ESS2 after processing.
  • the storage module is implemented by another memory independent of the memory module 110.
  • the storage module may be, for example, but not limited to, an optical disk, random access memory (RAM), read only memory (ROM), floppy disk, hard disk, or optical disk.
  • the storage module and the memory module 110 can also be implemented by the same memory.
  • FIG. 2 is a flowchart of a multi-stream image processing method 200 according to an embodiment of the invention.
  • the multi-stream image processing method 200 can be applied to the multi-stream image processing apparatus 100 shown in FIG. 1.
  • An embodiment of the multi-stream image processing method 200 shown in FIG. 2 includes the following steps S210 to S240.
  • a pre-stage circuit 120 generates a plurality of image streams MS, SS1, SS2 according to an identical image source IS.
  • the plurality of video streams include a main video stream MS and first and second sub-video streams SS1, SS2, and the resolution of the main video stream MS is higher than that of the sub-video streams SS1. SS2 resolution.
  • step S220 during an image frame processing time interval, the N-th sub-image frame of the sub-image streams SS1 and SS2 is temporarily stored in the memory module 110 by the front-end circuit 120 temporarily
  • the storage blocks 114A and 116A and the N-th main video frame of the main video stream MS are temporarily stored in the main video temporary storage block 112 of the memory module 110.
  • N is a positive integer greater than or equal to 2.
  • FIG. 3 is a sequence diagram of the multi-stream image processing device 100 performing image processing in an embodiment of the present invention.
  • the multi-stream image processing device 100 includes a plurality of video image processing time intervals in the processing timing, for example, three consecutive image image processing time intervals TN-1, TN, and TN+1 shown in the example shown in FIG. 3.
  • a synchronization signal time interval is included between each two adjacent image frame processing time zones, such as the synchronization signal time zone TSN between the image frame processing time zone TN-1 and TN and the image frame processing time zones TN and TN+1 The time interval between synchronization signals TSN+1.
  • the synchronization signal time interval TSN and the synchronization signal time interval TSN+1 correspond to the transmission of the synchronization signal SN and the synchronization signal SN+1, respectively, to mark the beginning of the image frame processing time intervals TN and TN+1, respectively.
  • the pre-stage circuit 120 and the post-stage circuit 130 do not perform data processing, but only in the video screen processing time interval TN-1, TN And TN+1 for data processing.
  • the processing timing of the pre-stage circuit 120 is indicated by slanted blocks. Since the pre-stage circuit 120 can generate the video streams MS, SS1 and SS2 at the same time, after the start of the video frame processing time interval TN, the pre-stage circuit 120 starts to synchronize the Nth sub-image of the temporary sub-image streams SS1 and SS2 The current first and second sub-image temporary storage blocks 114A and 116A, and the N-th main video frame of the main video stream MS are temporarily stored in the main video temporary storage block 112.
  • step S230 in the first sub-interval TS1 of the image frame processing time interval TN, the post-stage circuit 130 preferentially reads the existing first and second sub-image temporary storage blocks 114B and 116B in the memory module 110.
  • the N-1 sub-picture frames of the stored sub-picture streams SS1 and SS2 are processed.
  • the N-1th sub-image frame of the sub-image streams SS1 and SS2 has been passed by the pre-stage circuit 120 through the memory controller 115 , Are temporarily stored in the existing first and second sub-image temporary storage blocks 114B and 116B, respectively.
  • the processing sequence of the post-stage circuit 130 is represented by dotted blocks.
  • the post-stage circuit 130 operates in a time-sharing manner and processes one screen image at a time. Therefore, as shown in FIG. 3, in the first sub-interval TS1, the subsequent circuit 130 will preferentially read the existing first sub-image temporary storage block 114B through the memory controller 115, corresponding to the first sub-image stream SS1 And process the N-1th sub-image frame of the first image to generate the corresponding image frame in the processed first sub-image stream ESS1.
  • the post-stage circuit 130 reads the existing second sub-image temporary storage block 116B through the memory controller 115, corresponding to the Nth of the second sub-image stream SS2 -1 sub-image frames and process them to generate the corresponding image frames in the processed second sub-image stream ESS2.
  • step S240 in the second sub-interval TS2 after the first sub-interval TS1 in the video frame processing time interval TN, the N-th main video frame temporarily stored in the main video temporary storage block 112 is read by the post-stage circuit 130 For processing.
  • the capacity of the main image temporary storage block 112 is greater than or equal to the size of the Nth main image frame.
  • the post-stage circuit 130 may operate faster than the pre-stage circuit 120. Therefore, in order to avoid that the read speed of the data stored in the self-image temporary storage block 112 of the later stage circuit 130 exceeds the storage speed of the temporarily stored data of the previous circuit 120 to the main image temporary storage block 112, the synchronization circuit 150 can perform the previous stage circuit Coordination and synchronization between 120 and the post-stage circuit 130.
  • FIG. 4 is a block diagram of the synchronization circuit 150 in an embodiment of the present invention.
  • the synchronization circuit 150 includes a first comparison module 400, a second comparison module 402 and a synchronization processing module 404.
  • the first comparison module 400 and the second comparison module 402 can retrieve the content of the frame processing information FP1 and FP2 from the pre-stage circuit 120 and the post-stage circuit 130.
  • the screen processing information FP1 includes the number of front-stage frames F1 of the video screen being temporarily stored by the front-end circuit 120, and the number of storage lines RW of the video screen being temporarily stored
  • the screen processing information FP2 includes the one being read by the rear-stage circuit 130 The number of subsequent frames F2 of the video screen and the number of read lines RR of the video screen being read.
  • the first comparison module 400 compares the number of first-stage frames F1 of the video frames that the front-stage circuit 120 is temporarily storing with the number of second-stage frames F2 of the video frames being read by the second-stage circuit 130 to determine Whether the video frame being temporarily stored by the pre-stage circuit 120 and the video frame being read by the post-stage circuit 130 are the same.
  • the second comparison module 402 compares the number of storage lines RW of the video frame being temporarily stored by the previous stage circuit 120 and the number of read lines RR of the image frame being read by the rear stage circuit 130 to determine that the rear stage circuit 130 is reading Whether the number of read lines RR of the fetched video screen exceeds the storage line number RW of the video screen being temporarily stored by the pre-stage circuit 120.
  • the synchronization processing module 404 further determines the operation status of the pre-stage circuit 120 and the post-stage circuit 130 according to the comparison result of the first comparison module 400 and the second comparison module 402, and further determines the synchronization mechanism to be adopted.
  • FIG. 5 is a flowchart of a synchronization process 500 performed by the synchronization circuit 150 in an embodiment of the present invention. The operation mechanism of the synchronization circuit 150 will be described in more detail below based on FIGS. 4 and 5.
  • step S510 the first comparison module 400 retrieves the previous-stage frame number F1 and the second-stage frame number F2 in the frame processing information FP1 and FP2 from the pre-stage circuit 120 and the post-stage circuit 130 for comparison.
  • step S520 according to the first comparison result CR1 of the first comparison module 400, the synchronization processing module 404 further determines whether the image frame being temporarily stored by the front-end circuit 120 and the image frame being read by the rear-end circuit 130 are the same.
  • the post-stage circuit 130 reads the N-1 th sub-video frame and the pre-stage circuit 120 stores the N-th sub-picture For a main video frame, in step S530, the synchronization processing module 404 does not activate the synchronization mechanism of the synchronization circuit 150.
  • the N-th main video frame stored in the pre-stage circuit 120 and read by the post-stage circuit 130 are synchronized
  • the processing module 404 starts the operation of the synchronization mechanism, and in step S540, the second comparison module 402 retrieves the number of stored lines RW and the number of read lines RR in the frame processing information FP1 and FP2 from the front-end circuit 120 and the back-end circuit 130. Compare.
  • FIG. 6 is a schematic diagram of a main video screen 600 in an embodiment of the invention.
  • the synchronization processing module 404 determines whether the number of storage lines RW of the main image frame 600 being temporarily stored by the front-end circuit 120 exceeds the main image read by the back-end circuit 130 according to the second comparison result CR2 of the second comparison module 402 The number of read lines RR of the screen 600.
  • step S560 the synchronization processing module 404 allows the post-stage circuit 130 to continue reading the N-th main video frame for processing by the post-stage circuit control signal CC1. Conversely, when the number of stored lines RW does not exceed the number of read lines RR, it means that the content read by the subsequent circuit 130 will exceed the content stored by the previous circuit 120. Therefore, in step S570, the synchronization processing module 404 causes the post-stage circuit 130 to stop reading the N-th main video frame through the post-stage circuit control signal CC1, so as to avoid reading the wrong data content.
  • the post-stage circuit 130 sequentially reads the content of the N-th main video frame and performs processing to generate a main video frame corresponding to the processed main video stream EMS. It should be noted that, in this embodiment, the post-stage circuit 130 completes the processing of the Nth main video frame before the next synchronization signal time interval TSN+1.
  • the pre-stage circuit 120 may temporarily store the N+1th sub-picture frame of the sub-video streams SS1, SS2 and the main video stream MS in the above-mentioned manner. N+1th main video screen.
  • the post-stage circuit 130 may first read the Nth sub-image frames of the sub-image streams SS1 and SS2 for processing, and then read the N+1 main-image screen of the main video stream MS for processing.
  • the pre-stage circuit 120 temporarily stores the current first and second sub-images of the N-th sub-image frame of the sub-image streams SS1 and SS2 in the image frame processing time interval TN
  • the temporary storage blocks 114A and 116A become the existing sub-image temporary storage blocks for the image frame processing time interval TN+1.
  • the existing first and second sub-image temporary storage blocks 114B and 116B in the image frame processing time interval TN are used as the current sub-image temporary storage block for the image frame processing time interval TN+1, and are used to The N+1th sub-picture frame of the sub-picture streams SS1 and SS2 is stored.
  • the capacity of the main image temporary storage block 112 is greater than or equal to the size of the Nth main image frame.
  • the capacity of the main image temporary storage block 112 may also be smaller than the size of the Nth main image frame, and it operates as a ring buffer. That is, when the primary circuit 120 temporarily stores the main image frame to the highest address of the main image temporary storage block 112, the updated image data will be stored instead of the lowest address of the main image temporary storage block 112.
  • the post-stage circuit 130 reads the main image frame to the highest address of the main image temporary storage block 112, it will also return to the lowest address of the main image temporary storage block 112 to continue reading.
  • FIG. 7 is a block diagram of the synchronization circuit 150 in another embodiment of the present invention.
  • FIG. 8 is a flowchart of a synchronization process 800 performed by the synchronization circuit 150 in another embodiment of the present invention. 7 and 8, the implementation of the main image temporary storage block 112 operating in a circular buffer will be described in more detail.
  • the synchronization circuit 150 shown in FIG. 7 is similar to the synchronization circuit 150 shown in FIG. 4 and includes a first comparison module 400, a second comparison module 402, and a synchronization processing module 404.
  • the synchronization circuit 150 of FIG. 7 further includes a first calculation module 406.
  • the first calculation module 406 is configured to calculate the difference DR between the number of stored lines RW and the number of read lines RR based on the picture processing information FP1 and FP2.
  • the synchronization mechanism performed by the synchronization circuit 150 shown in FIG. 7 can execute the process of FIG. 8 at the same time except steps S510 to S570 in FIG. 5 to perform step S810, and the first calculation module 406 calculates the number of stored lines The distance DR between RW and the number of read lines RR.
  • step S820 the synchronization processing module 404 determines whether the gap DR is greater than the capacity of the main image temporary storage block 112.
  • the synchronization processing module 404 allows the front-end circuit 120 to continue to temporarily store the N-th main video frame through the front-end circuit control signal CC2.
  • the gap is larger than the capacity of the main image temporary storage block 112, it means that the storage speed of the pre-stage circuit 120 exceeds the reading speed of the post-stage circuit 130.
  • the synchronization processing module 404 will cause the front-end circuit 120 to stop temporarily storing the Nth main video frame through the front-end circuit control signal CC2, and further suspend the storage of the main video frame data.
  • FIG. 9 is a timing diagram of the multi-stream image processing device 100 performing image processing in another embodiment of the present invention.
  • the multi-stream image processing device 100 includes a plurality of video image processing time intervals in the processing timing, for example, three consecutive image image processing time intervals TN-1, TN, and TN+1 illustrated in FIG. 5 by way of example.
  • a synchronization signal time interval is included between each two adjacent video screen processing time zones, for example, a synchronization signal time interval TSN between video screen processing time intervals TN-1 and TN and a video screen processing time interval between TN and TN+1 The synchronization signal time interval TSN+1.
  • the synchronization signal time interval TSN and the synchronization signal time interval TSN+1 correspond to the transmission of the synchronization signal SN and the synchronization signal SN+1, respectively, to mark the beginning of the image frame processing time intervals TN and TN+1, respectively.
  • the N-1th sub-picture frame of the sub-picture streams SS1 and SS2 is still read and processed by the post-stage circuit 130, and the In the second sub-interval TS2 of the video screen processing time interval TN, the N-th main video screen is still read by the post-stage circuit 130 for processing.
  • the video frame processing time interval TN further includes a third sub-interval TS3 before the first sub-interval TS1.
  • the post-stage circuit 130 is allowed to process the synchronization signal in addition to processing in the second half of the video frame processing time interval TN-1. Processing is performed in the time interval TSN and the third sub-interval TS3.
  • the post-stage circuit 130 is also allowed to process the second in the second half of the video picture processing time interval TN, the synchronization signal time interval TSN+1, and the third sub-interval TS3 of the video picture processing time interval TN+1. N main video frames.
  • the Nth sub-picture frame of the sub-picture streams SS1 and SS2 and the N-th picture of the main video stream MS are still temporarily stored at the beginning of the video picture processing time interval TN Main image screen. Therefore, the main image temporary storage block 112 will simultaneously read the N-1th main image frame and temporarily store the Nth main image frame in the third sub-section TS3. Increasing this Ts3 also has the advantage that the processing speed of the subsequent circuit can be slower, that is, it can save power.
  • FIG. 10 is a block diagram of a synchronization circuit 150 in still another embodiment of the present invention.
  • FIG. 11 is a flowchart of a synchronization process 1100 performed by the synchronization circuit 150 in another embodiment of the present invention. The following will describe the implementation of the main image temporary storage block 112 operating in a ring buffer in more detail based on FIGS. 10 and 11.
  • the synchronization circuit 150 shown in FIG. 10 is similar to the synchronization circuit 150 shown in FIG. 7 and includes a first comparison module 400, a second comparison module 402, a synchronization processing module 404, and a first calculation module 406.
  • the synchronization circuit 150 of FIG. 10 further includes a second calculation module 408 configured to operate at different times when the image frame being stored by the pre-stage circuit 120 and the image frame being read by the post-stage circuit 130 are not simultaneously.
  • the synchronization mechanism performed by the synchronization circuit 150 shown in FIG. 10 has the same steps S510, S520, and S540 to S570 as in FIG. 5, and the same steps S810 to S840 as in FIG.
  • the synchronization circuit 150 shown in FIG. 10 will execute the process of FIG. 11 to start the operation of the synchronization mechanism.
  • step S1110 the second calculation module 408 first calculates the difference between the size FR of the N-1 main video frame and the number of read lines RR of the N-1 main video frame based on the screen processing information FP1 and FP2.
  • This gap represents the amount of data remaining in the N-1th main video frame that has not yet been read and processed by the post-stage circuit 130.
  • the second calculation module 408 calculates the sum SUM of this gap and the storage line number RWN of the Nth main video frame.
  • the storage line number RW represents the amount of data temporarily stored in the pre-stage circuit 120 in the Nth main video frame.
  • step S1120 the synchronization processing module 404 determines whether the sum SUM is greater than the capacity of the main image temporary storage block 112.
  • step S1130 the synchronization processing module 404 allows the front-end circuit 120 to continue to temporarily store the Nth main video frame through the front-end circuit control signal CC2.
  • step S1140 the synchronization processing module 404 will stop the front-end circuit 120 from generating the N-th main video frame by the front-end circuit control signal CC2, and further suspend the storage of the main video frame data.
  • the multi-stream image processing device 100 of the present invention can process multiple image streams generated by the front-end circuit in a single post-stage circuit in a time-sharing manner, avoiding the high installation cost of multiple post-stage circuits .
  • the memory module only needs to temporarily store the N-1th sub-image frame and the N-th sub-image frame for the sub-image screen with a lower resolution and a smaller amount of data.
  • the main image screen with high resolution and large data volume there is no need to additionally set two temporary storage blocks for temporary storage.
  • the space cost of the memory module it will also drop significantly.
  • the memory module can realize the main image temporary storage block in the form of a ring buffer, and the capacity will be reduced accordingly, further reducing the space cost of the memory module.
  • the multi-stream image processing device and method in the present invention can achieve the effects of reducing the hardware installation cost of the post-stage circuit and the memory space cost.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Studio Devices (AREA)
  • Television Systems (AREA)
  • Image Processing (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

本发明公开一种多串流影像处理裝置及方法，方法包含：由一前级电路产生包含一主影像串流以及至少一子影像串流的多个影像串流，且该主影像串流的分辨率高于该子影像串流的分辨率；于一影像画面处理时间区间中，由前级电路暂存第N个子影像画面于一存储器模块中的当下子影像暂存区块，以及暂存第N个主影像画面于该存储器模块中的主影像暂存区块；于该影像画面处理时间区间的第一子区间中，由一后级电路读取该存储器模块中的该现有子影像暂存区块中暂存的第N-1个子影像画面，以进行处理；于该影像画面处理时间区间的第二子区间中，由该后级电路读取第N个主影像画面，以进行处理。

Description

多串流影像处理装置及方法
本申请要求享有2018年12月26日提交的名称为“多串流影像处理装置及方法”的中国专利申请CN201811602356.7的优先权，其全部内容通过引用并入本文中。
技术领域
本发明涉及多串流影像处理技术，尤其涉及一种多串流影像处理装置与方法。
背景技术
在部分影像技术的应用中，会根据同一个影像来源产生多个影像串流。举例而言，网络监控摄影机(IP camera)可在撷取环境的影像后，产生相应的多个影像串流。部分影像串流的分辨率较高，可在对应的显示器上显示高画质的显示画面。而部分影像串流的分辨率较低，可显示于较低分辨率的显示器上提供例如，但不限于即时的监看。
然而，多个影像串流往往需要多个相应的电路来进行处理，而造成电路成本设置的高昂。
发明内容
鉴于现有技术的问题，本发明的一目的在于提供一种多串流影像处理装置与方法，以改善现有技术。
本发明的另一目的在于提供一种多串流影像处理装置与方法，可使多个影像串流藉由分时的方式以单一后级电路进行处理，避免多个后级电路的高昂设置成本。
本发明包含一种多串流影像处理方法，包含：由一前级电路根据一相同影像来源产生多个影像串流，其中该影像串流包含一主影像串流以及至少一子影像串流，且该主影像串流的分辨率高于该子影像串流的分辨率；于一影像画面处理时间区间中，由一前级电路暂存该子影像串流的第N个子影像画面于一存储器模块中的至少一当下子影像暂存区块，以及暂存该主影像串流的第N个主影像画面于该存储器模块中的一主影像暂存区块；于该影像画面处理时间区间的第一子区间中，由一后级电路读取该存储器模块中的至少一现有子影像暂存区块中暂存的该子影像串流的第N-1个子影像画面，以进行处理；于影像 画面处理时间区间中第一子区间后的第二子区间中，由该后级电路读取该主影像暂存区块中暂存的第N个主影像画面，以进行处理，其中，N为大于或者等于2的正整数。
本发明另包含一种多串流影像处理装置，包含：一存储器模块、一前级电路、一后级电路以及一处理电路。该处理电路电性耦接于该存储器模块、该前级电路以及该后级电路，并配置以执行软固件可执行指令，以执行一多串流影像处理方法，该多串流影像处理方法的步骤包含：由该前级电路根据一相同影像来源产生多个影像串流，其中影像串流包含一主影像串流以及至少一子影像串流，且该主影像串流的分辨率高于子影像串流的分辨率；于一影像画面处理时间区间中，由该前级电路暂存该子影像串流的第N个子影像画面于该存储器模块中的至少一当下子影像暂存区块，以及暂存该主影像串流的第N个主影像画面于该存储器模块中的主影像暂存区块；于该影像画面处理时间区间的一第一子区间中，由该后级电路读取该存储器模块中的至少一现有子影像暂存区块中暂存的子影像串流的第N-1个子影像画面，以进行处理；于影像画面处理时间区间中该第一子区间后的一第二子区间中，由该后级电路读取该主影像暂存区块中暂存的第N个主影像画面，以进行处理，其中，N为大于或者等于2的正整数。
以下结合附图和具体实施例对本发明进行详细描述，但不作为对本发明的限定。
附图说明
图1显示本发明的一实施例中，多串流影像处理装置的方框图；
图2显示本发明的一实施例中，一种多串流影像处理方法的流程图；
图3显示本发明的一实施例中，多串流影像处理装置进行影像处理的时序图；
图4显示本发明一实施例中，同步电路的方框图；
图5显示本发明一实施例中，同步电路运作时进行的同步流程的流程图；
图6显示本发明一实施例中，主影像画面的示意图；
图7显示本发明另一实施例中，同步电路的方框图；
图8显示本发明另一实施例中，同步电路运作时进行的同步流程的流程图；
图9显示本发明另一实施例中，多串流影像处理装置进行影像处理的时序图；
图10显示本发明又一实施例中，同步电路的方框图；以及
图11显示本发明又一实施例中，同步电路运作时进行的同步流程的流程图。
其中，附图标记
100 多串流影像处理装置
110 存储器模块
112 主影像暂存区块
114A 当下第一子影像暂存区块
114B 现有第一子影像暂存区块
116A 当下第二子影像暂存区块
116B 现有第二子影像暂存区块
115 存储器控制器
120 前级电路
130 后级电路
140 处理电路
141 软固件可执行指令
150 同步电路
200 多串流影像处理方法
400 第一比较模块
402 第二比较模块
404 同步处理模块
406 第一计算模块
408 第二计算模块
500 同步流程
600 主影像画面
800 同步流程
1100 同步流程
CC1 后级电路控制信号
CC2 前级电路控制信号
CR1 第一比较结果
CR2 第二比较结果
DR 差距
EMS 处理后主影像串流
ESS1 处理后第一子影像串流
ESS2 处理后第二子影像串流
F1 前级画面数
F2 后级画面数
FP1、FP2 画面处理信息
IS 影像来源
MS 主影像串流
RR 读取行数
RW 储存行数
SN、SN+1 同步信号
SS1 第一子影像串流
SS2 第二子影像串流
SUM 总和
S210～S240 步骤
S510～S570 步骤
S810～S840 步骤
S1110～S1140 步骤
TN-1、TN、TN+1 影像画面处理时间区间
TS1 第一子区间
TS2 第二子区间
TS3 第三子区间
TSN、TSN+1 同步信号时间区间
具体实施方式
下面结合附图对本发明的结构原理和工作原理作具体的描述：
本发明的一目的在于提供一种多串流影像处理装置与方法，可使多个影像串流藉由分时的方式以单一后级电路进行处理，避免多个后级电路的高昂设置成本。
请参照图1。图1为本发明的一实施例中，多串流影像处理装置100的方框图。多串流影像处理装置100可对环境进行影像的拍摄，以根据相同的影像来源IS产生多个影像串流MS(主影像串流)、SS1(第一子影像串流)、SS2(第二子影像串流)，并对影像串流MS、SS1、SS2进行后续的处理，产生处理后主影像串流EMS、处理后第一子影像串流ESS1、处理后第二子影像串流ESS2。
一多串流影像处理装置100包含：一存储器模块110、一前级电路120、一后级电路130、一处理电路140以及一同步电路150。
于一实施例中，存储器模块110具有不同的区块，以储存多串流影像处理所需的不同数据。存储器模块110可由速度较快的存储器实现，例如，但不限于双倍数据率同步动态随机读取存储器(double data rate synchronous dynamic random access memory；DDR SDRAM)。
于一实施例中，一多串流影像处理装置100可更包含一存储器控制器115。存储器控制器115可由例如，但不限于存储器介面单元(memory interface unit；MIU)实现。在多串流影像处理装置100中的其他电路模块，如前级电路120以及后级电路130，可通过存储器控制器115对存储器模块110进行读取与写入，以将数据储存于存储器模块110，或是自存储器模块110读取数据。
前级电路120于一实施例中，为影像信号处理器(image signal processor；ISP)，配置以根据一相同影像来源IS产生多个影像串流MS、SS1、SS2。其中，影像来源IS可为例如，但不限于网络监控摄影机的感光元件(未绘示)。在感光元件感测影像后，前级电路120将据以产生不同分辨率的影像串流MS、SS1、SS2，并通过存储器控制器115储存于存储器模块110中。
其中，影像串流MS为主影像串流，影像串流SS1、SS2为第一及第二子影像串流。主影像串流MS的分辨率较子影像串流SS1、SS2的分辨率为高。于一实施例中，主影像串流MS的分辨率为4K，第一子影像串流SS1的分辨率为1280×720，第二子影像串流SS2的分辨率为720×576。需注意的是，上述子影像串流的数目以及各影像串流的分辨率数值仅为一范例，本发明并不限于此。于一实施例中，子影像串流的数目可为一个或多 个。
后级电路130于一实施例中，为编码器，配置以通过存储器控制器115自存储器模块110读取影像串流MS、SS1、SS2来进行编码程序，并产生完成编码的处理后主影像串流EMS、处理后第一子影像串流ESS1、处理后第二子影像串流ESS2。于不同实施例中，编码程序可依照例如，但不限于H264、H265或是其他的编码标准进行。
同步电路150电性耦接于前级电路120、后级电路130以及处理电路140，配置以根据处理电路140的控制，使前级电路120对影像串流暂存于存储器模块110的储存速度以及后级电路130将影像串流自存储器模块110读取的读取速度能够达到平衡。
处理电路140电性耦接于存储器模块110、前级电路120、后级电路130以及同步电路150。处理电路140可执行一软固件可执行指令141，以执行多串流影像处理装置100的功能。更详细来说，处理电路140可从一多串流影像处理装置100中包含的储存模块(未绘示)撷取一软固件可执行指令141，且该软固件可执行指令141包含例如但不限于前级电路120、后级电路130以及同步电路150的韧体以及其他用以操作与控制前级电路120、后级电路130以及同步电路150的相关指令，进一步操作与控制前级电路120、后级电路130以及同步电路150，达到前述对影像串流影像串流MS、SS1、SS2进行暂存、读取及编码，并产生处理后主影像串流EMS、处理后第一子影像串流ESS1、处理后第二子影像串流ESS2的功效。
需注意的是，储存模块于一实施例中，是以与存储器模块110相独立的另一存储器来实现。举例而言，储存模块可由例如，但不限于光碟、随机读取存储器(random access memory；RAM)、只读存储器(read only memory；ROM)、软碟、硬碟或光学磁碟片。于另一实施例中，储存模块亦可与存储器模块110藉由同一存储器实现。
请同时参照图2。以下将搭配图1及图2，对于多串流影像处理装置100的详细功能进行更详细的说明。
图2为本发明的一实施例中，一种多串流影像处理方法200的流程图。多串流影像处理方法200可应用于图1所示的多串流影像处理装置100中。多串流影像处理方法200的一实施例如图2所示，包含下列步骤S210至步骤S240。
于步骤S210：由一前级电路120根据一相同影像来源IS产生多个影像串流MS、SS1、SS2。如上所述，该多个影像串流包含一主影像串流MS以及第一及第二子影像串流SS1、SS2，且该主影像串流MS的分辨率高于该子影像串流SS1、SS2的分辨率。
于步骤S220：于一影像画面处理时间区间中，由该前级电路120暂存子影像串流SS1、 SS2的第N个子影像画面于该存储器模块110中的当下第一及第二子影像暂存区块114A及116A，以及暂存主影像串流MS的第N个主影像画面于该存储器模块110中的主影像暂存区块112。其中，N为大于或者等于2的正整数。
请同时参照图3。图3为本发明的一实施例中，多串流影像处理装置100进行影像处理的时序图。
多串流影像处理装置100在处理的时序上，包含多个影像画面处理时间区间，例如第3图范例性绘示的连续三个影像画面处理时间区间TN-1、TN及TN+1。各两个邻接的影像画面处理时间区之间包含一同步信号时间区间，例如影像画面处理时间区间TN-1及TN之间的同步信号时间区间TSN以及影像画面处理时间区间TN及TN+1之间的同步信号时间区间TSN+1。同步信号时间区间TSN以及同步信号时间区间TSN+1是分别对应同步信号SN以及同步信号SN+1的传送，以分别标示影像画面处理时间区间TN及TN+1的起始。在本实施例中，在同步信号时间区间TSN以及同步信号时间区间TSN+1中，前级电路120和后级电路130均不进行数据处理，而仅在影像画面处理时间区间TN-1、TN及TN+1中进行数据处理。
在图3中，前级电路120的处理时序是以斜线区块表示。由于前级电路120可以同时产生影像串流MS、SS1以及SS2，因此在影像画面处理时间区间TN起始后，前级电路120即开始同步暂存子影像串流SS1、SS2的第N个子影像画面于当下第一及第二子影像暂存区块114A及116A，以及暂存主影像串流MS的第N个主影像画面于主影像暂存区块112。
于步骤S230：于影像画面处理时间区间TN的第一子区间TS1中，由后级电路130优先读取存储器模块110中的现有第一及第二子影像暂存区块114B及116B中暂存的子影像串流SS1及SS2的第N-1个子影像画面，以进行处理。
于一实施例中，如图3所示，在影像画面处理时间区间TN-1中，子影像串流SS1及SS2的第N-1个子影像画面即已由前级电路120通过存储器控制器115，分别被暂存于现有第一及第二子影像暂存区块114B及116B中。
后级电路130的处理时序是以点状区块表示。后级电路130是以分时的方式运作，一次针对一个画面影像进行处理。因此如图3所示，后级电路130在第一子区间TS1中，将优先通过存储器控制器115读取现有第一子影像暂存区块114B中，对应于第一子影像串流SS1的第N-1个子影像画面并进行处理，以产生处理后第一子影像串流ESS1中对应的影像画面。接着，后级电路130在第一子区间TS1剩下的时间中，再通过存储器控制 器115读取现有第二子影像暂存区块116B，对应于第二子影像串流SS2的第N-1个子影像画面并进行处理，以产生处理后第二子影像串流ESS2中对应的影像画面。
于步骤S240：于影像画面处理时间区间TN中第一子区间TS1后的第二子区间TS2中，由后级电路130读取主影像暂存区块112中暂存的第N个主影像画面，以进行处理。
于一实施例中，主影像暂存区块112的容量大于或等于第N个主影像画面的大小。于部分实施例中，后级电路130的运作速度可能较前级电路120为快。因此为了避免后级电路130自主影像暂存区块112读取数据的读取速度超过前级电路120暂存数据至主影像暂存区块112的储存速度，同步电路150将可进行前级电路120与后级电路130的协调与同步。
请参照图4。图4为本发明的一实施例中，同步电路150的方框图。
如图4所示，同步电路150包含第一比较模块400、第二比较模块402及同步处理模块404。
第一比较模块400及第二比较模块402可自前级电路120以及后级电路130撷取其画面处理信息FP1及FP2的内容。其中，画面处理信息FP1包含前级电路120正在暂存的影像画面的前级画面数F1，以及正在暂存的影像画面的储存行数RW，画面处理信息FP2包含后级电路130正在读取的影像画面的后级画面数F2，以及正在读取的影像画面的读取行数RR。
于一实施例中，第一比较模块400根据前级电路120正在暂存的影像画面的前级画面数F1以及后级电路130正在读取的影像画面的后级画面数F2进行比较，以判断前级电路120正在暂存的影像画面以及后级电路130正在读取的影像画面是否为相同。第二比较模块402则根据前级电路120正在暂存的影像画面的储存行数RW以及后级电路130正在读取的影像画面的读取行数RR进行比较，以判断后级电路130正在读取的影像画面的读取行数RR是否超过前级电路120正在暂存的影像画面的储存行数RW。
同步处理模块404进一步根据第一比较模块400以及第二比较模块402比较结果，判断前级电路120以及后级电路130的运作状况，进一步决定采用的同步机制。
请同时参照图5。图5为本发明的一实施例中，同步电路150运作时进行的一同步流程500的流程图。以下将根据图4及图5，对于同步电路150的运作机制进行更详细的说明。
如图5所示，于步骤S510，第一比较模块400自前级电路120以及后级电路130撷取其画面处理信息FP1及FP2中的前级画面数F1以及后级画面数F2进行比较。
于步骤S520，同步处理模块404根据第一比较模块400的第一比较结果CR1，进一步判断前级电路120正在暂存的影像画面以及后级电路130正在读取的影像画面是否为相同的。
当影像画面是不同的，例如在影像画面处理时间区间TN的第一子区间TS1中的情形，后级电路130读取的是第N-1个子影像画面而前级电路120储存的是第N个主影像画面，则于步骤S530，同步处理模块404不启动同步电路150的同步机制。
当影像画面是相同的，例如在影像画面处理时间区间TN的第二子区间TS2中的情形，前级电路120储存的以及后级电路130读取的均是第N个主影像画面时，同步处理模块404启动同步机制的运作，并于步骤S540，由第二比较模块402自前级电路120以及后级电路130撷取其画面处理信息FP1及FP2中的储存行数RW以及读取行数RR进行比较。
请同时参照图6。图6为本发明的一实施例中，主影像画面600的示意图。于步骤S550，同步处理模块404根据第二比较模块402的第二比较结果CR2，判断前级电路120正在暂存的主影像画面600的储存行数RW是否超过后级电路130读取的主影像画面600的读取行数RR。
在储存行数RW超过读取行数RR时，表示后级电路130读取的内容尚未超过前级电路120储存的内容。因此，于步骤S560，同步处理模块404藉由后级电路控制信号CC1允许后级电路130继续读取第N个主影像画面进行处理。反之，在储存行数RW不超过读取行数RR时，表示后级电路130读取的内容将超过前级电路120储存的内容。因此，于步骤S570，同步处理模块404藉由后级电路控制信号CC1使后级电路130停止读取第N个主影像画面，以避免读取到错误的数据内容。
因此，后级电路130将在同步电路150的协调与同步下，依序读取第N个主影像画面的内容，并进行处理以产生处理后主影像串流EMS对应的主影像画面。需注意的是，在本实施例中，后级电路130是在下一个同步信号时间区间TSN+1前即完成对于第N个主影像画面的处理。
进一步地，在影像画面处理时间区间TN+1中，前级电路120可依照上述的方式暂存子影像串流SS1、SS2的第N+1个子影像画面，以及暂存主影像串流MS的第N+1个主影像画面。后级电路130则可依照上述的方式先读取子影像串流SS1及SS2的第N个子影像画面进行处理后，再读取主影像串流MS的第N+1个主影像画面进行处理。
需注意的是，于一实施例中，前级电路120在影像画面处理时间区间TN中，用以暂 存子影像串流SS1、SS2的第N个子影像画面的当下第一及第二子影像暂存区块114A及116A，对于影像画面处理时间区间TN+1来说则成为现有子影像暂存区块。而在影像画面处理时间区间TN中的现有第一及第二子影像暂存区块114B及116B，对于影像画面处理时间区间TN+1来说则作为当下子影像暂存区块，并用来储存子影像串流SS1、SS2的第N+1个子影像画面。上述的实施方式是以主影像暂存区块112的容量大于或等于第N个主影像画面的大小为范例进行说明。于另一实施例中，主影像暂存区块112的容量亦可小于第N个主影像画面的大小，且运作为环状缓冲器(ring buffer)。亦即，当前级电路120对主影像画面的暂存到主影像暂存区块112的最高地址时，将把更新的影像数据取代主影像暂存区块112的最低地址进行储存。而当后级电路130对主影像画面的读取到主影像暂存区块112的最高地址时，亦将回到主影像暂存区块112的最低地址继续读取。
请同时参照图7及图8。图7为本发明的另一实施例中，同步电路150的方框图。图8为本发明的另一实施例中，同步电路150运作时进行的一同步流程800的流程图。以下将根据图7及图8，对于以环状缓冲器运作的主影像暂存区块112的实施方式进行更详细的说明。
图7所绘示的同步电路150与图4所绘示的同步电路150大同小异，包含第一比较模块400、第二比较模块402及同步处理模块404。于本实施例中，图7的同步电路150更包含第一计算模块406。第一计算模块406配置以根据画面处理信息FP1及FP2，计算储存行数RW与读取行数RR的差距DR。图7所绘示的同步电路150所进行的同步机制除了与图5相同的步骤S510至S570外，可更同时执行图8的流程，以进行步骤S810，由第一计算模块406计算储存行数RW与读取行数RR的差距DR。
于步骤S820，同步处理模块404判断差距DR是否大于主影像暂存区块112的容量。
在差距DR不大于主影像暂存区块112的容量时，表示主影像暂存区块112的内部有效未编码内容尚未占用完用完主影像暂存区块112的容量。因此，于步骤S830，同步处理模块404藉由前级电路控制信号CC2允许前级电路120继续暂存第N个主影像画面。反之，在差距大于主影像暂存区块112的容量时，表示前级电路120储存的速度超过后级电路130读取的速度，如果继续储存，将覆盖掉后级电路130尚未读取到的主影像画面的数据。因此，于步骤S840，同步处理模块404藉由前级电路控制信号CC2将使前级电路120停止暂存第N个主影像画面，进一步暂缓对于主影像画面数据的储存。
因此，藉由上述的机制，在以环状缓冲器运作的主影像暂存区块112的实施方式中，除可避免后级电路130读取的速度过快读到错误的数据外，亦可避免前级电路120储存的 速度过快而覆盖后级电路130尚未读取的数据。
请同时参照图9。图9为本发明另一实施例中，多串流影像处理装置100进行影像处理的时序图。
多串流影像处理装置100在处理的时序上，包含多个影像画面处理时间区间，例如图5范例性绘示的连续三个影像画面处理时间区间TN-1、TN及TN+1。各两个邻接的影像画面处理时间区之间包含同步信号时间区间，例如影像画面处理时间区间TN-1及TN之间的同步信号时间区间TSN以及影像画面处理时间区间TN及TN+1之间的同步信号时间区间TSN+1。同步信号时间区间TSN以及同步信号时间区间TSN+1是分别对应同步信号SN以及同步信号SN+1的传送，以分别标示影像画面处理时间区间TN及TN+1的起始。
在本实施例中，在影像画面处理时间区间TN的第一子区间TS1中，依旧是由后级电路130读取子影像串流SS1及SS2的第N-1个子影像画面进行处理，且在影像画面处理时间区间TN的第二子区间TS2中，也依旧是由后级电路130读取第N个主影像画面进行处理。
然而，在本实施例中，除了主影像暂存区块112是以环状缓冲器的方式实现外，影像画面处理时间区间TN在第一子区间TS1前更包含第三子区间TS3。对于影像画面处理时间区间TN-1所对应的第N-1个主影像画面，后级电路130除了在影像画面处理时间区间TN-1中的后半部进行处理外，也被允许在同步信号时间区间TSN以及第三子区间TS3中进行处理。以此类推，后级电路130亦被允许在影像画面处理时间区间TN中的后半部、同步信号时间区间TSN+1以及影像画面处理时间区间TN+1的第三子区间TS3中，处理第N个主影像画面。
然而，对于前级电路120来说，依旧是在影像画面处理时间区间TN起始时，即开始暂存子影像串流SS1、SS2的第N个子影像画面以及主影像串流MS的第N个主影像画面。因此，主影像暂存区块112将在第三子区间TS3中，同时进行第N-1个主影像画面的读取以及第N个主影像画面的暂存。增加此Ts3也具有的优点是后级电路的处理速度可以较慢，亦即可以较省电。
请同时参照图10及图11。图10为本发明的又一实施例中，同步电路150的方框图。图11为本发明的又一实施例中，同步电路150运作时进行的一同步流程1100的流程图。以下将根据图10及图11，对于以环状缓冲器运作的主影像暂存区块112的实施方式进行更详细的说明。
图10所绘示的同步电路150与图7所绘示的同步电路150大同小异，包含第一比较模块400、第二比较模块402、同步处理模块404及第一计算模块406。于本实施例中，图10的同步电路150更包含第二计算模块408，配置以在前级电路120正在暂存的影像画面以及后级电路130正在读取的影像画面不同时运作。
图10所绘示的同步电路150所进行的同步机制具有与图5相同的步骤S510、S520以及S540至S570，以及与图8相同的步骤S810至S840。然而，在本实施例中，当图5的步骤S520判断前级电路120正在暂存的影像画面以及后级电路130正在读取的影像画面是不同的，例如第三子区间TS3的情形时，图10所绘示的同步电路150将执行图11的流程，以启动同步机制的运作。于步骤S1110，由第二计算模块408根据画面处理信息FP1及FP2先计算第N-1个主影像画面的大小FR与第N-1个主影像画面的读取行数RR的差距。此差距代表第N-1个主影像画面中，剩下还未由后级电路130读取并进行处理的数据量。接着，第二计算模块408计算此差距与第N个主影像画面的储存行数RWN的总和SUM。其中，储存行数RW代表第N个主影像画面中，前级电路120已暂存的数据量。
于步骤S1120，同步处理模块404判断总和SUM是否大于主影像暂存区块112的容量。
在总和SUM不大于主影像暂存区块112的容量时，表示前级电路120的储存速度并未超过后级电路130的读取速度。因此，于步骤S1130，同步处理模块404藉由前级电路控制信号CC2允许前级电路120继续暂存第N个主影像画面。反之，在总和SUM大于主影像暂存区块112的容量时，表示前级电路120储存的速度超过后级电路130读取的速度，如果继续储存，将覆盖掉后级电路130尚未读取到的主影像画面的数据。因此，于步骤S1140，同步处理模块404藉由前级电路控制信号CC2将使前级电路120停止产生第N个主影像画面，进一步暂缓对于主影像画面数据的储存。
因此，本发明的多串流影像处理装置100可使前级电路所产生的多个影像串流，藉由分时的方式以单一后级电路进行处理，避免多个后级电路的高昂设置成本。
并且，藉由上述分时的方式，存储器模块仅需要对于因分辨率较低而数据量较小的子影像画面进行第N-1个子影像画面以及第N个子影像画面的暂存。对于分辨率高而数据量大的主影像画面，则不需要额外设置两个暂存区块进行暂存。对于存储器模块的空间成本来说，也将大幅下降。
进一步地，通过同步电路的设置，存储器模块可使主影像暂存区块以环状缓冲器的方 式实现，容量将因而缩减，更进一步降低存储器模块的空间成本。
需注意的是，上述的实施方式仅为一范例。于其他实施例中，本领域的通常知识者当可在不违背本发明的精神下进行更动。
综合上述，本发明中的多串流影像处理装置及方法可达到降低后级电路硬件设置成本以及降低存储器空间成本的功效。
当然，本发明还可有其它多种实施例，在不背离本发明精神及其实质的情况下，熟悉本领域的技术人员当可根据本发明作出各种相应的改变和变形，但这些相应的改变和变形都应属于本发明所附的权利要求的保护范围。

Claims (14)

  1. 一种多串流影像处理方法，其特征在于，包含：
    由一前级电路根据一相同影像来源产生多个影像串流，其中所述影像串流包含一主影像串流以及至少一子影像串流，且该主影像串流的分辨率高于该子影像串流的分辨率；
    于一影像画面处理时间区间中，由该前级电路暂存该子影像串流的一第N个子影像画面于一存储器模块中的至少一当下子影像暂存区块，以及暂存该主影像串流的一第N个主影像画面于该存储器模块中的一主影像暂存区块；
    于该影像画面处理时间区间的一第一子区间中，由一后级电路读取该存储器模块中的至少一现有子影像暂存区块中暂存的该子影像串流的一第N-1个子影像画面，以进行处理；以及
    于该影像画面处理时间区间中该第一子区间后的一第二子区间中，由该后级电路读取该主影像暂存区块中暂存的该第N个主影像画面，以进行处理，
    其中，N为大于或者等于2的正整数。
  2. 根据权利要求1所述的多串流影像处理方法，其特征在于，该主影像暂存区块的容量大于或等于该第N个主影像画面的大小。
  3. 根据权利要求1所述的多串流影像处理方法，其特征在于，更包含：
    由一同步电路撷取该前级电路以及该后级电路的一画面处理信息；
    由该同步电路根据该画面处理信息判断该前级电路暂存该第N个主影像画面的一储存行数是否超过该后级电路读取的该第N个主影像画面的一读取行数；
    在该储存行数超过该读取行数时，允许该后级电路继续读取该第N个主影像画面进行处理；以及
    在该储存行数不超过该读取行数时，使该后级电路停止读取该第N个主影像画面。
  4. 根据权利要求3所述的多串流影像处理方法，其特征在于，该主影像暂存区块的容量小于该第N个主影像画面的大小，且运作为一环状缓冲器，该多串流影像处理方法更包含：
    由该同步电路根据该画面处理信息判断该储存行数与该读取行数的一差距是否大于该主影像暂存区块的容量；
    在该差距不大于该主影像暂存区块的容量时，允许该前级电路继续暂存该第N个主 影像画面；以及
    在该差距大于该主影像暂存区块的容量时，使该前级电路停止暂存该第N个主影像画面。
  5. 根据权利要求1所述的多串流影像处理方法，其特征在于，该影像画面处理时间区间与一现有影像画面处理时间区间之间包含一同步信号时间区间，该前级电路以及该后级电路在该同步信号时间区间不进行数据处理。
  6. 根据权利要求1所述的多串流影像处理方法，其特征在于，该主影像暂存区块的容量小于该第N个主影像画面的大小，且运作为一环状缓冲器，该影像画面处理时间区间与一现有影像画面处理时间区间之间包含一同步信号时间区间，该多串流影像处理方法更包含：
    于该同步信号时间区间以及该影像画面处理时间区间中该第一子区间前的一第三子区间中，由该后级电路读取该主影像暂存区块中暂存的一第N-1个主影像画面，以进行处理；
    由一同步电路撷取该前级电路以及该后级电路的一画面处理信息；
    由该同步电路根据该画面处理信息计算该第N-1个主影像画面的大小与该第N-1个主影像画面的一读取行数的一差距；
    由该同步电路根据该画面处理信息判断该差距与该第N个主影像画面的一储存行数的一总和是否大于该主影像暂存区块的容量；
    在该总和不大于该主影像暂存区块的容量时，允许该前级电路继续暂存该第N个主影像画面；以及
    在该总和大于该主影像暂存区块的容量时，使该前级电路停止暂存该第N个主影像画面。
  7. 根据权利要求1所述的多串流影像处理方法，其特征在于，该前级电路为一影像信号处理器，该后级电路为一编码器。
  8. 一种多串流影像处理装置，其特征在于，包含：
    一存储器模块；
    一前级电路；
    一后级电路；以及
    一处理电路，电性耦接于该存储器模块、该前级电路以及该后级电路，并配置以执行多个软固件可执行指令，以执行一多串流影像处理方法，其中该多串流影像处理方法包含：
    由该前级电路根据一相同影像来源产生多个影像串流，其中该影像串流包含一主影像串流以及至少一子影像串流，且该主影像串流的分辨率高于该子影像串流的分辨率；
    于一影像画面处理时间区间中，由该前级电路暂存该子影像串流的一第N个子影像画面于该存储器模块中的至少一当下子影像暂存区块，以及暂存该主影像串流的一第N个主影像画面于该存储器模块中的一主影像暂存区块；
    于该影像画面处理时间区间的一第一子区间中，由一后级电路读取该存储器模块中的至少一现有子影像暂存区块中暂存的该子影像串流的一第N-1个子影像画面，以进行处理；以及
    于该影像画面处理时间区间中该第一子区间后的一第二子区间中，由该后级电路读取该主影像暂存区块中暂存的该第N个主影像画面，以进行处理，
    其中，N为大于或者等于2的正整数。
  9. 根据权利要求8所述的多串流影像处理装置，其特征在于，该主影像暂存区块的容量大于或等于该第N个主影像画面的大小。
  10. 根据权利要求8所述的多串流影像处理装置，其特征在于，更包含一同步电路，电性耦接于该处理电路、该前级电路以及该后级电路，该多串流影像处理方法更包含：
    由该同步电路撷取该前级电路以及该后级电路的一画面处理信息；
    由该同步电路根据该画面处理信息判断该前级电路暂存该第N个主影像画面的一储存行数是否超过该后级电路读取的该第N个主影像画面的一读取行数；
    在该储存行数超过该读取行数时，允许该后级电路继续读取该第N个主影像画面进行处理；以及
    在该储存行数不超过该读取行数时，使该后级电路停止读取该第N个主影像画面。
  11. 根据权利要求10所述的多串流影像处理装置，其特征在于，该主影像暂存区块的容量小于该第N个主影像画面的大小，且运作为一环状缓冲器，该多串流影像处理方法更包含：
    由该同步电路根据该画面处理信息判断该储存行数与该读取行数的一差距是否大于该主影像暂存区块的容量；
    在该差距不大于该主影像暂存区块的容量时，允许该前级电路继续暂存该第N个主 影像画面；以及
    在该差距大于该主影像暂存区块的容量时，使该前级电路停止暂存该第N个主影像画面。
  12. 根据权利要求8所述的多串流影像处理装置，其特征在于，该影像画面处理时间区间与一现有影像画面处理时间区间之间包含一同步信号时间区间，该前级电路以及该后级电路在该同步信号时间区间不进行数据处理。
  13. 根据权利要求8所述的多串流影像处理装置，其特征在于，更包含一同步电路，电性耦接于该处理电路、该前级电路以及该后级电路，其中该主影像暂存区块的容量小于该第N个主影像画面的大小，且运作为一环状缓冲器，该影像画面处理时间区间与一现有影像画面处理时间区间之间包含一同步信号时间区间，该多串流影像处理方法更包含：
    于该同步信号时间区间以及该影像画面处理时间区间中该第一子区间前的一第三子区间中，由该后级电路读取该主影像暂存区块中暂存的一第N-1个主影像画面，以进行处理；
    由一同步电路撷取该前级电路以及该后级电路的一画面处理信息；
    由该同步电路根据该画面处理信息计算该第N-1个主影像画面的大小与该第N-1个主影像画面的一读取行数的一差距；
    由该同步电路根据该画面处理信息判断该差距与该第N个主影像画面的一储存行数的一总和是否大于该主影像暂存区块的容量；
    在该总和不大于该主影像暂存区块的容量时，允许该前级电路继续暂存该第N个主影像画面；以及
    在该总和大于该主影像暂存区块的容量时，使该前级电路停止暂存该第N个主影像画面。
  14. 根据权利要求8所述的多串流影像处理装置，其特征在于，该前级电路为一影像信号处理器，该后级电路为一编码器。
PCT/CN2019/127870 2018-12-26 2019-12-24 多串流影像处理装置及方法 WO2020135407A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811602356.7 2018-12-26
CN201811602356.7A CN111372038B (zh) 2018-12-26 2018-12-26 多串流影像处理装置及方法

Publications (1)

Publication Number Publication Date
WO2020135407A1 true WO2020135407A1 (zh) 2020-07-02

Family

ID=68917538

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/127870 WO2020135407A1 (zh) 2018-12-26 2019-12-24 多串流影像处理装置及方法

Country Status (7)

Country Link
US (1) US10771798B2 (zh)
EP (1) EP3675028A1 (zh)
JP (1) JP6894473B2 (zh)
KR (1) KR102191387B1 (zh)
CN (1) CN111372038B (zh)
TW (1) TWI691202B (zh)
WO (1) WO2020135407A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA3205672A1 (en) * 2020-12-18 2022-06-23 Fasetto, Inc. Systems and methods for simultaneous multiple point of view video

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1697483A (zh) * 2004-05-13 2005-11-16 索尼公司 图像显示装置
CN101889446A (zh) * 2007-10-31 2010-11-17 松下电器产业株式会社 运动图像解码装置
CN105282597A (zh) * 2015-10-17 2016-01-27 浙江宇视科技有限公司 一种视频流缩放装置和方法
US20160119626A1 (en) * 2014-10-22 2016-04-28 Genetec Inc. System to dispatch video decoding to dedicated hardware resources
CN106534857A (zh) * 2016-11-02 2017-03-22 深圳市维海德技术股份有限公司 码流传输方法及基于usb的摄像机

Family Cites Families (41)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2703417B2 (ja) * 1991-04-05 1998-01-26 富士通株式会社 受信バッファ
US5367629A (en) * 1992-12-18 1994-11-22 Sharevision Technology, Inc. Digital video compression system utilizing vector adaptive transform
US5432554A (en) * 1993-06-16 1995-07-11 Intel Corporation Method and apparatus for decoding images using a specified data format
US5351085A (en) * 1993-06-16 1994-09-27 Intel Corporation Method and system for generating compressed image signals
US5511003A (en) * 1993-11-24 1996-04-23 Intel Corporation Encoding and decoding video signals using spatial filtering
JP2001103479A (ja) * 1999-09-29 2001-04-13 Toshiba Corp 動画像復号装置
JP2003259315A (ja) * 2002-02-28 2003-09-12 Matsushita Electric Ind Co Ltd 画像伝送装置および画像伝送方法
US7907665B2 (en) * 2003-03-14 2011-03-15 Lsi Corporation Multi-channel video compression system
JP4517592B2 (ja) * 2003-06-16 2010-08-04 ソニー株式会社 画像処理装置、画像処理方法、画像処理プログラムおよび記録媒体
AU2003272483A1 (en) * 2003-09-12 2005-04-27 Nielsen Media Research, Inc. Digital video signature apparatus and methods for use with video program identification systems
JP2005176001A (ja) * 2003-12-12 2005-06-30 Renesas Technology Corp 半導体装置及び画像処理装置
US7239355B2 (en) * 2004-05-17 2007-07-03 Mstar Semiconductor, Inc. Method of frame synchronization when scaling video and video scaling apparatus thereof
JP4689440B2 (ja) * 2005-11-04 2011-05-25 キヤノン株式会社 撮像装置、及び処理方法
US7657668B2 (en) * 2006-08-16 2010-02-02 Qnx Software Systems (Wavemakers), Inc. Clock synchronization of data streams
CN101202906A (zh) * 2006-12-11 2008-06-18 国际商业机器公司 在数字视频广播系统中对视频流进行处理的方法和设备
US9565419B2 (en) * 2007-04-13 2017-02-07 Ari M. Presler Digital camera system for recording, editing and visualizing images
TWI397899B (zh) 2007-04-30 2013-06-01 Mstar Semiconductor Inc 多視窗顯示控制器及相關方法
CN101312528B (zh) * 2007-05-24 2010-06-23 晶睿通讯股份有限公司 实时产生不同分辨率视讯串流的方法与系统
US7870224B1 (en) * 2007-06-08 2011-01-11 Adobe Systems Incorporated Managing online composite image content
CN102301729B (zh) * 2009-01-28 2014-12-17 日本电气株式会社 图像传输系统和图像传输方法
CN105472369B (zh) * 2009-01-29 2017-09-12 杜比实验室特许公司 视频装置
JP5533861B2 (ja) * 2009-04-30 2014-06-25 ソニー株式会社 表示制御装置、表示制御方法、及び、プログラム
JP2011223287A (ja) * 2010-04-09 2011-11-04 Sony Corp 情報処理装置、情報処理方法、及び、プログラム
CA2771433C (en) * 2010-08-09 2016-01-12 Panasonic Corporation Coding and decoding of stereoscopic images with base and enhancement layers
US10424274B2 (en) * 2010-11-24 2019-09-24 Ati Technologies Ulc Method and apparatus for providing temporal image processing using multi-stream field information
MY165813A (en) * 2011-02-25 2018-04-27 Photonis Netherlands B V Acquiring and displaying images in real-time
CN102346770B (zh) * 2011-09-21 2014-05-14 晨星软件研发(深圳)有限公司 WebKit浏览器网页内容加载方法及装置
US20150062353A1 (en) * 2013-08-30 2015-03-05 Microsoft Corporation Audio video playback synchronization for encoded media
US9538155B2 (en) * 2013-12-04 2017-01-03 Dolby Laboratories Licensing Corporation Decoding and display of high dynamic range video
TWI604432B (zh) * 2014-03-18 2017-11-01 緯創資通股份有限公司 顯示器及其視訊畫面顯示方法
US20160261908A1 (en) * 2015-03-05 2016-09-08 Htc Corporation Media streaming system and control method thereof
US9916252B2 (en) * 2015-05-19 2018-03-13 Linear Algebra Technologies Limited Systems and methods for addressing a cache with split-indexes
US9454820B1 (en) * 2015-06-12 2016-09-27 Google Inc. Using a scene illuminating infrared emitter array in a video monitoring camera for depth determination
WO2017150238A1 (ja) * 2016-03-03 2017-09-08 ソニー株式会社 表示制御装置、表示制御方法、及び、プログラム
US9967465B2 (en) * 2016-05-18 2018-05-08 Realtek Singapore Pte Ltd Image frame processing method
US9996894B2 (en) * 2016-05-18 2018-06-12 Realtek Singapore Pte Ltd Image processing device, video subsystem and video pipeline
US9992467B2 (en) * 2016-06-30 2018-06-05 Apple Inc. Parallel computer vision and image scaling architecture
CN106101545B (zh) * 2016-06-30 2019-10-15 维沃移动通信有限公司 一种图像处理方法及移动终端
US10341672B2 (en) * 2017-02-03 2019-07-02 Korea Advanced Institute Of Science And Technology Method and system for media synchronization
US10339421B2 (en) * 2017-03-30 2019-07-02 Toyota Motor Engineering & Manufacturing North America, Inc. RGB-D scene labeling with multimodal recurrent neural networks
CN111372085B (zh) * 2018-12-25 2021-07-09 厦门星宸科技有限公司 影像解码装置与方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1697483A (zh) * 2004-05-13 2005-11-16 索尼公司 图像显示装置
CN101889446A (zh) * 2007-10-31 2010-11-17 松下电器产业株式会社 运动图像解码装置
US20160119626A1 (en) * 2014-10-22 2016-04-28 Genetec Inc. System to dispatch video decoding to dedicated hardware resources
CN105282597A (zh) * 2015-10-17 2016-01-27 浙江宇视科技有限公司 一种视频流缩放装置和方法
CN106534857A (zh) * 2016-11-02 2017-03-22 深圳市维海德技术股份有限公司 码流传输方法及基于usb的摄像机

Also Published As

Publication number Publication date
US20200213601A1 (en) 2020-07-02
EP3675028A1 (en) 2020-07-01
CN111372038A (zh) 2020-07-03
US10771798B2 (en) 2020-09-08
KR102191387B1 (ko) 2020-12-15
TW202025787A (zh) 2020-07-01
TWI691202B (zh) 2020-04-11
JP6894473B2 (ja) 2021-06-30
JP2020108136A (ja) 2020-07-09
CN111372038B (zh) 2021-06-18
KR20200080095A (ko) 2020-07-06

Similar Documents

Publication Publication Date Title
US20180174551A1 (en) Sending frames using adjustable vertical blanking intervals
US8279344B2 (en) Synchronization of video presentation by video cadence modification
JP2007028261A (ja) 映像音声再生装置および映像音声再生方法
WO2020135407A1 (zh) 多串流影像处理装置及方法
JP2008154011A (ja) フレームシンクロナイザ回路
CN105472203A (zh) Genlock同步锁相系统及方法
JP2012169727A (ja) 映像信号処理装置および映像信号処理方法
CN110570793B (zh) 一种适配不同类型显示屏的测试方法、装置及终端设备
US9456180B2 (en) Image processing apparatus, communication system, and computer program
US6774950B1 (en) Displaying video images
CN112954432B (zh) 视频数据处理方法、装置、系统及可读存储介质
TW202037175A (zh) 使用單一視訊處理路徑來處理多個視訊的視訊處理電路及相關方法
US10212316B2 (en) Video processing apparatus
CN108965764B (zh) 图像处理方法及电子设备
JP2006191538A (ja) 圧縮ストリーム復号装置及び圧縮ストリーム復号方法
JP7480908B2 (ja) 映像合成装置及び映像合成方法
JP2006310906A (ja) 映像信号処理装置
CN116112627B (zh) 一种视频帧率自适应变换的方法和电路
US11538398B2 (en) Sub-frame synchronization of multiple displays with independent vertical refresh rates
JP3882688B2 (ja) 画像音声同期化装置
JP7304453B1 (ja) 符号化装置、符号化方法および符号化プログラム
JP3269331B2 (ja) 画像取り込み回路
JP2009122311A (ja) 画像処理システム、表示装置および画像処理方法
JPH1023402A (ja) 並列ビデオデコーダシステム
JP2015228588A (ja) 映像処理装置、その制御方法、およびプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19902990

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19902990

Country of ref document: EP

Kind code of ref document: A1