WO2022067683A1 - 图像传输方法、装置、设备和介质 - Google Patents

图像传输方法、装置、设备和介质 Download PDF

Info

Publication number
WO2022067683A1
WO2022067683A1 PCT/CN2020/119440 CN2020119440W WO2022067683A1 WO 2022067683 A1 WO2022067683 A1 WO 2022067683A1 CN 2020119440 W CN2020119440 W CN 2020119440W WO 2022067683 A1 WO2022067683 A1 WO 2022067683A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
transmitted
time
transmission
frame
Prior art date
Application number
PCT/CN2020/119440
Other languages
English (en)
French (fr)
Inventor
李林
何先铃
Original Assignee
浙江宇视科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 浙江宇视科技有限公司 filed Critical 浙江宇视科技有限公司
Priority to US18/027,032 priority Critical patent/US20230376259A1/en
Priority to EP20955708.1A priority patent/EP4224862A4/en
Priority to PCT/CN2020/119440 priority patent/WO2022067683A1/zh
Priority to CN202080104910.8A priority patent/CN116250229A/zh
Publication of WO2022067683A1 publication Critical patent/WO2022067683A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/363—Graphics controllers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/10—Program control for peripheral devices
    • G06F13/12—Program control for peripheral devices using hardware independent of the central processor, e.g. channel or peripheral processor
    • G06F13/124—Program control for peripheral devices using hardware independent of the central processor, e.g. channel or peripheral processor where hardware is a sequential transfer control unit, e.g. microprocessor, peripheral processor or state-machine
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38—Information transfer, e.g. on bus
    • G06F13/42—Bus transfer protocol, e.g. handshake; Synchronisation
    • G06F13/4282—Bus transfer protocol, e.g. handshake; Synchronisation on a serial bus, e.g. I2C bus, SPI bus
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/001—Arbitration of resources in a display system, e.g. control of access to frame buffer by video controller and/or main processor
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H04L65/752—Media network packet handling adapting media to network capabilities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H04L65/762—Media network packet handling at the source 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80—Responding to QoS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/75—Indicating network or usage conditions on the user display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/40—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2350/00—Solving problems of bandwidth in display systems

Definitions

  • the present application relates to the field of image transmission, for example, to an image transmission method, apparatus, device and medium.
  • Ultra-high-definition image display means that while ensuring the smoothness of image display, the image resolution is improved, that is, the amount of data transmitted per unit time will be greatly increased.
  • FPGA Field Programmable Gate Array
  • PCIE Peripheral Component Interconnect Express
  • the present application provides an image transmission method, apparatus, device and medium, so as to realize the orderly and rapid transmission and display of high-resolution images.
  • An image transmission method including:
  • the transmission bandwidth between the transmitting end and the receiving end determine the transmission required by the transmitting end to transmit one frame of the image to be transmitted in each channel of the transmitting end time;
  • the theoretical time corresponding to the transmission of one frame of the image to be transmitted is segmented to obtain the time period corresponding to the transmission time;
  • the sending end is controlled to send the to-be-transmitted image corresponding to the transmission time to the receiving end in the time period corresponding to the transmission time.
  • an image transmission device comprising:
  • the transmission time determination module is configured to determine that the transmitting end transmits one frame in each channel according to the data amount of the image to be transmitted in one frame in each channel of the transmitting end and the transmission bandwidth between the transmitting end and the receiving end The transmission time required for the image to be transmitted;
  • the segmentation module is configured to segment the theoretical time corresponding to the transmission of one frame of the image to be transmitted according to the transmission time required by the transmitting end to transmit one frame of the image to be transmitted in each channel, and obtain the time corresponding to the transmission time part;
  • the control module is configured to control the sending end to send the to-be-transmitted image corresponding to the transmission time to the receiving end in a time period corresponding to the transmission time.
  • an electronic device comprising:
  • the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the image transmission method as described above.
  • FIG. 1 is a schematic flowchart of an image transmission method provided by an embodiment of the present application.
  • FIG. 2 is a schematic diagram of a FPGA+PCIE distributed application system provided by an embodiment of the present application
  • FIG. 3 is a schematic flowchart of another image transmission method provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of time slice misalignment of a transmitter and a receiver provided by an embodiment of the present application
  • FIG. 5 is a schematic diagram of a synchronization pulse signal transmission provided by an embodiment of the present application.
  • FIG. 6 is a flowchart of an implementation of an image transmission method provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a theoretical flow of image data of a distributed application system of FPGA+PCIE provided by an embodiment of the present application;
  • FIG. 8 is a schematic diagram of distributing image data A and A' provided by an embodiment of the present application.
  • FIG. 9 is a schematic diagram of a kind of allocation image data A and A' sending preemption according to an embodiment of the present application.
  • FIG. 10 is a schematic diagram of distributing image data B and B' provided by an embodiment of the present application.
  • FIG. 11 is a schematic diagram of a transmission preemption occurring in allocating image data B and B' provided by an embodiment of the present application;
  • FIG. 12 is a schematic diagram of a preemption result of allocating image data provided by an embodiment of the present application.
  • FIG. 13 is a schematic diagram of instantaneous bandwidth overload of a display card provided by an embodiment of the present application.
  • FIG. 14 is a schematic diagram of the image data plan flow of a distributed application system of FPGA+PCIE provided by an embodiment of the present application;
  • 15 is a schematic diagram of time slice allocation of image data A and A' provided by an embodiment of the present application.
  • 16 is a schematic diagram of time slice allocation of image data B and B' provided by an embodiment of the present application.
  • 17 is a schematic diagram of time slice allocation of image data C and C' provided by an embodiment of the present application.
  • 19 is a schematic diagram of a comparison of instantaneous bandwidth in image transmission provided by an embodiment of the present application.
  • FIG. 20 is a schematic structural diagram of an image transmission apparatus provided by an embodiment of the present application.
  • FIG. 21 is a block diagram of an electronic device for implementing an image transmission method provided by an embodiment of the present application.
  • the sending end may be a node on the outflow side of the image data
  • the receiving end may be a node on the inflow side of the image data. It is not unique to the devices that are the sending end and the receiving end.
  • the sending end can be a capture card, a server, a single-chip microcomputer, etc.
  • the receiving end can be a display card, a memory, and the like.
  • the data transmission protocol or intermediate device between the sender and the receiver depends on the devices used as the sender and receiver. For example, if the sender is a capture card and the receiver is a display card, the PCIE bus can be used as the data transfer protocol.
  • FIG. 1 is a schematic flowchart of an image transmission method provided by an embodiment of the present application.
  • This embodiment can be applied to the situation in which the transmitting end transmits the image to the receiving end.
  • this embodiment may be applicable to a situation where multiple senders send images to one receiver, or multiple senders send images to multiple receivers.
  • the image transmission method disclosed in this embodiment may be performed by an image transmission apparatus, and the apparatus may be implemented by means of software and/or hardware, and may be integrated into an electronic device.
  • the image transmission method provided by this embodiment includes:
  • S110 Determine the transmission time required by the transmitting end to transmit one frame of the image to be transmitted in each channel of the transmitting end according to the data amount of the image to be transmitted in one frame of each channel of the transmitting end and the transmission bandwidth between the transmitting end and the receiving end .
  • FIG. 2 is a schematic diagram of an FPGA+PCIE distributed application system provided by an embodiment of the present application.
  • the system may include a main control board, multiple capture cards and multiple display cards.
  • the structure and form of the main control board are not limited, and only a device capable of managing and controlling the capture card and the display card may be used.
  • Multiple capture cards can respectively send multi-channel images to any display card, that is, the display card can receive multi-channel images sent by multiple capture cards.
  • only 4 channels of images are shown to be transmitted, which is not a limitation on the number of transmitted images. In fact, any channel of images can be transmitted.
  • the instantaneous bandwidth of the display card may be overloaded, and the images cannot be received normally.
  • the card freezes during display playback.
  • the main control board manages and controls the image sending process of the capture card, so that multiple capture cards send images in an orderly manner when sending images to the same display card, thereby improving the efficiency of image sending.
  • the time used by each capture card to transmit images so as to manage and control each capture card, all capture cards that need to send images to a display card are pre-determined, and each capture card needs to send images to the display card.
  • the transmission time required to transmit one frame of the image to be transmitted in each channel can be calculated.
  • the transmission bandwidth is the PCIE transmission bandwidth between EPs.
  • the ratio of the data amount of one frame of the image to be transmitted to the transmission bandwidth is the transmission time required to transmit one frame of the to-be-transmitted image in each channel.
  • the above scheme pre-determines the transmission time required to transmit a frame of images to be transmitted in each channel, and can comprehensively control the transmission time required for multiple capture cards to transmit images to be transmitted, so as to transmit the images to be transmitted according to the transmission time of the transmission. Time to effectively control the image sending process of the capture card.
  • Each frame of the image to be transmitted on each capture card has a corresponding transmission time, which is the time required to transmit the frame to be transmitted.
  • transmission time is the time required to transmit the frame to be transmitted.
  • the images to be transmitted that the capture card 1 needs to send to the display card 1 are image 1, image 2, and image 3, the transmission time required for transmitting image 1 is t1, and the transmission time required for transmitting image 2 is t2, The transmission time required to transmit image 3 is t3.
  • the images to be transmitted that the capture card 2 needs to send to the display card 1 are image 1', image 2', and image 3', the transmission time required to transmit image 1' is t4, and the transmission time required to transmit image 2' is t5 , the transmission time required to transmit the image 3' is t6.
  • the time corresponding to transmitting one frame of the image to be transmitted can be divided into 6 segments, corresponding to t1, t2, t3, t4, t5 and t6 respectively. That is, the theoretical time corresponding to transmitting a frame of images to be transmitted is divided into 0-t1, t1-(t1+t2), (t1+t2)-(t1+t2+t3), (t1+t2+t3)- (t1+t2+t3+t4), (t1+t2+t3+t4)-(t1+t2+t3+t4+t5), (t1+t2+t3+t4+t5)-(t1+t2+t3 +t4+t5+t6) 6 segments.
  • the order of the time segments obtained by segmenting the theoretical time corresponding to the transmission of a frame of images to be transmitted is not limited.
  • the six time segments correspond to t1, t2, t3, t4, and t5 respectively.
  • t6 it is also possible to make the six time periods obtained by segmentation correspond to t3, t5, t2, t1, t6, and t4, respectively, or correspond to other transmission times in other sequences, which are not limited here.
  • the above scheme divides the theoretical time corresponding to the transmission of one frame of images to be transmitted according to the time required to transmit one frame of images to be transmitted in each channel, so that the multiple frames of images to be transmitted are misaligned in the time period, so that there is program management and control.
  • a control instruction may be issued to the capture card to control the capture card to send the data to be transmitted in the corresponding time period, or it may be pre-delivered to all capture cards.
  • the acquisition card sends control instructions, and controls the acquisition card to send transmission data when the corresponding time period is reached.
  • the control instructions can also be actively obtained by the capture card from the main control board.
  • the capture card 1 controls the capture card 1 to send the image 3 to the display card 1 in the time period 0-t3, and control the capture
  • the card 2 sends the image 2' to the display card 1 in the t3-(t3+t5) time period, and controls the capture card 1 to send the image 2 to the display card 1 in the (t3+t5)-(t3+t5+t2) time period, and controls the Capture card 1 sends image 1 to display card 1 in the time period (t3+t5+t2)-(t3+t5+t2+t1), and controls capture card 2 to display card 1 at (t3+t5+t2+t1) - (t3+t5+t2+t1+t6) time period to send image 3', control capture card 2 to display card 1 at (t3+t5+t2+t1+t6)-
  • the above solution enables multiple capture cards to send images to be transmitted to the same display card in staggered peaks, improves the orderliness and transmission speed of images to be transmitted to the display card, and achieves fast images on the premise of ensuring image quality. transmission, and smooth playback.
  • the technical solution of the embodiment of the present application is to segment the theoretical time corresponding to the transmission of one frame of the to-be-transmitted image according to the transmission time required to transmit one frame of the to-be-transmitted image in each channel, and control the multi-frame to-be-transmitted image in the multiple channels.
  • the transmitted images are transmitted in an orderly manner in different time periods. On the premise of ensuring the image quality, the orderly and fast transmission of high-resolution images is realized.
  • FIG. 3 is a schematic flowchart of another image transmission method provided by an embodiment of the present application.
  • the embodiment of the present application is an optional solution proposed on the basis of the foregoing embodiment.
  • the image transmission method provided by the embodiment of the present application includes:
  • S210 Determine the transmission time required by the transmitting end to transmit one frame of the image to be transmitted in each channel according to the data amount of the image to be transmitted in each channel of the transmitting end and the transmission bandwidth between the transmitting end and the receiving end.
  • the theoretical time corresponding to the transmission of one frame of the image to be transmitted is segmented, and the corresponding transmission time is obtained.
  • the time period includes: if the transmission time required to transmit one frame of the image to be transmitted in the multiple channels is the same, then according to the number of channels of the sender and/or the theoretical time corresponding to the transmission of one frame of the image to be transmitted and the transmission time
  • a preset number is determined by the ratio of the preset number; according to the preset number, the theoretical time corresponding to the transmission of one frame of the image to be transmitted is averagely segmented to obtain a preset number of time periods.
  • Determining the preset number according to the number of channels at the sending end and/or the ratio of the theoretical time corresponding to the transmission of one frame of the image to be transmitted to the transmission time includes: setting the preset number equal to the number of channels at the sending end; or, The preset number is set to be equal to the ratio of the theoretical time corresponding to the transmission of one frame of the image to be transmitted to the transmission time; or, the preset number is set to be between the number of channels at the sending end and the ratio any value of .
  • capture card 1 and capture card n respectively capture and send 4 channels of images to be transmitted in YUV422 format with a resolution of 1920*1200 to display card 1 and video card n, and each capture card has 4 channels of images to be transmitted.
  • each display card has 4 channels of image input to be transmitted. Therefore, as long as it is ensured that the transmission of 4 frames of images to be transmitted corresponding to the 4 channels is completed within the theoretical time corresponding to the transmission of one frame of images to be transmitted, the theoretical time corresponding to the transmission of one frame of images to be transmitted can be divided into 4 segments on average.
  • the PCIE bandwidth between the capture card and the display card is X2.
  • the actual effective bandwidth is 6.6Gbps, that is, the bandwidth threshold between the sender and the receiver is 6.6Gbps.
  • the theoretical time corresponding to the image to be transmitted is divided into 6 segments equally to meet the transmission time requirement of each image to be transmitted.
  • the preset number can also be set between 4 and 6, for example, the preset number is 5, and the theoretical time corresponding to transmitting one frame of images to be transmitted is equally divided into 5 segments.
  • the transmission time the theoretical time corresponding to the transmission of one frame of the image to be transmitted is segmented, each time segment actually corresponds to the transmission time, and the transmission time is required for transmitting one frame of the image to be transmitted.
  • Each transmission time corresponds to the image to be transmitted, therefore, the image to be transmitted corresponding to each time period can be determined.
  • t1 is the time required to transmit image 1
  • t2 is the time required to transmit image 2
  • t3 is the time required to transmit image 3
  • t4 is the time required to transmit image 1'
  • t5 is the time required to transmit image 2'
  • a corresponding relationship is established, so that the sending end sends the image to be transmitted according to the corresponding relationship.
  • the process of establishing the corresponding relationship can be performed before all images start to be transmitted, or during the image transmission process, when it is detected that the capture card exists and needs to send a new image to be transmitted to the display card, the step of establishing the corresponding relationship can be performed again.
  • the above scheme establishes a corresponding relationship in advance, so that the capture card can perform image transmission in an orderly manner according to the corresponding relationship, and the image transmission efficiency is improved without changing the hardware structure while ensuring the image quality.
  • establishing the corresponding relationship between the time period, the image to be transmitted, the sending end and the receiving end of the image to be transmitted including: starting from the start time Start traversing multiple time periods to determine whether the receiving end is assigned to establish a corresponding relationship in the current time period; if the receiving end is not assigned to establish a corresponding relationship in the current time period, find the sender that has not been assigned to establish a corresponding relationship in the current time period. , and establish the corresponding relationship between the current time period, the sending end, the image to be transmitted to be allocated by the sending end, and the receiving end.
  • the transmission times corresponding to multiple time periods may be different or may be the same. If at least two frames of images to be transmitted require the same transmission time during transmission, there may be at least two frames of images to be transmitted corresponding to them in the same time period.
  • a corresponding relationship may also be established according to the situation in which the sending end and the receiving end are allocated in each time period. Exemplarily, in the 0-T1 time period, corresponding to the image to be transmitted 1" and the image to be transmitted 2", the image to be transmitted 1" is to be sent by the capture card 1, and the image to be transmitted 2" is to be sent by the capture card 2.
  • the display card 1 is not assigned to establish a corresponding relationship, and the capture card 1 and the capture card 2 are not assigned to establish a corresponding relationship. , the corresponding relationship between the capture card 1 and the display card 1.
  • the time period 0-T1 of the display card 1 has been allocated to establish a corresponding relationship, and the time period T1-2T1 of the display card 1 has not been allocated to establish a corresponding relationship.
  • Display card 2 has not been allocated to establish a corresponding relationship in the 0-T1 time period, capture card 1 has been allocated to establish a corresponding relationship in the 0-T1 time period, and capture card 2 has not been allocated in the 0-T1 time period.
  • the assignment establishes a corresponding relationship, so the corresponding relationship between the 0-T1 time period, the capture card 2, the image to be transmitted 3" and the display card 2 is established.
  • the display card 2 is not assigned to establish a corresponding relationship, and the capture card 2 In the T1-2T1 time period, it has been allocated to establish a corresponding relationship, and the capture card 1 has not been allocated to establish a corresponding relationship in the T1-2T1 time period.
  • the T1-2T1 time period, capture card 1, image to be transmitted 4" and display card 2 are established. corresponding relationship. And so on, until all the images to be transmitted, the capture card sending the image to be transmitted and the corresponding display card have established a corresponding relationship, or the theoretical time corresponding to the transmission of one frame of the image to be transmitted has been allocated to establish a relationship between the capture card and the display card. Correspondence.
  • the above solution can accurately and efficiently establish the corresponding relationship between the time period, the image to be transmitted, the sending end and the receiving end, so that the capture card can send images in an orderly manner according to the corresponding relationship.
  • a synchronization pulse signal is sent to the sending end and the receiving end to instruct the sending end to start sending the image to be transmitted, and instruct the receiving end to synchronously start receiving the image to be transmitted.
  • time slice t 0 of the sender n corresponds to the time slice t 1 of the sender 1
  • the time slice t 2 corresponding to the receiver n, which leads to the disorder of image data transmission and the randomness of congestion. sex.
  • the main control board sends a synchronization pulse signal to the sender and the receiver with the theoretical time corresponding to the transmission of one frame of the image to be transmitted as a cycle.
  • All FPGAs in the current system synchronously start data transmission and reception after receiving the pulse signal, ensuring that the time slice is absolutely synchronized when the acquisition card and the display card send and receive each frame of image data.
  • S260 Control the sending end to send the to-be-transmitted image corresponding to the transmission time to the receiving end in a time period corresponding to the transmission time.
  • the capture card can perform image transmission in an orderly manner according to the corresponding relationship, thereby improving the image transmission efficiency while ensuring the image quality and without changing the hardware structure.
  • the image transmission method of the present application is implemented in a distributed application system of FPGA+PCIE.
  • the FPGA acts as the EP end to send or receive data, and the amount of data that can be received or sent per unit time is limited by the bandwidth of the PCIE.
  • a capture card is mounted on the FPGA to perform data transmission as a transmitter, and a display card is mounted on the FPGA to receive data as a receiver.
  • the acquisition card can collect multiple image data at the same time, and can also send multiple image data to the outside at the same time.
  • the theoretical time (T) corresponding to the transmission of one frame of image is cut into multiple time equal parts according to the size of one frame of transmitted image in each channel, and all time slices are uniformly managed and allocated.
  • the capture card it needs to allocate a time slice before sending image data, so that it can send image data in the time period corresponding to the time slice.
  • the display card passively receives data, and the acquisition card actively sends data. Therefore, at the data sending end, that is, the side of the capture card, it is only necessary to control the order of the time slices of the image data sent by the capture card within one frame to solve the peak bandwidth overload problem of the data received by the display card.
  • the process of allocating time slices is described by taking the process of allocating time slices for one frame of image data to be sent in one channel on a capture card as an example.
  • the method for judging whether there is congestion is to obtain the bandwidth value of the image data normally transmitted by the display card on the time slice numbered t x (that is, the existing bandwidth value), and add the bandwidth value of the image data to be sent by the capture card. If the sum of the two is greater than the actual effective bandwidth of PCIE (that is, the bandwidth threshold between the display card and the capture card), it is regarded as congestion; if the sum of the two is not greater than the actual effective bandwidth of PCIE, it is regarded as not congested. If it is congested, it will continue to traverse the next time slice of the display card.
  • the time slot is used.
  • the existing bandwidth value of the time slice plus the bandwidth value of the image data to be sent by the current capture card is taken as the new existing bandwidth value.
  • the next time slice of the capture card is traversed continuously. Through the above steps, the time slice of the image data currently to be sent by the capture card can be obtained.
  • This embodiment proposes a method for slicing time, and proposes a method for controlling the sending time of the image data sending end within the theoretical time corresponding to the transmission of one frame of image data according to the congestion situation of each time slice, which effectively alleviates the problems caused by concurrency.
  • the PCIE bandwidth is congested at the data receiving end. And this solution greatly improves the bandwidth utilization of PCIE while preserving the image quality.
  • the embodiments of the present application compare and illustrate the image transmission method and the image transmission method proposed in the present application.
  • acquisition card 1 and acquisition card n collect and send 4-channel YUV422 format and 1920*1200 image data to display card 1 and display card n respectively.
  • Each capture card has 4 channels of image data output, and each display card has 4 channels of image data input.
  • the PCIE bandwidth between the capture card and the display card is X2.
  • the actual effective bandwidth is 6.6Gbps, that is, the bandwidth threshold between the sender and the receiver is 6.6Gbps.
  • T the theoretical time corresponding to the transmission of one frame of image data
  • 6 frames of image data corresponding to 6 channels can be sent and received.
  • the image data sent by the acquisition card 1 are A, B, C, and D.
  • the images sent by the capture card n are A', B', C', D', of which the image data A, B, A', B' are sent to the display card n, and the image data C, D, C', D' are sent to Display card 1.
  • Capture card 1 and capture card n may send image data to the same display card at the same time, resulting in instantaneous PCIE bandwidth overload on the display card.
  • the data is back pressured to the capture card, and the capture card alternately sends image data to the display card.
  • the image data can be sent by the capture card at time t, but in fact, it needs to be sent at 2t time, and the transmission frame rate is reduced, causing the playback to freeze.
  • image data A and A' are sent at the same time, resulting in the back pressure of image data A and image data A' to the capture card, and the data packets of image data A and image data A' are sent alternately, occupying a total of (2/T )time.
  • image data B and image data B' are sent from time 2/T. At this time, the two image data will be preempted at time (2/T) ⁇ (3/T) of display card n. .
  • the result of image data B and image data B' sending preemption results in the back pressure of image data B and image data B' to the capture card, and the data packets of image data B and image data B' are sent alternately, occupying a total of ( 2/T) ⁇ (4/T).
  • the image data D and the image data D' should have been sent within the time T, but they could not be sent due to the peak bandwidth overload of the display card.
  • the image data will be sent in a single frame of time.
  • the transmission of D and image data D' is completed.
  • the instantaneous bandwidth of the display card is overloaded, that is, in the same time period, the total data volume of image data sent by multiple capture cards on the display card exceeds the actual effective bandwidth of PCIE, resulting in instantaneous bandwidth overload of the display card.
  • the back pressure of the capture card caused the playback to freeze.
  • one frame time (T) is divided into multiple time equal parts according to the size of one frame of transmission image in each channel, and the time slices are uniformly managed and allocated.
  • the actual time occupied by the sliced time slice t x during the transmission of one frame of image is the total data amount of the image data to be transmitted corresponding to the current time slice and the actual effective bandwidth of PCIE X (Gbps ) ratio value. If the total amount of data corresponding to the time slice t 0 is Q 0 , then the actual time used for the time slice t 0 is Q 0 /X.
  • T Q 0 /X+Q 1 /X+...+Q p /X.
  • the size of all transmitted images is 1920*1200
  • the PCIE bandwidth is x2.
  • the actual effective bandwidth X is 6.6 Gbps.
  • the actual time of each time slice changes according to the size of the transmitted image, and the default number of time slices is allocated before image transmission, but the number of effective slices during actual image transmission is less than or equal to Default number of time slices.
  • the number of effective slices during actual image transmission is less than or equal to Default number of time slices.
  • 32 time slices are allocated to it, but only 6 time slices are actually valid.
  • the effective time slice is also calculated in real time according to the data allocation, that is, the number of effective slices cannot be determined before starting the data allocation in practical applications. If the amount of data sent by the capture card becomes larger or smaller, the number of effective slices will also change accordingly.
  • the data transmission specification is defined in advance, so effective slice data is calculated to help understanding.
  • the image transmission method of the present application is as follows:
  • the theoretical time T corresponding to the transmission of one frame of image data is divided into multiple time slices, for example, 32 time slices, but since the size of the transmitted images is the same, the effective number of sliced time slices within the theoretical time T corresponding to the transmission of one frame of image data is 6.
  • the images sent by capture card 1 are A, B, C, and D
  • the images sent by capture card n are A', B', C', D', where images A, B, A', B' Sent to graphics card n, images C, D, C', D' are sent to graphics card 1.
  • the time slice is first allocated according to the time slice congestion condition of the display card n.
  • the time slice t 0 of the display card n is not congested and the time slice t 0 of the capture card n is not occupied, and the sending time slice t 0 is allocated to it ;
  • the image A of the capture card 1 is sent to the display card n , the time slice t 0 of the display card n is congested, the time slice t 1 of the display card n is not congested, and the time slice t 1 of the acquisition card 1 is not occupied, and a time slice t 1 is allocated to it.
  • the time slice is first allocated according to the time slice congestion of the display card 1.
  • the time slice t 0 of the display card 1 is not congested, but the time slice t 0 of the capture card n has been occupied. At this time, the time slice t 0 of the capture card 1 is not occupied.
  • the time slice t 0 of the display card 1 and the time slice t 0 of the capture card 1 are allocated to it; when the image C' of the capture card n is continuously allocated, the time slice t 1 of the display card 1 is not congested and the capture The time slot t 1 of card n is not occupied, and the time slot t 1 is allocated to it.
  • the time slice is first allocated according to the time slice congestion of the display card 1.
  • the time slice t 2 of the display card 1 is not congested, but the time slice t 2 of the capture card n has been occupied. At this time, the time slice t 2 of the capture card 1 is not occupied.
  • the time slice t2 and the time slice t2 of the capture card 1 are allocated to the image D to be allocated by the capture card 1; when the image D' of the capture card n continues to be allocated, the time slice t3 of the display card 1 is not congested and the capture card n
  • the time slot t 3 is not occupied, and is allocated a transmission time slot t 3 .
  • the image data transmitted in the corresponding time slice is shown in Figure 18.
  • the actual number of slices of the capture card and the display card is 4.
  • the actual time of each time slice is calculated according to the total data volume of the image data in the time slice.
  • the actual time of the time slice t 0 of the capture card 1 is: the data amount of the image C divided by the actual effective bandwidth of the current PCIE.
  • the left side of FIG. 19 is a schematic diagram of the instantaneous bandwidth of data transmission in the related art
  • the right side of FIG. 19 is a schematic diagram of the instantaneous bandwidth of data transmission in the present application.
  • the image transmission method of the present application effectively alleviates the problem of image jamming caused by instantaneous bandwidth overload of PCIE in the left figure of Figure 19 under the condition that the total amount of data of one frame of image data remains unchanged, and effectively improves the utilization rate of PCIE bandwidth. .
  • FIG. 20 is a schematic structural diagram of an image transmission apparatus provided by an embodiment of the present application.
  • an embodiment of the present application discloses an image transmission apparatus 300 .
  • the apparatus 300 includes a transmission time determination module 301 , a segmentation module 302 and a control module 303 .
  • the transmission time determination module 301 is configured to determine that the transmitting end transmits one frame of the image to be transmitted in each channel according to the data amount of the image to be transmitted in each channel of the transmitting end and the transmission bandwidth between the transmitting end and the receiving end
  • the segmentation module 302 is set to segment the theoretical time corresponding to the transmission of one frame of the to-be-transmitted image according to the transmission time required by the transmitting end to transmit one frame of the to-be-transmitted image in each channel, to obtain The time period corresponding to the transmission time;
  • the control module 303 is configured to control the sending end to send the to-be-transmitted image corresponding to the transmission time to the receiving end in the time period corresponding to the transmission time.
  • the segmentation module 302 includes:
  • the preset number determination unit is set to be the same as the transmission time required to transmit a frame of images to be transmitted in the multiple channels respectively, then according to the number of channels of the transmitting end and/or the theoretical time corresponding to transmitting a frame of images to be transmitted is the same as the total transmission time.
  • the ratio of the transmission time determines a preset number; the time segment segmentation unit is set to averagely segment the theoretical time corresponding to transmitting a frame of images to be transmitted according to the preset number to obtain a preset number of time segments.
  • the preset number determination unit is set to:
  • the preset number is set equal to the number of channels of the sending end; or, the preset number is set equal to the ratio of the theoretical time corresponding to the transmission of one frame of the image to be transmitted to the transmission time; The number is set to any value between the number of channels at the sending end and the ratio.
  • the segmentation module 302 includes:
  • the alternate segmentation unit is set to alternately segment the theoretical time corresponding to the transmission of one frame of the image to be transmitted according to the transmission time required for transmitting one frame of the image to be transmitted corresponding to at least two transmitting ends, and obtain the time period corresponding to the transmission time .
  • the device further includes:
  • the image-to-be-transmitted determining module is configured to determine the images to be transmitted that are transmitted within the transmission time corresponding to each time period; the correspondence establishing module is configured to establish the time period, the image to be transmitted, and the transmission of the image to be transmitted The corresponding relationship between the terminal and the receiving terminal, so that the transmitting terminal sends the image to be transmitted according to the corresponding relationship.
  • the corresponding relationship establishing module includes:
  • the traversing unit is set to traverse a plurality of time periods from the start time, and determines whether the receiving end is allocated to establish a corresponding relationship in the current time period; the relationship establishing unit is set to if the receiving end is not allocated in the current time period.
  • search for the sending end that is not assigned to establish the corresponding relationship in the current time period, and establish the corresponding relationship between the current time period, the sending end, the image to be transmitted to be allocated by the sending end, and the receiving end.
  • control module 303 is set to:
  • a synchronization pulse signal is sent to the transmitting end and the receiving end to instruct the transmitting end to start the transmission of the to-be-transmitted image, and to instruct the receiving end to start synchronously The reception of the image to be transmitted.
  • the image transmission apparatus provided by the embodiment of the present application can execute the image transmission method provided by any embodiment of the present application, and has functional modules and effects corresponding to the execution method.
  • the present application further provides an electronic device and a readable storage medium.
  • FIG. 21 is a block diagram of an electronic device for implementing an image transmission method provided by an embodiment of the present application.
  • Electronic devices are intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers.
  • Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices.
  • the components shown herein, their connections and relationships, and their functions are by way of example only, and are not intended to limit implementations of the application described and/or claimed herein.
  • the electronic device includes: one or more processors 401, a memory 402, and an interface configured to connect a plurality of components, including a high-speed interface and a low-speed interface. Multiple components are interconnected using different buses, and may be mounted on a common motherboard or otherwise as desired.
  • the processor may process instructions for execution within the electronic device, including storing in or on memory to display a Graphical User Interface (GUI) on an external input/output device such as a display device coupled to the interface ) instructions for graphics information.
  • GUI Graphical User Interface
  • multiple processors and/or multiple buses may be used with multiple memories and multiple memories, if desired.
  • multiple electronic devices may be connected, with multiple devices providing some of the necessary operations (eg, as a server array, a group of blade servers, or a multiprocessor system).
  • a processor 401 is used as an example.
  • the memory 402 is the non-transitory computer-readable storage medium provided by the present application.
  • the memory stores instructions executable by at least one processor, so that the at least one processor executes the image transmission method provided by the present application.
  • the non-transitory computer-readable storage medium of the present application stores computer instructions for causing the computer to execute the image transmission method provided by the present application.
  • the memory 402 can be configured to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions/modules (for example, program instructions/modules corresponding to the image transmission method in the embodiments of the present application). , the transmission time determination module 301, the segmentation module 302 and the control module 303 shown in Figure 20).
  • the processor 401 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions and modules stored in the memory 402, ie, implements the image transmission method in the above method embodiments.
  • the memory 402 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the electronic device for image transmission, and the like. Additionally, memory 402 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 402 may optionally include memory located remotely from processor 401, which may be connected to the image transmission electronics via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
  • the electronic device of the image transmission method may further include: an input device 403 and an output device 404 .
  • the processor 401, the memory 402, the input device 403, and the output device 404 may be connected by a bus or in other ways, and the connection by a bus is taken as an example in FIG. 21 .
  • the input device 403 can receive input numerical or character information, and generate key signal input related to user settings and function control of the electronic device for image transmission, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, a pointing stick, a Or multiple input devices such as mouse buttons, trackballs, joysticks, etc.
  • the output device 404 may include a display device, an auxiliary lighting device (eg, Light Emitting Diode (LED)), a haptic feedback device (eg, a vibration motor), and the like.
  • the display device may include, but is not limited to, a liquid crystal display (LCD), an LED display, and a plasma display. In some implementations, the display device may be a touch screen.
  • Various implementations of the systems and techniques described herein can be implemented in digital electronic circuitry, integrated circuit systems, application specific integrated circuits (ASICs), computer hardware, firmware, software, and/or combinations thereof .
  • ASICs application specific integrated circuits
  • These various embodiments may include implementation in one or more computer programs executable and/or interpretable on a programmable system including at least one programmable processor that The processor, which may be a special purpose or general-purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device an output device.
  • machine-readable medium and “computer-readable medium” refer to any computer program product, apparatus, and/or apparatus arranged to provide machine instructions and/or data to a programmable processor (for example, a magnetic disk, an optical disk, a memory, a Programmable Logic Device (PLD)), including a machine-readable medium that receives machine instructions as a machine-readable signal.
  • machine-readable signal refers to any signal used to provide machine instructions and/or data to a programmable processor.
  • the systems and techniques described herein may be implemented on a computer having a display device (eg, a cathode ray tube (CRT) or an LCD monitor) configured to display information to the user ); and a keyboard and pointing device (eg, a mouse or trackball) through which a user can provide input to the computer.
  • a display device eg, a cathode ray tube (CRT) or an LCD monitor
  • a keyboard and pointing device eg, a mouse or trackball
  • Other kinds of devices may also be configured to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (eg, visual feedback, auditory feedback, or tactile feedback); and may be in any form (including acoustic input, voice input, or tactile input) to receive input from the user.
  • the systems and techniques described herein may be implemented on a computing system that includes back-end components (eg, as a data server), or a computing system that includes middleware components (eg, an application server), or a computing system that includes front-end components (eg, a user's computer having a graphical user interface or web browser through which a user may interact with implementations of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system.
  • the components of the system may be interconnected by any form or medium of digital data communication (eg, a communication network). Examples of communication networks include: Local Area Network (LAN), Wide Area Network (WAN), blockchain network, and the Internet.
  • a computer system can include clients and servers.
  • Clients and servers are generally remote from each other and usually interact through a communication network.
  • the relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other.
  • Steps can be reordered, added, or removed using the various forms of flow shown above.
  • steps described in the present application may be performed in parallel, sequentially or in different orders, and as long as the desired results of the technical solutions disclosed in the present application can be achieved, no limitation is imposed herein.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Human Computer Interaction (AREA)
  • Computer Graphics (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Computer Security & Cryptography (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本文公开了一种图像传输方法、装置、设备和介质。该图像传输方法包括：根据发送端的每路通道中的一帧待传输图像的数据量，以及所述发送端和接收端之间的传输带宽，确定所述发送端传输每路通道中一帧待传输图像所需的传输时间；根据所述发送端传输每路通道中一帧待传输图像所需的传输时间，对传输一帧待传输图像对应的理论时间进行分段，得到所述传输时间对应的时间段；控制所述发送端在所述传输时间对应的时间段中，向所述接收端发送所述传输时间对应的待传输图像。

Description

图像传输方法、装置、设备和介质 技术领域
本申请涉及图像传输领域，例如涉及一种图像传输方法、装置、设备和介质。
背景技术
随着科技的快速更迭发展，用户对图像显示清晰度的要求逐渐提高，逐渐追求超高清图像显示画质。超高清图像显示意味着在保证图像显示流畅度的同时，提高图像分辨率，也即在单位时间内传输的数据量将大大增加。
在图像采集、处理以及显示技术领域，现场可编程门阵列(Field Programmable Gate Array，FPGA)由于其功能强大、灵活及成本经济等优点被逐渐应用在此技术领域。在FPGA+高速串行计算机扩展总线标准(Peripheral Component Interconnect Express，PCIE)的分布式应用场景中，FPGA作为端点(Endpoin，EP)进行数据的发送或接收，其单位时间能接收或发送的数据量常受到PCIE带宽的限制。在多个采集卡对应一个显示卡或者多个采集卡对应多个显示卡进行图像采集以及发送过程中，由于采集卡端FPGA单位时间内传输的数据量增大，常造成显示卡端PCIE数据报文瞬时拥塞，进而引起图像播放卡顿。
为解决以上因数据报文顺势拥塞造成的图像卡顿，常采用的技术手段主要有三种，从硬件上扩大带宽、对图像进行压缩传输或在发生拥塞后停止传输。从硬件上扩大带宽的方案，虽解决了图像卡顿问题，但增加其硬件成本，降低PCIE的带宽使用率。对图像压缩传输的方案，会导致图像质量降低。拥塞后停止图像传输的方案，降低了系统资源利用率。
发明内容
本申请提供一种图像传输方法、装置、设备和介质，以实现高分辨率图像的有序快速传输和显示。
提供了一种图像传输方法，包括：
根据发送端的每路通道中的一帧待传输图像的数据量，以及所述发送端和接收端之间的传输带宽，确定所述发送端传输每路通道中一帧待传输图像所需的传输时间；
根据所述发送端传输每路通道中一帧待传输图像所需的传输时间，对传输 一帧待传输图像对应的理论时间进行分段，得到所述传输时间对应的时间段；
控制所述发送端在所述传输时间对应的时间段中，向所述接收端发送所述传输时间对应的待传输图像。
还提供了一种图像传输装置，包括：
传输时间确定模块，设置为根据发送端的每路通道中的一帧待传输图像的数据量，以及所述发送端和接收端之间的传输带宽，确定所述发送端传输每路通道中一帧待传输图像所需的传输时间；
分段模块，设置为根据所述发送端传输每路通道中一帧待传输图像所需的传输时间，对传输一帧待传输图像对应的理论时间进行分段，得到所述传输时间对应的时间段；
控制模块，设置为控制所述发送端在所述传输时间对应的时间段中，向所述接收端发送所述传输时间对应的待传输图像。
还公开了一种电子设备，包括：
至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，
所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行如上所述的图像传输方法。
还公开了一种存储有计算机指令的非瞬时计算机可读存储介质，所述计算机指令用于使所述计算机执行如上所述的图像传输方法。
附图说明
图1是本申请实施例提供的一种图像传输方法的流程示意图；
图2是本申请实施例提供的一种FPGA+PCIE分布式应用系统示意图；
图3是本申请实施例提供的另一种图像传输方法的流程示意图；
图4是本申请实施例提供的一种发送端和接收端时间片错位的示意图；
图5是本申请实施例提供的一种同步脉冲信号发送示意图；
图6是本申请实施例提供的一种图像传输方法实现的流程图；
图7是本申请实施例提供的一种FPGA+PCIE的分布式应用系统的图像数据理论流向示意图；
图8是本申请实施例提供的一种分配图像数据A和A＇的示意图；
图9是本申请实施例提供的一种分配图像数据A和A＇发生发送抢占的示意图；
图10是本申请实施例提供的一种分配图像数据B和B＇的示意图；
图11是本申请实施例提供的一种分配图像数据B和B＇发生发送抢占的示意图；
图12是本申请实施例提供的一种分配图像数据的抢占结果示意图；
图13是本申请实施例提供的一种显示卡瞬时带宽过载的示意图；
图14是本申请实施例提供的一种FPGA+PCIE的分布式应用系统的图像数据计划流向示意图；
图15是本申请实施例提供的一种图像数据A和A＇分配时间片的示意图；
图16是本申请实施例提供的一种图像数据B和B＇分配时间片的示意图；
图17是本申请实施例提供的一种图像数据C和C＇分配时间片的示意图；
图18是本申请实施例提供的一种图像数据D和D＇分配时间片的示意图；
图19是本申请实施例提供的一种图像传输中瞬时带宽的对比示意图；
图20是本申请实施例提供的一种图像传输装置的结构示意图；
图21是本申请实施例提供的一种用来实现图像传输方法的电子设备的框图。
具体实施方式
以下结合附图对本申请的示范性实施例做出说明，其中包括本申请实施例的多种细节以助于理解，应当将它们认为仅仅是示范性的。
在本申请实施例中，发送端可以为图像数据流出一侧的节点，接收端可以为图像数据流入一侧的节点。对于作为发送端和接收端的设备而言并不是唯一的，例如发送端可以为采集卡、服务器、单片机等，接收端可以为显示卡、存储器等。发送端和接收端之间的数据传输协议或中间设备根据作为发送端和接收端的设备而定，例如若发送端为采集卡，接收端为显示卡，则可以采用PCIE总线作为数据传输协议。
图1是本申请实施例提供的一种图像传输方法的流程示意图。本实施例可适用于发送端向接收端传输图像的情况。例如，本实施例可以适用于多个发送端向一个接收端，或者多个发送端向多个接收端发送图像的情况。本实施例公开的图像传输方法可以由一种图像传输装置来执行，该装置可以由软件和/或硬件的方式实现，可以集成于电子设备中。参见图1，本实施例提供的图像传输方 法包括：
S110、根据发送端的每路通道中的一帧待传输图像的数据量，以及发送端和接收端之间的传输带宽，确定发送端传输每路通道中的一帧待传输图像所需的传输时间。
待传输图像为发送端采集并需要向接收端发送的图像。图2是本申请实施例提供的一种FPGA+PCIE分布式应用系统示意图。如图2所示，系统中可以包括主控板、多个采集卡和多个显示卡。其中，对主控板的结构和形式不做限制，为能够对采集卡和显示卡进行管理控制的器件即可。多个采集卡可以分别向任一显示卡发送多路图像，即显示卡可以接收多个采集卡发送的多路图像。图2中只显示传输4路图像，并不是对传输图像数量的限制，实际可以传输任意路图像。在该系统中，多个采集卡同时向一个显示卡发送图像时，显示卡瞬时带宽可能会过载，图像无法正常接收，反压到采集卡上，再进行交替发送，导致传输帧率降低，显示卡在进行显示播放时出现卡顿的现象。
本申请实施例由主控板对采集卡的图像发送过程进行管理和控制，以使多个采集卡在向同一个显示卡发送图像时有序发送，提高图像发送的效率。本申请实施例为了明确每个采集卡传输图像所用的时间，以对每个采集卡进行管理和控制，预先确定需要向一显示卡上发送图像的所有采集卡，以及每个采集卡需要向该显示卡发送的待传输图像的数据量。根据待传输图像的数据量和传输带宽，可以计算传输每路通道中的一帧待传输图像所需的传输时间。传输带宽为EP端之间的PCIE传输带宽。一实施例中，一帧待传输图像的数据量与传输带宽的比值，即为传输每路通道中的一帧待传输图像所需的传输时间。
上述方案预先确定传输每路通道中的一帧待传输图像所需的传输时间，能够对多个采集卡传输待传输图像所需的传输时间进行全面的整体掌控，以根据传输待传输图像的传输时间对采集卡的图像发送过程进行有效管控。
S120、根据发送端传输每路通道中一帧待传输图像所需的传输时间，对传输一帧待传输图像对应的理论时间进行分段，得到所述传输时间对应的时间段。
每个采集卡的每帧待传输图像，都有对应的传输时间，为传输该帧待传输图像所需的时间。为了解决多个采集卡向一个显示卡上同时发送图像时导致显示卡上拥堵的问题，本申请实施例中针对一个显示卡，确定需要向该显示卡上发送图像的全部采集卡，进而确定需要向该显示卡上发送的全部待传输图像。确定全部待传输图像对应的传输时间，进而根据传输时间对传输一帧待传输图像对应的理论时间进行分段。
示例性的，采集卡1需要向显示卡1上发送的待传输图像为图像1、图像2、 图像3，传输图像1所需的传输时间为t1，传输图像2所需的传输时间为t2，传输图像3所需的传输时间为t3。采集卡2需要向显示卡1上发送的待传输图像为图像1’、图像2’、图像3’，传输图像1’所需的传输时间为t4，传输图像2’所需的传输时间为t5，传输图像3’所需的传输时间为t6。则可以将传输一帧待传输图像对应的时间分成6段，分别对应t1、t2、t3、t4、t5和t6。也就是，将传输一帧待传输图像对应的理论时间分为0-t1、t1-(t1+t2)、(t1+t2)-(t1+t2+t3)、(t1+t2+t3)-(t1+t2+t3+t4)、(t1+t2+t3+t4)-(t1+t2+t3+t4+t5)、(t1+t2+t3+t4+t5)-(t1+t2+t3+t4+t5+t6)6段。
在本申请实施例中，对传输一帧待传输图像对应的理论时间进行分段得到的时间段的顺序不作限定，例如上述举例中，6个时间段分别对应t1、t2、t3、t4、t5、t6。也可以令分段得到的6个时间段分别对应t3、t5、t2、t1、t6、t4，或是对应其他顺序的传输时间，在此不作限定。
上述方案根据传输每路通道中的一帧待传输图像所需的时间对传输一帧待传输图像对应的理论时间进行分段，从而使多帧待传输图像在时间段上进行错位，以进行有序管理和控制。
S130、控制所述发送端在所述传输时间对应的时间段中，向所述接收端发送所述传输时间对应的待传输图像。
在本申请实施例中，可以在即将到达每个采集卡发送待传输图像的时间段时，向采集卡下发控制指令，控制采集卡在对应的时间段发送待传输数据，也可以预先向所有采集卡发送控制指令，控制采集卡在到达对应的时间段时发送传输数据。控制指令也可以为采集卡向主控板主动获取的。
示例性，以S120中的举例说明。如果将传输一帧待传输图像对应的理论时间分成6段，分别对应t1、t2、t3、t4、t5和t6，则控制采集卡1向显示卡1在0-t1时间段发送图像1，控制采集卡1向显示卡1在t1-(t1+t2)时间段发送图像2，控制采集卡1向显示卡1在(t1+t2)-(t1+t2+t3)时间段发送图像3，控制采集卡2向显示卡1在(t1+t2+t3)-(t1+t2+t3+t4)时间段发送图像1’，控制采集卡2向显示卡1在(t1+t2+t3+t4)-(t1+t2+t3+t4+t5)时间段发送图像2’，控制采集卡2向显示卡1在(t1+t2+t3+t4+t5)-(t1+t2+t3+t4+t5+t6)时间段发送图像3’，由此实现多个采集卡向一个显示卡进行图像有序快速地传输。
如果将传输一帧待传输图像对应的理论时间分成6段，分别对应t3、t5、t2、t1、t6、t4，控制采集卡1向显示卡1在0-t3时间段发送图像3，控制采集卡2向显示卡1在t3-(t3+t5)时间段发送图像2’，控制采集卡1向显示卡1在(t3+t5)-(t3+t5+t2)时间段发送图像2，控制采集卡1向显示卡1在(t3+t5+t2)-(t3+t5+t2+t1)时间段发送图像1，控制采集卡2向显示卡1在(t3+t5+t2+t1) -(t3+t5+t2+t1+t6)时间段发送图像3’，控制采集卡2向显示卡1在(t3+t5+t2+t1+t6)-(t3+t5+t2+t1+t6+t4)时间段发送图像1’。以上只是举例，还可以为其他情况，可以根据实际情况确定分段的顺序以及图像传输的顺序。
上述方案使多个采集卡向同一个显示卡发送待传输图像时能够错峰发送，提高待传输图像向显示卡发送的有序性和传输速度，在保证图像质量的前提下，实现图像的快速传输，以及流畅播放。
本申请实施例的技术方案，根据传输每路通道中的一帧待传输图像所需的传输时间，对传输一帧待传输图像对应的理论时间进行分段，控制多路通道中的多帧待传输图像在不同的时间段进行有序传输，在保证图像质量的前提下，实现了高分辨率图像的有序快速传输。
图3是本申请实施例提供的另一种图像传输方法的流程示意图。本申请实施例是在上述实施例的基础上提出的一种可选方案。参见图3，本申请实施例提供的图像传输方法包括：
S210、根据发送端的每路通道中的一帧待传输图像的数据量，以及发送端和接收端之间的传输带宽，确定发送端传输每路通道中一帧待传输图像所需的传输时间。
S220、根据所述发送端传输每路通道中一帧待传输图像所需的传输时间，对传输一帧待传输图像对应的理论时间进行分段，得到所述传输时间对应的时间段。
在本申请实施例中，根据所述发送端传输每路通道中一帧待传输图像所需的传输时间，对传输一帧待传输图像对应的理论时间进行分段，得到所述传输时间对应的时间段，包括：若分别传输多路通道中的一帧待传输图像所需的传输时间相同，则根据发送端的通道路数和/或传输一帧待传输图像对应的理论时间与所述传输时间的比值确定预设数量；依据预设数量对传输一帧待传输图像对应的理论时间进行平均分段，得到预设数量个时间段。根据发送端的通道路数和/或传输一帧待传输图像对应的理论时间与所述传输时间的比值确定预设数量，包括：将所述预设数量设置为等于发送端的通道路数；或者，将所述预设数量设置为等于传输一帧待传输图像对应的理论时间与所述传输时间的比值；或者，将所述预设数量设置为介于发送端的通道路数与所述比值之间的任一数值。
示例性的，假设采集卡1和采集卡n分别采集和发送4路YUV422格式、分辨率为1920*1200的待传输图像到显示卡1和显示卡n，每块采集卡有4路待 传输图像输出，每块显示卡有4路待传输图像输入。因此只要保证在传输一帧待传输图像对应的理论时间内完成4路通道分别对应的4帧待传输图像的传输即可，可以将传输一帧待传输图像对应的理论时间平均分为4段。采集卡和显示卡之间的PCIE带宽为X2，根据PCIE带宽利用率可知其实际有效带宽为6.6Gbps，即发送端和接收端之间的带宽阈值为6.6Gbps。图像数据按照30帧/秒的速率在采集卡和显示卡之间传输，可计算出一路待传输图像所占带宽为1920*1200*16*30/1000/1000/1000＝1.1Gbps。针对采集卡和显示卡而言，传输一帧待传输图像对应的理论时间(T＝1/30s)之内就可以发送和接收6路通道分别对应的6帧待传输图像，可以将传输一帧待传输图像对应的理论时间平均分成6段，以满足每路待传输图像的传输时间需求。也可以令预设数量介于4和6之间，例如预设数量为5，将传输一帧待传输图像对应的理论时间平均分成5段。
S230、确定每个时间段对应的传输时间内传输的待传输图像。
示例性的，根据所述传输时间对传输一帧待传输图像对应的理论时间进行分段，每个时间分段实际上与传输时间是相对应的，传输时间为传输一帧待传输图像所需的时间，每个传输时间又和待传输图像相对应，因此，可以确定每个时间段对应的待传输图像。
如果将传输一帧待传输图像对应的理论时间分成6段，即0-t1、t1-(t1+t2)、(t1+t2)-(t1+t2+t3)、(t1+t2+t3)-(t1+t2+t3+t4)、(t1+t2+t3+t4)-(t1+t2+t3+t4+t5)、(t1+t2+t3+t4+t5)-(t1+t2+t3+t4+t5+t6)6段，分别对应t1、t2、t3、t4、t5和t6，t1为传输图像1所需的时间，与图像1对应，t2为传输图像2所需的时间，与图像2对应，t3为传输图像3所需的时间，与图像3对应，t4为传输图像1’所需的时间，与图像1’对应，t5为传输图像2’所需的时间，与图像2’对应，t6为传输图像3’所需的时间，与图像3’对应。由此，以传输时间为连接，确定每个时间段对应的待传输图像。
S240、建立所述时间段、所述待传输图像、所述待传输图像的发送端和接收端的对应关系。
在本申请实施例中，建立对应关系，以使所述发送端根据所述对应关系发送待传输图像。建立对应关系的过程可以在全部图像开始传输之前执行，也可以在图像传输过程中，当检测到存在采集卡需要向显示卡发送新的待传输图像时，再次执行对应关系建立的步骤。
对上述举例进行说明。建立时间段：0-t1、待传输图像：图像1、发送端：采集卡1、接收端：显示卡1的对应关系，以使采集卡1根据对应关系，在时间段0-t1内向显示卡1发送图像1。建立时间段：t1-(t1+t2)、待传输图像：图像2、发送端：采集卡1、接收端：显示卡1的对应关系，以使采集卡1根据对 应关系，在时间段t1-(t1+t2)内向显示卡1发送图像2，以此类推，建立每个时间段、对应的待传输图像、待传输图像的发送端和接收端的对应关系，以指示采集卡有序发送待传输图像。
上述方案通过预先建立对应关系，从而使采集卡根据对应关系进行有序的图像传输，在保证图像质量，以及不改变硬件结构的情况下，提高了图像的传输效率。
在本申请实施例中，若所述发送端为至少两个，则建立所述时间段、所述待传输图像、所述待传输图像的发送端和接收端的对应关系，包括：从起始时刻开始遍历多个时间段，确定接收端在当前时间段是否被分配建立对应关系；若接收端在当前时间段未被分配建立对应关系，则查找在当前时间段未被分配建立对应关系的发送端，建立当前时间段、所述发送端、所述发送端待分配的待传输图像以及所述接收端的对应关系。
示例性的，多个时间段对应的传输时间可能不同也可能相同，如果至少两帧待传输图像在传输时所需的传输时间相同，同一时间段可能存在与其对应的至少两帧待传输图像。在本申请实施例中，还可以根据每个时间段中发送端与接收端被分配的情况，建立对应关系。示例性的，在0-T1时间段，对应待传输图像1”和待传输图像2”，待传输图像1”为采集卡1待发送的，待传输图像2”为采集卡2待发送的。在0-T1时间段，显示卡1未被分配建立对应关系，采集卡1和采集卡2也未被分配建立对应关系，可以选择采集卡1，建立0-T1时间段、待传输图像1”、采集卡1和显示卡1的对应关系。显示卡1的0-T1时间段已经被分配建立对应关系，显示卡1的T1-2T1时间段，未被分配建立对应关系，因此，建立T1-2T1时间段、待传输图像2”、采集卡2和显示卡1的对应关系。
对于显示卡2同理，显示卡2在0-T1时间段未被分配建立对应关系，采集卡1在0-T1时间段已被分配建立对应关系，采集卡2在0-T1时间段未被分配建立对应关系，因此建立0-T1时间段、采集卡2、待传输图像3”和显示卡2的对应关系。在T1-2T1时间段，显示卡2未被分配建立对应关系，采集卡2在T1-2T1时间段已被分配建立对应关系，采集卡1在T1-2T1时间段未被分配建立对应关系，因此建立T1-2T1时间段、采集卡1、待传输图像4”和显示卡2的对应关系。以此类推，直到全部待传输图像、发送该待传输图像的采集卡与对应的显示卡均建立对应关系，或者传输一帧待传输图像对应的理论时间均被分配建立与采集卡和显示卡的对应关系。
上述方案能够准确高效的建立时间段、待传输图像、发送端与接收端之间的对应关系，从而使采集卡依据对应关系进行有序地图像发送。
S250、以传输一帧待传输图像对应的理论时间为周期，向所述发送端和所 述接收端发送同步脉冲信号。
示例性的，通过向所述发送端和所述接收端发送同步脉冲信号，以指示所述发送端启动待传输图像的发送，以及指示所述接收端同步启动待传输图像的接收。
由于不同的发送端和接收端之间开始数据发送或接收的时间点不一定是统一的，因此就有可能导致多个设备之间的时间片的不对应的情况。如图4所示，发送端n的时间片t 0对应发送端1的时间片t 1，其对应接收端n的时间片则为t 2，导致图像数据传输的无序性和存在拥塞的随机性。
如图5所示，为了实现采集卡与显示卡时间片的同步性，主控板以传输一帧待传输图像对应的理论时间为周期，向所述发送端和所述接收端发送同步脉冲信号。当前系统中所有FPGA收到该脉冲信号后同步启动数据的发送和接收，保证采集卡和显示卡发送和接收每一帧图像数据时时间片是保持绝对同步的。
S260、控制所述发送端在所述传输时间对应的时间段中，向所述接收端发送所述传输时间对应的待传输图像。
本申请实施例，通过预先建立对应关系，从而使采集卡根据对应关系进行有序的图像传输，在保证图像质量，以及不改变硬件结构的情况下，提高图像的传输效率。
本申请的图像传输方法在FPGA+PCIE的分布式应用系统中实施，FPGA作为EP端进行数据的发送或接收，其单位时间能接收或发送的数据量受到PCIE带宽的限制。本实施例搭载在FPGA上进行数据发送作为发送端的为采集卡，搭载在FPGA上进行数据接收作为接收端的为显示卡。其中采集卡可同时采集多路图像数据，也可同时向外发送多路图像数据。
本申请将传输一帧图像对应的理论时间(T)根据每路通道中的一帧传输图像的大小切割成多个时间等分，对所有时间片进行统一的管理分配。针对采集卡而言，在发送图像数据之前需要为其分配一个时间片，以使其在该时间片对应的时间段内发送图像数据。
在FPGA+PCIE的分布式应用系统中，显示卡为被动接收数据，采集卡主动发送数据。所以在数据发送端也就是采集卡一侧，只需控制采集卡的图像数据在一帧之内的发送时间片顺序即可达到解决显示卡接收数据出现峰值带宽过载问题。以对一张采集卡上的一路通道中的一帧待发送图像数据分配时间片的过程为例说明分配时间片的过程。
如图6所示，首先将依次判断时间片的标志位x(0≤x≤m)初始化为与初 始编号一致，由于本实施例中时间片的编号为t 0、t 1、t 2、…、t m-2、t m-1、t m，因此将x初始化为0。
判断显示卡在编号为t x的时间片内是否拥塞，该显示卡为接收采集卡待发送图像数据的显示卡。判断是否拥塞的方法为：获取显示卡在编号为t x的时间片上正常传输的图像数据的带宽值(即为已存在带宽值)，加上采集卡即将发送的图像数据的带宽值。如果两者之和大于PCIE的实际有效带宽(即显示卡和采集卡之间的带宽阈值)，则视为拥塞；如果两者之和不大于PCIE的实际有效带宽，则视为不拥塞。如果拥塞则继续遍历显示卡的下一个时间片。
如果将所有时间片都遍历完成都未找到不拥塞的时间片，则表示当前显示卡已经达到PCIE实际有效带宽，则停止本次分配操作。
如果找到时间片t x满足不拥塞的条件且在采集卡上的该时间片未被其他业务占用，则使用该时间片。并将该时间片的已存在带宽值加上当前采集卡即将发送的图像数据的带宽值作为新的已存在带宽值。
如果找到时间片t x满足不拥塞的条件但在采集卡上的该时间片被其他业务占用，则继续遍历采集卡的下一个时间片。通过以上步骤即可获取到采集卡当前待发送图像数据发送的时间片。
本实施例提出将时间切片的方法，并提出根据每个时间片的拥塞情况控制图像数据发送端在传输一帧图像数据对应的理论时间之内的发送时间的方法，有效的缓解了由于并发导致数据接收端出现PCIE带宽拥塞情况。且该方案在保留了图像质量的同时极大的提高了PCIE的带宽利用率。
本申请实施例以FPGA+PCIE的分布式应用系统为基础，对图像传输方法和本申请提出的图像传输方法进行对比说明。
如图7所示，FPGA+PCIE的分布式应用系统中，假设采集卡1和采集卡n分别采集和发送4路YUV422格式、分辨率为1920*1200图像数据到显示卡1和显示卡n，每块采集卡有4路图像数据输出，每块显示卡有4路图像数据输入。采集卡和显示卡之间的PCIE带宽为X2，根据PCIE带宽利用率可知其实际有效带宽为6.6Gbps，即发送端和接收端之间的带宽阈值为6.6Gbps。
图像数据按照30帧/秒的速率在采集卡和显示卡之间传输，由此可计算出一路图像数据所占带宽为1920*1200*16*30/1000/1000/1000＝1.1Gbps。针对采集卡和显示卡而言，传输一帧图像数据对应的理论时间(T＝1/30s)之内就可以发送和接收6路通道对应的6帧图像数据。
将传输一帧图像数据对应的理论时间(T)均分为6等份进行图像数据发送， 采集卡1发送的图像数据为A，B，C，D。采集卡n发送的图像为A＇，B＇，C＇，D＇，其中图像数据A，B，A＇，B＇发往显示卡n，图像数据C，D，C＇，D＇发往显示卡1。
在该分布式应用场景中存在多个采集卡同时向一个显示卡发送图像数据的情况，采集卡1和采集卡n可能同时向同一显示卡发送图像数据，导致显示卡出现PCIE瞬时带宽过载，将数据反压到采集卡，采集卡再交替向显示卡发送图像数据。理论上采集卡时间t能完成发送的图像数据，实际上需要在2t时间才完成发送，传输帧率降低进而引起播放卡顿。
针对图7中设定的FPGA+PCIE的分布式应用场景，一种传输方法如下：
如图8所示，当采集卡1的A图像数据和采集卡n的A＇图像数据同时向显示卡n发送时，造成图像数据A和图像数据A＇在显示卡n的0～(1/T)时刻上进行抢占。
如图9所示，图像数据A和A＇同时发送，导致图像数据A和图像数据A＇反压到采集卡，图像数据A和图像数据A＇的数据包交替发送，共占用(2/T)的时间。
如图10所示，从2/T时刻开始发送图像数据B与图像数据B＇，这时这两个图像数据又会在显示卡n的(2/T)～(3/T)时刻发生抢占。
如图11所示，图像数据B和图像数据B＇发送抢占的结果导致图像数据B和图像数据B＇反压到采集卡，图像数据B和图像数据B＇的数据包交替发送，共占用(2/T)～(4/T)的时间。
如图12所示，类似的图像数据C与图像数据C＇，图像数据D与图像数据D＇均会发送抢占，其最终的抢占结果均为导致发送时间延长。
本实施例中图像数据D与图像数据D＇本该在时间T内完成发送，但因为显示卡峰值带宽过载未能完成发送，这时由于PCIE发包机制，会单独用一帧的时间将图像数据D与图像数据D＇发送完成。针对图像数据A，B，C，D和图像数据A＇，B＇，C＇，D＇的发送过程而言，本该在一帧时间T内完成发送，实际使用了2T时间，总帧率为30/2＝15帧，帧率的降低引起图像卡顿。
如图13所示，显示卡瞬时带宽过载，即在同一时间段内，显示卡上多张采集卡发送的图像数据的总数据量超过了PCIE的实际有效带宽，造成了显示卡瞬时带宽过载。
理论上，针对显示卡，4路图像数据流入的PCIE带宽为4*1920*1200*16*30/1000/1000/1000＝4.4Gbps，该值明显小于6.6Gbps。但由于显示卡瞬时带宽过载，导致采集卡反压进而引起了播放卡顿。
假设当前PCIE实际有效带宽为X(Gbps)(其中(Gbps)为PCIE传输带宽单位)。一帧图像数据对应的数据量为Q，那么在传输一帧图像数据对应的理论时间之内(时间T)接收的数据带宽：
Figure PCTCN2020119440-appb-000001
在公式(1)中，当Q不变时，T越小，Q/T的值就越大。在图13中，t＜T，所以由于Q/t＞X(Gbps)进而引起了峰值带宽过载，峰值带宽过载导致该数据发送端进行带宽抢占，引起反压，反压后数据发送帧率降低进而引起播放卡顿。
因此，本申请实施例将一帧时间(T)根据每路通道中一帧传输图像的大小切割成多个时间等分，对时间片进行统一的管理分配。
如公式(1)所示，切片后的时间片t x在一帧图像传输过程中所占用的实际时间大小为当前时间片对应的待传输图像数据的总数据量与PCIE实际有效带宽X(Gbps)的比例值。若时间片t 0所对应的数据总量为Q 0，那么时间片t 0所用的实际时间为Q 0/X。
当传输一帧图像数据对应的理论时间被切片的数量为p且每一时间片内均分配有图像数据，则T＝Q 0/X+Q 1/X+…+Q p/X。
根据本申请实施例中数据传输规格的设定，传输的所有图像大小都为1920*1200，采用PCIE带宽为x2，根据PCIE带宽利用率可知其实际有效带宽X为6.6Gbps。采用30帧/秒速率进行传输，一帧时间T为1/30(s)＝33ms。传输一路通道中的一帧图像的时间片t x对应的数据总量Q x＝1920*1200*16/1000/1000/1000＝0.037Gb。由以上可计算t x＝0.037/6.6＝5.5(ms)，那么切片后的有效切片数量为33/5.5＝6。
根据本申请实施例的切片方法可知，每个时间片的实际时间会根据传输图像的大小变化而变化，在图像传输前分配了默认时间切片数量，但实际图像传输时其有效切片数量小于或等于默认时间切片数量。在该实施例的应用场景中为其分配了32个时间片，但实际有效的仅为6个时间片。
有效时间片也是根据数据分配进行实时计算的，即实际应用中在开始数据分配之前是无法确定有效切片数量的，若采集卡发送的数据量变大或变小，有效切片数量也会相应改变。本实施例中事先限定了数据传输规格，因此计算有效切片数据以帮助理解。
针对图7中设定的FPGA+PCIE的分布式应用场景，本申请的图像传输方法如下：
默认将传输一帧图像数据对应的理论时间T切割为多个时间片，例如可以 为32个，但由于发送图像大小相同，传输一帧图像数据对应的理论时间T内切割时间片有效使用数量为6。如图14所示，采集卡1发送图像为A，B，C，D，采集卡n发送的图像为A＇，B＇，C＇，D＇，其中图像A，B，A＇，B＇发往显示卡n，图像C，D，C＇，D＇发往显示卡1。
如图15所示，当采集卡n的图像A＇向显示卡n发送数据时，先根据显示卡n的时间片拥塞情况分配时间片。图像A＇发送时，显示卡n的时间片t 0未拥塞且采集卡n时间片t 0未被占用，为其分配发送时间片t 0；当采集卡1的图像A向显示卡n发送时，显示卡n的时间片t 0拥塞，显示卡n的时间片t 1未拥塞且采集卡1时间片t 1未被占用，为其分配时间片t 1。
如图16所示，采集卡n的图像B＇发送时，显示卡n的时间片t 0和t 1都已拥塞，显示卡n的时间片t 2未拥塞且采集卡n时间片t 2未被占用，为其分配发送时间片t 2；当采集卡1的图像B向显示卡n发送时，显示卡n的时间片t 0、t 1和t 2都已拥塞，显示卡n的时间片t 3未拥塞且采集卡1时间片t 3未被占用，为其分配时间片t 3。
如图17所示，采集卡n的图像C＇向显示卡1发送数据时，先根据显示卡1的时间片拥塞情况分配时间片。图像C＇发送时，显示卡1的时间片t 0未拥塞，但采集卡n的时间片t 0已经占用，此时采集卡1的时间片t 0未被占用，则针对采集卡1上的图像C而言，为其分配显示卡1的时间片t 0以及采集卡1的时间片t 0；当继续分配采集卡n的图像C＇时，显示卡1的时间片t 1未拥塞且采集卡n时间片t 1未被占用，为其分配发送时间片t 1。
如图18所示，采集卡n的图像D＇向显示卡1发送数据时，先根据显示卡1的时间片拥塞情况分配时间片。图像D＇发送时，显示卡1的时间片t 2未拥塞，但采集卡n的时间片t 2已经占用，此时采集卡1的时间片t 2未被占用，则先将显示卡1的时间片t 2以及采集卡1的时间片t 2分配给采集卡1待分配的图像D；当继续分配采集卡n的图像D＇时，显示卡1的时间片t 3未拥塞且采集卡n时间片t 3未被占用，为其分配发送时间片t 3。
分配完成后对应时间片传输的图像数据如图18所示，此时采集卡和显示卡的实际切片数量均为4，每个时间片的实际时间根据时间片内图像数据的总数据量计算得到，例如采集卡1的时间片t 0的实际时间为：图像C的数据量除以当前PCIE实际有效带宽。
如图19所示，图19左所示为相关技术中数据传输的瞬时带宽示意图，图19右所示为本申请中数据传输的瞬时带宽示意图。由图可知，本申请的图像传输方法在一帧图像数据的数据总量不变情况下有效缓解了如图19左图中由于PCIE瞬时带宽过载导致的图像卡顿问题，有效提高PCIE带宽利用率。
图20是本申请实施例提供的一种图像传输装置的结构示意图。参见图20，本申请实施例公开了一种图像传输装置300，该装置300包括：传输时间确定模块301、分段模块302和控制模块303。
传输时间确定模块301，设置为根据发送端的每路通道中的一帧待传输图像的数据量，以及发送端和接收端之间的传输带宽，确定发送端传输每路通道中一帧待传输图像所需的传输时间；分段模块302，设置为根据所述发送端传输每路通道中一帧待传输图像所需的传输时间，对传输一帧待传输图像对应的理论时间进行分段，得到所述传输时间对应的时间段；控制模块303，设置为控制所述发送端在所述传输时间对应的时间段中，向所述接收端发送所述传输时间对应的待传输图像。
在本申请实施例中，所述分段模块302，包括：
预设数量确定单元，设置为若分别传输多路通道中的一帧待传输图像所需的传输时间相同，则根据发送端的通道路数和/或传输一帧待传输图像对应的理论时间与所述传输时间的比值确定预设数量；时间段切分单元，设置为依据预设数量对传输一帧待传输图像对应的理论时间进行平均分段，得到预设数量个时间段。
在本申请实施例中，所述预设数量确定单元，是设置为：
将所述预设数量设置为等于发送端的通道路数；或者，将所述预设数量设置为等于传输一帧待传输图像对应的理论时间与所述传输时间的比值；或者，将所述预设数量设置为介于发送端的通道路数与所述比值之间的任一数值。
在本申请实施例中，若所述发送端为至少两个，所述分段模块302，包括：
交替分段单元，设置为交替依据至少两个发送端对应的传输一帧待传输图像所需的传输时间，对传输一帧待传输图像对应的理论时间进行分段，得到传输时间对应的时间段。
在本申请实施例中，所述装置还包括：
待传输图像确定模块，设置为确定每个时间段对应的传输时间内传输的待传输图像；对应关系建立模块，设置为建立所述时间段、所述待传输图像、所述待传输图像的发送端和接收端的对应关系，以使所述发送端根据所述对应关系发送待传输图像。
在本申请实施例中，若所述发送端为至少两个，则所述对应关系建立模块，包括：
遍历单元，设置为从起始时刻开始遍历多个时间段，确定接收端在当前时间段是否被分配建立对应关系；关系建立单元，设置为若所述接收端在所述当前时间段未被分配建立对应关系，则查找在当前时间段未被分配建立对应关系的发送端，建立当前时间段、所述发送端、所述发送端待分配的待传输图像以及所述接收端的对应关系。
在本申请实施例中，所述控制模块303，是设置为：
以传输一帧待传输图像对应的理论时间为周期，向所述发送端和所述接收端发送同步脉冲信号，以指示所述发送端启动待传输图像的发送，以及指示所述接收端同步启动待传输图像的接收。
本申请实施例所提供的图像传输装置可执行本申请任意实施例所提供的图像传输方法，具备执行方法相应的功能模块和效果。
根据本申请的实施例，本申请还提供了一种电子设备和一种可读存储介质。
如图21所示，图21是本申请实施例提供的一种用来实现图像传输方法的电子设备的框图。电子设备旨在表示多种形式的数字计算机，诸如，膝上型计算机、台式计算机、工作台、个人数字助理、服务器、刀片式服务器、大型计算机、和其它适合的计算机。电子设备还可以表示多种形式的移动装置，诸如，个人数字处理、蜂窝电话、智能电话、可穿戴设备和其它类似的计算装置。本文所示的部件、它们的连接和关系、以及它们的功能仅仅作为示例，并且不意在限制本文中描述的和/或者要求的本申请的实现。
如图21所示，该电子设备包括：一个或多个处理器401、存储器402，以及设置为连接多个部件的接口，包括高速接口和低速接口。多个部件利用不同的总线互相连接，并且可以被安装在公共主板上或者根据需要以其它方式安装。处理器可以对在电子设备内执行的指令进行处理，包括存储在存储器中或者存储器上以在外部输入/输出装置(诸如，耦合至接口的显示设备)上显示图形用户界面(Graphical User Interface，GUI)的图形信息的指令。在其它实施方式中，若需要，可以将多个处理器和/或多条总线与多个存储器和多个存储器一起使用。同样，可以连接多个电子设备，多个设备提供部分必要的操作(例如，作为服务器阵列、一组刀片式服务器、或者多处理器系统)。图21中以一个处理器401为例。
存储器402即为本申请所提供的非瞬时计算机可读存储介质。其中，所述存储器存储有可由至少一个处理器执行的指令，以使所述至少一个处理器执行本申请所提供的图像传输方法。本申请的非瞬时计算机可读存储介质存储计算 机指令，该计算机指令用于使计算机执行本申请所提供的图像传输方法。
存储器402作为一种非瞬时计算机可读存储介质，可设置为存储非瞬时软件程序、非瞬时计算机可执行程序以及模块，如本申请实施例中的图像传输的方法对应的程序指令/模块(例如，图20所示的传输时间确定模块301、分段模块302和控制模块303)。处理器401通过运行存储在存储器402中的非瞬时软件程序、指令以及模块，从而执行服务器的多种功能应用以及数据处理，即实现上述方法实施例中的图像传输方法。
存储器402可以包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需要的应用程序；存储数据区可存储根据图像传输的电子设备的使用所创建的数据等。此外，存储器402可以包括高速随机存取存储器，还可以包括非瞬时存储器，例如至少一个磁盘存储器件、闪存器件、或其他非瞬时固态存储器件。在一些实施例中，存储器402可选包括相对于处理器401远程设置的存储器，这些远程存储器可以通过网络连接至图像传输电子设备。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
图像传输方法的电子设备还可以包括：输入装置403和输出装置404。处理器401、存储器402、输入装置403和输出装置404可以通过总线或者其他方式连接，图21中以通过总线连接为例。
输入装置403可接收输入的数字或字符信息，以及产生与图像传输的电子设备的用户设置以及功能控制有关的键信号输入，例如触摸屏、小键盘、鼠标、轨迹板、触摸板、指示杆、一个或者多个鼠标按钮、轨迹球、操纵杆等输入装置。输出装置404可以包括显示设备、辅助照明装置(例如，发光二极管(Light Emitting Diode，LED))和触觉反馈装置(例如，振动电机)等。该显示设备可以包括但不限于，液晶显示器(Liquid Crystal Display，LCD)、LED显示器和等离子体显示器。在一些实施方式中，显示设备可以是触摸屏。
此处描述的系统和技术的多种实施方式可以在数字电子电路系统、集成电路系统、专用集成电路(Application Specific Integrated Circuit，ASIC)、计算机硬件、固件、软件、和/或它们的组合中实现。这些多种实施方式可以包括：实施在一个或者多个计算机程序中，该一个或者多个计算机程序可在包括至少一个可编程处理器的可编程系统上执行和/或解释，该可编程处理器可以是专用或者通用可编程处理器，可以从存储系统、至少一个输入装置、和至少一个输出装置接收数据和指令，并且将数据和指令传输至该存储系统、该至少一个输入装置、和该至少一个输出装置。
这些计算程序(也称作程序、软件、软件应用、或者代码)包括可编程处 理器的机器指令，并且可以利用高级过程和/或面向对象的编程语言、和/或汇编/机器语言来实施这些计算程序。如本文使用的，术语“机器可读介质”和“计算机可读介质”指的是设置为将机器指令和/或数据提供给可编程处理器的任何计算机程序产品、设备、和/或装置(例如，磁盘、光盘、存储器、可编程逻辑装置(Programmable Logic Device，PLD))，包括，接收作为机器可读信号的机器指令的机器可读介质。术语“机器可读信号”指的是用于将机器指令和/或数据提供给可编程处理器的任何信号。
为了提供与用户的交互，可以在计算机上实施此处描述的系统和技术，该计算机具有：设置为向用户显示信息的显示装置(例如，阴极射线管(Cathode Ray Tube，CRT)或者LCD监视器)；以及键盘和指向装置(例如，鼠标或者轨迹球)，用户可以通过该键盘和该指向装置来将输入提供给计算机。其它种类的装置还可以设置为提供与用户的交互；例如，提供给用户的反馈可以是任何形式的传感反馈(例如，视觉反馈、听觉反馈、或者触觉反馈)；并且可以用任何形式(包括声输入、语音输入或者、触觉输入)来接收来自用户的输入。
可以将此处描述的系统和技术实施在包括后台部件的计算系统(例如，作为数据服务器)、或者包括中间件部件的计算系统(例如，应用服务器)、或者包括前端部件的计算系统(例如，具有图形用户界面或者网络浏览器的用户计算机，用户可以通过该图形用户界面或者该网络浏览器来与此处描述的系统和技术的实施方式交互)、或者包括这种后台部件、中间件部件、或者前端部件的任何组合的计算系统中。可以通过任何形式或者介质的数字数据通信(例如，通信网络)来将系统的部件相互连接。通信网络的示例包括：局域网(Local Area Network，LAN)、广域网(Wide Area Network，WAN)、区块链网络和互联网。
计算机系统可以包括客户端和服务器。客户端和服务器一般远离彼此并且通常通过通信网络进行交互。通过在相应的计算机上运行并且彼此具有客户端-服务器关系的计算机程序来产生客户端和服务器的关系。
可以使用上面所示的多种形式的流程，重新排序、增加或删除步骤。例如，本发申请中记载的多个步骤可以并行地执行也可以顺序地执行也可以不同的次序执行，只要能够实现本申请公开的技术方案所期望的结果，本文在此不进行限制。

Claims (10)

  1. 一种图像传输方法，包括：
    根据发送端的每路通道中的一帧待传输图像的数据量，以及所述发送端和接收端之间的传输带宽，确定所述发送端传输每路通道中一帧待传输图像所需的传输时间；
    根据所述发送端传输每路通道中一帧待传输图像所需的传输时间，对传输一帧待传输图像对应的理论时间进行分段，得到所述传输时间对应的时间段；
    控制所述发送端在所述传输时间对应的时间段中，向所述接收端发送所述传输时间对应的待传输图像。
  2. 根据权利要求1所述的方法，其中，所述根据所述发送端传输每路通道中一帧待传输图像所需的传输时间，对传输一帧待传输图像对应的理论时间进行分段，得到所述传输时间对应的时间段，包括：
    在分别传输多路通道中的一帧待传输图像所需的传输时间相同的情况下，根据所述发送端的通道路数和所述传输一帧待传输图像对应的理论时间与所述传输时间的比值中的至少之一确定预设数量；
    依据所述预设数量对所述传输一帧待传输图像对应的理论时间进行平均分段，得到所述预设数量个时间段。
  3. 根据权利要求2所述的方法，其中，所述根据所述发送端的通道路数和所述传输一帧待传输图像对应的理论时间与所述传输时间的比值中的至少之一确定预设数量，包括以下之一：
    将所述预设数量设置为等于所述发送端的通道路数；
    将所述预设数量设置为等于所述传输一帧待传输图像对应的理论时间与所述传输时间的比值；
    将所述预设数量设置为介于所述发送端的通道路数与所述比值之间的一个数值。
  4. 根据权利要求1所述的方法，其中，在所述发送端为至少两个的情况下，所述根据所述发送端传输每路通道中一帧待传输图像所需的传输时间，对传输一帧待传输图像对应的理论时间进行分段，得到所述传输时间对应的时间段，包括：
    交替依据至少两个发送端对应的传输一帧待传输图像所需的传输时间，对所述传输一帧待传输图像对应的理论时间进行分段，得到所述传输时间对应的时间段。
  5. 根据权利要求1所述的方法，在所述控制所述发送端在所述传输时间对 应的时间段中，向所述接收端发送所述传输时间对应的待传输图像之前，还包括：
    确定每个时间段对应的传输时间内传输的待传输图像；
    建立所述时间段、所述待传输图像、所述待传输图像的发送端和所述待传输图像的接收端的对应关系，以使所述发送端根据所述对应关系发送所述待传输图像。
  6. 根据权利要求5所述的方法，其中，在所述发送端为至少两个的情况下，所述建立所述时间段、所述待传输图像、所述待传输图像的发送端和所述待传输图像的接收端的对应关系，包括：
    从起始时刻开始遍历多个时间段，确定所述接收端在当前时间段是否被分配建立对应关系；
    在所述接收端在所述当前时间段未被分配建立对应关系的情况下，查找在所述当前时间段未被分配建立对应关系的发送端，建立所述当前时间段、所述发送端、所述发送端待分配的待传输图像以及所述接收端的对应关系。
  7. 根据权利要求1-6中任一项所述的方法，其中，所述控制所述发送端在所述传输时间对应的时间段中，向所述接收端发送所述传输时间对应的待传输图像，包括：
    以所述传输一帧待传输图像对应的理论时间为周期，向所述发送端和所述接收端发送同步脉冲信号，以指示所述发送端启动待传输图像的发送，以及指示所述接收端同步启动待传输图像的接收。
  8. 一种图像传输装置，包括：
    传输时间确定模块，设置为根据发送端的每路通道中的一帧待传输图像的数据量，以及所述发送端和接收端之间的传输带宽，确定所述发送端传输每路通道中一帧待传输图像所需的传输时间；
    分段模块，设置为根据所述发送端传输每路通道中一帧待传输图像所需的传输时间，对传输一帧待传输图像对应的理论时间进行分段，得到所述传输时间对应的时间段；
    控制模块，设置为控制所述发送端在所述传输时间对应的时间段中，向所述接收端发送所述传输时间对应的待传输图像。
  9. 一种电子设备，包括：
    至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述 至少一个处理器执行，以使所述至少一个处理器执行权利要求1-7中任一项所述的图像传输方法。
  10. 一种存储有计算机指令的非瞬时计算机可读存储介质，所述计算机指令用于使所述计算机执行权利要求1-7中任一项所述的图像传输方法。
PCT/CN2020/119440 2020-09-30 2020-09-30 图像传输方法、装置、设备和介质 WO2022067683A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
US18/027,032 US20230376259A1 (en) 2020-09-30 2020-09-30 Image transmission method, apparatus and device, and medium
EP20955708.1A EP4224862A4 (en) 2020-09-30 2020-09-30 IMAGE TRANSMISSION METHOD, APPARATUS AND DEVICE AND MEDIUM
PCT/CN2020/119440 WO2022067683A1 (zh) 2020-09-30 2020-09-30 图像传输方法、装置、设备和介质
CN202080104910.8A CN116250229A (zh) 2020-09-30 2020-09-30 图像传输方法、装置、设备和介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/119440 WO2022067683A1 (zh) 2020-09-30 2020-09-30 图像传输方法、装置、设备和介质

Publications (1)

Publication Number Publication Date
WO2022067683A1 true WO2022067683A1 (zh) 2022-04-07

Family

ID=80949382

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/119440 WO2022067683A1 (zh) 2020-09-30 2020-09-30 图像传输方法、装置、设备和介质

Country Status (4)

Country Link
US (1) US20230376259A1 (zh)
EP (1) EP4224862A4 (zh)
CN (1) CN116250229A (zh)
WO (1) WO2022067683A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101043620A (zh) * 2006-06-19 2007-09-26 华为技术有限公司 一种电视节目传输方法及装置
CN102239696A (zh) * 2008-12-04 2011-11-09 日本电气株式会社 图像传输系统、图像传输设备和图像传输方法
US20150289279A1 (en) * 2014-04-06 2015-10-08 Hughes Network Systems, Llc Apparatus and method for an adaptive periodic bandwidth allocation approach in a shared bandwidth communications system
CN108924601A (zh) * 2018-06-25 2018-11-30 杭州雄迈信息技术有限公司 无线带宽多路视频传输方法以及装置

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6954234B2 (en) * 2001-10-10 2005-10-11 Koninklijke Philips Electronics N.V Digital video data signal processing system and method of processing digital video data signals for display by a DVI-compliant digital video display
CN107707626B (zh) * 2017-09-05 2020-04-07 百度在线网络技术（北京）有限公司 基于fpga的数据采集卡、数据采集系统及数据采集方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101043620A (zh) * 2006-06-19 2007-09-26 华为技术有限公司 一种电视节目传输方法及装置
CN102239696A (zh) * 2008-12-04 2011-11-09 日本电气株式会社 图像传输系统、图像传输设备和图像传输方法
US20150289279A1 (en) * 2014-04-06 2015-10-08 Hughes Network Systems, Llc Apparatus and method for an adaptive periodic bandwidth allocation approach in a shared bandwidth communications system
CN108924601A (zh) * 2018-06-25 2018-11-30 杭州雄迈信息技术有限公司 无线带宽多路视频传输方法以及装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4224862A4 *

Also Published As

Publication number Publication date
EP4224862A4 (en) 2024-04-17
CN116250229A (zh) 2023-06-09
EP4224862A1 (en) 2023-08-09
US20230376259A1 (en) 2023-11-23

Similar Documents

Publication Publication Date Title
US11233746B2 (en) Congestion control method and network device
CN104394486B (zh) 一种基于云桌面的视频流处理方法及装置
JP5920006B2 (ja) 画面更新制御プログラム、画面更新制御方法、および情報処理装置
WO2020140729A1 (zh) 数据传输方法、装置、计算机可读介质及电子设备
EP3860045A1 (en) Communication method and apparatus for ethernet, and device
WO2022174534A1 (zh) 一种资源请求方法及终端
EP2849080A1 (en) Image display method and device
WO2020026018A1 (zh) 文件的下载方法、装置、设备/终端/服务器及存储介质
CN103747077A (zh) 一种传输机制调整方法和服务端、客户端
CN116868553A (zh) 用于管理端点资源和拥塞缓解的数据中心网络上的动态网络接收器驱动的数据调度
CN108280018B (zh) 一种节点工作流通信开销效率分析优化方法及系统
WO2022067683A1 (zh) 图像传输方法、装置、设备和介质
CN112770358B (zh) 基于业务数据的多速率模式数据发送控制方法及装置
CN112769788B (zh) 计费业务数据处理方法、装置、电子设备及存储介质
CN111131081B (zh) 一种支持多进程的高性能单向传输的方法和装置
US8004991B1 (en) Method and system for processing network information
WO2022062799A1 (zh) 数据传输方法及相关装置
WO2022121610A1 (zh) 一种发送方法及装置
WO2022178690A1 (zh) 一种射频重调方法及装置
CN112866133B (zh) 用于获取共用最大分段大小mss的方法及装置
CN110661731B (zh) 一种报文处理方法及其装置
US9182941B2 (en) Flow control with buffer reclamation
US9626330B2 (en) Information processing apparatus, and information processing method
WO2023193689A1 (zh) 报文传输方法、装置、设备及计算机可读存储介质
WO2022165761A1 (zh) 提前终止方法、装置、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20955708

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2020955708

Country of ref document: EP

Effective date: 20230502