CN113015001B

CN113015001B - Video frame cache controller based on FPGA and control method thereof

Info

Publication number: CN113015001B
Application number: CN202110220823.5A
Authority: CN
Inventors: 曹捷
Original assignee: Shanghai Xianji Semiconductor Technology Co ltd
Current assignee: Shanghai Xianji Semiconductor Technology Co ltd
Priority date: 2021-02-26
Filing date: 2021-02-26
Publication date: 2022-04-08
Anticipated expiration: 2041-02-26
Also published as: CN113015001A

Abstract

The invention provides a video frame buffer controller based on FPGA and a control method thereof, wherein the video frame buffer controller comprises a detection configuration control module, a memory control module and a data interaction module, the detection configuration control module is used for obtaining instant reference clock frequency according to video data received each time, the memory control module is used for adjusting the clock frequency of a memory connected with the memory control module in real time according to the instant reference clock frequency, and the data interaction module is used for carrying out video data interaction with the memory through the memory control module. According to the invention, the clock frequency of the memory is adjusted in real time when the memory control module receives the low-bit-rate video stream, so that the power consumption of the video frame buffer controller is reduced, and the energy utilization rate is improved.

Description

Video frame cache controller based on FPGA and control method thereof

Technical Field

The invention relates to the field of image processing, in particular to a video frame cache controller based on an FPGA and a control method thereof.

Background

In the field of image processing, since the data size of one Frame of image is huge, a special memory is required to buffer the image data of each Frame, and a controller for controlling Video input, output and storage is called a Video Frame buffer (Video Frame buffer) controller.

The existing video formats are more in variety, such as standard definition (720x480), high definition (1280x720), full high definition (1920x1080), 4K ultra high definition (3840x2160), and 8K (7680 × 4320), so that the video resolution is larger and larger, the transmission bandwidth is higher and higher, and the data amount transmitted per second is larger. Therefore, in order to receive high-resolution and high-bandwidth video data, it is necessary to increase the clock frequency of the Memory, the Memory bandwidth, and use more FPGA on-chip RAM (Random Access Memory) to buffer the input data of one frame. Thus, the power consumption of the video frame buffer controller increases accordingly.

Further, since the existing memories are various in kind, the capacities, bandwidths, interfaces, and the like of these memories are different. At present, different types of memories can be easily hung externally through an FPGA to realize frame caching, but when different memories are used for video frame caching, the memory interface and an internal control circuit need to be modified, so that great inconvenience is brought to user design.

Disclosure of Invention

The invention aims to provide a video frame buffer controller based on an FPGA and a control method thereof, which aim to solve the problem of overlarge power consumption of the video frame buffer controller.

Another objective of the present invention is to provide a video frame buffer controller based on FPGA and a control method thereof, so as to solve the problem that it is necessary to modify a memory interface and an internal control circuit when different memories are used for video frame buffering, which brings great inconvenience to user design.

In order to solve the above problems, the present invention provides a video frame buffer controller based on FPGA, comprising:

the detection configuration control module is used for calculating the clock frequency of the memory according to the input pixel clock in the video data received each time so as to obtain instant clock configuration information;

the clock module is used for obtaining the instant reference clock frequency according to the instant clock configuration information;

the memory control module is used for adjusting the clock frequency of a memory connected with the memory control module in real time according to the instant reference clock frequency; and

and the data interaction module is used for carrying out video data interaction with the memory through the memory control module.

Optionally, the video data received by the detection configuration control module each time further includes an input field sync signal, an input data valid signal, and input data.

Further, the detection configuration control module is configured to detect a format of video data received each time, and obtain instant depth configuration information according to an input data valid signal and an input field synchronization signal received each time.

Further, the data interaction module comprises an input control module and an output control module;

the input control module is used for caching the video data received each time, configuring the caching depth of the video data received each time according to the instant depth configuration information, and writing the video data received each time into the memory through the memory control module; and

the output control module is used for caching the video data to be read out each time, configuring the caching depth of the video data to be read out each time according to the instant depth configuration information, and outputting the video data to be read out each time.

Furthermore, the data interaction module also comprises an arbitration control module;

the input control module is used for sending a write request to the arbitration control module when video data need to be written in, and writing the received video data into the memory through the arbitration control module and the memory control module when the arbitration control module responds to the write request;

the output control module is used for sending a reading request to the arbitration control module when video data needs to be read out, and reading the video data needing to be read out from the memory through the arbitration control module and the memory control module each time when the arbitration control module responds to the reading request; and

the arbitration control module is used for performing arbitration control on the write request of the input control module and the read request of the output control module, and writing the video data received each time into the memory or reading the video data to be read each time from the memory by controlling the write operation of the input control module and the read operation of the output control module.

Furthermore, the memory control module is provided with a memory interface, and the description of the memory interface is configured to be matched with the description of a target memory interface connected with the memory interface and then connected with the memory through the memory interface.

Further, the memory control module further has:

and the bus interface is used for receiving an external instruction and transmitting the instruction to the memory control circuit so as to control the memory control circuit to change the description of the memory interface and realize the connection of the memory interface and different types of memories.

Furthermore, the memory control module further comprises a memory control circuit, which is used for converting the description of the memory interface into the description of a target memory interface matched with the type of the memory connected with the memory control circuit, so as to convert a general write address, a write command and a write data format input through the arbitration control module into a write user format of the memory connected with the arbitration control module, convert a user format in the memory into a general read data format output by the arbitration control module, and is further used for obtaining the clock frequency of the memory according to the instant reference clock frequency and adjusting the clock frequency of the memory through the memory interface.

Further, the arbitration control module is configured to determine whether the memory is in an operating state or an idle state, suspend the received read request and/or the write request when the memory is in the operating state, and perform a corresponding response to the received read request and/or the write request when the memory is in the idle state to perform a read operation or a write operation on the memory,

wherein the operation state is that the memory is performing a read operation or a write operation of video data; the idle state is that the memory does not perform a read operation or a write operation of video data.

Further, when the memory is in the idle state and the arbitration control module receives a read request and a write request at the same time, the arbitration control module preferentially responds to the read request and suspends the write request.

Furthermore, the data interaction module also comprises a base address switching control module;

the input control module is used for sending an input field synchronizing signal to the base address switching control module after the current video data is written into the memory, and the base address switching control module is used for responding to the input field synchronizing signal and switching to a base address of the next video data so as to write the next video data; and

the output control module is used for sending an output field synchronizing signal to the base address switching control module after the current video data is read from the memory, and the base address switching control module is used for responding to the output field synchronizing signal and switching to the base address of the next video data so as to read the next video data.

On the other hand, the invention also provides a control method of the video frame buffer controller based on the FPGA, which comprises the following steps:

the detection configuration control module calculates the clock frequency of the memory according to the input pixel clock in the video data received each time to obtain instant clock configuration information;

the clock module obtains an instant reference clock frequency according to the instant clock configuration information;

the memory control module adjusts the clock frequency of a memory connected with the memory control module in real time according to the instant reference clock frequency; and

and the data interaction module performs video data interaction with the memory through the memory control module.

Furthermore, the detection configuration control module detects the format of the video data received each time, counts the resolution and frame rate of the video data received each time according to the effective signal of the input data received each time and the input field synchronous signal, and selects the instant depth configuration information meeting the requirements.

Further, the data interaction module performs video data interaction with the memory through the memory control module, and includes:

when video data needs to be written, an input control module sends a write request to an arbitration control module, and when the arbitration control module responds to the write request, the input control module caches the video data received each time, and the video data received each time is written into the memory through the arbitration control module and a memory control module, wherein the input control module dynamically configures the cache depth of the video data according to the instant depth configuration information, and the memory control module adjusts the clock frequency of the memory in real time according to the instant reference clock frequency; and

when video data needs to be read out, the output control module sends a read request to the arbitration control module, and when the arbitration control module responds to the read request, the output control module reads out the video data which needs to be read out each time from the memory through the arbitration control module and the memory control module, wherein the output control module dynamically configures the depth of the video data which needs to be read out each time according to the instant depth configuration information.

Further, when the input control module sends a write request to the arbitration control module, and/or the output control module sends a read request to the arbitration control module, and the memory is in an operating state, the arbitration control module suspends the received read request or write request, and suspends the write request of the video data of the input control module or the read request of the video data of the output control module;

when the input control module sends a write request to an arbitration control module and the memory is in an idle state, then the arbitration control module responds to the write request;

when the output control module issues a read request to the arbitration control module and the memory is in an idle state, then the arbitration control module responds to the read request; and

when the input control module sends a write request to the arbitration control module, the output control module sends a read request to the arbitration control module, and the memory is in an idle state, then the arbitration control module responds to the read request preferentially.

Further, when the arbitration control module responds to the write request of the input control module, the arbitration control module gives the control right of the memory to the input control module, the input control module dynamically configures the depth of the video data received each time by the cache according to the instant depth configuration information, receives the base address of the current video data, and sends a write command, a write address and write data to the arbitration control module;

the input control module sends an input field synchronizing signal to the base address switching control module after writing the current video data into the memory, and receives the base address of the next video data sent by the base address switching control module when the base address switching control module responds to the input field synchronizing signal;

after the arbitration control module receives the video data sent by the input control module, the arbitration control module sends the received write command, write address and write data to a memory control module; and

and the memory control module adjusts the clock frequency of the memory in real time according to the instant reference clock frequency and writes the write command, the write address and the write data into the memory.

Further, when the arbitration control module responds to the read request of the output control module, the arbitration control module gives the control right of the memory to the output control module, the output control module sends a read command and a read address to the arbitration control module, and the arbitration control module sends the read command and the read address to the memory control module;

the memory control module reads the video data which needs to be read out each time from the memory and sends the video data which needs to be read out each time to the arbitration control module, and the arbitration control module sends the video data which needs to be read out each time to the output control module;

the output control module dynamically configures and caches the depth of the video data to be read out each time according to the instant depth configuration information, and reads out the video data to be read out each time; and

after the current video data is read out from the memory, the output control module sends an output field synchronizing signal to the base address switching control module, and receives the base address of the next video data sent by the base address switching control module when the base address switching control module responds to the output field synchronizing signal.

Furthermore, the description of the memory interface of the memory control module is configured to be matched with the description of the target memory interface of the type of the memory connected with the memory control module, and then the memory control module is connected with the memory through the memory interface.

Further, the memory control circuit converts the description of the memory interface into the description of a target memory interface matched with the type of the memory connected with the memory control circuit, so as to convert a general write address, a write command and a write data format input through the arbitration control module into a write user format of the memory connected with the arbitration control module, convert the user format in the memory into a general read data format output by the arbitration control module, obtain the clock frequency of the memory according to the instant reference clock frequency, and adjust the clock frequency of the memory through the memory interface; and

the bus interface of the memory control module receives an external instruction and transmits the instruction to the memory control circuit to control the memory control circuit to change the description of the memory interface and enable the changed description of the memory interface to be matched with the type of the memory connected with the memory control circuit.

Compared with the prior art, the method has the following beneficial effects:

the invention provides a video frame cache controller based on FPGA and a control method thereof.A detection configuration control module is used for obtaining the instant reference clock frequency according to the video data received each time; the memory control module is used for adjusting the clock frequency of a memory connected with the memory control module in real time according to the instant reference clock frequency; and the data interaction module is used for carrying out video data interaction with the memory through the memory control module. Compared with the clock frequency and the memory bandwidth fixed by the clock when the memory receives the video data in the prior art, the clock frequency and the memory bandwidth of the memory connected with the memory are adjusted in real time according to the instant reference clock frequency through the memory control module, so that the clock frequency and the memory bandwidth of the memory are improved when the high-bit-rate video stream is received, and the clock frequency of the memory can be reduced when the low-bit-rate video stream is received, thereby reducing the power consumption of the video frame buffer controller and improving the energy utilization rate.

Drawings

FIG. 1 is a schematic diagram of an interface of a video frame buffer controller;

fig. 2 is a schematic structural diagram of an FPGA-based video frame buffer controller according to an embodiment of the present invention;

FIG. 3 is a block diagram of an arbitration control module according to an embodiment of the present invention;

FIG. 4 is a block diagram of a memory control module according to an embodiment of the invention.

Description of reference numerals:

1-a video input interface; 2-a video output interface; 3-a memory control interface; 4-a memory;

100-video frame buffer controller;

10-detecting a configuration control module; 20-a clock module; 30-an input control module; 31-input first-in first-out storage submodule; 40-arbitration control module, 41-arbiter status sub-module; 42-a data strobe submodule; 50-base address switching control module; 60-an output control module; 61-output first-in first-out storage submodule; 70-a memory control module; 71-a memory interface; 72-memory control circuitry; 73-bus interface; 80-memory.

Detailed Description

The following describes the video frame buffer controller based on FPGA and the control method thereof in further detail. The present invention will now be described in more detail with reference to the accompanying drawings, in which preferred embodiments of the invention are shown, it being understood that one skilled in the art may modify the invention herein described while still achieving the advantageous effects of the invention. Accordingly, the following description should be construed as broadly as possible to those skilled in the art and not as limiting the invention.

In the interest of clarity, not all features of an actual implementation are described. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. It will of course be appreciated that in the development of any such actual embodiment, numerous implementation-specific details must be set forth in order to achieve the developer's specific goals, such as compliance with system-related and business-related constraints, which will vary from one implementation to another. Moreover, it will be appreciated that such a development effort might be complex and time-consuming, but would nevertheless be a routine undertaking for those of ordinary skill in the art.

In order to make the objects and features of the present invention more comprehensible, embodiments of the present invention are described in detail below with reference to the accompanying drawings. It is to be noted that the drawings are in a very simplified form and are all used in a non-precise ratio for the purpose of facilitating and distinctly aiding in the description of the embodiments of the invention.

Fig. 1 is a schematic structural diagram of an interface of a video frame buffer controller. As shown in fig. 1, the buffering and control principle of the video frame buffer controller is as follows: the interface of the Video frame buffer controller comprises a Video input interface 1, a Video output interface 2 and a memory control interface 3, wherein the Video input interface 1 is responsible for receiving Video data (Video) and then writing the Video data into a memory 4, the Video output interface 2 is responsible for reading out the Video data in the memory 4 and outputting the Video data outwards, and the memory control interface 3 is responsible for connecting with the memory 4 and controlling the read operation and the write operation of the memory 4. Because the video frame buffer controller can only perform read operation or write operation on the memory at the same time, but cannot perform read operation and write operation at the same time, the video frame buffer controller must perform arbitration control on the read operation and the write operation, and perform the read operation and the write operation in turn. When a certain data is written in the writing operation, the control right of the memory is handed to the reading operation, and when a certain data is read out in the reading operation, the control right of the memory is handed to the writing operation, so that the reading and the writing of the video data and the caching are alternately realized in sequence.

Fig. 2 is a schematic structural diagram of a video frame buffer controller based on FPGA according to this embodiment. As shown in fig. 2, the present embodiment provides a video frame buffer controller 100 based on an FPGA (field programmable gate array), i.e. the video frame buffer controller 100 can be implemented by using hardware resources and logic resources built in the FPGA. The video frame buffer controller 100 is connected to the memory 80, and is configured to receive the video data, write the video data into the memory 80, and output the video data read from the memory 80. The video data at least includes an input pixel clock (I _ vin _ clk), an input field synchronizing signal (I _ vin _ vs), an input data valid signal (I _ vin _ de), input data (I _ vin _ data), and the like.

With continued reference to fig. 2, the video frame buffer controller 100 includes a detection configuration control module 10, a clock module 20, a data interaction module and a memory control module 70, wherein the data interaction module is configured to perform video data interaction with the memory through the memory control module. The data interaction module includes an input control module 30, an arbitration control module 40, a base address switching control module 50, and an output control module 60.

The detection configuration control module 10 is configured to receive an input pixel clock, an input field synchronization signal, and an input data valid signal in video data input each time, perform format detection on the input pixel clock, the input field synchronization signal, and the input data valid signal received each time, count a resolution and a frame rate of instant video data according to the input data valid signal and the input field synchronization signal received each time, and select a minimum configurable buffer depth (i.e., instant depth configuration information) that meets requirements to configure a buffer depth for the input control module 30 and the output control module 60. Further, the detection configuration controller 10 may also calculate the clock frequency of the memory 80 according to the input pixel clock received each time to obtain the instant clock configuration information.

The specific method for counting the resolution and frame rate of the video data is, for example: the horizontal resolution and the vertical resolution of the video data can be respectively counted according to the number of the continuous periods of the input data effective signals received each time and the number of times of the input data effective signals appearing in one frame, and the frame rate of the video data can be counted according to the number of pulses of the input field synchronizing signals within 1 second.

The method of selecting the minimum configurable cache depth that meets the requirements is for example: the instant depth configuration information configuration is larger than the horizontal resolution of the video data, the configurable buffer depth can be divided into several levels, for example, 1024, 2049, 4096, 8192, etc., and the minimum configurable buffer depth meeting the requirements is selected according to different horizontal resolutions.

The detection configuration control module 10 may provide instant depth configuration information to the input control module 30 and the output control module 60, respectively, and provide instant clock configuration information to the clock module 20, where the instant depth configuration information may enable the video frame buffer controller 100 to adjust the RAM resources of the input control module 30 and the output control module 60 in real time, so as to meet the bandwidth requirement on the video data, thereby enabling the video frame buffer controller to reduce the power consumption of the video frame buffer controller when receiving a low-bitrate video stream, and improving the energy utilization rate.

The method for detecting the clock frequency of the memory that the configuration control module 10 can calculate is, for example: setting the clock frequency of the memory as Fclk, the data bit width of the memory as Wmem, and the bandwidth utilization rate of the memory as e%, then the effective bandwidth of the memory is Fclk × Wmem × e%; setting the horizontal resolution of video data as Horin, the vertical resolution of the video data as Verin, the data bit width of the video data as Din, the frame rate of the video data as Fvsin, the horizontal resolution of the output video data as Horout, the vertical resolution of the output video data as Verout, the data bit width of the output video data as Dout, and the frame rate of the output video data as Fvsout, wherein the sum of the bandwidth of the video data and the bandwidth of the output video data is (Horin Verin Din Fvsin) + (Horout Verout Dout Fvsout); after calculating the sum of the bandwidth of the video data and the bandwidth of the output video data, the requirement for the instantaneous reference clock frequency due to the configuration of the memory control module is: the effective bandwidth of the memory must be greater than the sum of the bandwidth of the video data and the bandwidth of the output video data, and therefore, the clock frequency Fclk of the time memory is calculated as long as Fclk × Wmem > (Horin × Verin × Fvsin) + (Horout × Verout Dout × Fvsout) according to the effective bandwidth of the memory is greater than or equal to the sum of the bandwidth of the video data and the bandwidth of the output video data. And calculating the clock frequency of the memory according to the formula to obtain instant clock configuration information.

The clock module 20 is configured to configure the clock configuration information configured by the control module 10 according to the detection, and generate an instant reference clock frequency (memref _ clk) of the memory control module 70, so as to change the clock frequency of the memory 80, and dynamically adjust the clock frequency of the memory 80, so that the clock frequency and the memory bandwidth of the memory 80 are increased when the memory 80 receives a high-bit-rate video stream, and the clock frequency and the memory bandwidth of the memory are decreased when the memory 80 receives a low-bit-rate video stream, thereby reducing the power consumption of the video frame buffer controller and increasing the energy utilization rate.

The input control module 30 is configured to receive externally input video data, such as an input pixel clock (I _ vin _ clk), an input field synchronizing signal (I _ vin _ vs), an input data valid signal (I _ vin _ de), and input data (I _ vin _ data). The input control module 30 is further configured to, when receiving the video data, perform cache depth configuration on the input control module 30 according to the instant depth configuration information, and cache the video data received each time, so as to change the usage amount of RAM resources in the FPGA chip. In addition, the input control module 30 issues a write request to the arbitration control module 40 when it is necessary to write each received video data to the memory, and writes each received video data to the memory 80 via the arbitration control module 40 and the memory control module 70 when the arbitration control module 40 responds to the write request of the input control module 30. Further, the input control module 30 is further configured to send an input field sync signal to the base address switching control module 50 after the current video data is written into the memory 80, and receive a base address of the next video data sent by the base address switching control module 50 when the base address switching control module 50 responds to the input field sync signal, so that the memory 80 writes the next video data.

As a non-limiting example, the input control module 30 includes an input fifo memory sub-module 31, which is an on-chip memory of the FPGA, and when the input control module 30 receives external video data, the input fifo memory sub-module 31 configures a buffer depth for the input control module 30 according to the instant depth configuration information, and buffers the video data received each time, so as to adjust the resources of the input fifo memory sub-module 31 in real time for buffering, thereby reducing the number of resources used by the on-chip memory of the FPGA (i.e., the input fifo memory sub-module 31). For example, the input fifo storage sub-module 31 may adjust the input fifo storage sub-module 31 to buffer one frame of image in real time according to the instant depth configuration information.

With continued reference to fig. 2, the base address switching control module 50 is used to switch the base address of the write operation of the input control module 30 or the base address of the read operation of the output control module 60. In this embodiment, the base address switching control module 50 is configured to switch a base address corresponding to video data received by the input control module 30 each time or a base address corresponding to video data read by the output control module 60. Specifically, the input control module 30 is configured to, after the current data is written into the memory 80, send an input field sync signal to the base address switching control module 50, where the base address switching control module 50 may be configured to respond to the input field sync signal, switch to a base address (Wr _ base _ addr) of the next video data, and send a base address (i.e., a start address) of the next video data to the input control module 30, so as to write the next video data; after the output control module 60 reads the current video data from the memory 80, the output control module 60 sends an output field sync signal to the base address switching control module 50, and the base address switching control module 50 may be configured to switch to a base address (re _ base _ addr) of the next video data in response to the output field sync signal, and send the base address of the next video data to the output control module 60, so as to read the next video data. In this embodiment, each piece of video data is a frame of image, and in other embodiments, each piece of video data may also be a picture, but may also be other data. The base address switching control module 50 can switch the base address, so as to ensure that the writing or reading of the video data can be continuously performed, and the storage space of the memory is fully utilized, so that the memory is maximally utilized, and the size of the data amount which can be stored by the memory is increased.

The arbitration control module 40 is configured to arbitrate and control write requests of the input control module 30 and read requests of the output control module 60, and the arbitration control module 40 may implement writing of video data received each time into the memory 80 connected to the memory control module 70, or reading of video data to be read each time from the memory 80 and outputting the video data to the outside through the output control module 30 by controlling write operations of the input control module 30 and read operations of the output control module 60, so that the arbitration control module 40 may function to control input and output of data.

In this embodiment, whether the input control module 30 is to implement input of video data or the output control module 60 is to implement output of video data, the arbitration control module 40 must respond before execution. The arbitration control module 40 can effectively manage the input or output of the video data, so that the read and write operations of the video frame buffer controller 100 to the memory 80 can be controlled.

The arbitration control module 40 performs different response operations on the write request of the input control module 30 or the read request of the output control module 60 under different conditions by determining the video data read-write state of the memory 80. Specifically, the arbitration control module 40 may detect whether the memory 80 is performing reading or writing of video data, define that the memory 80 at this time is in an operating state if the memory 80 is performing reading or writing of video data, and define that the memory 80 at this time is in an idle state if the memory 80 is not performing reading or writing of video data.

In this embodiment, when the arbitration control module 40 receives a write request from the input control module 30 and/or a read request from the output control module 60, the arbitration control module 40 first determines which state the memory is in, and if the memory 80 is in an operating state, the arbitration control module 40 suspends the received read request or write request, and suspends the video data write request from the input control module 30 or the video data read request from the output control module 60; if the memory is in an idle state, the arbitration control module 40 will respond to the read request and/or the write request. Specifically, when the arbitration control module 40 receives a write request of the input control module 30 and a read request of the output control module 60 at the same time, and the memory 80 is in an idle state, the arbitration control module 40 preferably responds to the read request of the output control module 60 and suspends the write request, so as to ensure that the output video data is not interrupted, ensure continuity, and improve user experience. When the arbitration control module 40 receives a write request of the input control module 30 or a read request of the output control module 60 and the memory is in an idle state, the arbitration control module 40 responds to the write request of the input control module 30 or the read request of the output control module 60.

In other embodiments, when the arbitration control module 40 receives a write request from the input control module 30 and a read request from the output control module 60, and the memory is in an idle state, the arbitration control module 40 may preferably respond to the write request from the input control module 30 to ensure consistency of writing video data.

When the arbitration control module 40 responds to the write request of the input control module 30, the arbitration control module 40 gives the control right of the memory 80 to the input control module 30, and the input control module 30 can perform a write operation on the memory 80, specifically, the input control module 30 sends the write command, the write address (the base address of the current data) and the write data to the arbitration control module 40, and after receiving the video data sent by the input control module 30, the arbitration control module 40 sends the received write command, write address and write data to the memory control module 70. When the arbitration control module 40 responds to the read request of the output control module 60, the arbitration control module 40 gives the control right of the memory to the output control module 60, and the output control module 60 can read the memory. Specifically, the output control module 60 sends a read command and a read address (a base address of currently read video data) to the arbitration control module 40, the arbitration control module 40 sends the read command and the read address to the memory control module 70, the memory control module 70 sends the read data to the arbitration control module 40, and the arbitration control module 40 sends the read data to the output control module 60.

In other embodiments, the arbitration control module 40 is configured to suspend a write request or a read request for a predetermined time period, and to respond to the associated write request or read request regardless of the state of the memory.

Fig. 3 is a schematic structural diagram of an arbitration control module according to this embodiment. Referring to fig. 3 and fig. 2, the arbitration control module 40 includes an arbiter status sub-module 41 and a data strobe sub-module 42, and the arbiter status sub-module 41 is connected to and controls the data strobe sub-module 42. The arbiter status sub-module 41 is used for arbitration control of write requests from the input control module 30 and read requests from the output control module 60. The data strobe submodule 42 is configured to send the write command, the write address and the write data to the memory control module 70 when the arbiter status submodule 41 responds to the write request of the input control module 30; when the arbiter state sub-module 41 responds to the read request of the output control module 60, the read data is sent to the output control module 60.

The output control module 60 is respectively connected to the detection configuration control module 10, the arbitration control module 40 and the base address switching control module 50. The output control module 60 is configured to receive video data (i.e., read data) to be read each time, and is further configured to configure a cache depth of the output control module 60 according to the instant depth configuration information when the read data is received, and cache the read data to adjust the usage amount of RAM resources in the chip of the FPGA. And, the output controller 60 is further configured to issue a read request to the arbitration control module 40 when the video data needs to be read out, and read out the video data to be read out each time from the memory 80 through the arbitration control module 40 and the memory control module 70 when the arbitration control module 40 responds to the read request of the output control module 60. In addition, the output controller 60 is further configured to send an output field sync signal to the base address switching control module 50 after the current video data is read from the memory 80, and receive a base address of a next video data sent by the base address switching control module 50 when the base address switching control module 50 responds to the output field sync signal, so as to be used for a subsequent memory to read the next video data.

The output control module 60 includes an output fifo storage sub-module 61, and when the output control module 60 outputs video data, the output fifo storage sub-module 61 configures a cache depth of the output control module 60 according to the instant depth configuration information, and caches the video data to be read out each time, so as to adjust the resources of the output fifo storage sub-module 61 in real time for caching, thereby reducing the number of resources used by the on-chip memory of the FPGA. For example, the output fifo storage sub-module 61 may adjust the buffering of one line of one frame of image by the output fifo storage sub-module 61 according to the buffer depth configuration, and then the output control module 60 outputs the video data buffered from the output fifo storage sub-module 61 to the outside.

It is understood that the size of the buffer space that the input fifo sub-module 31 and the output fifo sub-module 61 can buffer the video data depends on the size of the space available for buffering, and as the space is larger, the video data that the input fifo sub-module 31 and the output fifo sub-module 61 can buffer is larger. By detecting the instant depth configuration information configured by the configuration control module 10, the input first-in first-out storage submodule 31 and the output first-in first-out storage submodule 61 can be dynamically adjusted, so that the RAM resource in the FPGA slice can be dynamically adjusted, the transmission speed of video data is improved, the video frame buffer controller 100 can more quickly buffer the video data, the power consumption of the video frame buffer controller 100 can be reduced when the low-bit-rate video stream is input and output, and the energy utilization rate is improved.

Under the condition that the write request of the input control module 30 or the read request of the output control module 60 is suspended, the input fifo storage submodule 31 of the input control module 30 and the output fifo storage submodule 61 of the output control module 60 may buffer video data first, and the fifo memory may play a role of buffering video data, so that when a write operation or a read operation of related video data is performed, video data may be input or output more quickly.

Referring to fig. 2, the memory control module 70 is configured to adjust the clock frequency of the memory connected to the memory control module in real time according to the instantaneous reference clock frequency, and further configured to write a write address and write data into the memory 80 under a write command of the input control module 30, and further configured to read data from the memory 80 and send the read data to the output control module 60 through the arbitration control module 40 under a read command of the output control module 60. And the memory control module 70 is also used for realizing connection with different kinds of memories 80 and exchanging video data with the memories 80.

Fig. 4 is a schematic structural diagram of the memory control module of the present embodiment. As shown in fig. 4, the memory control module 70 may have a memory interface 71, a memory control circuit 72, and a bus interface 73. The memory control circuit 72 is used to implement signal interaction between the memory control circuit 72 and the memory interface 71, and enables the memory control circuit 72 to perform the relevant operations described with respect to the memory control interface 71. For example, the memory control circuit 72 may be configured to convert the description of the memory interface 71 into a description of a target memory interface 71 matching the type of the memory 80 connected thereto, to convert a general write address, a write command, and a write data format input through the arbitration control module 40 into a write user format of the memory 80 connected thereto, and to convert a user format in the memory 80 connected to the memory interface 71 into a general read data format output to the arbitration control module 40, thereby enabling the memory control circuit 72 to exchange video data with the memory 80 through the memory interface 71; and is further configured to receive the clock frequency of the memory calculated by the detection configuration control module 10 through the instant reference clock frequency configured by the clock module 20, and adjust the clock frequency of the memory 80 through the memory interface 71, so that when the resolution of the video data changes, the clock frequency of the memory can be dynamically adjusted, thereby reducing power consumption when inputting a low-bit-rate video stream and improving energy utilization.

The bus interface 73 is configured to receive an external instruction, and transmit the instruction to the memory control circuit 72, so as to control the memory control circuit 72 to change the description of the memory interface 71, thereby implementing connection with different types of memories 80 by using the memory interface 71, that is, implementing selection of a corresponding memory by using the memory interface 71 for a read operation or a write operation. When the memory 80 needs to be connected to the video frame buffer controller 100, the memory 80 may be connected through the memory interface 71. The bus interface 73 may be an SPI bus interface, an I2C bus interface, a JTAG interface, a URART interface, an AXI bus interface, or the like.

The operations described above with respect to the memory interface 71 may be performed at different time periods. For example, before the video frame buffer controller 100 leaves the factory, the type of the memory 80 to be connected to the video frame buffer controller 100 may be predetermined, and then the connection with the bus interface 73 may be realized through the corresponding interface setting device in cooperation with corresponding setting software, and then the related instructions including the description of the memory interface 71 may be input to the memory control circuit 72 through the bus interface 73, and then the memory control circuit 72 may execute the related instructions, so as to realize the description of the memory interface 71. Of course, when the video frame buffer controller 100 and other electronic components are packaged together as a chip and sold to a user, the user may also set software by downloading a related interface and cooperate with the pin connection device, and the connection with the video frame buffer controller 100 is realized through the pins of the chip, so as to further perform the description of the memory interface 71. Further, after the chip packaged with the video frame buffer controller 100 is soldered on the circuit board, the user may also realize connection with the video frame buffer controller 100 through a gold finger of the circuit board, and further realize description of the memory interface 71 of the video frame buffer controller 100 through related interface setting software.

The memory control module 70 may be connected to the outside through the bus interface 73 so that the memory control module 70 may receive an instruction from the outside, which may be used to convert the description of the current memory interface 11 into a description of a target memory interface matching the type of the memory 80 connected to the interface. Of course, the external instruction may also be some related control instruction and object code, e.g.

A description of a target memory interface of the software interface that matches the type of memory 80. When an external instruction is transmitted to the memory control circuit 72 via the bus interface 73, the memory control circuit 72, through execution of the instruction, may further convert the description of the current memory interface 71 into the description of the target memory interface matching the type of the memory 80, so that the memory control circuit 72 can perform video data exchange with the memory 80 through the memory interface 71.

In this embodiment, by providing the memory interface 71, the memory control circuit 72, and the bus interface 73 in the memory control module 70, an external instruction may be received through the bus interface 73 first, so that a user may send an instruction to the memory control circuit 72 through the bus interface 73, and by providing the memory control circuit 72 that can execute these instructions, an operation of changing the description of the memory interface 71 by using the memory control circuit 72 may be implemented, thereby implementing an effect of converting the type of the memory interface 71. The converted memory interface 71 may be an interface corresponding to various memories 80, for example, an interface corresponding to SDRAM, DDR2/3SDRAM, PSRAM or Hyper RAM, and the converted memory interface 71 may be used to connect to the different types of memories 80, so as to finally implement video data exchange between the memory control module 70 and the memories 80.

Of course, in the case where the memory interface 71 has connected one type of memory, if it is necessary to replace the memory 80 with another type of memory 80, it is also possible to convert the description of the memory interface 71 matching the last type of memory 80 into the description of the target memory interface matching the type of the memory 80 after replacement by using the memory control circuit 72, so that the memory control circuit 72 can perform video data exchange with the memory 80 of another type after replacement through the memory interface 71. The specific operation mode may be that, in the case of power failure, a user erases the description about the memory interface originally used for matching with the memory of one type before replacement through the bus interface 73 of the memory control module 70, and then inputs a related instruction through the bus interface 73, where the instruction may include the description about the memory interface 71 matching with the memory 80 of another type after replacement, and then the memory control circuit 72 executes the inputted instruction to replace the description about the interface of the memory 80 with the description about the memory 80 after replacement, so that the memory control circuit 72 can exchange video data with the memory after replacement through the memory interface 71.

Under the condition that the description about the memory interface 71 already exists, the data input function of the bus interface 73 is utilized, so that the memory control circuit 72 can convert the description of the memory interface 71 into the description matched with the replaced type of the other type of the memory 80, one memory interface 71 can be connected with the plurality of types of the memories 80, the universality of the memory control module 70 is improved, and a user does not need to replace the matched memory control module under the condition of replacing the memory 80, namely the cost of the user is saved; and the clock frequency of the memory can be dynamically adjusted according to the resolution change of the input video stream, so that the power consumption can be reduced and the energy utilization rate can be improved when the low-bit-rate video stream is input.

After the connection of the memory control module 70 and the memory 80 is realized through the conversion memory interface 71, the video data can be buffered using the connected memory 80.

The embodiment also provides a control method of a video frame buffer controller based on the FPGA, which comprises the following steps:

the detection configuration control module obtains an instant reference clock frequency according to the video data received each time;

Referring to fig. 2-4, the step of obtaining the real-time reference clock frequency according to the video data received each time by the detection configuration control module specifically includes:

the detection configuration control module calculates the clock frequency of the memory according to the input pixel clock received each time to obtain instant clock configuration information; and the clock module obtains the instant reference clock frequency according to the instant clock configuration information.

The detection configuration control module 10 receives an input pixel clock, an input field synchronous signal, and an input data valid signal in video data input each time, performs format detection on the input pixel clock, the input field synchronous signal, and the input data valid signal received each time, counts a resolution and a frame rate of instant video data according to the input data valid signal and the input field synchronous signal received each time, and selects a minimum configurable buffer depth (i.e., instant depth configuration information) meeting requirements to configure buffer depths for the input control module 30 and the output control module 60, so that the video frame buffer controller 100 can adjust RAM resources of the input control module 30 and the output control module 60 in real time to meet bandwidth requirements for the video data.

The detection configuration control module 10 may further calculate a clock frequency of the memory 80 according to the input pixel clock received each time to obtain instant clock configuration information, and then the clock module 20 generates an instant reference clock frequency of the memory control module 70 according to the clock configuration information configured by the detection configuration control module 10.

In this step, the memory control module 70 may dynamically adjust the clock frequency of the memory by configuring the instantaneous reference clock frequency, so as to reduce power consumption and improve energy utilization when inputting a low-bit-rate video stream. The instant depth configuration information is respectively configured to the input control module 30 and the output control module 60, so that the RAM resources of the input control module 30 and the output control module 60 can be adjusted in real time, further, when a low-bit-rate video stream is input, the power consumption can be reduced, and the energy utilization rate can be improved.

The data interaction module performs video data interaction with the memory through the memory control module and comprises the following steps:

the input control module 30 sends a write request to the arbitration control module 40, and/or the output control module sends a read request to the arbitration control module, and the arbitration control module 40 performs different response operations on the write request of the input control module 30 or the read request of the output control module 60 under different conditions by judging the video data read and write states of the memory 80.

Specifically, when the read/write state of the video data in the memory 80 is an operation state, the arbitration control module 40 suspends the received read request or write request, and suspends the write request of the video data in the input control module 30 or the read request of the video data in the output control module 60;

when the read/write state of the video data of the memory 80 is idle and only the write request from the input control module 30 to the arbitration control module 40 is received, the arbitration control module 40 responds to the write request from the input control module and passes the control right of the memory 80 to the input control module 30, the input control module 30 dynamically configures the depth of the video data received and buffered each time according to the instant depth configuration information, receives the base address of the current video data from the base address switching control module 50, and sends the write command, the write address (the base address of the current video data) and the write data to the arbitration control module 40, and at the same time, the input control module 30 sends an input field synchronization signal to the base address switching control module 50 after writing the current share of video data into the memory 80, when the base address switching control module 50 responds to an input field synchronizing signal, the base address of the next piece of video data sent by the base address switching control module 50 is received, after the arbitration control module 40 receives the video data sent by the input control module 30, the arbitration control module 40 sends the received write command, write address and write data to the memory control module 70, the memory control module 70 adjusts the clock frequency of the memory in real time according to the instant reference clock frequency, and finally writes the write command, write address and write data into the memory;

when the read/write state of the video data of the memory 80 is idle and only the read request of the output control module 60 is received, the arbitration control module 40 responds to the read request of the output control module 60 and gives the control right of the memory to the output control module 60, the output control module 60 sends a read command and a read address (the base address of the currently read data) to the arbitration control module 40, the arbitration control module 40 sends the read command and the read address to the memory control module 70, the memory control module 70 reads the video data from the memory and sends the read video data to the arbitration control module 40, the arbitration control module 40 sends the video data to the output control module 60, the output control module 60 configures the buffer depth of the output control module 60 according to the instant depth configuration information, the video data is buffered and read out, after the current video data is read out from the memory, the output control module 60 sends an output field synchronizing signal to the base address switching control module 50, and receives the base address of the next video data sent by the base address switching control module 50 when the base address switching control module 50 responds to the output field synchronizing signal.

When the read/write state of the video data in the memory 80 is idle, and simultaneously receives the write request sent by the input control module 30 to the arbitration control module 40 and the read request sent by the output control module 60 to the arbitration control module 40, the read request of the output control module 60 is preferably responded, and the write request is suspended, so as to ensure that the output of the video data is not interrupted, ensure the consistency, and improve the user experience.

The memory control module 70 adjusts the clock frequency of the memory connected to it in real time according to the instant reference clock frequency, writes the write address and write data into the memory 80 under the write command of the input control module 30, and reads the read data from the memory 80 and sends the read data to the output control module 60 through the arbitration control module 40 under the read command of the output control module 60. And, the memory control module 70 implements connection with the heterogeneous memory 80 and exchanges video data with the memory 80.

Specifically, the memory control circuit 72 implements signal interaction between the memory control circuit 72 and the memory interface 71, and enables the memory control circuit 72 to perform the related operations described with respect to the memory control interface 71.

The memory control circuit 72 converts the description of the memory interface 71 into a description of a target memory interface matched with the type of the memory connected thereto, so as to convert a general write address, a write command and a write data format input through the arbitration control module into a write user format of the memory connected thereto, and convert a user format in the memory into a general read data format output by the arbitration control module, and also obtains a clock frequency of the memory according to an instantaneous reference clock frequency, and adjusts the clock frequency of the memory through the memory interface.

The description of the memory interface 71 is connected to the memory 80 through a memory interface after the description of the target memory interface configured by the memory control circuit 72 to match the type of memory 80 to which it is connected.

The bus interface 73 receives an external instruction and transmits the instruction to the memory control circuit 72 to control the memory control circuit 72 to change the description of the memory interface 71 and enable the changed description of the memory interface 71 to be matched with the type of the memory 80 connected with the memory interface, and the step can be implemented to be connected with various memories 80, so that the universality of the memory control module 70 is improved, and a user does not need to replace the memory control module matched with the memory interface under the condition of replacing the memory 80, namely, the cost of the user is saved.

In summary, in the video frame buffer controller based on the FPGA and the control method thereof provided by the present invention, the detection configuration control module is configured to obtain the real-time reference clock frequency according to the video data received each time; the memory control module is used for adjusting the clock frequency of a memory connected with the memory control module in real time according to the instant reference clock frequency; and the data interaction module is used for carrying out video data interaction with the memory through the memory control module. Compared with the clock frequency and the memory bandwidth fixed by the clock when the memory receives the video data in the prior art, the clock frequency of the memory connected with the memory is adjusted in real time according to the instant reference clock frequency through the memory control module, so that the clock frequency of the memory is increased when a high-bit-rate video stream is received, and the clock frequency and the memory bandwidth of the memory can be reduced when a low-bit-rate video stream is received, thereby reducing the power consumption of the video frame buffer controller and increasing the energy utilization rate.

It is to be understood that while the present invention has been described in conjunction with the preferred embodiments thereof, it is not intended to limit the invention to those embodiments. It will be apparent to those skilled in the art from this disclosure that many changes and modifications can be made, or equivalents modified, in the embodiments of the invention without departing from the scope of the invention. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention are still within the scope of the protection of the technical solution of the present invention, unless the contents of the technical solution of the present invention are departed.

Claims

1. A video frame buffer controller based on FPGA, comprising:

2. The FPGA-based video frame buffer controller of claim 1 wherein said detection configuration control module receives video data each time further comprising an input field sync signal, an input data valid signal, and input data.

3. The FPGA-based video frame buffer controller of claim 2 wherein said detection configuration control module is configured to detect a format of video data received each time and obtain said instantaneous depth configuration information according to an input data valid signal and an input field sync signal received each time.

4. The FPGA-based video frame cache controller of claim 2 wherein said data interaction module comprises an input control module and an output control module;

5. The FPGA-based video frame cache controller of claim 4, wherein the data interaction module further comprises an arbitration control module;

the output control module is used for sending a read request to the arbitration control module when video data needs to be read out, and reading the video data needing to be read out from the memory through the arbitration control module and the memory control module each time when the arbitration control module responds to the read request; and

6. The FPGA-based video frame cache controller of any one of claims 1-5, wherein the memory control module has a memory interface, and the description of the memory interface is configured to be connected with the memory through the memory interface after being configured as the description of a target memory interface matched with the type of the memory connected with the memory.

7. The FPGA-based video frame buffer controller of claim 6 wherein said memory control module further comprises a bus interface, said bus interface is configured to receive an external command and transmit said command to said memory control circuit, so as to control said memory control circuit to change the description of the memory interface, thereby enabling said memory interface to be connected to different types of memories.

8. The FPGA-based video frame cache controller of claim 5 wherein said memory control module further comprises memory control circuitry;

the memory control circuit is used for converting the description of the memory interface into the description of a target memory interface matched with the type of the memory connected with the memory control circuit, converting a general write address, a write command and a write data format input through the arbitration control module into a write user format of the memory connected with the arbitration control module, converting the user format in the memory into a general read data format output by the arbitration control module, and obtaining the clock frequency of the memory according to the instant reference clock frequency and adjusting the clock frequency of the memory through the memory interface.

9. The FPGA-based video frame cache controller of claim 5, wherein the arbitration control module is configured to determine whether the memory is in an operating state or an idle state, suspend the received read request and/or write request when the memory is in the operating state, and respond to the received read request and/or write request when the memory is in the idle state to perform a read operation or a write operation on the memory,

10. The FPGA-based video frame buffer controller of claim 9 wherein when said memory is in said idle state and said arbitration control module receives a read request and a write request simultaneously, said arbitration control module responds to said read request preferentially and suspends said write request.

11. The FPGA-based video frame cache controller of claim 4, wherein the data interaction module further comprises a base address switching control module;

12. A control method of a video frame buffer controller based on FPGA is characterized by comprising the following steps:

13. The FPGA-based video frame buffer controller control method of claim 12,

the detection configuration control module detects the format of the video data received each time, counts the resolution and frame rate of the video data received each time according to the effective signal of the input data received each time and the input field synchronous signal, and selects the instant depth configuration information meeting the requirements.

14. The method as claimed in claim 12, wherein the step of the data interaction module interacting with the memory via the memory control module comprises:

15. The FPGA-based video frame buffer controller control method of claim 14,

when the input control module sends a write request to an arbitration control module, and/or the output control module sends a read request to the arbitration control module, and the memory is in an operating state, the arbitration control module suspends the received read request or write request, and suspends the write request of the video data of the input control module or the read request of the video data of the output control module;

16. The FPGA-based video frame buffer controller control method of claim 15,

when the arbitration control module responds to the write request of the input control module, the arbitration control module gives the control right of the memory to the input control module, the input control module dynamically configures and caches the depth of the video data received each time according to the instant depth configuration information, receives the base address of the current video data, and sends a write command, a write address and write data to the arbitration control module;

17. The FPGA-based video frame buffer controller control method of claim 15,

when the arbitration control module responds to the read request of the output control module, the arbitration control module gives the control right of the memory to the output control module, the output control module sends a read command and a read address to the arbitration control module, and the arbitration control module sends the read command and the read address to the memory control module;

18. The method as claimed in claim 14, wherein the memory interface of the memory control module is configured to be connected to the memory through the memory interface after being configured to be a description of a target memory interface matching the type of the memory connected thereto.

19. The FPGA-based video frame buffer controller control method of claim 18,

the memory control circuit converts the description of the memory interface into the description of a target memory interface matched with the type of the memory connected with the memory control circuit, so as to convert a general write address, a write command and a write data format input by the arbitration control module into a write user format of the memory connected with the arbitration control module, convert the user format in the memory into a general read data format output by the arbitration control module, obtain the clock frequency of the memory according to the instant reference clock frequency and adjust the clock frequency of the memory through the memory interface; and

the bus interface of the memory control module receives an external instruction and transmits the instruction to the memory control circuit so as to control the memory control circuit to change the description of the memory interface and enable the changed description of the memory interface to be matched with the type of a memory connected with the memory control circuit.