CN118214820B

CN118214820B - Image data processing method, product, equipment and medium

Info

Publication number: CN118214820B
Application number: CN202410621039.9A
Authority: CN
Inventors: 张贞雷; 李拓; 李仁刚; 邹晓峰; 满宏涛; 周朗
Original assignee: Shandong Yunhai Guochuang Cloud Computing Equipment Industry Innovation Center Co Ltd
Current assignee: Shandong Yunhai Guochuang Cloud Computing Equipment Industry Innovation Center Co Ltd
Priority date: 2024-05-20
Filing date: 2024-05-20
Publication date: 2024-08-09
Anticipated expiration: 2044-05-20
Also published as: CN118214820A

Abstract

The invention discloses an image data processing method, an image data processing product, image data processing equipment and an image data processing medium, and relates to the technical field of servers. According to the scheme, the corresponding full time of each random access memory is respectively determined according to the read average rate, the write average rate and the residual space of each random access memory in the BMC, and the full time is equivalent to the actual running state of each module for processing the image data; the data processing strategy of the front module of the module where the random access memory is located is further adjusted according to the relation between the full time and the corresponding configuration threshold value, so that the writing rate of the random access memory of the current module is adjusted, dynamic unified adjustment among the image data processing rates of all modules on the image data processing path of the BMC is realized, the data loss of the rear module is avoided, and the frame loss rate of image data transmission is reduced.

Description

Image data processing method, product, equipment and medium

Technical Field

The present invention relates to the field of server technologies, and in particular, to an image data processing method, a product, an apparatus, and a medium.

Background

The basic management controller (Board Management Controller, BMC) is an important component of the server, and is capable of monitoring various states of the server. The BMC also has an important function of transmitting local video information to a remote end through a network for remote display and monitoring.

The Video processing process of the BMC mainly comprises the steps of receiving an original Video image at a host end through the BMC, sequentially passing through a Video graphics array (Video GRAPHICS ARRAY, VGA) module, a data capturing module, a color conversion module and a BLOCK (BLOCK) building module in a BMC chip, processing and transmitting the Video image by an image compression module and an output control module, and writing the Video image into a double-rate synchronous dynamic random access memory (Dual DATE RATE SDRAM, DDR). However, in practical applications, many functional modules occupy the system bus and read and write the DDR in the BMC, so that the writing state of the output control module of the BMC for writing the image data into the DDR is dynamically changed, specifically, writing is sometimes fast and writing is sometimes slow when writing into the DDR. On the premise that image data of a host end is continuously generated, the data processing speed between modules related to image transmission processing in the BMC is not dynamically managed, so that data of a later-stage module among the modules is about to be saturated, but the data of the current-stage module still occurs according to the original speed, the data of the later-stage module is lost, and the frame loss rate of image data transmission is improved.

In view of the above-mentioned problems, how to solve the problem that the data processing speed between the modules related to the image transmission processing in the BMC is not dynamically managed, so that the data of the later module is lost, and the frame loss rate of the image data transmission is improved, which is a problem to be solved by those skilled in the art.

Disclosure of Invention

The invention aims to provide an image data processing method, an image data processing product, image data processing equipment and an image data processing medium, which are used for solving the problems that the data processing speed between modules related to image transmission processing in a BMC is not dynamically managed, the data of a later module is lost, and the frame loss rate of image data transmission is improved.

In order to solve the above technical problems, the present invention provides an image data processing method, including:

Based on a preset time period, acquiring the reading average rate, the writing average rate and the residual space of each random access memory in the baseboard management controller; wherein, each random access memory is respectively corresponding to a plurality of modules for processing image data in the baseboard management controller one by one and respectively corresponding to and storing the image data processed by each module;

determining the corresponding full time of each random access memory according to the reading average rate, the writing average rate and the residual space of each random access memory;

When the full time of the random access memory is larger than a corresponding first configuration threshold value, the writing rate of a front-stage module of the module where the random access memory is located to the random access memory is improved;

when the full time of the random access memory is not more than the corresponding first configuration threshold value and is not less than the corresponding second configuration threshold value, maintaining the writing rate of the previous module of the module where the random access memory is positioned to the random access memory;

When the full time of the random access memory is smaller than a corresponding second configuration threshold value, reducing the writing rate of a front-stage module of the module where the random access memory is located to the random access memory; wherein the first configuration threshold is greater than the second configuration threshold.

In one aspect, the determining the corresponding full time of each random access memory according to the read average rate, the write average rate and the remaining space of each random access memory includes:

acquiring preset configuration parameters, and determining a difference value between the writing average rate and the reading average rate of the random access memory;

determining a quotient of the remaining space of the random access memory and the difference value;

And determining the product of the preset configuration parameter and the quotient to obtain the corresponding full time of the random access memory.

On the other hand, when the module where the random access memory is located is specifically an image compression module, and the full time of the random access memory is greater than the corresponding first configuration threshold, the increasing the writing rate of the previous module of the module where the random access memory is located to the random access memory includes:

reading image data of even pixel positions in each random access sub-memory based on the random access memory array of the block building module;

Based on the data arrangement sequence of the image data of each even pixel position, the image data of each even pixel position is sequentially written into the random access memory of the image compression module in a manner of repeated writing by taking single image data as a unit.

On the other hand, when the module where the random access memory is located is specifically an image compression module, and the full time of the random access memory is smaller than the corresponding second configuration threshold, the reducing the writing rate of the previous module of the module where the random access memory is located to the random access memory includes:

Determining a clock period at which the block building module generates single image data;

Taking a plurality of clock periods as reading periods, and reading image data in a random access sub-memory of a random access memory array of a block building module based on the reading periods;

And writing each image data into the random access memory of the image compression module in sequence.

On the other hand, when the module where the random access memory is located is specifically a block building module and the full time of the random access memory is greater than the corresponding first configuration threshold, the increasing the writing rate of the preceding module of the module where the random access memory is located to the random access memory includes:

Reading image data of a plurality of continuous addresses in the random access memory of the color conversion module according to a clock cycle;

Splicing the image data read in the same clock period to obtain a plurality of combined image data;

And according to the clock period, writing each piece of combined image data into the random access memory of the block building module in sequence.

On the other hand, when the module where the random access memory is located is specifically a block building module and the full time of the random access memory is smaller than the corresponding second configuration threshold, the reducing the writing rate of the preceding module of the module where the random access memory is located to the random access memory includes:

determining a clock period at which the color conversion module generates single image data;

Taking a plurality of clock periods as reading periods, and reading image data in the random access memory of the color conversion module based on the reading periods;

And writing each image data into the random access memory of the block building module in sequence.

On the other hand, when the module in which the random access memory is located is specifically a color conversion module, and the full time of the random access memory is greater than the corresponding first configuration threshold, the increasing the writing rate of the preceding module in the module in which the random access memory is located to the random access memory includes:

When the data capturing module is in a digital video interface mode or in a buffer mode, reading image data of a plurality of continuous addresses in the random access memory of the data capturing module according to a clock cycle;

sequentially writing each piece of combined image data into the random access memory of the color conversion module according to the clock period;

when the data capture module is in the buffer mode, the method further comprises increasing a burst length of the data capture module reading the double rate synchronous dynamic random access memory.

On the other hand, when the module in which the random access memory is located is specifically a color conversion module, and the full time of the random access memory is smaller than the corresponding second configuration threshold, the reducing the writing rate of the preceding module in the module in which the random access memory is located to the random access memory includes:

determining a clock period at which the data capture module generates single image data;

Taking a plurality of clock periods as reading periods, and reading image data in the random access memory of a data capturing module based on the reading periods;

sequentially writing each image data into the random access memory of the color conversion module;

the burst length of the double-rate synchronous dynamic random access memory read by the data capturing module is reduced.

In another aspect, the method further comprises:

When the data acquisition module is determined to be in a line blanking period, the writing rate of the data acquisition module to the random access memory of the color conversion module is reduced, and the writing rate of the color conversion module to the random access memory of the block building module is reduced.

In another aspect, the method further comprises:

And when the data acquisition module is determined to be in the vertical blanking period, reducing the writing rate of the data acquisition module to the random access memory of the color conversion module, reducing the writing rate of the color conversion module to the random access memory of the block building module, and reducing the writing rate of the block building module to the random access memory of the image compression module.

In another aspect, after reducing the writing rate of the data capturing module to the random access memory of the color conversion module, reducing the writing rate of the color conversion module to the random access memory of the block building module, and reducing the writing rate of the block building module to the random access memory of the image compression module, the method further includes:

Judging whether the residual space of the random access memory of the data capturing module and/or the residual space of the random access memory of the color conversion module and/or the residual space of the random access memory of the block building module is reduced in a preset period and reaches a corresponding space limit value;

if yes, carrying out frame loss processing on the image data of the current frame.

On the other hand, if the remaining space of the random access memory of the data capturing module and/or the remaining space of the random access memory of the color conversion module and/or the remaining space of the random access memory of the block building module is determined to decrease within a predetermined period and not reach the corresponding space limit, the method further comprises:

monitoring a space reduction rate of the remaining space;

when the space reduction rate reaches a preset rate, outputting alarm information;

The input of raw image data to the data capture module is paused.

To solve the above technical problem, the present invention also provides a computer program product, which includes a computer program/instruction that when executed by a processor implements the steps of the above image data processing method.

In order to solve the above technical problem, the present invention further provides an image data processing apparatus, including:

a memory for storing a computer program;

And a processor for implementing the steps of the image data processing method when executing the computer program.

To solve the above technical problem, the present invention further provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the steps of the image data processing method described above.

The image data processing method provided by the invention is used for acquiring the reading average speed, the writing average speed and the residual space of each random access memory in the substrate management controller based on a preset time period; wherein, each random access memory is respectively corresponding to a plurality of modules for processing image data in the baseboard management controller one by one, and respectively corresponding to and storing the image data processed by each module; determining the corresponding full time of each random access memory according to the reading average speed, the writing average speed and the residual space of each random access memory; when the full time of the random access memory is larger than the corresponding first configuration threshold value, the writing rate of a front-stage module of the module where the random access memory is positioned to the random access memory is improved; when the full time of the random access memory is not more than the corresponding first configuration threshold value and is not less than the corresponding second configuration threshold value, the writing rate of the previous module of the module where the current random access memory is positioned to the random access memory is maintained; when the full time of the random access memory is smaller than the corresponding second configuration threshold value, the writing rate of the front module of the module where the random access memory is located to the random access memory is reduced; wherein the first configuration threshold is greater than the second configuration threshold. The invention has the advantages that the corresponding full time of each random access memory is respectively determined according to the reading average speed, the writing average speed and the residual space of each random access memory in the BMC, which is equivalent to the actual running state of each module for processing the image data; the data processing strategy of the front module of the module where the random access memory is located is further adjusted according to the relation between the full time and the corresponding configuration threshold value, so that the writing rate of the random access memory of the current module is adjusted, dynamic unified adjustment among the image data processing rates of all modules on the image data processing path of the BMC is realized, the data loss of the rear module is avoided, and the frame loss rate of image data transmission is reduced.

Furthermore, the invention also provides a computer program product, an image data processing device and a computer readable storage medium, and the effects are the same.

Drawings

For a clearer description of embodiments of the present invention, the drawings that are required to be used in the embodiments will be briefly described, it being apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to the drawings without inventive effort for those skilled in the art.

Fig. 1 is a schematic diagram of a video function implementation of a server baseboard management controller according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a third RAM in a block configuration module according to an embodiment of the present invention;

FIG. 3 is a flowchart of an image data processing method according to an embodiment of the present invention;

Fig. 4 is a schematic diagram of a downlink blanking period and a vertical blanking period with 640×480 resolution according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of an image data processing apparatus according to an embodiment of the present invention;

fig. 6 is a schematic diagram of an image data processing apparatus according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without making any inventive effort are within the scope of the present invention.

The invention provides an image data processing method, an image data processing product, image data processing equipment and an image data processing medium, which are used for solving the problems that the data processing speed between modules related to image transmission processing in a BMC is not dynamically managed, the data of a later module is lost, and the frame loss rate of image data transmission is improved.

In order to better understand the aspects of the present invention, the present invention will be described in further detail with reference to the accompanying drawings and detailed description.

BMC is an important component of the server and can monitor various states of the server. The BMC also has an important function of transmitting local video information to a remote end through a network for remote display and monitoring.

Fig. 1 is a schematic diagram of video function implementation of a server baseboard management controller according to an embodiment of the present invention. As shown in fig. 1, in the video processing process of the BMC, an original video image at the host is received by the BMC, and is written into the DDR after being processed and transmitted by the VGA module, the data capturing module, the color conversion module, the BLOCK building module, the image compression module and the output control module in the BMC chip in sequence. The following describes the process of video processing by the BMC in detail with reference to fig. 1:

firstly, the host outputs the original data to the BMC, and the original video data is converted into video image data in Red Green Blue (RGB) format through the processing of the VGA module of the baseboard management control chip. Further, the data capture module obtains the raw RGB data in two ways: the first is a digital video interface (Digital Visual Interface, DVI) mode, where the data capture module is in a passive accept state and RGB data is sent directly from the DVI interface of the VGA module to the data capture module. It is understood that the DVI interface includes a line synchronization signal (Hs), a field synchronization signal (Vs), raw RGB DATA (rgb_data), an indication that the RGB DATA is valid (rgb_data_vld), and a pixel clock period (rgb_clk). The second is a Buffer (Buffer) mode, i.e., the data capture module actively goes to DDR to read the original RGB image data. It should be emphasized that there is a buffer of RGB data inside the data capturing module, i.e. the first random access memory (ram_a) in fig. 1; after the image data is cached, the image data is written into the RAM_A, and then read and sent to a later module.

The color conversion module converts the image data from RGB format to luminance-chrominance (YUV) format, and the conversion process is completed by using matrix conversion formula. It should be emphasized that there is a buffer of YUV data in the color conversion module, i.e. the second random access memory (ram_b) in fig. 1. After the image data is cached, the image data is written into the RAM_B, and then read and sent to a later-stage module.

The BLOCK building module is responsible for converting the image data in YUV format into YUV BLOCK data, namely BLOCK data. It will be appreciated that there is also a buffer memory for YUV BLOCK data within the BLOCK building BLOCK, i.e. the third random access memory (ram_array) in fig. 1. It should be noted that, according to the difference of YUV444/YUV420 compression formats, the BLOCK size may be 16×16 and 8×8, so 16Y data random access sub-memories (y_ram), 16U data random access sub-memories (u_ram) and 16V data random access sub-memories (v_ram) are required to perform buffering of y_block data, u_block data and v_block data, respectively. Wherein the Y data represents Luminance (luminence) information, also called gray values; which represents the darkness of the image. The U data and V data represent chrominance (Chrominance) information, also known as chromatic aberration, which represent the saturation and hue of colors.

The third random access memory RAM _ ARRAY is in practice a memory ARRAY made up of a plurality of random access sub-memories. Fig. 2 is a schematic diagram of an architecture of a third ram in a block building block according to an embodiment of the present invention. It can be seen that the third random access memory ram_array contains 16Y data random access sub-memories, 16U data random access sub-memories and 16V data random access sub-memories. The writing and reading processes for two compression formats of YUV444 and YUV420 are described below in conjunction with fig. 2:

Control logic on YUV data writing side:

(1) In YUV420 mode, in principle all Y data is retained, even rows and even columns of U/V data are retained, specifically, according to the resolution of the image, Y data of row 0/16/32/48 … … is written into y_ram_0, Y data of row 1/17/33/49 … … is written into y_ram_1, Y data of row 2/18/34/50 … … is written into y_ram_2, … …, Y data of row 15/31/47/63 … … is written into y_ram_15. Even column U data of row 0/16/32/48 … … is written into U_RAM_0, even column U data of row 2/18/34/50 … … is written into U_RAM_1, … …, and even column U data of row 14/30/46/62 … … is written into U_RAM_7. Even column U data of row 0/16/32/48 … … is written into V_RAM_0, even column U data of row 2/18/34/50 … … is written into V_RAM_1, … …, and even column U data of row 14/30/46/62 … … is written into V_RAM_7. (2) In YUV444 mode, Y/U/V data of all rows and all columns is reserved in principle. Specifically, according to the resolution of the image, the Y data of the 0/8/16/24 … … th row is written into y_ram_0, the Y data of the 1/9/17/25 th row … … is written into y_ram_1, the Y data of the 2/10/18/26 … … th row is written into y_ram_2, … …, and the Y data of the 7/15/23/31 … … th row is written into y_ram_7. U data of row 0/8/16/24 … … is written into U_RAM_0, U data of row 1/9/17/25 … … is written into U_RAM_1, U data of row 2/10/18/26 … … is written into U_RAM_2, … …, and U data of row 7/15/23/31 … … is written into U_RAM_7. V data of row 0/8/16/24 … … is written into V_RAM_0, V data of row 1/9/17/25 … … is written into V_RAM_1, V data of row 2/10/18/26 … … is written into V_RAM_2, … …, and V data of row 7/15/23/31 … … is written into V_RAM_7.

Control logic on YUV data read side:

(1) In YUV420 mode, sequentially reading 16 times Y_RAM_0, 16 times Y_RAM_1, … …, and 16 times Y_RAM_15 to form 16×16 Y_BLOCK data; sequentially reading 8 times of U_RAM_0,8 times of U_RAM_1, … … and 8 times of U_RAM_7 to form 8 multiplied by 8 U_BLOCK data; the v_ram_0, v_ram_1, … …, v_ram_7 are sequentially read 8 times, v_block data of 8×8 is formed, and then the remaining data are circularly read. (2) In YUV444 mode, sequentially reading 8 times Y_RAM_0,8 times Y_RAM_1, … … and 8 times Y_RAM_7 to form 8×8 Y_BLOCK data; sequentially reading 8 times of U_RAM_0,8 times of U_RAM_1, … … and 8 times of U_RAM_7 to form 8 multiplied by 8 U_BLOCK data; the v_ram_0, v_ram_1, … …, v_ram_7 are sequentially read 8 times, v_block data of 8×8 is formed, and then the remaining data are circularly read.

And finally, the BLOCK building module inputs the YUV BLOCK data into the image compression module to complete the video compression of the JPEG format, and writes the compressed JPEG image data into the DDR through the output control module. It is emphasized that the image compression module is internally provided with a fourth random access memory (ram_c) for buffering the received y_block data, u_block data and v_block data.

In practical application, many functional modules occupy system buses and read and write DDRs in the BMC, so that the writing state of the output control module of the BMC for writing the image data into the DDRs is dynamically changed, and the writing is particularly performed in the case of writing into the DDRs, and sometimes writing is performed quickly, and sometimes writing is performed slowly. On the premise that image data of a host end is continuously generated, the data processing speed between modules related to image transmission processing in the BMC is not dynamically managed, so that data of a later-stage module among the modules is about to be saturated, but the data of the current-stage module still occurs according to the original speed, the data of the later-stage module is lost, and the frame loss rate of image data transmission is improved. The present invention thus provides an image data processing method in order to solve the above-described problems.

Fig. 3 is a flowchart of an image data processing method according to an embodiment of the present invention. As shown in fig. 3, the method includes:

S10: based on a preset time period, the reading average speed, the writing average speed and the residual space of each random access memory in the baseboard management controller are obtained.

Each random access memory is respectively in one-to-one correspondence with a plurality of modules used for processing image data in the baseboard management controller, and respectively corresponds to and stores the image data processed by each module.

In a specific implementation, firstly, based on a preset time period, the average reading rate, the average writing rate and the remaining space of each random access memory in the BMC are obtained. It will be appreciated that each random access memory corresponds to a plurality of modules for processing image data in the baseboard management controller one by one, and the modules for processing image data in the BMC are respectively a data capturing module, a color conversion module, a block building module and an image compression module, so that the reading average rate, the writing average rate and the remaining space of the first random access memory ram_a, the second random access memory ram_b, the third random access memory ram_aray and the fourth random access memory ram_c need to be obtained based on the preset time period T.

The preset time period T is not limited in this embodiment, and depends on the specific implementation. It should be noted that, since the third ram_array is actually a random access memory ARRAY, which includes a plurality of random access sub-memories for storing y_block data, u_block data, and v_block data, respectively, and sequentially reading out y_block data, u_block data, and v_block data when the data is read out, the preset period of time must include at least one complete set of y_block data, u_block data, and v_block data in order to ensure accuracy of average rate acquisition, thereby ensuring accuracy of rate statistics.

S11: and respectively determining the corresponding full time of each random access memory according to the read average rate, the write average rate and the residual space of each random access memory.

Further, according to the read average rate, the write average rate and the remaining space of each random access memory, the corresponding full time of each random access memory is determined. Because the data capturing module in the BMC is used for acquiring the original RGB image data, the full-time of the first ram_a of the data capturing module is not required to be determined in the scheme, and the full-time of the second ram_b, the third ram_array and the fourth ram_c are mainly determined respectively.

It should be noted that, the full time refers to a time point when the ram reaches the maximum storage threshold, and can reflect the real-time running state of the current ram module. By calculating the full time of the random access memory, the operation condition of the front module of the module where the random access memory is located can be adjusted in the following steps. In the present embodiment, the process of obtaining the full time is not limited, and depends on the specific implementation.

S12: and when the full time of the random access memory is larger than the corresponding first configuration threshold value, improving the writing rate of the front-stage module of the module where the random access memory is positioned to the random access memory.

S13: and when the full time of the random access memory is not more than the corresponding first configuration threshold value and is not less than the corresponding second configuration threshold value, maintaining the writing rate of the previous module of the module where the current random access memory is positioned to the random access memory.

S14: and when the full time of the random access memory is smaller than the corresponding second configuration threshold value, reducing the writing rate of the front-stage module of the module where the random access memory is positioned to the random access memory.

Wherein the first configuration threshold is greater than the second configuration threshold.

In order to adjust the operation condition of the front-stage module of the module where the random access memory is located according to the full time, in this embodiment, two thresholds, namely a first configuration threshold and a second configuration threshold, are set for the full time, and the operation condition of the front-stage module of the module where the random access memory is located is specifically adjusted according to the magnitude relation between the full time and the corresponding configuration threshold.

The full time of each random access memory has a first configuration threshold and a second configuration threshold corresponding to each other. Further, the first configuration threshold is greater than the second configuration threshold. In this embodiment, the first configuration threshold and the second configuration threshold are not limited, and they depend on the specific implementation.

Specifically, when the full time of the random access memory is greater than the corresponding first configuration threshold, it indicates that the remaining space of the current random access memory is relatively large. In order to improve the compression transmission efficiency of the image data, the writing rate of the front module of the module where the random access memory is located to the random access memory can be improved. When the full time of the random access memory is not more than the corresponding first configuration threshold value and not less than the corresponding second configuration threshold value, the residual space of the current random access memory is proper, the compression transmission efficiency of the image data is moderate, and the writing rate of the front module of the module where the current random access memory is located to the random access memory can be kept. When the full time of the random access memory is smaller than the corresponding second configuration threshold, the remaining space of the current random access memory is smaller, and in order to reduce the compression transmission efficiency of the image data, the writing rate of the previous module of the module where the random access memory is located to the random access memory needs to be reduced. Therefore, the dynamic unified adjustment of the image data processing rates of all modules on the image data processing path of the BMC is realized.

It should be noted that, in this embodiment, the specific process of adjusting the write rate of the ram by the previous module of the module where the ram is located is not limited, and depends on the specific implementation.

In this embodiment, according to the read average rate, the write average rate and the remaining space of each random access memory in the BMC, the corresponding full time of each random access memory is determined, which is equivalent to obtaining the actual running state of each module for processing the image data; the data processing strategy of the front module of the module where the random access memory is located is further adjusted according to the relation between the full time and the corresponding configuration threshold value, so that the writing rate of the random access memory of the current module is adjusted, dynamic unified adjustment among the image data processing rates of all modules on the image data processing path of the BMC is realized, the data loss of the rear module is avoided, and the frame loss rate of image data transmission is reduced.

Based on the above embodiments, in some embodiments, determining the corresponding full time of each random access memory according to the read average rate, the write average rate and the remaining space of each random access memory includes:

S111: and acquiring preset configuration parameters, and determining the difference value between the writing average speed and the reading average speed of the random access memory.

S112: the quotient of the remaining space of the random access memory and the difference is determined.

S113: and determining the product of the preset configuration parameters and the quotient to obtain the corresponding full time of the random access memory.

To calculate the full time, the preset configuration parameters first need to be acquired. It should be noted that, since the total time of the three random access memories (the second random access memory ram_b, the third random access memory ram_array, and the fourth random access memory ram_c) is calculated, there are three preset configuration parameters. In this embodiment, each preset configuration parameter is not limited, and depends on the specific implementation.

The difference between the write average rate and the read average rate of the random access memory is further determined. The quotient of the remaining space of the random access memory and the difference is determined. And determining the product of the preset configuration parameters and the quotient to obtain the corresponding full time of the random access memory. The corresponding full time of the three random access memories is as follows:

T2=PARMA2×[SPACE_RAM_B/(RAM_B_WR_SPEED_AVG-RAM_B_RD_SPEED_AVG)]；

Wherein T2 is the full time of the second ram_b, PARMA is a preset configuration parameter of the second ram_b, space_ram_b is the remaining SPACE of the second ram_b, ram_b_wr_speed_avg is the average write rate of the second ram_b, and ram_b_rd_speed_avg is the average read rate of the second ram_b. T3= PARMA × [ space_ram ARRAY/(RAM_ARRAY\u) WR_SPEED_AVG-ram_array-RAM/u ARRAY; wherein T3 is the full time of the third ram_array, PARMA is a preset configuration parameter of the third ram_array, space_ram_array is the remaining SPACE of the third ram_array, ram_array_wr_speed_avg is the average write rate of the third ram_array, and ram_array_rd_speed_avg is the average read rate of the third ram_array. T4= PARMA × [ SPACE u ] ram_c/(ram_c_wr) SPEED_AVG-RAM_C/u rd_speed_avg) ]; wherein T4 is the full time of the fourth random access memory ram_c, PARMA4 is a preset configuration parameter of the fourth random access memory ram_c, space_ram_c is the remaining SPACE of the fourth random access memory ram_c, ram_c_wr_speed_avg is the average write rate of the fourth random access memory ram_c, and ram_c_rd_speed_avg is the average read rate of the fourth random access memory ram_c.

In this embodiment, the calculation of the full time corresponding to each random access memory is realized in the above manner, so that the operation condition of the previous module of the module where the random access memory is located is specifically adjusted in the subsequent step according to the size relationship between the full time and the corresponding configuration threshold.

Based on the content of the above embodiment and the specific module where the random access memory is located, a specific process of adjusting the operation condition of the previous module of the module where the random access memory is located according to the size relationship between the full time and the corresponding configuration threshold will be described in detail below:

The module where the random access memory is located is specifically an image compression module.

At this time, the random access memory is specifically a fourth random access memory ram_c, the corresponding full time is T4, the corresponding first configuration threshold is ram_c_high_ THREHOLD, and the corresponding second configuration threshold is ram_c_standby_ THREHOLD.

(1) When T4> ram_c_high_ THREHOLD, increasing the write rate of the previous module of the module where the random access memory is located to the random access memory, including:

S121: the image data of even pixel positions in each random access sub-memory is read based on the random access memory array of the block building block.

S122: based on the data arrangement sequence of the image data of each even pixel position, the image data of each even pixel position is sequentially written into the random access memory of the image compression module in a manner of repeated writing by taking single image data as a unit.

It will be appreciated that when T4> ram_c_high_ THREHOLD indicates that the remaining space of the fourth random access memory ram_c is relatively large, it is necessary to speed up the rate of image data output from the block building block to the image compression block in order to improve the compression efficiency. It should be noted that, in general, the BLOCK building BLOCK generates one y_block data or u_block data or v_block data in one clock cycle.

Specifically, the image data of even pixel positions in each random access sub-memory is read based on the random access memory ARRAY of the block building block, that is, based on the third random access memory ram_array of the block building block. For example, as shown in fig. 2, addresses such as address 0, address 2, address 4, etc. of the Y data random access sub-memory 0 are read to obtain y_block data such as 0 th y_block data, 2 nd y_block data, 4 th y_block data, etc. respectively. This achieves a doubling of the read-side rate of the third random access memory RAM _ ARRAY of the block building block.

The image data of each even pixel position is sequentially written into the random access memory of the image compression module in a manner of repeated writing for a plurality of times by taking single image data as a unit based on the data arrangement sequence of the image data of each even pixel position. In other words, in the same clock cycle of the fourth ram_c of the image compression module, the 0 th y_block data is written into the position 0, the position 1, … …, the position n (n is a positive integer), the 2 nd y_block data is written into the position n+1, the position n+2, … …, the position 2n+1, and so on, so that the writing rate of the fourth ram_c side is doubled, and the principle is that the information difference between the pixel points in the same BLOCK or adjacent positions in the same image is very small, and after JPEG compression, the difference is not seen by naked eyes. And n may be set to 1 in order to secure the image quality to the maximum.

(2) When RAM_C_HIGH_ THREHOLD is larger than or equal to T4 and larger than or equal to RAM_C_STANDARD_ THREHOLD, the JPEG compression rate is moderate, the residual space of the fourth random access memory RAM_C is proper, and the writing rate of the current block building module to the fourth random access memory RAM_C is kept.

(3) When ram_c_standby_ THREHOLD > T4, reducing the write rate of the previous module of the module where the random access memory is located to the random access memory, including:

s141: the determination block building block generates a clock cycle of the single image data.

S142: the image data in the random access sub-memory of the random access memory array of the block building block is read based on the read cycle with a plurality of clock cycles as the read cycle.

S143: and writing each image data into the random access memory of the image compression module in sequence.

It will be appreciated that when ram_c_standby_ THREHOLD > T4, indicating that the remaining space of the fourth random access memory ram_c is relatively small, then in order to reduce the compression efficiency, it is necessary to slow down the rate of image data output from the block building block, which is the preceding block, to the image compression block.

First, a clock period is determined at which the block building block generates single image data. It will be appreciated that in general, the BLOCK building BLOCK generates one Y_BLOCK data or U_BLOCK data or V_BLOCK data in one clock cycle, and one of the BLOCK data is read in one clock cycle. In order to slow down the rate of image data output from the BLOCK building BLOCK to the image compression BLOCK, the read-side logic of the third ram_array within the BLOCK building BLOCK is modified in this embodiment, and the image data in the random access sub-memory of the third ram_array of the BLOCK building BLOCK is read based on a plurality of clock cycles as a pair of read cycles, i.e., y_block data or u_block data or v_block data is read once every plurality of clock cycles. And finally, sequentially writing each image data into a fourth random access memory RAM_C of the image compression module. It should be noted that in some embodiments, Y_BLOCK data or U_BLOCK data or V_BLOCK data may be set to be read once every two clock cycles.

In summary, according to the actual running condition and the real-time compression rate of the buffer memory in the image compression module, the data reading strategy of the block building module and the data writing strategy of the image compression module are dynamically adjusted, so that the balance processing of the image data is realized.

The module where the random access memory is located is specifically a block building module.

At this time, the random access memory is specifically a third random access memory ram_array, the corresponding full time is T3, the corresponding first configuration threshold is ram_array_high_ THREHOLD, and the corresponding second configuration threshold is ram_array_standby_ THREHOLD.

(1) When T3> ram_array_high_ THREHOLD, increasing the write rate of the previous module of the module where the random access memory is located to the random access memory, including:

S123: image data of a plurality of consecutive addresses is read in a random access memory of the color conversion module according to a clock cycle.

S124: and splicing the image data read in the same clock period to obtain a plurality of combined image data.

S125: and writing each combined image data into the random access memory of the block building module in sequence according to the clock period.

It will be appreciated that when T3> ram_array_high_ THREHOLD indicates that the remaining space of the third random access memory ram_array is large, it is necessary to speed up the rate of image data output from the pre-stage module to the block building module by the color conversion module in order to improve compression efficiency. It should be noted that, in general, the color conversion module reads the second random access memory ram_b (24 bits wide) once in one clock cycle to obtain one YUV data.

Specifically, image data of a plurality of consecutive addresses is read in a random access memory of the color conversion module according to a clock cycle. That is, the image data of a plurality of continuous addresses in the second random access memory ram_b is read in one clock cycle, so as to obtain a plurality of YUV data, and the bit width of the YUV data is 24bits.

And further splicing the image data read in the same clock period to obtain a plurality of combined image data. That is, a plurality of YUV data are spliced to obtain a plurality of spliced combined image data. The bit width of the combined image data is the sum of the bit widths of a plurality of YUV data constituting the combined image data, and is an integer multiple of 24 bits.

And finally, according to the clock period, writing each combined image data into the random access memory of the block building module in sequence. That is, in a manner of writing one combined image data in one clock cycle, each combined image data is sequentially written into the third random access memory ram_array of the block building block, thereby realizing the writing of a plurality of adjacent address data in one clock cycle. For example, Y_0 is written to address 0 of Y data random access sub-memory 0, and Y_1 is written to address 1 of Y data random access sub-memory 0. It should be noted that in some embodiments, the two YUV data may be selectively concatenated to obtain the combined image data, where the combined image data has a bit width of 48bits.

(2) When RAM_ARRAY_HIGH_ THREHOLD is larger than or equal to T3 is larger than or equal to RAM_ARRAY_STANDARD_ THREHOLD, the compression rate is moderate, the residual space of the third random access memory RAM_ARRAY is proper, and the writing rate of the current color conversion module to the third random access memory RAM_ARRAY is kept.

(3) When ram_array_standby_ THREHOLD > T3, reducing the write rate of the previous module of the module where the random access memory is located to the random access memory, including:

s144: the clock period at which the color conversion module generates the individual image data is determined.

S145: the image data in the random access memory of the color conversion module is read based on the read cycle with a plurality of clock cycles as the read cycle.

S146: and writing each image data into the random access memory of the block building module in sequence.

It will be appreciated that when ram_array_standby_ THREHOLD > T3, indicating that the remaining space of the third random access memory ram_array is relatively small, then in order to reduce the compression efficiency, it is necessary to slow down the rate of image data output to the block building block by the color conversion block as the preceding block.

First, a clock period is determined at which the color conversion module generates single image data. It will be appreciated that in general, the color conversion module reads the second random access memory ram_b (24 bits wide) once in one clock cycle to obtain one YUV data. In order to slow down the rate of the image data output from the color conversion module to the block building module, the readout side logic of the second ram_b inside the color conversion module is modified in this embodiment, and the image data of the second ram_b of the color conversion module is read based on the read cycle with a plurality of clock cycles as the read cycle, that is, 24bits of YUV data is read once every a plurality of clock cycles. And finally, sequentially writing each image data into a third random access memory RAM_ARRAY of the block building module. It should be noted that in some embodiments, it may be provided that 24bits wide YUV data is read every two clock cycles.

In summary, according to the actual running condition and the real-time compression rate of the buffer memory in the block building module, the data reading strategy of the color conversion module and the data writing strategy of the block building module are dynamically adjusted, so that the balance processing of the image data is realized.

The module where the random access memory is located is specifically a color conversion module.

At this time, the random access memory is specifically a second random access memory ram_b, the corresponding full time is T2, the corresponding first configuration threshold is ram_b_high_ THREHOLD, and the corresponding second configuration threshold is ram_b_standby_ THREHOLD.

(1) When T2> ram_b_high_ THREHOLD, increasing the write rate of the previous module of the module where the random access memory is located to the random access memory, including:

S126: when the data capturing module is in the digital video interface mode or in the buffer mode, image data of a plurality of consecutive addresses is read in a random access memory of the data capturing module according to a clock cycle.

S127: and splicing the image data read in the same clock period to obtain a plurality of combined image data.

S128: and writing each combined image data into the random access memory of the color conversion module in sequence according to the clock period.

S129: when the data capture module is in the buffer mode, the method further comprises increasing a burst length of the data capture module reading the double rate synchronous dynamic random access memory.

It will be appreciated that when T2> ram_b_high_ THREHOLD indicates that the remaining space of the second random access memory ram_b is larger, the data capture module may choose to speed up the rate of image data output to the color conversion module in order to increase compression efficiency.

It should be noted that, since the data capturing module has two different modes, i.e. a digital video interface mode and a buffer mode, sources of the original RGB data in the two modes are different, so that different ways exist for speeding up the rate of the image data output from the data capturing module to the color conversion module. In addition, the data capture module typically reads the first random access memory ram_a (24 bits wide) once in one clock cycle to obtain one RGB data.

In one way, when the data capture module is in the digital video interface mode or in the buffer mode, image data of a plurality of consecutive addresses may be read in the random access memory of the data capture module according to a clock cycle. That is, the image data of a plurality of consecutive addresses in the first random access memory ram_a is read in one clock cycle to obtain a plurality of RGB data, and the bit width of the RGB data is 24bits.

And further splicing the image data read in the same clock period to obtain a plurality of combined image data. That is, a plurality of RGB data are spliced to obtain a plurality of spliced combined image data. The bit width of the combined image data is the sum of the bit widths of a plurality of RGB data constituting the combined image data, and is an integer multiple of 24 bits.

Finally, each combined image data is written into the random access memory of the color conversion module in sequence according to the clock period. That is, in a manner of writing one combined image data in one clock cycle, each combined image data is sequentially written into the second random access memory ram_b of the color conversion module, thereby realizing the writing of a plurality of adjacent address data in one clock cycle. It should be noted that, after receiving the combined image data composed of a plurality of RGB data, the color conversion module internally performs parallel color space conversion to calculate the conversion of a plurality of sets of RGB data at the same time. Furthermore, in some embodiments, the two RGB data may be optionally concatenated to obtain combined image data having a bit width of 48bits.

In another way, when the data capturing module is in the buffer mode, the Burst length (Burst) of the data capturing module for reading the DDR can be increased, so that the data capturing rate is increased, and the compression efficiency is improved.

(2) When RAM_B_HIGH_ THREHOLD is larger than or equal to T2 and larger than or equal to RAM_B_STANDARD_ THREHOLD, the compression rate is moderate, the residual space of the second random access memory RAM_B is proper, and the writing rate of the current data capturing module to the second random access memory RAM_B is kept.

(3) When ram_b_standby_ THREHOLD > T2, reducing the write rate of the previous module of the module where the random access memory is located to the random access memory, including:

S147: the clock period at which the data capture module generates the single image data is determined.

S148: the image data in the random access memory of the data capturing module is read based on the read cycle with a plurality of clock cycles as the read cycle.

S149: and writing each image data into the random access memory of the color conversion module in sequence.

S150: the burst length of the double-rate synchronous dynamic random access memory read by the data capturing module is reduced.

It will be appreciated that when ram_b_standby_ THREHOLD > T2, indicating that the remaining space of the second random access memory ram_b is relatively small, then in order to reduce the compression efficiency, it is necessary to slow down the rate of image data output to the color conversion module by the data capture module as a preceding module.

First, a clock period is determined at which the data capture module generates single image data. It will be appreciated that in general, the data capture module reads the first random access memory ram_a (24 bits wide) once in one clock cycle to obtain an RGB data. In order to slow down the rate of the image data output from the data capturing module to the color conversion module, the readout side logic of the first random access memory ram_a inside the data capturing module is modified in this embodiment, and the image data in the first random access memory ram_a of the data capturing module is read based on the read cycle with a plurality of clock cycles as the read cycle. That is, 24bits wide RGB data is read once every multiple clock cycles. And finally, sequentially writing each image data into a second random access memory RAM_B of the color conversion module. It should be noted that in some embodiments, it may be provided that 24bits wide RGB data is read every two clock cycles. On the basis, in this embodiment, the burst length of the data capturing module for reading the DDR needs to be reduced, so that the data capturing rate is further reduced.

In summary, according to the actual running condition and the real-time compression rate of the buffer memory in the color conversion module, the data reading strategy of the data capturing module and the data writing strategy of the color conversion module are dynamically adjusted, so that the balance processing of the image data is realized.

It is understood that the image data exists in a line blanking period and a field blanking period. The line blanking period occurs from line to line, the vertical blanking period occurs from frame to frame, and the line blanking period or the vertical blanking period is devoid of valid RGB data. The occurrence of the line blanking period and the field blanking period may be determined based on the data capture module.

Fig. 4 is a schematic diagram of a downlink blanking period and a vertical blanking period with 640×480 resolution according to an embodiment of the present invention. As shown in fig. 4, at 640×480 resolution, the line blanking period reaches 144 clock cycles, while the field blanking period is 29 lines, that is, no effective RGB data exists in the pixel clock cycles of 29 lines, 29×800=23200 clock cycles. This may occur when the buffer space inside the data capturing module and even the color conversion module is empty or there is little data, and the buffers inside the block building module and the image compression module will be full, resulting in an imbalance of video data in the overall buffer space. To solve this problem, in some embodiments, the method further includes, on the basis of the above embodiments:

S15: when the data acquisition module is determined to be in the line blanking period, the writing rate of the data acquisition module to the random access memory of the color conversion module is reduced, and the writing rate of the color conversion module to the random access memory of the block building module is reduced.

Specifically, when it is determined that the line blanking period is in, the first random access memory ram_a and the second random access memory ram_b are substantially in a state of very little data because the data capturing module is not inputted with the original RGB data at this time.

In order to avoid the imbalance of the video data in the whole buffer space, the read-side control logic of the first random access memory ram_a and the second random access memory ram_b needs to be changed, specifically, the write-in rate of the data capturing module to the second random access memory ram_b of the color converting module is reduced, the write-in rate of the color converting module to the third random access memory ram_array of the block building module is reduced, and finally, the write-in rates of the third random access memory ram_array and the fourth random access memory ram_c are slowed down, so that the video data is in a dynamic balanced state on the whole processing path.

Correspondingly, in some embodiments, the method further comprises:

S16: when the data capture module is determined to be in the vertical blanking period, the writing rate of the data capture module to the random access memory of the color conversion module is reduced, the writing rate of the color conversion module to the random access memory of the block building module is reduced, and the writing rate of the block building module to the random access memory of the image compression module is reduced.

Specifically, when the vertical blanking period occurs, the first random access memory ram_a, the second random access memory ram_b, and the third random access memory ram_array are substantially in a state of very little data because the data capture module is not inputted with the original RGB data at this time.

In order to avoid the imbalance of the video data in the whole buffer space, the read-side control logic of the first random access memory ram_a, the second random access memory ram_b and the third random access memory ram_array needs to be changed, specifically, the write-in rate of the data capturing module to the second random access memory ram_b of the color converting module is reduced, the write-in rate of the color converting module to the third random access memory ram_array of the block building module is reduced, the write-in rate of the block building module to the fourth random access memory ram_c of the image compressing module is reduced, and finally, the write-in rate of the fourth random access memory ram_c is reduced, so that the video data is in a dynamic balanced state on the whole processing path.

In summary, when in the line blanking period and the vertical blanking period, the balance of the video data on the whole processing path is realized by comprehensively scheduling the front and rear random access memories.

On the basis of the foregoing embodiments, in some embodiments, after reducing the writing rate of the data capturing module to the random access memory of the color conversion module, reducing the writing rate of the color conversion module to the random access memory of the block building module, and reducing the writing rate of the block building module to the random access memory of the image compression module, the method further includes:

S17: judging whether the residual space of the random access memory of the data capturing module and/or the residual space of the random access memory of the color conversion module and/or the residual space of the random access memory of the block building module is reduced in a preset period and reaches a corresponding space limit value; if yes, the process proceeds to step S18.

S18: and carrying out frame loss processing on the image data of the current frame.

In a specific implementation, after the writing rate of the data capturing module to the second random access memory ram_b of the color conversion module is reduced, the writing rate of the color conversion module to the third random access memory ram_array of the block building module is reduced, and the writing rate of the block building module to the fourth random access memory ram_c of the image compression module is reduced, it may be further determined whether the remaining space of the first random access memory ram_a of the data capturing module and/or the remaining space of the second random access memory ram_b of the color conversion module and/or the remaining space of the third random access memory ram_array of the block building module is reduced within a preset period, and the corresponding spatial limit is reached.

If the data is still being reduced and reaches the corresponding SPACE limit value space_ ALFULL, the residual SPACE of the memory is considered to be insufficient, the image data of the current frame cannot be captured continuously, and the frame loss processing is required to be carried out on the image data of the current frame, so that the data capture is reduced, the invalid compression is reduced, and the overall power consumption of the BMC chip is reduced.

In addition, if the remaining space of the random access memory of the data capturing module and/or the remaining space of the random access memory of the color conversion module and/or the remaining space of the random access memory of the block building module is determined to be reduced within the preset period and not reach the corresponding space limit, the method further comprises:

s19: monitoring a space reduction rate of the remaining space;

s20: when the space reduction rate reaches a preset rate, outputting alarm information;

S21: the input of raw image data to the data capture module is paused.

In a specific implementation, if the remaining space of the first ram_a of the data capturing module and/or the remaining space ram_b of the second RAM of the color conversion module and/or the remaining space of the third ram_array of the block building module is still decreasing within a preset period, but does not reach the corresponding space limit, it is considered that there is still a certain remaining space of the RAM, but a trend of decreasing the remaining space needs to be monitored, in particular, the rate of decreasing the space of the remaining space is monitored.

When the space reduction rate reaches the preset rate, the space reduction rate of the residual space is considered to be overlarge, and the alarm information is output at the moment after a short time, so that a worker is prompted to check the running condition of the corresponding random access memory in time, and corresponding measures are taken. Suspending the input of raw image data to the data capture module may be specifically selected to mitigate the reduction in the remaining space of the random access memory; the occupation of other modules in the BMC to read and write DDR can be selected to be suspended, so that DDR is only read and written by the BMC image data processing path, and the frame loss of image data is avoided.

It should be noted that the preset rate is not limited in this embodiment.

In the above embodiments, the image data processing method has been described in detail, and the present invention also provides a computer program product including a computer program/instruction which, when executed by a processor, performs the steps of the image data processing method described above.

Fig. 5 is a schematic diagram of an image data processing apparatus according to an embodiment of the present invention. As shown in fig. 5, the image data processing apparatus includes:

an obtaining module 10, configured to obtain, based on a preset period of time, a read average rate, a write average rate, and a remaining space of each random access memory in the baseboard management controller; wherein, each random access memory is respectively corresponding to a plurality of modules for processing image data in the baseboard management controller one by one, and respectively corresponding to and storing the image data processed by each module;

a determining module 11, configured to determine a full time corresponding to each random access memory according to the read average rate, the write average rate, and the remaining space of each random access memory;

The first adjustment module 12 is configured to increase the writing rate of the preceding module of the module where the random access memory is located to the random access memory when the full time of the random access memory is greater than the corresponding first configuration threshold;

The second adjustment module 13 is configured to maintain a write rate of the previous module of the module where the current random access memory is located to the random access memory when the full time of the random access memory is not greater than the corresponding first configuration threshold and not less than the corresponding second configuration threshold;

A third adjustment module 14, configured to reduce a writing rate of the preceding module of the module where the random access memory is located to the random access memory when the full time of the random access memory is less than the corresponding second configuration threshold; wherein the first configuration threshold is greater than the second configuration threshold.

In some embodiments, the determining module 11 includes:

The first acquisition sub-module is used for acquiring preset configuration parameters and determining the difference value between the writing average rate and the reading average rate of the random access memory;

The first determining submodule is used for determining the quotient of the residual space of the random access memory and the difference value;

and the second determining submodule is used for determining the product of the preset configuration parameter and the quotient value so as to obtain the corresponding full time of the random access memory.

In some embodiments, the first adjustment module 12 includes:

A first reading sub-module for reading image data of even pixel positions in each random access sub-memory based on the random access memory array of the block building module;

The first writing sub-module is used for sequentially writing the image data of each even pixel position into the random access memory of the image compression module in a mode of repeated writing for a plurality of times by taking single image data as a unit based on the data arrangement sequence of the image data of each even pixel position.

In some embodiments, the third adjustment module 14 includes:

a third determination sub-module for determining a clock period at which the block building module generates the single image data;

A second reading sub-module for reading the image data in the random access sub-memory of the random access memory array of the block building module based on the reading period with a plurality of clock periods as the reading period;

And the second writing sub-module is used for writing each image data into the random access memory of the image compression module in sequence.

In some embodiments, the first adjustment module 12 includes:

A third reading sub-module for reading image data of a plurality of continuous addresses in the random access memory of the color conversion module according to the clock period;

the first splicing sub-module is used for splicing the image data read in the same clock period to obtain a plurality of combined image data;

And the third writing sub-module is used for writing the combined image data into the random access memory of the block building module in sequence according to the clock period.

In some embodiments, the third adjustment module 14 includes:

A fourth determination sub-module for determining a clock period at which the color conversion module generates the single image data;

A fourth reading sub-module for reading the image data in the random access memory of the color conversion module based on the reading period with a plurality of clock periods as the reading period;

and the fourth writing sub-module is used for writing each image data into the random access memory of the block building module in sequence.

In some embodiments, the first adjustment module 12 includes:

A fifth reading sub-module for reading image data of a plurality of consecutive addresses in the random access memory of the data capturing module according to the clock period when the data capturing module is in the digital video interface mode or in the buffer mode;

The second splicing sub-module is used for splicing the image data read in the same clock period to obtain a plurality of combined image data;

A fifth writing sub-module, configured to sequentially write each combined image data into the random access memory of the color conversion module according to the clock period;

The adding sub-module is used for adding the burst length of the double-rate synchronous dynamic random access memory read by the data capturing module when the data capturing module is in the buffer mode.

In some embodiments, the third adjustment module 14 includes:

A fifth determining sub-module for determining a clock period at which the data capture module generates the single image data;

a sixth reading sub-module for reading the image data in the random access memory of the data capturing module based on the reading period with a plurality of clock periods as the reading period;

a sixth writing sub-module for sequentially writing each image data into the random access memory of the color conversion module;

the first reducing sub-module is used for reducing the burst length of the double-rate synchronous dynamic random access memory read by the data capturing module.

In some embodiments, further comprising:

and the second reducing sub-module is used for reducing the writing rate of the data capturing module to the random access memory of the color conversion module and reducing the writing rate of the color conversion module to the random access memory of the block building module when the data capturing module is determined to be in the line blanking period.

In some embodiments, further comprising:

And the third reducing sub-module is used for reducing the writing rate of the data capturing module to the random access memory of the color conversion module, reducing the writing rate of the color conversion module to the random access memory of the block building module and reducing the writing rate of the block building module to the random access memory of the image compression module when the third reducing sub-module is in the vertical blanking period.

In some embodiments, further comprising:

The judging sub-module is used for judging whether the residual space of the random access memory of the data capturing module and/or the residual space of the random access memory of the color conversion module and/or the residual space of the random access memory of the block building module is reduced in a preset period and reaches a corresponding space limit value; if yes, triggering a first processing sub-module;

And the processing sub-module is used for carrying out frame loss processing on the image data of the current frame.

In some embodiments, further comprising:

a monitoring sub-module for monitoring a rate of spatial reduction of the remaining space;

the alarm module is used for outputting alarm information when the space reduction rate reaches a preset rate;

and the pause module is used for pausing the input of the original image data of the data capturing module.

In this embodiment, the image data processing apparatus includes an acquisition module, a determination module, a first adjustment module, a second adjustment module, and a third adjustment module. The image data processing device is capable of implementing all the steps of the image data processing method described above when running. Specifically, according to the read average rate, the write average rate and the residual space of each random access memory in the BMC, the corresponding full time of each random access memory is respectively determined, which is equivalent to the actual running state of each module for processing the image data; the data processing strategy of the front module of the module where the random access memory is located is further adjusted according to the relation between the full time and the corresponding configuration threshold value, so that the writing rate of the random access memory of the current module is adjusted, dynamic unified adjustment among the image data processing rates of all modules on the image data processing path of the BMC is realized, the data loss of the rear module is avoided, and the frame loss rate of image data transmission is reduced.

Fig. 6 is a schematic diagram of an image data processing apparatus according to an embodiment of the present invention. As shown in fig. 6, the image data processing apparatus includes:

a memory 20 for storing a computer program.

A processor 21 for implementing the steps of the image data processing method as mentioned in the above embodiments when executing a computer program.

The image data processing device provided in this embodiment may include, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, or the like.

Processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The Processor 21 may be implemented in at least one hardware form of a digital signal Processor (DIGITAL SIGNAL Processor, DSP), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA), programmable logic array (Programmable Logic Array, PLA). The processor 21 may also include a main processor and a coprocessor, the main processor being a processor for processing data in an awake state, also referred to as a central processor (Central Processing Unit, CPU); a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 21 may integrate a graphics processor (Graphics Processing Unit, GPU) for rendering and drawing of content required to be displayed by the display screen. In some embodiments, the processor 21 may also include an artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) processor for processing computing operations related to machine learning.

Memory 20 may include one or more computer-readable storage media, which may be non-transitory. Memory 20 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used for storing a computer program 201, which, when loaded and executed by the processor 21, is capable of implementing the relevant steps of the image data processing method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 20 may further include an operating system 202, data 203, and the like, where the storage manner may be transient storage or permanent storage. Operating system 202 may include Windows, unix, linux, among other things. The data 203 may include, but is not limited to, data related to image data processing methods.

In some embodiments, the image data processing device may further include a display 22, an input/output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.

Those skilled in the art will appreciate that the structure shown in fig. 6 is not limiting of the image data processing apparatus and may include more or less components than illustrated.

In this embodiment, an image data processing apparatus includes a memory and a processor. The memory is used for storing a computer program. The processor is arranged to implement the steps of the image data processing method as mentioned in the above embodiments when executing the computer program. Specifically, according to the read average rate, the write average rate and the residual space of each random access memory in the BMC, the corresponding full time of each random access memory is respectively determined, which is equivalent to the actual running state of each module for processing the image data; the data processing strategy of the front module of the module where the random access memory is located is further adjusted according to the relation between the full time and the corresponding configuration threshold value, so that the writing rate of the random access memory of the current module is adjusted, dynamic unified adjustment among the image data processing rates of all modules on the image data processing path of the BMC is realized, the data loss of the rear module is avoided, and the frame loss rate of image data transmission is reduced.

Finally, the invention also provides a corresponding embodiment of the computer readable storage medium. The computer-readable storage medium has stored thereon a computer program which, when executed by a processor, performs the steps as described in the method embodiments above.

In this embodiment, a computer program is stored on a computer readable storage medium, and when the computer program is executed by a processor, the steps described in the above method embodiments are implemented. Specifically, according to the read average rate, the write average rate and the residual space of each random access memory in the BMC, the corresponding full time of each random access memory is respectively determined, which is equivalent to the actual running state of each module for processing the image data; the data processing strategy of the front module of the module where the random access memory is located is further adjusted according to the relation between the full time and the corresponding configuration threshold value, so that the writing rate of the random access memory of the current module is adjusted, dynamic unified adjustment among the image data processing rates of all modules on the image data processing path of the BMC is realized, the data loss of the rear module is avoided, and the frame loss rate of image data transmission is reduced.

Claims

1. An image data processing method, comprising:

Determining the corresponding full time of each random access memory according to the reading average rate, the writing average rate and the residual space of each random access memory; wherein, the full time is the time when the random access memory reaches the maximum storage critical value;

2. The image data processing method according to claim 1, wherein the determining the corresponding full time of each random access memory according to the read average rate, the write average rate, and the remaining space of each random access memory includes:

3. The method according to claim 2, wherein when the module of the ram is specifically an image compression module and the full time of the ram is greater than a corresponding first configuration threshold, the increasing the writing rate of the previous module of the ram to the ram includes:

4. The method according to claim 2, wherein when the module of the ram is specifically an image compression module and the full time of the ram is smaller than the corresponding second configuration threshold, the reducing the writing rate of the previous module of the ram to the ram includes:

5. The method according to claim 2, wherein when the module of the ram is a block building module and the full time of the ram is greater than the corresponding first configuration threshold, the increasing the writing rate of the preceding module of the ram to the ram includes:

6. The method according to claim 2, wherein when the module of the ram is a block building module and the full time of the ram is less than the corresponding second configuration threshold, the reducing the writing rate of the preceding module of the ram to the ram includes:

7. The method according to claim 2, wherein when the module of the ram is specifically a color conversion module and the full time of the ram is greater than the corresponding first configuration threshold, the increasing the writing rate of the preceding module of the ram to the ram includes:

8. The method according to claim 2, wherein when the module of the ram is a color conversion module and the full time of the ram is less than the corresponding second configuration threshold, the reducing the writing rate of the preceding module of the ram to the ram includes:

9. The image data processing method according to any one of claims 1 to 8, characterized by further comprising:

10. The image data processing method according to claim 9, characterized by further comprising:

11. The image data processing method according to claim 10, wherein after reducing a write rate of the data capturing module to the random access memory of the color conversion module, reducing a write rate of the color conversion module to the random access memory of the block building block, and reducing a write rate of the block building block to the random access memory of the image compression module, further comprising:

12. The image data processing method according to claim 11, wherein if the remaining space of the random access memory of the data capturing module and/or the remaining space of the random access memory of the color conversion module and/or the remaining space of the random access memory of the block building module is confirmed to be reduced within a predetermined period and the corresponding space limit is not reached, further comprising:

monitoring a space reduction rate of the remaining space;

The input of raw image data to the data capture module is paused.

13. A computer program product comprising computer programs/instructions which, when executed by a processor, implement the steps of the image data processing method of any one of claims 1 to 12.

14. An image data processing apparatus, characterized by comprising:

a memory for storing a computer program;

A processor for implementing the steps of the image data processing method according to any one of claims 1 to 12 when executing the computer program.

15. A computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the steps of the image data processing method according to any one of claims 1 to 12.