WO2023221217A1 - 一种数据处理方法及装置 - Google Patents

一种数据处理方法及装置 Download PDF

Info

Publication number
WO2023221217A1
WO2023221217A1 PCT/CN2022/098964 CN2022098964W WO2023221217A1 WO 2023221217 A1 WO2023221217 A1 WO 2023221217A1 CN 2022098964 W CN2022098964 W CN 2022098964W WO 2023221217 A1 WO2023221217 A1 WO 2023221217A1
Authority
WO
WIPO (PCT)
Prior art keywords
results
correlation
division
rearrangement
autocorrelation
Prior art date
Application number
PCT/CN2022/098964
Other languages
English (en)
French (fr)
Inventor
蒿杰
宋亚芳
方桂钿
梁俊
舒琳
卢进
范秋香
覃柱胜
Original Assignee
芯跳科技(广州)有限公司
广东人工智能与先进计算研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 芯跳科技(广州)有限公司, 广东人工智能与先进计算研究院 filed Critical 芯跳科技(广州)有限公司
Publication of WO2023221217A1 publication Critical patent/WO2023221217A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2455—Query execution
    • G06F16/24552—Database cache management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2457—Query processing with adaptation to user needs
    • G06F16/24578—Query processing with adaptation to user needs using ranking
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Definitions

  • This application relates to the field of resource allocation, specifically, to a data processing method and device.
  • the result obtained from antenna collection, filtering, and correlation processing is usually a real-time, high-speed data stream.
  • the data size transmitted per second during the transmission process often reaches several Gb or even more than ten Gb.
  • the cache processing of such a large amount of data not only affects the collection and processing performance of the entire system, but also means that the collection system needs to be equipped with huge storage space and hardware resources. Therefore, how to reasonably optimize and allocate system resources for big data caching has become one of the important issues in radio acquisition development work.
  • the purpose of the embodiments of the present application is to provide a data processing method and device that can reasonably optimize the allocation of system resources, thereby solving the problems of large resource occupancy and slow data processing speed encountered in the big data caching process.
  • the embodiment of the present application provides a data processing method, which may include:
  • each antenna in the multi-channel antenna includes multiple frequency points;
  • the autocorrelation partition and rearrangement results, the intra-group cross-correlation partition and rearrangement results and the inter-group cross-correlation partition and rearrangement results are subjected to correlation accumulation processing to obtain autocorrelation results, intra-group cross-correlation results and inter-group cross-correlation results. Relevant results;
  • the cached results are read in a loop.
  • this method can collect data from each antenna, then divide and rearrange the collected data according to the preset number of times and the preset number of groups, and then divide and rearrange the multiple division and rearrangement results. Perform correlation accumulation processing to obtain autocorrelation results, intra-group cross-correlation results and inter-group cross-correlation results; then, perform splicing and truncation caching of the auto-correlation results, intra-group cross-correlation results and inter-group cross-correlation results to obtain the cached results ; Finally, cache and read it to facilitate subsequent operations. It can be seen that implementing this implementation method can reasonably optimize the allocation of system resources, thereby solving the problems of large resource occupancy and slow data processing speed encountered in the big data caching process.
  • the collected data is divided and rearranged according to a preset number of times and a preset number of groups to obtain autocorrelation division and rearrangement results, intra-group cross-correlation division and rearrangement results and inter-group cross-correlation division and rearrangement results.
  • the resulting steps can include:
  • the autocorrelation division results and the cross-correlation division results are grouped into data groups according to the preset number of groups to obtain autocorrelation division and rearrangement results, intra-group cross-correlation division and rearrangement results and inter-group cross-correlation division and rearrangement results.
  • the method can first divide the data according to the preset number of times, and then divide and rearrange the data according to the preset number of groups. It can be seen that by implementing this implementation method, the collected data can be divided first and then divided and rearranged, thereby achieving the effect of rationally optimizing the allocation of system resources.
  • the step of dividing the collected data according to a preset number of times to obtain autocorrelation division results and cross-correlation division results may include:
  • the data of the same antenna and the same frequency point in the collected data are divided into data, and the autocorrelation division result is obtained;
  • the data between the same frequency points of different antennas in the collected data are divided according to the preset number of times to obtain a cross-correlation division result.
  • this method divides the collected data of the same antenna and the same frequency point in all frequency points, and obtains the autocorrelation division result; at the same time, it divides the data of the collected sub-data of the same frequency point of different antennas, and obtains the cross-correlation division result. It can be seen that by implementing this implementation method, the collected data corresponding to the N frequency points of the R-channel antenna can be reasonably divided, thereby obtaining two types of results: autocorrelation division results and cross-correlation division results, which is conducive to the subsequent steps of analyzing the two types of results. Different data grouping processing or different optimization processing.
  • the autocorrelation partitioning results and the cross-correlation partitioning results are grouped into data groups according to a preset number of groups to obtain autocorrelation partitioning and rearrangement results, intra-group cross-correlation partitioning and rearrangement results and inter-group cross-correlation results.
  • Steps to partition the rearranged results may include:
  • the cross-correlation division results are grouped according to the preset group number to obtain intra-group cross-correlation division and rearrangement results and inter-group cross-correlation division and rearrangement results.
  • this method can perform data grouping processing on both the autocorrelation division results and the cross-correlation division results according to the preset number of groups, and obtain the autocorrelation division and rearrangement results, the intra-group cross-correlation division and rearrangement results and the between-group Cross-correlation partitioning rearrangement results. It can be seen that by implementing this implementation method, three types of results can be obtained, so that the method can perform related accumulation processing on the three types of results, and can further use the hardware storage resource dual-port RAM inside the FPGA to obtain the related accumulation processing results. Caching data enables it to avoid parallel processing of large amounts of data and solve the problems of large resource footprint and slow data processing speed encountered in the process of big data caching.
  • the autocorrelation results, the intra-group cross-correlation results and the inter-group cross-correlation results are respectively subjected to splicing and truncation caching processing.
  • the step of obtaining the cached results may include:
  • the splicing and truncation results of each bit width are cached through different memories to obtain the cached results.
  • the method splices the autocorrelation results, intra-group cross-correlation results and inter-group cross-correlation results, and then performs 1 times, 2 times, 4 times, 8 times, 16 times and/or according to the bit width. Or truncation processing of 32 times the read data bit width, and the splicing and truncation results of different bit widths are cached separately. It can be seen that by implementing this implementation method, the multi-channel input related result data can be correspondingly truncated and spliced in frequency order according to the size of the FPGA hardware storage resources, so that it can be synchronized and cached, saving storage resources and solving the data problem. The problem of slow processing speed allows FPGA hardware storage resources to be fully utilized.
  • the splicing and truncation results of each bit width can be synchronized and cached in real time.
  • the autocorrelation partitioning result and/or the cross-correlation partitioning result is data with a large bit width and a small depth.
  • the embodiment of the present application also provides a data processing device.
  • the data processing device may include:
  • An acquisition unit configured to acquire collection data of multiple frequency points in a multi-channel antenna; each of the multi-channel antennas includes multiple frequency points;
  • a division and rearrangement unit configured to divide and rearrange the collected data according to a preset number of times and a preset number of groups to obtain autocorrelation division and rearrangement results, intra-group cross-correlation division and rearrangement results and inter-group mutual correlation results. Relevant division and rearrangement results;
  • a correlation accumulation unit configured to perform correlation accumulation processing on the autocorrelation partitioning and rearrangement results, the intra-group cross-correlation partitioning and rearrangement results, and the inter-group cross-correlation partitioning and rearrangement results, to obtain an autocorrelation result, Within-group cross-correlation results and between-group cross-correlation results;
  • a caching unit configured to perform splicing and truncation caching processing on the autocorrelation results, the intra-group cross-correlation results, and the inter-group cross-correlation results, respectively, to obtain cached results;
  • the reading unit is configured to read cached results cyclically according to a preset frequency sequence.
  • the data processing device can obtain the collection data of multiple frequency points in the multi-channel antenna through the acquisition unit; and divide the collection data according to the preset number of times and the preset number of groups through the division and rearrangement unit. rearrange to obtain the autocorrelation division rearrangement result, the intra-group cross-correlation division rearrangement result and the inter-group cross-correlation division rearrangement result; the autocorrelation division rearrangement result, the intra-group cross-correlation result are obtained through the correlation accumulation unit pair
  • the division and rearrangement results and the inter-group cross-correlation division and rearrangement results are subjected to correlation and accumulation processing to obtain the auto-correlation results, intra-group cross-correlation results and inter-group cross-correlation results; and then the auto-correlation results and intra-group cross-correlation results are obtained through the cache unit
  • the results and inter-group cross-correlation results are respectively spliced and intercepted and cached to obtain the cached results; finally,
  • the data partitioning and rearranging unit may include:
  • a dividing subunit configured to divide the collected data according to a preset number of times to obtain an autocorrelation division result and a cross-correlation division result;
  • the grouping subunit is configured to perform data grouping on the autocorrelation division results and the cross-correlation division results according to the preset group number, and obtain the autocorrelation division and rearrangement results, the intra-group cross-correlation division and rearrangement results and the group The results of cross-correlation partitioning and rearrangement.
  • the divided subunits may include:
  • An autocorrelation division module is configured to divide the data of the same antenna and the same frequency point in the collected data according to a preset number of times to obtain the autocorrelation division result;
  • the cross-correlation dividing module is configured to divide the data between the same frequency points of different antennas in the collected data according to a preset number of times to obtain the cross-correlation dividing result.
  • the grouping subunit may include:
  • An autocorrelation division and rearrangement module is configured to group the autocorrelation division results according to the preset group number to obtain the autocorrelation division and rearrangement results;
  • a cross-correlation division and rearrangement module configured to perform grouping processing on the cross-correlation division results according to the preset number of groups, and obtain the intra-group cross-correlation division and rearrangement results and the inter-group cross-correlation division and rearrangement results. Arrange results.
  • the data division and rearrangement unit can divide the collected data according to the preset number of times by dividing sub-units, and obtain the autocorrelation division results and cross-correlation division results; by dividing the sub-units, the data division and rearrangement unit can divide the collected data according to the preset times.
  • the number of groups groups the data of the autocorrelation division results and the cross-correlation division results, and obtains the autocorrelation division and rearrangement results, the intra-group cross-correlation division and rearrangement results and the inter-group cross-correlation division and rearrangement results.
  • the cache unit may include:
  • a processing subunit configured to perform splicing and truncation processing on the autocorrelation results, the intra-group cross-correlation results and the inter-group cross-correlation results respectively, to obtain splicing and truncation results of multiple bit widths;
  • the cache subunit is configured to cache the splicing and truncation results of each bit width through different memories to obtain cached results.
  • the cache unit can perform splicing and truncation processing on the autocorrelation results, the intra-group cross-correlation results and the inter-group cross-correlation results through the processing sub-unit, and obtain splicing truncation of multiple bit widths. bit result; the cache subunit is used to cache the splicing and truncation results of each bit width through different memories to obtain the cached result.
  • the multi-channel input related result data can be correspondingly truncated and spliced in frequency order according to the size of the FPGA hardware storage resources, so that it can be synchronized and cached, saving storage resources and solving the data problem. The problem of slow processing speed.
  • An embodiment of the present application also provides an electronic device.
  • the electronic device may include a memory and a processor.
  • the memory is used to store a computer program.
  • the processor runs the computer program to cause the electronic device to execute the implementation of the present application.
  • Embodiments of the present application also provide a computer-readable storage medium that stores computer program instructions.
  • the computer program instructions When the computer program instructions are read and run by a processor, any one of the first aspects of the embodiments of the present application is executed. the data processing methods described above.
  • Figure 1 is a schematic flowchart of a data processing method provided by some embodiments of the present application.
  • Figure 2 is a schematic structural diagram of a data processing device provided by other embodiments of the present application.
  • Figure 3 is a schematic structural diagram of a data processing device provided by some embodiments of the present application.
  • Figure 4 is a schematic diagram of the data processing flow of a data processing method provided by an embodiment of the present application.
  • Figure 1 is a schematic flowchart of a data processing method provided by some embodiments of the present application.
  • the data processing method may include the following steps S101-S109:
  • this step may be to obtain collection data of F different frequency points in each of the R antennas.
  • the method may further include:
  • the collected data are divided and rearranged according to the preset number of times and the preset number of groups, and the results of autocorrelation division and rearrangement, intra-group cross-correlation division and rearrangement results and inter-group cross-correlation division and rearrangement results are obtained.
  • Steps to rearrange results can include:
  • the step of dividing the collected data according to a preset number of times to obtain the autocorrelation division results and the cross-correlation division results may include steps S102 to S103 described later.
  • the autocorrelation division results and the cross-correlation division results are grouped according to the preset number of groups to obtain the autocorrelation division and rearrangement results, the intra-group cross-correlation division and rearrangement results and the inter-group cross-correlation results.
  • the step of dividing the rearrangement results may include steps S104 to S105 described later.
  • the preset number of times may be F/N times, where F may be the above-mentioned F frequency points, and N is the preset number of frequency points for a single processing.
  • this method can produce R*N autocorrelation division results each time. Determine W1 as the data bit width of each autocorrelation division result. Then, the data bit width of the R*N autocorrelation division results can be recorded as R*N*W1. Based on this, the total data bit width generated during F/N times of processing is R*N*W1*F/N.
  • R*N*W1 is the data bit width of the autocorrelation division result
  • F/N is the data depth
  • the data bit width of the result can be reduced by F/N times, and the depth can be increased by F/N times, and Choosing a smaller N can also achieve data conversion with smaller bit width and greater depth.
  • this step can generate R*F autocorrelation division results, and the total data bit width is R*F*W1.
  • W1 is the data bit width of each autocorrelation division result.
  • the result is a large bit width and small depth data.
  • this method can produce (R-1)*R/2*N cross-correlation division results each time. Determine W2 as the data bit width of each cross-correlation partition result. Then, the data bit width of the (R-1)*R/2*N cross-correlation division results can be recorded as R*(R-1)/2*N*W2. Based on this, the total data bit width generated during F/N times of processing is R*(R-1)/2*N*W2*F/N.
  • R*(R-1)/2*N*W2 is the data bit width of the cross-correlation division result
  • F/N is the data depth. Therefore, the data bit width of the result can be reduced by F/N times, and the depth By increasing F/N times and choosing a smaller N, data conversion with smaller bit width and greater depth can also be achieved.
  • this step can obtain (R-1)*R/2*F cross-correlation division results, and the total data bit width is R*(R-1)/2*F*W2.
  • W2 is the data bit width of each cross-correlation division result.
  • the result can be a large bit width and small depth data.
  • this method can group the N frequency points in the R/G antennas into G groups in order to obtain the autocorrelation division and rearrangement results, whose bit width is recorded as R/G*N*W1*(F/ N)*G.
  • the bit width of the autocorrelation partitioning and rearrangement result can be reduced by G times, and the data depth can be increased by G times.
  • S105 Group the cross-correlation division results according to the preset number of groups to obtain intra-group cross-correlation division and rearrangement results and inter-group cross-correlation division and rearrangement results.
  • this method can group the N frequency points in the R/G antenna in G groups in sequence, and obtain the intra-group cross-correlation partitioning and rearrangement results, whose bit width is recorded as R/G*(R/G- 1)/2*N*W2*(F/N)*G; at the same time, the cross-correlation partitioning and rearrangement results between groups can also be obtained, and the bit width is recorded as R/G*R/G*N*W2*(F /N)*(G*(G-1)/2).
  • the depth of the intra-group cross-correlation partitioning and rearrangement results is (F/N)*(G*(G-1)/2), which increases the depth of the intra-group cross-correlation partitioning and rearrangement results ( G*(G-1)/2).
  • G*(G-1)/2 the depth of each result in the intra-group cross-correlation partitioning and rearrangement results is increased by G times.
  • bit width of the cross-correlation partitioning and rearrangement results can be converted into two smaller bit width forms for subsequent processing. Among them, the depth of each result data will be increased by G times.
  • S106 Perform correlation accumulation processing on the autocorrelation division and rearrangement results, the intra-group cross-correlation division and rearrangement results and the inter-group cross-correlation division and rearrangement results to obtain the autocorrelation results, intra-group cross-correlation results and inter-group cross-correlation results.
  • step S106 After step S106, the following steps are also included: performing splicing and truncation caching processing on the autocorrelation results, intra-group cross-correlation results and inter-group cross-correlation results respectively to obtain cached results.
  • the autocorrelation results, intra-group cross-correlation results and inter-group cross-correlation results are respectively subjected to splicing and clipping caching processing.
  • the step of obtaining the cached results includes steps S107 to S108 described later.
  • the data bit widths of the above three related calculation results cannot be directly converted into the specific sending bit width M of the sending end.
  • the method truncates the multi-channel input related result data according to the size of the FPGA hardware storage resources and splices them in frequency order. Since the above design performs synchronous correlation calculations on N frequency points, it can be prioritized to determine whether the data bit width of one frequency point result of each of the above correlation results is greater than K32*M. If it is greater than the value, the result data will be truncated to obtain DATA_K32 with a bit width of K32*M and a depth of F/N*G. At the same time, a dual-port RAM with the corresponding bit width and depth will be instantiated to cache the result data.
  • the remaining data bit width will be The judgment will continue; if it is less than that, it will be spliced with the next frequency point result data of the same channel, and then it will be judged whether the splicing bit width is greater than K32*M, and the truncation operation will be performed accordingly. Until the result data of N frequency points calculated simultaneously are spliced and the splicing bit width is not greater than K32*M, then it is judged whether the remaining data bit width after truncation is greater than K16*M. If it is greater, the bit width obtained by truncation is K16* M, DATA_K16 with a depth of F/N*G is cached.
  • various combinations of DATA_K32, DATA_K16, DATA_K8, DATA_K4, DATA_K2 or DATA_K1 can be obtained according to the data bit width of different results, but the data depth is all F/N*G, so Complete multi-channel data splicing and truncation processing with large bit width and small depth. That is, the above three related results can be divided into K32*M*F/N*G, K16*M*F/N*G, K8*M*F/N*G, K4*M*F/N*G, The splicing and truncation results of K2*M*F/N*G or K1*M*F/N*G with various bit widths.
  • this method can synchronize and cache the splicing and clipping results in real time.
  • the system will instantiate RAM_K32, RAM_K16, RAM_K8, RAM_K4, RAM_K2 or RAM_K1 dual-port RAMs with different bit widths but the same depth according to the splicing and truncation of the above three related calculation result data, and according to the valid input of the following data stream Signal synchronization caches these data and obtains cached results.
  • the autocorrelation results, intra-group cross-correlation results and inter-group cross-correlation results can be separately spliced and clipped and cached to obtain cached results.
  • this method can perform a consistent operation on the above three related results through a read control logic that reads out the data of all cache RAMs one by one at a frequency, so that the output bit width of all cache RAMs is M .
  • a read control logic that reads out the data of all cache RAMs one by one at a frequency, so that the output bit width of all cache RAMs is M .
  • the synchronous calculation of the cached N frequency point related results will be completed.
  • the first step is to divide the data of N frequency points in the R antennas in F/N times in sequence; the second step is to divide the data of N frequency points in F/N times. Each result is divided into G groups and the data of N frequency points of the R/G antenna are sequentially calculated and divided and rearranged; the third step is to splice the truncated buffered data of each of the G results according to the number of frequency points N. Steps; The fourth step is the step of cyclically reading the data of each RAM_KX in the order of frequency points.
  • the execution subject of this method may be a computing device such as a computer or a server, which is not limited in this embodiment.
  • the execution subject of this method can also be a smart device such as a smartphone or a tablet computer, which is not limited in this embodiment.
  • the data in each antenna can be collected, and then the collected data can be divided and rearranged according to the preset number of times and the preset number of groups, and then the division and rearrangement can be performed.
  • the multiple data obtained by arranging are subjected to correlation and accumulation processing to obtain autocorrelation results, intra-group cross-correlation results and inter-group cross-correlation results; then, the auto-correlation results, intra-group cross-correlation results and inter-group cross-correlation results are spliced and intercepted. bit cache to obtain the cached result; finally, cache it and read it to facilitate subsequent operations. It can be seen that implementing this implementation method can reasonably optimize the allocation of system resources, thereby solving the problems of large resource occupancy and slow data processing speed encountered in the big data caching process.
  • Figure 2 is a schematic structural diagram of a data processing device provided by other embodiments of the present application. As shown in Figure 2, the data processing device includes:
  • the acquisition unit 210 is configured to acquire collection data of multiple frequency points in a multi-channel antenna; each antenna in the multi-channel antenna includes multiple frequency points;
  • the data division and rearrangement unit 220 is configured to perform data division and rearrangement of the collected data according to the preset number of times and the preset number of groups, and obtain the autocorrelation division and rearrangement results, intra-group cross-correlation division and rearrangement results and inter-group cross-correlation division and rearrangement results. Relevant division and rearrangement results;
  • the correlation accumulation unit 230 is configured to perform correlation accumulation processing on the autocorrelation division and rearrangement results, the intra-group cross-correlation division and rearrangement results, and the inter-group cross-correlation division and rearrangement results, to obtain the autocorrelation results and the intra-group cross-correlation results. and cross-correlation results between groups;
  • the caching unit 240 is configured to perform splicing and truncation caching processing on the autocorrelation results, intra-group cross-correlation results and inter-group cross-correlation results respectively, to obtain cached results;
  • the reading unit 250 is configured to read cached results cyclically according to a preset frequency sequence.
  • the implementation of the data processing device described in this embodiment can reasonably optimize the allocation of system resources, thereby solving the problems of large resource occupancy and slow data processing speed encountered in the big data caching process.
  • FIG. 3 is a schematic structural diagram of a data processing device provided by some embodiments of the present application. Among them, the data processing device shown in FIG. 3 is optimized from the data processing device shown in FIG. 2 . As shown in Figure 3, the data partitioning and rearranging unit 220 includes:
  • the division subunit 221 is configured to divide the collected data according to a preset number of times to obtain autocorrelation division results and cross-correlation division results;
  • the grouping subunit 222 is configured to perform data grouping on the autocorrelation division results and the cross-correlation division results according to the preset number of groups, and obtain the autocorrelation division and rearrangement results, the intra-group cross-correlation division and rearrangement results and the inter-group cross-correlation results. Divide and rearrange the results.
  • the dividing subunit 221 includes:
  • the autocorrelation division module is configured to divide the data of the same antenna and the same frequency point in the collected data according to a preset number of times to obtain the autocorrelation division result;
  • the cross-correlation division module is configured to divide the data between the same frequency points of different antennas in the collected data according to a preset number of times to obtain a cross-correlation division result.
  • the grouping subunit 222 includes:
  • the autocorrelation division and rearrangement module is configured to group the autocorrelation division results according to the preset group number to obtain the autocorrelation division and rearrangement results;
  • the cross-correlation partitioning and rearranging module is configured to group the cross-correlation partitioning results according to the preset number of groups, and obtain the intra-group cross-correlation partitioning and rearranging results and the inter-group cross-correlation partitioning and rearranging results.
  • the cache unit 240 includes:
  • the processing subunit 241 is configured to perform splicing and truncation processing on the autocorrelation results, intra-group cross-correlation results and inter-group cross-correlation results respectively, to obtain splicing and truncation results of multiple bit widths;
  • the cache subunit 242 is configured to cache the splicing and truncation results of each bit width through different memories to obtain cached results.
  • the implementation of the data processing device described in this embodiment can reasonably optimize the allocation of system resources, thereby solving the problems of large resource occupancy and slow data processing speed encountered in the big data caching process.
  • Embodiments of the present application provide an electronic device, including a memory and a processor.
  • the memory is used to store a computer program.
  • the processor runs the computer program to cause the electronic device to execute any one of the data processing methods in Embodiment 1 or 2 of the present application. .
  • Embodiments of the present application provide a computer-readable storage medium that stores computer program instructions. When the computer program instructions are read and run by a processor, any data processing in Embodiment 1 or Embodiment 2 of the present application is performed. method.
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more components for implementing the specified logical function(s). Executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures.
  • each block of the block diagram and/or flowchart illustration, and combinations of blocks in the block diagram and/or flowchart illustration can be implemented by special purpose hardware-based systems that perform the specified functions or acts. , or can be implemented using a combination of specialized hardware and computer instructions.
  • each functional module in each embodiment of the present application can be integrated together to form an independent part, each module can exist alone, or two or more modules can be integrated to form an independent part.
  • the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium.
  • the technical solution of the present application is essentially or the part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program code. .
  • This application provides a data processing method and device.
  • the method and device can collect data from each antenna, and then divide and rearrange the collected data according to the preset number of times and the preset number of groups, and then divide and rearrange the data.
  • the obtained multiple division and rearrangement results are subjected to correlation and accumulation processing to obtain autocorrelation results, intra-group cross-correlation results and inter-group cross-correlation results; then, the auto-correlation results, intra-group cross-correlation results and inter-group cross-correlation results are obtained.
  • Splice the truncation cache to get the cached result; finally, cache and read it to facilitate subsequent operations, so that system resources can be reasonably optimized and configured to solve the problems of large resource footprint and slow data processing speed encountered in the big data caching process. The problem.
  • the data processing method and device of the present application are reproducible and can be used in a variety of industrial applications.
  • the data processing method and device of the present application can be used in the field of resource allocation.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Radar Systems Or Details Thereof (AREA)
  • Variable-Direction Aerials And Aerial Arrays (AREA)

Abstract

本申请实施例提供一种数据处理方法及装置，该数据处理方法包括：获取多路天线中多个频点的采集数据；根据预设次数和预设组数对采集数据进行数据划分重排，得到自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果；对上述三种划分重排结果进行相关累加处理，得到自相关结果、组内互相关结果和组间互相关结果；再对自相关结果、组内互相关结果和组间互相关结果分别进行拼接截位缓存处理，得到缓存结果；根据预设的频点顺序，循环读取缓存结果。可见，实施这种实施方式，能够合理优化配置系统资源，从而解决大数据缓存过程中遇到的资源占用空间大和数据处理速度慢的问题。

Description

一种数据处理方法及装置
相关申请的交叉引用
本申请要求于2022年05月19日提交中国国家知识产权局的申请号为202210554412.4、名称为“一种数据处理方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及资源配置领域，具体而言，涉及一种数据处理方法及装置。
背景技术
目前，在大型射电阵列数字信号处理系统中，由天线采集、滤波到经相关处理后所得到的结果通常是一种实时、高速的数据流。其中，传输过程中每秒传输的数据大小往往会达到几个Gb甚至十几个Gb。在实践中发现，这样大量数据的缓存处理不仅影响着整个系统的采集处理性能，还意味着采集系统需要配备巨大的存储空间和硬件资源。因此，如何合理优化配置系统资源以进行大数据缓存成为了射电采集开发工作的重要问题之一。
发明内容
本申请实施例的目的在于提供一种数据处理方法及装置，能够合理优化配置系统资源，从而解决大数据缓存过程中遇到的资源占用空间大和数据处理速度慢的问题。
本申请实施例提供了一种数据处理方法，所述方法可以包括：
获取多路天线中多个频点的采集数据；所述多路天线中每路天线都包括多个频点；
根据预设次数和预设组数对所述采集数据进行数据划分重排，得到自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果；
对所述自相关划分重排结果、所述组内互相关划分重排结果和所述组间互相关划分重排结果进行相关累加处理，得到自相关结果、组内互相关结果和组间互相关结果；
对所述自相关结果、所述组内互相关结果和所述组间互相关结果分别进行拼接截位缓存处理，得到缓存结果；
根据预设的频点顺序，循环读取缓存结果。
在上述实现过程中，该方法可以采集每路天线中的数据，然后根据预设次数和预设组数对采集数据进行数据划分重排，然后再对划分重排得到的多个划分重排结果进行相关累加处理，得到自相关结果、组内互相关结果和组间互相关结果；再后，对自相关结果、组内互相关结果和组间互相关结果进行拼接截位缓存，得到缓存结果；最后，再对其进行缓存读取，以便于后续操作。可见，实施这种实施方式，能够合理优化配置系统资源，从而解决大数据缓存过程中遇到的资源占用空间大和数据处理速度慢的问题。
可选地，所述根据预设次数和预设组数对所述采集数据进行数据划分重排，得到自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果的步骤可以包括：
根据预设次数对所述采集数据进行数据划分，得到自相关划分结果和互相关划分结果；
根据预设组数对所述自相关划分结果和所述互相关划分结果进行数据分组，得到自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果。
在上述实现过程中，该方法可以先按照预设次数对数据进行划分，然后再按照预设组数对数据进行划分重排。可见，实施这种实施方式，能够对采集数据先进行划分在进行划分重排，从而能够实现合理优化配置系统资源的效果。
可选地，所述根据预设次数对所述采集数据进行数据划分，得到自相关划分结果和互相关划分结果的步骤可以包括：
根据预设次数对所述采集数据中相同天线相同频点的数据进行数据划分，得到自相关划分结果；
根据所述预设次数对所述采集数据中不同天线相同频点间的数据进行数据划分，得到互相关划分结果。
在上述实现过程中，该方法对所有频点中相同天线相同频点的采集数据进行数据划分，得到自相关划分结果；同时对不同天线相同频点的采集子数据进行数据划分，得到互相关划分结果。可见，实施这种实施方式，能够对R路天线N个频点对应的采集数据进行合理划分，从而得到自相关划分结果和互相关划分结果两类结果，从而有利于后续步骤对两类结果进行不同的数据分组处理或不同的优化处理。
可选地，所述根据预设组数对所述自相关划分结果和所述互相关划分结果进行数据分组，得到自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果的步骤可以包括：
根据预设组数对所述自相关划分结果进行分组处理，得到自相关划分重排结果；
根据所述预设组数对所述互相关划分结果进行分组处理，得到组内互相关划分重排结果和组间互相关划分重排结果。
在上述实现过程中，该方法可以根据预设组数对自相关划分结果和互相关划分结果两者进行数据分组处理，得到自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果。可见，实施这种实施方式，能够获取到三类结果，从而使得该方法可以对该三类结果进行相关累加处理，并可以进一步使用FPGA内部的硬件存储资源双端口RAM对得到的相关累加处理结果进行数据缓存，进而使得其能够避免大量数据的并行处理，并解决大数据缓存过程中遇到的资源占用空间大和数据处理速度慢的问题。
可选地，所述对所述自相关结果、所述组内互相关结果和所述组间互相关结果分别进 行拼接截位缓存处理，得到缓存结果的步骤可以包括：
对所述自相关结果、所述组内互相关结果和所述组间互相关结果分别进行拼接截位处理，得到多种位宽的拼接截位结果；
通过不同的存储器对每种位宽的拼接截位结果进行缓存，得到缓存结果。
在上述实现过程中，该方法对自相关结果、组内互相关结果和组间互相关结果进行了拼接处理，然后根据位宽进行1倍、2倍、4倍、8倍、16倍和/或32倍读出数据位宽的截位处理，并对不同位宽的拼接截位结果进行分别缓存。可见，实施这种实施方式，能够根据FPGA硬件存储资源大小对多路输入相关结果数据进行相应的截位和按频点顺序拼接的处理，从而能够对其进行同步缓存，节省存储资源并解决数据处理速度慢的问题，进而使得FPGA硬件存储资源能够被充分利用。
可选地，可以对每种位宽的拼接截位结果进行实时同步缓存。
可选地，所述自相关划分结果和/或所述互相关划分结果是大位宽小深度的数据。
本申请实施例还提供了一种数据处理装置，所述数据处理装置可以包括：
获取单元，被配置成用于获取多路天线中多个频点的采集数据；所述多路天线中每路天线都包括多个频点；
划分重排单元，被配置成用于根据预设次数和预设组数对所述采集数据进行数据划分重排，得到自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果；
相关累加单元，被配置成用于对所述自相关划分重排结果、所述组内互相关划分重排结果和所述组间互相关划分重排结果进行相关累加处理，得到自相关结果、组内互相关结果和组间互相关结果；
缓存单元，被配置成用于对所述自相关结果、所述组内互相关结果和所述组间互相关结果分别进行拼接截位缓存处理，得到缓存结果；
读取单元，被配置成用于根据预设的频点顺序，循环读取缓存结果。
在上述实现过程中，数据处理装置可以通过获取单元来获取多路天线中多个频点的采集数据；通过划分重排单元来根据预设次数和预设组数对所述采集数据进行数据划分重排，得到自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果；通过相关累加单元对对所述自相关划分重排结果、所述组内互相关划分重排结果和所述组间互相关划分重排结果进行相关累加处理，得到自相关结果、组内互相关结果和组间互相关结果；再通过缓存单元对自相关结果、组内互相关结果和组间互相关结果分别进行拼接截位缓存处理，得到缓存结果；最后，通过读取单元来根据预设的频点顺序，循环读取缓存结果。可见，实施这种实施方式，能够合理优化配置系统资源，从而解决大数据缓存过程中遇到的资源占用空间大和数据处理速度慢的问题。
可选地，所述数据划分重排单元可以包括：
划分子单元，被配置成用于根据预设次数对所述采集数据进行数据划分，得到自相关划分结果和互相关划分结果；
分组子单元，被配置成用于根据预设组数对所述自相关划分结果和所述互相关划分结果进行数据分组，得到自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果。
可选地，所述划分子单元可以包括：
自相关划分模块，被配置成用于根据预设次数对所述采集数据中相同天线相同频点的数据进行数据划分，得到所述自相关划分结果；
互相关划分模块，被配置成用于根据预设次数对所述采集数据中不同天线相同频点间的数据进行数据划分，得到所述互相关划分结果。
可选地，所述分组子单元可以包括：
自相关划分重排模块，被配置成用于根据所述预设组数对所述自相关划分结果进行分组处理，得到所述自相关划分重排结果；
互相关划分重排模块，被配置成用于根据所述预设组数对所述互相关划分结果进行分组处理，得到所述组内互相关划分重排结果和所述组间互相关划分重排结果。
在上述实现过程中，该数据划分重排单元可以通过划分子单元来根据预设次数对采集数据进行数据划分，得到自相关划分结果和互相关划分结果；通过划分重排子单元来根据预设组数对自相关划分结果和互相关划分结果进行数据分组，得到自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果。可见，实施这种实施方式，能够对采集数据进行划分重排，从而能够实现合理优化配置系统资源的效果。
可选地，所述缓存单元可以包括：
处理子单元，被配置成用于对所述自相关结果、所述组内互相关结果和所述组间互相关结果分别进行拼接截位处理，得到多种位宽的拼接截位结果；
缓存子单元，被配置成用于通过不同的存储器对每种位宽的拼接截位结果进行缓存，得到缓存结果。
在上述实现过程中，缓存单元可以通过处理子单元对所述自相关结果、所述组内互相关结果和所述组间互相关结果分别进行拼接截位处理，得到多种位宽的拼接截位结果；通过缓存子单元来通过不同的存储器对每种位宽的拼接截位结果进行缓存，得到缓存结果。可见，实施这种实施方式，能够根据FPGA硬件存储资源大小对多路输入相关结果数据进行相应的截位和按频点顺序拼接的处理，从而能够对其进行同步缓存，节省存储资源并解决数据处理速度慢的问题。
本申请实施例还提供了一种电子设备，该电子设备可以包括存储器以及处理器，所述存储器用于存储计算机程序，所述处理器运行所述计算机程序以使所述电子设备执行本申请实施例第一方面中任一项所述的数据处理方法。
本申请实施例还提供了一种计算机可读存储介质，其存储有计算机程序指令，所述计算机程序指令被一处理器读取并运行时，执行本申请实施例第一方面中任一项所述的数据处理方法。
附图说明
为了更清楚地说明本申请实施例的技术方案，下面将对本实施例中所需要使用的附图作简单地介绍，应当理解，以下附图仅示出了本申请的某些实施例，因此不应被看作是对范围的限定，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。
图1为本申请的一些实施例提供的一种数据处理方法的流程示意图；
图2为本申请的另一些实施例提供的一种数据处理装置的结构示意图；
图3为本申请的又一些实施例提供的一种数据处理装置的结构示意图；
图4为本申请实施例提供的一种数据处理方法的数据处理流程示意图。
具体实施方式
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整的描述。显然，所描述的实施例是本公开的一部分实施例，而不是全部的实施例。因此，以下对在附图中提供的本公开的实施例的详细描述并非旨在限制要求保护的本公开的范围，而是仅仅表示本公开的选定实施例。基于本公开中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步定义和解释。同时，在本申请的描述中，术语“第一”、“第二”等仅用于区分描述，而不能理解为指示或暗示相对重要性。
请参看图1，图1为本申请的一些实施例提供的一种数据处理方法的流程示意图。如图1所示，其中，该数据处理方法可以包括以下步骤S101-S109：
S101、获取多路天线中多个频点的采集数据；多路天线中每路天线都包括多个频点。
可选地，该步骤可以为获取R路天线中每路天线中F个不同频点的采集数据。
作为一种可选的实施方式，步骤S101之后，该方法还可以包括：
根据预设次数和预设组数对采集数据进行数据划分重排，得到自相关划分重排结果、 组内互相关划分重排结果和组间互相关划分重排结果。
作为一种进一步可选的实施方式，根据预设次数和预设组数对采集数据进行数据划分重排，得到自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果的步骤可以包括：
根据预设次数对采集数据进行数据划分，得到自相关划分结果和互相关划分结果；
根据预设组数对自相关划分结果和互相关划分结果进行数据分组，得到自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果。
作为一种进一步可选的实施方式，根据预设次数对采集数据进行数据划分，得到自相关划分结果和互相关划分结果的步骤可以包括稍后描述的步骤S102～S103。
作为一种进一步可选的实施方式，根据预设组数对自相关划分结果和互相关划分结果进行数据分组，得到自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果的步骤可以包括稍后描述的步骤S104～S105。
S102、根据预设次数对采集数据中相同天线相同频点的数据进行数据划分，得到自相关划分结果。
可选地，预设次数可以为F/N次，其中F可以为上述的F个频点，N为预设的单次处理频点数量。
可选地，该方法可以将F个不同频点数据分成F/N次顺序处理，以使该方法在后续的处理过程中分多次对N(N＝1，2，3…)个频点进行并行处理，从而减少原本所需并行处理的频点数量。在上述过程中，该方法可以每次都产生R*N个自相关划分结果。将W1确定为每个自相关划分结果的数据位宽。那么，该R*N个自相关划分结果的数据位宽便可以记为R*N*W1。基于此，F/N次的处理过程中将产生的总数据位宽为R*N*W1*F/N。其中，R*N*W1为自相关划分结果的数据位宽，F/N为数据深度，由此便可将结果的数据位宽减小F/N倍，深度增大F/N倍，并且选择越小的N，还可以实现更小位宽和更大深度的数据转换。
可选地，该步骤可以得到产生R*F个自相关划分结果，总数据位宽为R*F*W1。其中W1为每个自相关划分结果的数据位宽。
可选地，该结果是一种大位宽小深度的数据。
S103、根据预设次数对采集数据中不同天线相同频点间的数据进行数据划分，得到互相关划分结果。
可选地，该方法可以将F个不同频点数据分成F/N次顺序处理，以使该方法在后续的处理过程中分多次对N(N＝1，2，3…)个频点进行并行处理，从而减少原本所需并行处理的频点数量。在上述过程中，该方法可以每次都将产生(R-1)*R/2*N个互相关划分结 果。将W2确定为每个互相关划分结果的数据位宽。那么，该(R-1)*R/2*N个互相关划分结果的数据位宽便可以记为R*(R-1)/2*N*W2。基于此，F/N次的处理过程中将产生的总数据位宽为R*(R-1)/2*N*W2*F/N。其中，R*(R-1)/2*N*W2为互相关划分结果的数据位宽，F/N为数据深度，由此便可将结果的数据位宽减小F/N倍，深度增大F/N倍，并且选择越小的N，还可以实现更小位宽和更大深度的数据转换。
可选地，该步骤可以得到(R-1)*R/2*F个互相关划分结果，总数据位宽为R*(R-1)/2*F*W2。其中，W2为每个互相关划分结果的数据位宽。
可选地，该结果可以是一种大位宽小深度的数据。
S104、根据预设组数对自相关划分结果进行分组处理，得到自相关划分重排结果。
可选地，由于大型射电阵列采集系统往往需要处理几十路甚至几百路的天线数据，这也导致了产生的相关结果数据位宽极大。基于此，本方法可以将R路天线拆分为G(G＝2，3…)组处理，其中，G为预设组数。
可选地，该方法可以分G组顺序对R/G路天线中N个频点进行分组处理，得到自相关划分重排结果，其位宽记为R/G*N*W1*(F/N)*G。
实施这种实施方式，能够将自相关划分重排结果的位宽缩小G倍，并将数据深度增大G倍。
S105、根据预设组数对互相关划分结果进行分组处理，得到组内互相关划分重排结果和组间互相关划分重排结果。
本实施例中，由于大型射电阵列采集系统往往需要处理几十路甚至几百路的天线数据，这也导致了产生的相关结果数据位宽极大。基于此，本方法可以将R路天线拆分为G(G＝2，3…)组处理，其中，G为预设组数。
可选地，该方法可以分G组顺序对R/G路天线中N个频点进行分组处理，得到组内互相关划分重排结果，其位宽记为R/G*(R/G-1)/2*N*W2*(F/N)*G；同时，还能够得到组间互相关划分重排结果，其位宽记为R/G*R/G*N*W2*(F/N)*(G*(G-1)/2)。
可选地，该组内互相关划分重排结果的深度为(F/N)*(G*(G-1)/2)，这就使得组内互相关划分重排结果的深度增大(G*(G-1)/2)。这里因为存在(G-1)/2个组，所以组内互相关划分重排结果中每个结果的深度都增大G倍。
实施这种实施方式，能够将互相关划分重排结果位宽转变成两种更小位宽的形式，以便后续处理。其中，每种结果数据的深度都将增大G倍。
S106、对自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果进行相关累加处理，得到自相关结果、组内互相关结果和组间互相关结果。
在步骤S106之后，还包括以下步骤：对自相关结果、组内互相关结果和组间互相关结 果分别进行拼接截位缓存处理，得到缓存结果。
作为一种可选的实施方式，对自相关结果、组内互相关结果和组间互相关结果分别进行拼接截位缓存处理，得到缓存结果的步骤包括稍后描述的步骤S107～S108。
S107、对自相关结果、组内互相关结果和组间互相关结果分别进行拼接截位处理，得到多种位宽的拼接截位结果。
本实施例中，上述三种相关计算结果的数据位宽均无法直接转换成发送端特定的发送位宽M，使用FPGA内部的硬件存储资源双端口RAM进行数据缓存，需要通过将其数据位宽补零拓展到K*M(K＝1，2，4，8，16，32)才实现，这不仅浪费FPGA的存储资源，更是需要大量消耗FPGA的布线和逻辑资源，使得FPGA的面积过于庞大，并且存在相关结果的数据位宽过大的情况，无法直接进行缓存和位宽转换。
本实施例中，FPGA的硬件存储资源BLOCK RAM要完成M位宽的输出位宽转换时，其输入位宽需要是K*M(K＝1，2，4，8，16，32)，当K取相应倍数时记为K1，K2，K4，K8，K16和K32。
本实施例中，该方法根据FPGA硬件存储资源大小对多路输入相关结果数据进行相应的截位和按频点顺序拼接的处理方法。由于上述设计进行N个频点的同步相关计算，可以优先判断上述各路相关结果的1个频点结果数据位宽是否大于K32*M。若大于则将结果数据截位得到位宽为K32*M，深度为F/N*G的DATA_K32，同时例化相应位宽和深度的双端口RAM进行结果数据的缓存，剩下的数据位宽将继续进行判断；若小于则和相同一路的下一个频点结果数据拼接，再判断拼接位宽是否大于K32*M，并相应进行截位操作。直到同步计算的N个频点结果数据都拼接完并且拼接位宽没有大于K32*M，接着判断截位后剩余的数据位宽是否大于K16*M，若大于则截位得到位宽为K16*M，深度为F/N*G的DATA_K16进行缓存。
在本实施例，根据上述方法可以根据不同结果的数据位宽情况，相应可以得到DATA_K32、DATA_K16、DATA_K8、DATA_K4、DATA_K2或者DATA_K1的多种组合情况，但数据深度都是F/N*G，便完成多路大位宽小深度数据拼接截位处理。即上述三种相关结果可以截位分成K32*M*F/N*G、K16*M*F/N*G、K8*M*F/N*G、K4*M*F/N*G、K2*M*F/N*G或K1*M*F/N*G多种位宽的拼接截位结果。
S108、通过不同的存储器对每种位宽的拼接截位结果进行缓存，得到缓存结果。
可选地，该方法可以对拼接截位结果进行实时同步缓存。系统将根据上述三种相关计算结果数据的拼接截位情况，相应例化RAM_K32、RAM_K16、RAM_K8、RAM_K4、RAM_K2或RAM_K1的不同位宽却相同深度的双端口RAM，并根据跟随数据流输入的有效信号同步将这些数据进行缓存，得到缓存结果。
本实施例中，实施上述步骤S106～步骤S107，能够对自相关结果、组内互相关结果和组间互相关结果分别进行拼接截位缓存处理，得到缓存结果。
S109、根据预设的频点顺序，循环读取缓存结果。
可选地，该方法可以通过一种将所有缓存RAM的数据以频点逐一读出的方式的读控制逻辑，对上述三种相关结果进行一致性操作，使得全部缓存RAM的输出位宽为M。同时，对同一种相关结果的可能例化的RAM_K32、RAM_K16、RAM_K8、RAM_K4、RAM_K2或RAM_K1的缓存RAM，设计将RAM_KX(X＝32，16，8，4，2，1)读出KX次M位宽数据，再继续读KX次下一个缓存RAM_KX的M位宽数据，读完所有缓存RAM_KX则完成同步计算缓存的N个频点相关结果，循环读完F/N*G次后则完成同一种相关计算的所有频点结果的读出，直到上述三种相关结果所有频点全部读出，这样便实现了数据位宽为M的大深度小位宽的实时数据流输出。
实施这种实施方式，能够通过该种新型的大数据缓存处理方法很好地完成了系统的存储资源的合理分配，避免大位宽数据的冗余处理，同时对缓存RAM读出逻辑的合理实现，极大简化大位宽小深度的数据处理过程和FPGA硬件实现。
请参阅图4，图4提供了一种数据处理方法的数据处理流程示意图。由图4可知，对应于本实施例中的描述，其第一步为分F/N次顺序对R路天线中N个频点的数据进行划分的步骤；第二步为对F/N个结果的每一个结果分G组顺序计算R/G天线N频点的数据进行划分重排的步骤；第三步为对G个结果的每一个结果按频点N的个数拼接截位缓存的步骤；第四步为按频点的顺序循环读出各个RAM_KX的数据的步骤。
可选地，该方法的执行主体可以为计算机、服务器等计算装置，对此本实施例中不作任何限定。
可选地，该方法的执行主体还可以为智能手机、平板电脑等智能设备，对此本实施例中不作任何限定。
可见，实施本实施例所描述的数据处理方法，能够对每路天线中的数据采集，然后根据预设次数和预设组数对采集到的采集数据进行数据划分重排，然后再对划分重排得到的多个数据进行相关累加处理，得到自相关结果、组内互相关结果和组间互相关结果；再然后，对自相关结果、组内互相关结果和组间互相关结果进行拼接截位缓存，得到缓存结果；最后，再对其进行缓存读取，以便于后续操作。可见，实施这种实施方式，能够合理优化配置系统资源，从而解决大数据缓存过程中遇到的资源占用空间大和数据处理速度慢的问题。
请参看图2，图2为本申请的另一些实施例提供的一种数据处理装置的结构示意图。如 图2所示，该数据处理装置包括：
获取单元210被配置成用于获取多路天线中多个频点的采集数据；多路天线中每路天线都包括多个频点；
数据划分重排单元220，被配置成用于根据预设次数和预设组数对采集数据进行数据划分重排，得到自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果；
相关累加单元230，被配置成用于对自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果进行相关累加处理，得到自相关结果、组内互相关结果和组间互相关结果；
缓存单元240，被配置成用于对自相关结果、组内互相关结果和组间互相关结果分别进行拼接截位缓存处理，得到缓存结果；
读取单元250，被配置成用于根据预设的频点顺序，循环读取缓存结果。
本实施例中，对于数据处理装置的解释说明可以参照实施例1中的描述，对此本实施例中不再多加赘述。
可见，实施本实施例所描述的数据处理装置，能够合理优化配置系统资源，从而解决大数据缓存过程中遇到的资源占用空间大和数据处理速度慢的问题。
请一并参阅图3，图3是本申请的又一些实施例提供的一种数据处理装置的结构示意图。其中，图3所示的数据处理装置是由图2所示的数据处理装置进行优化得到的。如图3所示，数据划分重排单元220包括：
划分子单元221，被配置成用于根据预设次数对采集数据进行数据划分，得到自相关划分结果和互相关划分结果；
分组子单元222，被配置成用于根据预设组数对自相关划分结果和互相关划分结果进行数据分组，得到自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果。
作为一种可选的实施方式，划分子单元221包括：
自相关划分模块，被配置成用于根据预设次数对采集数据中相同天线相同频点的数据进行数据划分，得到自相关划分结果；
互相关划分模块，被配置成用于根据预设次数对采集数据中不同天线相同频点间的数据进行数据划分，得到互相关划分结果。
作为一种可选的实施方式，分组子单元222包括：
自相关划分重排模块，被配置成用于根据预设组数对自相关划分结果进行分组处理，得到自相关划分重排结果；
互相关划分重排模块，被配置成用于根据预设组数对互相关划分结果进行分组处理，得到组内互相关划分重排结果和组间互相关划分重排结果。
作为一种可选的实施方式，缓存单元240包括：
处理子单元241，被配置成用于对自相关结果、组内互相关结果和组间互相关结果分别进行拼接截位处理，得到多种位宽的拼接截位结果；
缓存子单元242，被配置成用于通过不同的存储器对每种位宽的拼接截位结果进行缓存，得到缓存结果。
本实施例中，对于数据处理装置的解释说明可以参照前述实施例中的描述，对此本实施例中不再多加赘述。
可见，实施本实施例所描述的数据处理装置，能够合理优化配置系统资源，从而解决大数据缓存过程中遇到的资源占用空间大和数据处理速度慢的问题。
本申请实施例提供了一种电子设备，包括存储器以及处理器，存储器用于存储计算机程序，处理器运行计算机程序以使电子设备执行本申请实施例1或实施例2中任一项数据处理方法。
本申请实施例提供了一种计算机可读存储介质，其存储有计算机程序指令，计算机程序指令被一处理器读取并运行时，执行本申请实施例1或实施例2中任一项数据处理方法。
在本申请所提供的几个实施例中，应该理解到，所揭露的装置和方法，也可以通过其它的方式实现。以上所描述的装置实施例仅仅是示意性的，例如，附图中的流程图和框图显示了根据本申请的多个实施例的装置、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或代码的一部分，所述模块、程序段或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现方式中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
另外，在本申请各个实施例中的各功能模块可以集成在一起形成一个独立的部分，也可以是各个模块单独存在，也可以两个或两个以上模块集成形成一个独立的部分。
所述功能如果以软件功能模块的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计 算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述仅为本申请的实施例而已，并不用于限制本申请的保护范围，对于本领域的技术人员来说，本申请可以有各种更改和变化。凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步定义和解释。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应所述以权利要求的保护范围为准。
需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
工业实用性
本申请提供了一种数据处理方法及装置，该方法及装置可以采集每路天线中的数据，然后根据预设次数和预设组数对采集数据进行数据划分重排，然后再对划分重排得到的多个划分重排结果进行相关累加处理，得到自相关结果、组内互相关结果和组间互相关结果；再后，对自相关结果、组内互相关结果和组间互相关结果进行拼接截位缓存，得到缓存结果；最后，再对其进行缓存读取，以便于后续操作，从而能够合理优化配置系统资源，从而解决大数据缓存过程中遇到的资源占用空间大和数据处理速度慢的问题。
此外，可以理解的是，本申请的数据处理方法及装置是可以重现的，并且可以用在多种工业应用中。例如，本申请的数据处理方法及装置可以用于资源配置领域。

Claims (14)

  1. 一种数据处理方法，其特征在于，所述方法包括：
    获取多路天线中多个频点的采集数据；所述多路天线中每路天线都包括多个频点；
    根据预设次数和预设组数对所述采集数据进行数据划分重排，得到自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果；
    对所述自相关划分重排结果、所述组内互相关划分重排结果和所述组间互相关划分重排结果进行相关累加处理，得到自相关结果、组内互相关结果和组间互相关结果；
    对所述自相关结果、所述组内互相关结果和所述组间互相关结果分别进行拼接截位缓存处理，得到缓存结果；
    根据预设的频点顺序，循环读取缓存结果。
  2. 根据权利要求1所述的数据处理方法，其特征在于，所述根据预设次数和预设组数对所述采集数据进行数据划分重排，得到自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果的步骤包括：
    根据预设次数对所述采集数据进行数据划分，得到自相关划分结果和互相关划分结果；
    根据预设组数对所述自相关划分结果和所述互相关划分结果进行数据分组，得到自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果。
  3. 根据权利要求2所述的数据处理方法，其特征在于，所述根据预设次数对所述采集数据进行数据划分，得到自相关划分结果和互相关划分结果的步骤包括：
    根据预设次数对所述采集数据中相同天线相同频点的数据进行数据划分，得到自相关划分结果；
    根据所述预设次数对所述采集数据中不同天线相同频点间的数据进行数据划分，得到互相关划分结果。
  4. 根据权利要求2或3所述的数据处理方法，其特征在于，所述根据预设组数对所述自相关划分结果和所述互相关划分结果进行数据分组，得到自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果的步骤包括：
    根据预设组数对所述自相关划分结果进行分组处理，得到自相关划分重排结果；
    根据所述预设组数对所述互相关划分结果进行分组处理，得到组内互相关划分重排结果和组间互相关划分重排结果。
  5. 根据权利要求1至4中任一项所述的数据处理方法，其特征在于，所述对所述自相关结果、所述组内互相关结果和所述组间互相关结果分别进行拼接截位缓存处理，得到缓存结果的步骤包括：
    对所述自相关结果、所述组内互相关结果和所述组间互相关结果分别进行拼接截位处理，得到多种位宽的拼接截位结果；
    通过不同的存储器对每种位宽的拼接截位结果进行缓存，得到缓存结果。
  6. 根据权利要求5所述的数据处理方法，其特征在于，对每种位宽的拼接截位结果进行实时同步缓存。
  7. 根据权利要求1至6中任一项所述的数据处理方法，其特征在于，所述自相关划分结果和/或所述互相关划分结果是大位宽小深度的数据。
  8. 一种数据处理装置，其特征在于，所述数据处理装置包括：
    获取单元，被配置成用于获取多路天线中多个频点的采集数据；所述多路天线中每路天线都包括多个频点；
    数据划分重排单元，被配置成用于根据预设次数和预设组数对所述采集数据进行数据划分重排，得到自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果；
    相关累加单元，被配置成用于对所述自相关划分重排结果、所述组内互相关划分重排结果和所述组间互相关划分重排结果进行相关累加处理，得到自相关结果、组内互相关结果和组间互相关结果；
    缓存单元，被配置成用于对所述自相关结果、所述组内互相关结果和所述组间互相关结果分别进行拼接截位缓存处理，得到缓存结果；
    读取单元，被配置成用于根据预设的频点顺序，循环读取缓存结果。
  9. 根据权利要求8所述的数据处理装置，其特征在于，所述数据划分重排单元包括：
    划分子单元，被配置成用于根据预设次数对所述采集数据进行数据划分，得到自相关划分结果和互相关划分结果；
    分组子单元，被配置成用于根据预设组数对所述自相关划分结果和所述互相关划分结果进行数据分组，得到自相关划分重排结果、组内互相关划分重排结果和组间互相关划分重排结果。
  10. 根据权利要求9所述的数据处理装置，其特征在于，所述划分子单元包括：
    自相关划分模块，被配置成用于根据预设次数对所述采集数据中相同天线相同频点的数据进行数据划分，得到所述自相关划分结果；
    互相关划分模块，被配置成用于根据预设次数对所述采集数据中不同天线相同频点间的数据进行数据划分，得到所述互相关划分结果。
  11. 根据权利要求9至10中任一项所述的数据处理装置，其特征在于，所述分组子单元包括：
    自相关划分重排模块，被配置成用于根据所述预设组数对所述自相关划分结果进行分组处理，得到所述自相关划分重排结果；
    互相关划分重排模块，被配置成用于根据所述预设组数对所述互相关划分结果进行分组处理，得到所述组内互相关划分重排结果和所述组间互相关划分重排结果。
  12. 根据权利要求8至10中任一项所述的数据处理装置，其特征在于，所述缓存单元包括：
    处理子单元，被配置成用于对所述自相关结果、所述组内互相关结果和所述组间互相关结果分别进行拼接截位处理，得到多种位宽的拼接截位结果；
    缓存子单元，被配置成用于通过不同的存储器对每种位宽的拼接截位结果进行缓存，得到缓存结果。
  13. 一种电子设备，其特征在于，所述电子设备包括存储器以及处理器，所述存储器用于存储计算机程序，所述处理器运行所述计算机程序以使所述电子设备执行权利要求1至7中任一项所述的数据处理方法。
  14. 一种可读存储介质，其特征在于，所述可读存储介质中存储有计算机程序指令，所述计算机程序指令被一处理器读取并运行时，执行权利要求1至7中任一项所述的数据处理方法。
PCT/CN2022/098964 2022-05-19 2022-06-15 一种数据处理方法及装置 WO2023221217A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210554412.4 2022-05-19
CN202210554412.4A CN114911832B (zh) 2022-05-19 2022-05-19 一种数据处理方法及装置

Publications (1)

Publication Number Publication Date
WO2023221217A1 true WO2023221217A1 (zh) 2023-11-23

Family

ID=82768039

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/098964 WO2023221217A1 (zh) 2022-05-19 2022-06-15 一种数据处理方法及装置

Country Status (2)

Country Link
CN (1) CN114911832B (zh)
WO (1) WO2023221217A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080205561A1 (en) * 2007-02-14 2008-08-28 Samsung Electronics Co. Ltd. Apparatus and method for interference cancellation in broadband wireless access communication system
CN108828592A (zh) * 2018-04-25 2018-11-16 中国科学院电子学研究所 基于mimo矩形平面阵列的方位向成像方法
CN111339484A (zh) * 2020-02-20 2020-06-26 中国科学院自动化研究所 基于fpga的大型射电干涉阵列相关器的实现方法及装置
CN111931669A (zh) * 2020-08-14 2020-11-13 山东大学 一种太阳射电观测系统的信号自适应截位方法及系统

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8533245B1 (en) * 2010-03-03 2013-09-10 Altera Corporation Multipliers with a reduced number of memory blocks
CN101997566B (zh) * 2010-11-08 2014-07-09 北京理工大学 一种基于最大绝对值累积平均和栅格比较的自适应截位方法
CN102291197B (zh) * 2011-05-19 2015-07-08 重庆大学 组间正交互补序列集的生成方法
CN103037032B (zh) * 2012-12-20 2016-06-29 北京四方继保自动化股份有限公司 一种使用fpga实现sv数据32位寻址访问的方法
CN104809996B (zh) * 2015-04-30 2017-06-16 武汉精测电子技术股份有限公司 基于fpga实现mipi多种lane数的数据信号的方法和装置
CN104820207B (zh) * 2015-05-08 2017-06-27 中国科学院新疆天文台 基于fpga、gpu和cpu混合架构的实时相关器
WO2018119035A1 (en) * 2016-12-22 2018-06-28 Ip Reservoir, Llc Pipelines for hardware-accelerated machine learning
CN106685887B (zh) * 2017-01-12 2020-04-07 重庆邮电大学 一种用于fpga的ufmc发射机的频域实现方法
CN109542799B (zh) * 2018-11-05 2023-03-28 西安智多晶微电子有限公司 块存储器拼接方法、拼接模块、存储装置及现场可编程门阵列
CN110262773B (zh) * 2019-04-28 2020-08-04 阿里巴巴集团控股有限公司 一种计算机数据处理方法及装置
CN111047037B (zh) * 2019-12-27 2024-05-24 北京市商汤科技开发有限公司 数据处理方法、装置、设备及存储介质
CN111506294B (zh) * 2020-04-13 2022-07-29 中国科学院自动化研究所 基于块浮点的fblms算法的fpga实现装置及方法
CN112487354B (zh) * 2020-11-30 2022-10-04 北京航空航天大学 一种基于比较器和fpga的多通道高速实时互相关运算装置
CN113590853B (zh) * 2021-07-13 2024-08-09 深圳市洲明科技股份有限公司 一种灰阶自适应扩展方法、fpga系统、设备及介质
CN113960682A (zh) * 2021-08-31 2022-01-21 西安电子科技大学 一种基于fpga的多通道数字相关器及其相关方法
CN113555737B (zh) * 2021-09-17 2021-12-17 长沙泰科阳微电子有限公司 一种sar图像检测方法及系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080205561A1 (en) * 2007-02-14 2008-08-28 Samsung Electronics Co. Ltd. Apparatus and method for interference cancellation in broadband wireless access communication system
CN108828592A (zh) * 2018-04-25 2018-11-16 中国科学院电子学研究所 基于mimo矩形平面阵列的方位向成像方法
CN111339484A (zh) * 2020-02-20 2020-06-26 中国科学院自动化研究所 基于fpga的大型射电干涉阵列相关器的实现方法及装置
CN111931669A (zh) * 2020-08-14 2020-11-13 山东大学 一种太阳射电观测系统的信号自适应截位方法及系统

Also Published As

Publication number Publication date
CN114911832B (zh) 2023-06-23
CN114911832A (zh) 2022-08-16

Similar Documents

Publication Publication Date Title
US10268398B2 (en) Storage system, recording medium for storing control program and control method for storage system
CN109155763B (zh) 数据流上的数字信号处理
CN109063196B (zh) 数据处理方法、装置、电子设备及计算机可读存储介质
US20160253220A1 (en) Data center operation
US8321476B2 (en) Method and system for determining boundary values dynamically defining key value bounds of two or more disjoint subsets of sort run-based parallel processing of data from databases
CN103324592A (zh) 一种数据迁移控制方法、数据迁移方法及装置
JP2020524853A (ja) データベースアクセラレータに対する並列計算オフロード
CN110245134A (zh) 一种应用于搜索服务的增量同步方法
CN108337286A (zh) 一种切包方法及装置
WO2021109665A1 (zh) 一种数据处理装置、方法、基站和存储介质
KR20200141208A (ko) 그래프 데이터 처리 방법 및 그래프 데이터 처리 장치
CN115203210A (zh) 哈希表处理方法、装置、设备及计算机可读存储介质
WO2023221217A1 (zh) 一种数据处理方法及装置
CN109358820B (zh) 数据访问方法、装置、电子设备及计算机可读存储介质
EP3243144B1 (en) Data partition and transformation methods and apparatuses
EP3528133A1 (en) Data stream join method and device
CN110120959B (zh) 大数据推送方法、装置、系统、设备及可读存储介质
CN115269719A (zh) 数据同步方法及装置
CN112685619A (zh) 一种网页表格数据通用抓取方法
CN113726499A (zh) 数字采样数据高速插值同步方法及系统
CN115210694A (zh) 数据传输方法及装置
CN112260896A (zh) 一种网络传输测试方法、装置、设备及可读存储介质
CN112711588A (zh) 多表连接的方法和装置
CN113965584B (zh) 报文处理方法、设备、装置及存储介质
CN110225211B (zh) 多通道脉冲编码调制语音交换系统及方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22942249

Country of ref document: EP

Kind code of ref document: A1