CN110825343B - Rapid data screening method and system - Google Patents
Rapid data screening method and system Download PDFInfo
- Publication number
- CN110825343B CN110825343B CN201911071810.5A CN201911071810A CN110825343B CN 110825343 B CN110825343 B CN 110825343B CN 201911071810 A CN201911071810 A CN 201911071810A CN 110825343 B CN110825343 B CN 110825343B
- Authority
- CN
- China
- Prior art keywords
- data
- priority
- comparison
- module
- comparison result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F5/00—Methods or arrangements for data conversion without changing the order or content of the data handled
- G06F5/06—Methods or arrangements for data conversion without changing the order or content of the data handled for changing the speed of data flow, i.e. speed regularising or timing, e.g. delay lines, FIFO buffers; over- or underrun control therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Complex Calculations (AREA)
- Logic Circuits (AREA)
Abstract
The utility model provides a quick data screening method and system, aiming at the massive channel data, firstly, the threshold comparator is used to judge the data validity of the input data, and the availability of the data to the next stage processing is judged according to the comparison result, thus reducing the resource pressure. The rapid screening of the data priority is completed on the basis of guaranteeing the timeliness and the resource availability by utilizing the rapid data comparison, the generation of the priority group sequence and the gating of the output data, the rapid screening of the priority of the multi-channel data has a good effect, the data screening speed is high, the real-time performance is good under the condition that the data volume is increased, the FPGA-based hardware design can be well operated on platforms based on different FPGAs, and the portability degree is high.
Description
Technical Field
The disclosure relates to the technical field of detection and identification, in particular to a rapid data screening method and system.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
The rapid accumulation of data volume in a big data environment is very important for analyzing the value contained in mass data and screening out valuable data. And data screening is in a crucial position in the whole data processing flow. The rapid data screening is an important content in the field of detection and identification, and is widely applied in the fields of digital signal processing, digital image processing, pattern identification and the like. Particularly with the advent of the high definition era, it is becoming a hot topic to ensure how to screen the most interesting data points at high data volumes. For example, in a complex electromagnetic environment, how to quickly locate the band where the signal is located in the full band and quickly make a response becomes an important research for ensuring the real-time performance of a modern digital signal processing system. When dealing with massive input data, how to quickly screen out high-priority data is an important research topic. At present, new requirements are provided for data screening, time feasibility and space feasibility are required to be considered, and good real-time performance and reliability can be guaranteed under the condition that the data volume is increased.
In the prior art, the fast screening method for multi-channel data includes software-based screening methods, such as insert sorting screening, hill sorting screening, heap sorting screening, merge sorting screening, fast sorting screening, and the like. The software-based sorting method is used as a mature sorting and screening algorithm and has high reliability, but with the increase of data volume, the resource consumption is huge, and the screening completion time is prolonged. Therefore, in the occasion with high real-time requirement, the screening algorithm realized based on software is difficult to meet the requirement.
In order to take time and resources into consideration, some hardware-based screening algorithms are developed, such as serial-parallel set sorting screening, multi-level merging sorting screening, hardware sorting system screening, and the like.
The two aspects of real-time requirement and resource realizability are considered in the serial-parallel set sequencing screening, for the sequencing screening of the N data, the comparison result of the single data and the N data is not required to be completed in one clock cycle, and instead, the time-sharing idea is utilized to complete the output of all the comparison results in the N cycles. The method greatly reduces the consumption of hardware resources in the algorithm implementation process. However, the number of clock cycles for completing the screening is equivalent to the number of data to be processed, the real-time performance is limited by the number of data, the index in the aspect of real-time performance is reduced, and the availability of the algorithm needs to be considered by combining with the real-time performance requirement during application.
The multilevel merging and sorting screening has two realization modes of register-based and FIFO-based. The most basic operation of this algorithm is to merge two already sorted data tables, so the applicability of this algorithm is limited. Furthermore, the worst running time of the algorithm is o (nlogn), and the real-time performance becomes worse as the amount of data to be processed increases.
The hardware sorting system screens, and the core idea is to insert sorting directly. The data which needs to be inserted currently is compared with the data which is sorted previously, so that the correct position where the data should be inserted can be calculated after one clock cycle. Through the analysis principle, the algorithm adopts the parallel comparison idea, but along with the increase of the number of the data to be processed, the time required for completing the data priority screening is increased linearly, and the real-time performance is relatively poor.
In conclusion, the data screening algorithm realized based on pure software occupies a large amount of resources along with the increase of the data volume to be processed, and no method is guaranteed in the aspects of instantaneity and processing speed. The data screening algorithm based on hardware implementation: the serial-parallel set sequencing screening algorithm optimizes the resource quantity required by the algorithm, but the processing speed still has no way to meet the occasion with high real-time requirement; multi-level merging, sorting and screening are carried out, the execution condition is good on the occasion of realizing two sorted data tables, but the application scene is limited; the screening of the hardware sorting system adopts a comparison and insertion idea to screen the priority of data, but the processing speed is reduced and the real-time performance is deteriorated as the data volume is increased. Therefore, an algorithm is needed, which can ensure the real-time performance of data screening, and when the data amount increases, the real-time performance will not be deteriorated, and the processing speed is reliable.
Disclosure of Invention
Aiming at massive channel data, the threshold comparator is used for judging the data validity of input data, and the availability of the data to the next-stage processing is judged according to the comparison result, so that the resource pressure is reduced. The rapid screening of the data priority is completed on the basis of guaranteeing the timeliness and the resource availability by utilizing the rapid data comparison, the generation of the priority group sequence and the gating of the output data, the rapid screening of the priority of the multi-channel data has a good effect, the data screening speed is high, the real-time performance is good under the condition that the data volume is increased, the FPGA-based hardware design can be well operated on platforms based on different FPGAs, and the portability degree is high.
In order to achieve the purpose, the following technical scheme is adopted in the disclosure:
one or more embodiments provide a fast data screening system, including M threshold comparators with M output terminals, M fast data comparison modules, an input terminal of each fast data comparison module being connected in parallel to the M output terminals of the threshold comparator, an output terminal of each fast data comparison module being connected to a priority array sequence generation module, and an output data gating module connected to the priority array sequence generation module;
the threshold comparator is used for comparing a set threshold with input data and outputting effective data to the rapid data comparison module according to a comparison result;
a fast data comparison module: the device comprises a priority group sequence generation module, a threshold comparator, a priority group sequence generation module and a comparison module, wherein the priority group sequence generation module is used for comparing any two data in the received effective data of M output ends of the threshold comparator and transmitting results of pairwise comparison to the priority group sequence generation module;
the priority array sequence generation module: the priority sequence is used for generating a priority sequence of the data according to the comparison result, and the priority sequence is arranged in order from high to low according to the priority level;
an output data strobe module: and the device is used for strobing and outputting the first Q data with high priority according to the set strobe quantity Q and the priority sequence.
Furthermore, the threshold comparator, the rapid data comparison module, the priority group sequence generation module and the output data gating module are all realized on the basis of an FPGA chip, the functions of the modules are realized by adopting hardware resources on the FPGA chip, and a writeable file is generated by adopting hardware description language programming and loaded into the FPGA chip.
Further, each of the fast data comparison modules comprises:
a setting module: the main data used for respectively defining the Mth data at the input end of the fast data comparison module as the main data of the Mth fast data comparison module;
a first comparison module: and the comparison module is used for comparing the main data of the current fast data comparison module with other M-1 data at the input end of the fast data comparison module to obtain the comparison result of the current main data.
Further, the comparison result of the current main data is that when the main data is larger than other data currently being compared, the output is that the comparison result is 0, otherwise, the comparison result is 1, M-1 comparison results of the main data are obtained, variables are set and assigned as the comparison result of the current main data;
or
When the comparison result is the result obtained by the method of setting the variable and assigning the variable, the priority sequence of the data is generated according to the comparison result, and the priority group sequence generation module comprises:
a priority value calculation module: the main data priority value is obtained by weighting the comparison result of the main data and other data aiming at each main data;
an arrangement module: and the priority sequence is used for arranging the main data from large to small according to the priority value aiming at each main data to obtain the data.
A fast data screening method is used for outputting data after judging the priority of input data, and comprises the following steps:
comparing the set threshold value with the input data, and screening according to the comparison result to obtain effective data;
comparing any two data in the obtained effective data, and outputting results of pairwise comparison;
generating a priority sequence of the data according to the comparison result, wherein the priority sequence is orderly arranged from high to low according to the priority level;
and according to the set gating quantity Q and the priority sequence, gating and outputting the first Q data with high priority.
Further, the method is realized based on an FPGA chip, the steps of the method are realized by adopting hardware resources on the FPGA chip, and a writeable file is generated by adopting hardware description language programming and is loaded into the FPGA chip.
Further, valid data refers to data that is greater than a set threshold;
or
The method for comparing any two data in the obtained effective data and outputting the result of pairwise comparison specifically comprises the following steps:
defining each data in the effective data as main data respectively and sequentially;
comparing the size of each main data with other effective data to obtain a comparison result of each main data with other effective data;
further, the comparison result of the main data and the other data is a result obtained by a method of setting a variable and assigning a value to the variable, and specifically includes: defining a variable, outputting a comparison result of 0 to the variable when the main data is larger than other data currently being compared, and assigning 1 to the variable if the main data is not larger than the other data currently being compared, and obtaining M-1 comparison results of the main data;
or the comparison result is the number of times that the statistical main data is larger than other data.
Further, when the comparison result is that the number of times that the statistical main data is larger than other data is counted, the priority sequence of the data is obtained by arranging according to the size of the number of times.
Further, when the comparison result is a result obtained by setting a variable and assigning the variable, a priority sequence of the data is generated according to the comparison result, and the method for arranging the priority sequence in order from high to low according to the priority level specifically comprises the following steps:
weighting the comparison result of the main data and other data aiming at each main data to obtain the priority value of the current main data;
and arranging the main data from large to small according to the priority value aiming at each main data to obtain the priority sequence of the data.
Compared with the prior art, the beneficial effect of this disclosure is:
the system and the method can complete the priority rapid screening of multi-channel data based on FPGA, and have the advantages of large data processing quantity, high real-time performance and high portability.
1) The amount of data that can be processed is large: the modules with the same function are arranged in parallel, the module number can be set according to the channel number, the operation is executed in parallel, the priority rapid screening of hundreds of channels, thousands of channels and ten thousands of channels of data can be realized, and the priority screening of big data can be completed rapidly by combining a threshold comparator, rapid data comparison, priority array sequence generation and output gating. And the algorithm flow is simple and easy to realize.
2) The real-time performance is high: the requirement of multi-channel data priority rapid screening can be met, and massive channel data priority screening can be completed within 4 clock periods. And with the increase of the data volume, the real-time performance of the algorithm is not deteriorated, and the processing speed is high.
3) The portability is high: the FPGA-based hardware design can well run on platforms based on different FPGAs.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure and not to limit the disclosure.
Fig. 1 is a block diagram of a system of embodiment 1 of the present disclosure;
fig. 2 is a schematic structural diagram of a system implemented based on an FPGA in embodiment 1 of the present disclosure.
The specific implementation mode is as follows:
the present disclosure is further described with reference to the following drawings and examples.
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise. It should be noted that, in the case of no conflict, the embodiments and features in the embodiments in the present disclosure may be combined with each other. The embodiments will be described in detail below with reference to the accompanying drawings.
The field programmable gate array is abbreviated as FPGA: the semi-custom circuit belongs to an application-specific integrated circuit, is a programmable logic array, and integrates various hardware resources, such as a multiplier, an LUT lookup table, an RAM and the like.
Example 1
In the technical solutions disclosed in one or more embodiments, as shown in fig. 1 and 2, a fast data screening system includes M threshold comparators with output ends, M fast data comparison modules, an input end of each fast data comparison module being connected in parallel with the M output ends of the threshold comparators, an output end of each fast data comparison module being connected with a priority array sequence generation module, and an output data gating module connected with the priority array sequence generation module; wherein M is the quantity of setting for, and M is the numerical value that sets up according to the quantity of on-the-spot data collection, can set up according to concrete needs.
The threshold comparator is used for comparing a set threshold with input data and outputting effective data to the rapid data comparison module according to a comparison result;
a fast data comparison module: the priority array sequence generation module is used for comparing the received data of the M output ends of the threshold comparator and transmitting the comparison result to the priority array sequence generation module;
the priority array sequence generation module: the priority sequence is used for generating a priority sequence of the data according to the comparison result, and the priority sequence is arranged in order from high to low according to the priority level;
an output data strobe module: and the device is used for strobing and outputting the first Q data with high priority according to the set strobe quantity Q and the priority sequence.
Optionally, the threshold comparator, the fast data comparison module, the priority group sequence generation module and the output data gating module may be implemented based on an FPGA chip, the functions of the modules are implemented by using hardware resources on the FPGA chip, and the writeable file is generated by using hardware description language programming and loaded into the FPGA chip.
Realizing a functional algorithm of the rapid data comparison module on the FPGA based on a hardware description language, and outputting a comparison result of the main data and other data; realizing the algorithm of the priority array sequence generation module on the FPGA based on a hardware description language, and outputting array data with ordered priorities; and realizing an algorithm of an output data gating module on the FPGA based on a hardware description language, and outputting a preset number of high-priority data.
Optionally, the threshold comparator compares a set threshold with input data, and outputs data larger than the set threshold as valid data, where when there are M output ends, the number of valid data output in parallel at a time is less than or equal to M.
In this embodiment, optionally, when the number of valid input data at the input end of the fast data comparison module is less than M, the number of input data is increased to M by using zero value supplementation.
As a further improvement, each fast data comparison module comprises:
a setting module: the main data used for respectively defining the Mth data at the input end of the fast data comparison module as the main data of the Mth fast data comparison module; the main data set by different rapid data comparison modules are non-repetitive data and respectively correspond to each output end of the threshold comparator.
A first comparison module: the device comprises a data comparison module, a comparison module and a comparison module, wherein the data comparison module is used for comparing the main data of the current fast data comparison module with other M-1 data at the input end of the fast data comparison module to obtain a comparison result of the current main data;
optionally, the comparison result of the current main data is that when the main data is larger than other data currently being compared, the output is that the comparison result is 0, otherwise, the output is 1, M-1 comparison results of the main data are obtained, and a variable is set and assigned as the comparison result of the current main data; or counting the times that the main data is larger than other data as a comparison result.
M quick data comparison modules respectively compare, and each quick data comparison module outputs M-1 comparison results of main data set by the quick data comparison module.
Specifically, in this embodiment, the comparison result of the current main data is, as shown in fig. 2, when the main data is larger than other data currently being compared, the output is 0, otherwise, the comparison result is 1, and M-1 comparison results of the main data are obtained as a comparison method for assigning a variable. The steps executed by the fast data comparison module are described by taking the fast comparison module 1 as an example: index _1_2 is assigned a value of 0 when din _ pre _1 is greater than din _ pre _2, otherwise index _1_2 is assigned a value of 1, and so on, comparing din _ pre _1 with din _ pre _3, din _ pre _4 … … din _ pre _ M, respectively, and index _1_ M is assigned a value of 0 when din _ pre _1 is greater than din _ pre _ M, otherwise index _1_ M is assigned a value of 1. After M-1 comparisons, the assignment from index _1_2 to index _1_ M is complete. M-1 output data of the rapid comparison module 1 are input of the next-level priority array sequence generation module. Similarly, the specific operation algorithms of the fast comparison module 2 to the fast comparison module M are the same as those of the fast comparison module 1. Through the operation of the fast comparison portion, M ﹡ (M-1) output data can be generated.
In some embodiments, the priority group sequence generation module: for judging the priority of the data and obtaining the priority sequence of the data according to the priority. When the comparison result of the rapid data comparison module is the times of counting the main data larger than other data, the data can be directly arranged according to the times to obtain the priority sequence of the data.
When the comparison result is the result obtained by the method of setting and assigning the variable, the priority group sequence generation module comprises:
a priority value calculation module: the main data priority value is obtained by weighting the comparison result of the main data and other data aiming at each main data;
an arrangement module: and the priority sequence is used for arranging the main data from large to small according to the priority value aiming at each main data to obtain the data.
Based on FPGA implementation, specifically, the priority group sequence generation module firstly initializes a register type two-dimensional array: reg [ D-1:0] ch [ M-1:0 ]. The array can store D bits of binary data, and the number of the data which can be stored is M. The (M-1) ﹡ M output data generated at the previous stage are used to construct a priority screening sequence. If a hardware description language on the FPGA is adopted, the numerical value of the corresponding main data may be assigned to the corresponding array, and for the priority array sequence generation module 1: ch [ index _1_2+ index _1_3+ … … + index _1_ M ] < ═ din _ pre _ 1. By analogy, for the priority array sequence generation module M: ch [ index _ M _1+ index _ M _2+ … … + index _ M _ (M-1) ] ] < ═ din _ pre _ M. The data after being processed by the priority array sequence generation part are orderly arranged from ch [0] to ch [ M-1 ].
The output data gating module is used for outputting data gating. The priority number of the data needing to be gated can be set to be Q, and Q < ═ M. The specific operation is as follows: ch _ Sel [1] < ═ Ch [0], … …, Ch _ Sel [ Q ] < ═ Ch [ Q-1 ]. The whole operation of rapidly screening out Q high-priority data from N data is completed.
Example 2
The embodiment provides a fast data screening method, as shown in fig. 2, for strobing and outputting data after determining the priority of input data, including the following steps:
step 2, comparing any two data in the obtained effective data, and outputting results of pairwise comparison;
step 3, generating a priority sequence of the data according to the comparison result, wherein the priority sequence is orderly arranged from high to low according to the priority level;
and 4, gating and outputting the first Q data with high priority according to the set gating quantity Q and the priority sequence.
Preferably, the steps 1 to 4 are implemented based on an FPGA chip, the method steps in the steps 1 to 4 are implemented by using hardware resources on the FPGA chip, and the writeable file is generated by using hardware description language programming and loaded into the FPGA chip.
Optionally, the valid data in step 1 refers to data greater than a set threshold;
the method for comparing any two data in the obtained effective data in the step 2 and outputting the result of pairwise comparison specifically comprises the following steps:
1) defining each data in the effective data as main data respectively and sequentially;
a fast data screening system according to embodiment 1, that is, defining main data of an mth fast data comparison module for the mth data at the input end of the fast data comparison module;
2) comparing the size of each main data with other data of the effective data respectively to obtain a comparison result of each main data with other data;
further, the comparison result may be a result obtained by setting a variable and assigning the variable, specifically, when the main data is larger than other data currently being compared, the comparison result is output as 0 and assigned to the variable, otherwise, the comparison result is 1 and assigned to the variable, and M-1 comparison results of the main data are obtained;
optionally, the comparison result may also be the number of times that the statistical main data is greater than the other data.
When the comparison result is that the times of the statistical main data are greater than those of other data, the data can be directly arranged according to the times to obtain the priority sequence of the data.
When the comparison result is the result obtained by setting the variable and assigning the variable, in step 4, a priority sequence of the data is generated according to the comparison result, and the method for arranging the priority sequence in order from high to low according to the priority level specifically comprises the following steps:
41. weighting the comparison result of the main data and other data aiming at each main data to obtain the priority value of the current main data;
42. and arranging the main data from large to small according to the priority value aiming at each main data to obtain the priority sequence of the data.
The embodiment provides a fast data screening method, which can complete the fast priority screening of multi-channel data, and has the following outstanding advantages: 1) the amount of data which can be processed is large; 2) the real-time performance is high; 3) the portability is high.
1) The amount of data that can be processed is large: the method can realize the rapid screening of the priority of hundreds of channels, thousands of channels to ten thousands of channels of data, and can rapidly complete the priority screening of the big data by combining a threshold comparator, rapid data comparison, priority array sequence generation and output gating. And the algorithm flow is simple and easy to realize.
2) The real-time performance is high: the requirement of multi-channel data priority rapid screening can be met, and massive channel data priority screening can be completed within 4 clock periods. And with the increase of the data volume, the real-time performance of the algorithm is not deteriorated, and the processing speed is high.
3) The portability is high: the method can be well operated on platforms based on different FPGAs.
The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure, and various modifications and changes may be made to the present disclosure by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.
Although the present disclosure has been described with reference to specific embodiments, it should be understood that the scope of the present disclosure is not limited thereto, and those skilled in the art will appreciate that various modifications and changes can be made without departing from the spirit and scope of the present disclosure.
Claims (11)
1. A rapid data screening system is characterized in that: the device comprises a threshold comparator with M output ends, M rapid data comparison modules, an output data gating module and a priority array sequence generation module, wherein the input end of each rapid data comparison module is connected with the M output ends of the threshold comparator in parallel;
the threshold comparator is used for comparing a set threshold with input data and outputting effective data to the rapid data comparison module according to a comparison result;
a fast data comparison module: the device comprises a priority group sequence generation module, a threshold comparator, a priority group sequence generation module and a comparison module, wherein the priority group sequence generation module is used for comparing any two data in the received effective data of M output ends of the threshold comparator and transmitting results of pairwise comparison to the priority group sequence generation module;
the priority array sequence generation module: the priority sequence is used for generating a priority sequence of the data according to the comparison result, and the priority sequence is arranged in order from high to low according to the priority level;
an output data strobe module: the system comprises a data strobe output unit, a priority sequence and a control unit, wherein the data strobe output unit is used for strobing and outputting the first Q data with high priority according to the set strobe quantity Q and the priority sequence;
each of the fast data comparison modules includes:
a setting module: the main data used for respectively defining M data at the input end of the rapid data comparison module as the main data of the M rapid data comparison modules;
a first comparison module: and the comparison module is used for comparing the main data of the current fast data comparison module with other M-1 data at the input end of the fast data comparison module to obtain the comparison result of the current main data.
2. The rapid data screening system of claim 1, wherein: the threshold comparator, the rapid data comparison module, the priority group sequence generation module and the output data gating module are all realized on the basis of an FPGA chip, the functions of the modules are realized by adopting hardware resources on the FPGA chip, and a writeable file is generated by adopting hardware description language programming and loaded into the FPGA chip.
3. The rapid data screening system of claim 1, wherein: and the comparison result of the current main data is that when the main data is larger than other data currently being compared, the output is that the comparison result is 0, otherwise, the output is 1, M-1 comparison results of the main data are obtained, variables are set and assigned as the comparison result of the current main data.
4. The rapid data screening system of claim 3, wherein: when the comparison result is the result obtained by the method of setting the variable and assigning the variable, the priority sequence of the data is generated according to the comparison result, and the priority group sequence generation module comprises:
a priority value calculation module: the main data priority value is obtained by weighting the comparison result of the main data and other data aiming at each main data;
an arrangement module: and the priority sequence is used for arranging the main data from large to small according to the priority value aiming at each main data to obtain the data.
5. A fast data screening method is characterized in that the method is used for outputting data after judging the priority of input data, and comprises the following steps:
comparing the set threshold value with the input data, and screening according to the comparison result to obtain effective data;
comparing any two data in the obtained effective data, and outputting results of pairwise comparison;
generating a priority sequence of the data according to the comparison result, wherein the priority sequence is orderly arranged from high to low according to the priority level;
according to the set gating quantity Q and the priority sequence, gating and outputting the first Q data with high priority;
the method for comparing any two data in the obtained effective data and outputting the result of pairwise comparison specifically comprises the following steps:
respectively defining M data in the effective data as M main data in sequence;
and comparing the size of each main data with other data of the effective data respectively to obtain a comparison result of each main data with other data.
6. The method of claim 5, wherein the method comprises: the method of claim 5 is implemented based on an FPGA chip, using hardware resources on the FPGA chip, and using hardware description language programming to generate a writeable file to be loaded into the FPGA chip.
7. The method of claim 5, wherein the method comprises: valid data refers to data that is greater than a set threshold.
8. The method of claim 5, wherein the method comprises: the comparison result of the main data and other data is obtained by setting variables and assigning values to the variables, and specifically includes: and defining a variable, outputting a comparison result of 0 to the variable when the main data is larger than other data currently being compared, and assigning 1 to the variable if the main data is not larger than the other data currently being compared, thereby obtaining M-1 comparison results of the main data.
9. The method of claim 5, wherein the method comprises: and the comparison result of the main data and other data is the frequency of counting the main data to be larger than other data.
10. The method of claim 9, wherein the method comprises: and when the comparison result is that the times of counting the main data are greater than the times of counting other data, arranging according to the times to obtain the priority sequence of the data.
11. The method of claim 8, wherein the method comprises: when the comparison result is the result obtained by setting the variable and assigning the variable, a priority sequence of the data is generated according to the comparison result, and the method for arranging the priority sequence in order from high to low according to the priority level specifically comprises the following steps:
weighting the comparison result of the main data and other data aiming at each main data to obtain the priority value of the current main data;
and arranging the main data from large to small according to the priority value aiming at each main data to obtain the priority sequence of the data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911071810.5A CN110825343B (en) | 2019-11-05 | 2019-11-05 | Rapid data screening method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911071810.5A CN110825343B (en) | 2019-11-05 | 2019-11-05 | Rapid data screening method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110825343A CN110825343A (en) | 2020-02-21 |
| CN110825343B true CN110825343B (en) | 2021-12-03 |
Family
ID=69552530
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911071810.5A Active CN110825343B (en) | 2019-11-05 | 2019-11-05 | Rapid data screening method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110825343B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1285064A (en) * | 1997-12-16 | 2001-02-21 | 英特尔公司 | System for ordering load and store instructions that performs out-of-order multithread execution |
| CN102394661A (en) * | 2011-11-08 | 2012-03-28 | 北京邮电大学 | LDPC (low density parity check) decoder and decoding method based on layer decoding processing |
| CN104317549A (en) * | 2014-10-15 | 2015-01-28 | 中国航天科技集团公司第九研究院第七七一研究所 | Cascade structure circuit and method for realizing data sorting |
| CN105162474A (en) * | 2015-09-09 | 2015-12-16 | 中国科学院自动化研究所 | Addition comparison selection calculation method and addition comparison selection calculation device in radix-4 algorithm |
| CN105512179A (en) * | 2015-11-25 | 2016-04-20 | 中国科学院计算技术研究所 | Data sorting device, method and data processing chip achieved by hardware |
| CN106250097A (en) * | 2016-06-22 | 2016-12-21 | 中国科学院计算技术研究所 | A kind of acceleration collator towards big data, method, chip, processor |
| CN107359953A (en) * | 2017-06-30 | 2017-11-17 | 京信通信系统（中国）有限公司 | A kind of TD LTE synchronous method and synchronizer |
| CN108242051A (en) * | 2016-12-23 | 2018-07-03 | 想象技术有限公司 | Median determines |
| CN109426484A (en) * | 2017-08-28 | 2019-03-05 | 华为技术有限公司 | A kind of data sorting device, method and chip |
| CN109460210A (en) * | 2018-10-22 | 2019-03-12 | 重庆中科云丛科技有限公司 | Ordering system and data processing method |
| CN110033426A (en) * | 2018-01-12 | 2019-07-19 | 杭州海康威视数字技术股份有限公司 | A kind of device for being handled disparity estimation image |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7908283B2 (en) * | 2007-08-29 | 2011-03-15 | Red Hat, Inc. | Finding superlatives in an unordered list |
-
2019
- 2019-11-05 CN CN201911071810.5A patent/CN110825343B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1285064A (en) * | 1997-12-16 | 2001-02-21 | 英特尔公司 | System for ordering load and store instructions that performs out-of-order multithread execution |
| CN102394661A (en) * | 2011-11-08 | 2012-03-28 | 北京邮电大学 | LDPC (low density parity check) decoder and decoding method based on layer decoding processing |
| CN104317549A (en) * | 2014-10-15 | 2015-01-28 | 中国航天科技集团公司第九研究院第七七一研究所 | Cascade structure circuit and method for realizing data sorting |
| CN105162474A (en) * | 2015-09-09 | 2015-12-16 | 中国科学院自动化研究所 | Addition comparison selection calculation method and addition comparison selection calculation device in radix-4 algorithm |
| CN105512179A (en) * | 2015-11-25 | 2016-04-20 | 中国科学院计算技术研究所 | Data sorting device, method and data processing chip achieved by hardware |
| CN106250097A (en) * | 2016-06-22 | 2016-12-21 | 中国科学院计算技术研究所 | A kind of acceleration collator towards big data, method, chip, processor |
| CN108242051A (en) * | 2016-12-23 | 2018-07-03 | 想象技术有限公司 | Median determines |
| CN107359953A (en) * | 2017-06-30 | 2017-11-17 | 京信通信系统（中国）有限公司 | A kind of TD LTE synchronous method and synchronizer |
| CN109426484A (en) * | 2017-08-28 | 2019-03-05 | 华为技术有限公司 | A kind of data sorting device, method and chip |
| CN110033426A (en) * | 2018-01-12 | 2019-07-19 | 杭州海康威视数字技术股份有限公司 | A kind of device for being handled disparity estimation image |
| CN109460210A (en) * | 2018-10-22 | 2019-03-12 | 重庆中科云丛科技有限公司 | Ordering system and data processing method |
Non-Patent Citations (1)
| Title |
|---|
| 基于TMS28035的MMC的控制系统设计与实验研究;马凯等;《电源学报》;20160819;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110825343A (en) | 2020-02-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9886017B2 (en) | Counter operation in a state machine lattice | |
| CN109426484B (en) | Data sorting device, method and chip | |
| US9147123B2 (en) | System and method to perform feature detection and to determine a feature score | |
| US8775685B1 (en) | Parallel processing of network packets | |
| US20070192572A1 (en) | Minimum processor instruction for implementing weighted fair queuing and other priority queuing | |
| CN108132986B (en) | Rapid processing method for test data of mass sensors of aircraft | |
| CN105573843A (en) | Data processing method and system | |
| EP3893400A1 (en) | Sorting method and apparatus, and electronic device and medium | |
| CN110825343B (en) | Rapid data screening method and system | |
| CN114547201A (en) | Processing method of block chain transaction, block chain node and electronic equipment | |
| CN112527505B (en) | Multithreading comprehensive control method and device | |
| US8196082B1 (en) | Signal routing and pin placement | |
| CN110008382B (en) | Method, system and equipment for determining TopN data | |
| WO2022253131A1 (en) | Data parsing method and apparatus, computer device, and storage medium | |
| US11449344B1 (en) | Regular expression processor and parallel processing architecture | |
| CN111722829B (en) | Double-concurrent pre-reading high-performance merging and sorting method and application | |
| CN109164982B (en) | Data processing circuit, data processing method and data storage device | |
| CN104735824A (en) | Data processing system | |
| CN112118012A (en) | Method and device for determining iteration times of decoder | |
| CN117077603B (en) | Verification method, chip, system, electronic device and readable storage medium | |
| CN110020765B (en) | Service flow switching method and device | |
| CN117112162A (en) | Data processing method, device, equipment and storage medium | |
| CN117852630A (en) | Application method of configurable multi-morphology neural network accelerator | |
| CN116610361A (en) | Floating point arithmetic unit, configuration method and device thereof, artificial intelligent chip and accelerator | |
| CN113346877A (en) | Clock period detection method and circuit based on dichotomy |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 266555 No. 98 Xiangjiang Road, Huangdao District, Qingdao City, Shandong Province Applicant after: CLP kesiyi Technology Co., Ltd Address before: 266555 No. 98 Xiangjiang Road, Huangdao District, Qingdao City, Shandong Province Applicant before: China Electronics Technology Instrument Co., Ltd |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |