CN110659290A

CN110659290A - Data processing method and device and related product

Info

Publication number: CN110659290A
Application number: CN201910894302.0A
Authority: CN
Inventors: 不公告发明人
Original assignee: Beijing Zhongke Cambrian Technology Co Ltd
Current assignee: Cambricon Technologies Corp Ltd; Beijing Zhongke Cambrian Technology Co Ltd
Priority date: 2019-09-20
Filing date: 2019-09-20
Publication date: 2020-01-07
Anticipated expiration: 2039-09-20
Also published as: CN110659290B

Abstract

The present disclosure relates to a data processing method and apparatus and related products. The product includes a control module, the control module including: the device comprises an instruction cache unit, an instruction processing unit and a storage queue unit; the instruction cache unit is used for storing the calculation instruction associated with the artificial neural network operation; the instruction processing unit is used for analyzing the calculation instruction to obtain a plurality of operation instructions; the storage queue unit is configured to store an instruction queue, where the instruction queue includes: and a plurality of operation instructions or calculation instructions to be executed according to the front and back sequence of the queue. Through the method, the operation efficiency of the related product in the operation of the neural network model can be improved.

Description

Data processing method and device and related product

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a data processing method and apparatus, and a related product.

Background

In the data processing process, it is usually necessary to select a part of the data that satisfies a condition from the large amount of data, for example, to select the data with the highest K values from the large amount of data. The traditional processing method needs to sort the data first and then take the first K sorted data. When the amount of data to be sorted is large, there are problems of long sorting time, low processing efficiency, and the like.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a data processing method and apparatus and related products capable of improving data processing efficiency.

According to an aspect of the present disclosure, there is provided a data processing method applied to a processor including a plurality of processing units, the method including: when the first quantity of the data to be processed is larger than or equal to the preset quantity, determining a first processing unit for processing the data to be processed from the plurality of processing units; dividing the data to be processed into a plurality of first data groups according to the second number of the first processing units; selecting K first data meeting a first preset condition from each first data group, wherein K is a positive integer; and selecting K target data meeting the first preset condition from a second data group, wherein the second data group comprises corresponding first data in each first data group.

According to another aspect of the present disclosure, there is provided a data processing apparatus, the apparatus being applied to a processor including a plurality of processing units, the apparatus including: the processing unit determining module is used for determining a first processing unit for processing the data to be processed from the plurality of processing units when the first quantity of the data to be processed is greater than or equal to a preset quantity; the data group dividing module is used for dividing the data to be processed into a plurality of first data groups according to the second number of the first processing units; the first data selection module is used for selecting K first data meeting a first preset condition in each first data group, wherein K is a positive integer; and the second data selection module is used for selecting K target data meeting the first preset condition from a second data group, wherein the second data group comprises corresponding first data in each first data group.

According to another aspect of the present disclosure, there is provided an artificial intelligence chip comprising a data processing apparatus as described above.

According to another aspect of the present disclosure, there is provided an electronic device including the artificial intelligence chip as described above.

According to another aspect of the present disclosure, a board card is provided, which includes: a memory device, an interface device and a control device and an artificial intelligence chip as described above; wherein, the artificial intelligence chip is respectively connected with the storage device, the control device and the interface device; the storage device is used for storing data; the interface device is used for realizing data transmission between the artificial intelligence chip and external equipment; and the control device is used for monitoring the state of the artificial intelligence chip.

According to the embodiment of the disclosure, when the amount of data to be processed exceeds the preset amount, the data to be processed is divided into multiple groups according to the number of available processing units, K data meeting the preset condition are selected from each group of data, and K target data meeting the preset condition are selected from the K data in each group, so that the advantage of multiple cores (processing units) of a processor can be fully utilized to process large-scale data, the processing pressure of a single core is reduced, the data processing time is further reduced, and the data processing efficiency is improved.

Through deducing technical characteristics in the claims, the beneficial effects corresponding to the technical problems in the background art can be achieved. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

Fig. 1 shows a schematic diagram of a processor of a data processing method according to an embodiment of the present disclosure.

Fig. 2 shows a flow diagram of a data processing method according to an embodiment of the present disclosure.

Fig. 3 shows a flowchart of step S13 of the data processing method according to an embodiment of the present disclosure.

Fig. 4 shows a flowchart of step S14 of the data processing method according to an embodiment of the present disclosure.

Fig. 5 shows a flowchart of step S14 of the data processing method according to an embodiment of the present disclosure.

Fig. 6 is a schematic diagram showing an application example of the data processing method according to the embodiment of the present disclosure.

Fig. 7 is a schematic diagram showing an application example of the data processing method according to the embodiment of the present disclosure.

Fig. 8 shows a block diagram of a data processing apparatus according to an embodiment of the present disclosure.

Fig. 9 shows a block diagram of a board card according to an embodiment of the present disclosure.

Detailed Description

The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are some, but not all embodiments of the present disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.

It should be understood that the terms "first," "second," "third," and "fourth," etc. in the claims, description, and drawings of the present disclosure are used to distinguish between different objects and are not used to describe a particular order. The terms "comprises" and "comprising," when used in the specification and claims of this disclosure, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It is also to be understood that the terminology used in the description of the disclosure herein is for the purpose of describing particular embodiments only, and is not intended to be limiting of the disclosure. As used in the specification and claims of this disclosure, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should be further understood that the term "and/or" as used in the specification and claims of this disclosure refers to any and all possible combinations of one or more of the associated listed items and includes such combinations.

As used in this specification and claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to a determination" or "in response to a detection". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".

The data Processing method according to the embodiment of the present disclosure may be applied to a processor, which may be a general-purpose processor, such as a Central Processing Unit (CPU), or an artificial Intelligence Processor (IPU) for performing artificial intelligence operations. The artificial intelligence operations may include machine learning operations, brain-like operations, and the like. The machine learning operation comprises neural network operation, k-means operation, support vector machine operation and the like. The artificial intelligence processor may include, for example, one or a combination of a GPU (Graphics Processing Unit), a NPU (Neural-Network Processing Unit), a DSP (Digital Signal Processing Unit), and a Field Programmable Gate Array (FPGA) chip. The present disclosure is not limited to a particular type of processor.

In one possible implementation, the processor referred to in this disclosure may include multiple processing units, each of which may independently run various tasks assigned thereto, such as: a convolution operation task, a pooling task, a full connection task, or the like. The present disclosure is not limited to processing units and tasks executed by processing units.

Fig. 1 shows a schematic diagram of a processor of a data processing method according to an embodiment of the present disclosure. As shown in fig. 1, processor 100 includes a plurality of processing units 101 and a storage unit 102, where the plurality of processing units 101 is configured to execute instruction sequences, and the storage unit 102 is configured to store data, and may include a Random Access Memory (RAM) and a register file. Multiple processing units 101 in processor 100 may share a portion of memory, such as a portion of RAM memory and a register file, or may have separate memory spaces at the same time.

Fig. 2 shows a flow diagram of a data processing method according to an embodiment of the present disclosure. As shown in fig. 2, the data processing method includes:

in step S11, when the first number of the data to be processed is greater than or equal to the preset number, a first processing unit for processing the data to be processed is determined from the plurality of processing units.

Wherein the predetermined number may be used to represent the size of the data. The preset number may be represented by the number of data, for example, the preset number is 100 ten thousand data; the preset amount may also be represented by the size of the storage space occupied by the data, for example, the preset amount is 10GB of data. The preset number may be determined according to actual conditions, for example, the preset number may be determined according to time limit requirements of data processing. The present disclosure does not limit the setting manner and specific values of the preset number.

In one possible implementation, a first amount of data to be processed may be determined first, and then the first amount may be compared with a preset amount. When the first number is greater than or equal to the preset number, the data to be processed can be regarded as large-scale data, and the data can be processed by using a plurality of processing units, so that the data processing efficiency is improved. A currently available processing unit (e.g., a processing unit that is currently idle or is about to be idle) may be selected from among the plurality of processing units of the processor, and the selected processing unit may be determined as the first processing unit for processing the data to be processed. Those skilled in the art can determine the available processing units according to practical situations, and the present disclosure is not limited thereto.

Step S12, dividing the data to be processed into a plurality of first data groups according to the second number of the first processing units.

In one possible implementation manner, the data to be processed may be divided into a plurality of first data groups according to the second number of the first processing units. The number of the first data groups may be equal to the second number of the first processing units, that is, all the first processing units are used to process the data to be processed, for example, the second number of the first processing units is 5, the data to be processed may be divided into 5 first data groups, and each first processing unit processes one first data group.

In a possible implementation manner, the number of the first data groups may also be smaller than the second number of the first processing units, that is, a part of the first processing units is selected to process the data to be processed, and the rest of the first processing units may be used for other processing according to actual situations. For example, the second number of the first processing units is 5, 4 first processing units may be selected from the first processing units to process the data to be processed, and the data to be processed is divided into 4 first data groups, and the remaining 1 first processing unit may be used to process the processed data again.

In one possible implementation, the number of data in the first data group may be the same or different. The skilled person can determine the amount of data in each first data set according to practical situations, and the present disclosure does not limit this.

In a possible implementation manner, after dividing the first data group, the first processing unit may obtain the data in the first data group according to a data address of the first data in the first data group and a data length of the first data group. The data address may be an absolute address, i.e. a physical address of the first data, or a relative address, i.e. an offset of the first data with respect to other addresses.

Step S13, in each first data group, K first data satisfying a first preset condition are selected, where K is a positive integer.

The first preset condition may be used to indicate a condition for selecting data, for example, the first preset condition may be to select data with the largest K number or select data with the smallest K number, and the specific content of the first preset condition is not limited in the present disclosure.

In a possible implementation manner, the first processing unit may process, for example, compare, sort, and the like, data in the first data group according to a first preset condition, and then select K first data that satisfy the first preset condition.

Step S14, select K target data satisfying the first preset condition from a second data group, where the second data group includes corresponding first data in each first data group.

In a possible implementation manner, the K first data sets respectively selected from each first data set may be merged to determine the second data set. The order of data merging may be set according to the selection completion order of the K first data, may also be set randomly, or may be set according to other manners. For example, after two or more first data groups complete the selection of K first data, the selected first data groups begin to be merged until all the first data groups are merged to form a second data group; or after the K first data of each first data group are selected, merging the data to form a second data group; or otherwise merge the data. The present disclosure does not limit the order or manner of data merging.

In a possible implementation manner, after the second data group is determined, K target data meeting the first preset condition may be selected from the second data group. The K selected target data may be a numerical value of the target data, an identifier or a subscript (which may be used to indicate a position index of the target data in the data to be processed) of the target data, or in other manners, which is not limited in this disclosure.

Fig. 3 shows a flowchart of step S13 of the data processing method according to an embodiment of the present disclosure. As shown in fig. 3, step S13 may include:

step S131, reading N data from the first data group as first input data, where N is a positive integer and N is greater than K.

In one possible implementation, N may be used to represent the maximum amount of data that the first processing unit may process in parallel in one computational cycle.

In a possible implementation manner, when the first processing unit processes the data in the first data group, N data may be first read from the first data group as the first input data. When the N data are read, the N data may be sequentially read in sequence, may be randomly read, or may be read in other sequences. The present disclosure does not limit the reading order of the N data.

Step S132, selecting K sixth data satisfying a first preset condition from the first input data.

In a possible implementation manner, after reading the first input data, the first processing unit may select the first input data according to a first preset condition, and select K sixth data that satisfy the first preset condition from the first input data. For example, the first preset condition is to select K pieces of data with the largest number, and the first processing unit may select the first input data according to the first preset condition by using data processing methods such as comparison and sorting, and select the K pieces of data with the largest number from the first input data, that is, select K pieces of sixth data that satisfy the first preset condition.

Step S133, using N-K data of the K sixth data and the data other than the first input data in the first data group as second input data.

In a possible implementation manner, after the first processing unit selects K sixth data from the first input data, N-K data may be read from data other than the first input data in the first data group, and the K sixth data and the read N-K data may be used as the second input data. The reading manner of the N-K data is similar to that of the N data in step S131, and is not described herein again.

Step S134, selecting K seventh data satisfying a first preset condition from the second input data.

In a possible implementation manner, after determining the second input data, the first processing unit may select K seventh data satisfying the first preset condition from the second input data. For example, the first preset condition is to select the K pieces of data with the largest number, and the first processing unit may select the second input data according to the first preset condition by using data processing manners such as comparison and sorting, and select the K pieces of data with the largest number from the second input data, that is, select the K pieces of seventh data that satisfy the first preset condition.

In a possible implementation manner, after the K seventh data meeting the first preset condition are selected from the second input data, the first processing unit may continue to select the remaining data in the first data group (i.e., the unprocessed data in the first data group) by using the above method, that is, read N-K data from the remaining data in the first data group, use the K seventh data and the read N-K data as new input data, and then reselect the K seventh data meeting the first preset condition from the new input data. The above process is repeated until all the data in the first data set are selected.

In step S135, after all the data in the first data group are selected, the last selected K seventh data are determined as K first data meeting the first preset condition.

In a possible implementation manner, since the input data each time includes the data selected last time, after all the data in the first data group are selected, the K seventh data selected last time may be determined as the K first data selected from the first data group and satisfying the first preset condition.

In this embodiment, when the first processing unit selects K first data satisfying the first preset condition from the first data group, n data may be first read from the first data group and K data satisfying a first preset condition may be selected from the N data according to the data processing capability of the first processing unit, then N-K data are read from the remaining data in the first data group (i.e. the data in the first data group that has not been read), and reselects K data satisfying the first preset condition from the read N-K data and the K data selected last time, repeats the process, iteratively selecting the data in the first data group until all data in the first data group are selected, and determining the last selected K data as K data meeting a first preset condition. The data processing capacity of the first processing unit can be fully utilized, and meanwhile, the data in the first data group can be selected in an iteration mode, so that the data selecting speed is increased, and the data processing efficiency is improved.

In one possible implementation manner, step S132 may include: comparing data in the first input data with a first reference threshold respectively, and determining M eighth data meeting a third preset condition in the first input data; and determining the eighth data as K sixth data satisfying a first preset condition in the case that M is equal to K.

Wherein the first reference threshold may be a reference value used to compare with data in the first input data. The first data in the first input data may be selected as the first reference threshold, or one data may be randomly selected from the first input data as the first reference threshold, or the first reference threshold may be selected in other manners. The present disclosure does not limit the selection manner and specific value of the first reference threshold.

In a possible implementation manner, after the first reference threshold is determined, the data in the first input data may be respectively compared with the first reference threshold, and M eighth data satisfying a third preset condition in the first input data are determined. The third preset condition may be a condition related to the first preset condition, for example, the first preset condition is to select the data with the largest K values, and the third preset condition is to select the data larger than the first reference threshold. The third preset condition can be set by those skilled in the art according to actual needs, and the disclosure does not limit the specific content of the third preset condition.

In a possible implementation manner, when the data in the first input data are respectively compared with the first reference threshold, the index data may be set to identify whether the data in the first input data satisfy a third preset condition, so as to obtain an index data group corresponding to the first input data. For example, the value of the index data may be set to 0 and 1, where 1 may indicate that the data corresponding to the index data satisfies the third preset condition, and 0 may indicate that the data corresponding to the index data does not satisfy the third preset condition; the value of the index data may also be set in other ways, which is not limited in this disclosure.

In a possible implementation manner, the first input data may be selected according to the index data group corresponding to the first input data and the value of the index data, so as to obtain eighth data meeting a third preset condition and determine the number M of the eighth data. For example, a value of 1 for the index data indicates that the data corresponding to the index data satisfies the third preset condition, and according to the index data having a value of 1 in the index data group, eighth data satisfying the third preset condition may be selected from the first input data and the number M of the eighth data may be determined.

In a possible implementation manner, after the M eighth data are determined, a relationship between the number M of the selected eighth data and the number K of the sixth data to be selected may be determined. If M is equal to K, that is, the eighth data selected according to the third preset condition also meets the first preset condition, the M eighth data may be determined as K sixth data meeting the first preset condition.

For example, the first preset condition is to select the data with the largest K number, where K is 5, and the third preset condition is to select the data larger than the first reference threshold. And selecting the first input data according to a third preset condition to obtain M-5 eighth data. The number M of the eighth data is equal to K, and M eighth data may be determined as K sixth data satisfying the first preset condition.

In this embodiment, by determining the first reference threshold, and comparing the data in the first input data with the first reference threshold, M eighth data meeting the third preset condition are selected from the first input data, and when M is equal to K, the eighth data is determined as K sixth data meeting the first preset condition, so that the K sixth data meeting the first preset condition can be selected from the first input data in a data comparison manner, which is simple and fast, thereby reducing the complexity of data processing and improving the data processing efficiency.

In a possible implementation manner, step S132 may further include: and determining the eighth data and the first reference threshold as K sixth data satisfying a first preset condition if M is equal to K-1.

Where M is equal to K-1, it may indicate that the number of the eighth data selected according to the third preset condition is less than 1 of the number of the sixth data to be selected, and 1 data needs to be additionally selected from the first input data.

In this implementation manner, when M is equal to K-1, since only one data needs to be complementarily selected, the first reference threshold may be directly used as the complementarily selected data, and then the M eighth data and the first reference threshold are determined as the K sixth data satisfying the first preset condition.

For example, the first preset condition is to select the data with the largest K number, where K is 5, and the third preset condition is to select the data larger than the first reference threshold. And selecting the first input data according to a third preset condition to obtain 4 eighth data. In order to select the K sixth data, when M is equal to K-1, 1 data equal to the first reference threshold needs to be additionally selected. Since the first reference threshold is data selected from the first input data, the first reference threshold may be directly used as the complementarily selected data, and then the eighth data and the first reference threshold may be determined as the K sixth data satisfying the first preset condition.

In this embodiment, when M is equal to K-1, that is, when 1 data needs to be complementarily selected, the first reference threshold may be directly used as the complementarily selected data without performing data selection again, and the M eighth data and the first reference threshold are determined as the K sixth data satisfying the first preset condition, so that the selection efficiency of the sixth data can be improved, and the data processing efficiency is further improved.

In a possible implementation manner, step S132 may further include: comparing the eighth data with a second reference threshold respectively under the condition that M is larger than K, and determining ninth data meeting a fourth preset condition in the eighth data; determining the ninth data as K sixth data satisfying a first preset condition, in a case where the number of the ninth data is equal to K.

Where M is greater than K may indicate that the number of eighth data selected according to the third preset condition exceeds the number of sixth data to be selected. And in the case that M is larger than K, data selection needs to be carried out again from M eighth data.

In a possible implementation manner, the fourth preset condition may be used to select data again, and the fourth preset condition may be a condition similar to the third preset condition, for example, the third preset condition is to select data larger than the first reference threshold, the fourth preset condition is to select data larger than the second reference threshold, and the second reference threshold may be larger than the first reference threshold. The selection manner of the second reference threshold is similar to that of the first reference threshold, and is not repeated here.

In a possible implementation manner, after the M eighth data are determined, a second reference threshold may be selected from the M eighth data when M is greater than K, and the M eighth data are respectively compared with the second reference threshold, so as to determine ninth data that satisfies a fourth preset condition in the eighth data.

In a possible implementation manner, after the ninth data meeting the fourth preset condition is selected, the relationship between the number of the ninth data and the number K of the sixth data to be selected may be determined. If M is equal to K, that is, the ninth data selected according to the fourth preset condition also meets the first preset condition, the ninth data may be determined as K sixth data meeting the first preset condition.

In a possible implementation manner, if the number of the selected ninth data is greater than K, a new reference threshold may be determined again, and data selection is performed again in the ninth data. The data selection may be performed iteratively for a plurality of times until the number of the ninth data selected last is equal to K or equal to K-1 or less than K-1, and then different processes are performed according to the relationship between the number of the ninth data selected last and K, respectively.

In this embodiment, when M is greater than K, ninth data meeting a fourth preset condition is determined from the eighth data by comparing the eighth data with the second reference threshold, and when the number of the ninth data is equal to K, the ninth data is determined as K sixth data meeting the first preset condition, so that when the ninth data is selected, the ninth data is selected from the eighth data without being selected from the first input data, thereby reducing the number of data comparison and improving the data processing efficiency.

In a possible implementation manner, step S132 may further include: under the condition that M is smaller than K-1, K-M tenth data are selected from data except the eighth data in the first input data; and taking the eighth data and the tenth data as K sixth data meeting a first preset condition.

Where M is smaller than K-1 may indicate that the number of eighth data selected according to the third preset condition is smaller than the number of sixth data to be selected. In case M is smaller than K-1, K-M data need to be additionally selected from the first input data.

In a possible implementation manner, when M is smaller than K-1, K-M tenth data may be selected from data other than the eighth data in the first input data, and the eighth data and the tenth data may be used as K sixth data meeting a first preset condition. For example, the first preset condition is to select the data with the largest K number, where K is 5, and the third preset condition is to select the data larger than the first reference threshold. Firstly, the first input data can be selected according to a third preset condition, and M is 3 eighth data; according to M < K-1, K-M is required to be selected as 2 tenth data; then, 2 tenth data may be selected from data other than the eighth data in the first input data, and the eighth data and the tenth data may be used as K sixth data satisfying the first preset condition.

In this embodiment, when M < K-1, that is, when the number of the eighth data is insufficient, K-M tenth data can be selected from data other than the eighth data in the first input data according to the number (K-M) of the missing data, and the eighth data and the tenth data are used as K sixth data satisfying the first preset condition, so that when the number of the eighth data is insufficient, the data can be selected from the unselected data in the first input data in a supplementary manner, thereby reducing the number of data comparison and improving the data processing efficiency.

In a possible implementation manner, selecting K-M tenth data from data other than the eighth data in the first input data may include: comparing data except the eighth data in the first input data with the first reference threshold respectively, and determining T eleventh data meeting a fifth preset condition; in the case where T is equal to K-M, the eleventh data is determined as tenth data.

The fifth preset condition may be used to supplement the selected data in the data other than the eighth data in the first input data. The fifth predetermined condition may be related to but not overlapping with the third predetermined condition, for example, the third predetermined condition is to select data larger than the first reference threshold, and the fifth predetermined condition may be to select data equal to the first reference threshold. The fifth preset condition can be set by those skilled in the art according to actual needs, and the disclosure does not limit this.

In a possible implementation manner, when K-M tenth data are selected from data other than the eighth data in the first input data, the data other than the eighth data in the first input data may be respectively compared with the first reference threshold according to a fifth preset condition to determine T eleventh data satisfying the fifth preset condition, and then a relationship between the number T of the selected eleventh data and the number K-M of the tenth data to be selected may be determined. In the case where T is equal to K-M, eleventh data may be determined as tenth data.

In this embodiment, a fifth preset condition related to the third preset condition but not overlapped is used, T eleventh data are selected from data other than the eighth data in the first input data, and the eleventh data are determined as the tenth data when T is equal to K-M, so that multiple data selections caused by overlapping selection conditions can be avoided, and the data processing efficiency can be improved.

In a possible implementation manner, selecting K-M tenth data from data other than the eighth data in the first input data, further includes: and under the condition that T is larger than K-M, selecting K-M eleventh data from the T eleventh data to determine the tenth data.

That is, in the case that the number T of eleventh data satisfying the fifth preset condition is greater than K-M, K-M eleventh data may be directly selected from the T eleventh data in an input order or randomly or in another manner, and the selected K-M eleventh data may be determined as tenth data. It should be understood that the present disclosure does not limit the manner in which K-M eleventh data are selected from the T eleventh data.

In the embodiment, when T is greater than K-M, K-M eleventh data are directly selected from the T eleventh data to be determined as tenth data without data comparison, which is simple and fast and can improve data processing efficiency.

In a possible implementation manner, selecting K-M tenth data from data other than the eighth data in the first input data, further includes: selecting K-M-T twelfth data meeting a sixth preset condition from the eighth data and the data except the eleventh data in the first input data under the condition that T is smaller than K-M; and determining the eleventh data and the twelfth data as tenth data.

The sixth preset condition may be used to supplement the selected data in the data other than the eighth data and the eleventh data in the first input data. The sixth preset condition may be similar to the third preset condition, for example, the third preset condition is to select data larger than the first reference threshold, and the sixth preset condition may be to select data larger than the third reference threshold. The third reference threshold may be less than the first reference threshold. The selection manner of the third reference threshold is similar to that of the first reference threshold, and is not repeated here.

In a possible implementation manner, when T is less than K-M, data other than the eighth data and the eleventh data in the first input data may be respectively compared with a third reference threshold according to a sixth preset condition, K-M-T twelfth data meeting the sixth preset condition may be selected, and the eleventh data and the twelfth data may be determined as tenth data.

In one possible implementation, in the case that the number of the selected twelfth data is greater than K-M-T, a new reference threshold may be re-selected using a similar processing method as "in the case that M is greater than K", data is re-selected in the twelfth data, and data selection may be performed in multiple iterations until the number of selected data is equal to K-M-T or equal to K-M-T-1 or less than K-M-T-1. In the case where the number of pieces of picked-up data is smaller than K-M-T-1, the picked-up data may be supplemented using a processing method similar to that in the case where M is smaller than K-1 until the number of pieces of picked-up data is equal to K-M-T.

In one possible implementation, in the case that the number of the selected twelfth data is equal to K-M-T-1, the eleventh data, the twelfth data, and the third reference threshold may be determined as the tenth data using a similar processing method as "in the case that M is equal to K-1".

In one possible implementation, in the case that the number of the selected twelfth data is less than K-M-T-1, the selected data may be supplemented using a similar processing method as "in the case that M is less than K-1" until the number of the selected data is equal to K-M-T.

In this embodiment, when T is less than K-M-1, that is, when the number of eleventh data is insufficient, twelfth data can be selected from the eighth data and the data other than the eleventh data in the first input data according to the number of missing data (K-M-T), and the eleventh data and the twelfth data can be used as tenth data, so that when the number of eleventh data is insufficient, the selected data can be supplemented from the data that is not selected in the first input data, thereby improving the data processing efficiency.

In a possible implementation manner, the data selecting manner of step S134 (selecting K seventh data satisfying the first preset condition from the second input data) is similar to that of step S132, and is not described herein again.

Fig. 4 shows a flowchart of step S14 of the data processing method according to an embodiment of the present disclosure. As shown in fig. 4, step S14 includes:

in step S141, when the third number of data in the second data group is greater than or equal to the preset number, a second processing unit for processing the data in the second data group is determined from the plurality of processing units.

In one possible implementation manner, K first data selected from each first data group may be determined as a second data group, a third number of data in the second data group is determined, and then the third number is compared with a preset number.

In a possible implementation manner, when the third number is greater than or equal to the preset number, the data in the second data group may be regarded as large-scale data, and a plurality of processing units may be used for data processing, so as to improve data processing efficiency. A currently available processing unit (e.g., a processing unit that is currently idle or about to be idle) may be selected from the plurality of processing units and the selected processing unit may be determined as the second processing unit for processing data in the second data group. Those skilled in the art can determine the available processing units according to practical situations, and the present disclosure is not limited thereto.

In a possible implementation manner, since the third amount of data in the second data group is smaller than the first amount of data to be processed, a part of the first processing units may be selected from the first processing units as the second processing units. For example, when the number of the second processing units is 1/4 of the first processing unit, that is, the second number of the first processing units is 8, 2 of the processing units may be selected as the second processing units.

Step S142, dividing the data in the second data group into a plurality of third data groups according to the fourth number of the second processing units.

In a possible implementation manner, after the second processing units for processing the data in the second data group are determined, the data in the second data group may be divided into a plurality of third data groups according to the fourth number of the second processing units. The third data group is divided in a manner similar to that of the first data group in step S12, and will not be described again here.

Step S143, selecting K second data meeting a first preset condition in each third data group;

in a possible implementation manner, after the plurality of third data groups are divided, K second data meeting the first preset condition may be respectively selected from each third data group. The selection manner of the second data is similar to the selection manner of the first data in step S13, and is not described herein again.

Step S144, selecting K target data meeting the first preset condition from a fourth data group, where the fourth data group includes K second data in each third data group.

In a possible implementation manner, the K second data respectively selected from the third data group may be determined as a fourth data group, and the data in the fourth data group is compared, sorted, and the like, and K target data meeting the first preset condition is selected from the fourth data group.

In a possible implementation manner, if the number of data in the fourth data set is still greater than or equal to the preset number, the steps S141 to S143 may be executed iteratively until the number of data in the last selected fourth data set is smaller than the preset number, and then the step S144 is executed to select K target data meeting the first preset condition from the fourth data set.

In a possible implementation manner, if the number of data in the fourth data group is less than the preset number, the data selecting manner described in step S144 may be adopted, the data may be selected by a data heap (see steps S145 to S148 for details), or other manners, which is not limited by the present disclosure.

In this embodiment, when the number of data in the second data group exceeds the preset number, the data in the second data group may be divided into multiple groups according to the number of processing units in the processor, which are used for processing the data in the second data group, K data meeting the preset condition are selected from each group, and K target data meeting the preset condition are selected from the K data in each group, so that when the data in the second data group is large-scale data, the advantage of multiple cores (processing units) of the processor may be fully utilized, the processing pressure of a single core is reduced, the data processing time is reduced, and the data processing efficiency is improved.

Fig. 5 shows a flowchart of step S14 of the data processing method according to an embodiment of the present disclosure. As shown in fig. 5, step S14 includes:

step S145, when the third number of data in the second data group is smaller than the preset number, selecting K third data from the second data group;

in a possible implementation manner, when the third amount of data in the second data is smaller than the preset amount, the data in the second data group may be considered as small-scale data, and K target data may be selected from the second data group by one processing unit.

In one possible implementation, the processing unit may first select K third data from the second data set. When the K third data are selected, the K third data may be randomly selected from the second data group, or the first K third data may be read in sequence, or other selection manners may be adopted, which is not limited in this disclosure.

Step S146, according to the first preset condition, setting the K third data as data heaps, where the data heaps include a maximum heap or a minimum heap.

The data heap is a special data structure, and can be regarded as a group of data of a complete binary tree structure, and is characterized in that: the value of a node in the heap is always no greater or no less than the value of its parent node. The data heaps may include a maximum heap (the data heap where the value of the root node is the largest) or a minimum heap (the data heap where the value of the root node is the smallest).

In one possible implementation manner, the K third data may be set as a maximum heap or a minimum heap according to a first preset condition. For example, the first preset condition is to select K pieces of data with the largest numerical value, and according to the first preset condition, the K pieces of third data can be set as the minimum heap; the first preset condition is to select K pieces of data with the smallest numerical value, and the K pieces of third data can be set as the largest heap according to the first preset condition.

Step S147, adjusting the data pile according to fourth data to obtain an adjusted data pile, where the fourth data includes data other than the K third data in the second data group.

In a possible implementation manner, after the K third data are set as the data heap, the data heap may be adjusted according to the fourth data, that is, data other than the K third data in the second data group. For example, the first preset condition is to select K data with the largest numerical value, according to the first preset condition, the K third data may be set as the minimum heap, then the fourth data other than the K third data in the second data group is read, and the data heap is adjusted according to the numerical value of the fourth data until the fourth data is read, so as to obtain the adjusted data heap.

Step S148, determining K target data satisfying the first preset condition in the second data group according to the adjusted data stack.

In a possible implementation manner, the adjusted data stack includes K data that is selected from the second data group and satisfies the first preset condition, so that the K data in the adjusted data stack may be determined as K target data that satisfies the first preset condition in the second data group.

In this embodiment, when the third quantity of data in the second data group is smaller than the preset quantity, K third data are selected from the second data group, the K third data are set as the data pile according to the first preset condition, then the adjusted data pile is obtained according to fourth data except the K third data in the second data group, and the data in the adjusted data pile is determined as K target data meeting the first preset condition in the second data group, so that when small-scale data are processed, data selection can be performed quickly by setting and adjusting the data pile, and the data processing efficiency is improved.

In one possible implementation manner, step S147 may include: comparing fifth data with a root node of the data pile, and judging whether the fifth data meets a second preset condition, wherein the fifth data is any one of the fourth data; when the fifth data meet the second preset condition, replacing a root node of the data heap with the fifth data, and adjusting the data heap according to the category of the data heap; and comparing each fifth data with the root node of the data pile to obtain the data pile after the last adjustment.

In one possible implementation, the second preset condition may be set according to the first preset condition. For example, if the first preset condition is that K pieces of data with the largest values are selected, the second preset condition may be that the fifth data is larger than the root node; if the first preset condition is that K pieces of data with the minimum value are selected, the second preset condition may be that the fifth data is smaller than the root node. The specific content of the second preset condition can be set by those skilled in the art according to actual needs, and the disclosure does not limit this.

In a possible implementation manner, when the data heap is adjusted according to the fourth data, the fifth data (i.e., any one of the fourth data) may be compared with the root node of the data heap, and whether the fifth data satisfies the second preset condition is determined.

In a possible implementation manner, when the fifth data meets a second preset condition, the root node of the data pile can be replaced by the fifth data, and then the data pile is adjusted according to the category of the data pile; and when the fifth data does not meet the second preset condition, the data pile is not adjusted. And after each fifth data is compared with the root node of the data heap, the data heap after the last adjustment can be obtained.

For example, the data heap is the smallest heap (i.e. the value of the root node of the data heap is the smallest in the data heap), the second preset condition is that the fifth data is larger than the root node, and the fifth data is compared with the root node of the data heap. And if the fifth data is larger than the root node, replacing the root node of the data pile with the fifth data, and adjusting the data pile according to the characteristics of the minimum pile (for example, the numerical value of any non-terminal node is not larger than the numerical value of the child node of the non-terminal node, and the numerical value of the root node is the minimum value in the data pile), so that the adjusted data pile is still the minimum pile. And if the fifth data is smaller than the root node, the data pile is not adjusted, and the next fifth data is continuously selected to be compared with the root node of the data pile. And after each fifth data is compared with the root node of the data heap, the data heap after the last adjustment can be obtained.

In this embodiment, whether the fifth data meets the second preset condition can be judged by comparing the fifth data with the root node of the data pile, when the fifth data meets the second preset condition, the root node of the data pile is replaced by the fifth data, the data pile is adjusted according to the category of the data class, after each fifth data is compared with the root node of the data pile, the data pile after the last adjustment is obtained, the adjusted data pile can be obtained by data comparison, replacement and data pile adjustment, the data selection is simple and efficient, and the data processing efficiency can be improved.

In one possible implementation, the method further includes: and when the first quantity is smaller than the preset quantity, K target data meeting the first preset condition are selected from the data to be processed.

In this implementation manner, when the first amount of the data to be processed is smaller than the preset amount, the data to be processed may be regarded as small-scale data, and K target data may be selected from the data to be processed by one processing unit.

In a possible implementation manner, K target data are selected from the data to be processed by one processing unit, and K target data meeting a first preset condition are selected from the data to be processed by data comparison in a data selection manner similar to that in step S13; k target data satisfying the first preset condition may be selected from the data to be processed by setting and adjusting the data pile in a data selection manner similar to that in step S14; or other data selection means. The present disclosure is not so limited.

In this embodiment, when the first amount of the data to be processed is smaller than the preset amount, that is, when the data to be processed is small-scale data, K target data can be selected from the data to be processed by one processing unit, so that the complexity of data selection can be reduced, and the data selection efficiency can be improved.

Fig. 6 is a schematic diagram showing an application example of the data processing method according to the embodiment of the present disclosure. As shown in fig. 6, this application example is applicable to a processor including a plurality of processing units, and is used to select K pieces of data satisfying a preset condition, that is, K pieces of target data, from data to be processed.

The number of data to be processed may be determined first in step S601; then, in step S602, it is determined whether the number of the data to be processed is greater than or equal to a preset number;

if the number of the data to be processed is greater than or equal to the preset number, that is, the data to be processed is large-scale data, executing step S603, and dividing the data to be processed into a plurality of data groups; then, step S604 is executed, a plurality of processing units are used to select K pieces of data meeting the preset condition from each data group, then step S605 is executed, the K pieces of data selected from each data group are merged, whether the number of the merged data is greater than or equal to the preset number is judged, if the number of the merged data is greater than or equal to the preset number, that is, the data obtained by merging the K pieces of data selected from each data group is large-scale data, step 603 is executed, the data are continuously divided into a plurality of data groups, and a plurality of processing units are used to select the data;

if the number of the data to be processed is smaller than the preset number, that is, the data to be processed is small-scale data, step S606 is executed, and K target data are selected from the data to be processed.

If the number of the data obtained by merging the K selected data in each data group is smaller than the preset number, that is, the data obtained by merging the K selected data in each data group is small-scale data, step S606 is executed to select K target data from the K selected data in each data group.

Fig. 7 is a schematic diagram showing an application example of the data processing method according to the embodiment of the present disclosure. As shown in fig. 7, this application example is applicable to data processing of one processing unit in a processor, and can be used for the processing unit to select K data with the largest value from a data group.

First, in step S701, it is determined whether the processing unit selects data for the first time, and if so, step 702 is executed to read N data from the data set as first input data; otherwise, executing step S717, reading N-K data from the remaining data in the data group, and using the K data selected last time and the N-K data read this time as second input data;

after reading the first input data or the second input data, in step S703, one data may be selected from the input data as a reference threshold, and then step S704 is performed to select M eighth data larger than the reference threshold from the input data by comparing the input data with the reference threshold;

after selecting M eighth data in step S704, in step S705, determining whether M is greater than K, if M is greater than K, executing step S706, re-selecting data from M eighth data that are greater than the reference threshold, otherwise, executing step S707, determining whether M is equal to K, if M is equal to K, executing step S708, taking M eighth data as the K data selected this time, otherwise, executing step S709, determining whether M is equal to K-1, if M is equal to K-1, executing step 710, taking M eighth data and the reference threshold as the K data selected this time, otherwise, executing step S711, and selecting T eleventh data that are equal to the reference threshold from data other than the eighth data in the input data;

after the T eleventh data are selected in step S711, it is determined whether T is greater than or equal to K in step S712, if T is greater than or equal to K, step S713 is executed to select K-M eleventh data from the T eleventh data and take the K-M eleventh data and the M eighth data as the K data selected this time, otherwise, step S714 is executed to select K-M-T twelfth data from the data smaller than the reference threshold through data comparison, and in step S715, the selected K-M-T twelfth data, the T eleventh data and the M eighth data are taken as the K data selected this time.

After the K data are selected in step S708 or step S710 or step S713 or step S517, it may be determined whether the data in the data group are completely selected in step S716, and if the data in the data group are not completely selected, the method jumps to step S701, continues to read the data, and performs data selection.

If it is determined in step S716 that the data selection in the data group is completed, the last selected K data are determined as the K data with the largest value in the data group, and the values and/or subscripts of the K data with the largest value are output.

According to the embodiment of the disclosure, when a plurality of data meeting a preset condition are selected from data to be processed, the data scale of the data to be processed can be determined according to the preset number, if the number of the data to be processed is greater than or equal to the preset number, the data to be processed is considered to be large-scale data, when the data are selected, the data to be processed can be divided into a plurality of data groups, the data are selected through a plurality of processing units respectively, the multi-core advantage of a processor is fully utilized, the single-core processing pressure is reduced, meanwhile, the data selection time can be reduced, and the data processing efficiency is improved; if the number of the data to be processed is smaller than the preset number, the data to be processed is considered to be small-scale data, and when data selection is carried out, the data selection can be carried out through one processing unit, so that the data selection complexity is reduced, and the data processing efficiency is improved.

In the data processing process, the multi-core advantage of the processor can be fully exerted, the data processing efficiency is improved, the resource waste of the processor can be reduced, and the resource utilization efficiency is improved by matching the number of the processing units for executing data processing with the scale of the data to be processed.

It is noted that while for simplicity of explanation, the foregoing method embodiments have been described as a series of acts or combination of acts, it will be appreciated by those skilled in the art that the present disclosure is not limited by the order of acts, as some steps may, in accordance with the present disclosure, occur in other orders and concurrently. Further, those skilled in the art should also appreciate that the embodiments described in the specification are exemplary embodiments and that acts and modules referred to are not necessarily required by the disclosure.

It should be further noted that, although the steps in the flowcharts are shown in sequence as indicated by the arrows, the steps are not necessarily executed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in each flowchart may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

Fig. 8 shows a block diagram of a data processing apparatus according to an embodiment of the present disclosure. The data processing device is applied to a processor which comprises a plurality of processing units. As shown in fig. 8, the data processing apparatus includes:

a processing unit determining module 81, configured to determine, when a first number of to-be-processed data is greater than or equal to a preset number, a first processing unit for processing the to-be-processed data from the plurality of processing units;

a data group dividing module 82, configured to divide the data to be processed into a plurality of first data groups according to the second number of the first processing units;

a first data selecting module 83, configured to select, in each first data group, K first data that satisfy a first preset condition, where K is a positive integer;

a second data selecting module 84, configured to select K target data that meet the first preset condition from a second data group, where the second data group includes corresponding first data in each first data group.

In a possible implementation manner, the second data selecting module 84 includes: the processing unit determining sub-module is used for determining a second processing unit for processing the data in the second data group from the plurality of processing units when the third quantity of the data in the second data group is greater than or equal to the preset quantity; the data group dividing submodule is used for dividing the data in the second data group into a plurality of third data groups according to the fourth quantity of the second processing units; the first data selection submodule is used for selecting K second data meeting a first preset condition in each third data group; and the second data selection submodule is used for selecting K target data meeting a first preset condition from a fourth data group, and the fourth data group comprises corresponding second data in each third data group.

In a possible implementation manner, the second data selecting module 84 includes: a third data selection submodule, configured to select K third data from the second data group when a third number of data in the second data group is smaller than the preset number; the data pile setting submodule is used for setting the K third data into a data pile according to the first preset condition, wherein the data pile comprises a maximum pile or a minimum pile; the data pile adjusting submodule is used for adjusting the data pile according to fourth data to obtain an adjusted data pile, and the fourth data comprise data except the K third data in the second data group; and the first data determining submodule is used for determining K target data meeting the first preset condition in the second data group according to the adjusted data pile.

In one possible implementation, the data heap adjustment submodule includes: the first comparison submodule is used for comparing fifth data with a root node of the data heap and judging whether the fifth data meets a second preset condition, wherein the fifth data is any one of the fourth data; the replacing submodule is used for replacing a root node of the data pile through the fifth data when the fifth data meet the second preset condition, and adjusting the data pile according to the category of the data pile; and the adjusting submodule is used for comparing each fifth data with the root node of the data pile to obtain the data pile after the last adjustment.

In a possible implementation manner, the first data selecting module 83 includes: the data reading submodule is used for reading N data from the first data group as first input data, wherein N is a positive integer and is greater than K; the fourth data selection submodule is used for selecting K sixth data meeting a first preset condition from the first input data; an input determination submodule configured to use the K sixth data and N-K data of data other than the first input data in the first data group as second input data; the fifth data selection submodule is used for selecting K seventh data meeting a first preset condition from the second input data; and the second data determining submodule is used for determining the last selected K seventh data as K first data meeting a first preset condition after all the data in the first data group are selected.

In a possible implementation manner, the fourth data selecting submodule includes: the second comparison submodule is used for comparing the data in the first input data with a first reference threshold respectively and determining M eighth data meeting a third preset condition in the first input data; and the third data determining submodule is used for determining the eighth data as K sixth data meeting the first preset condition under the condition that M is equal to K.

In a possible implementation manner, the fourth data selecting sub-module further includes: a fourth data determining sub-module, configured to determine, when M is equal to K-1, the eighth data and the first reference threshold as K sixth data that satisfy a first preset condition.

In a possible implementation manner, the fourth data selecting sub-module further includes: the third comparison submodule is used for comparing the eighth data with a second reference threshold respectively under the condition that M is larger than K, and determining ninth data meeting a fourth preset condition in the eighth data; a fifth data determining sub-module, configured to determine the ninth data as K sixth data that satisfy a first preset condition when the number of the ninth data is equal to K.

In a possible implementation manner, the fourth data selecting sub-module further includes: a sixth data selection submodule, configured to select K-M tenth data from data other than the eighth data in the first input data when M is smaller than K-1; and the sixth data determining submodule is used for taking the eighth data and the tenth data as K sixth data meeting a first preset condition.

In a possible implementation manner, the sixth data selecting submodule is configured to: comparing data except the eighth data in the first input data with the first reference threshold respectively, and determining T eleventh data meeting a fifth preset condition; in the case where T is equal to K-M, the eleventh data is determined as tenth data.

In a possible implementation manner, the sixth data selecting submodule is further configured to: and under the condition that T is larger than K-M, selecting K-M eleventh data from the T eleventh data to determine the tenth data.

In a possible implementation manner, the sixth data selecting submodule is further configured to: selecting K-M-T twelfth data meeting a sixth preset condition from the eighth data and the data except the eleventh data in the first input data under the condition that T is smaller than K-M; and determining the eleventh data and the twelfth data as tenth data.

In one possible implementation, the apparatus further includes: and the third data selection module is used for selecting K target data meeting the first preset condition from the data to be processed when the first quantity is smaller than the preset quantity.

It should be understood that the above-described apparatus embodiments are merely illustrative and that the apparatus of the present disclosure may be implemented in other ways. For example, the division of the units/modules in the above embodiments is only one logical function division, and there may be another division manner in actual implementation. For example, multiple units, modules, or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented.

In addition, unless otherwise specified, each functional unit/module in each embodiment of the present disclosure may be integrated into one unit/module, each unit/module may exist alone physically, or two or more units/modules may be integrated together. The integrated units/modules may be implemented in the form of hardware or software program modules.

If the integrated unit/module is implemented in hardware, the hardware may be digital circuits, analog circuits, etc. Physical implementations of hardware structures include, but are not limited to, transistors, memristors, and the like. The artificial intelligence processor may be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, ASIC, etc., unless otherwise specified. Unless otherwise specified, the Memory unit may be any suitable magnetic storage medium or magneto-optical storage medium, such as resistive Random Access Memory rram (resistive Random Access Memory), Dynamic Random Access Memory dram (Dynamic Random Access Memory), Static Random Access Memory SRAM (Static Random-Access Memory), enhanced Dynamic Random Access Memory edram (enhanced Dynamic Random Access Memory), High-Bandwidth Memory HBM (High-Bandwidth Memory), hybrid Memory cubic hmc (hybrid Memory cube), and so on.

The integrated units/modules, if implemented in the form of software program modules and sold or used as a stand-alone product, may be stored in a computer readable memory. Based on such understanding, the technical solution of the present disclosure may be embodied in the form of a software product, which is stored in a memory and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present disclosure. And the aforementioned memory comprises: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

In a possible implementation manner, an artificial intelligence chip is also disclosed, which comprises the data processing device.

In a possible implementation manner, a board card is further disclosed, which comprises a storage device, an interface device, a control device and the artificial intelligence chip; wherein, the artificial intelligence chip is respectively connected with the storage device, the control device and the interface device; the storage device is used for storing data; the interface device is used for realizing data transmission between the artificial intelligence chip and external equipment; and the control device is used for monitoring the state of the artificial intelligence chip.

Fig. 9 shows a block diagram of a board according to an embodiment of the present disclosure, and referring to fig. 9, the board may include other kit components besides the chip 389, where the kit components include, but are not limited to: memory device 390, interface device 391 and control device 392;

the storage device 390 is connected to the artificial intelligence chip through a bus for storing data. The memory device may include a plurality of groups of memory cells 393. Each group of the storage units is connected with the artificial intelligence chip through a bus. It is understood that each group of the memory cells may be a DDR SDRAM (Double Data Rate SDRAM).

DDR can double the speed of SDRAM without increasing the clock frequency. DDR allows data to be read out on the rising and falling edges of the clock pulse. DDR is twice as fast as standard SDRAM. In one embodiment, the storage device may include 4 sets of the storage unit. Each group of the memory cells may include a plurality of DDR4 particles (chips). In one embodiment, the artificial intelligence chip may include 4 72-bit DDR4 controllers, and 64 bits of the 72-bit DDR4 controller are used for data transmission, and 8 bits are used for ECC check. It can be understood that when DDR4-3200 particles are adopted in each group of memory cells, the theoretical bandwidth of data transmission can reach 25600 MB/s.

In one embodiment, each group of the memory cells includes a plurality of double rate synchronous dynamic random access memories arranged in parallel. DDR can transfer data twice in one clock cycle. And a controller for controlling DDR is arranged in the chip and is used for controlling data transmission and data storage of each memory unit.

The interface device is electrically connected with the artificial intelligence chip. The interface device is used for realizing data transmission between the artificial intelligence chip and external equipment (such as a server or a computer). For example, in one embodiment, the interface device may be a standard PCIE interface. For example, the data to be processed is transmitted to the chip by the server through the standard PCIE interface, so as to implement data transfer. Preferably, when PCIE 3.0X 16 interface transmission is adopted, the theoretical bandwidth can reach 16000 MB/s. In another embodiment, the interface device may also be another interface, and the disclosure does not limit the specific expression of the other interface, and the interface unit may implement the switching function. In addition, the calculation result of the artificial intelligence chip is still transmitted back to the external device (e.g. server) by the interface device.

The control device is electrically connected with the artificial intelligence chip. The control device is used for monitoring the state of the artificial intelligence chip. Specifically, the artificial intelligence chip and the control device can be electrically connected through an SPI interface. The control device may include a single chip Microcomputer (MCU). As the artificial intelligence chip can comprise a plurality of processing chips, a plurality of processing cores or a plurality of processing circuits, a plurality of loads can be driven. Therefore, the artificial intelligence chip can be in different working states such as multi-load and light load. The control device can realize the regulation and control of the working states of a plurality of processing chips, a plurality of processing circuits and/or a plurality of processing circuits in the artificial intelligence chip.

In one possible implementation, an electronic device is disclosed that includes the artificial intelligence chip described above. The electronic device comprises a data processing device, a robot, a computer, a printer, a scanner, a tablet computer, an intelligent terminal, a mobile phone, a vehicle data recorder, a navigator, a sensor, a camera, a server, a cloud server, a camera, a video camera, a projector, a watch, an earphone, a mobile storage, a wearable device, a vehicle, a household appliance, and/or a medical device. The vehicle comprises an airplane, a ship and/or a vehicle; the household appliances comprise a television, an air conditioner, a microwave oven, a refrigerator, an electric cooker, a humidifier, a washing machine, an electric lamp, a gas stove and a range hood; the medical equipment comprises a nuclear magnetic resonance apparatus, a B-ultrasonic apparatus and/or an electrocardiograph.

In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments. The technical features of the embodiments may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The foregoing may be better understood in light of the following clauses:

clause a1, a data processing method applied to a processor comprising a plurality of processing units, the method comprising:

when the first quantity of the data to be processed is larger than or equal to the preset quantity, determining a first processing unit for processing the data to be processed from the plurality of processing units;

dividing the data to be processed into a plurality of first data groups according to the second number of the first processing units;

selecting K first data meeting a first preset condition from each first data group, wherein K is a positive integer;

and selecting K target data meeting the first preset condition from a second data group, wherein the second data group comprises corresponding first data in each first data group.

Clause a2, selecting K target data satisfying the first preset condition from a second data group according to the method of clause a1, including:

when the third quantity of the data in the second data group is larger than or equal to the preset quantity, determining a second processing unit for processing the data in the second data group from the plurality of processing units;

dividing the data in the second data group into a plurality of third data groups according to the fourth number of the second processing units;

selecting K second data meeting a first preset condition from each third data group;

k target data meeting a first preset condition are selected from a fourth data group, and the fourth data group comprises corresponding second data in each third data group.

Clause A3, selecting K target data satisfying the first preset condition from a second data group according to the method of clause a1, including:

when the third quantity of the data in the second data group is smaller than the preset quantity, K third data are selected from the second data group;

setting the K third data as a data heap according to the first preset condition, wherein the data heap comprises a maximum heap or a minimum heap;

adjusting the data pile according to fourth data to obtain an adjusted data pile, wherein the fourth data comprises data except the K third data in a second data group;

and determining K target data meeting the first preset condition in the second data group according to the adjusted data stack.

Clause a4, adjusting the data heap according to the fourth data according to the method of clause A3, to obtain an adjusted data heap, including:

comparing fifth data with a root node of the data pile, and judging whether the fifth data meets a second preset condition, wherein the fifth data is any one of the fourth data;

when the fifth data meet the second preset condition, replacing a root node of the data heap with the fifth data, and adjusting the data heap according to the category of the data heap;

and comparing each fifth data with the root node of the data pile to obtain the data pile after the last adjustment.

Clause a5, selecting K first data satisfying a first preset condition in each first data group according to the method of clause a1, including:

reading N data from the first data group as first input data, wherein N is a positive integer and is greater than K;

selecting K pieces of sixth data meeting a first preset condition from the first input data;

taking the K pieces of sixth data and N-K pieces of data in the first data group except the first input data as second input data;

selecting K pieces of seventh data meeting a first preset condition from the second input data;

after all the data in the first data group are selected, determining the last selected K seventh data as K first data meeting a first preset condition.

Clause a6, selecting K sixth data satisfying a first preset condition from the first input data according to the method of clause a5, including:

comparing data in the first input data with a first reference threshold respectively, and determining M eighth data meeting a third preset condition in the first input data;

and determining the eighth data as K sixth data satisfying a first preset condition in the case that M is equal to K.

Clause a7, selecting K sixth data satisfying a first preset condition from the first input data according to the method of clause a6, further comprising:

and determining the eighth data and the first reference threshold as K sixth data satisfying a first preset condition if M is equal to K-1.

Clause A8, selecting K sixth data satisfying a first preset condition from the first input data according to the method of clause a6, further comprising:

comparing the eighth data with a second reference threshold respectively under the condition that M is larger than K, and determining ninth data meeting a fourth preset condition in the eighth data;

determining the ninth data as K sixth data satisfying a first preset condition, in a case where the number of the ninth data is equal to K.

Clause a9, selecting K sixth data satisfying a first preset condition from the first input data according to the method of clause a6, further comprising:

under the condition that M is smaller than K-1, K-M tenth data are selected from data except the eighth data in the first input data;

and taking the eighth data and the tenth data as K sixth data meeting a first preset condition.

Clause a10, selecting K-M tenth data from the data other than the eighth data in the first input data according to the method of clause a9, including:

comparing data except the eighth data in the first input data with the first reference threshold respectively, and determining T eleventh data meeting a fifth preset condition;

in the case where T is equal to K-M, the eleventh data is determined as tenth data.

Clause a11, selecting K-M tenth data from the data other than the eighth data in the first input data according to the method of clause a10, further comprising:

and under the condition that T is larger than K-M, selecting K-M eleventh data from the T eleventh data to determine the tenth data.

Clause a12, selecting K-M tenth data from the data other than the eighth data in the first input data according to the method of clause a10 or a11, further comprising:

selecting K-M-T twelfth data meeting a sixth preset condition from the eighth data and the data except the eleventh data in the first input data under the condition that T is smaller than K-M;

and determining the eleventh data and the twelfth data as tenth data.

Clause a13, the method of clause a1-a12, further comprising:

and when the first quantity is smaller than the preset quantity, K target data meeting the first preset condition are selected from the data to be processed.

Clause a14, a data processing apparatus applied to a processor comprising a plurality of processing units, the apparatus comprising:

the processing unit determining module is used for determining a first processing unit for processing the data to be processed from the plurality of processing units when the first quantity of the data to be processed is greater than or equal to a preset quantity;

the data group dividing module is used for dividing the data to be processed into a plurality of first data groups according to the second number of the first processing units;

the first data selection module is used for selecting K first data meeting a first preset condition in each first data group, wherein K is a positive integer;

and the second data selection module is used for selecting K target data meeting the first preset condition from a second data group, wherein the second data group comprises corresponding first data in each first data group.

Clause a15, the apparatus of clause a14, the second data selection module, comprising:

the processing unit determining sub-module is used for determining a second processing unit for processing the data in the second data group from the plurality of processing units when the third quantity of the data in the second data group is greater than or equal to the preset quantity;

the data group dividing submodule is used for dividing the data in the second data group into a plurality of third data groups according to the fourth quantity of the second processing units;

the first data selection submodule is used for selecting K second data meeting a first preset condition in each third data group;

and the second data selection submodule is used for selecting K target data meeting a first preset condition from a fourth data group, and the fourth data group comprises corresponding second data in each third data group.

Clause a16, the apparatus of clause a14, the second data selection module, comprising:

a third data selection submodule, configured to select K third data from the second data group when a third number of data in the second data group is smaller than the preset number;

the data pile setting submodule is used for setting the K third data into a data pile according to the first preset condition, wherein the data pile comprises a maximum pile or a minimum pile;

the data pile adjusting submodule is used for adjusting the data pile according to fourth data to obtain an adjusted data pile, and the fourth data comprise data except the K third data in the second data group;

and the first data determining submodule is used for determining K target data meeting the first preset condition in the second data group according to the adjusted data pile.

Clause a17, the apparatus of clause a16, the data heap adjustment submodule, comprising:

the first comparison submodule is used for comparing fifth data with a root node of the data heap and judging whether the fifth data meets a second preset condition, wherein the fifth data is any one of the fourth data;

the replacing submodule is used for replacing a root node of the data pile through the fifth data when the fifth data meet the second preset condition, and adjusting the data pile according to the category of the data pile;

and the adjusting submodule is used for comparing each fifth data with the root node of the data pile to obtain the data pile after the last adjustment.

Clause a18, the apparatus of clause a14, the first data extraction module comprising:

the data reading submodule is used for reading N data from the first data group as first input data, wherein N is a positive integer and is greater than K;

the fourth data selection submodule is used for selecting K sixth data meeting a first preset condition from the first input data;

an input determination submodule configured to use the K sixth data and N-K data of data other than the first input data in the first data group as second input data;

the fifth data selection submodule is used for selecting K seventh data meeting a first preset condition from the second input data;

and the second data determining submodule is used for determining the last selected K seventh data as K first data meeting a first preset condition after all the data in the first data group are selected.

Clause a19, the apparatus of clause a18, the fourth data extraction submodule comprising:

the second comparison submodule is used for comparing the data in the first input data with a first reference threshold respectively and determining M eighth data meeting a third preset condition in the first input data;

and the third data determining submodule is used for determining the eighth data as K sixth data meeting the first preset condition under the condition that M is equal to K.

Clause a20, the apparatus of clause a19, the fourth data extraction submodule, further comprising:

a fourth data determining sub-module, configured to determine, when M is equal to K-1, the eighth data and the first reference threshold as K sixth data that satisfy a first preset condition.

Clause a21, the apparatus of clause a19, the fourth data extraction submodule, further comprising:

the third comparison submodule is used for comparing the eighth data with a second reference threshold respectively under the condition that M is larger than K, and determining ninth data meeting a fourth preset condition in the eighth data;

a fifth data determining sub-module, configured to determine the ninth data as K sixth data that satisfy a first preset condition when the number of the ninth data is equal to K.

Clause a22, the apparatus of clause a19, the fourth data extraction submodule, further comprising:

a sixth data selection submodule, configured to select K-M tenth data from data other than the eighth data in the first input data when M is smaller than K-1;

and the sixth data determining submodule is used for taking the eighth data and the tenth data as K sixth data meeting a first preset condition.

Clause a23, the apparatus of clause a22, the sixth data extraction submodule, for:

Clause a24, the apparatus of clause a23, the sixth data extraction submodule, further configured to:

Clause a25, the apparatus of clause a23 or a24, the sixth data extraction submodule further configured to:

and determining the eleventh data and the twelfth data as tenth data.

Clause a26, the apparatus of clause a14-a25, further comprising:

and the third data selection module is used for selecting K target data meeting the first preset condition from the data to be processed when the first quantity is smaller than the preset quantity.

Clause a27, an artificial intelligence chip, the chip comprising the data processing apparatus of any one of clauses a13-a 26.

Clause a28, an electronic device comprising the artificial intelligence chip of clause a 27.

Clause a29, a card, comprising: a memory device, an interface device and a control device and an artificial intelligence chip as described in clause a 27;

wherein, the artificial intelligence chip is respectively connected with the storage device, the control device and the interface device;

the storage device is used for storing data;

the interface device is used for realizing data transmission between the artificial intelligence chip and external equipment;

and the control device is used for monitoring the state of the artificial intelligence chip.

The embodiments of the present disclosure have been described in detail, and the principles and embodiments of the present disclosure are explained herein using specific examples, which are provided only to help understand the method and the core idea of the present disclosure. Meanwhile, a person skilled in the art should, based on the idea of the present disclosure, change or modify the specific embodiments and application scope of the present disclosure. In view of the above, the description is not intended to limit the present disclosure.

Claims

1. A data processing method applied to a processor including a plurality of processing units, the method comprising:

2. The method of claim 1, wherein selecting K target data from a second data group that satisfy the first predetermined condition comprises:

3. The method of claim 1, wherein selecting K target data from a second data group that satisfy the first predetermined condition comprises:

4. The method of claim 3, wherein adjusting the data heap according to the fourth data to obtain an adjusted data heap comprises:

5. The method of claim 1, wherein selecting K first data satisfying a first predetermined condition in each first data group comprises:

6. The method according to claim 5, wherein selecting K sixth data satisfying a first preset condition from the first input data comprises:

7. The method according to claim 6, wherein selecting K sixth data satisfying a first preset condition from the first input data further comprises:

8. The method according to claim 6, wherein selecting K sixth data satisfying a first preset condition from the first input data further comprises:

9. The method according to claim 6, wherein selecting K sixth data satisfying a first preset condition from the first input data further comprises:

10. The method according to claim 9, wherein selecting K-M tenth data from data other than the eighth data in the first input data comprises:

11. The method according to claim 10, wherein K-M tenth data are selected from data other than the eighth data in the first input data, further comprising:

12. The method according to claim 10 or 11, wherein K-M tenth data are selected from data other than the eighth data in the first input data, further comprising:

and determining the eleventh data and the twelfth data as tenth data.

13. The method of claims 1-12, further comprising:

14. A data processing apparatus, applied to a processor comprising a plurality of processing units, the apparatus comprising:

15. The apparatus of claim 14, wherein the second data extracting module comprises:

16. The apparatus of claim 14, wherein the second data extracting module comprises:

17. The apparatus of claim 16, wherein the data heap adjustment submodule comprises:

18. The apparatus of claim 14, wherein the first data selecting module comprises:

19. The apparatus of claim 18, wherein the fourth data selection submodule comprises:

20. The apparatus of claim 19, wherein the fourth data selection submodule further comprises:

21. The apparatus of claim 19, wherein the fourth data selection submodule further comprises:

22. The apparatus of claim 19, wherein the fourth data selection submodule further comprises:

23. The apparatus of claim 22, wherein the sixth data selection submodule is configured to:

24. The apparatus of claim 23, wherein the sixth data selection submodule is further configured to:

25. The apparatus according to claim 23 or 24, wherein the sixth data selecting sub-module is further configured to:

and determining the eleventh data and the twelfth data as tenth data.

26. The apparatus of claims 14-25, further comprising:

27. An artificial intelligence chip, wherein the chip comprises a data processing apparatus according to any one of claims 13 to 26.

28. An electronic device, characterized in that the electronic device comprises an artificial intelligence chip according to claim 27.

29. The utility model provides a board card, its characterized in that, the board card includes: a memory device, an interface device and a control device and an artificial intelligence chip according to claim 27;

the storage device is used for storing data;