CN113961944A - Method, apparatus, device and storage medium for processing data - Google Patents
Method, apparatus, device and storage medium for processing data Download PDFInfo
- Publication number
- CN113961944A CN113961944A CN202010700636.2A CN202010700636A CN113961944A CN 113961944 A CN113961944 A CN 113961944A CN 202010700636 A CN202010700636 A CN 202010700636A CN 113961944 A CN113961944 A CN 113961944A
- Authority
- CN
- China
- Prior art keywords
- data
- processing
- parameter
- target
- parameter value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 title claims abstract description 149
- 238000000034 method Methods 0.000 title claims abstract description 80
- 230000008569 process Effects 0.000 claims abstract description 33
- 230000015654 memory Effects 0.000 claims abstract description 25
- 238000004364 calculation method Methods 0.000 claims abstract description 23
- 230000006870 function Effects 0.000 claims description 27
- 230000004044 response Effects 0.000 claims description 8
- 238000012217 deletion Methods 0.000 claims description 5
- 230000037430 deletion Effects 0.000 claims description 5
- 238000010586 diagram Methods 0.000 description 8
- 239000003795 chemical substances by application Substances 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 206010012601 diabetes mellitus Diseases 0.000 description 5
- 238000004590 computer program Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 241000430521 Alyssum Species 0.000 description 1
- 241000864438 Parablechnum minus Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000012938 design process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/08—Key distribution or management, e.g. generation, sharing or updating, of cryptographic keys or passwords
- H04L9/0861—Generation of secret information including derivation or calculation of cryptographic keys or passwords
- H04L9/0869—Generation of secret information including derivation or calculation of cryptographic keys or passwords involving random numbers or seeds
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L2209/00—Additional information or applications relating to cryptographic mechanisms or cryptographic arrangements for secret or secure communication H04L9/00
- H04L2209/46—Secure multiparty computation, e.g. millionaire problem
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a method, a device, equipment and a storage medium for processing data, and relates to the fields of secure multi-party computing, cryptography and data processing. The specific implementation scheme is as follows: acquiring processing information aiming at target data; determining the target calling times of each parameter in the processing information; sending a data processing instruction to the at least two computing nodes according to the processing information so that the at least two computing nodes can process the target data; acquiring parameter values obtained in the data processing process and the actual calling times of the parameters from at least two calculation nodes in real time; and processing each parameter value according to the target calling times and the actual calling times. The realization mode can judge whether the parameter is still used according to the actual calling times and the target calling times of the parameter, and process the parameter value according to the judgment result, thereby effectively reducing the occupation of the memory on the premise of ensuring the correct calculation.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to the fields of secure multiparty computing, cryptography, and data processing, and in particular, to a method, an apparatus, a device, and a storage medium for processing data.
Background
Secure multiparty computing is actually a security protocol, as a sub-field of cryptography, which allows multiple data owners to perform collaborative computing without trust, output the computed results, and ensure that no one party can obtain any information other than the computed results. In other words, secure multiparty computing technology can capture data usage value without revealing the original data content.
The secure multiparty computing protocol requires considerable mathematical and cryptographic knowledge, such as encryption regimes, secret sharing, inadvertent transmission, etc., during the design process. A large number of parameters are generated in the calculation process, and if the parameters are stored, a large amount of memory is occupied.
Disclosure of Invention
A method, apparatus, device, and storage medium for processing data are provided.
According to a first aspect, there is provided a method for processing data, comprising: acquiring processing information aiming at target data; determining the target calling times of each parameter in the processing information; sending a data processing instruction to the at least two computing nodes according to the processing information so that the at least two computing nodes can process the target data; acquiring parameter values obtained in the data processing process and the actual calling times of the parameters from at least two calculation nodes in real time; and processing each parameter value according to the target calling times and the actual calling times.
According to a second aspect, there is provided an apparatus for processing data, comprising: an information acquisition unit configured to acquire processing information for target data; a number determination unit configured to determine a target number of calls for each parameter in the processing information; a processing instruction sending unit configured to send data processing instructions to the at least two computing nodes according to the processing information, so that the at least two computing nodes process the target data; the number obtaining unit is configured to obtain each parameter value obtained in the data processing process and the actual calling number of each parameter from at least two calculation nodes in real time; and the parameter processing unit is configured to process each parameter value according to the target calling times and the actual calling times.
According to a third aspect, there is provided an electronic device for processing data, comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described in the first aspect.
According to a fourth aspect, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method as described in the first aspect.
According to the technology of the application, whether the parameter can still be used or not can be judged according to the actual calling times and the target calling times of the parameter, and the parameter value is processed according to the judgment result, so that the occupation of the memory is effectively reduced on the premise of ensuring correct calculation.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:
FIG. 1 is an exemplary system architecture diagram in which one embodiment of the present application may be applied;
FIG. 2 is a flow diagram of one embodiment of a method for processing data according to the present application;
FIG. 3 is a schematic diagram of an application scenario of a method for processing data according to the present application;
FIG. 4 is a flow diagram of another embodiment of a method for processing data according to the present application;
FIG. 5 is a schematic block diagram illustrating one embodiment of an apparatus for processing data according to the present application;
fig. 6 is a block diagram of an electronic device for implementing a method for processing data according to an embodiment of the present application.
Detailed Description
The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
Fig. 1 shows an exemplary system architecture 100 to which embodiments of the present method for processing data or apparatus for processing data may be applied.
As shown in fig. 1, the system architecture 100 may include data providing nodes 101, 102, a management node 103, computing nodes 104, 105, and a privacy providing node 106. The nodes can communicate with each other through a network. The network may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The data providing nodes 101, 102 may store various data available for processing to the computing nodes 104, 105, so that the computing nodes 104, 105 can process the data. These data cannot be acquired by other devices. The data providing nodes 101 and 102 may output the data by encrypting the data.
The computing nodes 104, 105 may be various electronic devices that can be used to process data, and may receive a processing instruction sent by the management node 103 and perform various operations on data received from the data providing nodes 101, 102 according to the processing instruction.
The privacy providing node 106 is used for providing random numbers in the calculation process of the calculation nodes 104 and 105, so that the calculation nodes 104 and 105 can obtain the correct processing result of the original data on the basis that the original data cannot be obtained.
It should be noted that the data providing nodes 101 and 102, the management node 103, the computing nodes 104 and 105, and the privacy providing node 106 may be hardware or software. When the data providing nodes 101 and 102, the management node 103, the computing nodes 104 and 105, and the privacy providing node 106 are hardware, they may be implemented as a distributed cluster formed by a plurality of electronic devices, or may be implemented as a single electronic device. When the data providing nodes 101, 102, the management node 103, the computing nodes 104, 105 and the privacy providing node 106 are software, they may be implemented as a plurality of software or software modules (for example, to provide distributed services), or as a single software or software module. And is not particularly limited herein.
It should be noted that the method for processing data provided in the embodiment of the present application is generally performed by the management node 103. Accordingly, means for processing data are typically provided in the management node 103.
It should be understood that the number of data providing nodes, management nodes, computing nodes and privacy providing nodes in fig. 1 is merely illustrative. There may be any number of data providing nodes, management nodes, computing nodes, and privacy providing nodes, as desired for implementation.
With continued reference to FIG. 2, a flow 200 of one embodiment of a method for processing data in accordance with the present application is shown. The method for processing data of the embodiment comprises the following steps:
in step 201, processing information for target data is acquired.
In this embodiment, an execution subject (e.g., a management node shown in fig. 1) of the method for processing data may acquire processing information for target data. Here, the processing information may be description information of a processing flow of the target data, and the processing information may include code information, formula information, text, and the like. The technician can write the processing information through the terminal used by the technician, and the execution main body can be connected with the terminal used by the technician and acquire the processing information written by the technician. Here, the target data refers to data received by a computing node (e.g., computing nodes 104, 105 shown in fig. 1). According to the system architecture shown in fig. 1, the computing node may receive data provided by both the data providing node and the privacy providing node.
In the field of secure multiparty computing, since no electronic device other than a data providing node (e.g., the data providing nodes 101 and 102 shown in fig. 1) can obtain original data, the data obtained by the computing node may be data encrypted by the data providing node. For example, the original data includes two numbers, x is 3 and y is 5. The data received by the two computing nodes is different from the original data, and the data received by the computing node 1 may be x-1, and the data received by the computing node 2 may be x-2. Of course, to distinguish between values destined for different compute nodes, the two values may be labeled with subscripts. For example, the data sent to the compute node 1 is labeled x1The data sent to the compute node 2 is denoted x2. Similarly, y may be divided into two values, sent to two computing nodes, respectively, e.g., y1＝4，y21. If the calculation for x and y is z x y, the resulting z value is 15. The privacy providing node needs to provide some random numbers in the calculation process so that the final calculation results of the calculation node 1 and the calculation node 2 are equal to 15.
After the execution main body acquires the processing information, the execution main body can analyze the processing information and determine the target calling times of each parameter in the processing information. The analysis here may include analyzing the call relation of each function, and determining the target call times of each parameter according to the call relation. It can be understood that, since the target data includes data sent by the data provider to the computing node and data sent by the privacy providing node to the computing node, many parameters exist in the calculation process. The execution subject may count the number of calls for each parameter.
For example, the target number of calls for parameter x in the following code is 2. The 1 st call is in w.dot (x), and the 2 nd word is in dz.dot (x.transpose ()).
SharedEntity z＝w.dot(x).plus(b)；
SharedEntity a＝z.sigmoid()；
SharedEntity dz＝a.minus(y)；
SharedEntity dw＝dz.dot(x.transpose()).div(samples)；
SharedEntity db＝dz.sumH().div(samples)；
w＝w.minus(alpha.multiply(dw))；
b＝b.minus(alpha.multiply(db))。
After acquiring the processing information, the execution main body may send a data processing instruction to at least two computing nodes. Each computing node, upon receiving the data processing instruction, may process data received from the data providing node and the privacy providing node. The data processing instruction here may be an instruction obtained by analyzing the processing information by the execution subject. For example, the data processing instruction may be w.dot (x). The compute node, upon receiving the instruction, may compute for the w value and the x value. It will be appreciated that after computing a completed data processing instruction, the computing node may report to the executing agent that the computation has been completed, so that the executing agent sends a new data processing instruction to the computing node for the resulting computation.
And step 204, acquiring each parameter value obtained in the data processing process and the actual calling times of each parameter from at least two computing nodes in real time.
In this embodiment, the execution main body may further obtain, in real time, each parameter value obtained in the data processing process and the actual number of calls of each parameter from each computation node. Specifically, the execution agent may receive a report from the compute node and determine the data processing process. And determining the calculated parameter values and the actual calling times of the parameters according to the data processing process.
And step 205, processing each parameter value according to the target calling times and the actual calling times.
The execution main body can compare the obtained actual calling times of the parameters with the target calling times, and if the actual calling times and the target calling times are equal, the parameters are not called again. A part of the parameter values can be deleted, which can effectively save memory space. Correspondingly, if the actual calling times are less than the target calling times, the parameter is still called, and the parameter value needs to be stored to ensure the calculation accuracy.
With continued reference to fig. 3, a schematic diagram of one application scenario of a method for processing data according to the present application is shown. In the application scenario of fig. 3, the database 301 of hospital 1 stores medical record information of the diabetic patient, and the database 302 of hospital 2 also stores medical record information of the diabetic patient. Due to the existing privacy data protection scheme, the medical history information of the diabetes patients of hospital 1 and hospital 2 cannot be shared with other electronic devices. In secure multiparty computing, hospital 1 and hospital 2 may encrypt medical record information of a diabetic patient and send the encrypted medical record information to computing nodes 303 and 304, respectively, and the private data provider 305 may also send a random number to computing nodes 303 and 304 for processing. The server 306 can obtain processing information for the above-described diabetes patient medical record information. After the processing in steps 202 to 205, after the data processing is completed, the server 306 may delete the parameter values obtained in the calculation process.
The method for processing data provided by the embodiment of the application can judge whether the parameter is still used according to the actual calling times and the target calling times of the parameter, and process the parameter value according to the judgment result, thereby effectively reducing the occupation of the memory on the premise of ensuring correct calculation.
With continued reference to FIG. 4, a flow 400 of another embodiment of a method for processing data according to the present application is shown. As shown in fig. 4, the method for processing data of the present embodiment may include the steps of:
In this embodiment, the execution subject may send an instruction to the privacy providing node and the at least one data providing node to notify the privacy providing node and the at least one data providing node to send the target data to the at least two computing nodes. In some specific application scenarios, the privacy providing node, each data providing node, and each computing node need to register with the execution subject first after being started. After the registration is completed, the execution subject determines that each node is in a normal working state, and then may send an instruction to the privacy providing node and the at least one data providing node, so that the privacy providing node and the at least one data providing node send target data to the at least two computing nodes. The compute node, upon receiving the data, may obtain the compute logic from the execution body to perform computations on the received data.
At step 402, processing information for target data is obtained.
In this embodiment, the execution subject may determine the function call chain according to the processing information. Specifically, the execution subject may first analyze the processing information to determine a calling relationship of a function in the code. And connecting the calling relations of the functions in series to obtain a function calling chain.
And step 404, determining the target calling times of each parameter according to the function calling chain.
After the function call chain is obtained, the execution subject may count the number of occurrences of each parameter in the function call chain, and use the obtained number as the target call number.
And 406, acquiring parameter values obtained in the data processing process and the actual calling times of the parameters from at least two computing nodes in real time.
The principle of steps 405 to 406 is similar to that of steps 203 to 204, and is not described herein again.
In this embodiment, if the target number of calls of each parameter value is greater than the actual number of calls, it is described that the parameter value is still called. In this case, the execution body may acquire the calculated parameter values from the respective calculation nodes and store the respective parameter values. The simultaneous execution agent may also send a storage instruction to at least two compute nodes, i.e., notify each compute node to store each parameter value.
In this embodiment, if the target number of calls of each parameter value is equal to the actual number of calls, it indicates that the computation logic has been executed. At this point, the execution agent may only retain the parameter values at the top of the function call chain. Here, the parameter at the top of the function call chain may be at least one calculated value finally obtained. The execution agent may retain only these parameter values for subsequent calculations. And taking the rest parameter values as parameter values to be deleted, and deleting the previously stored parameter values to be deleted. Meanwhile, the execution main body can also send a deletion instruction to at least two computing nodes, namely, each computing node is informed to delete the parameter value to be deleted.
In this embodiment, after determining that each computing node completes data processing, the execution main body may return a data processing result to the data providing node. However, when there are a plurality of data providing nodes, the data processing result of the a data providing node cannot be transmitted to the B data providing node due to the private data protection scheme. Since the data sent by each data providing node to each computing node has an identifier, the execution main body can send an instruction to each data providing node according to the identifier of the data provided by each data providing node, so that the execution main body can acquire a data processing result corresponding to the identifier of the data from the computing node. Therefore, the original data can be guaranteed to be utilized, and the processing result of the original data can be guaranteed not to be revealed.
The method for processing data provided by the above embodiment of the present application may determine the target call times of each parameter according to the function call chain, and determine whether the parameter value is still used according to the target call times and the actual call times. After determining that it will not be used, only the parameter values at the top of the function call chain are saved for subsequent use. In addition, the data processing result can be returned to the data providing node providing the corresponding data.
With further reference to fig. 5, as an implementation of the methods shown in the above-mentioned figures, the present application provides an embodiment of an apparatus for processing data, which corresponds to the method embodiment shown in fig. 2, and which is particularly applicable to various electronic devices.
As shown in fig. 5, the apparatus 500 for processing data of the present embodiment includes: an information acquisition unit 501, a number-of-times determination unit 502, a processing instruction transmission unit 503, a number-of-times acquisition unit 504, and a parameter processing unit 505.
An information acquisition unit 501 configured to acquire processing information for target data.
A number determining unit 502 configured to determine the target number of calls for each parameter in the processing information.
A processing instruction sending unit 503 configured to send a data processing instruction to the at least two computing nodes according to the processing information, so that the at least two computing nodes process the target data.
The number obtaining unit 504 is configured to obtain, from at least two computing nodes, parameter values obtained in the data processing process and actual call numbers of the parameters in real time.
And a parameter processing unit 505 configured to process each parameter value according to the target call times and the actual call times.
In some optional implementations of this embodiment, the number determining unit 502 may be further configured to: determining a function call chain according to the processing information; and determining the target calling times of each parameter according to the function calling chain.
In some optional implementations of this embodiment, the parameter processing unit 505 may be further configured to: in response to determining that the target number of calls for each parameter value is greater than the actual number of calls, storing each parameter value and sending a store instruction to at least two compute nodes to store each parameter value.
In some optional implementations of this embodiment, the parameter processing unit 505 may be further configured to: in response to determining that the target calling times of the parameter values are equal to the actual calling times, determining the parameter values except the parameter value at the top of the function calling chain as the parameter values to be deleted; deleting the parameter values to be deleted and sending a deletion instruction to the at least two computing nodes to delete the parameter values to be deleted.
In some optional implementations of this embodiment, the apparatus 500 may further include a first instruction sending unit, not shown in fig. 5, configured to send an instruction to the privacy providing node and the at least one data providing node, so that the privacy providing node and the at least one data providing node send the target data to the at least two computing nodes.
In some optional implementations of this embodiment, the apparatus 500 may further include a second instruction sending unit, not shown in fig. 5, configured to send, according to the data provided by the at least one data providing node, an instruction to the at least one data providing node to cause the at least one data providing node to obtain the data processing result from the at least two computing nodes.
It should be understood that units 501 to 505, which are described in the apparatus 500 for processing data, correspond to the respective steps in the method described with reference to fig. 2, respectively. Thus, the operations and features described above for the method for processing data are equally applicable to the apparatus 500 and the units included therein and will not be described again here.
According to an embodiment of the present application, an electronic device and a readable storage medium are also provided.
As shown in fig. 6, is a block diagram of an electronic device executing a method for processing data according to an embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.
As shown in fig. 6, the electronic apparatus includes: one or more processors 601, memory 602, and interfaces for connecting the various components, including a high-speed interface and a low-speed interface. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input/output apparatus (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). In fig. 6, one processor 601 is taken as an example.
The memory 602 is a non-transitory computer readable storage medium as provided herein. Wherein the memory stores instructions executable by at least one processor to cause the at least one processor to perform the methods provided herein for processing data. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform the methods provided herein for processing data.
The memory 602, which is a non-transitory computer-readable storage medium, may be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions/modules corresponding to the method for processing data executed in the embodiment of the present application (for example, the information acquisition unit 501, the number determination unit 502, the processing instruction transmission unit 503, the number acquisition unit 504, and the parameter processing unit 505 shown in fig. 5). The processor 601 executes various functional applications of the server and data processing by executing non-transitory software programs, instructions and modules stored in the memory 602, that is, implements the method for processing data executed in the above method embodiment.
The memory 602 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of an electronic device that performs processing of data, and the like. Further, the memory 602 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 602 optionally includes memory located remotely from the processor 601, which may be connected via a network to an electronic device executing instructions for processing data. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device performing the method for processing data may further include: an input device 603 and an output device 604. The processor 601, the memory 602, the input device 603 and the output device 604 may be connected by a bus or other means, and fig. 6 illustrates the connection by a bus as an example.
The input device 603 may receive input numeric or character information and generate key signal inputs related to performing user settings and function control of an electronic apparatus for processing data, such as an input device like a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointing stick, one or more mouse buttons, a track ball, a joystick, etc. The output devices 604 may include a display device, auxiliary lighting devices (e.g., LEDs), and tactile feedback devices (e.g., vibrating motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device can be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
According to the technical scheme of the embodiment of the application, whether the parameter is still used or not can be judged according to the actual calling times and the target calling times of the parameter, and the parameter value is processed according to the judgment result, so that the occupation of the memory is effectively reduced on the premise of ensuring correct calculation.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and the present invention is not limited thereto as long as the desired results of the technical solutions disclosed in the present application can be achieved.
The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (14)
1. A method for processing data, comprising:
acquiring processing information aiming at target data;
determining the target calling times of each parameter in the processing information;
sending a data processing instruction to at least two computing nodes according to the processing information so that the at least two computing nodes can process the target data;
acquiring parameter values obtained in the data processing process and the actual calling times of the parameters from the at least two calculation nodes in real time;
and processing each parameter value according to the target calling times and the actual calling times.
2. The method of claim 1, wherein the determining the number of target calls for each parameter in the processing information comprises:
determining a function call chain according to the processing information;
and determining the target calling times of each parameter according to the function calling chain.
3. The method of claim 2, wherein the processing the parameter values according to the target number of calls and the actual number of calls comprises:
in response to determining that the target number of calls for each parameter value is greater than the actual number of calls, storing each parameter value and sending a store instruction to the at least two compute nodes to store each parameter value.
4. The method of claim 3, wherein the processing the parameter values according to the target number of calls and the actual number of calls comprises:
in response to determining that the target calling times of the parameter values are equal to the actual calling times, determining the parameter values except the parameter value at the top of the function calling chain as the parameter values to be deleted;
deleting the parameter value to be deleted and sending a deletion instruction to the at least two computing nodes to delete the parameter value to be deleted.
5. The method of claim 1, wherein the method further comprises:
sending instructions to a privacy providing node, at least one data providing node, to cause the privacy providing node, the at least one data providing node to send the target data to the at least two computing nodes.
6. The method of claim 5, wherein the method further comprises:
and sending instructions to the at least one data providing node according to the data provided by the at least one data providing node so as to enable the at least one data providing node to obtain data processing results from the at least two computing nodes.
7. An apparatus for processing data, comprising:
an information acquisition unit configured to acquire processing information for target data;
a number determination unit configured to determine a target number of calls for each parameter in the processing information;
a processing instruction sending unit configured to send data processing instructions to at least two computing nodes according to the processing information, so that the at least two computing nodes process the target data;
the number obtaining unit is configured to obtain each parameter value obtained in the data processing process and the actual calling number of each parameter from the at least two calculation nodes in real time;
and the parameter processing unit is configured to process each parameter value according to the target calling times and the actual calling times.
8. The apparatus of claim 7, wherein the number determination unit is further configured to:
determining a function call chain according to the processing information;
and determining the target calling times of each parameter according to the function calling chain.
9. The apparatus of claim 8, wherein the parameter processing unit is further configured to:
in response to determining that the target number of calls for each parameter value is greater than the actual number of calls, storing each parameter value and sending a store instruction to the at least two compute nodes to store each parameter value.
10. The apparatus of claim 9, wherein the parameter processing unit is further configured to:
in response to determining that the target calling times of the parameter values are equal to the actual calling times, determining the parameter values except the parameter value at the top of the function calling chain as the parameter values to be deleted;
deleting the parameter value to be deleted and sending a deletion instruction to the at least two computing nodes to delete the parameter value to be deleted.
11. The apparatus of claim 7, wherein the apparatus further comprises:
a first instruction sending unit configured to send an instruction to a privacy providing node and at least one data providing node to cause the privacy providing node and the at least one data providing node to send the target data to the at least two computing nodes.
12. The apparatus of claim 11, wherein the apparatus further comprises:
a second instruction sending unit configured to send an instruction to the at least one data providing node according to the data provided by the at least one data providing node, so that the at least one data providing node obtains data processing results from the at least two computing nodes.
13. An electronic device for processing data, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.
14. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010700636.2A CN113961944A (en) | 2020-07-20 | 2020-07-20 | Method, apparatus, device and storage medium for processing data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010700636.2A CN113961944A (en) | 2020-07-20 | 2020-07-20 | Method, apparatus, device and storage medium for processing data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113961944A true CN113961944A (en) | 2022-01-21 |
Family
ID=79459748
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010700636.2A Pending CN113961944A (en) | 2020-07-20 | 2020-07-20 | Method, apparatus, device and storage medium for processing data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113961944A (en) |
-
2020
- 2020-07-20 CN CN202010700636.2A patent/CN113961944A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111460429B (en) | Task processing method, device, equipment and medium based on trusted execution environment | |
| CN112016110B (en) | Method, device, equipment and storage medium for storing data | |
| JP2022054386A (en) | Model joint training method, apparatus, electronic device, storage medium, and computer program | |
| US9250895B2 (en) | Establishing subsystem boundaries based on call flow graph topology | |
| CN113722755B (en) | Data processing system, method, device and equipment for realizing privacy protection | |
| CN112416632B (en) | Event communication method and device, electronic equipment and computer readable medium | |
| EP4002752A1 (en) | Protecting cloud application secret key with multi-party computation algorithm | |
| CN112615852A (en) | Data processing method, related device and computer program product | |
| CN114422237B (en) | Data transmission method and device, electronic equipment and medium | |
| CN113238943A (en) | Method and device for continuous integration test | |
| US9800473B2 (en) | Network based virtual collaborative problem solving space | |
| CN115801220A (en) | Acceleration apparatus, computing system, and acceleration method | |
| CN110059064B (en) | Log file processing method and device and computer readable storage medium | |
| CN112565225B (en) | Method and device for data transmission, electronic equipment and readable storage medium | |
| US20170017602A1 (en) | Storage system cabling analysis | |
| CN112527416A (en) | Task processing method and device, computer equipment and storage medium | |
| CN111767149A (en) | Scheduling method, device, equipment and storage equipment | |
| CN113961944A (en) | Method, apparatus, device and storage medium for processing data | |
| US10754644B2 (en) | Annotations in software development | |
| CN112735601B (en) | Test method, device and equipment for determining infectious disease close-contact population by using Bluetooth | |
| CN112000491B (en) | Application program interface calling method, device, equipment and storage medium | |
| CN111818046B (en) | Method, apparatus, device and storage medium for interaction information | |
| CN111522771B (en) | Fundus image processing method, terminal device and storage medium | |
| CN114745179A (en) | Multiparty privacy intersection method, device, equipment and storage medium | |
| CN115526953A (en) | Method and device for generating view, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |