CN111177765A - Financial big data processing method, storage medium and system - Google Patents
Financial big data processing method, storage medium and system Download PDFInfo
- Publication number
- CN111177765A CN111177765A CN202010009956.3A CN202010009956A CN111177765A CN 111177765 A CN111177765 A CN 111177765A CN 202010009956 A CN202010009956 A CN 202010009956A CN 111177765 A CN111177765 A CN 111177765A
- Authority
- CN
- China
- Prior art keywords
- data
- big data
- data processing
- type
- processing node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2452—Query translation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/52—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems during program execution, e.g. stack integrity ; Preventing unwanted data erasure; Buffer overflow
- G06F21/53—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems during program execution, e.g. stack integrity ; Preventing unwanted data erasure; Buffer overflow by executing in a restricted environment, e.g. sandbox or secure virtual machine
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/55—Detecting local intrusion or implementing counter-measures
- G06F21/56—Computer malware detection or handling, e.g. anti-virus arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2221/00—Indexing scheme relating to security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F2221/03—Indexing scheme relating to G06F21/50, monitoring users, programs or devices to maintain the integrity of platforms
- G06F2221/033—Test or assess software
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Bioethics (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Virology (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Storage Device Security (AREA)
Abstract
The invention provides a financial big data processing method, which comprises the following steps: the big data processing node receives data sent by a user through a terminal through a link; the big data processing node analyzes the type of the sent data; if the type is the query type, the big data processing node supports data query and generates a result based on the protected big data; if the type is a program package type, the big data processing node analyzes the malicious degree of the code of the big data processing node, and accordingly determines a response and generates a result or rejects the request of the user; and the big data processing node sends the generated result back to the user and the terminal through the link based on the query type or the non-malicious program package request. In addition, the method, the computer readable storage medium and the system reduce the risk of infection and leakage of the big data storage caused by malicious codes or programs and avoid leakage of private information in the big data caused by the inquiry request.
Description
Technical Field
The present invention relates generally to the field of electrical signal processing, and more particularly, to a financial big data processing method, storage medium, and system.
Background
With the rapid development of information network technology, big data has been incorporated into various aspects of user work, life, such as big data analysis based on market behavior at work, business analysis such as payment and financial transactions at life, and analysis of medical data, and so on. Analysis of big data provides more accurate and efficient decision reference for human beings. Big data is promoting the revolution of society, changing the behavior and habits of the user in work and life. The method proposes a strategy for implementing national big data for our country, and takes the big data as basic strategic resources.
However, the wide application of big data brings great convenience to people's work and life, and also brings great threat to the security of network systems and personal information. For example, a user's private information may be known or inferred for a particular behavioral analysis, which may lead to information leakage. In addition, under a big data environment, due to the extremely rapid increase of the data volume, the safety management of the big data storage can follow up in time, so that a malicious user can adopt a query request such as malicious codes or programs or after disguise to carry out massive query and multi-dimensional prediction on information in the big data storage, and not only is the great risk that the big data storage is infected and leaked due to the malicious codes or programs, but also the leakage of private information in the big data due to the query request exists. There is a strong need in the art for a targeted technique to effectively solve the technical problem.
Disclosure of Invention
One of the objects of the present invention is to provide a financial big data processing method, storage medium and system. By the financial big data processing method, risks of infection and leakage of big data storage caused by malicious codes or programs can be reduced, and leakage of private information in big data caused by query requests is avoided.
The technical scheme adopted by the invention to solve the technical problems is as follows: a financial big data processing method comprises the following steps: in step S1, the big data processing node receives data sent by the user through the terminal via the link; in step S2, the big data processing node analyzes the type of the transmitted data; in step S3, if the type is a query type, the big data processing node supports data query and generates a result based on the protected big data; in step S4, if the type is a package type, the big data processing node analyzes the malicious degree of its code, and accordingly determines a response and generates a result or rejects the request of the user; and in step S5, the big data processing node sends the generated result back to the user and the terminal via the link based on the query type or the non-malware package request.
According to one embodiment, in steps S1 and S5, the link is a wireless link. Wherein the wireless link comprises a link formed from at least one of Wi-Fi, Bluetooth (BT), Near Field Communication (NFC), Global Positioning System (GPS), and cellular communication; the cellular communication includes LTE, LTE-A, CDMA, WCDMA, UMTS, WiBro, GSM.
According to one embodiment, wherein in step S2, the big data processing node analyzing the type of the sent data further comprises: s21, the big data processing node confirms the format of the data, if the data is a compressed text, the decompression module is called to decompress the data, if the data is an uncompressed text, the step S23 is directly entered; s22, if the decompressed data is encrypted data, the data is decrypted by using the private key in the asymmetric encryption to obtain the decompressed data, and the process proceeds to step S23.
According to one embodiment, in S23, the big data processing node performs type judgment on the data in the preprocessing module; wherein if the data is not automatically run in the pre-processing module and no installation and local data acquisition is performed, and if a first line comprising a plurality of lines of data is determined to be a starting line, zero or more fields following the starting line, each field containing a name and a value separated by a colon and the header ending with an empty line, and the empty line followed by binary data or text, determining that the type is a text-type query; and if the data automatically runs in the preprocessing module and is installed and local data is acquired, determining that the data is of the package type.
According to one embodiment, in S23, the big data processing node performs type judgment on the data in the preprocessing module; wherein if the data is not automatically run in the pre-processing module and no installation and local data acquisition is performed, and if a first line comprising a plurality of lines of data is determined to be a starting line, zero or more fields following the starting line, each field containing a name and a value separated by a colon and the header ending with an empty line, and the empty line followed by binary data or text, determining that the type is a text-type query; if the data automatically runs in the preprocessing module and is installed and locally acquired, the preprocessing module in the big data processing node firstly uses data (package = packages) operation to display the data set in the package, and then determines that the type of the data set is the package based on the result.
According to one embodiment, in step S3, if the type is a query type, the big data processing node supports data query and generates a result based on the protected big data; before the data to be queried is stored in the big data node, the following processing needs to be carried out: taking the personal identification of the owner of the data as first information, and encrypting the data record of the owner of the data in an autonomous encryption mode; transmitting the first information and the encrypted data record to a storage module and updating periodically; extracting analyzed data with the type of query, and finding out the storage address of the data expected to be queried in the analyzed data by using an encryption retrieval algorithm through a storage module; the storage module requests the calling module to acquire target data so as to acquire data to be inquired; the calling module generates symmetrically encrypted target data to the storage module by using a secure transmission protocol, so that only the user and the terminal can understand the queried big data result in the process of sending the data back to the user and the terminal through the link, and other users and terminals cannot analyze the queried big data result, wherein in order to reduce the potential safety hazard caused by intercepting and analyzing the big data result to the storage module, the queried big data result does not contain identification information of the storage module.
According to one embodiment, in step S4, if the type is a package type, the big data processing node analyzes the degree of maliciousness of its code, determines a response accordingly and generates a result or rejects the user' S request. The process of analyzing the malicious degree is as follows: step S41, the receiving module of the big data processing node receives the data of the program package type, and distributes the data to the program running module based on the running state of the big data processing node, and the program running module utilizes the simulator to realize the function of simulating the running program package, and further collects the characteristic vector of the running program package; the program package analysis module processes the characteristic vectors which are simulated and collected by the program operation module, classifies data based on a program package information base and a machine learning technology, and further detects the data of a data package with a code having a malicious intention; the program package information base stores program package information which is analyzed by a big data processing node once and program packages which are captured from a networked program package website or a program package sharing platform based on a program package capturing module, so that the capability of fast analysis is provided for the program package analyzing module, and after malicious programs of the program packages are determined and malicious degree identification is carried out, the determined program packages are used as training data in the program package analyzing module; step S42, when the program package is determined to be a non-malicious program package, extracting the query request of the program package, determining a response based on the query request and querying to generate a result; in step S43, when the package is determined to be a malicious package or the level of malicious intent cannot be determined, the user' S request is rejected.
According to one embodiment, in step S41, the machine learning of the package includes the steps of: obtaining a configuration file of the program package through decompiling, and converting the configuration file into a text format which can be executed by a machine; restoring the binary file of the source code through the open source software; scanning a source code by using a matching algorithm, counting keywords and the frequency and the position of the keywords in a configuration file, and obtaining the similarity distance between each pair of keywords by using a distance formula; classifying the keywords based on the similarity distance, searching by taking each keyword as a target, counting the keywords with the similarity distance smaller than a first threshold value, comparing the counted keywords with the positions of the keywords, and removing the keywords in different classifications; and further converting the keywords with the similarity distance smaller than the first threshold into the same category and recording the same as a single feature.
According to one embodiment, in step S41, the determination of the malicious package includes the steps of: step S41', analyzing the bytecode instruction by analyzing the program package, and extracting the sensitive function and function call relation of the program package based on the function record in the storage module in the big data storage node; the extraction comprises traversing functions in the program package to search for sensitive functions, and determining whether to call the sensitive functions or not based on function names and parameters in the function call relation and tracking operands in the sensitive functions; wherein tracing comprises comparing an execution operation path of the package with path records in storage modules in the big data storage node to determine a degree of maliciousness of the package; when the same parameters exist, determining the type of the data to be malicious or non-malicious based on the record of the storage module, otherwise, determining the data to be uncertain; step S42', when the package is determined to be a non-malicious package, extracting a query request of the package, determining a response based on the query request, and performing a query to generate a result; step S43 ', when the package is determined to be a malicious package or the level of maliciousness cannot be determined, the user' S request is rejected.
According to one embodiment, in step S41, the determination of the malicious package includes the steps of: step S41 '', the preprocessing module of the big data node adds monitoring codes to the program package to monitor the calling behavior of the program package during installation and operation; loading the program package added with the monitoring code into a sandbox, monitoring the operation of the program package added with the monitoring code in the sandbox by a monitoring module of the big data node, allocating a corresponding thread and recording the behavior and the operation path in the sandbox once the operation behavior occurs until no new operation and event occurs in the sandbox; analyzing the calling operation in the running process; wherein adding the monitoring code comprises: decompiling the program package, traversing the codes, if finding the calling parameters in the configuration file, adding the monitoring codes and repacking the monitoring codes, so that after the monitoring codes are called during the running of the program package, the log records the related log data, and the log data is extracted to obtain the calling information of the program package; determining an in-sandbox execution parameter of the program package based on the analyzed execution path, time and sandbox state of the program package, and further comparing the parameter with a corresponding parameter in a storage module in the big data storage node to determine the malicious degree of the program package; when the same parameters exist, determining the type of the data to be malicious or non-malicious based on the record of the storage module, otherwise, determining the data to be uncertain; step S42 '', when the program package is determined to be a non-malicious program package, extracting the query request of the program package, determining a response based on the query request and performing a query to generate a result; at step S43 ″, when the package is determined to be a malicious package or the level of malicious intent cannot be determined, the user' S request is denied.
According to one embodiment, in step S5, the big data processing node sends the generated result back to the user and the terminal via the link based on the query type or the non-malware package request; and after the user and the terminal receive the generated result, decrypting and combing and summarizing the received result data to obtain a plaintext data set.
According to another aspect of the invention, a computer-readable storage medium is also disclosed, comprising computer-executable instructions stored thereon, which when executed, implement the financial big data processing method as described above.
According to another aspect of the invention, a system for implementing the financial big data processing method is also disclosed.
Drawings
Embodiments of the invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which:
fig. 1 illustrates a schematic flow chart of a financial big data processing method according to an exemplary embodiment of the present invention.
Detailed Description
In the following description, reference is made to the accompanying drawings that show, by way of illustration, several specific embodiments. It will be understood that: other embodiments are contemplated and may be made without departing from the scope or spirit of the present disclosure. The following detailed description is, therefore, not to be taken in a limiting sense.
Fig. 1 illustrates a schematic flow chart of a financial big data processing method according to an exemplary embodiment of the present invention.
In step S1, the big data processing node receives data sent by the user through the terminal via the link;
in step S2, the big data processing node analyzes the type of the transmitted data;
in step S3, if the type is a query type, the big data processing node supports data query and generates a result based on the protected big data;
in step S4, if the type is a package type, the big data processing node analyzes the malicious degree of its code, and accordingly determines a response and generates a result or rejects the request of the user; and
in step S5, the big data processing node sends the generated result back to the user and the terminal via the link based on the query type or the non-malware package request.
Preferably, in steps S1 and S5, the link is a wireless link, for example, a link formed by at least one of Wi-Fi, Bluetooth (BT), Near Field Communication (NFC), Global Positioning System (GPS), and cellular communication (e.g., LTE-a, CDMA, WCDMA, UMTS, WiBro, GSM, etc.).
Preferably, in step S2, the big data processing node analyzing the type of the sent data further includes: s21, the big data processing node confirms the format of the data, if the data is a compressed text, the decompression module is called to decompress the data, if the data is an uncompressed text, the step S23 is directly entered; s22, if the decompressed data is encrypted data, the data is decrypted at the big data processing node by using the private key in the asymmetric encryption, thereby obtaining the decompressed data, and the step S23 is entered; s23, the big data processing node judges the type of the data in the preprocessing module; wherein if the data is not automatically run in the pre-processing module and no installation and local data acquisition is performed, and if a first line comprising a plurality of lines of data is determined to be a starting line, zero or more fields following the starting line, each field containing a name and a value separated by a colon and the header ending with an empty line, and the empty line followed by binary data or text, determining that the type is a text-type query; and if the data automatically runs in the preprocessing module and is installed and local data is acquired, determining that the data is of the package type.
Alternatively, in step S23, after determining that the data is automatically run in the preprocessing module and the installation and local data acquisition are performed, the preprocessing module in the big data processing node first displays the data set in the package using a data (package = packages) operation, and then determines that the type is the package based on the result.
Preferably, in step S3, if the type is a query type, the big data processing node supports data query and generates a result based on the protected big data; before the data to be queried is stored in the big data node, the following processing needs to be carried out: taking the personal identification of the owner of the data as first information, and encrypting the data record of the owner of the data in an autonomous encryption mode; the first information and the encrypted data record are transmitted to the storage module and periodically updated.
Preferably, the step S3 further includes: extracting analyzed data with the type of query, and finding out the storage address of the data expected to be queried in the analyzed data by using an encryption retrieval algorithm through a storage module; the storage module requests the calling module to acquire target data so as to acquire data to be inquired; the calling module generates symmetrically encrypted target data to the storage module by using a secure transmission protocol, so that only the user and the terminal can understand the queried big data result in the process of sending the data back to the user and the terminal through the link, and other users and terminals cannot analyze the queried big data result, wherein in order to reduce the potential safety hazard caused by intercepting and analyzing the big data result to the storage module, the queried big data result does not contain identification information of the storage module.
Preferably, in step S4, if the type is a package type, the big data processing node analyzes the degree of maliciousness of its code, determines a response accordingly and generates a result or rejects the user' S request. The process of analyzing the malicious degree is as follows:
step S41, the receiving module of the big data processing node receives the data of the program package type, and distributes the data to the program running module based on the running state of the big data processing node, and the program running module utilizes the simulator to realize the function of simulating the running program package, and further collects the characteristic vector of the running program package; the program package analysis module processes the characteristic vectors which are simulated and collected by the program operation module, classifies data based on a program package information base and a machine learning technology, and further detects the data of a data package with a code having a malicious intention; the program package information base stores program package information which is analyzed by a big data processing node once and program packages which are captured from a networked program package website or a program package sharing platform based on a program package capturing module, so that the capability of fast analysis is provided for the program package analyzing module, and after malicious programs of the program packages are determined and malicious degree identification is carried out, the determined program packages are used as training data in the program package analyzing module;
step S42, when the program package is determined to be a non-malicious program package, extracting the query request of the program package, determining a response based on the query request and querying to generate a result;
in step S43, when the package is determined to be a malicious package or the level of malicious intent cannot be determined, the user' S request is rejected.
Preferably, in step S41, the machine learning of the package includes the steps of: obtaining a configuration file of the program package through decompiling, and converting the configuration file into a text format which can be executed by a machine; restoring the binary file of the source code through the open source software; scanning a source code by using a matching algorithm, counting keywords and the frequency and the position of the keywords in a configuration file, and obtaining the similarity distance between each pair of keywords by using a distance formula; classifying the keywords based on the similarity distance, searching by taking each keyword as a target, counting the keywords with the similarity distance smaller than a first threshold value, comparing the counted keywords with the positions of the keywords, and removing the keywords in different classifications; and further converting the keywords with the similarity distance smaller than the first threshold into the same category and recording the same as a single feature.
Preferably, in step S41, the determination of the malicious program package includes the steps of:
step S41', analyzing the bytecode instruction by analyzing the program package, and extracting the sensitive function and function call relation of the program package based on the function record in the storage module in the big data storage node; the extraction comprises traversing functions in the program package to search for sensitive functions, and determining whether to call the sensitive functions or not based on function names and parameters in the function call relation and tracking operands in the sensitive functions; wherein tracing comprises comparing an execution operation path of the package with path records in storage modules in the big data storage node to determine a degree of maliciousness of the package; when the same parameters exist, determining the type of the data to be malicious or non-malicious based on the record of the storage module, otherwise, determining the data to be uncertain;
step S42', when the package is determined to be a non-malicious package, extracting a query request of the package, determining a response based on the query request, and performing a query to generate a result;
step S43 ', when the package is determined to be a malicious package or the level of maliciousness cannot be determined, the user' S request is rejected.
Alternatively, in step S41, the determination of the malicious package includes the steps of:
step S41 '', the preprocessing module of the big data node adds monitoring codes to the program package to monitor the calling behavior of the program package during installation and operation; loading the program package added with the monitoring code into a sandbox, monitoring the operation of the program package added with the monitoring code in the sandbox by a monitoring module of the big data node, allocating a corresponding thread and recording the behavior and the operation path in the sandbox once the operation behavior occurs until no new operation and event occurs in the sandbox; analyzing the calling operation in the running process; wherein adding the monitoring code comprises: decompiling the program package, traversing the codes, if finding the calling parameters in the configuration file, adding the monitoring codes and repacking the monitoring codes, so that after the monitoring codes are called during the running of the program package, the log records the related log data, and the log data is extracted to obtain the calling information of the program package; determining an in-sandbox execution parameter of the program package based on the analyzed execution path, time and sandbox state of the program package, and further comparing the parameter with a corresponding parameter in a storage module in the big data storage node to determine the malicious degree of the program package; when the same parameters exist, determining the type of the data to be malicious or non-malicious based on the record of the storage module, otherwise, determining the data to be uncertain;
step S42 '', when the program package is determined to be a non-malicious program package, extracting the query request of the program package, determining a response based on the query request and performing a query to generate a result;
at step S43 ″, when the package is determined to be a malicious package or the level of malicious intent cannot be determined, the user' S request is denied.
Preferably, in step S5, the big data processing node sends the generated result back to the user and the terminal via the link based on the query type or the non-malicious package request; and after the user and the terminal receive the generated result, decrypting and combing and summarizing the received result data to obtain a plaintext data set.
According to an embodiment of the present invention, there is also disclosed a storage medium including computer-executable instructions stored thereon which, when executed, implement the financial big data processing method as described above.
According to the embodiment of the invention, the financial big data processing system is also disclosed, which is used for realizing the financial big data processing method.
In summary, in the technical solution of the present invention, a financial big data processing method, a storage medium and a system are adopted. By the method, the storage medium and the system, risks of infection and leakage of the large data storage caused by malicious codes or programs can be reduced, and leakage of private information in the large data caused by the query request is avoided.
It will be understood that: the examples and embodiments of the invention may be implemented in hardware, software, or a combination of hardware and software. As described above, any body performing such operations may be stored, either in volatile or non-volatile storage, e.g., in a storage device such as a ROM, whether erasable or rewritable or not, or in memory such as, for example, RAM, memory chips, devices or integrated circuits, or on an optically or magnetically readable medium such as, for example, a CD, DVD, magnetic disk or magnetic tape. It will be understood that: storage devices and storage media are examples of machine-readable storage suitable for storing one or more programs that, when executed, implement examples of the present invention. Examples of the present invention may be conveyed electronically via any medium, such as a communications signal carried by a wired or wireless coupling, and the examples contain the same where appropriate.
It should be noted that: since the present invention solves the technical problems of reducing the risk of infection and leakage of big data storage caused by malicious code or program and avoiding the leakage of private information in big data caused by query request, adopts the technical means understood by those skilled in the computer technical field according to the teaching after reading the present specification, and obtains the beneficial technical effects of reducing the risk of infection and leakage of big data storage caused by malicious code or program and avoiding the leakage of private information in big data caused by query request, the solution claimed in the appended claims belongs to the technical solution in the meaning of patent law. Furthermore, the solution claimed in the appended claims has utility since it can be manufactured or used in industry.
The above description is only a preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Unless expressly stated otherwise, each feature disclosed is one example only of a generic series of equivalent or similar features. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A financial big data processing method is characterized by comprising the following steps:
in step S1, the big data processing node receives data sent by the user through the terminal via the link;
in step S2, the big data processing node analyzes the type of the transmitted data;
in step S3, if the type is a query type, the big data processing node supports data query and generates a result based on the protected big data;
in step S4, if the type is a package type, the big data processing node analyzes the malicious degree of its code, and accordingly determines a response and generates a result or rejects the request of the user; and
in step S5, the big data processing node sends the generated result back to the user and the terminal via the link based on the query type or the non-malware package request.
2. The financial big data processing method according to claim 1, wherein:
in steps S1 and S5, the link is a wireless link.
3. The financial big data processing method according to claim 2, wherein:
the wireless link comprises a link formed by at least one of Wi-Fi, Bluetooth (BT), Near Field Communication (NFC), Global Positioning System (GPS), and cellular communication; the cellular communication includes LTE, LTE-A, CDMA, WCDMA, UMTS, WiBro, GSM.
4. The financial big data processing method of claim 3, wherein the big data processing node analyzing the type of the transmitted data at step S2 further comprises:
s21, the big data processing node confirms the format of the data, if the data is a compressed text, the decompression module is called to decompress the data, if the data is an uncompressed text, the step S23 is directly entered; s22, if the decompressed data is encrypted data, the data is decrypted by using the private key in the asymmetric encryption to obtain the decompressed data, and the process proceeds to step S23.
5. The financial big data processing method according to claim 4, wherein the step S2 further comprises:
s23, the big data processing node judges the type of the data in the preprocessing module; wherein if the data is not automatically run in the pre-processing module and no installation and local data acquisition is performed, and if a first line comprising a plurality of lines of data is determined to be a starting line, zero or more fields following the starting line, each field containing a name and a value separated by a colon and the header ending with an empty line, and the empty line followed by binary data or text, determining that the type is a text-type query; and if the data automatically runs in the preprocessing module and is installed and local data is acquired, determining that the data is of the package type.
6. The financial big data processing method according to claim 4, wherein the step S2 further comprises:
s23, the big data processing node judges the type of the data in the preprocessing module; wherein if the data is not automatically run in the pre-processing module and no installation and local data acquisition is performed, and if a first line comprising a plurality of lines of data is determined to be a starting line, zero or more fields following the starting line, each field containing a name and a value separated by a colon and the header ending with an empty line, and the empty line followed by binary data or text, determining that the type is a text-type query; if the data automatically runs in the preprocessing module and is installed and locally acquired, the preprocessing module in the big data processing node firstly uses data (package = packages) operation to display the data set in the package, and then determines that the type of the data set is the package based on the result.
7. The financial big data processing method according to claim 5 or 6, wherein
In step S3, if the type is a query type, the big data processing node supports data query and generates a result based on the protected big data; before the data to be queried is stored in the big data node, the following processing needs to be carried out: taking the personal identification of the owner of the data as first information, and encrypting the data record of the owner of the data in an autonomous encryption mode; transmitting the first information and the encrypted data record to a storage module and updating periodically; extracting analyzed data with the type of query, and finding out the storage address of the data expected to be queried in the analyzed data by using an encryption retrieval algorithm through a storage module; the storage module requests the calling module to acquire target data so as to acquire data to be inquired; the calling module generates symmetrically encrypted target data to the storage module by using a secure transmission protocol, so that only the user and the terminal can understand the queried big data result in the process of sending the data back to the user and the terminal through the link, and other users and terminals cannot analyze the queried big data result, wherein in order to reduce the potential safety hazard caused by intercepting and analyzing the big data result to the storage module, the queried big data result does not contain identification information of the storage module.
8. The financial big data processing method according to claim 7, wherein:
in step S5, the big data processing node sends the generated result back to the user and the terminal via the link based on the query type or the non-malware package request; and after the user and the terminal receive the generated result, decrypting and combing and summarizing the received result data to obtain a plaintext data set.
9. A computer-readable storage medium comprising computer-executable instructions stored thereon that, when executed, implement the financial big data processing method of claim 8.
10. A system for implementing the financial big data processing method according to claim 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010009956.3A CN111177765A (en) | 2020-01-06 | 2020-01-06 | Financial big data processing method, storage medium and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010009956.3A CN111177765A (en) | 2020-01-06 | 2020-01-06 | Financial big data processing method, storage medium and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111177765A true CN111177765A (en) | 2020-05-19 |
Family
ID=70656182
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010009956.3A Withdrawn CN111177765A (en) | 2020-01-06 | 2020-01-06 | Financial big data processing method, storage medium and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111177765A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111882416A (en) * | 2020-07-24 | 2020-11-03 | 未鲲(上海)科技服务有限公司 | Training method and related device of risk prediction model |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1506895A (en) * | 2002-12-12 | 2004-06-23 | 国际商业机器公司 | System and method for providing polling routine execution for representation prediction service |
| CN102394885A (en) * | 2011-11-09 | 2012-03-28 | 中国人民解放军信息工程大学 | Information classification protection automatic verification method based on data stream |
| CN103473381A (en) * | 2013-10-13 | 2013-12-25 | 陈志德 | Database security assessment method |
| CN105678188A (en) * | 2016-01-07 | 2016-06-15 | 杨龙频 | Anti-leakage protocol identification method and device for database |
| CN105989179A (en) * | 2015-03-06 | 2016-10-05 | 北京邮电大学 | Financial data processing method and system |
| CN106227899A (en) * | 2016-08-31 | 2016-12-14 | 北京京航计算通讯研究所 | The storage of the big data of a kind of internet of things oriented and querying method |
| CN107085684A (en) * | 2016-02-16 | 2017-08-22 | 腾讯科技（深圳）有限公司 | The detection method and device of performance of program |
| CN107404473A (en) * | 2017-06-06 | 2017-11-28 | 西安电子科技大学 | Based on Mshield machine learning multi-mode Web application means of defences |
| CN107547525A (en) * | 2017-08-14 | 2018-01-05 | 复旦大学 | A kind of method for secret protection of big data query processing |
| CN109194646A (en) * | 2018-08-30 | 2019-01-11 | 东北大学 | A kind of safety certification data access method based on block chain |
| CN109271371A (en) * | 2018-08-21 | 2019-01-25 | 广东工业大学 | A kind of Distributed-tier big data analysis processing model based on Spark |
| US20190034476A1 (en) * | 2015-12-16 | 2019-01-31 | American Express Travel Related Services Company, Inc. | Converting a language type of a query |
| CN109586282A (en) * | 2018-11-29 | 2019-04-05 | 安徽继远软件有限公司 | A kind of unknown threat detection system of power grid and method |
| CN109981606A (en) * | 2019-03-07 | 2019-07-05 | 北京华安普特网络科技有限公司 | The hardware firewall detection device of universal serial bus |
| CN110223047A (en) * | 2019-06-17 | 2019-09-10 | 谢昱 | One kind being used for ancient books printed matter that indicates the type form or partial content of books and periodicals searching and managing platform and method |
| CN110532804A (en) * | 2019-09-05 | 2019-12-03 | 广州知弘科技有限公司 | A kind of secure storage control method based on big data |
-
2020
- 2020-01-06 CN CN202010009956.3A patent/CN111177765A/en not_active Withdrawn
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1506895A (en) * | 2002-12-12 | 2004-06-23 | 国际商业机器公司 | System and method for providing polling routine execution for representation prediction service |
| CN102394885A (en) * | 2011-11-09 | 2012-03-28 | 中国人民解放军信息工程大学 | Information classification protection automatic verification method based on data stream |
| CN103473381A (en) * | 2013-10-13 | 2013-12-25 | 陈志德 | Database security assessment method |
| CN105989179A (en) * | 2015-03-06 | 2016-10-05 | 北京邮电大学 | Financial data processing method and system |
| US20190034476A1 (en) * | 2015-12-16 | 2019-01-31 | American Express Travel Related Services Company, Inc. | Converting a language type of a query |
| CN105678188A (en) * | 2016-01-07 | 2016-06-15 | 杨龙频 | Anti-leakage protocol identification method and device for database |
| CN107085684A (en) * | 2016-02-16 | 2017-08-22 | 腾讯科技（深圳）有限公司 | The detection method and device of performance of program |
| CN106227899A (en) * | 2016-08-31 | 2016-12-14 | 北京京航计算通讯研究所 | The storage of the big data of a kind of internet of things oriented and querying method |
| CN107404473A (en) * | 2017-06-06 | 2017-11-28 | 西安电子科技大学 | Based on Mshield machine learning multi-mode Web application means of defences |
| CN107547525A (en) * | 2017-08-14 | 2018-01-05 | 复旦大学 | A kind of method for secret protection of big data query processing |
| CN109271371A (en) * | 2018-08-21 | 2019-01-25 | 广东工业大学 | A kind of Distributed-tier big data analysis processing model based on Spark |
| CN109194646A (en) * | 2018-08-30 | 2019-01-11 | 东北大学 | A kind of safety certification data access method based on block chain |
| CN109586282A (en) * | 2018-11-29 | 2019-04-05 | 安徽继远软件有限公司 | A kind of unknown threat detection system of power grid and method |
| CN109981606A (en) * | 2019-03-07 | 2019-07-05 | 北京华安普特网络科技有限公司 | The hardware firewall detection device of universal serial bus |
| CN110223047A (en) * | 2019-06-17 | 2019-09-10 | 谢昱 | One kind being used for ancient books printed matter that indicates the type form or partial content of books and periodicals searching and managing platform and method |
| CN110532804A (en) * | 2019-09-05 | 2019-12-03 | 广州知弘科技有限公司 | A kind of secure storage control method based on big data |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111882416A (en) * | 2020-07-24 | 2020-11-03 | 未鲲(上海)科技服务有限公司 | Training method and related device of risk prediction model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109753800B (en) | Android malicious application detection method and system fusing frequent item set and random forest algorithm | |
| ES2808954T3 (en) | Procedure and device for use in risk management of application information | |
| CN105956474B (en) | Android platform software unusual checking system | |
| US20130247187A1 (en) | Computing device to detect malware | |
| CN104246788A (en) | Detection and prevention of installation of malicious mobile applications | |
| CN110417718B (en) | Method, device, equipment and storage medium for processing risk data in website | |
| US20140195793A1 (en) | Remotely Establishing Device Platform Integrity | |
| CN113872965B (en) | SQL injection detection method based on Snort engine | |
| CN110276195A (en) | A kind of smart machine intrusion detection method, equipment and storage medium | |
| Sun et al. | Malware detection on Android smartphones using keywords vector and SVM | |
| CN116956080A (en) | Data processing method, device and storage medium | |
| CN108833500B (en) | Service calling method, service providing method, data transmission method and server | |
| CN111177765A (en) | Financial big data processing method, storage medium and system | |
| CN109871685B (en) | RTF file analysis method and device | |
| US20120089849A1 (en) | Cookie management system and method | |
| CN115600201A (en) | User account information safety processing method for power grid system software | |
| CN103095714A (en) | Trojan horse detection method based on Trojan horse virus type classification modeling | |
| Hindarto et al. | Android-manifest extraction and labeling method for malware compilation and dataset creation. | |
| CN116049822A (en) | Application program supervision method, system, electronic device and storage medium | |
| CN115544558A (en) | Sensitive information detection method and device, computer equipment and storage medium | |
| CN111125701B (en) | File detection method, equipment, storage medium and device | |
| CN111221803B (en) | Feature library management method and coprocessor | |
| Kumari et al. | Malware and piracy detection in android applications | |
| KR102249758B1 (en) | Artificial intelligence personal privacy data security system applying case based reasoning technology and block chain method and server thereof | |
| CN109214202B (en) | Data analysis and diagnosis system, device, method and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20200519 |
|
| WW01 | Invention patent application withdrawn after publication |