CN112667888A - Big data processing system based on block chain - Google Patents
Big data processing system based on block chain Download PDFInfo
- Publication number
- CN112667888A CN112667888A CN202011536944.2A CN202011536944A CN112667888A CN 112667888 A CN112667888 A CN 112667888A CN 202011536944 A CN202011536944 A CN 202011536944A CN 112667888 A CN112667888 A CN 112667888A
- Authority
- CN
- China
- Prior art keywords
- data
- module
- information
- pushing
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 title claims abstract description 32
- 238000007405 data analysis Methods 0.000 claims abstract description 70
- 230000000694 effects Effects 0.000 claims abstract description 25
- 238000000034 method Methods 0.000 claims abstract description 23
- 230000005540 biological transmission Effects 0.000 claims description 29
- 238000012795 verification Methods 0.000 claims description 8
- 238000004458 analytical method Methods 0.000 claims description 6
- 238000013500 data storage Methods 0.000 claims description 6
- 230000003631 expected effect Effects 0.000 claims description 6
- 238000005516 engineering process Methods 0.000 claims description 5
- 230000007547 defect Effects 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 4
- 238000013329 compounding Methods 0.000 claims description 3
- 230000001629 suppression Effects 0.000 claims description 3
- 230000008901 benefit Effects 0.000 description 5
- 238000010276 construction Methods 0.000 description 4
- 230000011218 segmentation Effects 0.000 description 3
- 238000000586 desensitisation Methods 0.000 description 2
- 230000002401 inhibitory effect Effects 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 230000014759 maintenance of location Effects 0.000 description 2
- 230000009323 psychological health Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
Images
Landscapes
- Storage Device Security (AREA)
Abstract
The invention belongs to the technical field of big data, and particularly relates to a big data processing system based on a block chain, which comprises a data acquisition module, a data analysis module, a data confusion module and a data pushing module; the data acquisition module uninterruptedly collects content related information watched by a user and sends the content related information to the data confusion module; the data confusion module processes the data after receiving the related data to obtain nondirectional preference data, and sends the data to the data analysis module after the processing is finished; after receiving the nondirectional data, the data analysis module analyzes the data to obtain preference state information of the user in the corresponding area and sends the preference state information to the data pushing module; the data pushing module pushes relevant information in a targeted manner according to the user preference state information; the invention ensures the privacy and safety of big data and has good information popularization effect.
Description
Technical Field
The invention belongs to the technical field of big data, and particularly relates to a big data processing system based on a block chain.
Background
Big data (bigdata), which refers to a data set that cannot be captured, managed and processed by a conventional software tool within a certain time range, is an information asset that needs a new processing mode to have stronger decision-making power, insight discovery power and flow optimization capability to adapt to a large amount, high growth rate and diversification. With the rapid development of electronic informatization, electronic information data gradually becomes the key point of research of people, and people can not leave various data in daily life, so that big data becomes a hot spot of current research.
In addition, the conventional blockchain technology is a data technology which tends to be decentralized, non-falsifiable and data public, and once the stored big data is acquired by all nodes, the big data privacy and the big data security cannot be guaranteed. In the real application process, it is a necessary prerequisite that the data service provider guarantees the data security and privacy of the customer in the process of providing services for the customer, and if the data security cannot be guaranteed, the data service provider will not be accepted and approved by the customer.
Disclosure of Invention
In order to make up for the defects of the prior art, improve the privacy safety of big data based on a block chain and improve the information popularization effect, the invention provides a big data processing system based on the block chain.
The technical scheme adopted by the invention for solving the technical problems is as follows: the invention relates to a big data processing system based on a block chain, which comprises a data acquisition module, a data analysis module, a data confusion module and a data pushing module; the data acquisition module adopts a crawler technology to uninterruptedly collect content related information watched by a user; the data acquisition module sends the acquired data to the data confusion module; after the data confusion module receives the related data, the data is processed by adopting a black box mode according to a specific algorithm, and the relevance between a specific individual user and specific preference data in the data is removed or disturbed to obtain the non-directional preference data; approximate geographic information and approximate time information are still stored in the nondirectional data; the basic processing unit of the rough geographic information is a county-level region; the approximate time information is accurate to an hour; the data confusion module processes the acquired data and sends the acquired omnidirectional data to the data analysis module; after receiving the nondirectional data, the data analysis module analyzes the data to obtain preference state information of the user in the corresponding area; after the data analysis module obtains the user preference state information, the information is sent to a data pushing module; the data pushing module pushes relevant information in different regions at different time points according to the user preference state information, so that the popularization effect is improved;
during operation, the data acquisition module acquires user preference data, the data confusion module processes the acquired data to complete data desensitization to obtain nondirectional data, the acquired data is prevented from being associated with a specific user to cause leakage of user privacy information, and meanwhile, after the nondirectional data are obtained, the data analysis module analyzes the nondirectional data to obtain preference information of users in related regions, so that the data push module can push the data pertinently to improve the popularization effect, meanwhile, the user preference information of the related regions can be provided to corresponding cooperative units to promote the service quality of the related units in the specific regions to be improved, the retention rate of consumers is improved, the economic benefit is improved, and meanwhile, the data analysis module analyzes the preference information of the users in a specific time period to predict the psychological state of the users at a specific time point, the specific region is directionally popularized at a specific time point, and the popularization effect of the data push module is further improved.
Preferably, the data analysis module analyzes the received omnidirectional data, divides user groups according to different browsing information contents, and improves the effect of directional popularization of the data pushing module; the data analysis module analyzes the nondirectional data, and individually distinguishes parts belonging to the violation information, so that user groups, places and time which prefer to browse the violation information are obtained according to the violation information analysis, and violation information characteristic data are obtained; the data analysis module sends violation information characteristic data to the data pushing module; the data pushing module is used for increasing the quantity and frequency of pushing general laws and education information to a specific area at a specific time point according to the received violation information characteristic data;
when the system works, the nondirectional data are analyzed by the data analysis module, so that the region where the user group prefers to browse the violation information in daily life and the proportion of the violation information which the user group prefers to browse in the region can be found out, therefore, the data push module pushes the general law and the education information to the users in the region, and by increasing the quantity and frequency of pushing the general law and the education information, the law control consciousness of the user group in the region is improved in the process of subtlety, the law control and culture level construction degree of the user in the region is enhanced, the condition that the user group quality is reduced, the law control and culture level construction are reduced due to the fact that the user in the region browses too much violation information is avoided, meanwhile, the data pushing module is used for increasing the pushed general law and education information, inhibiting or eliminating potential illegal intentions of a few users, and improving the willingness and level of the users in the region to watch law and understand law.
Preferably, after receiving the omnidirectional data, the data analysis module analyzes the part of data, and performs suppression information marking on information belonging to a negative state in the part of data; the data analysis module divides user groups according to the marked depression information marks, distinguishes the user groups, places and time which prefer to browse negative information, and obtains pressure characteristic data; the data analysis module sends the pressure characteristic data to the data pushing module; the data pushing module increases the number and frequency of front information release in a specific area according to the received pressure characteristic data; the positive information includes but is not limited to positive energy videos, articles; when the data pushing module pushes the front information in a specific area, the pushing force for the whole day time period is increased relative to the historical pushing force, and the force for pushing the front information in the specific time period is increased to a peak value;
during operation, nondirectional data are analyzed through the data analysis module, regions and groups which are preferential to browse negative information are found out, then the strength of pushing positive information to the regions is increased through the data pushing module, psychological depression or depression caused by long-term preferential browse negative information of users in the regions is avoided, the psychological health level of the users in the regions is reduced, the people in the regions are not good in happiness, meanwhile, the positive information of the strength pushing is increased through the data pushing module, the potential depression state of the user groups in the regions can be relieved or eliminated to a certain extent in a subtlety mode, and most of the users in the user groups can actively, optimistically and upwards see life.
Preferably, the data analysis module analyzes the received omnidirectional data and establishes a user preference model; the data analysis module obtains predicted data of the user browsing negative information and violation information through analysis of the model; the data analysis module analyzes the received nondirectional data to obtain statistical data of the negative information and the violation information browsed by the user; the data analysis module compares the predicted data with the statistical data to obtain an absolute value of a comparison result; the absolute value of the comparison result is larger than or equal to zero, the data analysis module judges that the general law, the education information and the positive information which are pushed by the data pushing module in a pertinence manner obtain the expected effect, and the user group browsing the violation information and the negative information is reduced or unchanged; the absolute value of the comparison result is less than zero, the general law, the education information and the positive information which are pushed in a targeted manner by the data pushing module do not obtain the expected effect, and the data analysis module judges that the user group browsing the violation information and the negative information is expanded; the absolute value of the comparison result is less than zero, and the data analysis module sends a pushing force increasing instruction to the data pushing module; after the data pushing module receives the instruction, the data pushing module increases daily information pushing force;
during operation, through the user preference model that data analysis module established, and obtain the forecast data through the model, through the contrast to forecast data and statistical data, can clearly reflect the effect that relevant information of data propelling movement module pertinence propelling movement gained, guarantee that data propelling movement module is effectual to the popularization of relevant information, avoid data propelling movement module not good to relevant information's popularization effect, simultaneously, can reflect whether suitable to the propelling movement dynamics of data propelling movement module through the contrast to forecast data and statistical data, avoid data propelling movement module to carry out the in-process propelling movement dynamics undersize or too big to relevant information, lead to the propelling movement effect not good or arouse user's crowd's reaction, reduce the propelling movement effect.
Preferably, the processing system comprises a data storage module, a data encryption module and a node propagation module; the data storage module is used for storing data generated by the processing system in the working process and collected user data; the data encryption module encrypts the omnidirectional data output by the data confusion module to obtain secret data and a decryption key; after the data encryption module obtains the security level data, the security level data are sent to the node transmission module; after receiving the secret level data, the node transmission module transmits the secret level data to the block chain network in a broadcasting mode and receives stored signals returned by other nodes in the block chain network; after receiving return signals of half of nodes except the nodes in the block chain network, the node transmission module judges that the confidential data is successfully linked and stops broadcasting the confidential data; when the data encryption module obtains the security level data, a verification MD5 value of the security level data is generated at the same time, and the verification MD5 value is divided into two ends with equal length and is respectively attached to the head and the tail of the security level data; after receiving the complete security level data, other nodes in the block chain network check the security level data in the background; when the other nodes check the secret data, extracting check MD5 values attached to the head and tail of the secret data, and performing comparison MD5 value obtained by calculating the check MD5 value and the secret data; if the comparison MD5 value is consistent with the check MD5 value, other nodes judge that the received security level data is complete and correct, and store the security level data and broadcast the stored signal; if the comparison MD5 value is inconsistent with the check MD5 value, other nodes judge that the received security level data has defects and does not accord with the expected received data, and other nodes discard the received security level data, keep silent and do not broadcast the stored signals;
when the system works, collected omnidirectional data is encrypted by the data encryption module and then broadcast to the blockchain network through the node transmission module for storage, the characteristics of the blockchain network are utilized to ensure the safety, the openness and the transparency of the data and avoid the data from being attacked and falsified, meanwhile, in the process of uploading confidential data to the blockchain network, the MD5 value of the confidential data is added to the confidential data before and after the confidential data, the integrity and the reliability of the confidential data received by other nodes in the blockchain network can be ensured, the confidential data is prevented from being disturbed and attacked in the process of transmitting the confidential data in the blockchain network, the loss or falsification of the confidential data is caused, the stored confidential data in the blockchain network is deviated, and meanwhile, the verification is carried out by comparing the MD5 value, the confidential data stored in the blockchain network is ensured to be completely consistent with the data broadcast by the node transmission module, the method and the device avoid that other nodes in the block network cannot verify the secret data after the secret data is tampered and interfered and has a deviation in the transmission process, and directly store the wrong secret data, so that the block network authenticates the wrong secret data as correct data, acknowledges the correctness of the wrong secret data, and rejects the correctness of the correct secret data initially sent by the node transmission module.
Preferably, when encrypting the non-directional data, the data encryption module segments the received non-directional data according to a predetermined segmentation algorithm to obtain data blocks with the same size and arranged in sequence, and then divides and stores the data blocks obtained after segmentation into local storage data and block network storage data by using a division algorithm; the data encryption module encrypts the block network data by using a Hash encryption algorithm to obtain secret data, and the secret data is sent to the node transmission module and broadcasted to the block chain network through the node transmission module for storage; the ratio of the local storage data to the block network storage data is (8-9): 1; the data block is divided and stored by adopting a division algorithm comprising a time function, block network storage data are extracted from the data blocks which are same in size and are arranged in sequence according to the division algorithm, and the residual data blocks after extraction are stored as local storage data; the time function is the current working time point of the data encryption module; the local storage data does not comprise data blocks which are extracted and used as block network storage data; when the processing system or other users use the nondirectional data, downloading the secret data from the block chain network, decrypting the secret data through the data encryption module, reversely running the division algorithm, compounding the block network storage data and the local storage data in the secret data to obtain data blocks which are same in size and are sequentially arranged after being divided, and then reversely running the division algorithm to obtain original nondirectional data;
when the system works, the data encryption module is used for segmenting and extracting the non-directional data to obtain a relatively small number of block stored data from the non-directional data, the block stored data are uploaded to the block chain network as secret data, the data amount required to be transmitted by the node transmission module is reduced, the load of the block chain network is reduced, the block chain network is prevented from being too bulky, meanwhile, the complete non-directional data are segmented, and a part of the encrypted data are sent to the block chain network as the secret data in a broadcasting mode, so that the safety of the whole data can be ensured under the condition that a secret key is accidentally leaked, the condition that lawless persons acquire all data from the block chain network and decrypt the data through the secret key to cause data leakage can be avoided, meanwhile, the data can be further improved by segmenting the data, the data stored in the block network is guaranteed not to be tampered, the fact that the block chain selected by a processing system is relatively small in size and cannot be tampered with through a 51% attack mode, the data stored in the block chain network is tampered with the advantage of computing power, the safety and accuracy of the data are affected, when the secret level data stored in the block chain network has deviation, the data cannot be combined with the locally stored data after decryption, and the fact that the data are affected or misled by tampering or deviation data is avoided.
The invention has the following beneficial effects:
1. according to the big data processing system based on the block chain, the data analysis module is used for analyzing the collected information, so that the data push module can push the related information of a specific region and a user group in a targeted manner, the information push effect is improved, meanwhile, the data analysis module is used for establishing a user preference model, the expected data obtained by analyzing the model is compared with the statistical data, the effect of the data push module for pushing the related information is obtained, the push strength of the data push module is dynamically adjusted, and the push effect of the related information is improved.
2. The big data processing system based on the block chain divides the nondirectional data through the data encryption module, extracts part of the data from the nondirectional data to serve as secret data, broadcasts the secret data to the block chain network through the node transmission module after encryption, stores the data in the block chain network, ensures the authenticity of the data, avoids the data from being tampered, reduces the data volume uploaded to the block chain network through the division and encryption of the data, and ensures the privacy and the safety of the data.
Drawings
The invention will be further explained with reference to the drawings.
FIG. 1 is a system flow diagram of the present invention;
Detailed Description
In order to make the technical means, the creation characteristics, the achievement purposes and the effects of the invention easy to understand, the invention is further described with the specific embodiments.
As shown in fig. 1, the big data processing system based on the block chain includes a data acquisition module, a data analysis module, a data confusion module, and a data pushing module; the data acquisition module adopts a crawler technology to uninterruptedly collect content related information watched by a user; the data acquisition module sends the acquired data to the data confusion module; after the data confusion module receives the related data, the data is processed by adopting a black box mode according to a specific algorithm, and the relevance between a specific individual user and specific preference data in the data is removed or disturbed to obtain the non-directional preference data; approximate geographic information and approximate time information are still stored in the nondirectional data; the basic processing unit of the rough geographic information is a county-level region; the approximate time information is accurate to an hour; the data confusion module processes the acquired data and sends the acquired omnidirectional data to the data analysis module; after receiving the nondirectional data, the data analysis module analyzes the data to obtain preference state information of the user in the corresponding area; after the data analysis module obtains the user preference state information, the information is sent to a data pushing module; the data pushing module pushes relevant information in different regions at different time points according to the user preference state information, so that the popularization effect is improved;
during operation, the data acquisition module acquires user preference data, the data confusion module processes the acquired data to complete data desensitization to obtain nondirectional data, the acquired data is prevented from being associated with a specific user to cause leakage of user privacy information, and meanwhile, after the nondirectional data are obtained, the data analysis module analyzes the nondirectional data to obtain preference information of users in related regions, so that the data push module can push the data pertinently to improve the popularization effect, meanwhile, the user preference information of the related regions can be provided to corresponding cooperative units to promote the service quality of the related units in the specific regions to be improved, the retention rate of consumers is improved, the economic benefit is improved, and meanwhile, the data analysis module analyzes the preference information of the users in a specific time period to predict the psychological state of the users at a specific time point, the specific region is directionally popularized at a specific time point, and the popularization effect of the data push module is further improved.
As an implementation manner of the present invention, the data analysis module analyzes the received omnidirectional data, divides the user group according to the different browsing information contents, and improves the effect of the data pushing module on directional popularization; the data analysis module analyzes the nondirectional data, and individually distinguishes parts belonging to the violation information, so that user groups, places and time which prefer to browse the violation information are obtained according to the violation information analysis, and violation information characteristic data are obtained; the data analysis module sends violation information characteristic data to the data pushing module; the data pushing module is used for increasing the quantity and frequency of pushing general laws and education information to a specific area at a specific time point according to the received violation information characteristic data;
when the system works, the nondirectional data are analyzed by the data analysis module, so that the region where the user group prefers to browse the violation information in daily life and the proportion of the violation information which the user group prefers to browse in the region can be found out, therefore, the data push module pushes the general law and the education information to the users in the region, and by increasing the quantity and frequency of pushing the general law and the education information, the law control consciousness of the user group in the region is improved in the process of subtlety, the law control and culture level construction degree of the user in the region is enhanced, the condition that the user group quality is reduced, the law control and culture level construction are reduced due to the fact that the user in the region browses too much violation information is avoided, meanwhile, the data pushing module is used for increasing the pushed general law and education information, inhibiting or eliminating potential illegal intentions of a few users, and improving the willingness and level of the users in the region to watch law and understand law.
As an embodiment of the present invention, after receiving the nondirectional data, the data analysis module analyzes the partial data, and performs suppression information marking on information belonging to a negative state in the partial data; the data analysis module divides user groups according to the marked depression information marks, distinguishes the user groups, places and time which prefer to browse negative information, and obtains pressure characteristic data; the data analysis module sends the pressure characteristic data to the data pushing module; the data pushing module increases the number and frequency of front information release in a specific area according to the received pressure characteristic data; the positive information includes but is not limited to positive energy videos, articles; when the data pushing module pushes the front information in a specific area, the pushing force for the whole day time period is increased relative to the historical pushing force, and the force for pushing the front information in the specific time period is increased to a peak value;
during operation, nondirectional data are analyzed through the data analysis module, regions and groups which are preferential to browse negative information are found out, then the strength of pushing positive information to the regions is increased through the data pushing module, psychological depression or depression caused by long-term preferential browse negative information of users in the regions is avoided, the psychological health level of the users in the regions is reduced, the people in the regions are not good in happiness, meanwhile, the positive information of the strength pushing is increased through the data pushing module, the potential depression state of the user groups in the regions can be relieved or eliminated to a certain extent in a subtlety mode, and most of the users in the user groups can actively, optimistically and upwards see life.
As an embodiment of the present invention, the data analysis module analyzes the received omnidirectional data and establishes a user preference model; the data analysis module obtains predicted data of the user browsing negative information and violation information through analysis of the model; the data analysis module analyzes the received nondirectional data to obtain statistical data of the negative information and the violation information browsed by the user; the data analysis module compares the predicted data with the statistical data to obtain an absolute value of a comparison result; the absolute value of the comparison result is larger than or equal to zero, the data analysis module judges that the general law, the education information and the positive information which are pushed by the data pushing module in a pertinence manner obtain the expected effect, and the user group browsing the violation information and the negative information is reduced or unchanged; the absolute value of the comparison result is less than zero, the general law, the education information and the positive information which are pushed in a targeted manner by the data pushing module do not obtain the expected effect, and the data analysis module judges that the user group browsing the violation information and the negative information is expanded; the absolute value of the comparison result is less than zero, and the data analysis module sends a pushing force increasing instruction to the data pushing module; after the data pushing module receives the instruction, the data pushing module increases daily information pushing force;
during operation, through the user preference model that data analysis module established, and obtain the forecast data through the model, through the contrast to forecast data and statistical data, can clearly reflect the effect that relevant information of data propelling movement module pertinence propelling movement gained, guarantee that data propelling movement module is effectual to the popularization of relevant information, avoid data propelling movement module not good to relevant information's popularization effect, simultaneously, can reflect whether suitable to the propelling movement dynamics of data propelling movement module through the contrast to forecast data and statistical data, avoid data propelling movement module to carry out the in-process propelling movement dynamics undersize or too big to relevant information, lead to the propelling movement effect not good or arouse user's crowd's reaction, reduce the propelling movement effect.
As an implementation mode of the invention, the processing system comprises a data storage module, a data encryption module and a node propagation module; the data storage module is used for storing data generated by the processing system in the working process and collected user data; the data encryption module encrypts the omnidirectional data output by the data confusion module to obtain secret data and a decryption key; after the data encryption module obtains the security level data, the security level data are sent to the node transmission module; after receiving the secret level data, the node transmission module transmits the secret level data to the block chain network in a broadcasting mode and receives stored signals returned by other nodes in the block chain network; after receiving return signals of half of nodes except the nodes in the block chain network, the node transmission module judges that the confidential data is successfully linked and stops broadcasting the confidential data; when the data encryption module obtains the security level data, a verification MD5 value of the security level data is generated at the same time, and the verification MD5 value is divided into two ends with equal length and is respectively attached to the head and the tail of the security level data; after receiving the complete security level data, other nodes in the block chain network check the security level data in the background; when the other nodes check the secret data, extracting check MD5 values attached to the head and tail of the secret data, and performing comparison MD5 value obtained by calculating the check MD5 value and the secret data; if the comparison MD5 value is consistent with the check MD5 value, other nodes judge that the received security level data is complete and correct, and store the security level data and broadcast the stored signal; if the comparison MD5 value is inconsistent with the check MD5 value, other nodes judge that the received security level data has defects and does not accord with the expected received data, and other nodes discard the received security level data, keep silent and do not broadcast the stored signals;
when the system works, collected omnidirectional data is encrypted by the data encryption module and then broadcast to the blockchain network through the node transmission module for storage, the characteristics of the blockchain network are utilized to ensure the safety, the openness and the transparency of the data and avoid the data from being attacked and falsified, meanwhile, in the process of uploading confidential data to the blockchain network, the MD5 value of the confidential data is added to the confidential data before and after the confidential data, the integrity and the reliability of the confidential data received by other nodes in the blockchain network can be ensured, the confidential data is prevented from being disturbed and attacked in the process of transmitting the confidential data in the blockchain network, the loss or falsification of the confidential data is caused, the stored confidential data in the blockchain network is deviated, and meanwhile, the verification is carried out by comparing the MD5 value, the confidential data stored in the blockchain network is ensured to be completely consistent with the data broadcast by the node transmission module, the method and the device avoid that other nodes in the block network cannot verify the secret data after the secret data is tampered and interfered and has a deviation in the transmission process, and directly store the wrong secret data, so that the block network authenticates the wrong secret data as correct data, acknowledges the correctness of the wrong secret data, and rejects the correctness of the correct secret data initially sent by the node transmission module.
As an embodiment of the present invention, when encrypting the non-directional data, the data encryption module divides the received non-directional data according to a predetermined division algorithm to obtain data blocks with the same size and arranged in sequence, and then divides and stores the data blocks obtained after division into local storage data and block network storage data by using a division algorithm; the data encryption module encrypts the block network data by using a Hash encryption algorithm to obtain secret data, and the secret data is sent to the node transmission module and broadcasted to the block chain network through the node transmission module for storage; the ratio of the local storage data to the block network storage data is (8-9): 1; the data block is divided and stored by adopting a division algorithm comprising a time function, block network storage data are extracted from the data blocks which are same in size and are arranged in sequence according to the division algorithm, and the residual data blocks after extraction are stored as local storage data; the time function is the current working time point of the data encryption module; the local storage data does not comprise data blocks which are extracted and used as block network storage data; when the processing system or other users use the nondirectional data, downloading the secret data from the block chain network, decrypting the secret data through the data encryption module, reversely running the division algorithm, compounding the block network storage data and the local storage data in the secret data to obtain data blocks which are same in size and are sequentially arranged after being divided, and then reversely running the division algorithm to obtain original nondirectional data;
when the system works, the data encryption module is used for segmenting and extracting the non-directional data to obtain a relatively small number of block stored data from the non-directional data, the block stored data are uploaded to the block chain network as secret data, the data amount required to be transmitted by the node transmission module is reduced, the load of the block chain network is reduced, the block chain network is prevented from being too bulky, meanwhile, the complete non-directional data are segmented, and a part of the encrypted data are sent to the block chain network as the secret data in a broadcasting mode, so that the safety of the whole data can be ensured under the condition that a secret key is accidentally leaked, the condition that lawless persons acquire all data from the block chain network and decrypt the data through the secret key to cause data leakage can be avoided, meanwhile, the data can be further improved by segmenting the data, the data stored in the block network is guaranteed not to be tampered, the fact that the block chain selected by a processing system is relatively small in size and cannot be tampered with through a 51% attack mode, the data stored in the block chain network is tampered with the advantage of computing power, the safety and accuracy of the data are affected, when the secret level data stored in the block chain network has deviation, the data cannot be combined with the locally stored data after decryption, and the fact that the data are affected or misled by tampering or deviation data is avoided.
The specific working process is as follows:
during work, the data acquisition module is used for acquiring user preference data, the data confusion module is used for processing the acquired data, and the data analysis module is used for analyzing the data so as to obtain preference information of users in related regions, so that the data push module can conveniently push the preference information in a targeted manner; analyzing the nondirectional data through a data analysis module, finding out a region where a user group prefers to browse the violation information in daily life and the proportion of the violation information which the user group prefers to browse in the region, and pushing general law and education information to a user in the region through a data pushing module; the data analysis module analyzes the nondirectional data, finds out regions and crowds which prefer to browse negative information, and then increases the strength of pushing positive information to the regions through the data pushing module; the user preference model established by the data analysis module is used for obtaining predicted data through the model, and the effect obtained by the data pushing module in the process of pushing relevant information in a targeted manner is reflected through the comparison between the predicted data and the statistical data; the acquired omnidirectional data is encrypted by the data encryption module and then broadcast to the block chain network for storage by the node transmission module, and the safety, the openness and the transparency of the data are ensured by utilizing the characteristics of the block chain network, so that the data is prevented from being attacked and tampered; by means of the segmentation and extraction of the nondirectional data by the data encryption module, the block storage data with a relatively small number is obtained from the nondirectional data, and the block storage data is uploaded to the block chain network as secret data, so that the data volume required to be transmitted by the node transmission module is reduced.
The foregoing illustrates and describes the principles, general features, and advantages of the present invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (6)
1. A big data processing system based on a block chain is characterized in that: the device comprises a data acquisition module, a data analysis module, a data confusion module and a data pushing module; the data acquisition module adopts a crawler technology to uninterruptedly collect content related information watched by a user; the data acquisition module sends the acquired data to the data confusion module; after the data confusion module receives the related data, the data is processed by adopting a black box mode according to a specific algorithm, and the relevance between a specific individual user and specific preference data in the data is removed or disturbed to obtain the non-directional preference data; approximate geographic information and approximate time information are still stored in the nondirectional data; the basic processing unit of the rough geographic information is a county-level region; the approximate time information is accurate to an hour; the data confusion module processes the acquired data and sends the acquired omnidirectional data to the data analysis module; after receiving the nondirectional data, the data analysis module analyzes the data to obtain preference state information of the user in the corresponding area; after the data analysis module obtains the user preference state information, the information is sent to a data pushing module; the data pushing module pushes relevant information in different regions at different time points according to the user preference state information, and therefore the popularization effect is improved.
2. The big data processing system based on the block chain as claimed in claim 1, wherein: the data analysis module analyzes the received omnidirectional data, divides user groups according to different browsing information contents, and improves the effect of directional popularization of the data pushing module; the data analysis module analyzes the nondirectional data, and individually distinguishes parts belonging to the violation information, so that user groups, places and time which prefer to browse the violation information are obtained according to the violation information analysis, and violation information characteristic data are obtained; the data analysis module sends violation information characteristic data to the data pushing module; and the data pushing module increases the quantity and frequency of pushing general laws and education information to a specific area at a specific time point according to the received violation information characteristic data.
3. The big data processing system based on the block chain as claimed in claim 2, wherein: after receiving the nondirectional data, the data analysis module analyzes the partial data and performs suppression information marking on the information belonging to the negative state in the partial data; the data analysis module divides user groups according to the marked depression information marks, distinguishes the user groups, places and time which prefer to browse negative information, and obtains pressure characteristic data; the data analysis module sends the pressure characteristic data to the data pushing module; the data pushing module increases the number and frequency of front information release in a specific area according to the received pressure characteristic data; the positive information includes but is not limited to positive energy videos, articles; when the data pushing module pushes the front information in a specific area, the pushing force for the whole day time period is increased relative to the historical pushing force, and the force for pushing the front information in the specific time period is increased to a peak value.
4. The big data processing system based on the block chain as claimed in claim 3, wherein: the data analysis module analyzes the received omnidirectional data and establishes a user preference model; the data analysis module obtains predicted data of the user browsing negative information and violation information through analysis of the model; the data analysis module analyzes the received nondirectional data to obtain statistical data of the negative information and the violation information browsed by the user; the data analysis module compares the predicted data with the statistical data to obtain an absolute value of a comparison result; the absolute value of the comparison result is larger than or equal to zero, the data analysis module judges that the general law, the education information and the positive information which are pushed by the data pushing module in a pertinence manner obtain the expected effect, and the user group browsing the violation information and the negative information is reduced or unchanged; the absolute value of the comparison result is less than zero, the general law, the education information and the positive information which are pushed in a targeted manner by the data pushing module do not obtain the expected effect, and the data analysis module judges that the user group browsing the violation information and the negative information is expanded; the absolute value of the comparison result is less than zero, and the data analysis module sends a pushing force increasing instruction to the data pushing module; after the data pushing module receives the instruction, the data pushing module increases daily information pushing force.
5. The big data processing system based on the block chain as claimed in claim 1, wherein: the processing system comprises a data storage module, a data encryption module and a node transmission module; the data storage module is used for storing data generated by the processing system in the working process and collected user data; the data encryption module encrypts the omnidirectional data output by the data confusion module to obtain secret data and a decryption key; after the data encryption module obtains the security level data, the security level data are sent to the node transmission module; after receiving the secret level data, the node transmission module transmits the secret level data to the block chain network in a broadcasting mode and receives stored signals returned by other nodes in the block chain network; after receiving return signals of half of nodes except the nodes in the block chain network, the node transmission module judges that the confidential data is successfully linked and stops broadcasting the confidential data; when the data encryption module obtains the security level data, a verification MD5 value of the security level data is generated at the same time, and the verification MD5 value is divided into two ends with equal length and is respectively attached to the head and the tail of the security level data; after receiving the complete security level data, other nodes in the block chain network check the security level data in the background; when the other nodes check the secret data, extracting check MD5 values attached to the head and tail of the secret data, and performing comparison MD5 value obtained by calculating the check MD5 value and the secret data; if the comparison MD5 value is consistent with the check MD5 value, other nodes judge that the received security level data is complete and correct, and store the security level data and broadcast the stored signal; and if the comparison MD5 value is inconsistent with the check MD5 value, other nodes judge that the received security level data has defects and does not accord with the expected received data, and other nodes discard the received security level data, keep silent and do not broadcast the stored signal.
6. The big data processing system based on the block chain as claimed in claim 5, wherein: when encrypting the nondirectional data, the data encryption module divides the received nondirectional data according to a preset division algorithm to obtain data blocks which are the same in size and are arranged in sequence, and then divides and stores the data blocks obtained after division into local storage data and block network storage data by adopting a division algorithm; the data encryption module encrypts the block network data by using a Hash encryption algorithm to obtain secret data, and the secret data is sent to the node transmission module and broadcasted to the block chain network through the node transmission module for storage; the ratio of the local storage data to the block network storage data is (8-9): 1; the data block is divided and stored by adopting a division algorithm comprising a time function, block network storage data are extracted from the data blocks which are same in size and are arranged in sequence according to the division algorithm, and the residual data blocks after extraction are stored as local storage data; the time function is the current working time point of the data encryption module; the local storage data does not comprise data blocks which are extracted and used as block network storage data; when the processing system or other users use the nondirectional data, downloading the secret data from the block chain network, decrypting the secret data through the data encryption module, reversely running the division algorithm, compounding the block network storage data and the local storage data in the secret data to obtain the data blocks which are same in size and are sequentially arranged after being divided, and then reversely running the division algorithm to obtain the original nondirectional data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011536944.2A CN112667888A (en) | 2020-12-23 | 2020-12-23 | Big data processing system based on block chain |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011536944.2A CN112667888A (en) | 2020-12-23 | 2020-12-23 | Big data processing system based on block chain |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112667888A true CN112667888A (en) | 2021-04-16 |
Family
ID=75408123
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011536944.2A Pending CN112667888A (en) | 2020-12-23 | 2020-12-23 | Big data processing system based on block chain |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112667888A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113569262A (en) * | 2021-07-30 | 2021-10-29 | 立信(重庆)数据科技股份有限公司 | Ciphertext storage method and system based on block chain |
| CN113674819A (en) * | 2021-08-25 | 2021-11-19 | 昆明智迪教育信息咨询有限责任公司 | Mental health management work monitoring and evaluating method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105956048A (en) * | 2016-04-27 | 2016-09-21 | 上海遥薇(集团)有限公司 | Community service big data algorithm mining system |
| CN110557385A (en) * | 2019-08-22 | 2019-12-10 | 西安电子科技大学 | information hiding access method and system based on behavior confusion, and server |
| CN111953670A (en) * | 2020-07-30 | 2020-11-17 | 江苏大学 | Self-adaptive confusion method and system based on Meek transmission plug-in and computer storage medium |
-
2020
- 2020-12-23 CN CN202011536944.2A patent/CN112667888A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105956048A (en) * | 2016-04-27 | 2016-09-21 | 上海遥薇(集团)有限公司 | Community service big data algorithm mining system |
| CN110557385A (en) * | 2019-08-22 | 2019-12-10 | 西安电子科技大学 | information hiding access method and system based on behavior confusion, and server |
| CN111953670A (en) * | 2020-07-30 | 2020-11-17 | 江苏大学 | Self-adaptive confusion method and system based on Meek transmission plug-in and computer storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113569262A (en) * | 2021-07-30 | 2021-10-29 | 立信(重庆)数据科技股份有限公司 | Ciphertext storage method and system based on block chain |
| CN113674819A (en) * | 2021-08-25 | 2021-11-19 | 昆明智迪教育信息咨询有限责任公司 | Mental health management work monitoring and evaluating method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111756522B (en) | Data processing method and system | |
| CN110677411B (en) | Data sharing method and system based on cloud storage | |
| CN113449048B (en) | Data label distribution determining method and device, computer equipment and storage medium | |
| CN115062328B (en) | Intelligent information analysis method based on cross-modal data fusion | |
| CN108809990B (en) | Crowdsourcing data security encryption method, server and storage medium | |
| CN112667888A (en) | Big data processing system based on block chain | |
| CN114024744B (en) | Information protection method based on cloud computing and blockchain service and artificial intelligent platform | |
| US20160063278A1 (en) | Privacy Compliance Event Analysis System | |
| CN113766085B (en) | Image processing method and related device | |
| CN112949545A (en) | Method, apparatus, computing device and medium for recognizing face image | |
| CN111639355B (en) | Data security management method and system | |
| CN116383793B (en) | Face data processing method, device, electronic equipment and computer readable medium | |
| CN105337742A (en) | LFSR (Linear Feedback Shift Register) file encryption and decryption methods based on human face image features and GPS (Global Position System) information | |
| WO2017107431A1 (en) | Method, device and electronic equipment for sorting images | |
| CN115296908A (en) | Encryption method and device for sensitive information, electronic equipment and storage medium | |
| CN111899741A (en) | Audio keyword encryption method and device, storage medium and electronic device | |
| CN113709513B (en) | Equipment fingerprint processing method, user side, server, system and storage medium | |
| CN116090024A (en) | Reliable data storage device, system and method | |
| CN117692257B (en) | High-speed encryption method and device for service data of electric power Internet of things | |
| CN110222187B (en) | Common activity detection and data sharing method for protecting user privacy | |
| CN108900472A (en) | The transmission method and device of information | |
| CN115690969A (en) | Non-contact fingerprint authentication method, terminal, access control equipment and storage medium | |
| CN112733188B (en) | Sensitive file management method | |
| CN109146522A (en) | Food safety trace back method and system based on block chain | |
| CN111582954B (en) | False data identification method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |