CN111258981A - Data processing method, device, equipment and storage medium - Google Patents
Data processing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN111258981A CN111258981A CN202010031154.2A CN202010031154A CN111258981A CN 111258981 A CN111258981 A CN 111258981A CN 202010031154 A CN202010031154 A CN 202010031154A CN 111258981 A CN111258981 A CN 111258981A
- Authority
- CN
- China
- Prior art keywords
- data
- field
- target
- supervision
- supervision data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the invention discloses a data processing method, a device, equipment and a storage medium, wherein the method comprises the following steps: determining first supervision data of at least two target objects supervised by a first group of authorities; determining the data reliability corresponding to each first supervision data according to field weight information and field source reliability information in a preset rule configuration file; and merging the first supervision data according to the credibility of the data to determine the target supervision data of the target object. By the technical scheme of the embodiment of the invention, the flexibility of data merging can be improved, and the accuracy of data merging is ensured.
Description
Technical Field
Embodiments of the present invention relate to computer technologies, and in particular, to a data processing method, an apparatus, a device, and a storage medium.
Background
An organization may refer to a business institution established pursuant to law, such as a provincial institution, a government institution, a bureau of the provincial, a district setting institution, etc. Generally, a superior-subordinate relationship exists between organizations, and subordinate organizations need to report acquired supervision data to a superior organization, so that the superior organization can know the situation of the subordinate organizations in real time.
Because different lower-level organizations can simultaneously supervise the same object, the higher-level organizations can obtain different supervision data of the same object, and the supervision data needs to be merged and processed into one piece of data so as to be checked and managed. In the prior art, merging is generally performed according to the sequence of the acquisition time of each piece of supervision data, for example, the latest obtained supervision data is directly overlaid on the existing supervision data of the object, so as to store the latest supervision data of the object.
However, in the process of implementing the present invention, the inventor finds that at least the following problems exist in the prior art:
the existing merging mode is only based on the parameter of data acquisition time for merging, so that the merging flexibility is poor, and the reliability of data sources is not considered, so that the accuracy of data merging cannot be ensured.
Disclosure of Invention
The embodiment of the invention provides a data processing method, a data processing device, data processing equipment and a storage medium, which are used for improving the flexibility of data merging and ensuring the accuracy of data merging.
In a first aspect, an embodiment of the present invention provides a data processing method, including:
determining first supervision data of at least two target objects supervised by a first group of authorities;
determining the data reliability corresponding to each first supervision data according to field weight information and field source reliability information in a preset rule configuration file;
and merging the first supervision data according to the data credibility to determine the target supervision data of the target object.
In a second aspect, an embodiment of the present invention further provides a data processing apparatus, including:
a first supervision data determining module for determining first supervision data of at least two first groups of target objects supervised by the organization;
the data reliability determining module is used for determining the data reliability corresponding to each first supervision data according to field weight information and field source reliability information in a preset rule configuration file;
and the data merging processing module is used for merging each first supervision data according to each data credibility and determining the target supervision data of the target object.
In a third aspect, an embodiment of the present invention further provides an apparatus, where the apparatus includes:
one or more processors;
a memory for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement a data processing method as provided by any of the embodiments of the invention.
In a fourth aspect, the embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the data processing method provided in any embodiment of the present invention.
The embodiment of the invention has the following advantages or beneficial effects:
determining first supervision data of at least two first groups of supervised target objects; determining the data reliability corresponding to each first supervision data according to field weight information and field source reliability information in a preset rule configuration file; the first supervision data are merged according to the credibility of the data, so that the data can be merged based on the reliability of a data source, the accuracy of merging the data is ensured, and a preset rule configuration file can be preset based on business requirements so as to be merged based on the preset rule configuration file, so that the flexibility of merging the data is improved, and the personalized requirements of users are met.
Drawings
Fig. 1 is a flowchart of a data processing method according to an embodiment of the present invention;
fig. 2 is a flowchart of a data processing method according to a second embodiment of the present invention;
fig. 3 is a schematic structural diagram of a data processing apparatus according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of an apparatus according to a fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1 is a flowchart of a data processing method according to an embodiment of the present invention, which is applicable to merging supervision data of each lower-level organization. The method may be performed by a data processing apparatus, which may be implemented by means of software and/or hardware, integrated in a device having data processing capabilities. The method specifically comprises the following steps:
s110, determining first supervision data of at least two target objects supervised by a first group of mechanisms.
The first group of organizations may refer to organizations belonging to the same class and needing to report data. For example, the first set of organization may be, but is not limited to, a respective hall office organization within a province, and the like. The target object may be the same object supervised by at least two first groups of organizations. For example, the target object may be, but is not limited to: law enforcement, business, individual industrial and commercial businesses, natural persons, administrative relatives, and the like. The first supervision data may refer to a piece of data of a target object supervised by the first group of organizations. For example, when the target object is a law enforcement officer, the first regulatory data is law enforcement officer data, which may include, but is not limited to: at least one of personal information, unit information, law enforcement post information, law enforcement certificate number information, and qualification information. The first supervision data in this embodiment may be stored in a data table manner.
Specifically, a plurality of lower-level organizations, that is, second organizations, may exist under each first-level organization, so that each first-level organization may supervise through the respective second-level organizations under the first-level organization, and report supervision data obtained by supervision to a data processing device provided in a higher-level organization of the first organization by means of a front-end processor. If the data processing device receives a piece of supervision data of the target object reported by a certain first group of mechanisms, the piece of supervision data can be determined as the first supervision data corresponding to the first group of mechanisms. If the data processing device receives at least two pieces of supervision data of the target object reported by a certain first group of mechanisms, the data processing device can perform data merging processing on the at least two pieces of supervision data to determine first supervision data corresponding to the first group of mechanisms.
Exemplarily, when the supervision object is zhang zai of law enforcement officer, table 1 shows the first supervision data of zhang zai of law enforcement officer supervised by the judicial hall and the municipality. The identity card number in table 1 may be used as a primary key identifier for distinguishing different objects, and all the first supervision data of the same object may be obtained based on the primary key identifier.
TABLE 1 first supervision data of Zhang III Law Enforcement supervised by the judicial parlor and municipal administration
And S120, determining the data reliability corresponding to each first supervision data according to the field weight information and the field source reliability information in the preset rule configuration file.
The preset rule configuration file may be pre-configured based on the service requirement and the scenario, so as to improve the flexibility of data processing. The field weight information may include weights of the respective supervision fields corresponding to each object, and the sum of the weights of the respective supervision fields corresponding to the same object is equal to 1. The weight of a regulatory field may be used to characterize the importance of the regulatory field. For example, a higher weight for a regulatory field indicates a higher importance for the object to which the regulatory field corresponds. The field source credibility information may include credibility of field value information of respective regulatory fields corresponding to each object obtained from respective source authorities. When the credibility of the field value information of a certain supervision field obtained from a certain source authority is higher, it indicates that the authenticity of the field value information obtained from the source authority is higher. Data trustworthiness may refer to the degree of trustworthiness of each first regulatory data evaluated as a whole. The field weight information and the field source reliability information in the implementation can be represented in a data table mode. For example, table 2 gives an example of field weight information for law enforcement officers; table 3 gives an example of the source confidence information for a field corresponding to law enforcement officer.
TABLE 2 field weight information for Law Enforcement
| Object | Supervision field | Weight of |
| Law enforcement personnel | Age (age) | 0.2 |
| Law enforcement personnel | Address | 0.1 |
| Law enforcement personnel | Law enforcement certificate number | 0.7 |
TABLE 3 Law enforcement officer corresponding field Source credibility information
| Table name | Name of field | First data source mechanism | Degree of confidence |
| Law enforcement personnel watch | Age (age) | Judicial hall | 1 |
| Law enforcement personnel watch | Age (age) | City supervising office | 0.5 |
| Law enforcement personnel watch | Address | Judicial hall | 1 |
| Law enforcement personnel watch | Address | City supervising office | 1 |
| Law enforcement personnel watch | Law enforcement certificate number | Judicial hall | 1 |
| Law enforcement personnel watch | Law enforcement certificate number | City supervising office | 0.5 |
Specifically, when determining first supervision data of the same object (i.e., a target object) supervised by at least two first groups of authorities, for each piece of first supervision data, the field weight information and the field source reliability information in the preset rule configuration file may be used to determine the reliability of each supervision field in the first supervision data, and add the reliabilities of the supervision fields to obtain the data reliability corresponding to the first supervision data.
Exemplarily, S120 may include: acquiring each effective field in each first supervision data and a first data source mechanism corresponding to the first supervision data; determining the credibility of the target field corresponding to each effective field according to the first data source mechanism, each effective field, and field weight information and field source credibility information in the preset rule configuration file; and determining the data reliability corresponding to the first supervision data according to the reliability of each target field.
The first data source mechanism corresponding to the first supervision data may refer to a first organization mechanism reporting the first supervision data. The valid field may refer to a supervision field in which field value information in the first supervision data is non-empty. The reliability of the target field can be used for representing the reliability and trueness of the field value information of the effective field reported by the first data source mechanism.
Specifically, there may be supervision fields in each first supervision data, where field value information is null, so that it is required to detect whether field value information of each supervision field in each first supervision data is null information, and determine that field value information in each first supervision data is a respective valid field of non-null information. For each valid field in each first supervision data, the credibility of the target field corresponding to the valid field can be determined according to field weight information and field source credibility information in a preset rule configuration file. By adding the target field credibility corresponding to each valid field in the first supervision data, the obtained addition result can be determined as the data credibility corresponding to the first supervision data. Similarly, the data reliability corresponding to each first supervision data can be determined.
For example, determining the reliability of the target field corresponding to each valid field according to the first data source mechanism, each valid field, and the field weight information and the field source reliability information in the preset rule configuration file may include:
determining the target weight corresponding to each effective field according to field weight information in a preset rule configuration file; determining the target credibility corresponding to each effective field according to field source credibility information in a preset rule configuration file and a first data source mechanism; and determining the target field reliability corresponding to each effective field according to the target weight and the target reliability corresponding to each effective field.
Specifically, based on the target object identifier corresponding to each valid field, the target weight corresponding to each valid field under the target object identifier may be queried from field weight information in the preset rule configuration file. The target credibility corresponding to each effective field corresponding to the first data source mechanism under the target object identifier can be inquired from the field source credibility information in the preset rule configuration file based on the target object identifier corresponding to each effective field and the first data source mechanism. And multiplying the target weight and the target reliability corresponding to the same effective field, and determining the obtained multiplication result as the target field reliability corresponding to the effective field. Similarly, the credibility of the target field corresponding to each effective field can be determined.
It should be noted that, the reliability of the target field corresponding to each invalid field in which the field value information in the first supervision data is empty is zero. Illustratively, based on the contents given in tables 1-3, table 4 gives an example of the credibility of the target field for each valid field in the first supervision data.
TABLE 4 target field confidence for each valid field in the first supervisory data
And S130, merging the first supervision data according to the credibility of the data, and determining the target supervision data of the target object.
Specifically, the credibility of each piece of data is compared, the first supervision data with the highest data credibility is used as target supervision data, and the supervision field with empty field value information in the target supervision data can be supplemented based on other first supervision data, so that more real and reliable complete supervision data can be obtained, and the accuracy of data merging is ensured.
Illustratively, S130 may include: taking the first supervision data corresponding to the highest data reliability as target supervision data, and detecting whether a first field with null field value information exists in the target supervision data; if yes, determining first field value information corresponding to the first field according to other first supervision data except the target supervision data, and adding the first field value information to the target supervision data.
Specifically, whether a first field with null field value information exists in first supervision data (namely target supervision data) with highest credibility is detected; if not, the current target supervision data is a complete supervision data, data supplement is not needed, and the current target supervision data can be directly used as final supervision data of the target object obtained after data merging; if yes, the first field value information corresponding to each first field can be determined based on other first supervision data except the target supervision data, and each first field value information is added into the target supervision data, so that the target supervision data is complete supervision data, and the integrity of data merging is guaranteed.
For example, determining first field value information corresponding to the first field according to other first supervision data except the target supervision data may include: determining each target first supervision data in which the value information of the first field is stored, among other first supervision data except the target supervision data; and determining first field value information corresponding to the first field according to the data credibility corresponding to the first supervision data of each target.
Specifically, in other first supervision data except for the target supervision data, whether non-null value information corresponding to the first field is stored in each first supervision data is detected, and if yes, the first supervision data is determined to be the target first supervision data, so that each target first supervision data in the other first supervision data can be obtained. By comparing the data reliability of each target first supervision data, the target first supervision data with the highest data reliability can be obtained. If only one target first supervision data with the highest data credibility exists, the non-null value information corresponding to the first field in the target first supervision data can be directly determined as the first field value information corresponding to the first field. If at least two target first supervision data with the highest data credibility exist, the non-empty field value information corresponding to the first field in the first supervision data obtained latest can be determined as the first field value information corresponding to the first field based on the acquisition time of each target first supervision data with the highest data credibility, so that the first field value information with the highest credibility and the latest information can be obtained in a merged manner, and the accuracy of the data is further ensured.
Illustratively, after the two pieces of first supervision data given in table 4 are merged, the target supervision data of law enforcement officer Zhao is obtained as shown in table 5.
TABLE 5 Law Enforcement supervision data for Zhang III
According to the technical scheme of the embodiment, first supervision data of target objects supervised by at least two first groups of mechanisms are determined; determining the data reliability corresponding to each first supervision data according to field weight information and field source reliability information in a preset rule configuration file; the first supervision data are merged according to the credibility of the data, so that the data can be merged based on the reliability of a data source, the accuracy of merging the data is ensured, and a preset rule configuration file can be preset based on business requirements so as to be merged based on the preset rule configuration file, so that the flexibility of merging the data is improved, and the personalized requirements of users are met.
On the basis of the above technical solution, after S130, the method may further include: and allocating a filing identifier to the target supervision data, adding the filing identifier, the reliability of the target field corresponding to each effective field and the reliability of the data to the corresponding first supervision data, and storing each added first supervision data.
The archiving identification can be composed of at least one of numbers, letters and symbols to distinguish different target supervision data, and each first supervision data before merging processing corresponding to the target supervision data can be linked based on the archiving identification, so that data query is facilitated. For example, the archive identification may be an incremental value of a preset length, such as 1 increment every allocation, starting at 00000001.
Specifically, a new archive identifier may be assigned to the target supervisory data of the target object based on the currently assigned archive identifier to uniquely identify the target supervisory data. The embodiment may add the allocated archive identifier to the target supervision data for data tracing. For example, table 6 shows target supervision data after adding an archive identifier.
Table 6 target supervision data with added filing identification
The embodiment can add the filing identification corresponding to the target supervision data, the target field reliability corresponding to each effective field in the first supervision data and the data reliability corresponding to the first supervision data in each first supervision data for merging processing, and archive and store the added first supervision data, so that a user can conveniently check the whole merging process, data tracing is realized, error correction is facilitated, and function diversification is realized. Illustratively, Table 7 sets forth first regulatory data for an archive deposit.
Table 7 first supervision data filed and stored
Example two
Fig. 2 is a flowchart of a data processing method according to a second embodiment of the present invention, and this embodiment optimizes the step of determining the first supervision data of the at least two first groups of target objects to be supervised by the organization based on the above embodiment. Wherein explanations of the same or corresponding terms as those of the above-described embodiments are omitted.
Referring to fig. 2, the data processing method provided in this embodiment specifically includes the following steps:
s210, if it is detected that second supervision data of the target object obtained by supervision of at least two second organization groups exist in the first organization, determining priority information of a second data source organization corresponding to each second supervision data according to a preset priority configuration file.
Wherein the first organization mechanism is a superior mechanism of the second organization mechanism. The number of second organisation mechanisms under the first set of organisation may be plural. For example, when the first organization is a judicial hall of a certain province, the second organization under the first organization may be a judicial office of each city in the province. The second supervision data may refer to supervision data of the target object reported by the second group of organizations. The preset priority profile may be preset based on business needs and scenarios in order to increase flexibility of data processing. The preset priority profile may contain priority information for each second organization to characterize the importance of each second organization. For example, the priority information may be characterized numerically, such as the higher the number, the lower the priority level. The second data source authority may refer to a second organization that reports second regulatory data.
Specifically, when the data processing apparatus receives second supervision data of the same target object reported by at least two second groups of mechanisms under any one first group of mechanisms, that is, the first group of mechanisms corresponds to at least two pieces of second supervision data, all the second supervision data need to be merged to determine the first supervision data corresponding to the first group of mechanisms, and at this time, priority information corresponding to each second data source mechanism can be obtained from a preset priority configuration file according to an mechanism identifier of the second data source mechanism corresponding to each second supervision data.
S220, merging the second supervision data according to the priority information, and determining first supervision data of a target object supervised by the first group of mechanisms.
Specifically, according to the priority information of each second data source mechanism, second supervision data with the highest priority can be determined, the second supervision data with the highest priority is used as first supervision data of a target object, and whether a second field with null field value information exists in the first supervision data or not is detected; if so, determining second field value information corresponding to the second field according to other second supervision data except the first supervision data, and adding the second field value information to the first supervision data, so that merging operation of the second supervision data is realized, and the first supervision data of the target object supervised by the first group of organizations is obtained.
For example, determining second field value information corresponding to the second field according to other second supervision data except the first supervision data may include: determining each target second supervision data in which the value information of the second field is stored, among other second supervision data except the first supervision data; and determining second field value information corresponding to the second field according to the priority information corresponding to the second supervision data of each target.
Specifically, in other second supervision data except the first supervision data, whether non-null value information corresponding to the second field is stored in each second supervision data is detected, and if yes, the second supervision data is determined as target second supervision data, so that each target second supervision data in the other second supervision data can be obtained. By comparing the priority information of each target second supervision data, the target second supervision data with the highest priority can be obtained. If only one target second supervision data with the highest priority exists, the non-null value information corresponding to the second field in the target second supervision data can be directly determined as the second field value information corresponding to the second field. If at least two target second supervision data with the highest priority exist, the non-null field value information corresponding to the second field in the second supervision data obtained latest can be determined as the second field value information corresponding to the second field based on the acquisition time of each target second supervision data with the highest priority, so that the second field value information with the highest priority and the latest second field value information can be merged, and the accuracy of the data is further ensured.
And S230, determining the data credibility corresponding to each first supervision data according to the field weight information and the field source credibility information in the preset rule configuration file.
S240, merging the first supervision data according to the credibility of the data, and determining the target supervision data of the target object.
According to the technical scheme of this embodiment, when it is detected that there are at least two second supervision data of the target object supervised by the second organization, according to the preset priority configuration file, the priority information of the second data source organization corresponding to each second supervision data may be determined, and each second supervision data is merged based on each priority information, so that merging processing may be performed based on the priority of the second organization, and the first supervision data of the target object supervised by the first organization may be determined, so that merging processing may be performed subsequently on the first supervision data of each different first organization.
The following is an embodiment of a data processing apparatus according to an embodiment of the present invention, which belongs to the same inventive concept as the data processing methods of the above embodiments, and reference may be made to the above embodiments of the data processing method for details that are not described in detail in the embodiments of the data processing apparatus.
EXAMPLE III
Fig. 3 is a schematic structural diagram of a data processing apparatus according to a third embodiment of the present invention, where the present embodiment is applicable to a case of merging and processing supervision data of each lower organization, and the apparatus specifically includes: a first supervisory data determination module 310, a data trustworthiness determination module 320, and a data merge processing module 330.
Wherein, the first supervision data determining module 310 is configured to determine first supervision data of at least two target objects supervised by a first group of organizations; the data reliability determining module 320 is configured to determine, according to field weight information and field source reliability information in a preset rule configuration file, a data reliability corresponding to each first supervision data; and the data merging processing module 330 is configured to merge the first supervision data according to the data credibility to determine target supervision data of the target object.
Optionally, the first supervision data determining module 310 is specifically configured to:
if second supervision data of the target object, which are obtained by supervision of at least two second organization groups, are detected to exist in the first organization, the priority information of a second data source organization corresponding to each second supervision data is determined according to a preset priority configuration file;
merging each second supervision data according to each priority information, and determining first supervision data of a target object supervised by a first group of mechanisms;
wherein the first organization mechanism is a superior mechanism of the second organization mechanism.
Optionally, data credibility determination module 320 includes:
the effective field acquisition unit is used for acquiring each effective field in each piece of first supervision data and a first data source mechanism corresponding to the first supervision data;
the target field credibility determining unit is used for determining the credibility of the target field corresponding to each effective field according to the first data source mechanism, each effective field, and the field weight information and the field source credibility information in the preset rule configuration file;
and the data reliability determining unit is used for determining the data reliability corresponding to the first supervision data according to the reliability of each target field.
Optionally, the target field reliability determining unit is specifically configured to:
determining the target weight corresponding to each effective field according to field weight information in a preset rule configuration file; determining the target credibility corresponding to each effective field according to field source credibility information in a preset rule configuration file and a first data source mechanism; and determining the target field reliability corresponding to each effective field according to the target weight and the target reliability corresponding to each effective field.
Optionally, the data merging processing module 330 includes:
the field value information detection unit is used for taking the first supervision data corresponding to the highest data reliability as target supervision data and detecting whether a first field with empty field value information exists in the target supervision data;
and the first field value information determining unit is used for determining first field value information corresponding to the first field according to other first supervision data except the target supervision data if the first field value information is determined, and adding the first field value information to the target supervision data.
Optionally, the first field value information determining unit is specifically configured to:
determining each target first supervision data in which the value information of the first field is stored, among other first supervision data except the target supervision data; and determining first field value information corresponding to the first field according to the data credibility corresponding to the first supervision data of each target.
Optionally, the apparatus further comprises:
and the filing identifier adding module is used for distributing a filing identifier to the target supervision data after the target supervision data of the target object is determined, adding the filing identifier, the reliability of the target field corresponding to each effective field and the reliability of the data to the corresponding first supervision data, and storing each added first supervision data.
The data processing device provided by the embodiment of the invention can execute the data processing method provided by any embodiment of the invention, and has the corresponding functional module and the beneficial effect of executing the data processing method.
It should be noted that, in the embodiment of the data processing apparatus, the modules and units included in the embodiment are only divided according to functional logic, but are not limited to the above division as long as the corresponding functions can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
Example four
Fig. 4 is a schematic structural diagram of an apparatus according to a fourth embodiment of the present invention. Referring to fig. 4, the apparatus includes:
one or more processors 410;
a memory 420 for storing one or more programs;
when the one or more programs are executed by the one or more processors 410, the one or more processors 410 are caused to implement a data processing method as provided in any of the embodiments above, the method comprising:
determining first supervision data of at least two target objects supervised by a first group of authorities;
determining the data reliability corresponding to each first supervision data according to field weight information and field source reliability information in a preset rule configuration file;
and merging the first supervision data according to the credibility of the data to determine the target supervision data of the target object.
In FIG. 4, a processor 410 is illustrated as an example; the processor 410 and the memory 420 in the device may be connected by a bus or other means, as exemplified by the bus connection in fig. 4.
The memory 420 serves as a computer-readable storage medium, and may be used for storing software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the data processing method in the embodiment of the present invention (for example, the first supervision data determination module 310, the data credibility determination module 320, and the data merging processing module 330 in the data processing apparatus). The processor 410 executes various functional applications of the device and data processing, i.e., implements the above-described data processing method, by executing software programs, instructions, and modules stored in the memory 420.
The memory 420 mainly includes a program storage area and a data storage area, wherein the program storage area can store an operating system and an application program required by at least one function; the storage data area may store data created according to use of the device, and the like. Further, the memory 420 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, memory 420 may further include memory located remotely from processor 410, which may be connected to devices through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The apparatus proposed in this embodiment is the same as the data processing method proposed in the above embodiment, and the technical details that are not described in detail in this embodiment can be referred to the above embodiment, and this embodiment has the same advantageous effects as the data processing method.
EXAMPLE five
The present embodiment provides a computer-readable storage medium on which a computer program is stored, the program, when executed by a processor, implementing a data processing method as provided in any embodiment of the present invention, the method comprising:
determining first supervision data of at least two target objects supervised by a first group of authorities;
determining the data reliability corresponding to each first supervision data according to field weight information and field source reliability information in a preset rule configuration file;
and merging the first supervision data according to the credibility of the data to determine the target supervision data of the target object.
Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer-readable storage medium may be, for example but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It will be understood by those skilled in the art that the modules or steps of the invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of computing devices, and optionally they may be implemented by program code executable by a computing device, such that it may be stored in a memory device and executed by a computing device, or it may be separately fabricated into various integrated circuit modules, or it may be fabricated by fabricating a plurality of modules or steps thereof into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A data processing method, comprising:
determining first supervision data of at least two target objects supervised by a first group of authorities;
determining the data reliability corresponding to each first supervision data according to field weight information and field source reliability information in a preset rule configuration file;
and merging the first supervision data according to the data credibility to determine the target supervision data of the target object.
2. The method of claim 1, wherein determining first regulatory data for at least two first set of regulatory target objects comprises:
if second supervision data of a target object, which are obtained by supervision of at least two second organization groups, are detected to exist in the first organization, determining priority information of a second data source organization corresponding to each second supervision data according to a preset priority configuration file;
merging each second supervision data according to each priority information to determine first supervision data of a target object supervised by the first group of mechanisms;
wherein the first organization mechanism is a superior mechanism of the second organization mechanism.
3. The method of claim 1, wherein determining the data reliability corresponding to each of the first supervision data according to field weight information and field source reliability information in a preset rule configuration file comprises:
obtaining each effective field in each first supervision data and a first data source mechanism corresponding to the first supervision data;
determining the credibility of the target field corresponding to each effective field according to the first data source mechanism, each effective field, and field weight information and field source credibility information in a preset rule configuration file;
and determining the data reliability corresponding to the first supervision data according to the reliability of each target field.
4. The method of claim 3, wherein determining the credibility of the target field corresponding to each valid field according to the first data source organization, each valid field, and field weight information and field source credibility information in a preset rule configuration file comprises:
determining a target weight corresponding to each effective field according to field weight information in a preset rule configuration file;
determining a target credibility corresponding to each effective field according to field source credibility information in the preset rule configuration file and the first data source mechanism;
and determining the reliability of the target field corresponding to each effective field according to the target weight and the target reliability corresponding to each effective field.
5. The method of claim 1, wherein merging each of the first supervisory data according to each of the data credibility to determine target supervisory data for the target object comprises:
taking first supervision data corresponding to the highest data reliability as target supervision data, and detecting whether a first field with null field value information exists in the target supervision data;
if yes, determining first field value information corresponding to the first field according to other first supervision data except the target supervision data, and adding the first field value information to the target supervision data.
6. The method of claim 5, wherein determining, according to other first supervision data except the target supervision data, first field value information corresponding to the first field comprises:
determining each target first supervision data in which the value information of the first field is stored, among other first supervision data except the target supervision data;
and determining first field value information corresponding to the first field according to the data credibility corresponding to each target first supervision data.
7. The method of claim 3, after determining the target regulatory data for the target object, further comprising:
and allocating a filing identifier to the target supervision data, adding the filing identifier, the reliability of the target field corresponding to each effective field and the reliability of the data to corresponding first supervision data, and storing each added first supervision data.
8. A data processing apparatus, comprising:
a first supervision data determining module for determining first supervision data of at least two first groups of target objects supervised by the organization;
the data reliability determining module is used for determining the data reliability corresponding to each first supervision data according to field weight information and field source reliability information in a preset rule configuration file;
and the data merging processing module is used for merging each first supervision data according to each data credibility and determining the target supervision data of the target object.
9. An apparatus, characterized in that the apparatus comprises:
one or more processors;
a memory for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement a data processing method as claimed in any one of claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the data processing method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010031154.2A CN111258981A (en) | 2020-01-13 | 2020-01-13 | Data processing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010031154.2A CN111258981A (en) | 2020-01-13 | 2020-01-13 | Data processing method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111258981A true CN111258981A (en) | 2020-06-09 |
Family
ID=70952105
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010031154.2A Pending CN111258981A (en) | 2020-01-13 | 2020-01-13 | Data processing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111258981A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104717078A (en) * | 2013-12-13 | 2015-06-17 | 贵州电网公司信息通信分公司 | Configuration-based data merging method |
| WO2016004813A1 (en) * | 2014-07-07 | 2016-01-14 | 阿里巴巴集团控股有限公司 | Data storage method, query method and device |
| CN107784111A (en) * | 2017-11-06 | 2018-03-09 | 北京锐安科技有限公司 | Data digging method, device, equipment and storage medium |
| CN107844402A (en) * | 2017-11-17 | 2018-03-27 | 北京联想超融合科技有限公司 | A kind of resource monitoring method, device and terminal based on super fusion storage system |
| CN108255788A (en) * | 2016-12-27 | 2018-07-06 | 方正国际软件(北京)有限公司 | A kind of method and device for the confidence level for assessing data |
| CA2996960A1 (en) * | 2017-03-02 | 2018-09-02 | Discovered Intelligence Inc. | System for aggregation and prioritization of it asset field values from real-time event logs and method thereof |
| CN109783490A (en) * | 2018-12-25 | 2019-05-21 | 杭州数梦工场科技有限公司 | Data fusion method, device, computer equipment and storage medium |
| CN110276060A (en) * | 2019-06-24 | 2019-09-24 | 北京明略软件系统有限公司 | The method and device of data processing |
| CN110347683A (en) * | 2019-06-27 | 2019-10-18 | 北京明略软件系统有限公司 | A kind of tables of data merging treatment method and device |
| CN110472216A (en) * | 2019-08-19 | 2019-11-19 | 北京明略软件系统有限公司 | Determination method, apparatus, storage medium and the electronic device of field |
-
2020
- 2020-01-13 CN CN202010031154.2A patent/CN111258981A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104717078A (en) * | 2013-12-13 | 2015-06-17 | 贵州电网公司信息通信分公司 | Configuration-based data merging method |
| WO2016004813A1 (en) * | 2014-07-07 | 2016-01-14 | 阿里巴巴集团控股有限公司 | Data storage method, query method and device |
| CN108255788A (en) * | 2016-12-27 | 2018-07-06 | 方正国际软件(北京)有限公司 | A kind of method and device for the confidence level for assessing data |
| CA2996960A1 (en) * | 2017-03-02 | 2018-09-02 | Discovered Intelligence Inc. | System for aggregation and prioritization of it asset field values from real-time event logs and method thereof |
| CN107784111A (en) * | 2017-11-06 | 2018-03-09 | 北京锐安科技有限公司 | Data digging method, device, equipment and storage medium |
| CN107844402A (en) * | 2017-11-17 | 2018-03-27 | 北京联想超融合科技有限公司 | A kind of resource monitoring method, device and terminal based on super fusion storage system |
| CN109783490A (en) * | 2018-12-25 | 2019-05-21 | 杭州数梦工场科技有限公司 | Data fusion method, device, computer equipment and storage medium |
| CN110276060A (en) * | 2019-06-24 | 2019-09-24 | 北京明略软件系统有限公司 | The method and device of data processing |
| CN110347683A (en) * | 2019-06-27 | 2019-10-18 | 北京明略软件系统有限公司 | A kind of tables of data merging treatment method and device |
| CN110472216A (en) * | 2019-08-19 | 2019-11-19 | 北京明略软件系统有限公司 | Determination method, apparatus, storage medium and the electronic device of field |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108595157B (en) | Block chain data processing method, device, equipment and storage medium | |
| JP6716727B2 (en) | Streaming data distributed processing method and apparatus | |
| CN107358116B (en) | A kind of method for secret protection in multi-sensitive attributes data publication | |
| CN109086182A (en) | The method and terminal device of database auto-alarming | |
| CN105549508A (en) | Alarm method based on information combination and apparatus thereof | |
| CN111680104B (en) | Data synchronization method, device, computer equipment and readable storage medium | |
| CN111010282A (en) | Information processing method based on block chain and related device | |
| US8396877B2 (en) | Method and apparatus for generating a fused view of one or more people | |
| CN110825919B (en) | ID data processing method and device | |
| CN111258981A (en) | Data processing method, device, equipment and storage medium | |
| CN112433753A (en) | Interface document generation method, device, equipment and medium based on parameter information | |
| CN109428906B (en) | Request processing method, device, system and terminal | |
| CN113065323B (en) | Method, device and equipment for importing contract template data and storage medium | |
| CN110457332B (en) | Information processing method and related equipment | |
| CN110708414B (en) | Telephone number sorting method and device and electronic equipment | |
| CN114238349A (en) | Data verification method, device, equipment and medium | |
| CN111639057A (en) | Log message processing method and device, computer equipment and storage medium | |
| CN112039893A (en) | Private transaction processing method and device, electronic equipment and readable storage medium | |
| CN111611056A (en) | Data processing method and device, computer equipment and storage medium | |
| CN111221847A (en) | Monitoring data storage method and device and computer readable storage medium | |
| CN108932305A (en) | A kind of data processing method, device, electronic equipment and storage medium | |
| CN112883038B (en) | Data management method based on block chain, computer and readable storage medium | |
| CN115102920B (en) | Individual transmission and management control method based on relational network | |
| US10261945B1 (en) | Systems and methods for storing and accessing monitoring data | |
| CN103413224A (en) | Standardization and aggregation system with a plurality of data sources |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220927 Address after: 25 Financial Street, Xicheng District, Beijing 100033 Applicant after: CHINA CONSTRUCTION BANK Corp. Address before: 25 Financial Street, Xicheng District, Beijing 100033 Applicant before: CHINA CONSTRUCTION BANK Corp. Applicant before: Jianxin Financial Science and Technology Co.,Ltd. |
|
| TA01 | Transfer of patent application right |