CN110704476A

CN110704476A - Data processing method, device, equipment and storage medium

Info

Publication number: CN110704476A
Application number: CN201910951264.8A
Authority: CN
Inventors: 武新明; 程强; 万月亮
Original assignee: Beijing Ruian Technology Co Ltd
Current assignee: Beijing Ruian Technology Co Ltd
Priority date: 2019-10-08
Filing date: 2019-10-08
Publication date: 2020-01-17

Abstract

The embodiment of the invention discloses a data processing method, a data processing device, data processing equipment and a storage medium. The method comprises the following steps: responding to a query instruction acting in a target interface, and determining at least two target attributes to be queried; determining the attribute to be queried and the key value of the attribute to be queried according to the query information in the query instruction; according to the mapping relation between the user and the known attribute information, the user with the attribute to be inquired and the key value is taken as a target user, and the target attribute value of the target user is obtained from the mapping relation; the mapping relation between the user and the known attribute information is generated according to the historical query record of the user, so that the data information and the logic relation between the data can be rapidly and comprehensively displayed.

Description

Data processing method, device, equipment and storage medium

Technical Field

Embodiments of the present invention relate to data processing technologies, and in particular, to a data processing method, an apparatus, a device, and a storage medium.

Background

At present, big data information is widely applied in the aspect of life.

When a web application is built or a business process of the web application is tested, background data is often required to support, particularly, at present, background data storage is diversified, and various data such as relational data mysql, oracle, non-relational data hbase and the like are used.

Under the circumstance, a data extraction method is urgently needed, a plurality of databases are not needed to be built, and data information and the logic relation among data can be rapidly and comprehensively shown.

Disclosure of Invention

The invention provides a data processing method, a data processing device, data processing equipment and a storage medium, which are used for rapidly and comprehensively showing data information and logic relation among data.

In a first aspect, an embodiment of the present invention provides a data processing method, including:

responding to a query instruction acting in a target interface, and determining at least two target attributes to be queried;

determining the attribute to be queried and the key value of the attribute to be queried according to the query information in the query instruction;

according to the mapping relation between the user and the known attribute information, the user with the attribute to be inquired and the key value is taken as a target user, and the target attribute value of the target user is obtained from the mapping relation;

and generating a mapping relation between the user and the known attribute information according to the historical query record of the user.

In a second aspect, an embodiment of the present invention further provides a data processing apparatus, including:

the target attribute determining module is used for responding to a query instruction acting in the target interface and determining at least two target attributes to be queried;

the attribute to be queried determining module is used for determining the attribute to be queried and the key value of the attribute to be queried according to the query information in the query instruction;

a target attribute value obtaining module, configured to take a user with an attribute to be queried, which includes the key value, as a target user according to a mapping relationship between the user and known attribute information, and obtain a target attribute value of the target user from the mapping relationship;

In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the data processing method according to any one of the embodiments of the present invention when executing the program.

In a fourth aspect, the embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the data processing method according to any one of the embodiments of the present invention.

The method comprises the steps of determining at least two target attributes to be queried by responding to a query instruction acting in a target interface; determining the attribute to be queried and the key value of the attribute to be queried according to the query information in the query instruction; according to the mapping relation between the user and the known attribute information, the user with the attribute to be inquired and the key value is taken as a target user, and the target attribute value of the target user is obtained from the mapping relation; the mapping relation between the user and the known attribute information is generated according to the historical query record of the user, so that the data information and the logic relation between the data can be rapidly and comprehensively displayed.

Drawings

Fig. 1 is a schematic flow chart of a data processing method according to an embodiment of the present invention;

fig. 2a is a schematic flowchart of a data processing method according to a second embodiment of the present invention;

fig. 2b is a schematic structural diagram of establishing a mapping relationship between a user and at least two pieces of known attribute information according to a second embodiment of the present invention;

fig. 3 is a schematic structural diagram of a data processing apparatus according to a third embodiment of the present invention;

fig. 4 is a schematic structural diagram of an apparatus according to a fourth embodiment of the present invention.

Detailed Description

The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.

Before discussing exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart may describe the steps as a sequential process, many of the steps can be performed in parallel, concurrently or simultaneously. In addition, the order of the steps may be rearranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figure. The processes may correspond to methods, functions, procedures, subroutines, and the like.

Example one

Fig. 1 is a schematic flow chart of a data processing method according to an embodiment of the present invention, where the present embodiment is applicable to a case of demonstrating functions of a system or for developing data required for comprehensive functions, and the method may be executed by a data processing apparatus, where the apparatus may be implemented in a software and/or hardware manner, and may be integrated in an electronic device, and specifically includes the following steps:

step 110, in response to a query instruction acting in the target interface, at least two target attributes to be queried are determined.

In this embodiment, the target interface is an input interface of the Web, and a user can access an application program to be accessed through a browser without installing other software. The method is realized by accessing the application program needing to be accessed through the Web application program. The application program has two modes C/S, B/S. C/S are client/server side programs, meaning that such programs generally run independently. And B/S is a browser-side/server-side application, which is generally run by means of a browser such as IE. Web applications are typically B/S mode. Web applications are first "applications" and are not substantially different from programs written in standard programming languages, such as C, C + +, and the like. However, Web applications have their own unique place in that they are Web-based, rather than operating using traditional methods.

In this embodiment, the target attribute refers to information that the user wants to obtain, and exemplarily, the target attribute includes, but is not limited to, an identity card number, a home address, a mobile phone number, and a user mailbox.

In this embodiment, a user inputs a query instruction through an input interface of the Web, and the query instruction returns information required by the user to the user, so that the user can view and use the information.

And step 120, determining the attribute to be queried and the key value of the attribute to be queried according to the query information in the query instruction.

In this embodiment, the attribute to be queried includes, but is not limited to, an identity card number, a home address, a mobile phone number, and a user mailbox, and when the attribute to be queried is a mobile phone number, a key value of the attribute to be queried may be the first three digits of the mobile phone number or the last four digits of the mobile phone number.

Step 130, according to the mapping relationship between the user and the known attribute information, taking the user with the attribute to be queried including the key value as a target user, and obtaining the target attribute value of the target user from the mapping relationship.

In this embodiment, when the known attributes of the user include a mobile phone number, a home address, an identity card number, and a user mailbox, and it is determined that the attribute to be queried is the mobile phone number according to the query information, and the key value in the attribute to be queried is a user with the first three digits of the mobile phone number being 136, all users with the first three digits of the mobile phone number being 136 are taken as target users. Illustratively, the mobile phone number of user a is 13622 × and the mobile phone number of user B is 13633 × so that user a and user B are both target users. Further, when the target attribute is the identity card number, the identity card number of the user A and the identity card number of the user B can be determined by determining that the user A and the user B are target users and the attribute information that the user A and the user B are bound with.

Optionally, before the user with the target attribute included in the known attribute is taken as the target user according to the mapping relationship between the user and the known attribute information, the method further includes:

respectively extracting at least two pieces of known attribute information of a user from at least two historical query records of the user;

and establishing a mapping relation between the user and the at least two pieces of known attribute information.

In this embodiment, extracting at least two pieces of known attribute information of a user through a history query record means that when the mobile phone numbers of the user are queried in an input interface of the Web for the first time, the mobile phone numbers of a plurality of users are returned to the input interface, and then the mobile phone numbers of different users are classified according to user IDs. And when the identity card numbers of the users are inquired in the input interface of the Web for the second time, returning to the input interface of the identity card numbers of the users, and then classifying the identity card numbers of different users according to the user IDs. The method can be used for inquiring different attribute information, so that a plurality of different attribute information of different users can be bound, the attribute information is the known attribute information of the users, and the mapping relation between the users and at least two pieces of known attribute information can be established.

According to the technical scheme of the embodiment of the invention, at least two target attributes to be queried are determined by responding to a query instruction acting in a target interface; determining the attribute to be queried and the key value of the attribute to be queried according to the query information in the query instruction; according to the mapping relation between the user and the known attribute information, the user with the attribute to be inquired and the key value is taken as a target user, and the target attribute value of the target user is obtained from the mapping relation; and various databases do not need to be built, and the data information and the logical relationship among the data can be rapidly and comprehensively displayed.

Example two

Fig. 2 is a flowchart of a data processing method according to a second embodiment of the present invention, where this embodiment is applicable to a case where functions of a presentation system or data required for developing the functions are relatively comprehensive, and the method may be executed by a data processing apparatus, and the apparatus may be implemented in a software and/or hardware manner, and may be integrated in an electronic device, and specifically includes the following steps:

step 210, in response to a query instruction acting in the target interface, determining at least two target attributes to be queried.

Step 220, respectively extracting at least two pieces of known attribute information of the user from at least two historical query records of the user; and establishing a mapping relation between the user and the at least two pieces of known attribute information.

In this embodiment, before the web application queries the target attribute, that is, before the web application calls the data service interface, a set of rest interface services, that is, a swagger interface program, is constructed. Among them, the rest (representational statetransfer) interface, the representation of the resource and the state conversion. The URL is used to locate the resource and the operation is described by an http verb. Which is used to specify how the client interacts data with the API provider at the http level.

When the known attribute information of the user is acquired, the specific operation process is as follows:

firstly, a query request is sent to a database, a swagger interface program forwards the query request, query request parameters are analyzed according to forwarding service, and queried known attribute information is stored in a MongoDB, wherein the MongoDB is a database based on distributed file storage and is written in C + + language, and the MongoDB aims to provide an extensible high-performance data storage solution for web application.

MongoDB is a product between relational databases and non-relational databases, and among the non-relational databases, the MongoDB has the most abundant functions and is most similar to the relational databases. The data structure supported by the method is very loose and is in a json-like bson format, so that more complex data types can be stored. Because MongoDB only stores the json format, the query request in the process is converted into the json format if the query request is not in the json format.

When the query request calls the data service to return a result, the corresponding relation between the request parameter and the url and the returned result is stored in the MongoDB in a json format, namely the known attribute information of the user and the mapping relation between the user and the at least two pieces of known attribute information are stored in the MongoDB.

In this embodiment, optionally, after extracting at least two pieces of known attribute information of the user from at least two historical query records of the user, the method further includes:

and if any one of the target attributes is absent in the known attributes of any user, refusing to establish the mapping relation of the user.

For example, if the mapping relationship between the user a and the mobile phone number, the home address and the identity card number can be known in the mongoDB, when the target attribute is the identity card number of the user a, the identity card number of the user a cannot be searched, and the mapping relationship between the user a and the mobile phone number, the home address and the identity card number is refused to be established in the MongoDB. The setting has the advantages that when the attribute information of the user is searched in the MongoDB, the more the attribute information bound by the user is, the higher the information value of the user is and the more comprehensive the information is due to the requirement of a service scene.

Step 230, selecting the attribute to be queried from the known attribute information; and constructing an index field of the attribute to be queried, and providing data query service of the attribute to be queried.

In this embodiment, the index field of the attribute to be queried refers to a search mode capable of finding the attribute to be queried, and for example, the attribute to be queried is an id number, and the index field of the attribute to be queried may be a field where the id number belongs to shanxi. Or, when the attribute to be queried is the id number, the index field of the attribute to be queried may be a field in which the last bit of the id number is X.

In this embodiment, the obtained known attributes are analyzed, and the index field of each known attribute information is constructed, so that each known attribute information can be found when being used as an attribute to be queried. Specifically, refer to a schematic structural diagram of fig. 2b, which illustrates a mapping relationship between a user and at least two pieces of known attribute information.

After known attribute information is analyzed in the MongoDB, an index field constructed by the known attribute information is stored in the solr, and a schema file in the solr carries out service scene configuration on the index field of the known attribute information in the MongoDB.

When the known attribute information is inquired in the MongoDB, a connector plug-in is used for establishing an index in the solr in an incremental mode by taking the known attribute information in the MongoDB as a source and taking the shema file of the solr as an index condition. Further, if the known attribute information in the MongoDB is deleted, the index in the solr is also deleted accordingly. The method has the advantages that under the condition that no business application is built, the known attribute information in the MongoDB can still be inquired by using the solr interface to build the indexed field, and the integrity of the data and the corresponding relation between the business scene and the data are verified.

Step 240, determining the attribute to be queried and the key value of the attribute to be queried according to the query information in the query instruction, including: judging whether an index field in the query information is in a constructed index field or not according to the query information in the query instruction; and if so, determining the attribute to be queried and the key value of the attribute to be queried.

In this embodiment, for example, the index field in the query information is a field in which the identification number attribution is shanxi, and then whether the index field in which the identification number attribution is shanxi has been established is searched for in the constructed index field, and if so, it is determined that the attribute to be queried is the identification number and the first two bits of the identification number are the key value of the attribute to be queried is 14.

Under the condition that various known attribute information is inquired through an inquiry interface of the solr and inquiry can be carried out according to the established index field, a business application is simulated in the MongoDB to use api of restemplate to call an inquiry interface of a data provider, and swagger plug-in is used for providing rest interface service to the outside, so that a post or get request debugging, namely an inquiry instruction, can be conveniently sent through a browser interface, and each packaged inquiry interface related to the business can be checked.

And step 250, according to the mapping relationship between the user and the known attribute information, taking the user with the attribute to be queried including the key value as a target user, and acquiring the target attribute value of the target user from the mapping relationship.

In this embodiment, for example, if the target attribute of the target user is the mobile phone number of the shanxi user, the attribute to be queried is the identity card number, and the first two digits of the identity card number are 14 as the key value of the attribute to be queried, it is determined that the user meeting the condition is the target user, that is, the shanxi user, and the mobile phone number of the shanxi user is determined in the mapping relationship of the already attribute information of the shanxi user, and then the value of the target attribute is 11 digits of the mobile phone number.

According to the technical scheme, the swagger plug-in is used, the rest interface published outside the debugging web service can be checked through the browser, the debugging operation is simple, the test extraction can be repeated, the best effect is achieved, the data association is strong, too many databases are not required to be built, the data are stored and transmitted in a json format, the MongoDB and the solr are matched for use, the created indexes are synchronized in real-time increment, and the checking and the analysis are convenient.

EXAMPLE III

Fig. 3 is a schematic structural diagram of a data processing apparatus according to a third embodiment of the present invention. The data processing device provided by the embodiment of the invention can execute the data processing method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method. As shown in fig. 3, the apparatus includes:

a target attribute determining module 301, configured to determine at least two target attributes to be queried in response to a query instruction acting in a target interface;

a to-be-queried attribute determining module 302, configured to determine, according to query information in the query instruction, a to-be-queried attribute and a key value of the to-be-queried attribute;

a target attribute value obtaining module 303, configured to take a user with an attribute to be queried, which includes the key value, as a target user according to a mapping relationship between the user and known attribute information, and obtain a target attribute value of the target user from the mapping relationship;

The device further comprises:

the known attribute information extraction module is used for respectively extracting at least two pieces of known attribute information of the user from at least two historical query records of the user;

and the mapping relation establishing module is used for establishing the mapping relation between the user and the at least two pieces of known attribute information.

The device further comprises:

and the target attribute judging module is used for refusing to establish the mapping relation of the user if any one target attribute is lacked in the known attributes of any user.

The device, still include:

the attribute to be queried selecting module is used for selecting the attribute to be queried from the known attribute information;

and the index field construction module is used for constructing the index field of the attribute to be inquired and providing data inquiry service of the attribute to be inquired.

The to-be-queried attribute determining module 302 is further configured to determine, according to query information in the query instruction, whether an index field in the query information is in a constructed index field;

and if so, determining the attribute to be queried and the key value of the attribute to be queried.

It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the above-described apparatus may refer to the corresponding process in the foregoing method embodiment, and is not described herein again.

Example four

Fig. 4 is a schematic structural diagram of an apparatus according to a fourth embodiment of the present invention, and fig. 4 is a schematic structural diagram of an exemplary apparatus suitable for implementing the embodiment of the present invention. The device 12 shown in fig. 4 is only an example and should not bring any limitation to the function and scope of use of the embodiments of the present invention.

As shown in FIG. 4, device 12 is in the form of a general purpose computing device. The components of device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16.

Bus 18 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, such architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, micro-channel architecture (MAC) bus, enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

Device 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by device 12 and includes both volatile and nonvolatile media, removable and non-removable media.

The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)30 and/or cache memory 32. Device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 4, and commonly referred to as a "hard drive"). Although not shown in FIG. 4, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.

A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of embodiments described herein.

Device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), with one or more devices that enable a user to interact with device 12, and/or with any devices (e.g., network card, modem, etc.) that enable device 12 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 22. Also, the device 12 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet) via the network adapter 20. As shown in FIG. 4, the network adapter 20 communicates with the other modules of the device 12 via the bus 18. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

The processing unit 16 executes various functional applications and data processing by executing programs stored in the system memory 28, for example, to implement a data processing method provided by an embodiment of the present invention, including:

EXAMPLE five

An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program (or referred to as a computer-executable instruction) is stored, where the computer program, when executed by a processor, can implement the data processing method according to any of the embodiments described above, and the method includes:

and generating a mapping relation between the user and the known attribute information according to the historical query record of the user. Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for embodiments of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A data processing method, comprising:

2. The method according to claim 1, wherein before the user with the target attribute included in the known attribute is taken as the target user according to the mapping relationship between the user and the known attribute information, the method further comprises:

3. The method of claim 2, further comprising, after extracting at least two known attribute information of the user from at least two historical query records of the user, respectively:

4. The method of claim 2, wherein after establishing the mapping relationship between the user and the at least two known attribute information, further comprising:

selecting attributes to be queried from the known attribute information;

and constructing an index field of the attribute to be queried, and providing data query service of the attribute to be queried.

5. The method of claim 4, wherein determining the attribute to be queried and the key value of the attribute to be queried according to the query information in the query instruction comprises:

judging whether an index field in the query information is in a constructed index field or not according to the query information in the query instruction;

6. A data processing apparatus, comprising:

7. The apparatus of claim 6, further comprising:

8. The apparatus of claim 7, further comprising:

9. Computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the data processing method according to any of claims 1-5 when executing the program.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the data processing method of any one of claims 1 to 5.