CN110704486A - Data processing method, device, system, storage medium and server - Google Patents
Data processing method, device, system, storage medium and server Download PDFInfo
- Publication number
- CN110704486A CN110704486A CN201910923997.0A CN201910923997A CN110704486A CN 110704486 A CN110704486 A CN 110704486A CN 201910923997 A CN201910923997 A CN 201910923997A CN 110704486 A CN110704486 A CN 110704486A
- Authority
- CN
- China
- Prior art keywords
- information
- index
- data
- search
- data processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
Abstract
The embodiment of the invention provides a data processing method, a data processing device, a data processing system, a storage medium and a server. The method comprises the following steps: acquiring data information, wherein the data information comprises primary key information and content information, and the primary key information is used for identifying the data information; constructing an index according to the main key information and index information extracted from the content information, and storing the index to a search server; and storing the data information to a database. In the scheme of the embodiment of the invention, the constructed index is stored in the search server, the data information is stored in the database, and the index and the stored information are respectively stored, so that the index can be acquired from the search server and the data information of the main key information identification of the index can be acquired from the database during data search, and the hierarchical query of the index and the data information in the search process is realized, thereby improving the search efficiency and reducing the search time.
Description
[ technical field ] A method for producing a semiconductor device
The embodiment of the invention relates to the technical field of internet, in particular to a data processing method, a device, a system, a storage medium and a server.
[ background of the invention ]
At present, in the field of electronic commerce, a shopping website or an operator of mobile payment needs to process a large number of auditing tasks every day, and data information to be audited by the auditing tasks mainly aims at data information of transactions or accounts with risks.
Before an auditor of an operator audits data information, the data information needs to be searched from a database. In the prior art, the search of data information is usually based on the MySQL database, so the search process is indexed based on the MySQL SQL statement.
However, when the search is performed by using the MySQL SQL statement, if there is a joint search or the search condition is complicated, the search efficiency is low, and the search takes a long time.
[ summary of the invention ]
In view of this, embodiments of the present invention provide a data processing method, apparatus, system, storage medium, and server, which are used to improve search efficiency and reduce search time.
In one aspect, an embodiment of the present invention provides a data processing method, including:
acquiring data information, wherein the data information comprises primary key information and content information, and the primary key information is used for identifying the data information;
constructing an index according to the main key information and index information extracted from the content information, and storing the index to a search server;
and storing the data information to a database.
Optionally, the index information extracted from the content information includes index information of a specified type;
the constructing an index according to the primary key information and the index information extracted from the content information includes:
and constructing an index according to the main key information and the index information of the specified type extracted from the content information.
Optionally, the obtaining data information further includes:
receiving a data description message sent by a first client, wherein the data description message comprises the primary key information and description information;
the acquiring data information includes:
extracting description information of a specified type from the description information;
and generating the data information according to the primary key information and the description information of the specified type, wherein the content information comprises the description information of the specified type.
Optionally, the receiving the data description message sent by the first client includes:
and receiving the data description message sent by the first client through message middleware.
Optionally, the search server comprises an ES server, and the database comprises a MySQL database.
In another aspect, an embodiment of the present invention provides a data processing method, including:
receiving a search request sent by a second client, wherein the search request comprises search conditions;
according to the search condition, obtaining an index matched with the search condition from a search server;
acquiring data information of the primary key information identification from a database according to the indexed primary key information;
and sending the data information to the second client.
Optionally, the index includes primary key information and index information;
the obtaining of the index matched with the search condition from the search server according to the search condition comprises:
inquiring index information matched with the search condition from the search server;
determining an index including index information matching the search condition as an index matching the search condition.
Optionally, the search server comprises an ES server, and the database comprises a MySQL database.
In another aspect, an embodiment of the present invention provides a data processing apparatus, including:
the system comprises an acquisition module, a storage module and a display module, wherein the acquisition module is used for acquiring data information, the data information comprises primary key information and content information, and the primary key information is used for identifying the data information;
the index component module is used for constructing an index according to the main key information and the index information extracted from the content information and storing the index to a search server;
and the database component module is used for storing the data information to a database.
Optionally, the index information extracted from the content information includes index information of a specified type;
the index component module is specifically configured to construct an index according to the primary key information and index information of a specified type extracted from the content information, and store the index to a search server.
Optionally, the method further comprises:
the receiving module is used for receiving a data description message sent by a first client, wherein the data description message comprises the primary key information and description information;
the acquisition module is specifically used for extracting the description information of the specified type from the description information; and generating the data information according to the primary key information and the description information of the specified type, wherein the content information comprises the description information of the specified type.
Optionally, the search server comprises an ES server, and the database comprises a MySQL database.
In another aspect, an embodiment of the present invention provides a data processing apparatus, including:
the receiving module is used for receiving a search request sent by a second client, and the search request comprises a search condition;
the index component module is used for acquiring indexes matched with the search conditions from a search server according to the search conditions;
the database component module is used for acquiring the data information of the primary key information identifier from a database according to the primary key information of the index;
and the sending module is used for sending the data information to the second client.
Optionally, the index includes primary key information and index information;
the index component module is specifically used for inquiring index information matched with the search condition from the search server; determining an index including index information matching the search condition as an index matching the search condition.
Optionally, the search server comprises an ES server, and the database comprises a MySQL database.
In another aspect, an embodiment of the present invention provides a data processing system, including: a data processing device, a search server and a database;
the data processing device is used for acquiring data information, wherein the data information comprises primary key information and content information, and the primary key information is used for identifying the data information; constructing an index according to the main key information and index information extracted from the content information, and storing the index to a search server; storing the data information to a database;
the search server is used for storing the index;
and the database is used for storing the data information.
In another aspect, an embodiment of the present invention provides a data processing system, including: the system comprises a data processing device, a search server, a database and a second client;
the data processing device is used for receiving a search request sent by a second client, and the search request comprises a search condition; according to the search condition, obtaining an index matched with the search condition from a search server; acquiring data information of the primary key information identification from a database according to the indexed primary key information; sending the data information to the second client;
and the second client is used for sending the search request and receiving the data information sent by the data processing device.
On the other hand, the embodiment of the present invention provides a storage medium, where the storage medium includes a stored program, and when the program runs, a device in which the storage medium is located is controlled to execute the data processing method.
On the other hand, the embodiment of the present invention provides a storage medium, where the storage medium includes a stored program, and when the program runs, a device in which the storage medium is located is controlled to execute the data processing method.
In another aspect, an embodiment of the present invention provides a server, including a memory and a processor, where the memory is used to store information including program instructions, and the processor is used to control execution of the program instructions, and the program instructions are loaded by the processor and executed to implement the steps of the data processing method.
In another aspect, an embodiment of the present invention provides a server, including a memory and a processor, where the memory is used to store information including program instructions, and the processor is used to control execution of the program instructions, and the program instructions are loaded by the processor and executed to implement the steps of the data processing method.
In the scheme of the embodiment of the invention, the constructed index is stored in the search server, the data information is stored in the database, and the index and the stored information are respectively stored, so that the index can be acquired from the search server and the data information of the main key information identification of the index can be acquired from the database during data search, and the hierarchical query of the index and the data information in the search process is realized, thereby improving the search efficiency and reducing the search time.
[ description of the drawings ]
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive labor.
FIG. 1 is a block diagram of a data processing system according to an embodiment of the present invention;
fig. 2 is a flowchart of a data processing method according to an embodiment of the present invention;
FIG. 3 is a flow chart of obtaining data information in an embodiment of the present invention;
FIG. 4 is a diagram illustrating a data processing method according to an embodiment of the present invention;
FIG. 5 is a flow chart of another data processing method according to an embodiment of the invention;
FIG. 6 is a diagram illustrating another data processing method according to an embodiment of the present invention;
FIG. 7 is a flow chart of index acquisition in an embodiment of the present invention;
fig. 8 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present invention;
FIG. 9 is a block diagram of another data processing apparatus according to an embodiment of the present invention;
FIG. 10 is a block diagram of a data processing system according to an embodiment of the present invention;
FIG. 11 is a block diagram of a data processing system according to an embodiment of the present invention;
fig. 12 is a schematic diagram of a server according to an embodiment of the present invention.
[ detailed description ] embodiments
For better understanding of the technical solutions of the present invention, the following detailed descriptions of the embodiments of the present invention are provided with reference to the accompanying drawings.
It should be understood that the described embodiments are only some embodiments of the invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terminology used in the embodiments of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the examples of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be understood that the term "and/or" as used herein is merely one type of associative relationship that describes an associated object, meaning that three types of relationships may exist, e.g., A and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.
Fig. 1 is a schematic structural framework diagram of a data processing system according to an embodiment of the present invention, as shown in fig. 1, the system may include: the system comprises a first client, a second client, a data processing device, an ES server and a MySQL database, wherein the data processing device is provided with an ES component module and a MySQL component module. The ES component module is connected with the ES server, and the data processing device can perform data interaction with the ES server through the ES component module, wherein the ES (elastic search) server is a search server based on Lucene. The MySQL component module is connected with the MySQL database, and the data processing device can perform data interaction between the MySQL component module and the MySQL database. As shown in fig. 1, the first client may be a client used by a customer, for example, the first client may include a mobile phone, a tablet computer, a notebook computer or a desktop computer; the second client may be a client used by an auditor of the operator, for example, the second client may include a cell phone, a tablet computer, a notebook computer, or a desktop computer.
Fig. 2 is a flowchart of a data processing method according to an embodiment of the present invention, where the data processing method shown in fig. 2 is executed by the data processing apparatus shown in fig. 1, and as shown in fig. 2, the method includes:
In the embodiment of the present invention, the data processing apparatus may acquire the data information from the data description message sent by the first client.
Fig. 3 is a flowchart of acquiring data information according to an embodiment of the present invention, as shown in fig. 3, before step 101, the method further includes:
Fig. 4 is a schematic diagram of a data processing method in an embodiment of the present invention, as shown in fig. 4, a user purchases a tea leaf on line on a shopping website through a first client, and after a period of time, the user finds that a merchant has a false shipment, and then applies for a refund on the shopping website through the first client. The user can fill in refund applying information on a designated webpage of a shopping website to carry out refund applying operation, the first client generates a data description message according to the refund applying information filled by the user, and the first client generates a piece of key information when generating a data description message, wherein the key information uniquely identifies the data description message and is a data ID. The data description message may include multiple types of description information, for example, the multiple types may include creation time, creator, APP version, location of the creator, merchant, transaction product, transaction amount, reporting reason, and reporting content, and accordingly, each type of description information may specifically include: the establishment time is 14:00, the creator is a user account, the APP version is 4.0, the location of the creator is Beijing, the merchant is an XX tea shop, the transaction product is tea, the transaction amount is 1, the transaction amount is 100 yuan, the reporting reason is false delivery, and the reporting content is application refund.
In step 100, the data processing apparatus may receive a data description message sent by a first client through message middleware. The message transmission is carried out through the message middleware, so that the reliability and the efficiency of data transmission can be improved.
After receiving the data description message in step 100, the data processing apparatus continues to execute step 101 to obtain data information according to the data description message.
Specifically, step 101 may include:
in step 1011, description information of a specified type is extracted from the description information.
Specifically, description information of a specified type can be extracted from a plurality of types of description information. For example, the specified type may include creation time, creator, merchant, transaction product, transaction amount, transaction date, shipping date, and reporting content, and the description information of the specified type may include: the establishment time is 14:00, the creator is a user account, the merchant is an XX tea shop, the transaction product is tea, the transaction amount is 1, the transaction amount is 100 yuan, the transaction date is 7 and 24 days in 2019, the delivery date is 7 and 25 days in 2019, the reporting reason is false delivery, and the reporting content is application refund. It should be noted that: the specified types of the description information include, but are not limited to, the above types, and in practical application, the description information can be set as required, so that the universality and the expansibility of the description information of the specified types are improved.
The data information generated in this step may include primary key information and content information, where the content information may include 14:00, a user account, XX tea shop, tea, 1, 100 yuan, a transaction date of 2019, 7, and 24 days, a shipping date of 2019, 7, and 25 days, a false shipment, and a refund application. Because the data information comprises the description information of the specified type except the primary key information, the structure for generating the data information is a universal structure, in other words, the data information has the universal structure, so that convenience is provided for subsequently constructing indexes, storing data and searching data, and the cost of later maintenance and upgrading is reduced.
And 102, constructing an index according to the main key information and the index information extracted from the content information, and storing the index to a search server.
In the embodiment of the invention, the index information extracted from the content information comprises the index information of the specified type. For example: the specified type of index information may include creation time, creator. It should be noted that: the designated type of the index information includes, but is not limited to, the above types, and in practical application, the index information can be set as required, so that the universality and the expansibility of the index information of the designated type are improved.
In step 102, the step of constructing an index according to the primary key information and the index information extracted from the content information specifically includes: and constructing an index according to the main key information and the index information of the specified type extracted from the content information. The structure of the index may be as shown in table 1 below.
TABLE 1
| Primary key information | Creation time | Creator |
| ID 1 | 14:00 | User |
| ID | ||
| 2 | 15:00 | User account B |
| …… | …… | …… |
As shown in table 1 above, each row of information is an index, and the index includes index information of a specific type except the primary key information, so that the structure of the index is a general structure, in other words, the index has a general structure, so that data search can be performed through the same type of search conditions when data are subsequently searched, and the universality of data search in this embodiment is improved, thereby providing convenience for subsequent data search, improving search efficiency, and reducing costs of later maintenance and upgrade.
In an embodiment of the present invention, the search server includes an ES server. As shown in fig. 1, in this step, the data processing apparatus may store the index to the ES server through the ES component module.
And step 103, storing the data information into a database.
In an embodiment of the invention, the database comprises a MySQL database. As shown in fig. 1, in this step, the data processing apparatus may store the data information in the MySQL database through the MySQL component module.
Fig. 5 is a flowchart of another data processing method according to an embodiment of the present invention, where the data processing method shown in fig. 5 is executed by the data processing apparatus shown in fig. 1, and as shown in fig. 5, the method includes:
Fig. 6 is a schematic diagram of another data processing method in an embodiment of the present invention, as shown in fig. 6, an auditor of an operator inputs a search condition on a second client to generate a search request. The number of search conditions may be one or more. When the search condition is plural, the plural search conditions may form a combined search condition.
For example: the retrieval condition may include data information of which creation time ranges from 13:00 to 17: 00.
In the embodiment of the invention, the index comprises primary key information and index information.
Fig. 7 is a flowchart of obtaining an index according to an embodiment of the present invention, and as shown in fig. 7, step 202 may specifically include:
As shown in table 1 above, for example, the data processing apparatus may query the index information with the creation time of 14:00 and the creation time of 15:00 from the search server through the ES component module.
For example, the data processing apparatus may determine, as an index matching the search condition, an index including index information of a creation time of 14:00 and a creation time of 15:00 through the ES component module.
And 203, acquiring data information of the primary key information identifier from the database according to the indexed primary key information.
As shown in table 1 above, the primary key information of the index matching the search condition includes ID1 and ID2, and the data processing apparatus may acquire the data information identified by ID1 and the data information identified by ID2 from the MySQL database through the MySQL component module.
And step 204, sending the data information to the second client.
Specifically, the data information identified by the primary key information in step 203 is returned to the second client. And the auditor audits the data information on the second client and returns the audited result to the data processing device. For example: and aiming at the data information identified by the ID1, the auditor audits the data identified by the ID1, determines the false shipment of the merchant, judges the refund of the merchant to the user, and returns the audit result of the refund of the merchant to the user to the data processing device. The data processing apparatus may send a notification message to the first client, the notification message including the audit result.
According to the technical scheme of the data processing method, the constructed index is stored in the search server, the data information is stored in the database, and the index and the stored information are respectively stored, so that the index can be acquired from the search server and the data information of the primary key information identification of the index can be acquired from the database during data search, and the hierarchical query of the index and the data information in the search process is realized, so that the search efficiency is improved, the search time is reduced, and the operation efficiency is further improved. In the embodiment of the invention, the search server adopts the ES server, and the ES server can realize the function of quickly searching the index, thereby further improving the search efficiency and reducing the search time; the database adopts the MySQL database, and the function of the universal database of the MySQL database is used without changing the bottom storage, so that the scheme of the embodiment of the invention is suitable for most service scenes, and has no influence on the existing service, thereby improving the user experience of the product. In the embodiment of the invention, the ES server only stores the index without storing other information in the data information, thereby reducing the use of storage resources in the ES server.
Fig. 8 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present invention, where the data processing apparatus is applicable to a server, and as shown in fig. 8, the data processing apparatus includes: an acquisition module 11, an indexing component module 12 and a database component module 13.
The obtaining module 11 is configured to obtain data information, where the data information includes primary key information and content information, and the primary key information is used to identify the data information.
The index component module 12 is configured to construct an index according to the primary key information and the index information extracted from the content information, and store the index in a search server.
The database component module 13 is configured to store the data information in a database.
Further, the index information extracted from the content information includes index information of a specified type.
The index component module 12 is specifically configured to construct an index according to the primary key information and the index information of the specified type extracted from the content information, and store the index in the search server.
Further, the apparatus further comprises: a receiving module 14.
The receiving module 14 is configured to receive a data description message sent by the first client, where the data description message includes the primary key information and description information.
The obtaining module 11 is specifically configured to extract description information of a specified type from the description information; and generating the data information according to the primary key information and the description information of the specified type, wherein the content information comprises the description information of the specified type.
In the embodiment of the invention, the search server comprises an ES server, and the database comprises a MySQL database.
In the embodiment of the present invention, the index component module 12 is the ES component module in fig. 1; the database component module 13 is the MySQL component module in fig. 1.
Fig. 9 is a schematic structural diagram of another data processing apparatus according to an embodiment of the present invention, where the data processing apparatus is applicable to a server, and as shown in fig. 9, the data processing apparatus includes: a receiving module 14, an indexing component module 12, a database component module 13 and a sending module 15.
The receiving module 14 is configured to receive a search request sent by a second client, where the search request includes a search condition;
the index component module 12 is configured to obtain an index matching the search condition from a search server according to the search condition;
the database component module 13 is configured to obtain data information of the primary key information identifier from a database according to the primary key information of the index;
the sending module 15 is configured to send the data information to the second client.
Further, the index includes primary key information and index information.
The index component module 12 is specifically configured to query the search server for index information matching the search condition; determining an index including index information matching the search condition as an index matching the search condition.
In the embodiment of the invention, the search server comprises an ES server, and the database comprises a MySQL database.
In the embodiment of the present invention, the index component module 12 is the ES component module in fig. 1; the database component module 13 is the MySQL component module in fig. 1.
According to the technical scheme of the data processing device, the constructed index is stored in the search server, the data information is stored in the database, and the index and the stored information are respectively stored, so that the index can be acquired from the search server and the data information of the main key information identification of the index can be acquired from the database during data search, the hierarchical query of the index and the data information in the search process is realized, the search efficiency is improved, and the search time is shortened.
Fig. 10 is a schematic structural diagram of a data processing system according to an embodiment of the present invention, and as shown in fig. 10, the system includes: a data processing apparatus 1, a search server 2 and a database 3.
The data processing device 1 is used for acquiring data information, wherein the data information comprises primary key information and content information, and the primary key information is used for identifying the data information; constructing an index according to the main key information and the index information extracted from the content information, and storing the index to a search server 2; storing the data information to a database 3;
the search server 2 is used to store the index.
The database 3 is used for storing the data information.
Further, the system further comprises: a first client 4.
The first client 4 is configured to send data description information to the data processing apparatus 1, where the data description information includes the primary key information and description information. The data processing device 1 is specifically configured to extract description information of a specified type from the description information; and generating the data information according to the primary key information and the description information of the specified type, wherein the content information comprises the description information of the specified type.
In the embodiment of the invention, the search server comprises an ES server, and the database comprises a MySQL database.
Fig. 11 is a schematic structural diagram of a data processing system according to an embodiment of the present invention, and as shown in fig. 11, the system includes: a data processing device 1, a search server 2, a database 3 and a second client 5.
The data processing device 1 is configured to receive a search request sent by the second client 5, where the search request includes a search condition; according to the search condition, obtaining an index matched with the search condition from the search server 2; acquiring data information of the primary key information identifier from a database 3 according to the indexed primary key information; sending the data information to the second client 5;
the second client 5 is configured to send the search request and receive data information sent by the data processing apparatus 1.
In the embodiment of the invention, the search server comprises an ES server, and the database comprises a MySQL database.
According to the technical scheme of the data processing system, the constructed index is stored in the search server, the data information is stored in the database, and the index and the stored information are respectively stored, so that the index can be acquired from the search server and the data information of the main key information identification of the index can be acquired from the database during data search, the hierarchical query of the index and the data information in the search process is realized, the search efficiency is improved, and the search time is shortened.
Embodiments of the present invention provide a storage medium, where the storage medium includes a stored program, where, when the program runs, a device on which the storage medium is located is controlled to execute each step of each embodiment of the data processing method, and specific descriptions may refer to each embodiment of the data processing method.
Embodiments of the present invention provide a server, including a memory and a processor, where the memory is used to store information including program instructions, and the processor is used to control execution of the program instructions, and the program instructions are loaded by the processor and executed to implement steps of various embodiments of the data processing method, and specific descriptions may refer to various embodiments of the data processing method.
Fig. 12 is a schematic diagram of a server according to an embodiment of the present invention. As shown in fig. 12, the server 20 of this embodiment includes: the processor 21, the memory 22, and the computer program 23 stored in the memory 22 and capable of running on the processor 21, where the computer program 23 is executed by the processor 21 to implement the data processing method in the embodiment, and in order to avoid repetition, the details are not repeated herein. Alternatively, the computer program is executed by the processor 21 to implement the functions of each model/unit in the data processing apparatus in the embodiments, which are not described herein again to avoid redundancy.
The server 20 may be a computing device such as a desktop computer, a notebook, a palm top computer, and a cloud server. The computer device may include, but is not limited to, a processor 21, a memory 22. Those skilled in the art will appreciate that fig. 12 is merely an example of a server 20 and does not constitute a limitation of server 20 and may include more or fewer components than shown, or some components in combination, or different components, e.g., server 20 may also include input-output devices, network access devices, buses, etc.
The Processor 21 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The storage 22 may be an internal storage unit of the server 20, such as a hard disk or a memory of the server 20. The memory 22 may also be an external storage device of the server 20, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), or the like provided on the server 20. Further, the memory 22 may also include both an internal storage unit of the server 20 and an external storage device. The memory 22 is used for storing computer programs and other programs and data required by the computer device. The memory 22 may also be used to temporarily store data that has been output or is to be output.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the embodiments provided in the present invention, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions in actual implementation, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) or a Processor (Processor) to execute some steps of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (21)
1. A data processing method, comprising:
acquiring data information, wherein the data information comprises primary key information and content information, and the primary key information is used for identifying the data information;
constructing an index according to the main key information and index information extracted from the content information, and storing the index to a search server;
and storing the data information to a database.
2. The data processing method according to claim 1, wherein the index information extracted from the content information includes index information of a specified type;
the constructing an index according to the primary key information and the index information extracted from the content information includes:
and constructing an index according to the main key information and the index information of the specified type extracted from the content information.
3. The data processing method of claim 1, wherein the obtaining data information further comprises:
receiving a data description message sent by a first client, wherein the data description message comprises the primary key information and description information;
the acquiring data information includes:
extracting description information of a specified type from the description information;
and generating the data information according to the primary key information and the description information of the specified type, wherein the content information comprises the description information of the specified type.
4. The data processing method of claim 3, wherein the receiving the data description message sent by the first client comprises:
and receiving the data description message sent by the first client through message middleware.
5. The data processing method of any one of claims 1 to 4, wherein the search server comprises an ES server and the database comprises a MySQL database.
6. A data processing method, comprising:
receiving a search request sent by a second client, wherein the search request comprises search conditions;
according to the search condition, obtaining an index matched with the search condition from a search server;
acquiring data information of the primary key information identification from a database according to the indexed primary key information;
and sending the data information to the second client.
7. The data processing method of claim 6, wherein the index includes primary key information and index information;
the obtaining of the index matched with the search condition from the search server according to the search condition comprises:
inquiring index information matched with the search condition from the search server;
determining an index including index information matching the search condition as an index matching the search condition.
8. The data processing method of claim 6 or 7, wherein the search server comprises an ES server and the database comprises a MySQL database.
9. A data processing apparatus, comprising:
the system comprises an acquisition module, a storage module and a display module, wherein the acquisition module is used for acquiring data information, the data information comprises primary key information and content information, and the primary key information is used for identifying the data information;
the index component module is used for constructing an index according to the main key information and the index information extracted from the content information and storing the index to a search server;
and the database component module is used for storing the data information to a database.
10. The data processing apparatus according to claim 9, wherein the index information extracted from the content information includes index information of a specified type;
the index component module is specifically configured to construct an index according to the primary key information and index information of a specified type extracted from the content information, and store the index to a search server.
11. The data processing apparatus of claim 9, further comprising:
the receiving module is used for receiving a data description message sent by a first client, wherein the data description message comprises the primary key information and description information;
the acquisition module is specifically used for extracting the description information of the specified type from the description information; and generating the data information according to the primary key information and the description information of the specified type, wherein the content information comprises the description information of the specified type.
12. A data processing apparatus according to any of claims 9 to 11, wherein the search server comprises an ES server and the database comprises a MySQL database.
13. A data processing apparatus, comprising:
the receiving module is used for receiving a search request sent by a second client, and the search request comprises a search condition;
the index component module is used for acquiring indexes matched with the search conditions from a search server according to the search conditions;
the database component module is used for acquiring the data information of the primary key information identifier from a database according to the primary key information of the index;
and the sending module is used for sending the data information to the second client.
14. The data processing apparatus of claim 13, wherein the index includes primary key information and index information;
the index component module is specifically used for inquiring index information matched with the search condition from the search server; determining an index including index information matching the search condition as an index matching the search condition.
15. The data processing apparatus according to claim 13 or 14, wherein the search server comprises an ES server and the database comprises a MySQL database.
16. A data processing system, comprising: a data processing device, a search server and a database;
the data processing device is used for acquiring data information, wherein the data information comprises primary key information and content information, and the primary key information is used for identifying the data information; constructing an index according to the main key information and index information extracted from the content information, and storing the index to a search server; storing the data information to a database;
the search server is used for storing the index;
and the database is used for storing the data information.
17. A data processing system, comprising: the system comprises a data processing device, a search server, a database and a second client;
the data processing device is used for receiving a search request sent by a second client, and the search request comprises a search condition; according to the search condition, obtaining an index matched with the search condition from a search server; acquiring data information of the primary key information identification from a database according to the indexed primary key information; sending the data information to the second client;
and the second client is used for sending the search request and receiving the data information sent by the data processing device.
18. A storage medium, characterized in that the storage medium comprises a stored program, wherein when the program runs, a device in which the storage medium is located is controlled to execute the data processing method according to any one of claims 1 to 5.
19. A storage medium, characterized in that the storage medium comprises a stored program, wherein when the program runs, a device in which the storage medium is located is controlled to execute the data processing method according to any one of claims 6 to 8.
20. A server comprising a memory for storing information including program instructions and a processor for controlling the execution of the program instructions, characterized in that the program instructions are loaded and executed by the processor to implement the steps of the data processing method according to any one of claims 1 to 5.
21. A server comprising a memory for storing information including program instructions and a processor for controlling the execution of the program instructions, characterized in that the program instructions are loaded and executed by the processor to implement the steps of the data processing method according to any one of claims 6 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910923997.0A CN110704486B (en) | 2019-09-27 | 2019-09-27 | Data processing method, device, system, storage medium and server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910923997.0A CN110704486B (en) | 2019-09-27 | 2019-09-27 | Data processing method, device, system, storage medium and server |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110704486A true CN110704486A (en) | 2020-01-17 |
| CN110704486B CN110704486B (en) | 2022-12-02 |
Family
ID=69197851
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910923997.0A Active CN110704486B (en) | 2019-09-27 | 2019-09-27 | Data processing method, device, system, storage medium and server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110704486B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113347220A (en) * | 2020-03-02 | 2021-09-03 | 北京金山云网络技术有限公司 | Inter-service data sharing method, device, server and medium |
| CN114205641A (en) * | 2021-11-30 | 2022-03-18 | 杭州华橙软件技术有限公司 | Video data processing method and device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106383860A (en) * | 2016-08-31 | 2017-02-08 | 无锡雅座在线科技发展有限公司 | Data query method and apparatus |
| US9633074B1 (en) * | 2014-01-03 | 2017-04-25 | Amazon Technologies, Inc. | Querying data set tables in a non-transactional database |
| US20170206249A1 (en) * | 2016-01-20 | 2017-07-20 | Systems And Methods For Implementing Urban Voices | Systems and methods for implementing urban voices |
| CN107103011A (en) * | 2016-02-23 | 2017-08-29 | 阿里巴巴集团控股有限公司 | The implementation method and device of terminal data search |
| CN107133362A (en) * | 2017-06-01 | 2017-09-05 | 北京凤凰理理它信息技术有限公司 | Commodity Information Search method, system, computer program and electronic equipment |
| CN110019213A (en) * | 2017-12-04 | 2019-07-16 | 北京京东尚科信息技术有限公司 | Data managing method, device, electronic equipment and storage medium |
| CN110109948A (en) * | 2019-04-25 | 2019-08-09 | 数译(成都)信息技术有限公司 | Data query method, computer equipment and computer readable storage medium |
-
2019
- 2019-09-27 CN CN201910923997.0A patent/CN110704486B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9633074B1 (en) * | 2014-01-03 | 2017-04-25 | Amazon Technologies, Inc. | Querying data set tables in a non-transactional database |
| US20170206249A1 (en) * | 2016-01-20 | 2017-07-20 | Systems And Methods For Implementing Urban Voices | Systems and methods for implementing urban voices |
| CN107103011A (en) * | 2016-02-23 | 2017-08-29 | 阿里巴巴集团控股有限公司 | The implementation method and device of terminal data search |
| CN106383860A (en) * | 2016-08-31 | 2017-02-08 | 无锡雅座在线科技发展有限公司 | Data query method and apparatus |
| CN107133362A (en) * | 2017-06-01 | 2017-09-05 | 北京凤凰理理它信息技术有限公司 | Commodity Information Search method, system, computer program and electronic equipment |
| CN110019213A (en) * | 2017-12-04 | 2019-07-16 | 北京京东尚科信息技术有限公司 | Data managing method, device, electronic equipment and storage medium |
| CN110109948A (en) * | 2019-04-25 | 2019-08-09 | 数译(成都)信息技术有限公司 | Data query method, computer equipment and computer readable storage medium |
Non-Patent Citations (2)
| Title |
|---|
| HUANG BIN ET AL: "An efficient two-level bitmap index for cloud data management", 《2011 IEEE 3RD INTERNATIONAL CONFERENCE ON COMMUNICATION SOFTWARE AND NETWORKS》 * |
| 徐婧: "数据的存储和检索", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113347220A (en) * | 2020-03-02 | 2021-09-03 | 北京金山云网络技术有限公司 | Inter-service data sharing method, device, server and medium |
| CN113347220B (en) * | 2020-03-02 | 2022-10-04 | 北京金山云网络技术有限公司 | Inter-service data sharing method, device, server and medium |
| CN114205641A (en) * | 2021-11-30 | 2022-03-18 | 杭州华橙软件技术有限公司 | Video data processing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110704486B (en) | 2022-12-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109034988B (en) | Accounting entry generation method and device | |
| WO2020000720A1 (en) | Server, packet processing method, program, and computer-readable storage medium | |
| CN106326243B (en) | Data processing method and device | |
| US10977703B2 (en) | Method and system for determining confidence of a seller in a user marketplace | |
| CN110704486B (en) | Data processing method, device, system, storage medium and server | |
| CN115544991A (en) | Report generation method, device, equipment, medium and program product | |
| CN106202440B (en) | Data processing method, device and equipment | |
| CN108959514B (en) | Data processing method and device | |
| CN114066533A (en) | Product recommendation method and device, electronic equipment and storage medium | |
| CN112597192A (en) | Data query method, device, server and medium | |
| CN116167860A (en) | Account checking processing method and device, electronic equipment and storage medium | |
| CN110471918A (en) | A kind of relevant database is synchronized to the method and device of block chain | |
| CN107038617B (en) | Pre-creation method and device of payment order | |
| CN114840388A (en) | Data monitoring method and device, electronic equipment and storage medium | |
| CN115145587A (en) | Product parameter checking method and device, electronic equipment and storage medium | |
| CN114238475A (en) | Data processing method of data warehouse | |
| CN110706051B (en) | Order data aggregation method and device and server | |
| CN113592470A (en) | Service processing method and device, electronic equipment and storage medium | |
| CN111611056A (en) | Data processing method and device, computer equipment and storage medium | |
| CN112541645A (en) | Data processing method and system along with vehicle product project development and related device | |
| CN113971007B (en) | Information processing method, device, electronic equipment and medium | |
| CN110782310B (en) | Method, device and system for asynchronously acquiring user attribute information from third-party platform | |
| CN111260477B (en) | Writing and reading method and processing system for product object data information | |
| CN113763097A (en) | Method and device for updating article information | |
| CN113609000A (en) | Test data generation method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |