CN110276002B - Search application data processing method and device, computer equipment and storage medium - Google Patents
Search application data processing method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN110276002B CN110276002B CN201910562858.XA CN201910562858A CN110276002B CN 110276002 B CN110276002 B CN 110276002B CN 201910562858 A CN201910562858 A CN 201910562858A CN 110276002 B CN110276002 B CN 110276002B
- Authority
- CN
- China
- Prior art keywords
- data
- application
- identifier
- data table
- search
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9532—Query formulation
Abstract
The present application relates to the field of data retrieval technologies, and in particular, to a search application data processing method and apparatus, a computer device, and a storage medium. The method comprises the following steps: monitoring a database log, and reading a first database identifier and a first data table identifier from a newly-added data stream when monitoring that the newly-added data stream is inserted into the database log; searching a first metadata structure corresponding to the first database identifier and the first data table identifier, and acquiring a first application identifier of the first metadata structure; processing data fields in the newly added data stream according to the first metadata structure to generate application metadata fields, and generating search application records according to the application metadata fields; the search application record is added to the search storage engine according to the first application identification. By adopting the method, the data query performance can be improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a search application data processing method and apparatus, a computer device, and a storage medium.
Background
The product of the internet company is updated quickly, and each business party of the product can generate a large amount of product business data every day in the business processing process of the product, so most products need to inquire a database or support other data retrieval functions.
Most of existing data search engines are based on a relational database, high-order data of various services are stored in the relational database, however, service requirements of internet companies are greatly changed, different service parties have different data query requirements, sometimes indexes are built on the relational database according to the requirements, and when too many indexes are built on the relational database by a plurality of service parties, due to the fact that effective layering can not be carried out on service data, data query performance is reduced, and the data query requirements of each service party can not be met.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a search application data processing method, apparatus, computer device and storage medium capable of improving data query performance and meeting requirements of multi-party data query.
A search application data processing method, the method comprising:
monitoring a database log, and reading a first database identifier and a first data table identifier from a newly added data stream when monitoring that the newly added data stream is inserted into the database log;
searching a first metadata structure corresponding to the first database identifier and the first data table identifier, and acquiring a first application identifier of the first metadata structure;
processing data fields in the newly added data stream according to the first metadata structure to generate application metadata fields, and generating search application records according to the application metadata fields;
and adding the search application record to a search storage engine according to the first application identifier.
In one embodiment, the processing the data field in the newly added data stream according to the first metadata structure to generate the application metadata field includes:
acquiring a service type corresponding to the first metadata structure, and searching and starting a data processing thread corresponding to the service type;
in the data processing thread, acquiring preset processing logic of each data element in the first metadata structure; and searching a data field corresponding to each data element from the newly added data stream, and processing the data field according to the preset processing logic to generate an application element field.
In one embodiment, after obtaining the service type corresponding to the first application metadata structure, the method further includes:
caching the newly increased data flow to a cache region corresponding to the service type in the message middleware;
before searching for the data field corresponding to each data element from the newly added data stream, the method includes:
and reading the newly added data stream from a buffer area corresponding to the service type of the message middleware.
In one embodiment, the method further comprises:
receiving a data table selection instruction sent by a terminal, wherein the data table selection instruction carries a second application identifier, a second database identifier and a second data table identifier;
searching a data table corresponding to the second data table identifier from a database corresponding to the second database identifier, acquiring the table structure information of the data table and returning the table structure information to the terminal;
receiving data element information sent by a terminal, wherein the data element information is formed by carrying out logic processing and editing on the table structure information, and a second element data structure is generated according to the data element information and the second application identifier;
and storing the second metadata structure in association with the second database identifier and the second data table identifier.
In one embodiment, the method further comprises:
analyzing field information and index information of each data element from the data element information, and generating an application data template according to the field information;
creating an application data table corresponding to the second application identifier in the search storage engine according to the application data template;
and creating an index for the application data table according to the index information.
In one embodiment, adding the search application record to a search storage engine according to the first application identification comprises:
searching an application data table corresponding to the first application identification from the search storage engine;
and adding each application meta-field in the search application record to a corresponding column of a new record row in the searched application data table.
In one embodiment, the method further comprises:
receiving a data query instruction sent by a terminal, wherein the data query instruction carries a target application identifier and a retrieval logic formula;
searching an application data table corresponding to the target application identification from the search storage engine;
and according to the retrieval logic formula, performing data retrieval on the application data table corresponding to the target application identifier to obtain target data, and returning the target data to the terminal.
A search application data processing apparatus, the apparatus comprising:
the log monitoring module is used for monitoring the database log, and reading a first database identifier and a first data table identifier from a newly-added data stream when the situation that the newly-added data stream is inserted into the database log is monitored;
the meta-structure searching module is used for searching a first meta-data structure corresponding to the first database identifier and the first data table identifier and acquiring a first application identifier of the first meta-data structure;
the data processing module is used for processing the data fields in the newly added data stream according to the first metadata structure to generate application metadata fields and generating search application records according to the application metadata fields;
and the engine data adding module is used for adding the search application record into a search storage engine according to the first application identifier.
In one embodiment, the data processing module comprises:
the thread searching unit is used for acquiring the service type corresponding to the first metadata structure and searching and starting a data processing thread corresponding to the service type;
a field processing unit, configured to obtain, in the data processing thread, a preset processing logic of each data element in the first metadata structure; and searching a data field corresponding to each data element from the newly added data stream, and processing the data field according to the preset processing logic to generate an application element field.
In one embodiment, the apparatus further comprises:
the data caching module is used for caching the newly increased data flow into a caching area corresponding to the service type in the message middleware;
and the buffer reading module is used for reading the newly added data stream from the buffer area corresponding to the service type of the message middleware.
In one embodiment, the apparatus further comprises:
the selection instruction receiving module is used for receiving a data table selection instruction sent by the terminal, wherein the data table selection instruction carries a second application identifier, a second database identifier and a second data table identifier;
the table structure returning module is used for searching the data table corresponding to the second data table identifier from the database corresponding to the second database identifier, acquiring the table structure information of the data table and returning the table structure information to the terminal;
a result generating module, configured to receive data metadata information sent by the terminal, where the data metadata information is formed by performing logic processing and editing on the table structure information, and generate a second metadata structure according to the data metadata information and the second application identifier;
and the structure storage module is used for storing the second metadata structure in association with the second database identifier and the second data table identifier.
In one embodiment, the apparatus further comprises:
the template generating module is used for analyzing the field information and the index information of each data element from the data element information and generating an application data template according to the field information;
a data table creating module, configured to create, in the search storage engine, an application data table corresponding to the second application identifier according to the application data template;
and the index creating module is used for creating an index for the application data table according to the index information.
In one embodiment, the engine data adding module comprises:
the data table searching unit is used for searching an application data table corresponding to the first application identifier from the search storage engine;
and the field recording unit is used for adding each application meta field in the search application record to a corresponding column of a new record row in the searched application data table.
In one embodiment, the apparatus further comprises:
the query instruction receiving module is used for receiving a data query instruction sent by the terminal, wherein the data query instruction carries a target application identifier and a retrieval logic formula;
the data table searching module is used for searching an application data table corresponding to the target application identifier from the search storage engine;
and the data retrieval module is used for performing data retrieval on the application data table corresponding to the target application identifier according to the retrieval logic formula to obtain target data and returning the target data to the terminal.
A computer device comprising a memory storing a computer program and a processor implementing the steps of the above method when executing the computer program.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method.
The method, the device, the computer equipment and the storage medium for processing the search application data monitor the database logs, when a new data stream is detected to be inserted into the database, read the data table information to be inserted into the new data stream and the application identifier of the corresponding search application, search the metadata structure corresponding to the data table information, perform field processing on the new data stream according to the metadata structure to generate the search application record meeting the search application requirement, add the processed new search application record into the search storage engine according to the application identifier, since the search application record in the search storage engine is divided according to the application identifier, when a business side performs data search, the corresponding target search application record in the search storage engine can be locked through selecting the search application and then performs search based on the target search application record, the isolation between the service data is realized, and the data retrieval efficiency is improved.
Drawings
FIG. 1 is a diagram of an application scenario for a search application data processing method in one embodiment;
FIG. 2 is a flow diagram illustrating a process for searching application data in one embodiment;
FIG. 3 is a data relationship map in one embodiment;
FIG. 4 is a flowchart that illustrates the steps of creating an application metadata structure, in one embodiment;
FIG. 5 is a flowchart illustrating the field processing steps in one embodiment;
FIG. 6 is a block diagram showing the structure of a search application data processing apparatus according to an embodiment;
FIG. 7 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
It will be understood that, as used herein, the terms "first," "second," and the like may be used herein to describe various elements, but these elements are not limited by these terms unless otherwise specified. These terms are only used to distinguish one element from another. For example, a first server may be referred to as a second server, and similarly, a second server may be referred to as a first server, without departing from the scope of the present application.
The search application data processing method provided by the application can be applied to the application environment shown in fig. 1. Wherein the service terminal 102 communicates with the server 104 via a network. Specifically, the user may select an existing data table, select a field in the data table, edit attributes such as a data format, an association relation, whether an index is established, and the like of the field, generate a new metadata structure after the editing is confirmed, the service terminal 102 sends the new metadata structure to the server 104, and the server 104 stores the new metadata structure in association with a data table identifier of the selected data table and a database identifier of a database to which the new metadata structure belongs. The server 104 monitors a newly added data stream newly inserted into the database, when the newly added data stream is monitored, a first database identifier and a first data table identifier corresponding to a written data table are read from the newly added data stream, a first metadata structure of a search application corresponding to the first database identifier and the first data table identifier is searched, a first application identifier corresponding to the search application of the first metadata structure is obtained, the server 104 processes a data field in the newly added data stream according to the first metadata structure to generate an application metadata field, a search application record is generated according to the application metadata field, and the search application record is added into a search storage engine according to the first application identifier.
The service terminal 102 may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the server 104 may be implemented by an independent server or a server cluster composed of a plurality of servers. When the server 104 is an independent server, a plurality of databases may be deployed in the server 104, and each database may store a service data table of a specific service; when the server 104 is a server cluster composed of a plurality of servers, a service data table of a specific service may be stored in a database disposed in each server.
In one embodiment, as shown in fig. 2, a search application data processing method is provided, which is described by taking the method as an example of being applied to the server 104 in fig. 1, and in other embodiments, the method can also be applied to a terminal, and the method includes the following steps:
And the server monitors the database logs of each deployed database and monitors whether a new data stream is inserted into the database logs or not. When operations such as addition, deletion, and modification are performed on data in a data table in a database, corresponding operations are recorded in a database log in a data stream form, and a server monitors operation behaviors in the database. For example, when data is newly added to a data table, a data stream is inserted into a database log through an insert instruction, and the data stream inserted into the database includes data table information of an operation, database information to which the data table belongs, and specific data content.
When the server monitors that a newly added data stream is inserted into the database log, the server reads a first database identifier and a first data table identifier from the newly added data stream. The server may be deployed with a plurality of databases, each database may store a plurality of data tables, the data tables are directed to a plurality of business parties, data of the business parties may be stored in the databases, and the data stored in the data tables may be classified according to business types. The data table identification is used for uniquely identifying each data table, and the database identification is used for uniquely identifying the database stored in the data table. In this embodiment, the first data table identifier is an identifier of a data table into which the new data is inserted, and the first database identifier is an identifier of a database stored in the data table of the new data.
In order to facilitate data search, users of each service party can create search applications according to own needs, and set metadata structures needed by the search applications. The metadata structure sets data fields in a data table related to search application, data format of each data field, information such as whether the fields are added with indexes, incidence relation with other fields, field operation logic and the like. The metadata structure is stored in association with the identification information of the data table referenced by each data field in the structure. The data table referred to by the search application may be plural, and thus, the data table corresponding to the metadata structure may also be plural. In addition, a data table may also be referenced by search applications of multiple business parties. Please refer to fig. 3 for a data relationship map among the service party, the search application, and the data table. A business party may set a plurality of search applications, in fig. 3, business party 1 sets search application a and search application b, the number of data tables referred by each search application may be one or more, search application a refers to two data tables including data table a and data table b, and search application b refers to one data table c.
The server searches for a first metadata structure corresponding to the first database identifier and the first data table identifier, and the number of the first metadata structures searched by the server may be multiple. The metadata structure is identified with an application identification of the search application, and the server obtains the application identification of the first metadata structure.
And step 230, processing the data field in the newly added data stream according to the first metadata structure to generate an application metadata field, and generating a search application record according to the application metadata field.
The server obtains the data format of each data field in the first metadata structure, the field operation logic, whether to establish field information such as indexes on the field, reads the data stream field written in the data table from the newly added data stream, processes the corresponding data stream field according to the field information to generate the application metadata field, if the data format is adjusted, operates the field value according to the field operation logic to generate a new field, establishes indexes on some data stream fields, and the like.
And the server sorts the application element fields generated after processing according to a standard format to generate search application records. For example, the field names of the data fields may be standardized to generate a standardized search application record.
The search engine is used for processing data search requests of users, and the search storage engine is used for storing search data used by the search engine. And the server searches a data storage area corresponding to the first application identifier in the search storage engine and adds the search application record to the corresponding data storage area. When the user of the business side carries out data retrieval query, the user can select the search application and the retrieval formula which need to be used, and the search engine carries out data retrieval from the data corresponding to the search application according to the retrieval formula, so that the range of the retrieved data can be effectively reduced, and the data query retrieval efficiency is improved.
In this embodiment, the server monitors the database log, and when it is detected that a new data stream is inserted into the database, reading the data table information to be inserted by the new data stream and the application identifier of the corresponding search application, and searching a metadata structure corresponding to the data table information, performing field processing on the newly added data stream according to the metadata structure to generate a search application record meeting the search application requirement, adding the newly processed search application record into a search storage engine according to an application identifier, since the search application records in the search storage engine are divided according to the application identifiers, when a service side searches data, corresponding target search application records in the search storage engine can be locked through selecting the search application, and then searching is carried out based on the target search application records, so that isolation among service data is realized, and data retrieval efficiency is improved.
In one embodiment, the step of processing the data field in the newly added data stream according to the first metadata structure to generate the application metadata field may include: acquiring a service type corresponding to the first metadata structure, and searching and starting a data processing thread corresponding to the service type; in a data processing thread, acquiring preset processing logic of each data element in a first metadata structure; and searching a data field corresponding to each data element from the newly-added data stream, and processing the data field according to preset processing logic to generate an application element field.
In this embodiment, the server sets a plurality of data processing threads, and the plurality of data processing threads are divided according to the service type of the data to be processed. For example, the service type may include private service, public service, and the like, and when the search application is built, the service type of the query data corresponding to the search application is also added to the metadata structure.
The server extracts the service type information from the first metadata structure, and the service type information can be service type identification, service name and the like. And the server searches a data processing thread corresponding to the service type and informs the data processing thread to perform data processing on the data field in the newly added data stream.
In an embodiment, the server may directly distribute the newly added data stream to the data processing thread, and after receiving the newly added data stream, the data processing thread acquires the preset processing logic of each data element from the first metadata structure, where the data element is a structure field of the corresponding application structure before the application element field is processed. The data processing thread searches for the data field corresponding to each data element from the newly added data stream, processes the data field according to the preset processing logic and generates the application element field, and the preset processing logic may be the logic of field format processing, field value operation, and the like described in the previous embodiment.
In the embodiment, by performing the multithread processing on the newly added data stream according to the service type, the data streams of different service types can be isolated, the parallel processing of the multi-service data is realized, and the data processing efficiency is improved.
In an embodiment, after the step of obtaining the service type corresponding to the first application metadata structure, the server may cache the newly added data stream into a cache region corresponding to the service type in the message middleware, and the step before the data processing thread searches for the data field corresponding to each data element from the newly added data stream may include: and reading the newly added data stream from the buffer area corresponding to the service type of the message middleware.
In this embodiment, the server does not directly distribute the newly added data stream to the corresponding data processing thread for processing, but caches all the newly added data streams in the message middleware, and then notifies the corresponding data processing thread to process. The message middleware is provided with a plurality of cache regions, the cache regions are divided according to the service types, the server searches the cache region corresponding to the service type from the message middleware and stores the newly added data stream into the cache region. And the server informs the data processing thread to read the newly added data stream from the cache region corresponding to the service type and then perform data processing when processing the newly added data stream. Furthermore, the read new data stream is marked in the message middleware, and the read new data stream in each cache region is cleaned regularly. By caching the newly added data stream into the message middleware, excessive data table data can be prevented from being backed up in the server, and data redundancy is reduced.
In one embodiment, as shown in fig. 4, the search application data processing method may further include the following creation step of the application metadata structure:
step 202: and receiving a data table selection instruction sent by the terminal, wherein the data table selection instruction carries a second application identifier, a second database identifier and a second data table identifier.
When a user of a service side has a new search application requirement, the creation operation of a newly added application metadata structure can be performed on a service terminal. The user can set application identification such as an application name and basic information such as a service type of the application on the terminal, then a data table in a database which needs to be quoted by the search application is selected, after the user selects the database, the terminal displays the data table in the database, the user can select the data table, and after the data table is selected, the terminal generates a data table selection instruction according to the data table identification of the selected data table, the database identification of the selected data table and the set application identification and sends the data table selection instruction to the server.
For example, the name of the search application set by the user may be "shop," the service type to which the search application belongs, such as loans and the like, the user may select a "centralized" database, the terminal displays the data table stored in the database, and the user selects the data table, for example, the data table may include baseshop, shop _ extra, shop _ auth, and the like.
And after receiving a data table selection instruction sent by the terminal, the server reads a second application identifier of the search application set by the user, a second database identifier of the selected database and a second data table identifier of the selected data table from the data table selection instruction.
Step 204: and searching the data table corresponding to the second data table identifier from the database corresponding to the second database identifier, acquiring the table structure information of the data table and returning the table structure information to the terminal.
The server searches the database corresponding to the second database identifier, searches the data table corresponding to the second database identifier from the searched database, obtains table structure information of the data table, wherein the table structure information can comprise field information such as field names and data formats of fields in the data table, and the server returns the obtained table structure information to the terminal.
Step 206: and receiving data element information sent by a terminal, wherein the data element information is formed by carrying out logic processing and editing on the table structure information, and a second element data structure is generated according to the data element information and a second application identifier.
And after receiving the table structure information, the terminal displays the table structure information in a visual way, such as listing field names and field data formats in a list form, and providing an editing operation option for each field. The editing operation selection may include deleting, editing field type, adding external link, setting association relationship with fields in other tables, setting word segmenter, setting index, and other operation options. The word segmentation device is a preset field numerical value processing logic, and the processing logic is coded and packaged, so that the word segmentation device can be used for all fields. For example, the field value processing logic may be logic to convert Chinese characters to pinyin, separate numbers in the field values from Chinese characters, and so on. The index is set to add an index to the field in a data table that is then generated from the metadata structure.
And editing the displayed table structure information by the user, and after the editing is finished and confirmed, acquiring the editing information of the fields in the table structure finally selected by the user, namely the data element information by the terminal, wherein each field selected and edited by the user is called as a data element. And the server assembles the data meta information to generate a second metadata structure, and identifies the second metadata structure according to the second application identifier.
Step 208: and storing the second metadata structure in association with the second database identification and the second data table identification.
When the server stores the second metadata structure, the server associates the second library identifier and the second data table identifier corresponding to the referenced data table with the second metadata structure, for example, a mapping relationship between the second application identifier and the second database identifier and between the second application identifier and the second data table identifier may be recorded. Therefore, when new data is written into the data table, the search application which refers to the data table can be found, and the search storage engine corresponding to the search application can be updated conveniently.
In one embodiment, after the step of storing the second metadata structure in association with the second database identifier and the second data table identifier, the server may further include: analyzing field information and index information of each data element from the data element information, and generating an application data template according to the field information; creating an application data table corresponding to the second application identifier in the search storage engine according to the application data template; and creating an index for the application data table according to the index information.
The server analyzes field information and index information of each data element from the data element information of the second element data structure, wherein the field information can comprise field names, field types such as integer and numerical values, information such as whether the field is a main key, and the like, the index information is used for establishing an index for the data element, in addition, the server creates an application data template according to the acquired field information of each data element, the application data template is a field information set of the data element, and the field information set can be stored in a key-value character string form for example. The method comprises the steps that a server sets a field structure of an application data table according to information of each field in an application data template, the application data table can be files in excel, csv and other formats, if an elastic search engine is used by the server, the application data table can be a data table in an elastic search file format, and indexes are set for columns where corresponding fields in the application data table are located according to index information of each data element. And the server identifies the generated application data table according to the second application identification.
In one embodiment, the step of adding the search application record to the search storage engine based on the first application identification may comprise: searching an application data table corresponding to the first application identifier from a search storage engine; and adding each application meta field in the search application record to a corresponding column of a new record row in the searched application data table.
When the server monitors a newly added data stream corresponding to the search application of the first application identifier and processes the newly added data stream to generate a search application record, the server searches an application data table corresponding to the first application identifier from a search storage engine, creates a new record row in the application data table, matches the name of each application meta-field in the search application record with the field name in the application data table, and adds the field value of each application meta-field to the corresponding column of the matched field in the new record row, so that a new data record is generated in the application data table.
In one embodiment, the user can also modify, delete and the like the metadata structure of the created search application on the terminal. When a user selects a created search application on the terminal, the terminal displays the metadata structure of the application in a visual mode, and the user can edit and modify the data elements in the search application, including modifying the field type, adding indexes or not, changing a word segmentation device and other attributes. And after the user confirms that the modification is finished, the terminal acquires the modified data metadata information to generate a modified metadata structure and returns the modified metadata structure to the server. And the server replaces the original metadata structure of the search application after receiving the modified metadata structure, and correspondingly modifies the created application data template and the created application data table according to the modified metadata structure.
When a user selects to delete a certain search application, the terminal acquires an application identifier of the search application selected to be deleted by the user, sends a search application deletion instruction carrying the application identifier to the server, extracts the application identifier of the search application to be deleted from the search application instruction by the server, and deletes a metadata structure, an application data template and an application data table corresponding to the application identifier.
In one embodiment, the search application data processing method may further include the steps of: receiving a data query instruction sent by a terminal, wherein the data query instruction carries a target application identifier and a retrieval logic formula; searching an application data table corresponding to the target application identifier from a search storage engine; and performing data retrieval on the application data table corresponding to the target application identifier according to the retrieval logic formula to obtain target data, performing field sorting and other processing on the target data according to a preset retrieval output format, and returning the processed target data to the terminal.
When a user needs to perform data query, a target application identifier of a target search application which needs to be used can be selected on a data search page of the terminal, and a search logical formula is input, wherein the search logical formula can be a simple key field, a combination of the key field and a search condition, and the like. When the user confirms retrieval, the terminal obtains the target application identification selected by the user and the input retrieval logic formula, generates a data query instruction according to the target application identification and the retrieval logic formula and sends the data query instruction to the server.
The server receives a data query instruction sent by the terminal, extracts a target application identifier and a retrieval logic formula from the data query instruction, searches an application data table corresponding to the target application identifier from a search storage engine, performs retrieval query on data in the application data table according to the retrieval logic formula to obtain target data conforming to the retrieval logic formula, and returns the queried target data to the terminal. Massive business data can be reduced to the data size of the application data table by constructing search application, so that the search speed magnitude can be improved.
In one embodiment, as shown in fig. 5, the field processing step of processing the data field in the newly added data stream according to the first metadata structure to generate the application metadata field may include:
When a plurality of data tables are referred by the search application created by the business user, hierarchical information among the data tables needs to be set when the metadata structure is edited, namely, one data table needs to be set as a main table, other tables are auxiliary tables, data association is carried out between the auxiliary tables and the main table through a main key or other fields, and fields capable of establishing association relationship between the tables are association fields. The other fields except the associated fields in the auxiliary table can be mutually independent from the fields in the main table, and can also be required to be attached to the fields in the main table for operation. If the user selects two data tables, namely, the shop _ extra data table and the shop _ auth data table, the shop _ extra data table can be selected as a main table, and the shop _ auth data table can be selected as an auxiliary table.
When a server processes a field of a newly added data stream, it needs to first judge whether a data table corresponding to the data stream is a main table in a metadata structure, the server extracts data surface level information from a first metadata structure, judges whether the data surface level information has a main table identifier, if so, judges that the data table corresponding to a second data table identifier is the main table, and if so, judges that the data table corresponding to the second data table identifier is an auxiliary table.
And step 234, when the data table corresponding to the second data table identification is judged not to be the main table, extracting the associated field from the data table corresponding to the second data table identification, and searching the record row which corresponds to the associated field and is generated according to the main table from the search storage engine.
The data stream corresponding to the primary table is typically written into the database prior to the data of the secondary table. Therefore, when the data table corresponding to the newly added data stream is the main table, the fields in the newly added data stream can be directly processed according to the metadata structure of the search application, the fields processed according to the data stream of the main table can be firstly stored in the data column corresponding to the main table fields of the newly added record row in the search storage engine, and the auxiliary table fields in the newly added record row are vacated. When the data table corresponding to the newly added data stream is the secondary table, since the secondary table field may need to be attached to the primary table field for processing, when the secondary table data stream is processed, the search record generated according to the primary table data stream needs to be acquired from the search storage engine according to the primary key in the secondary table. Specifically, when the server determines that the data table corresponding to the second data table identifier is not the main table, the server extracts the associated field from the data table corresponding to the second data table identifier, searches the storage area corresponding to the first application identifier from the search storage engine, and searches the record line data consistent with the main key field from the storage area. And the server acquires the associated fields between the data table and the main table in the data surface level information and extracts field data corresponding to the associated fields from the recorded line data.
And 236, processing the data field in the newly added data stream to generate an application meta-field according to the first meta-data structure and the found field in the record line.
And the server processes the data fields in the newly added data stream according to the first metadata structure and the field data in the extracted record row, namely, processing logic for extracting each application element field from the first metadata structure, performs operation processing according to the data fields in the newly added data stream of the processing logic, and generates the application element fields by performing operation processing on the associated fields and the data fields according to the processing logic when the data fields are fields associated with the master table.
The step 240 of adding the search application record to the search storage engine according to the first application identifier may specifically include: the application meta-field is supplemented to the corresponding column in the record row.
And the server integrates the application element fields obtained after the auxiliary table processing to generate a search application record, and adds each application element field in the search application record to the corresponding data column of the searched record row.
It should be understood that although the various steps in the flow charts of fig. 2-5 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2-5 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternating with other steps or at least some of the sub-steps or stages of other steps.
In one embodiment, as shown in fig. 6, there is provided a search application data processing apparatus including: a log monitoring module 610, a meta structure searching module 620, a data processing module 630 and an engine data adding module 640, wherein:
the log monitoring module 610 is configured to monitor the database log, and when it is monitored that a newly added data stream is inserted into the database log, read the first database identifier and the first data table identifier from the newly added data stream.
And the meta-structure searching module 620 is configured to search for a first meta-data structure corresponding to the first database identifier and the first data table identifier, and obtain a first application identifier of the first meta-data structure.
The data processing module 630 is configured to process the data field in the newly added data stream according to the first metadata structure to generate an application meta field, and generate a search application record according to the application meta field.
An engine data adding module 640, configured to add the search application record to the search storage engine according to the first application identifier.
In one embodiment, the data processing module 630 may include:
and the thread searching unit is used for acquiring the service type corresponding to the first metadata structure, and searching and starting the data processing thread corresponding to the service type.
The field processing unit is used for acquiring the preset processing logic of each data element in the first metadata structure in the data processing thread; and searching a data field corresponding to each data element from the newly-added data stream, and processing the data field according to preset processing logic to generate an application element field.
In one embodiment, the search application data processing apparatus may further include:
and the data caching module is used for caching the newly added data flow into a caching area corresponding to the service type in the message middleware.
And the cache reading module is used for reading the newly-added data stream from the cache region corresponding to the service type of the message middleware.
In one embodiment, the search application data processing apparatus may further include:
and the selection instruction receiving module is used for receiving a data table selection instruction sent by the service terminal, wherein the data table selection instruction carries a second application identifier, a second database identifier and a second data table identifier.
And the table structure returning module is used for searching the data table corresponding to the second data table identifier from the database corresponding to the second database identifier, acquiring the table structure information of the data table and returning the table structure information to the terminal.
And the result generation module is used for receiving the data metadata information which is sent by the terminal and is obtained by logically processing and editing the table structure information, and generating a second metadata structure according to the data metadata information and the second application identifier.
And the structure storage module is used for storing the second metadata structure in association with the second database identifier and the second data table identifier.
In one embodiment, the apparatus may further comprise:
and the template generating module is used for analyzing the field information and the index information of each data element from the data element information and generating the application data template according to the field information.
And the data table creating module is used for creating an application data table corresponding to the second application identifier in the search storage engine according to the application data template.
And the index creating module is used for creating an index for the application data table according to the index information.
In one embodiment, the engine data addition module 640 may include:
and the data table searching unit is used for searching the application data table corresponding to the first application identification from the search storage engine.
And the field recording unit is used for adding each application element field in the searched application record to a corresponding column of a new record row in the searched application data table.
In one embodiment, the apparatus may further comprise:
and the query instruction receiving module is used for receiving a data query instruction sent by the terminal, wherein the data query instruction carries the target application identifier and the retrieval logic formula.
And the data table searching module is used for searching the application data table corresponding to the target application identifier from the search storage engine.
And the data retrieval module is used for performing data retrieval on the application data table corresponding to the target application identifier according to the retrieval logic formula to obtain target data and returning the target data to the terminal.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 7. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for searching application data processing related data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a search application data processing method.
Those skilled in the art will appreciate that the architecture shown in fig. 7 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, there is provided a computer device comprising a memory storing a computer program and a processor implementing the following steps when the processor executes the computer program: monitoring a database log, and reading a first database identifier and a first data table identifier from a newly-added data stream when monitoring that the newly-added data stream is inserted into the database log; searching a first metadata structure corresponding to the first database identifier and the first data table identifier, and acquiring a first application identifier of the first metadata structure; processing data fields in the newly added data stream according to the first metadata structure to generate application metadata fields, and generating search application records according to the application metadata fields; the search application record is added to the search storage engine according to the first application identification.
In one embodiment, the processor is further configured to obtain a service type corresponding to the first metadata structure and search and enable a data processing thread corresponding to the service type when the processor executes the computer program to perform the step of processing the data field in the newly added data stream according to the first metadata structure to generate the application metadata field; in a data processing thread, acquiring preset processing logic of each data element in a first metadata structure; and searching a data field corresponding to each data element from the newly-added data stream, and processing the data field according to preset processing logic to generate an application element field.
In one embodiment, the processor, when executing the computer program, further performs the steps of: caching the newly added data flow to a cache region corresponding to the service type in the message middleware; and reading the newly added data stream from the buffer area corresponding to the service type of the message middleware.
In one embodiment, the processor, when executing the computer program, further performs the steps of: receiving a data table selection instruction sent by the terminal, wherein the data table selection instruction carries a second application identifier, a second database identifier and a second data table identifier; searching a data table corresponding to the second data table identifier from a database corresponding to the second database identifier, acquiring the table structure information of the data table and returning the table structure information to the terminal; receiving data element information sent by a terminal, wherein the data element information is formed by carrying out logic processing and editing on table structure information, and a second element data structure is generated according to the data element information and a second application identifier; and storing the second metadata structure in association with the second database identification and the second data table identification.
In one embodiment, the processor, when executing the computer program, further performs the steps of: analyzing field information and index information of each data element from the data element information, and generating an application data template according to the field information; creating an application data table corresponding to the second application identifier in the search storage engine according to the application data template; and creating an index for the application data table according to the index information.
In one embodiment, the processor when executing the computer program when performing the step of adding the search application record to the search storage engine based on the first application identification is further configured to: searching an application data table corresponding to the first application identifier from a search storage engine; and adding each application meta field in the search application record to a corresponding column of a new record row in the searched application data table.
In one embodiment, the processor, when executing the computer program, further performs the steps of: receiving a data query instruction sent by a terminal, wherein the data query instruction carries a target application identifier and a retrieval logic formula; searching an application data table corresponding to the target application identifier from a search storage engine; and performing data retrieval on the application data table corresponding to the target application identifier according to the retrieval logic formula to obtain target data, and returning the target data to the terminal.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of: monitoring a database log, and reading a first database identifier and a first data table identifier from a newly-added data stream when monitoring that the newly-added data stream is inserted into the database log; searching a first metadata structure corresponding to the first database identifier and the first data table identifier, and acquiring a first application identifier of the first metadata structure; processing data fields in the newly added data stream according to the first metadata structure to generate application metadata fields, and generating search application records according to the application metadata fields; the search application record is added to the search storage engine according to the first application identification.
In one embodiment, when executed by the processor, the computer program is further configured to obtain a service type corresponding to the first metadata structure and search for and enable a data processing thread corresponding to the service type when the step of processing the data field in the newly added data stream according to the first metadata structure to generate the application metadata field is performed; in a data processing thread, acquiring preset processing logic of each data element in a first metadata structure; and searching a data field corresponding to each data element from the newly-added data stream, and processing the data field according to preset processing logic to generate an application element field.
In one embodiment, the computer program when executed by the processor further performs the steps of: caching the newly added data flow to a cache region corresponding to the service type in the message middleware; and reading the newly added data stream from the buffer area corresponding to the service type of the message middleware.
In one embodiment, the computer program when executed by the processor further performs the steps of: receiving a data table selection instruction sent by the service terminal, wherein the data table selection instruction carries a second application identifier, a second database identifier and a second data table identifier; searching a data table corresponding to the second data table identifier from a database corresponding to the second database identifier, acquiring the table structure information of the data table and returning the table structure information to the terminal; receiving data element information sent by a terminal, wherein the data element information is formed by carrying out logic processing and editing on table structure information, and a second element data structure is generated according to the data element information and a second application identifier; and storing the second metadata structure in association with the second database identification and the second data table identification.
In one embodiment, the computer program when executed by the processor further performs the steps of: analyzing field information and index information of each data element from the data element information, and generating an application data template according to the field information; in a search storage engine, creating an application data table corresponding to the second application identifier according to the application data template; and creating an index for the application data table according to the index information.
In one embodiment, the computer program when executed by the processor further performs the step of adding the search application record to the search storage engine in accordance with the first application identification further for: searching an application data table corresponding to the first application identifier from a search storage engine; and adding each application meta field in the search application record to a corresponding column of a new record row in the searched application data table.
In one embodiment, the computer program when executed by the processor further performs the steps of: receiving a data query instruction sent by a terminal, wherein the data query instruction carries a target application identifier and a retrieval logic formula; searching an application data table corresponding to the target application identifier from a search storage engine; and according to the retrieval logic formula, performing data retrieval on the application data table corresponding to the target application identifier to obtain target data, and returning the target data to the terminal.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (14)
1. A search application data processing method, characterized in that the method comprises:
receiving a data table selection instruction sent by a terminal, wherein the data table selection instruction carries a first application identifier, a first database identifier and a first data table identifier;
searching a data table corresponding to the first data table identifier from a database corresponding to the first database identifier, acquiring table structure information of the data table and returning the table structure information to the terminal;
receiving data element information sent by the terminal, wherein the data element information is formed by carrying out logic processing and editing on the table structure information, and a first metadata structure is generated according to the data element information and the first application identifier;
storing the first metadata structure in association with the first database identifier and the first data table identifier;
monitoring a database log, and reading a first database identifier and a first data table identifier from a newly added data stream when monitoring that the newly added data stream is inserted into the database log;
searching a first metadata structure corresponding to the first database identifier and the first data table identifier, and acquiring a first application identifier of the first metadata structure;
processing data fields in the newly added data stream according to the first metadata structure to generate application metadata fields, and generating search application records according to the application metadata fields;
and adding the search application record to a search storage engine according to the first application identifier.
2. The method of claim 1, wherein the processing the data field in the newly added data stream according to the first metadata structure to generate an application meta field comprises:
acquiring a service type corresponding to the first metadata structure, and searching and starting a data processing thread corresponding to the service type;
in the data processing thread, acquiring preset processing logic of each data element in the first metadata structure; and searching a data field corresponding to each data element from the newly added data stream, and processing the data field according to the preset processing logic to generate an application element field.
3. The method of claim 2, wherein after obtaining the service type corresponding to the first application metadata structure, further comprising:
caching the newly increased data flow to a cache region corresponding to the service type in the message middleware;
before searching for the data field corresponding to each data element from the newly added data stream, the method includes:
and reading the newly added data stream from a buffer area corresponding to the service type of the message middleware.
4. The method of claim 1, further comprising:
analyzing field information and index information of each data element from the data element information, and generating an application data template according to the field information;
creating an application data table corresponding to the first application identifier in the search storage engine according to the application data template;
and creating an index for the application data table according to the index information.
5. The method of claim 4, wherein adding the search application record to a search storage engine according to the first application identification comprises:
searching an application data table corresponding to the first application identification from the search storage engine;
and adding each application meta-field in the search application record to a corresponding column of a new record row in the searched application data table.
6. The method of claim 4, further comprising:
receiving a data query instruction sent by a terminal, wherein the data query instruction carries a target application identifier and a retrieval logic formula;
searching an application data table corresponding to the target application identification from the search storage engine;
and according to the retrieval logic formula, performing data retrieval on the application data table corresponding to the target application identifier to obtain target data, and returning the target data to the terminal.
7. A search application data processing apparatus, characterized in that the apparatus comprises:
the selection instruction receiving module is used for receiving a data table selection instruction sent by the terminal, wherein the data table selection instruction carries a first application identifier, a first database identifier and a first data table identifier;
the table structure returning module is used for searching the data table corresponding to the first data table identifier from the database corresponding to the first database identifier, acquiring the table structure information of the data table and returning the table structure information to the terminal;
a result generating module, configured to receive data metadata information sent by the terminal, where the data metadata information is formed by performing logic processing and editing on the table structure information, and generate a first metadata structure according to the data metadata information and the first application identifier;
the structure storage module is used for storing the first metadata structure, the first database identifier and the first data table identifier in a correlation manner;
the log monitoring module is used for monitoring the database log, and reading a first database identifier and a first data table identifier from a newly-added data stream when the situation that the newly-added data stream is inserted into the database log is monitored;
the meta-structure searching module is used for searching a first meta-data structure corresponding to the first database identifier and the first data table identifier and acquiring a first application identifier of the first meta-data structure;
the data processing module is used for processing the data fields in the newly added data stream according to the first metadata structure to generate application metadata fields and generating search application records according to the application metadata fields;
and the engine data adding module is used for adding the search application record into a search storage engine according to the first application identifier.
8. The apparatus of claim 7, wherein the data processing module comprises:
the thread searching unit is used for acquiring the service type corresponding to the first metadata structure and searching and starting a data processing thread corresponding to the service type;
a field processing unit, configured to obtain, in the data processing thread, a preset processing logic of each data element in the first metadata structure; and searching a data field corresponding to each data element from the newly added data stream, and processing the data field according to the preset processing logic to generate an application element field.
9. The apparatus of claim 8, further comprising:
the data caching module is used for caching the newly increased data flow into a caching area corresponding to the service type in the message middleware;
and the buffer reading module is used for reading the newly added data stream from the buffer area corresponding to the service type of the message middleware.
10. The apparatus of claim 7, further comprising:
the template generating module is used for analyzing the field information and the index information of each data element from the data element information and generating an application data template according to the field information;
a data table creating module, configured to create, in the search storage engine, an application data table corresponding to the first application identifier according to the application data template;
and the index creating module is used for creating an index for the application data table according to the index information.
11. The apparatus of claim 10, wherein the engine data addition module comprises:
the data table searching unit is used for searching an application data table corresponding to the first application identifier from the search storage engine;
and the field recording unit is used for adding each application meta field in the search application record to a corresponding column of a new record row in the searched application data table.
12. The apparatus of claim 10, further comprising:
the query instruction receiving module is used for receiving a data query instruction sent by the terminal, wherein the data query instruction carries a target application identifier and a retrieval logic formula;
the data table searching module is used for searching an application data table corresponding to the target application identifier from the search storage engine;
and the data retrieval module is used for performing data retrieval on the application data table corresponding to the target application identifier according to the retrieval logic formula to obtain target data and returning the target data to the terminal.
13. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method of any one of claims 1 to 6 when executing the computer program.
14. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910562858.XA CN110276002B (en) | 2019-06-26 | 2019-06-26 | Search application data processing method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910562858.XA CN110276002B (en) | 2019-06-26 | 2019-06-26 | Search application data processing method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110276002A CN110276002A (en) | 2019-09-24 |
| CN110276002B true CN110276002B (en) | 2021-08-03 |
Family
ID=67963358
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910562858.XA Active CN110276002B (en) | 2019-06-26 | 2019-06-26 | Search application data processing method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110276002B (en) |
Families Citing this family (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110866491B (en) * | 2019-11-13 | 2023-11-24 | 腾讯科技（深圳）有限公司 | Target retrieval method, apparatus, computer-readable storage medium, and computer device |
| CN111026757B (en) * | 2019-12-10 | 2023-10-10 | 首都医科大学附属北京友谊医院 | Method, device, equipment and storage medium for generating standard statistical format data |
| CN111061739B (en) * | 2019-12-17 | 2023-07-04 | 医渡云（北京）技术有限公司 | Method and device for warehousing massive medical data, electronic equipment and storage medium |
| CN111159192B (en) * | 2019-12-30 | 2023-09-05 | 北京因特睿软件有限公司 | Big data based data warehousing method and device, storage medium and processor |
| CN111274350B (en) * | 2020-02-03 | 2023-06-23 | 广州极尚网络技术有限公司 | Data processing method, device, computer equipment and storage medium |
| CN113268530A (en) * | 2020-02-17 | 2021-08-17 | 浙江大搜车软件技术有限公司 | Mass heterogeneous data acquisition method and system, computer equipment and storage medium |
| CN111324510B (en) * | 2020-02-21 | 2024-04-02 | 腾讯科技（深圳）有限公司 | Log processing method and device and electronic equipment |
| CN111858779A (en) * | 2020-06-05 | 2020-10-30 | 北京旷视科技有限公司 | Data analysis method and device, electronic equipment and storage medium |
| CN112287269B (en) * | 2020-06-22 | 2024-03-22 | 上海柯林布瑞信息技术有限公司 | Data loading method and device, computing equipment and computer readable storage medium |
| CN111813783B (en) * | 2020-07-27 | 2024-03-26 | 南方电网数字电网研究院有限公司 | Data processing method, device, computer equipment and storage medium |
| CN112286932B (en) * | 2020-10-28 | 2021-09-14 | 况客科技（北京）有限公司 | Data processing system |
| CN112420168B (en) * | 2020-11-12 | 2023-06-23 | 武汉联影医疗科技有限公司 | Method, device, equipment and storage medium for writing data into database |
| CN112685417B (en) * | 2020-12-30 | 2024-04-05 | 京东科技控股股份有限公司 | Database operation method, system, device, server and storage medium |
| CN112801136B (en) * | 2020-12-31 | 2024-04-26 | 广州技象科技有限公司 | Internet of things gateway data processing method and device with characteristic identification |
| CN113761016A (en) * | 2021-01-15 | 2021-12-07 | 北京沃东天骏信息技术有限公司 | Data query method, device, equipment and storage medium |
| CN113094370A (en) * | 2021-04-13 | 2021-07-09 | 北京精鸿软件科技有限公司 | Data index construction method and device, storage medium and electronic equipment |
| CN113377809A (en) * | 2021-06-23 | 2021-09-10 | 北京百度网讯科技有限公司 | Data processing method and apparatus, computing device, and medium |
| CN115391408A (en) * | 2022-10-27 | 2022-11-25 | 深圳市华曦达科技股份有限公司 | Video log tracking method and device and electronic equipment |
| CN117112632B (en) * | 2023-10-23 | 2024-01-12 | 北京纷扬科技有限责任公司 | Isolation method, device and storage medium for preventing data impact |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104750826A (en) * | 2015-03-31 | 2015-07-01 | 克拉玛依红有软件有限责任公司 | Structural data resource metadata automatically-identifying and dynamically-registering method |
| CN107391612A (en) * | 2017-07-04 | 2017-11-24 | 石化盈科信息技术有限责任公司 | Field configuration method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8195689B2 (en) * | 2009-06-10 | 2012-06-05 | Zeitera, Llc | Media fingerprinting and identification system |
-
2019
- 2019-06-26 CN CN201910562858.XA patent/CN110276002B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104750826A (en) * | 2015-03-31 | 2015-07-01 | 克拉玛依红有软件有限责任公司 | Structural data resource metadata automatically-identifying and dynamically-registering method |
| CN107391612A (en) * | 2017-07-04 | 2017-11-24 | 石化盈科信息技术有限责任公司 | Field configuration method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110276002A (en) | 2019-09-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110276002B (en) | Search application data processing method and device, computer equipment and storage medium | |
| CN110069449B (en) | File processing method, device, computer equipment and storage medium | |
| CN110287192B (en) | Search application data processing method and device, computer equipment and storage medium | |
| US10725981B1 (en) | Analyzing big data | |
| US9361320B1 (en) | Modeling big data | |
| CN111177302B (en) | Service bill processing method, device, computer equipment and storage medium | |
| US20140046928A1 (en) | Query plans with parameter markers in place of object identifiers | |
| US8880463B2 (en) | Standardized framework for reporting archived legacy system data | |
| CN110888837B (en) | Object storage small file merging method and device | |
| CN108614837B (en) | File storage and retrieval method and device | |
| CN111078559B (en) | Method, device, medium and computer equipment for extracting function call in java code | |
| CN109739828B (en) | Data processing method and device and computer readable storage medium | |
| CN109783457B (en) | CGI interface management method, device, computer equipment and storage medium | |
| CN111176901B (en) | HDFS deleted file recovery method, terminal device and storage medium | |
| JP4879193B2 (en) | System log management support apparatus and system log management support method | |
| CN112948504A (en) | Data acquisition method and device, computer equipment and storage medium | |
| CN112785248A (en) | Human resource data cross-organization interaction method, device, equipment and storage medium | |
| CN107430633B (en) | System and method for data storage and computer readable medium | |
| US20230153455A1 (en) | Query-based database redaction | |
| US10528229B2 (en) | Mandatory comment on action or modification | |
| CN109460500B (en) | Hotspot event discovery method and device, computer equipment and storage medium | |
| CN113391916A (en) | Organization architecture data processing method, device, computer equipment and storage medium | |
| CN112835886A (en) | Data table field adding method and device | |
| CN113515518A (en) | Data storage method and device, computer equipment and storage medium | |
| CN111813842B (en) | Data processing method, device, system, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |