CN113268500A - Service processing method and device and electronic equipment - Google Patents
Service processing method and device and electronic equipment Download PDFInfo
- Publication number
- CN113268500A CN113268500A CN202110638853.8A CN202110638853A CN113268500A CN 113268500 A CN113268500 A CN 113268500A CN 202110638853 A CN202110638853 A CN 202110638853A CN 113268500 A CN113268500 A CN 113268500A
- Authority
- CN
- China
- Prior art keywords
- data
- service
- metadata information
- task
- information set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The embodiment of the invention provides a service processing method, a service processing device and electronic equipment, wherein the method comprises the steps of receiving a data query request corresponding to a service to be processed, wherein the data query request comprises a service identifier, acquiring service source data corresponding to the service identifier from a first type database and a second type database according to the data query request, wherein the service source data comprises basic service data and a task scheduling relation, and processing the basic service data according to the task scheduling relation, so that the service to be processed is realized, the efficiency of data query and calling is improved, the waiting time of a user is reduced, and the use experience of the user is improved.
Description
Technical Field
The embodiment of the invention relates to the technical field of data processing, in particular to a service processing method and device and electronic equipment.
Background
With the development of the internet, the number of users and the number of information source nodes are increased explosively, so that the data form is diversified. Although the relational database has extremely strong management capability, information sharing and data exchange are seriously influenced due to differences among various database systems and heterogeneous types of the relied-on operating systems.
In the prior art, when related services are realized, a plurality of reports are often needed, and data personnel can also extract various data and task scheduling information for processing the data and uniformly store the extracted data and task scheduling information in a database.
However, since the extracted data and the task scheduling information are various, in the subsequent query or call process, it takes a long time to acquire the corresponding data from the database, so that the efficiency of data query and call is reduced, the waiting time of the user is increased, and the use experience of the user is reduced.
Disclosure of Invention
The embodiment of the invention provides a service processing method, a service processing device and electronic equipment, and aims to improve the efficiency of data query and calling.
In a first aspect, an embodiment of the present invention provides a service processing method, including:
receiving a data query request corresponding to a service to be processed, wherein the data query request comprises a service identifier;
acquiring service source data corresponding to the service identifier from a first type database and a second type database according to the data query request, wherein the service source data comprises basic service data and a task scheduling relation;
and processing the basic service data according to the task scheduling relationship so as to realize the service to be processed.
Optionally, the first type database is a search type database, the second type database is a graph type database, and the service source data corresponding to the service identifier is obtained from the first type database and the second type database according to the data query request, where the service source data includes basic service data and a task scheduling relationship, including:
acquiring service source data corresponding to the service identifier from the search class database and the graph class database according to the data query request;
the service source data comprises basic service data and a task scheduling relation, wherein the basic service data comprises basic service data of a graphic type and basic service data of a character type.
Optionally, before the receiving the data query request corresponding to the service to be processed, the method further includes:
acquiring metadata information stored in a report system, a scheduling system and different types of data storage systems to obtain a metadata information set;
integrating and classifying different types of metadata information in the metadata information sets according to a preset data integration frame to obtain a first sub-metadata information set and a second sub-metadata information set after integration;
and storing the integrated first sub-metadata information set into the first type database, and storing the integrated second sub-metadata information set into the second type database.
Optionally, the integrating and classifying different types of metadata information in the metadata information sets according to a preset data integration frame to obtain an integrated first sub-metadata information set and a second sub-metadata information set, including:
acquiring a target task dependency chain corresponding to different types of metadata information in the metadata information set;
acquiring all task variables from the target task dependency chain;
judging whether each task variable is a preset type of task variable to obtain a judgment result;
and if the judgment result is that the task variable is a preset type of task variable, integrating and classifying the metadata information corresponding to the task variable in the metadata information set according to a preset data integration frame to obtain a first sub-metadata information set and a second sub-metadata information set after integration.
Optionally, the method further includes:
and if the judgment result is that the task variable is not the task variable of the preset type, deleting the metadata information corresponding to the task variable in the metadata information set.
Optionally, the metadata information includes at least one data table, and each data table includes metadata, a column number, a comment of the table itself, and a column comment.
In a second aspect, an embodiment of the present invention provides a service processing apparatus, including:
the system comprises a receiving module, a processing module and a processing module, wherein the receiving module is used for receiving a data query request corresponding to a service to be processed, and the data query request comprises a service identifier;
the processing module is used for acquiring service source data corresponding to the service identifier from a first type database and a second type database according to the data query request, wherein the service source data comprises basic service data and a task scheduling relation;
and the processing module is also used for processing the basic service data according to the task scheduling relationship so as to realize the service to be processed.
In a third aspect, an embodiment of the present invention provides an electronic device, including: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executing the computer-executable instructions stored by the memory causes the at least one processor to perform the business process method of any of the first aspects.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer executes instructions, and when a processor executes the computer to execute the instructions, the service processing method according to any one of the first aspect is implemented.
In a fifth aspect, an embodiment of the present invention provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the business processing method according to the first aspect and various possible designs of the first aspect is implemented.
The embodiment of the invention provides a service processing method, a device and electronic equipment, after the scheme is adopted, a data query request which is corresponding to a service to be processed and contains a service identifier can be received, then service source data which corresponds to the service identifier is obtained from a first type database and a second type database according to the data query request, wherein the service source data contains basic service data and a task scheduling relation, the basic service data is processed according to the task scheduling relation, the service to be processed is further realized, different types of service source data are stored by adopting two different types of databases of the first type database and the second type database, different query and calling modes are adopted for the different types of service source data, the efficiency of data query and calling is improved, and the waiting time of a user is further reduced, the use experience of the user is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a schematic structural diagram of an application system of a service processing method according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a service processing method according to an embodiment of the present invention;
fig. 3 is a schematic diagram illustrating an application of the parsing process according to the embodiment of the present invention;
FIG. 4 is a schematic diagram of a metadata structure provided by an embodiment of the invention;
fig. 5 is a schematic structural diagram of a service processing apparatus according to an embodiment of the present invention;
fig. 6 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of including other sequential examples in addition to those illustrated or described. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
In the prior art, when related services are realized, a plurality of reports are often needed, and data personnel can also extract various data and task scheduling information for processing the data and uniformly store the extracted data and task scheduling information in a database. However, since the extracted data and the task scheduling information are various, it takes a long time to acquire the corresponding data from the database in the subsequent query or call process. Illustratively, the extracted data and task scheduling information include two types, namely type a and type B, and the data and task scheduling information corresponding to the type a and the type B are all uniformly stored in one database, and then the data storage mode corresponding to the database may not be suitable for the storage of the type a or type B data, so that in the subsequent query or call process, a long time is consumed to acquire the corresponding data from the database, the efficiency of data query and call is reduced, the waiting time of a user is increased, and the use experience of the user is reduced.
Based on the above problems, the method and the device for storing the service source data in different types adopt the two different types of databases, namely the first type database and the second type database, so that different types of service source data are inquired and called in different manners, the data inquiry and calling efficiency is improved, the waiting time of a user is shortened, and the use experience of the user is improved.
Fig. 1 is a schematic architecture diagram of an application system of a service processing method according to an embodiment of the present invention, and as shown in fig. 1, the application system may include: a search server 101, a metadata server 102, a front-end server 103, a data source 104, and a server 105. The search server 101 and the metadata server 102 may correspond to different types of databases, and are used to implement storage of different types of basic service data and task scheduling relationships. The front-end server 103 is responsible for the Web UI interacting with the user, and may be, for example, a flash-based Web application, and the display layer of the front-end server is constructed by using read with reduce, boottrap, Webpack, and Babel. The data source 104 can be a reporting system, a scheduling system, and different types of data storage systems, and illustratively can be Oracle, Hive, MySQL, and the like. The server 105 may obtain corresponding data from the search server 101 and the metadata server 102 according to the received data query request corresponding to the service to be processed, and then further process the data according to the obtained data, thereby implementing the service to be processed.
The search server 101 and the metadata server 102 may be deployed separately or collectively in the server 105. In this embodiment, the search server 101 and the metadata server 102 are deployed separately.
The technical solution of the present invention will be described in detail below with specific examples. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments.
Fig. 2 is a schematic flow diagram of a service processing method provided in an embodiment of the present invention, where the method in this embodiment may be executed by a server, as shown in fig. 2, the method in this embodiment may include:
s201: and receiving a data query request corresponding to the service to be processed, wherein the data query request comprises a service identifier.
In this embodiment, a plurality of to-be-processed services may be displayed in the front-end server, and the plurality of to-be-processed services may be arranged and displayed according to a preset display style, and when a user wants to implement any one of the to-be-processed services, the to-be-processed service may be triggered in a touch operation manner.
Further, the touch operation may be a single-click operation or a double-click operation with a mouse, or a single-click operation, a double-click operation, a long-press operation, or a long-press operation on a touch screen.
The server may receive a data query request corresponding to a to-be-processed service triggered by a touch operation, where the data query request may include a service identifier for uniquely identifying the to-be-processed service.
The type of the service identifier may be set by a user, and is not limited in detail herein.
S202: and acquiring service source data corresponding to the service identifier from the first type database and the second type database according to the data query request, wherein the service source data comprises basic service data and a task scheduling relation.
In this embodiment, different types of service source data may be stored in different types of databases, so that the efficiency of subsequent service source data query and scheduling may be improved.
Further, if the first type database is a search type database and the second type database is a graph type database, the service source data corresponding to the service identifier is obtained from the first type database and the second type database according to the data query request, where the service source data includes basic service data and a task scheduling relationship, and the method specifically includes:
and acquiring service source data corresponding to the service identification from the search database and the graph database according to the data query request.
The service source data comprises basic service data and a task scheduling relation, and the basic service data comprises basic service data of a graphic type and basic service data of a character type.
Specifically, the service source data is stored in a mode of combining the search class database and the graph class database, so that most types of data can be covered, and the query and scheduling efficiency of the data is obviously improved. Illustratively, the search class database may be an Elasticsearch, the graph class database may be a Neo4J graph database, and the structured data may be stored in a network.
In addition, the more kinds of the stored databases, the more pressure the server calls may be, most of the data query and call cases may be satisfied by storing the data in two types of databases, and in a special case, the data may also be stored in at least three types of databases.
S203: and processing the basic service data according to the task scheduling relation, thereby realizing the service to be processed.
In this embodiment, the service source data may include basic service data and a task scheduling relationship, where the basic service data may be various types of data required for implementing the service to be processed, and may be, for example, text, an image, or the like. The task scheduling relationship is an overall process for implementing the to-be-processed service, illustratively, the to-be-processed service includes three sub-service modules, and the task scheduling relationship is a scheduling relationship among the three sub-service modules. Correspondingly, after the task scheduling relationship is determined, the basic service data can be processed according to the task scheduling relationship, and the service to be processed is further realized.
After the scheme is adopted, the data query request which is corresponding to the service to be processed and contains the service identifier can be received firstly, then the service source data which is corresponding to the service identifier is obtained from the first type database and the second type database according to the data query request, the basic service data and the task scheduling relation are contained in the service source data, then the basic service data are processed according to the task scheduling relation, the service to be processed is further realized, different query and calling modes are adopted for the different types of service source data through the mode that the different types of service source data are stored by adopting the first type database and the second type database, the efficiency of data query and calling is improved, the waiting time of a user is reduced, and the use experience of the user is improved.
Based on the method of fig. 2, the present specification also provides some specific embodiments of the method, which are described below.
In addition, in another embodiment, before S201, the method may further include:
and acquiring metadata information stored in a report system, a scheduling system and different types of data storage systems to obtain a metadata information set.
And integrating and classifying different types of metadata information in the metadata information sets according to a preset data integration frame to obtain a first sub-metadata information set and a second sub-metadata information set after integration.
And storing the integrated first sub-metadata information set into a first type database, and storing the integrated second sub-metadata information set into a second type database.
In this embodiment, the metadata information set may include a reporting system, a scheduling system, and different types of storage systems, where the different types of storage systems may be Oracle, Hive, MySQL, and the like.
Further, according to a preset data integration framework, integrating and classifying different types of metadata information in the metadata information sets to obtain an integrated first sub-metadata information set and a second sub-metadata information set, which may specifically include:
and acquiring a target task dependency chain corresponding to different types of metadata information in the metadata information set.
And acquiring all task variables from the target task dependency chain.
And judging whether the task variable is a preset type of task variable or not according to each task variable to obtain a judgment result.
And if the judgment result is that the task variable is a preset type of task variable, integrating and classifying the metadata information corresponding to the task variable in the metadata information set according to a preset data integration frame to obtain a first sub-metadata information set and a second sub-metadata information set after integration.
Further, the method may further include:
and if the judgment result is that the task variable is not the task variable of the preset type, deleting the metadata information corresponding to the task variable in the metadata information set.
Specifically, different types of metadata information in the metadata information set may correspond to different target task dependency chains, and the specific corresponding rule may be determined by using an existing corresponding manner, which is not discussed in detail herein. Illustratively, the target task dependency chain may be a DAG in the airflow, and specifically may be a directed acyclic graph composed of a plurality of task dependencies.
The metadata service interacts with the Neo4j graph database using the Python language interface of Neo4j and provides metadata for front-end services. In the metadata model, a table is the core of the entire metadata model, and first, the table is stored in a database of some type, which may be Oracle, Impala, or the like. Because the metadata that is searched for in data discovery may be stored in different types of databases, each database is named for differentiation. The database itself also includes a similar schema, distinguishing the database tables. In summary, we use the database type, the database name and the schema table to make unique distinction, and the specific and only unique table can be located by the database type, the database name and the schema.
When the generation of the table is determined, it is also necessary to determine where the data in the table comes. Scheduling tasks associated with table generation can be found by parsing the Airflow DAG file. Fig. 3 is an application schematic diagram of an analysis process provided in an embodiment of the present invention, and as shown in fig. 3, in this embodiment, all task variables are obtained from a DAG file of an Airflow, where, for example, the task variables may be operators, and for each Operator, whether the Operator is a preset type of task variable may be determined, and for example, whether the Operator is an SQL Operator or a DataX Operator may be determined. And if the metadata information is not the SQL Operator or the DataX Operator, the Operator is not processed any more, and the metadata information corresponding to the Operator is deleted. On the contrary, if the SQL Operator is used, the SQL statement is extracted from the Operator, the SQL statement is analyzed by a Parse method, and the TABLE name behind the INTO/OVERWRETE/TABLE is extracted from the SQL statement. If the data operation is the DataX Operator, Json is analyzed, and then the table name corresponding to the metadata information is extracted according to the appointed path, so that the subsequent calling is convenient when the to-be-processed service is realized.
In addition, the data integration framework can adopt an existing framework and is responsible for extracting metadata from a data source and importing the metadata into a first type database and a second type database, and exemplarily can be imported into Neo4J and an Elasticissearch database.
Furthermore, in another embodiment, the metadata information includes at least one data table, each of which includes metadata, a column number, a comment on the table itself, and a column comment.
In this embodiment, the metadata information may be composed of different data tables, and each data table may further include, in addition to its own metadata, a column number, a table own comment, and a column comment, which facilitates description of the table from multiple dimensions for subsequent invocation.
Fig. 4 is a schematic diagram of a metadata structure according to an embodiment of the present invention, as shown in fig. 4, in this embodiment, data related to a table may be extracted from a preset metadata base, so that which billboard the table may be applied to may be determined. Illustratively, it may be a Superset metadata repository. In addition, meta information of the table, such as which columns are in the table, comments on the table itself, and column comments, may also be included in the table.
Based on the same idea, an embodiment of this specification further provides a device corresponding to the foregoing method, and fig. 5 is a schematic structural diagram of a service processing device provided in an embodiment of the present invention, and as shown in fig. 5, the method may include:
the receiving module 501 is configured to receive a data query request corresponding to a service to be processed, where the data query request includes a service identifier.
A processing module 502, configured to obtain, according to the data query request, service source data corresponding to the service identifier from a first type database and a second type database, where the service source data includes basic service data and a task scheduling relationship.
In this embodiment, the first type database is a search type database, the second type database is a graph type database, and the processing module 502 is further configured to:
and acquiring service source data corresponding to the service identification from the search class database and the graph class database according to the data query request.
The service source data comprises basic service data and a task scheduling relation, wherein the basic service data comprises basic service data of a graphic type and basic service data of a character type.
The processing module 502 is further configured to process the basic service data according to the task scheduling relationship, so as to implement the service to be processed.
Moreover, in another embodiment, the processing module 502 is further configured to:
and acquiring metadata information stored in a report system, a scheduling system and different types of data storage systems to obtain a metadata information set.
And integrating and classifying different types of metadata information in the metadata information sets according to a preset data integration frame to obtain a first sub-metadata information set and a second sub-metadata information set after integration.
And storing the integrated first sub-metadata information set into the first type database, and storing the integrated second sub-metadata information set into the second type database.
Moreover, in another embodiment, the processing module 502 is further configured to:
and acquiring a target task dependency chain corresponding to different types of metadata information in the metadata information set.
And acquiring all task variables from the target task dependency chain.
And judging whether the task variable is a preset type of task variable or not according to each task variable to obtain a judgment result.
And if the judgment result is that the task variable is a preset type of task variable, integrating and classifying the metadata information corresponding to the task variable in the metadata information set according to a preset data integration frame to obtain a first sub-metadata information set and a second sub-metadata information set after integration.
In addition, the processing module 502 is further configured to:
and if the judgment result is that the task variable is not the task variable of the preset type, deleting the metadata information corresponding to the task variable in the metadata information set.
The metadata information comprises at least one data table, and each data table comprises metadata, column numbers, notes of the table and column notes.
The apparatus provided in the embodiment of the present invention may implement the method in the embodiment shown in fig. 2, and the implementation principle and the technical effect are similar, which are not described herein again.
Fig. 6 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present invention, and as shown in fig. 6, a device 600 according to the embodiment includes: at least one processor 601 and memory 602. The processor 601 and the memory 602 are connected by a bus 603.
In a specific implementation, at least one processor 601 executes computer-executable instructions stored by the memory 602 to cause the at least one processor 601 to perform the methods of the above-described method embodiments.
For a specific implementation process of the processor 601, reference may be made to the above method embodiments, which implement the principle and the technical effect similarly, and details of this embodiment are not described herein again.
In the embodiment shown in fig. 6, it should be understood that the Processor may be a Central Processing Unit (CPU), other general purpose processors, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of a method disclosed in connection with the present invention may be embodied directly in a hardware processor, or in a combination of the hardware and software modules within the processor.
The memory may comprise high speed RAM memory and may also include non-volatile storage NVM, such as at least one disk memory.
The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, the buses in the figures of the present application are not limited to only one bus or one type of bus.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer execution instruction is stored in the computer-readable storage medium, and when a processor executes the computer execution instruction, the service processing method of the foregoing method embodiment is implemented.
An embodiment of the present invention further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the service processing method described above is implemented.
The computer-readable storage medium may be implemented by any type of volatile or non-volatile memory device or combination thereof, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk. Readable storage media can be any available media that can be accessed by a general purpose or special purpose computer.
An exemplary readable storage medium is coupled to the processor such the processor can read information from, and write information to, the readable storage medium. Of course, the readable storage medium may also be an integral part of the processor. The processor and the readable storage medium may reside in an Application Specific Integrated Circuits (ASIC). Of course, the processor and the readable storage medium may also reside as discrete components in the apparatus.
Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for processing a service, comprising:
receiving a data query request corresponding to a service to be processed, wherein the data query request comprises a service identifier;
acquiring service source data corresponding to the service identifier from a first type database and a second type database according to the data query request, wherein the service source data comprises basic service data and a task scheduling relation;
and processing the basic service data according to the task scheduling relationship so as to realize the service to be processed.
2. The method according to claim 1, wherein the first type database is a search type database, the second type database is a graph type database, and the obtaining of the service source data corresponding to the service identifier from the first type database and the second type database according to the data query request includes basic service data and a task scheduling relationship, including:
acquiring service source data corresponding to the service identifier from the search class database and the graph class database according to the data query request;
the service source data comprises basic service data and a task scheduling relation, wherein the basic service data comprises basic service data of a graphic type and basic service data of a character type.
3. The method according to claim 1 or 2, wherein before the receiving the data query request corresponding to the service to be processed, the method further comprises:
acquiring metadata information stored in a report system, a scheduling system and different types of data storage systems to obtain a metadata information set;
integrating and classifying different types of metadata information in the metadata information sets according to a preset data integration frame to obtain a first sub-metadata information set and a second sub-metadata information set after integration;
and storing the integrated first sub-metadata information set into the first type database, and storing the integrated second sub-metadata information set into the second type database.
4. The method according to claim 3, wherein the integrating and classifying different types of metadata information in the metadata information sets according to a preset data integration framework to obtain an integrated first sub-metadata information set and a second sub-metadata information set includes:
acquiring a target task dependency chain corresponding to different types of metadata information in the metadata information set;
acquiring all task variables from the target task dependency chain;
judging whether each task variable is a preset type of task variable to obtain a judgment result;
and if the judgment result is that the task variable is a preset type of task variable, integrating and classifying the metadata information corresponding to the task variable in the metadata information set according to a preset data integration frame to obtain a first sub-metadata information set and a second sub-metadata information set after integration.
5. The method of claim 4, further comprising:
and if the judgment result is that the task variable is not the task variable of the preset type, deleting the metadata information corresponding to the task variable in the metadata information set.
6. The method of claim 3, wherein the metadata information comprises at least one data table, each of the data tables comprising metadata, a column number, a comment on the table itself, and a column comment.
7. A traffic processing apparatus, comprising:
the system comprises a receiving module, a processing module and a processing module, wherein the receiving module is used for receiving a data query request corresponding to a service to be processed, and the data query request comprises a service identifier;
the processing module is used for acquiring service source data corresponding to the service identifier from a first type database and a second type database according to the data query request, wherein the service source data comprises basic service data and a task scheduling relation;
and the processing module is also used for processing the basic service data according to the task scheduling relationship so as to realize the service to be processed.
8. An electronic device, comprising: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executing the computer-executable instructions stored by the memory causes the at least one processor to perform the business process method of any of claims 1 to 6.
9. A computer-readable storage medium, having stored thereon computer-executable instructions, which, when executed by a processor, implement the business process method of any one of claims 1 to 6.
10. A computer program product comprising a computer program, characterized in that the computer program realizes the business process method of any of claims 1 to 6 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110638853.8A CN113268500B (en) | 2021-06-08 | 2021-06-08 | Service processing method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110638853.8A CN113268500B (en) | 2021-06-08 | 2021-06-08 | Service processing method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113268500A true CN113268500A (en) | 2021-08-17 |
| CN113268500B CN113268500B (en) | 2023-06-02 |
Family
ID=77234546
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110638853.8A Active CN113268500B (en) | 2021-06-08 | 2021-06-08 | Service processing method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113268500B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115145992A (en) * | 2022-09-05 | 2022-10-04 | 深圳市明源云科技有限公司 | Multi-source data processing method and device, electronic equipment and readable storage medium |
| CN115269561A (en) * | 2022-09-21 | 2022-11-01 | 国网智能电网研究院有限公司 | Hybrid database management method and device, hybrid database and electronic equipment |
| CN115375380A (en) * | 2022-10-26 | 2022-11-22 | 南昌惠联网络技术有限公司 | Service data processing method and processing device based on attribute classification |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110072465A1 (en) * | 2009-09-20 | 2011-03-24 | Lg Electronics Inc. | Method of processing epg metadata in network device and the network device for controlling the same |
| CN103970899A (en) * | 2014-05-27 | 2014-08-06 | 重庆大学 | Service-oriented metadata relevance extraction management method and management system |
| CN106815270A (en) * | 2015-12-02 | 2017-06-09 | 大唐移动通信设备有限公司 | A kind of method and device for servicing dynamic management |
| CN108701325A (en) * | 2015-12-21 | 2018-10-23 | 星茶瓦公司 | The transaction system that is self-regulated and the method for the system |
| CN109857756A (en) * | 2019-01-14 | 2019-06-07 | 中国联合网络通信集团有限公司 | Generation method, device, equipment and the readable storage medium storing program for executing of Database Report |
| US10673716B1 (en) * | 2017-10-31 | 2020-06-02 | Amazon Technologies, Inc. | Graph-based generation of dependency-adherent execution plans for data center migrations |
| CN111708794A (en) * | 2020-06-22 | 2020-09-25 | 中国平安财产保险股份有限公司 | Data comparison method and device based on big data platform and computer equipment |
| CN112507010A (en) * | 2020-12-14 | 2021-03-16 | 深圳佑驾创新科技有限公司 | Service data processing method and device, computer equipment and storage medium |
-
2021
- 2021-06-08 CN CN202110638853.8A patent/CN113268500B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110072465A1 (en) * | 2009-09-20 | 2011-03-24 | Lg Electronics Inc. | Method of processing epg metadata in network device and the network device for controlling the same |
| CN103970899A (en) * | 2014-05-27 | 2014-08-06 | 重庆大学 | Service-oriented metadata relevance extraction management method and management system |
| CN106815270A (en) * | 2015-12-02 | 2017-06-09 | 大唐移动通信设备有限公司 | A kind of method and device for servicing dynamic management |
| CN108701325A (en) * | 2015-12-21 | 2018-10-23 | 星茶瓦公司 | The transaction system that is self-regulated and the method for the system |
| US10673716B1 (en) * | 2017-10-31 | 2020-06-02 | Amazon Technologies, Inc. | Graph-based generation of dependency-adherent execution plans for data center migrations |
| CN109857756A (en) * | 2019-01-14 | 2019-06-07 | 中国联合网络通信集团有限公司 | Generation method, device, equipment and the readable storage medium storing program for executing of Database Report |
| CN111708794A (en) * | 2020-06-22 | 2020-09-25 | 中国平安财产保险股份有限公司 | Data comparison method and device based on big data platform and computer equipment |
| CN112507010A (en) * | 2020-12-14 | 2021-03-16 | 深圳佑驾创新科技有限公司 | Service data processing method and device, computer equipment and storage medium |
Non-Patent Citations (3)
| Title |
|---|
| ISA INUWA-DUTSE ET AL.: "Detection of spam-posting accounts on Twitter", 《NEUROCOMPUTING》, pages 496 - 511 * |
| 程珅: "数据网格中目录服务的研究与应用", 《中国优秀博硕士学位论文全文数据库 (硕士) 信息科技辑》, pages 139 - 159 * |
| 苏俊妮: "基于继电保护故障及信息系统的定值自动校核", 《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》, pages 042 - 574 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115145992A (en) * | 2022-09-05 | 2022-10-04 | 深圳市明源云科技有限公司 | Multi-source data processing method and device, electronic equipment and readable storage medium |
| CN115145992B (en) * | 2022-09-05 | 2022-12-27 | 深圳市明源云科技有限公司 | Multi-source data processing method and device, electronic equipment and readable storage medium |
| CN115269561A (en) * | 2022-09-21 | 2022-11-01 | 国网智能电网研究院有限公司 | Hybrid database management method and device, hybrid database and electronic equipment |
| CN115269561B (en) * | 2022-09-21 | 2023-01-24 | 国网智能电网研究院有限公司 | Hybrid database management method and device, hybrid database and electronic equipment |
| WO2024060956A1 (en) * | 2022-09-21 | 2024-03-28 | 国网智能电网研究院有限公司 | Hybrid database management method and apparatus, hybrid database, and electronic device |
| CN115375380A (en) * | 2022-10-26 | 2022-11-22 | 南昌惠联网络技术有限公司 | Service data processing method and processing device based on attribute classification |
| CN115375380B (en) * | 2022-10-26 | 2023-02-03 | 南昌惠联网络技术有限公司 | Service data processing method and processing device based on attribute classification |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113268500B (en) | 2023-06-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111177176B (en) | Data detection method, device and storage medium | |
| CN113268500B (en) | Service processing method and device and electronic equipment | |
| US8983895B2 (en) | Representation of multiplicities for Docflow reporting | |
| CN111177231A (en) | Report generation method and report generation device | |
| JP2020522790A (en) | Automatic dependency analyzer for heterogeneously programmed data processing systems | |
| CN108171528B (en) | Attribution method and attribution system | |
| CN110795455A (en) | Dependency relationship analysis method, electronic device, computer device and readable storage medium | |
| CN110795697A (en) | Logic expression obtaining method and device, storage medium and electronic device | |
| CN108345658B (en) | Decomposition processing method of algorithm calculation track, server and storage medium | |
| CN113297458A (en) | Paging query method, device and equipment | |
| CN112650909A (en) | Product display method and device, electronic equipment and storage medium | |
| CN114003818A (en) | Page recommendation method, device, server and storage medium | |
| CN110019182B (en) | Data tracing method and device | |
| CN112347748A (en) | Data report generation method and device, computer equipment and readable storage medium | |
| CN111553133B (en) | Report generation method and device, electronic equipment and storage medium | |
| CN117009397A (en) | Data query method, data query device, electronic equipment and storage medium | |
| CN104750823B (en) | Method and device for inquiring promotion condition data | |
| CN114328149B (en) | Method and device for counting compiling duration, electronic equipment and readable medium | |
| CN112307019B (en) | Data processing method, device and non-transitory computer readable storage medium | |
| CN111311329B (en) | Tag data acquisition method, device, equipment and readable storage medium | |
| US11960560B1 (en) | Methods for analyzing recurring accessibility issues with dynamic web site behavior and devices thereof | |
| US20240104297A1 (en) | Analysis of spreadsheet table in response to user input | |
| CN114489886A (en) | Resource page generation method, device, equipment and medium | |
| CN115481147A (en) | Drive relationship searching method and device, electronic equipment and storage medium | |
| CN110956269A (en) | Data model generation method, device, equipment and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |