CN112732758A - Heterogeneous data processing method and device, electronic equipment and readable storage medium - Google Patents
Heterogeneous data processing method and device, electronic equipment and readable storage medium Download PDFInfo
- Publication number
- CN112732758A CN112732758A CN202011637761.XA CN202011637761A CN112732758A CN 112732758 A CN112732758 A CN 112732758A CN 202011637761 A CN202011637761 A CN 202011637761A CN 112732758 A CN112732758 A CN 112732758A
- Authority
- CN
- China
- Prior art keywords
- data
- topic
- heterogeneous
- heterogeneous data
- data source
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
Abstract
The invention provides a heterogeneous data processing method and device, electronic equipment and a readable storage medium. The heterogeneous data processing platform is provided with at least two heterogeneous data sources, first input of data topics can be received on the heterogeneous data processing platform, data query information is matched according to topic identification in response to the first input, and since the data topics correspond to at least one heterogeneous data source, heterogeneous data can be queried in the heterogeneous data sources corresponding to the data topics according to the data query information. At this moment, since the heterogeneous data source is accessed to the heterogeneous data display platform, the heterogeneous data source can be directly accessed through the heterogeneous data display platform after the thematic query information is determined, and data services are not required to be provided after heterogeneous data in different heterogeneous data sources are moved and integrated to a data center, so that the operation, maintenance and data integration costs of the platform are reduced, and the efficiency of analysis and display of the heterogeneous data is improved.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a heterogeneous data processing method and apparatus, an electronic device, and a readable storage medium.
Background
Different heterogeneous data refers to service data adopting different storage modes, and is usually generated in a large amount in an accumulated manner due to the influences of construction, implementation stages, technical levels, economic conditions and other human factors of a service system, a data management system and the like in the data informatization process, and the data management systems adopted by different heterogeneous data are different, and can be a simple file database or a complex network database.
At present, in order to perform unified management and display on different heterogeneous data, a data center is generally established, data is extracted from data sources of each heterogeneous data and is stored in a centralized manner, and then corresponding data services are provided. However, as the data types and data volumes of the heterogeneous data gradually increase, the operation, maintenance and data integration costs of the data center gradually increase, the modeling difficulty gradually increases, and the data migration difficulty is difficult, so that the efficiency of analyzing, integrating and displaying the heterogeneous data is seriously affected.
Disclosure of Invention
The invention provides a heterogeneous data processing method, a heterogeneous data processing device, electronic equipment and a readable storage medium, and aims to solve the problems that in the prior art, heterogeneous data among heterogeneous data sources in multiple storage modes are difficult to analyze, integrate and display, high in cost and low in efficiency.
In order to solve the above problems, the present invention is realized by:
in a first aspect, an embodiment of the present invention provides a heterogeneous data processing method, which is applied to a heterogeneous data processing platform, where the heterogeneous data processing platform accesses at least two heterogeneous data sources, and the at least two heterogeneous data sources respectively store heterogeneous data in different storage manners, where the method includes: receiving a first input of a data topic, wherein the data topic corresponds to at least one heterogeneous data source; responding to the first input, and matching data query information corresponding to the data topic according to the topic identification corresponding to the data topic; and querying in a heterogeneous data source corresponding to the data topic according to the data query information to obtain the heterogeneous data.
Optionally, before the receiving the first input of the data topic, the method further includes:
displaying a data topic list, wherein the data topic list comprises at least one data topic corresponding to a heterogeneous data source in a user permission range;
the receiving a first input of a data topic, comprising:
receiving a selection operation of the data topic in the data topic list as the first input.
Optionally, the obtaining of the heterogeneous data by querying in the heterogeneous data source corresponding to the data topic according to the data query information includes:
generating query conditions according to the data query information, wherein the data query information comprises at least one of an address, a storage mode, a database table and an access mode of the heterogeneous data source, and the access mode comprises any one of remote access and local access;
and accessing the heterogeneous data source according to the query condition, and querying to obtain heterogeneous data.
Optionally, after receiving the first input of the data topic, the method further includes:
receiving a second input to the data topic;
in response to the second input, performing an editing operation on the data topic, wherein the editing operation comprises at least one of adding, modifying and deleting field filtering conditions when the heterogeneous data is queried;
optionally, the obtaining of the heterogeneous data by querying in the heterogeneous data source corresponding to the data topic according to the data query information includes:
and querying a heterogeneous data source corresponding to the data topic according to the data query information and the field filtering condition to obtain heterogeneous data.
Optionally, before the receiving the first input of the data topic, the method further includes:
accessing a heterogeneous data source by adopting a preset access mode, wherein the preset access mode comprises any one of remote access and local access;
acquiring the address of the heterogeneous data source;
introducing a corresponding database table according to the heterogeneous data source, and determining a storage mode of the database under the condition that the database is in a specified storage mode;
setting thematic identifiers corresponding to data thematic, and acquiring data query information corresponding to the data thematic, wherein the thematic identifiers comprise at least one of thematic names, thematic contents and thematic classifications, and the data query information comprises at least one of the address of the heterogeneous data source, the database table, the storage mode and an access mode corresponding to the preset access mode;
and creating the data topic according to the topic identification and the data query information.
Optionally, after the accessing the heterogeneous data source by using the predetermined access method, the method further includes:
and under the condition that any one of the accessed heterogeneous data sources is repeated with the accessed heterogeneous data source, displaying data source information of the accessed heterogeneous data source, wherein the data source information comprises at least one of an operation object, operation time, a backup mode, an access record and an exception notification.
Optionally, after the corresponding database table is introduced according to the heterogeneous data source, the method includes:
setting the read-write permission of the database table according to the user permission corresponding to the heterogeneous data source;
after the data topic is created according to the topic identification and the data query information, the method further comprises the following steps:
and determining an open state corresponding to the data topic, wherein the open state comprises at least one of closing and starting.
In a second aspect, an embodiment of the present invention provides a heterogeneous data processing apparatus, which is applied to a heterogeneous data processing platform, where the heterogeneous data processing platform accesses at least two heterogeneous data sources, and the at least two heterogeneous data sources respectively store heterogeneous data in different storage manners, where the method includes: the system comprises a first input receiving module, a first data processing module and a first data processing module, wherein the first input receiving module is used for receiving first input of data topics, and the data topics correspond to at least one heterogeneous data source; the data thematic matching module is used for responding to the first input and matching data query information corresponding to the data thematic according to the thematic identification corresponding to the data thematic; and the heterogeneous data query module is used for querying the heterogeneous data in the heterogeneous data source corresponding to the data topic according to the data query information to obtain the heterogeneous data.
Optionally, the apparatus further comprises:
and the data list display module is used for displaying a data topic list, wherein the data topic list comprises at least one data topic corresponding to a heterogeneous data source in a user permission range.
The first input receiving module is specifically configured to receive a selection operation of the data topic in the data topic list as the first input.
Optionally, the heterogeneous data query module includes:
the query condition generation sub-module is used for generating a query condition according to the data query information, wherein the data query information comprises at least one of an address, a storage mode, a database table and an access mode of the heterogeneous data source, and the access mode comprises any one of remote access and local access;
and the heterogeneous data query submodule is used for accessing a heterogeneous data source according to the query condition and querying to obtain heterogeneous data.
Optionally, the apparatus further comprises:
the second input receiving module is used for receiving second input of the data topic;
the field filtering editing module is used for responding to the second input and executing editing operation on the data topic, wherein the editing operation comprises at least one of adding, modifying and deleting field filtering conditions when the heterogeneous data is inquired;
optionally, the heterogeneous data query module is specifically configured to query, according to the data query information and the field filtering condition, a heterogeneous data source corresponding to the data topic, so as to obtain heterogeneous data.
Optionally, the apparatus further comprises:
the heterogeneous data source access module is used for accessing a heterogeneous data source by adopting a preset access mode, wherein the preset access mode comprises any one of remote access and local access;
the heterogeneous data source address acquisition module is used for acquiring the address of the heterogeneous data source;
the database table introducing module is used for introducing a corresponding database table according to the heterogeneous data source and determining a storage mode of the database under the condition that the database is in a specified storage mode;
the data topic information acquisition module is used for setting topic identifications corresponding to data topics and acquiring data query information corresponding to the data topics, wherein the topic identifications comprise at least one of topic names, topic contents and topic classifications, and the data query information comprises at least one of the addresses of the heterogeneous data sources, the database table, the storage modes and access modes corresponding to the preset access modes;
and the data topic creating module is used for creating the data topic according to the topic identification and the data query information.
Optionally, the apparatus further comprises:
the heterogeneous data source verification module is used for displaying data source information of the accessed heterogeneous data source under the condition that any one of the accessed heterogeneous data sources is repeated with the accessed heterogeneous data source, and the data source information comprises at least one of an operation object, operation time, a backup mode, an access record and an exception notification.
Optionally, the apparatus further comprises:
and the database table authority determining module is used for setting the read-write authority of the database table according to the user authority corresponding to the heterogeneous data source.
Optionally, the apparatus further comprises:
and the data topic state determining module is used for determining an open state corresponding to the data topic, wherein the open state comprises at least one of closing and starting.
In a third aspect, an embodiment of the present invention provides an electronic device, including: a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the steps of the heterogeneous data processing method of any of the above.
In a fourth aspect, the embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when being executed by a processor, the computer program implements the steps of the heterogeneous data processing method described in any one of the above.
Compared with the prior art, the invention has the following advantages:
in the embodiment of the invention, a heterogeneous data processing platform accessed with at least two heterogeneous data sources is provided, a first input for a data topic can be received on the heterogeneous data processing platform, the first input is responded, data query information is matched according to a topic identifier corresponding to the data topic, and the data topic corresponds to at least one heterogeneous data source, so after the data query information is determined according to the topic identifier matched with the topic identifier, heterogeneous data can be queried in the heterogeneous data source corresponding to the data topic according to the data query information. At this moment, in the heterogeneous data sources with different storage modes among cross-platforms, cross-services and cross-regions, since the heterogeneous data sources are accessed to the heterogeneous data display platform, the heterogeneous data sources can be directly accessed through the heterogeneous data display platform after the thematic query information is determined, and data services are not required to be provided after the heterogeneous data in different heterogeneous data sources are moved and integrated to a data center, so that the operation, maintenance and data integration costs of the platform are reduced, and the analysis and display efficiency of the heterogeneous data is improved.
Drawings
Fig. 1 is a flowchart illustrating steps of a method for processing heterogeneous data according to an embodiment of the present invention;
FIG. 2 is a flow chart illustrating steps of another method for processing heterogeneous data according to an embodiment of the present invention;
fig. 3 is a flowchart illustrating that a heterogeneous data source accesses a heterogeneous data processing platform according to an embodiment of the present invention;
FIG. 4 is a flow chart illustrating a process for creating a data topic provided by an embodiment of the invention;
FIG. 5 is a flow chart illustrating a method for querying heterogeneous data according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram illustrating a heterogeneous data processing apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanying figures are described in detail below, and it is apparent that the embodiments described are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The following describes in detail a heterogeneous data processing method, an apparatus, an electronic device, and a readable storage medium provided in the embodiments of the present application with specific embodiments and application scenarios thereof in conjunction with the accompanying drawings.
Referring to fig. 1, a flowchart illustrating steps of a heterogeneous data processing method provided in an embodiment of the present invention is shown, and is applied to a heterogeneous data processing platform, where the heterogeneous data processing platform accesses at least two heterogeneous data sources, and the at least two heterogeneous data sources respectively store heterogeneous data in different storage manners, where the method specifically includes the following steps:
in the embodiment of the invention, the heterogeneous data processing platform can be accessed to more than two heterogeneous data sources, the service data stored in the heterogeneous data sources are heterogeneous data, wherein different heterogeneous data sources can be different databases under different servers, the storage modes of different databases are different, for example, the heterogeneous data sources can be databases such as greenplus Database, MySQL (relational Database), Oracle Database and the like, and specifically, the heterogeneous data processing platform can be connected with various different databases across platforms, services and regions through built-in various Database drivers.
In the embodiment of the invention, the data topic pointer classifies and concentrates the heterogeneous data sources for the specific object, optionally, the specific object can be a special data type, a data format or a self-defined specific object, for example, different data topics can represent heterogeneous data sources corresponding to different special data types, for example, the data topics can respectively correspond to the heterogeneous data sources storing entertainment data, information data, scientific research data and the like; or heterogeneous data sources corresponding to different data formats, for example, data topics may correspond to heterogeneous data sources storing text data, image data, audio data, video data, and the like; the data sources can also be customized heterogeneous data sources, and after different heterogeneous data sources are accessed, different data topics can be customized according to data types and data formats of the heterogeneous data sources. At this time, the data topic may be a cultural and educational entertainment topic, a scientific research development topic, a news information topic, or the like, or may be a text topic, an image topic, an audio topic, a video topic, or the like, which is not limited in this embodiment of the present invention.
In the embodiment of the invention, the first input is selection input of data topics, which is used for determining to query heterogeneous data sources corresponding to the data topics, and optionally, the first input may be selection input of topic identification showing the data topics, or received information of the topic identification or the heterogeneous data sources when the information of the heterogeneous data sources corresponding to the data topics is displayed; or, the text input, the voice input, and the like corresponding to the topic identifier of the data topic may be received, and the first input form is not particularly limited in the embodiment of the present invention.
For example, taking the topic identification of the data topic "traffic monitoring polling topic" as "topic a", a selection input for "topic a" is received when "topic a" is presented, or a text input, a voice input, etc., including the content of "topic a" is received when "topic a" is not presented.
And step 102, responding to the first input, and matching the data query information corresponding to the data topic according to the topic identification corresponding to the data topic.
In the embodiment of the invention, when first input of the data topic is received, heterogeneous data in a heterogeneous data source corresponding to the data topic can be determined to be displayed, at the moment, the data topic information can be matched according to a topic identifier corresponding to the data topic in response to the first input, and as the heterogeneous data processing platform can store more than two data topics and the corresponding relation between the data topic and the heterogeneous data source, the topic identifier can be preset to distinguish different data topics, wherein the topic identifier can be an ID (Identity Document), a number, a topic name and the like corresponding to the data topic. The data query information refers to information related to querying heterogeneous data in a heterogeneous data source corresponding to a data topic, and optionally, the data query information may include an address of the heterogeneous data source, a field of a requirement, and the like, so that a part of the required heterogeneous data can be obtained in the corresponding heterogeneous data source. Because the thematic identification corresponds to the data thematic and the data query information corresponds to the data thematic, the data query information can be obtained by matching according to the thematic identification.
For example, the data query information corresponding to the traffic monitoring polling topic may include an X-path monitoring data source, a Y-path monitoring data source, a Z-path monitoring data source, monitoring time 20xx year, X month 10 to 20xx year, X month 20, and the like, and the data query information may be matched according to the identifier of the "topic a".
And 103, inquiring in a heterogeneous data source corresponding to the data topic according to the data inquiry information to obtain the heterogeneous data.
In the embodiment of the invention, after the data query information is acquired, the heterogeneous data can be queried and acquired in the heterogeneous data source corresponding to the data topic, and at the moment, the heterogeneous data source is accessed to the heterogeneous data processing platform, so that the heterogeneous data source can be directly accessed through the heterogeneous data processing platform, the heterogeneous data in the heterogeneous data source does not need to be extracted and integrated in advance, and the cost for processing the heterogeneous data is reduced. Optionally, according to the service requirement, data analysis, data display, and the like may be performed on the obtained heterogeneous data, and the processing mode of the heterogeneous data obtained by query is not particularly limited in the embodiment of the present invention.
For example, the data query information corresponding to the traffic monitoring polling topic includes an X-path monitoring data source, a Y-path monitoring data source, a Z-path monitoring data source, monitoring time 20xx year, X month 10 to 20xx year, X month 20, and the like, and the monitoring data recorded in the corresponding time range is obtained from each monitoring data source, at this time, the monitoring data can be displayed in time and source according to the information of time, the monitoring data source, and the like, and service analysis can also be performed according to the monitoring data.
Optionally, after the step 101, the method further includes:
and step S11, receiving a second input of the data topic.
In the embodiment of the present invention, under the condition that a first input for a data topic is received, so as to determine a heterogeneous data source corresponding to a query data topic, a second input for the data topic may be further received, so as to determine a further editing operation for the data topic, where an input form of the second input may be the same as that of the first input, or may be different from that of the first input.
For example, after receiving a selection input for "topic a," a second input may also be received to determine whether to perform further editing operations on the traffic monitoring polling topic corresponding to "topic a.
And step S12, responding to the second input, and executing editing operation on the data topic, wherein the editing operation comprises at least one of adding, modifying and deleting field filtering conditions when the heterogeneous data is queried.
In this embodiment of the present invention, in response to the second input, an editing operation may be performed on the data topic, and optionally, the editing operation may include adding, modifying, deleting a field filter condition and the like when querying the heterogeneous data, where the field filter condition includes that a part including the field in the heterogeneous data is not returned, or a part including the field is returned, and the like.
For example, in response to the second input, a field filter condition is added to the query for the traffic monitoring poll topic that filters the field "car accident".
At this time, the step 103 specifically includes:
and querying a heterogeneous data source corresponding to the data topic according to the data query information and the field filtering condition to obtain heterogeneous data.
In the embodiment of the invention, under the condition that the received second input data topic has the field filtering condition, the heterogeneous data source corresponding to the data topic can be inquired according to the data inquiry information and the field filtering condition to obtain heterogeneous data.
In the embodiment of the invention, a heterogeneous data processing platform accessed with at least two heterogeneous data sources is provided, a first input of data topics can be received on the heterogeneous data processing platform, data topic information is matched according to topic identifications corresponding to the data topics in response to the first input, the data topic information comprises topic identifications corresponding to the data topics and book query information, and the data topics correspond to at least one heterogeneous data source, so that after the data topic information is determined according to the topic identifications matched with the topic identifications, heterogeneous data can be queried in the heterogeneous data sources corresponding to the data topics according to the data query information in the data topic information to obtain the heterogeneous data. At this moment, in the heterogeneous data sources with different storage modes among cross-platforms, cross-services and cross-regions, since the heterogeneous data sources are accessed to the heterogeneous data display platform, the heterogeneous data sources can be directly accessed through the heterogeneous data display platform after the thematic query information is determined, and data services are not required to be provided after the heterogeneous data in different heterogeneous data sources are moved and integrated to a data center, so that the operation, maintenance and data integration costs of the platform are reduced, and the analysis and display efficiency of the heterogeneous data is improved.
Referring to fig. 2, a flowchart illustrating steps of a heterogeneous data processing method provided in an embodiment of the present invention is shown, and is applied to a heterogeneous data processing platform, where the heterogeneous data processing platform accesses at least two heterogeneous data sources, and the at least two heterogeneous data sources respectively store heterogeneous data in different storage manners, where the method specifically includes the following steps:
In the embodiment of the invention, the heterogeneous data sources can be accessed to the heterogeneous data processing platform in advance, optionally, the heterogeneous data sources can be grouped according to data types, data formats or customized standards, and the corresponding groups are selected to be accessed respectively when the heterogeneous data sources are accessed, so that the management efficiency of the heterogeneous data sources is improved.
In the embodiment of the present invention, the heterogeneous data sources may be accessed through a predetermined access manner, where the predetermined access manner may include remote access, for example, remote access is performed through an IP (Internet Protocol) address, a host name, a port number, a driver, an account password, and the like of the heterogeneous data sources, and when it is determined that the heterogeneous data sources to which the information points are connectable, the heterogeneous data sources are remotely accessed to the heterogeneous data processing platform; or, the predetermined access mode may also include local access, such as locally creating a heterogeneous data source in the heterogeneous data processing platform, or importing a heterogeneous data source of another server into a database of the heterogeneous data processing platform, so as to perform local access.
In the embodiment of the invention, in the process of accessing the heterogeneous data sources, if the heterogeneous data processing platform has accessed at least one heterogeneous data source, whether the currently accessed heterogeneous data source is repeated with the accessed heterogeneous data source can be checked, and if the heterogeneous data processing platform has not accessed the heterogeneous data source, verification is not required, wherein the repetition with the accessed heterogeneous data source can be that any one of the accessed heterogeneous data sources is the same as or similar to the data source information of the accessed heterogeneous data source, and can also be that the heterogeneous data in the heterogeneous data source is the same as or similar to the data source information of the accessed heterogeneous data source. Optionally, under the condition that any one of the accessed heterogeneous data sources is repeated with the accessed heterogeneous data source, the data source information of the accessed heterogeneous data source may be displayed, at this time, operations such as modification and replacement of the data source information may be received, and the access operation to the repeated heterogeneous data source may also be cancelled.
In the embodiment of the present invention, the data source information refers to basic attribute information of the heterogeneous data source, and optionally, may include at least one of an operation object, an operation time, a backup manner, an access record, and an exception notification, where the operation object may be an operation user, an operation terminal, or an IP address of the operation user or the operation terminal, which creates or accesses the heterogeneous data source; the operation time can be the creation time, the access time and the like of an abnormal data source; the backup mode can be backup address, backup period, etc.; the exception notification may be exception notification content, exception notification conditions, or the like; the access record may be an access through a heterogeneous data processing platform, or may be all accesses, and the embodiment of the present invention is not limited in this respect. After the heterogeneous data source is accessed, the operation object, the operation time, the backup mode, the access record, the abnormal notification and the like of the heterogeneous data source can be recorded according to the data source information of the heterogeneous data source, so that the management of the heterogeneous data source is facilitated.
Referring to fig. 3, which is a flowchart illustrating that a heterogeneous data source accesses a heterogeneous data processing platform provided in an embodiment of the present invention, as shown in fig. 3, data sources A, B and C complete a process of accessing a data storage of the heterogeneous data source to the heterogeneous data processing platform by setting a heterogeneous data source access engine to perform processing steps of data source grouping, data source access, data source verification, data source information perfection, and the like, respectively.
And step 203, acquiring the address of the heterogeneous data source.
In the embodiment of the invention, the address of the heterogeneous data source can be acquired as information required when the heterogeneous data source is inquired, wherein the address can be an IP address of the heterogeneous data source.
And 204, introducing a corresponding database table according to the heterogeneous data source, and determining a storage mode of the database under the condition that the database is in a specified storage mode.
In the embodiment of the present invention, the database table is used for screening fields in a heterogeneous data source, so as to adapt to different data analysis and display requirements, optionally, the database table may be introduced in a template import manner, where a template includes displayable columns for defining data characteristics, heterogeneous data in the heterogeneous data source is filtered by introducing the template, and heterogeneous data corresponding to the displayable columns in the template in the heterogeneous data is displayed, so as to achieve a field filtering effect, where the format of the template may be txt (text format), db (data base ), dbf (data base file, database file), csv (Comma-segmented Value), html (hypertext mark-up Language), xml (Extensible Markup Language), xls (Microsoft Excel, spreadsheet format), and so on, xlsx (spreadsheet format), wk1(LOTUS 1-2-3, spreadsheet format), wq1(Quattro pro, spreadsheet format), json (JavaScript Object Notation, JS Object Notation), mdb (Microsoft Database, Database file storage format), accdb (Access Database, Microsoft Access software storage format), odbc (Open Database Connectivity, Open Database interconnect), etc.; or, the displayable fields can be introduced in a customized manner, and the fields except the fields which can be displayed in the customized manner are filtered to obtain the corresponding database table.
In the embodiment of the present invention, after the database table is introduced, the fields in the database table may also be checked, optionally, whether a duplicate column, a special character column, or the like exists in the database table may be checked, the introduction of the database table is confirmed after the correction, more than two different database tables may be introduced into the same heterogeneous data source according to the actual application requirements, and the number of the database tables is not specifically limited in the embodiment of the present invention.
In the embodiment of the invention, a part of database is in a specified storage mode, wherein the storage mode can be row storage or column storage, and when the specified storage mode exists in the database table, the storage mode can be acquired as information required when a heterogeneous data source is queried.
And step 205, setting the read-write permission of the database table according to the user permission corresponding to the heterogeneous data source.
In the embodiment of the invention, the user authority refers to a heterogeneous data source which can be queried, obtained, edited and the like by a user, different users have different user authorities, the heterogeneous data sources of the heterogeneous data which can be queried and obtained by the user in different user authorities are different, at the moment, the read-write authority of the database table can be set according to the user authority corresponding to the heterogeneous data source, for example, when the user authority comprises the heterogeneous data source, the user corresponding to the user authority is set to have the read-write authority of the heterogeneous data source corresponding to the database, and when the user authority does not comprise the heterogeneous data source, the user corresponding to the user authority is set to have no read-write authority of the heterogeneous data source corresponding to the database or only have read authority. As shown in fig. 3, different database tables are introduced into heterogeneous data sources, and the heterogeneous data processing platform may grant read-write rights to the database tables according to user rights.
If the user 1 has the user permissions of the heterogeneous data source a and the heterogeneous data source B, as shown in fig. 3, the read-write permission of the base table a may be granted to the user 1, and the read-write permission of the base table 1 corresponding to the heterogeneous data source C may not be granted to the user 1.
In the embodiment of the invention, the data topic can be used for managing a heterogeneous data source, specifically, the data topic can be used for managing and introducing a database table, optionally, a topic identification corresponding to the data topic can be set, including setting a topic name, adding a topic description and the like, selecting topic classification and the like, and different data topics can be effectively distinguished according to the topic identification so as to facilitate matching in the query process, wherein the topic description can be further supplementary explanation for the topic name, the topic classification and the like.
For example, the topic identification corresponding to the data topic may be set by receiving text input, voice input, etc. of the user, such as a text "topic a" according to the topic name input by the user, and a text "traffic monitoring polling topic" described by the topic includes monitoring data processing for 20xx year x month 10 to 20xx year x month 20, and a text "monitoring polling class" classified by the topic.
In the embodiment of the invention, the acquisition of the query information corresponding to the data topic can be realized by setting the topic identification, selecting the corresponding heterogeneous data source, the storage mode specified by the heterogeneous data source and the database table introduced by the heterogeneous data source so as to determine the data query information corresponding to the data topic, querying the heterogeneous data source through the data query information so as to quickly and accurately determine the corresponding heterogeneous data source, and acquiring the required field through the database table so as to acquire the heterogeneous data to be analyzed and displayed.
For example, under the special topic of traffic monitoring and polling, an X-path monitoring data source, a Y-path monitoring data source and a Z-path monitoring data source which store monitoring data are selected, and a database table from 20xx year X month 10 to 20xx year X month 20 is selected, so that data query information is obtained.
And step 207, creating the data topic according to the topic identification and the data query information.
In the embodiment of the present invention, the set topic identifier may be associated with the acquired data query information, so as to create the corresponding data topic, optionally, form verification may be performed on the data topic in the process of creating the data topic, whether duplication, error, deletion, or the like exists in the topic identifier and the data query information between different data topics is detected, when form verification fails, step 206 may be executed again, so as to modify or innovate the data topic, and when form verification passes, creation of the data topic is completed.
And 208, determining an open state corresponding to the data topic, wherein the open state comprises at least one of closing and starting.
In the embodiment of the present invention, the data topic may correspond to different open states, where the open state may include at least one of closing and enabling, and optionally, in a case that the open state of the data topic is closed, the data topic does not receive a selection and an editing operation of a user who does not create the data topic, or allows query and display, but does not allow writing into a database table, or the like; when the open state of the data topic is enabled, the data topic may receive part or all of the operations of a user that does not create the data topic.
For example, the traffic monitoring polling topic is obtained by setting a topic identifier by the user 1 and selecting data query information to create, at this time, under the condition that the open state of the traffic monitoring polling topic is enabled, the user 2 can perform read-write operation on the database table under the traffic monitoring polling topic, and under the condition that the open state of the traffic monitoring polling topic is closed, the user 2 cannot perform read-write operation on the database table under the traffic monitoring polling topic.
Alternatively, taking the data topic "XX fraud and certification topic" as an example, the topic identification and data query information for creating the data topic may be as follows:
topic name: xx special topic of making false certificates
Topic description:
(1) processing counterfeit and shoddy commodity data aiming at xx areas in xx city of xx province;
(2) and (4) data processing of fake equipment and fake certificate fake chapters in xx district of xx province xx city.
Topic classification: counterfeit and certificate making
Data source (address): 25.31.0.1 counterfeit proof with 3308/theomatic xx
Database: the matic
The storage mode: the matic _ model (partial database with storage mode)
The database table: the matic _ table
Fig. 4 shows a flowchart for creating a data topic according to an embodiment of the present invention, and as shown in fig. 4, after the creation of a data topic is started, a topic name can be set, a topic description is added, a topic classification is selected, and a heterogeneous data source and a database table corresponding to the data topic are further selected to determine an open state; and after determining each item of information, performing form verification, if the form verification is passed, storing the information to complete the creation of a data topic, and if the form verification is not passed, resetting and selecting the information again.
And 209, displaying a data topic list, wherein the data topic list comprises at least one data topic corresponding to a heterogeneous data source in a user permission range.
In the embodiment of the present invention, the data topic may be displayed in a form of a data topic list, where the data topic list includes at least one data topic corresponding to a heterogeneous data source within a user permission range, and therefore, the data topic list may change correspondingly according to a change of the heterogeneous data source included in the user permission range, and when the data topic list displays the data topic, the data topic list may display a topic identifier corresponding to the data topic, and may also display other information such as information of the heterogeneous data source corresponding to the data topic, which is not limited in the embodiment of the present invention. At this time, different data topics can be obtained through the simple steps and processes, wherein the different data topics can correspond to the same or different heterogeneous data sources, that is, in the process of querying data, a user only needs to query the data topics without considering a specific heterogeneous data source, so that the querying efficiency of the heterogeneous data under the multi-heterogeneous data source and high-data-volume querying environment is effectively improved.
For example, in the "XX fraud certification topic" example, the topic names may be displayed in the data topic list, the topic names, the topic descriptions, and the topic classifications may be displayed, or all may be displayed for the user to select.
In the embodiment of the present invention, the operation of selecting the data topic in the data topic list may be an operation of clicking, long pressing, or the like on the corresponding data identifier in the data topic list, or a dragging operation of dragging the corresponding data identifier to a preset area on the display area, so that the query node corresponding to the data topic is added to the preset area, and a plurality of query nodes may be placed in the preset area. Optionally, when the number of data topics is large, the topic classification to which the data topic belongs may be selected first, or the data topic is retrieved in the display page to determine the required data topic.
In the embodiment of the invention, the query node in the preset area can further receive an expansion operation, so that expansion options corresponding to the query node are displayed, different expansion options can correspond to different expansion functions, and all field data under the data special topic corresponding to the query node can be queried according to different expansion options; or field filtering conditions can be added, modified or deleted, and fields with different lengths and contents are added in a self-defined mode for filtering, so that heterogeneous data are further filtered on the basis of a database table in subsequent query, and the required query street is obtained; the query node may also be removed from the preset area, thereby canceling the query for the data topic. The form of the expansion operation may correspond to the form of the first input referred to above.
And step 211, responding to the first input, and matching the data query information corresponding to the data topic according to the topic identification corresponding to the data topic.
In the embodiment of the present invention, step 211 may refer to the related description of step 102, and is not repeated herein to avoid repetition.
In the embodiment of the present invention, Query conditions may be generated according to the data Query information, where the Query conditions are used to determine corresponding heterogeneous data sources, and field screening is performed on the heterogeneous data sources to obtain required heterogeneous data, and optionally, dynamic SQL (Structured Query Language) may be generated according to the data Query information, so as to implement simple and efficient addition and replacement of multiple conditions, for example, a Query address condition is added according to an address of a heterogeneous data source, a database storage mode condition is added according to a storage mode, a screening field condition is added according to a database table, and the like, thereby improving efficiency of data Query, where the data Query information may refer to the relevant description in step 206, and is not described herein again to avoid repetition.
In the embodiment of the present invention, after the query condition is generated according to the data query information, the heterogeneous data source may be accessed according to the query condition, and different access manners are also different according to access manners of the heterogeneous data source, for example, the heterogeneous data source that is remotely accessed may be a remote access, the heterogeneous data source that is locally accessed may be a local access heterogeneous data source, and after the heterogeneous data source is successfully accessed, the heterogeneous data may be screened according to the query condition to obtain the heterogeneous data.
Fig. 5 shows a flowchart for querying heterogeneous data according to an embodiment of the present invention, as shown in fig. 5, a data topic list within a user permission range is first displayed, a required data topic is determined according to topic classification, and a query node corresponding to the data topic is added to a first input of the data topic; according to the expansion operation received by the query node, the heterogeneous data corresponding to the query data topic can be provided, the field filtering condition is edited, or the expansion functions such as the query node and the like are deleted; screening fields according to field filtering conditions, initiating a heterogeneous data query request which can include thematic identifications, and acquiring data query information corresponding to the data thematic according to the data thematic identifications; and generating a query condition according to the data query information, determining an access mode of the heterogeneous data source according to the data query information, remotely accessing the heterogeneous data source and acquiring heterogeneous data according to the query condition under the condition that the access mode is remote access, and locally accessing the heterogeneous data source and acquiring the heterogeneous data according to the query condition under the condition that the access mode is not remote access.
Optionally, as shown in fig. 5, in case of a failure of remote access, a data topic list may be returned to add a query node again; after the heterogeneous data is obtained, the heterogeneous data can be returned to a display end of the data topic list to display the heterogeneous data.
In the embodiment of the invention, a heterogeneous data processing platform accessed with at least two heterogeneous data sources is provided, a first input of data topics can be received on the heterogeneous data processing platform, data topic information is matched according to topic identifications corresponding to the data topics in response to the first input, the data topic information comprises topic identifications corresponding to the data topics and book query information, and the data topics correspond to at least one heterogeneous data source, so that after the data topic information is determined according to the topic identifications matched with the topic identifications, heterogeneous data can be queried in the heterogeneous data sources corresponding to the data topics according to the data query information in the data topic information to obtain the heterogeneous data. At this moment, in the heterogeneous data sources with different storage modes among cross-platforms, cross-services and cross-regions, since the heterogeneous data sources are accessed to the heterogeneous data display platform, the heterogeneous data sources can be directly accessed through the heterogeneous data display platform after the thematic query information is determined, and data services are not required to be provided after the heterogeneous data in different heterogeneous data sources are moved and integrated to a data center, so that the operation, maintenance and data integration costs of the platform are reduced, and the analysis and display efficiency of the heterogeneous data is improved.
It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.
Example two
Referring to fig. 6, a schematic structural diagram of a heterogeneous data processing apparatus 30 according to an embodiment of the present invention is shown, and is applied to a heterogeneous data processing platform, where the heterogeneous data processing platform accesses at least two heterogeneous data sources, and the at least two heterogeneous data sources respectively store heterogeneous data in different storage manners, where the method includes: a first input receiving module 301, configured to receive a first input of a data topic, where the data topic corresponds to at least one heterogeneous data source; a data topic matching module 302, configured to, in response to the first input, match data query information corresponding to the data topic according to a topic identifier corresponding to the data topic; and the heterogeneous data query module 303 is configured to query the heterogeneous data sources corresponding to the data topics according to the data query information to obtain the heterogeneous data.
Optionally, the apparatus further comprises:
and the data list display module is used for displaying a data topic list, wherein the data topic list comprises at least one data topic corresponding to a heterogeneous data source in a user permission range.
The first input receiving module 301 is specifically configured to receive a selection operation of the data topic in the data topic list as the first input.
Optionally, the heterogeneous data query module 303 includes:
the query condition generation sub-module is used for generating a query condition according to the data query information, wherein the data query information comprises at least one of an address, a storage mode, a database table and an access mode of the heterogeneous data source, and the access mode comprises any one of remote access and local access;
and the heterogeneous data query submodule is used for accessing a heterogeneous data source according to the query condition and querying to obtain heterogeneous data.
Optionally, the apparatus further comprises:
the second input receiving module is used for receiving second input of the data topic;
the field filtering editing module is used for responding to the second input and executing editing operation on the data topic, wherein the editing operation comprises at least one of adding, modifying and deleting field filtering conditions when the heterogeneous data is inquired;
optionally, the heterogeneous data query module 303 is specifically configured to query, according to the data query information and the field filtering condition, a heterogeneous data source corresponding to the data topic, so as to obtain heterogeneous data.
Optionally, the apparatus further comprises:
the heterogeneous data source access module is used for accessing a heterogeneous data source by adopting a preset access mode, wherein the preset access mode comprises any one of remote access and local access;
the heterogeneous data source address acquisition module is used for acquiring the address of the heterogeneous data source;
the database table introducing module is used for introducing a corresponding database table according to the heterogeneous data source and determining a storage mode of the database under the condition that the database is in a specified storage mode;
the data topic information acquisition module is used for setting topic identifications corresponding to data topics and acquiring data query information corresponding to the data topics, wherein the topic identifications comprise at least one of topic names, topic contents and topic classifications, and the data query information comprises at least one of the addresses of the heterogeneous data sources, the database table, the storage modes and access modes corresponding to the preset access modes;
and the data topic creating module is used for creating the data topic according to the topic identification and the data query information.
Optionally, the apparatus further comprises:
the heterogeneous data source verification module is used for displaying data source information of the accessed heterogeneous data source under the condition that any one of the accessed heterogeneous data sources is repeated with the accessed heterogeneous data source, and the data source information comprises at least one of an operation object, operation time, a backup mode, an access record and an exception notification.
Optionally, the apparatus further comprises:
and the database table authority determining module is used for setting the read-write authority of the database table according to the user authority corresponding to the heterogeneous data source.
Optionally, the apparatus further comprises:
and the data topic state determining module is used for determining an open state corresponding to the data topic, wherein the open state comprises at least one of closing and starting.
In the embodiment of the invention, a heterogeneous data processing platform accessed with at least two heterogeneous data sources is provided, a first input for a data topic can be received on the heterogeneous data processing platform, the first input is responded, data query information is matched according to a topic identifier corresponding to the data topic, and the data topic corresponds to at least one heterogeneous data source, so after the data query information is determined according to the topic identifier matched with the topic identifier, heterogeneous data can be queried in the heterogeneous data source corresponding to the data topic according to the data query information. At this moment, in the heterogeneous data sources with different storage modes among cross-platforms, cross-services and cross-regions, since the heterogeneous data sources are accessed to the heterogeneous data display platform, the heterogeneous data sources can be directly accessed through the heterogeneous data display platform after the thematic query information is determined, and data services are not required to be provided after the heterogeneous data in different heterogeneous data sources are moved and integrated to a data center, so that the operation, maintenance and data integration costs of the platform are reduced, and the analysis and display efficiency of the heterogeneous data is improved.
For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
Preferably, an embodiment of the present invention further provides a terminal, which includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, where the computer program, when executed by the processor, implements each process of the foregoing heterogeneous data processing method embodiment, and can achieve the same technical effect, and details are not repeated here to avoid repetition.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements each process of the above-mentioned heterogeneous data processing method embodiment, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here. The computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminals (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create a system for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction system which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.
The above provides a method and a device for processing heterogeneous data, and a specific example is applied in the present disclosure to explain the principle and the implementation of the present invention, and the above description of the embodiment is only used to help understanding the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. A heterogeneous data processing method is applied to a heterogeneous data processing platform, the heterogeneous data processing platform accesses at least two heterogeneous data sources, and the at least two heterogeneous data sources respectively adopt different storage modes to store heterogeneous data, and the method comprises the following steps:
receiving a first input of a data topic, wherein the data topic corresponds to at least one heterogeneous data source;
responding to the first input, and matching data query information corresponding to the data topic according to the topic identification corresponding to the data topic;
and querying in a heterogeneous data source corresponding to the data topic according to the data query information to obtain the heterogeneous data.
2. The method of claim 1, wherein prior to receiving the first input for the data topic, further comprising:
displaying a data topic list, wherein the data topic list comprises at least one data topic corresponding to a heterogeneous data source in a user permission range;
the receiving a first input of a data topic, comprising:
receiving a selection operation of the data topic in the data topic list as the first input.
3. The method according to claim 1, wherein the obtaining of the heterogeneous data by querying in the heterogeneous data source corresponding to the data topic according to the data query information comprises:
generating query conditions according to the data query information, wherein the data query information comprises at least one of an address, a storage mode, a database table and an access mode of the heterogeneous data source, and the access mode comprises any one of remote access and local access;
and accessing the heterogeneous data source according to the query condition, and querying to obtain heterogeneous data.
4. The method of claim 1, wherein after receiving the first input of the data topic, further comprising:
receiving a second input to the data topic;
in response to the second input, performing an editing operation on the data topic, wherein the editing operation comprises at least one of adding, modifying and deleting field filtering conditions when the heterogeneous data is queried;
the obtaining of the heterogeneous data by querying in the heterogeneous data source corresponding to the data topic according to the data query information includes:
and querying a heterogeneous data source corresponding to the data topic according to the data query information and the field filtering condition to obtain heterogeneous data.
5. The method of claim 1, wherein prior to receiving the first input for the data topic, further comprising:
accessing a heterogeneous data source by adopting a preset access mode, wherein the preset access mode comprises any one of remote access and local access;
acquiring the address of the heterogeneous data source;
introducing a corresponding database table according to the heterogeneous data source, and determining a storage mode of the database under the condition that the database is in a specified storage mode;
setting thematic identifiers corresponding to data thematic, and acquiring data query information corresponding to the data thematic, wherein the thematic identifiers comprise at least one of thematic names, thematic contents and thematic classifications, and the data query information comprises at least one of the address of the heterogeneous data source, the database table, the storage mode and an access mode corresponding to the preset access mode;
and creating the data topic according to the topic identification and the data query information.
6. The method of claim 5, wherein after accessing the heterogeneous data sources by using the predetermined access method, further comprising:
and under the condition that any one of the accessed heterogeneous data sources is repeated with the accessed heterogeneous data source, displaying data source information of the accessed heterogeneous data source, wherein the data source information comprises at least one of an operation object, operation time, a backup mode, an access record and an exception notification.
7. The method of claim 5, wherein after the introducing the corresponding database table according to the heterogeneous data source, the method comprises:
setting the read-write permission of the database table according to the user permission corresponding to the heterogeneous data source;
after the data topic is created according to the topic identification and the data query information, the method further comprises the following steps:
and determining an open state corresponding to the data topic, wherein the open state comprises at least one of closing and starting.
8. The utility model provides a heterogeneous data processing apparatus, its characterized in that is applied to heterogeneous data processing platform, heterogeneous data processing platform inserts at least two heterogeneous data sources, and at least two heterogeneous data sources adopt different storage methods to store heterogeneous data respectively, include:
the system comprises a first input receiving module, a first data processing module and a first data processing module, wherein the first input receiving module is used for receiving first input of data topics, and the data topics correspond to at least one heterogeneous data source;
the data thematic matching module is used for responding to the first input and matching data thematic information according to thematic identification corresponding to the data thematic, and the data thematic information comprises thematic identification corresponding to the data thematic and data query information;
and the heterogeneous data query module is used for querying the heterogeneous data in the heterogeneous data source corresponding to the data topic according to the data query information to obtain the heterogeneous data.
9. An electronic device comprising a processor, a memory, and a program or instructions stored on the memory and executable on the processor, the program or instructions when executed by the processor implementing the steps of the heterogeneous data processing method of claims 1-7.
10. A storage medium, characterized in that it stores thereon a program or instructions which, when executed by a processor, implement the steps of the heterogeneous data processing method according to claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011637761.XA CN112732758A (en) | 2020-12-31 | 2020-12-31 | Heterogeneous data processing method and device, electronic equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011637761.XA CN112732758A (en) | 2020-12-31 | 2020-12-31 | Heterogeneous data processing method and device, electronic equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112732758A true CN112732758A (en) | 2021-04-30 |
Family
ID=75608884
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011637761.XA Pending CN112732758A (en) | 2020-12-31 | 2020-12-31 | Heterogeneous data processing method and device, electronic equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112732758A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113268473A (en) * | 2021-07-20 | 2021-08-17 | 国能信控互联技术有限公司 | Self-adaptive data acquisition system and method for multiple time sequence databases |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107066499A (en) * | 2016-12-30 | 2017-08-18 | 江苏瑞中数据股份有限公司 | The data query method of multi-source data management and visualization system is stored towards isomery |
| CN108874848A (en) * | 2018-01-02 | 2018-11-23 | 国家测绘地理信息局第六地形测量队 | A kind of visualization system and implementation method for geographical national conditions knowledge services |
| CN110837585A (en) * | 2019-11-07 | 2020-02-25 | 中盈优创资讯科技有限公司 | Multi-source heterogeneous data association query method and system |
-
2020
- 2020-12-31 CN CN202011637761.XA patent/CN112732758A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107066499A (en) * | 2016-12-30 | 2017-08-18 | 江苏瑞中数据股份有限公司 | The data query method of multi-source data management and visualization system is stored towards isomery |
| CN108874848A (en) * | 2018-01-02 | 2018-11-23 | 国家测绘地理信息局第六地形测量队 | A kind of visualization system and implementation method for geographical national conditions knowledge services |
| CN110837585A (en) * | 2019-11-07 | 2020-02-25 | 中盈优创资讯科技有限公司 | Multi-source heterogeneous data association query method and system |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113268473A (en) * | 2021-07-20 | 2021-08-17 | 国能信控互联技术有限公司 | Self-adaptive data acquisition system and method for multiple time sequence databases |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP4726545B2 (en) | Method, system and apparatus for discovering and connecting data sources | |
| CN111104392B (en) | Database migration method and device, electronic equipment and storage medium | |
| US8892534B2 (en) | System and method for integrating data quality metrics into enterprise data management processes | |
| EP3876107A1 (en) | Apparatus, systems, and methods for analyzing movements of target entities | |
| CN106844730B (en) | Method and device for displaying file content | |
| CN109284631A (en) | A kind of document desensitization system and method based on big data | |
| CN111177252A (en) | Service data processing method and device | |
| CN111914135A (en) | Data query method and device, electronic equipment and storage medium | |
| CN110414259B (en) | Method and equipment for constructing data category and realizing data sharing | |
| CN114428822B (en) | Data processing method and device, electronic equipment and storage medium | |
| Manu et al. | Research information management systems: a comparative study | |
| CN115544183A (en) | Data visualization method and device, computer equipment and storage medium | |
| CN111522817A (en) | Table content mapping system, method and non-transitory storage medium | |
| CN112732758A (en) | Heterogeneous data processing method and device, electronic equipment and readable storage medium | |
| CN113037505B (en) | Method and system for realizing trusted Web application | |
| US20140229815A1 (en) | Computerised data entry form processing | |
| CN112330308B (en) | Case management system and method and electronic device | |
| Kawu et al. | FAIR Guidelines and data regulatory framework for digital health in Nigeria | |
| CN107239568B (en) | Distributed index implementation method and device | |
| CN110019542B (en) | Generation of enterprise relationship, generation of organization member database and identification of same name member | |
| CN111858627A (en) | Academic degree and academic calendar query system and method based on block chain | |
| CN108170867A (en) | A kind of metadata service system | |
| CN111190986B (en) | Map data comparison method and device | |
| CN113934729A (en) | Data management method based on knowledge graph, related equipment and medium | |
| Bunawan et al. | The challenges in preserving the electronic records metadata |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |