CN113468379A - Data source processing method and device and intelligent analysis platform - Google Patents
Data source processing method and device and intelligent analysis platform Download PDFInfo
- Publication number
- CN113468379A CN113468379A CN202010241184.6A CN202010241184A CN113468379A CN 113468379 A CN113468379 A CN 113468379A CN 202010241184 A CN202010241184 A CN 202010241184A CN 113468379 A CN113468379 A CN 113468379A
- Authority
- CN
- China
- Prior art keywords
- data
- standardized
- data sources
- sources
- various data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
Abstract
The application provides a data source processing method, a data source processing device and an intelligent analysis platform, wherein the data source processing method comprises the following steps: acquiring a plurality of data sources; carrying out statistical analysis on the various data sources and obtaining data information in the data sources; setting a standardized metadata list for various data sources according to various data information and combining with preset classification logic; training to obtain a standard data query engine based on the data information and the standardized metadata list as a sample set; and extracting various data sources to a standardized data query engine, and processing the various data sources through the standardized data query engine to obtain the summarized standardized data sources. According to the method, various data sources can be unified and standardized, data developers can know the states of the various data sources more quickly, metadata lists can be automatically analyzed, and working efficiency is improved.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data source processing method and apparatus, and an intelligent analysis platform.
Background
At present, data of each product often has many connections and dependencies, many repeated development works are needed to unify data sources, and the tools are difficult to be utilized in other products, so that the data need to be further processed so as to be more conveniently extracted and used by users.
Disclosure of Invention
In view of this, the present application provides a data source processing method, an apparatus and an intelligent analysis platform, which can perform standardized processing on various data sources, and facilitate data use.
In order to solve the technical problem, the following technical scheme is adopted in the application:
in a first aspect, an embodiment of the present application provides a data source processing method, including the following steps:
acquiring various types of data sources; and extracting the various types of data sources to a standardized data query engine, and processing the data sources through the standardized data query engine to obtain various summarized standardized data sources. For example, various data sources are accessed through an adaptive interface, and extracted to summarize the various data sources, wherein the various data sources can be a collection of multiple different types of data sources, and machine learning is performed by combining a standardized data query engine based on the various data sources to obtain standardized and unified various data sources.
The method comprises the steps of carrying out statistical analysis on various data sources, obtaining data information in the data sources, wherein the data information can be information such as character content and digital content, setting standardized metadata lists for the various data sources according to the various data information and combining with preset classification logic, training to obtain a standard data query engine based on the data information and the standardized metadata lists as a sample set, and carrying out unified standardized processing on the various data sources when the various data sources extract the standard data query engine, so that data developers can know the states of the various data sources more quickly.
According to the data source processing method, various data sources can be unified and standardized, data developers can know the states of the various data sources more quickly, metadata lists can be automatically analyzed, and working efficiency is improved.
As an embodiment of the first aspect of the present application, the data source may include: one or more of files, web pages, relational databases, time sequence databases, analytical databases, real-time message queues and data extraction interfaces are convenient for the development or use of various data by unifying and standardizing various data sources.
As an embodiment of the first aspect of the present application, the metadata list includes: one or more of type, scope, meaning, and distribution of individual data fields in the data source.
As an embodiment of the first aspect of the present application, the predetermined classification logic includes:
judging whether the data information meets a threshold value of a set attribute, for example, the data information is 18 digits, the attribute is an identity card, the threshold value of the identity card can be set to be 15 digits or 18 digits, and the current data information is 18 digits and meets the threshold value of the set identity card;
and when the data information is judged to accord with the threshold value of the set attribute, attributing the data information to the attribute, and recording the attribute in the standardized metadata list. And then can directly obtain the data information of the data source as the ID card through the record attribute of the metadata list, the convenient data user knows the state of the data source fast.
As an embodiment of the first aspect of the present application, acquiring multiple types of data sources includes:
and setting an interface adaptive to various data sources to acquire the various data sources, extracting the various data sources to a standard data query engine, and calling the various data sources through the interface of the data source so as to facilitate the summarization of the data source.
In a second aspect, the present application discloses a processing apparatus for a data source, including:
the acquisition module is used for acquiring various data sources;
the analysis module is used for carrying out statistical analysis on various data sources and obtaining data information in the data sources;
the processing module is used for setting a standardized metadata list for various data sources according to various data information and combining with preset classification logic;
the processing module is used for training to obtain a standard data query engine based on the data information and a standardized data source list as a sample set;
and the extraction module is used for extracting the multiple data sources to the standardized data query engine so that the multiple data sources are processed by the standardized data query engine to obtain the summarized standardized data sources.
According to the data source processing device, various data sources can be unified and standardized, data developers can conveniently know the states of the various data sources more quickly, metadata lists can be automatically analyzed, and working efficiency is improved.
As an embodiment of the second aspect of the present application, the data source includes: a file, a web page, a relational database, a time series database, an analytical database, a real-time message queue, or a data extraction interface.
As an embodiment of the second aspect of the present application, the metadata list includes: one or more of type, scope, meaning, and distribution of individual data fields in the data source.
As an embodiment of the second aspect of the present application, the processing module is specifically configured to:
judging whether the data information meets a threshold value of a set attribute;
and when the data information is judged to accord with the threshold value of the set attribute, attributing the data information to the attribute, and recording the attribute in the standardized metadata list.
As an embodiment of the second aspect of the present application, the obtaining module is further configured to: and setting an interface adaptive to various data sources to acquire the various data sources, and further extracting the various data sources to a standard data query engine to summarize the various data sources.
In a third aspect, the present application discloses an intelligent analysis platform, which includes the processing apparatus of the data source in the foregoing embodiment.
In a fourth aspect, the present application discloses an electronic device, comprising: a processor; and a memory having computer program instructions stored therein,
wherein the computer program instructions, when executed by the processor, cause the processor to perform the method of the above embodiment.
The technical scheme of the application has at least one of the following beneficial effects:
according to the data source processing method and device and the intelligent analysis platform, various data sources can be unified and standardized, and data developers can know the states of the various data sources more quickly.
Drawings
FIG. 1 is a flow chart of a method of processing a data source according to one embodiment of the present application;
FIG. 2 is a schematic diagram of a data source processing apparatus according to an embodiment of the present application;
fig. 3 is a schematic diagram of an electronic device according to an embodiment of the invention.
Detailed Description
In order that the content of the present application may be more clearly understood, specific embodiments of the present application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are intended to illustrate the present application but are not intended to limit the scope of the present application.
A method for processing a data source according to an embodiment of the present application is described below with reference to the accompanying drawings, and fig. 1 shows a flowchart of the method for processing a data source, and as shown in fig. 1, the method includes the following steps:
step S110, acquiring a plurality of data sources, wherein the data sources comprise: one or more of a file, a web page, a relational database, a time series database, an analytical database, a real-time message queue, and a data extraction interface. Obtaining multiple types of data sources may include: and setting an interface adapting to various data sources to acquire multiple types of data sources. Various data sources can be called through the data source interface so as to facilitate the summarization of the various data sources.
And step S120, performing statistical analysis on various data sources, and obtaining data information in the data sources. The statistical analysis of the data source includes analyzing metadata in the data source, and analyzing information such as type, range, meaning, distribution, etc. of each field in the data source, for example, meaning of a field is male or female, in which data source the field is distributed, etc.
And step 130, setting a standardized metadata list for each data source according to each data information and combining with preset classification logic, wherein the metadata list is used for recording attribute information about the data source, such as gender, identity cards, mobile phone numbers and the like.
Step 140, training to obtain a standard data query engine based on the data information and the standardized data source list as a sample set, and then rapidly obtaining the state of each data source through the standard data query engine.
Step S150, extracting the plurality of data sources to a standard data query engine, and processing the data sources by the standardized data query engine to obtain various summarized standardized data sources. That is to say, the standard data query engine is used as a standard model, which can be obtained by training based on a large amount of historical empirical data in the above steps, for example, 1000 pieces of data can be analyzed, wherein the number of 11 digits can be marked as a mobile phone number, and data information about men and women is marked as gender, etc., and the standard data query engine is obtained by training based on the data information, and various data sources can be unified and standardized by the standard data query engine, and summarized, so that data users can quickly know the states of the various data sources, and work efficiency is improved.
The standard data query engine of the embodiment of the application can perform regular training according to the continuous accumulation of the data source information so as to improve the accuracy of the standard data query engine in processing the data source.
According to one embodiment of the present application, a standardized data query engine is obtained through machine learning.
According to one embodiment of the application, the predetermined classification logic comprises: judging whether the data information meets a threshold value of a set attribute; and when the data information is judged to accord with the threshold value of the set attribute, classifying the data information as the attribute, wherein the attribute can be information such as an identity card, gender, a license plate, a mobile phone number and the like, and recording the attribute in the standardized metadata list. The predetermined classification logic of the present application may classify the data source according to different types of data sources, for example, a set classification logic about numbers, when the data information of the data source is analyzed to be a number, the number is judged to be 18-bit or 15-bit number, and the threshold value (18-bit number or 15-bit number) of the identity card is met, so that the data information can be judged to be the identity card, and the identity card is recorded in a standardized metadata list. For another example, if the data information is 11 digits, it is determined that the digital information is a threshold (11 digits) of a set mobile phone number, and the mobile phone number is recorded in a standardized metadata list as a mark, and after analyzing a large amount of data, the data source is trained as a training sample set to obtain a standard query engine.
In addition, in the present application, a probabilistic analysis may be performed on a part of wrongly written data information or a data source which is difficult to judge, for example, the data information is 14 bits, and it does not satisfy a set threshold value of the identity card, and in this case, a probability of 50% may be marked, and the probability may be marked in a standardized metadata list, and a probability of 90% or higher or lower may be marked for satisfying the threshold value of the identity card, so as to objectively analyze the data source.
Therefore, according to the data source processing method, various data sources can be unified and standardized, data developers can know the states of the various data sources more quickly, and working efficiency is improved.
Based on the above description, the processing apparatus of the data source of the present application is specifically described below.
Fig. 2 shows a processing apparatus of a data source according to an embodiment of the present application, and as shown in fig. 2, the apparatus includes:
an obtaining module 310, configured to obtain multiple types of data sources, where the data sources include: one or more of a file, a web page, a relational database, a time series database, an analytical database, a real-time message queue, and a data extraction interface.
The analysis module 320 is configured to perform statistical analysis on various data sources and obtain data information in the data sources.
And the processing module 330 is configured to set a standardized metadata list for each data source according to each data information and by combining with a predetermined classification logic.
The processing module 330 trains a standard data query engine based on the data information and the normalized data source list as a sample set.
The extracting module 340 is configured to extract the multiple data sources into the standardized data query engine, so that the multiple data sources are processed by the standardized data query engine to obtain the summarized standardized data sources. The data query engine with the standard can unify and standardize various data sources and gather the data sources, so that data users can quickly know the states of the various data sources, and the working efficiency is improved.
According to one embodiment of the application, the data source comprises: a file, a web page, a relational database, a time series database, an analytical database, a real-time message queue, or a data extraction interface.
According to one embodiment of the present application, the metadata list includes: one or more of type, scope, meaning, and distribution of individual data fields in the data source.
According to an embodiment of the present application, the processing module 330 is specifically configured to:
judging whether the data information meets a threshold value of a set attribute;
and when the data information is judged to accord with the threshold value of the set attribute, attributing the data information to the attribute, and recording the attribute in the standardized metadata list.
According to an embodiment of the present application, the obtaining module 310 is further configured to:
and setting an interface adapting to various data sources so as to extract various types of data sources to a standard data query engine.
The components and the workflow of the data source processing apparatus in the embodiment of the present application have been described in detail in the above embodiment, and refer to the processing method in the above embodiment specifically, and are not described herein again.
According to the data source processing device, various data sources can be unified and standardized, data developers can conveniently know the states of the various data sources more quickly, metadata lists can be automatically analyzed, and working efficiency is improved.
The application also discloses an intelligent analysis platform which comprises the data source processing device of the embodiment. For the components and the workflow of the processing apparatus of the data source, detailed descriptions have been given in the above embodiments, and specific reference may be made to the apparatus of the above embodiments, which are not repeated herein.
Other structures and operations of the intelligent analysis platform according to the embodiments of the present application are understood and easily implemented by those skilled in the art, and thus will not be described in detail.
Referring now to FIG. 3, shown is a block diagram of an apparatus 1200 in accordance with one embodiment of the present application. The device 1200 may include one or more processors 1201 coupled to a controller hub 1203. For at least one embodiment, the controller hub 1203 communicates with the processor 1201 via a multi-drop Bus such as a Front Side Bus (FSB), a point-to-point interface such as a Quick Path Interconnect (QPI), or similar connection 1206. The processor 1201 executes instructions that control general types of data processing operations. In one embodiment, Controller Hub 1203 includes, but is not limited to, a Graphics Memory Controller Hub (GMCH) (not shown) and an Input/Output Hub (IOH) (which may be on separate chips) (not shown), where the GMCH includes a Memory and a Graphics Controller and is coupled to the IOH.
The device 1200 may also include a coprocessor 1202 and a memory 1204 coupled to the controller hub 1203. Alternatively, one or both of the memory and GMCH may be integrated within the processor (as described herein), with the memory 1204 and coprocessor 1202 being directly coupled to the processor 1201 and to the controller hub 1203, with the controller hub 1203 and IOH being in a single chip. The Memory 1204 may be, for example, a Dynamic Random Access Memory (DRAM), a Phase Change Memory (PCM), or a combination of the two. In one embodiment, coprocessor 1202 is a special-Purpose processor, such as, for example, a high-throughput MIC processor (MIC), a network or communication processor, compression engine, graphics processor, General Purpose Graphics Processor (GPGPU), embedded processor, or the like. The optional nature of coprocessor 1202 is represented in FIG. 3 by dashed lines.
In one embodiment, device 1200 may further include a Network Interface Controller (NIC) 1206. Network interface 1206 may include a transceiver to provide a radio interface for device 1200 to communicate with any other suitable device (e.g., front end module, antenna, etc.). In various embodiments, the network interface 1206 may be integrated with other components of the device 1200. The network interface 1206 may implement the functions of the communication unit in the above-described embodiments.
The device 1200 may further include an Input/Output (I/O) device 1205. I/O1205 may include: a user interface designed to enable a user to interact with the device 1200; the design of the peripheral component interface enables peripheral components to also interact with the device 1200; and/or sensors may be configured to determine environmental conditions and/or location information associated with device 1200.
It is noted that fig. 3 is merely exemplary. That is, although fig. 3 shows that the apparatus 1200 includes a plurality of devices, such as the processor 1201, the controller hub 1203, the memory 1204, etc., in practical applications, an apparatus using the methods of the present application may include only a part of the devices of the apparatus 1200, for example, only the processor 1201 and the NIC1206 may be included. The nature of the optional device in fig. 3 is shown in dashed lines.
According to some embodiments of the present application, the memory 1204 serving as a computer-readable storage medium stores instructions, which when executed on a computer, cause the system 1200 to perform the processing method according to the above embodiments, which may specifically refer to the method of the above embodiments, and will not be described herein again.
Embodiments of the mechanisms disclosed herein may be implemented in hardware, software, firmware, or a combination of these implementations. Embodiments of the application may be implemented as computer programs or program code executing on programmable systems comprising at least one processor, a storage system (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device.
Program code may be applied to input instructions to perform the functions described herein and generate output information. The output information may be applied to one or more output devices in a known manner. For purposes of this Application, a processing system includes any system having a Processor such as, for example, a Digital Signal Processor (DSP), a microcontroller, an Application Specific Integrated Circuit (ASIC), or a microprocessor.
The program code may be implemented in a high level procedural or object oriented programming language to communicate with a processing system. The program code can also be implemented in assembly or machine language, if desired. Indeed, the mechanisms described in this application are not limited in scope to any particular programming language. In any case, the language may be a compiled or interpreted language.
In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried by or stored on one or more transitory or non-transitory machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or via other computer readable media. Thus, a machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including, but not limited to, floppy diskettes, optical disks, Compact disk Read Only memories (CD-ROMs), magneto-optical disks, Read Only Memories (ROMs), Random Access Memories (RAMs), Erasable Programmable Read Only Memories (EPROMs), Electrically Erasable Programmable Read Only Memories (EEPROMs), magnetic or optical cards, flash Memory, or a tangible machine-readable Memory for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in electrical, optical, acoustical or other forms of propagated signals. Thus, a machine-readable medium includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).
In the drawings, some features of the structures or methods may be shown in a particular arrangement and/or order. However, it is to be understood that such specific arrangement and/or ordering may not be required. Rather, in some embodiments, the features may be arranged in a manner and/or order different from that shown in the figures. In addition, the inclusion of a structural or methodical feature in a particular figure is not meant to imply that such feature is required in all embodiments, and in some embodiments, may not be included or may be combined with other features.
It should be noted that, in the embodiments of the apparatuses in the present application, each unit/module is a logical unit/module, and physically, one logical unit/module may be one physical unit/module, or may be a part of one physical unit/module, and may also be implemented by a combination of multiple physical units/modules, where the physical implementation manner of the logical unit/module itself is not the most important, and the combination of the functions implemented by the logical unit/module is the key to solve the technical problem provided by the present application. Furthermore, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units/modules which are not so closely related to solve the technical problems presented in the present application, which does not indicate that no other units/modules exist in the above-mentioned device embodiments.
It is noted that, in the examples and descriptions of this patent, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, the use of the verb "comprise a" to define an element does not exclude the presence of another, same element in a process, method, article, or apparatus that comprises the element.
While the present application has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present application.
Claims (10)
1. A method for processing a data source, comprising:
acquiring a plurality of data sources;
carrying out statistical analysis on the various data sources and obtaining data information in the data sources;
setting a standardized metadata list for various data sources according to various data information and combining with preset classification logic;
training to obtain a standard data query engine based on the data information and the standardized metadata list as a sample set;
and extracting various data sources to a standardized data query engine, and processing the various data sources through the standardized data query engine to obtain the summarized standardized data sources.
2. The method of claim 1, wherein the data source comprises: one or more of a file, a web page, a relational database, a time series database, an analytical database, a real-time message queue, and a data extraction interface.
3. The method of claim 1, wherein the metadata list comprises: one or more of type, scope, meaning, and distribution of individual data fields in the data source.
4. The method of claim 1, wherein the predetermined classification logic comprises:
judging whether the data information meets a threshold value of a set attribute;
and when the data information is judged to accord with the threshold value of the set attribute, attributing the data information to the attribute, and recording the attribute in the standardized metadata list.
5. A data source processing apparatus, comprising:
the acquisition module is used for acquiring various data sources;
the analysis module is used for carrying out statistical analysis on various data sources and obtaining data information in the data sources;
the processing module is used for setting a standardized metadata list for various data sources according to various data information and combining with preset classification logic;
the processing module is used for training to obtain a standard data query engine based on the data information and a standardized data source list as a sample set;
and the extraction module is used for extracting the various data sources to a standardized data query engine and processing the data sources through the standardized data query engine to obtain the summarized standardized data sources.
6. The apparatus of claim 5, wherein the data source comprises: a file, a web page, a relational database, a time series database, an analytical database, a real-time message queue, or a data extraction interface.
7. The apparatus of claim 5, wherein the metadata list comprises: one or more of type, scope, meaning, and distribution of individual data fields in the data source.
8. The apparatus of claim 5, wherein the processing module is specifically configured to:
judging whether the data information meets a threshold value of a set attribute;
and when the data information is judged to accord with the threshold value of the set attribute, attributing the data information to the attribute, and recording the attribute in the standardized metadata list.
9. An intelligent analysis platform comprising processing means of the data source of any one of claims 5 to 8.
10. An electronic device, comprising: a processor; and a memory having computer program instructions stored therein,
wherein the computer program instructions, when executed by the processor, cause the processor to perform the method of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010241184.6A CN113468379A (en) | 2020-03-31 | 2020-03-31 | Data source processing method and device and intelligent analysis platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010241184.6A CN113468379A (en) | 2020-03-31 | 2020-03-31 | Data source processing method and device and intelligent analysis platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113468379A true CN113468379A (en) | 2021-10-01 |
Family
ID=77865982
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010241184.6A Pending CN113468379A (en) | 2020-03-31 | 2020-03-31 | Data source processing method and device and intelligent analysis platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113468379A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115185923A (en) * | 2022-07-07 | 2022-10-14 | 中国气象局气象探测中心 | Method, system and intelligent terminal for managing meteorological observation metadata |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102495892A (en) * | 2011-12-09 | 2012-06-13 | 北京大学 | Webpage information extraction method |
| US8375014B1 (en) * | 2008-06-19 | 2013-02-12 | BioFortis, Inc. | Database query builder |
| CN105095436A (en) * | 2015-07-23 | 2015-11-25 | 苏州国云数据科技有限公司 | Automatic modeling method for data of data sources |
| CN105701176A (en) * | 2016-01-04 | 2016-06-22 | 浪潮软件股份有限公司 | Data integration method and apparatus |
| CN108292323A (en) * | 2016-01-08 | 2018-07-17 | 微软技术许可有限责任公司 | Use the database manipulation of the metadata of data source |
| CN108595571A (en) * | 2018-04-16 | 2018-09-28 | 深圳零壹云医科技有限公司 | A kind of Data Integration management method, device, system and user terminal |
| CN109558443A (en) * | 2018-11-29 | 2019-04-02 | 北京数聚鑫云信息技术有限公司 | A kind of method and device of data in integrated data sources |
-
2020
- 2020-03-31 CN CN202010241184.6A patent/CN113468379A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8375014B1 (en) * | 2008-06-19 | 2013-02-12 | BioFortis, Inc. | Database query builder |
| CN102495892A (en) * | 2011-12-09 | 2012-06-13 | 北京大学 | Webpage information extraction method |
| CN105095436A (en) * | 2015-07-23 | 2015-11-25 | 苏州国云数据科技有限公司 | Automatic modeling method for data of data sources |
| CN105701176A (en) * | 2016-01-04 | 2016-06-22 | 浪潮软件股份有限公司 | Data integration method and apparatus |
| CN108292323A (en) * | 2016-01-08 | 2018-07-17 | 微软技术许可有限责任公司 | Use the database manipulation of the metadata of data source |
| CN108595571A (en) * | 2018-04-16 | 2018-09-28 | 深圳零壹云医科技有限公司 | A kind of Data Integration management method, device, system and user terminal |
| CN109558443A (en) * | 2018-11-29 | 2019-04-02 | 北京数聚鑫云信息技术有限公司 | A kind of method and device of data in integrated data sources |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115185923A (en) * | 2022-07-07 | 2022-10-14 | 中国气象局气象探测中心 | Method, system and intelligent terminal for managing meteorological observation metadata |
| CN115185923B (en) * | 2022-07-07 | 2023-03-07 | 中国气象局气象探测中心 | Method and system for managing meteorological observation metadata and intelligent terminal |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107423278B (en) | Evaluation element identification method, device and system | |
| CN106484915B (en) | A kind of cleaning method and system of mass data | |
| CN111796957B (en) | Transaction abnormal root cause analysis method and system based on application log | |
| CN104063450A (en) | Hot spot information analyzing method and equipment | |
| CN110737821B (en) | Similar event query method, device, storage medium and terminal equipment | |
| CN108241867B (en) | Classification method and device | |
| CN115237857A (en) | Log processing method and device, computer equipment and storage medium | |
| CN109697155B (en) | IT system performance evaluation method, device, equipment and readable storage medium | |
| CN113468379A (en) | Data source processing method and device and intelligent analysis platform | |
| CN110147482B (en) | Method and device for acquiring burst hotspot theme | |
| CN105678557A (en) | Method and device for generating model, method and device for evaluating service quality | |
| CN115495498B (en) | Data association method, system, electronic equipment and storage medium | |
| CN107430633A (en) | The representative content through related optimization being associated to data-storage system | |
| CN113360313B (en) | Behavior analysis method based on massive system logs | |
| US11640558B2 (en) | Unbalanced sample classification method and apparatus | |
| CN111931229B (en) | Data identification method, device and storage medium | |
| CN110309273A (en) | Answering method and device | |
| CN114021716A (en) | Model training method and system and electronic equipment | |
| CN111027296A (en) | Report generation method and system based on knowledge base | |
| US11354274B1 (en) | System and method for performing data minimization without reading data content | |
| CN113569879B (en) | Training method of abnormal recognition model, abnormal account recognition method and related device | |
| CN111901307B (en) | Encrypted traffic identification method, device, equipment and medium | |
| CN113626385B (en) | Method and system based on text data reading | |
| CN108153817B (en) | Intelligent web page data acquisition method | |
| CN115033179A (en) | Data storage method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |