CN117667454A - Metadata acquisition method, device, equipment and medium - Google Patents

Metadata acquisition method, device, equipment and medium Download PDF

Info

Publication number
CN117667454A
CN117667454A CN202311678519.0A CN202311678519A CN117667454A CN 117667454 A CN117667454 A CN 117667454A CN 202311678519 A CN202311678519 A CN 202311678519A CN 117667454 A CN117667454 A CN 117667454A
Authority
CN
China
Prior art keywords
target
metadata
acquisition
data source
preset
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311678519.0A
Other languages
Chinese (zh)
Inventor
沈滔
刘航
庞攀
顾美营
胥果
马华均
张登辉
王树超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chengdu Sefon Software Co Ltd
Original Assignee
Chengdu Sefon Software Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chengdu Sefon Software Co Ltd filed Critical Chengdu Sefon Software Co Ltd
Priority to CN202311678519.0A priority Critical patent/CN117667454A/en
Publication of CN117667454A publication Critical patent/CN117667454A/en
Pending legal-status Critical Current

Links

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a metadata acquisition method, a device, equipment and a medium, which comprise the following steps: configuring a target acquisition task for a target client; determining a target data source to be acquired according to the target acquisition task; configuring a target acquisition plug-in for a target client, wherein the target acquisition plug-in at least comprises a preset data structure, a preset connection protocol, a preset acquisition tool and a preset acquisition rule; based on the acquisition executor, the target client adopts a target acquisition plug-in to acquire first metadata in a target data source to be acquired. Compared with the prior art, only single-type metadata can be acquired, the metadata acquisition method and device can acquire various types of metadata, the coding amount is smaller, the metadata acquisition efficiency is improved, the labor intensity of staff is reduced, and the economic benefit is improved.

Description

Metadata acquisition method, device, equipment and medium
Technical Field
The present invention relates to the field of software communications, and in particular, to a metadata collection method, apparatus, device, and medium.
Background
Metadata collection refers to the process of collecting and recording descriptive information about data. Metadata is data about data that provides information about the context, structure, content, and attributes of the data. Metadata may describe information in terms of source, format, quality, update frequency, etc. of the data, helping a user understand and use the data efficiently.
When extracting, converting and loading metadata, the metadata is generally only used for single data, and may have difficulty in collecting or comparing data sources of different industries, and when extracting, converting and loading metadata for different types of metadata, a large amount of scripts and programs need to be written, and development and maintenance costs are required. Therefore, how to improve the metadata collection efficiency is a problem to be solved.
Disclosure of Invention
According to the metadata acquisition method, device, equipment and medium, the technical problem of low metadata acquisition efficiency in the prior art is solved, and the technical effect of improving metadata acquisition efficiency is achieved.
In a first aspect, the present application provides a metadata collection method, including:
configuring a target acquisition task for a target client;
determining a target data source to be acquired according to the target acquisition task;
configuring a target acquisition plug-in for a target client, wherein the target acquisition plug-in at least comprises a preset data structure, a preset connection protocol, a preset acquisition tool and a preset acquisition rule;
based on the acquisition executor, the target client adopts a target acquisition plug-in to acquire first metadata in a target data source to be acquired.
Further, determining a target data source to be acquired according to the target acquisition task includes:
constructing a definition interface collection library, wherein the definition interface collection library comprises a plurality of preset definition interfaces;
constructing a data source library to be acquired;
determining a target definition interface from a definition interface collection library according to the target acquisition task;
and determining a target data source to be acquired corresponding to the target definition interface from the data source library to be acquired according to the target acquisition task.
Further, the method comprises the steps of:
dynamically expanding a data source library to be acquired;
and updating the definition interface collection library according to the dynamic expansion result of the data source library to be acquired.
Further, the method further comprises:
constructing a metadata base, wherein the metadata base comprises a plurality of metadata;
inputting the collected first metadata into a metadata base;
based on a preset analysis model, the first metadata is analyzed, and the blood-edge relation between the first metadata and each metadata in the metadata base is determined.
Further, before configuring the target acquisition task for the target client, the method includes:
the target client sends a login request to the target server;
the target server receives the login request and verifies the login request to generate a verification result, and the target server returns the verification result to the target client;
and the target client determines whether a target acquisition task can be configured for the target client through the front end according to the received verification result.
Further, the method further comprises:
adding a timestamp to the first metadata;
after the target client collects the first metadata, judging the time interval between the time stamp in the first metadata and the current moment;
and determining the network transmission state between the target client and the data source library to be acquired according to the preset response time and the time interval.
In a second aspect, the present application provides a metadata collection apparatus, the apparatus comprising:
the task configuration module is used for configuring a target acquisition task for the target client;
the data source determining module is used for determining a target data source to be acquired according to the target acquisition task;
the plug-in determining module is used for configuring a target acquisition plug-in for the target client, wherein the target acquisition plug-in at least comprises a preset data structure, a preset connection protocol, a preset acquisition tool and a preset acquisition rule;
the acquisition module is used for acquiring first metadata in a target data source to be acquired by the target client based on the acquisition executor by adopting the target acquisition plug-in.
In a third aspect, the present application provides an electronic device, including:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to execute to implement a metadata collection method as provided in the first aspect.
In a fourth aspect, the present application provides a non-transitory computer readable storage medium, which when executed by a processor of an electronic device, causes the electronic device to perform implementing a metadata collection method as provided in the first aspect.
One or more technical solutions provided in the embodiments of the present application at least have the following technical effects or advantages:
the method comprises the steps of configuring a target acquisition task for a target client; determining a target data source to be acquired according to the target acquisition task; configuring a target acquisition plug-in for a target client, wherein the target acquisition plug-in at least comprises a preset data structure, a preset connection protocol, a preset acquisition tool and a preset acquisition rule; based on the acquisition executor, the target client adopts a target acquisition plug-in to acquire first metadata in a target data source to be acquired. Compared with the prior art, only single-type metadata can be acquired, the metadata acquisition method and device can acquire various types of metadata, the coding amount is smaller, the metadata acquisition efficiency is improved, the labor intensity of staff is reduced, and the economic benefit is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a metadata collection method provided in the present application;
FIG. 2 is a flow chart of metadata collection, comparison and analysis provided herein;
fig. 3 is a schematic structural diagram of a metadata collection device provided in the present application;
fig. 4 is a schematic structural diagram of an electronic device provided in the present application.
Detailed Description
The metadata acquisition method solves the technical problem of low metadata acquisition efficiency in the prior art.
The technical scheme of the embodiment of the application aims to solve the technical problems, and the overall thought is as follows:
a metadata collection method, the method comprising: configuring a target acquisition task for a target client; determining a target data source to be acquired according to the target acquisition task; configuring a target acquisition plug-in for a target client, wherein the target acquisition plug-in at least comprises a preset data structure, a preset connection protocol, a preset acquisition tool and a preset acquisition rule; based on the acquisition executor, the target client adopts a target acquisition plug-in to acquire first metadata in a target data source to be acquired.
In order to better understand the above technical solutions, the following detailed description will refer to the accompanying drawings and specific embodiments.
First, the term "and/or" appearing herein is merely an association relationship describing associated objects, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship.
The application provides a metadata acquisition method as shown in fig. 1, which comprises the following steps of S11-S14:
step S11, configuring a target acquisition task for a target client.
And step S12, determining a target data source to be acquired according to the target acquisition task.
Step S13, configuring a target acquisition plug-in for the target client, wherein the target acquisition plug-in at least comprises a preset data structure, a preset connection protocol, a preset acquisition tool and a preset acquisition rule.
Step S14, based on the acquisition executor, the target client adopts the target acquisition plug-in to acquire first metadata in the target data source to be acquired.
With respect to step S11, a target acquisition task is configured for the target client.
For the target client, when metadata acquisition is required to be performed through the target client, a target acquisition task can be configured for the target client, and the target client can be a mobile terminal, such as a mobile phone, a computer, an intelligent wearable device and the like.
Before the target client is configured with the target acquisition task, in order to ensure the acquisition safety, before the target client is configured with the target acquisition task, the method further comprises the following steps: the target client sends a login request to the target server; the target server receives the login request and verifies the login request to generate a verification result, and the target server returns the verification result to the target client; and the target client determines whether a target acquisition task can be configured for the target client through the front end according to the received verification result.
The target server receives the login request and verifies the login request, the target server verifies the login request according to the user identity credentials, when the login request is legal, the verification result is verification passing, and when the login request is illegal, the verification result is verification failing. The target server returns the verification result, i.e. "verification passed" or "verification failed", to the target client. When the verification result is 'verification passing', the user name and the password sent by the target client can be matched with the user name and the password stored in the target server, and the target client can interact with the target server; when the verification result is "verification failed", it means that the user name and the password sent by the target client cannot be matched with the user name and the password stored in the target server, and the target client cannot interact with the target server.
With respect to step S12, a target data source to be acquired is determined according to the target acquisition task.
The determination of the target data source to be acquired may be based on the target acquisition task, and specifically may include:
constructing a definition interface collection library, wherein the definition interface collection library comprises a plurality of preset definition interfaces; constructing a data source library to be acquired; determining a target definition interface from a definition interface collection library according to the target acquisition task; and determining a target data source to be acquired corresponding to the target definition interface from the data source library to be acquired according to the target acquisition task.
A definition interface library may be constructed, where the definition interface library includes a plurality of preset definition interfaces, and each definition interface includes a method of attribute information to be collected. The definition interface may define a variety of general methods for obtaining metadata and processing metadata. It will be appreciated that collection efficiency may be improved by building a library of defined interfaces.
The data source library to be acquired can also be constructed, and the data source library to be acquired comprises a plurality of data sources to be acquired. It can be appreciated that constructing a database to be acquired can also improve the acquisition efficiency.
The task of target acquisition can be realized, the target definition interface is determined from the definition interface collection library, and the target data source to be acquired corresponding to the target definition interface is determined from the data source to be acquired, and it can be understood that the target definition interface corresponds to the target data source to be acquired one by one.
Furthermore, the dynamic expansion of the database to be acquired can be performed; and updating the definition interface collection library according to the dynamic expansion result of the data source library to be acquired.
The type of the dynamic expansion metadata can be realized by using mechanisms such as dynamic expansion and reflection of a virtual machine in Java so as to achieve the purpose of updating a data source library to be acquired. The method has the advantage that new metadata can be quickly adapted at low cost without restarting or relying on the outside.
Regarding step S13, a target acquisition plug-in is configured for the target client, where the target acquisition plug-in at least includes a preset data structure, a preset connection protocol, a preset acquisition tool, and a preset acquisition rule.
The target acquisition plug-in is used for providing acquisition of metadata of different types for a target client, in other words, the purpose of the application is not acquisition of metadata of a single type. Therefore, the above objective can be achieved by the preset data structure, the preset connection protocol, the preset collection tool and the preset collection rule in the target collection plug-in.
With respect to step S14, based on the acquisition executor, the target client acquires the first metadata in the target data source to be acquired using the target acquisition plug-in.
And the target client adopts a target acquisition plug-in to acquire first metadata from a target data source to be acquired.
Furthermore, in the process of collecting the first metadata, a time stamp can be added to the first metadata; after the target client collects the first metadata, judging the time interval between the time stamp in the first metadata and the current moment; and determining the network transmission state between the target client and the data source library to be acquired according to the preset response time and the time interval.
The first metadata and the timestamp may be combined in the form of:
first metadata # timestamp
Such as: 1789# 1454634164, 1789 before the # symbol is the first metadata, 14546354164 after the # symbol is the timestamp.
Determining a time interval according to the added time stamp 14546354164 and the time when the target client receives the time stamp 14546354164, determining the transmission state of the network corresponding to the target client and the data source library to be acquired according to the time interval and the preset response time, and if the time interval is smaller than the preset response time, indicating that the transmission state of the network corresponding to the target client and the data source library to be acquired is normal; if the time interval is greater than the preset response time, the abnormal transmission state of the network corresponding to the target client and the data source library to be collected is indicated, and the relevant responsible person can check the network transmission state of the network corresponding to the target client and the data source library to be collected.
After collecting the first metadata, further comprising: constructing a metadata base, wherein the metadata base comprises a plurality of metadata; inputting the collected first metadata into a metadata base; based on a preset analysis model, the first metadata is analyzed, and the blood-edge relationship between the first metadata and each metadata in the metadata base is determined, and in particular, reference may be made to fig. 2.
In summary, the application configures a target acquisition task for a target client; determining a target data source to be acquired according to the target acquisition task; configuring a target acquisition plug-in for a target client, wherein the target acquisition plug-in at least comprises a preset data structure, a preset connection protocol, a preset acquisition tool and a preset acquisition rule; based on the acquisition executor, the target client adopts a target acquisition plug-in to acquire first metadata in a target data source to be acquired. Compared with the prior art, only single-type metadata can be acquired, the metadata acquisition method and device can acquire various types of metadata, the coding amount is smaller, the metadata acquisition efficiency is improved, the labor intensity of staff is reduced, and the economic benefit is improved.
Based on the same inventive concept, the present application provides a metadata acquisition apparatus as shown in fig. 3, the apparatus comprising:
a task configuration module 31, configured to configure a target acquisition task for a target client;
a data source determining module 32, configured to determine a target data source to be collected according to a target collection task;
the plug-in determining module 33 is configured to configure a target acquisition plug-in for the target client, where the target acquisition plug-in at least includes a preset data structure, a preset connection protocol, a preset acquisition tool, and a preset acquisition rule;
the collection module 34 is configured to collect, based on the collection executor, first metadata in the target to-be-collected data source by using the target collection plug-in.
Further, the data source determining module 32 is configured to:
determining a target data source to be acquired according to a target acquisition task, including: constructing a definition interface collection library, wherein the definition interface collection library comprises a plurality of preset definition interfaces;
constructing a data source library to be acquired;
determining a target definition interface from a definition interface collection library according to the target acquisition task;
and determining a target data source to be acquired corresponding to the target definition interface from the data source library to be acquired according to the target acquisition task.
Further, the data source determining module 32 is configured to:
dynamically expanding a data source library to be acquired;
and updating the definition interface collection library according to the dynamic expansion result of the data source library to be acquired.
Further, the acquisition module 34 is configured to:
constructing a metadata base, wherein the metadata base comprises a plurality of metadata;
inputting the collected first metadata into a metadata base;
based on a preset analysis model, the first metadata is analyzed, and the blood-edge relation between the first metadata and each metadata in the metadata base is determined.
Further, the task configuration module 31 is configured to:
the target client sends a login request to the target server;
the target server receives the login request and verifies the login request to generate a verification result, and the target server returns the verification result to the target client;
and the target client determines whether a target acquisition task can be configured for the target client through the front end according to the received verification result.
Further, the acquisition module 34 is configured to:
adding a timestamp to the first metadata;
after the target client collects the first metadata, judging the time interval between the time stamp in the first metadata and the current moment;
and determining the network transmission state between the target client and the data source library to be acquired according to the preset response time and the time interval.
Based on the same inventive concept, the present application provides an electronic device as shown in fig. 4, including:
a processor 41;
a memory 42 for storing instructions executable by the processor 41;
wherein the processor 41 is configured to execute to implement a metadata collection method as provided previously.
Based on the same inventive concept, the present application also provides a non-transitory computer readable storage medium, which when executed by the processor 41 of the electronic device, enables the electronic device to perform a metadata collection method as provided above.
Since the electronic device described in this embodiment is an electronic device used to implement the method of information processing in this embodiment, those skilled in the art will be able to understand the specific implementation of the electronic device and various modifications thereof based on the method of information processing described in this embodiment, so how the method of this embodiment is implemented in this electronic device will not be described in detail herein. The electronic device used by those skilled in the art to implement the information processing method in the embodiments of the present application falls within the scope of protection intended by the present application.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims (9)

1. A method of metadata collection, the method comprising:
configuring a target acquisition task for a target client;
determining a target data source to be acquired according to the target acquisition task;
configuring a target acquisition plug-in for the target client, wherein the target acquisition plug-in at least comprises a preset data structure, a preset connection protocol, a preset acquisition tool and a preset acquisition rule;
based on an acquisition executor, the target client adopts the target acquisition plug-in to acquire first metadata in the target data source to be acquired.
2. The method for collecting metadata as claimed in claim 1, wherein determining a target data source to be collected according to the target collection task comprises:
constructing a definition interface collection library, wherein the definition interface collection library comprises a plurality of preset definition interfaces;
constructing a data source library to be acquired;
determining a target definition interface from the definition interface collection library according to the target acquisition task;
and determining the target data source to be acquired corresponding to the target definition interface from the data source library to be acquired according to the target acquisition task.
3. A method of metadata acquisition as claimed in claim 2, wherein the method comprises:
dynamically expanding the data source library to be acquired;
and updating the definition interface collection library according to the dynamic expansion result of the data source library to be acquired.
4. A method of metadata acquisition as claimed in claim 2, wherein the method further comprises:
constructing a metadata base, wherein the metadata base comprises a plurality of metadata;
inputting the acquired first metadata into the metadata base;
and analyzing the first metadata based on a preset analysis model, and determining the blood-edge relationship between the first metadata and each metadata in the metadata base.
5. The method of claim 2, wherein before configuring the target client with a target acquisition task, the method comprises:
the target client sends a login request to a target server;
the target server receives the login request and verifies the login request to generate a verification result, and the target server returns the verification result to the target client;
and the target client determines whether the target acquisition task can be configured for the target client through the front end according to the received verification result.
6. A method of metadata acquisition as claimed in claim 2, wherein the method further comprises:
adding a timestamp to the first metadata;
after the target client acquires the first metadata, judging the time interval between the time stamp in the first metadata and the current moment;
and determining the network transmission state between the target client and the data source library to be acquired according to the preset response time and the time interval.
7. A metadata acquisition device, the device comprising:
the task configuration module is used for configuring a target acquisition task for the target client;
the data source determining module is used for determining a target data source to be acquired according to the target acquisition task;
the plug-in determining module is used for configuring a target acquisition plug-in for the target client, wherein the target acquisition plug-in at least comprises a preset data structure, a preset connection protocol, a preset acquisition tool and a preset acquisition rule;
and the acquisition module is used for acquiring the first metadata in the target data source to be acquired by the target client side by adopting the target acquisition plug-in based on the acquisition executor.
8. An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute to implement a metadata collection method as claimed in any one of claims 1 to 6.
9. A non-transitory computer readable storage medium, which when executed by a processor of an electronic device, causes the electronic device to perform a metadata collection method implementing any of claims 1 to 6.
CN202311678519.0A 2023-12-07 2023-12-07 Metadata acquisition method, device, equipment and medium Pending CN117667454A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311678519.0A CN117667454A (en) 2023-12-07 2023-12-07 Metadata acquisition method, device, equipment and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311678519.0A CN117667454A (en) 2023-12-07 2023-12-07 Metadata acquisition method, device, equipment and medium

Publications (1)

Publication Number Publication Date
CN117667454A true CN117667454A (en) 2024-03-08

Family

ID=90074838

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311678519.0A Pending CN117667454A (en) 2023-12-07 2023-12-07 Metadata acquisition method, device, equipment and medium

Country Status (1)

Country Link
CN (1) CN117667454A (en)

Similar Documents

Publication Publication Date Title
CN112380093A (en) Operation and maintenance processing method and device and computer equipment
EP2769304B1 (en) Telemetry file hash and conflict detection
CN109376069B (en) Method and device for generating test report
CN105049287A (en) Log processing method and log processing devices
CN104246767A (en) Telemetry system for a cloud synchronization system
CN107168844B (en) Performance monitoring method and device
WO2019029281A1 (en) Collision check data processing method and apparatus, electronic device, and storage medium
CN111552633A (en) Interface abnormal call testing method and device, computer equipment and storage medium
CN110798445B (en) Public gateway interface testing method and device, computer equipment and storage medium
CN104980421B (en) Batch request processing method and system
CN111814441A (en) Report generation method and device, electronic equipment and storage medium
CN111597388A (en) Sample collection method, device, equipment and medium based on distributed system
CN106301975B (en) Data detection method and device
CN111427796A (en) System testing method and device and electronic equipment
CN107368407B (en) Information processing method and device
CN108595178B (en) Hook-based data acquisition method, device and equipment
CN117667454A (en) Metadata acquisition method, device, equipment and medium
CN114650211A (en) Fault repairing method, device, electronic equipment and computer readable storage medium
CN114239296A (en) Data processing method and device and electronic equipment
CN107948316B (en) File synchronization method, device and equipment
CN113448985A (en) API (application program interface) interface generation method, calling method and device and electronic equipment
CN109981548B (en) Method and device for analyzing charging message
CN112948195A (en) Interface testing method and device, electronic equipment and storage medium
CN112487997A (en) Portrait feature extraction method and device
CN114116544A (en) Method, device and equipment for acquiring slot information and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination