CN111274326A - Feature data importing method, feature data managing and controlling method, feature data importing device, feature data managing and controlling device and electronic equipment - Google Patents

Feature data importing method, feature data managing and controlling method, feature data importing device, feature data managing and controlling device and electronic equipment Download PDF

Info

Publication number
CN111274326A
CN111274326A CN201811478228.6A CN201811478228A CN111274326A CN 111274326 A CN111274326 A CN 111274326A CN 201811478228 A CN201811478228 A CN 201811478228A CN 111274326 A CN111274326 A CN 111274326A
Authority
CN
China
Prior art keywords
feature
metadata
authority
user
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201811478228.6A
Other languages
Chinese (zh)
Inventor
王志明
何龙
杜龙志
付俊强
范育峰
余芳
李奘
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Didi Infinity Technology and Development Co Ltd
Original Assignee
Beijing Didi Infinity Technology and Development Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Didi Infinity Technology and Development Co Ltd filed Critical Beijing Didi Infinity Technology and Development Co Ltd
Priority to CN201811478228.6A priority Critical patent/CN111274326A/en
Publication of CN111274326A publication Critical patent/CN111274326A/en
Pending legal-status Critical Current

Links

Images

Abstract

The application provides a feature data importing method, a feature data management and control method, a feature data importing device and an electronic device, wherein the feature data importing method is applied to a feature platform server and comprises the following steps: acquiring feature metadata imported from a feature table to a metadata table by a user, wherein the feature table is a structured table formed by feature data acquired by the user, and the metadata table is a table corresponding to a template in a feature database; checking whether the user has import authority of the feature metadata; if the user has import authority, checking whether the characteristic metadata meets the requirement of a metadata table; if the feature metadata meets the requirements, the feature metadata is written into a metadata table. Compared with the existing mode of importing the feature data through manual input, the feature data importing efficiency is greatly improved.

Description

Feature data importing method, feature data managing and controlling method, feature data importing device, feature data managing and controlling device and electronic equipment
Technical Field
The application relates to the technical field of big data, in particular to a feature data importing method, a feature data managing and controlling method, a feature data importing device, a feature data managing and controlling device and electronic equipment.
Background
With the rapid development of neural network technology, more and more company businesses can use machine learning or model training, and need to acquire a large amount of sample data or characteristics, so that it is very important to establish a characteristic platform and manage the characteristic data through the characteristic platform. The existing feature platform usually needs to acquire manual form filling of workers, form building is completed, feature data importing is completed, and working efficiency is poor.
Disclosure of Invention
In view of this, embodiments of the present application provide a feature data importing method, a feature data managing and controlling method, a feature data importing device, and an electronic device, where feature metadata are first obtained from a feature table of a user, and then data is imported when it is verified that the user has permission and the data meets requirements of the metadata table.
According to one aspect of the present application, an electronic device is provided that may include one or more storage media and one or more processors in communication with the storage media. One or more storage media store machine-readable instructions executable by a processor. When the electronic device runs, the processor and the storage medium are communicated through the bus, and the processor executes the machine readable instructions to execute one or more methods in the feature data importing method and/or the feature data managing method.
According to another aspect of the present application, there is provided a method for importing feature data, the method being applied to a feature platform server, and including: acquiring feature metadata imported from a feature table to a metadata table by a user, wherein the feature table is a structured table formed by the feature data acquired by the user, and the metadata table is a table corresponding to a template in a feature database; checking whether the user has the import authority of the feature metadata; if the user has import authority, checking whether the feature metadata meets the requirement of the metadata table; and if the characteristic metadata meets the requirement, writing the characteristic metadata into the metadata table.
In a preferred embodiment of the present application, the step of obtaining the feature metadata imported from the feature table to the metadata table by the user includes: monitoring whether a user drags the feature data in the feature table; and if so, acquiring the feature metadata of the dragged feature data and a metadata table corresponding to the feature metadata.
In a preferred embodiment of the present application, the step of obtaining the metadata table corresponding to the feature metadata includes: if a new command of the user is received, a metadata table is newly established for the feature metadata; and if an adding instruction of the user is received, searching a metadata table matched with the feature metadata, and taking the searched metadata table as the metadata table corresponding to the feature metadata.
In a preferred embodiment of the present application, the step of checking whether the user has the import right of the feature metadata includes: sending a query request to an authority server; the query request carries the account of the user and the identifier of the metadata table; if receiving prompt information with permission returned by the permission server, determining that the user has the import permission of the feature metadata; the permission prompt information is generated when the permission server finds that the account of the user corresponds to the metadata table.
In a preferred embodiment of the present application, the requirements of the metadata table include: a partition requirement for the feature table, a country code requirement for the feature table with a country code, and a data uniqueness requirement.
In a preferred embodiment of the present application, the feature table is a Hive table.
According to another aspect of the present application, there is provided a method for managing and controlling feature data, the method being applied to a feature platform server, the feature platform server storing a feature database, feature metadata in a feature metadata table in the feature database being imported by the method in the previous aspect, the method including: receiving a characteristic data operation request of a user; wherein the characteristic data operation request carries an operation type; searching the management authority of the user; and responding to the operation corresponding to the operation type if the searched management authority comprises the operation authority corresponding to the operation type.
In a preferred embodiment of the present application, if the found management authority includes an operation authority corresponding to the operation type, the step of responding to the operation corresponding to the operation type includes: if the searched management authority is a viewing authority and the operation type is a viewing operation, displaying data corresponding to the characteristic data operation request; if the searched management authority is an extraction authority or an approval authority and the operation type is a viewing operation or an extraction operation, providing data corresponding to the characteristic data operation request; and if the searched management authority is an approval authority and the operation type is an approval operation, allocating a viewing authority or an extraction authority of the data corresponding to the characteristic data operation request to the user corresponding to the approval operation.
In a preferred embodiment of the present application, the method further comprises: distributing the management authority of the feature metadata in the metadata table according to a preset authority dimension; wherein the preset authority dimension comprises: feature type, user type, region scope.
In a preferred embodiment of the present application, the step of assigning the management authority of the feature metadata in the metadata table according to a preset authority dimension includes: if the preset authority dimension comprises the characteristic type, acquiring a security label corresponding to the characteristic type of the current metadata table; and distributing the management authority of the feature metadata in the metadata table according to the security label.
In a preferred embodiment of the present application, the step of assigning the management authority of the feature metadata in the metadata table according to a preset authority dimension includes: and if the preset authority dimension comprises the user type, distributing the management authority of the feature metadata in the metadata table according to the management level and/or the value level in the user type.
In a preferred embodiment of the present application, the step of assigning the management authority of the feature metadata in the metadata table according to a preset authority dimension includes: if the preset authority dimension comprises an area range, dividing the feature metadata in the metadata table according to the area range, and distributing management authority to the divided feature metadata.
According to another aspect of the present application, there is provided an apparatus for importing feature data, the apparatus being applied to a feature platform server, including: the data acquisition module is used for acquiring feature metadata imported from a feature table to a metadata table by a user, wherein the feature table is a structured table formed by the feature data acquired by the user, and the metadata table is a table corresponding to a template in a feature database; the first checking module is used for checking whether the user has the import permission of the feature metadata; the second checking module is used for checking whether the characteristic metadata meets the requirement of the metadata table or not if the user has the import permission; and the data writing module is used for writing the characteristic metadata into the metadata table if the characteristic metadata meets the requirements.
In a preferred embodiment of the present application, the data obtaining module includes: the monitoring module is used for monitoring whether a user drags the feature data in the feature table; and the data acquisition submodule is used for acquiring the feature metadata of the dragged feature data and the metadata table corresponding to the feature metadata when the dragging operation of the user on the feature data in the feature table is monitored.
In a preferred embodiment of the present application, the data obtaining sub-module is further configured to: if a new command of the user is received, a metadata table is newly established for the feature metadata; and if an adding instruction of the user is received, searching a metadata table matched with the feature metadata, and taking the searched metadata table as the metadata table corresponding to the feature metadata.
In a preferred embodiment of the present application, the first checking module is further configured to: sending a query request to an authority server; the query request carries the account of the user and the identifier of the metadata table; if receiving prompt information with permission returned by the permission server, determining that the user has the import permission of the feature metadata; the permission prompt information is generated when the permission server finds that the account of the user corresponds to the metadata table.
In a preferred embodiment of the present application, the requirements of the metadata table include: a partition requirement for the feature table, a country code requirement for the feature table with a country code, and a data uniqueness requirement.
In a preferred embodiment of the present application, the feature table is a Hive table.
According to another aspect of the present application, there is provided an apparatus for managing and controlling feature data, the apparatus is applied to a feature platform server, the feature platform server stores a feature database, and feature metadata in a feature metadata table in the feature database is imported by the apparatus in the previous aspect, the apparatus includes: the request receiving module is used for receiving a characteristic data operation request of a user; wherein the characteristic data operation request carries an operation type; the authority searching module is used for searching the management authority of the user; and the operation response module is used for responding to the operation corresponding to the operation type if the searched management authority contains the operation authority corresponding to the operation type.
In a preferred embodiment of the present application, the operation response module is further configured to: if the searched management authority is a viewing authority and the operation type is a viewing operation, displaying data corresponding to the characteristic data operation request; if the searched management authority is an extraction authority or an approval authority and the operation type is a viewing operation or an extraction operation, providing data corresponding to the characteristic data operation request; and if the searched management authority is an approval authority and the operation type is an approval operation, allocating a viewing authority or an extraction authority of the data corresponding to the characteristic data operation request to the user corresponding to the approval operation.
In a preferred embodiment of the present application, the apparatus further comprises: the authority distribution module is used for distributing the management authority of the feature metadata in the metadata table according to a preset authority dimension; wherein the preset authority dimension comprises: feature type, user type, region scope.
In a preferred embodiment of the present application, the right assignment module is further configured to: if the preset authority dimension comprises the characteristic type, acquiring a security label corresponding to the characteristic type of the current metadata table; and distributing the management authority of the feature metadata in the metadata table according to the security label.
In a preferred embodiment of the present application, the right assignment module is further configured to: and if the preset authority dimension comprises the user type, distributing the management authority of the feature metadata in the metadata table according to the management level and/or the value level in the user type.
In a preferred embodiment of the present application, the right assignment module is further configured to: if the preset authority dimension comprises an area range, dividing the feature metadata in the metadata table according to the area range, and distributing management authority to the divided feature metadata.
According to another aspect of the present application, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method for importing feature data or the method for managing feature data as described in any one of the above.
In the embodiment of the application, firstly, feature metadata are obtained from a feature table of a user, the feature table is a structured table formed by feature data collected by the user, the metadata table is a table corresponding to a template in a feature database, and then when the condition that the user has import permission of the feature metadata and the feature metadata meet the requirements of the metadata table is verified, the feature metadata are written into the metadata table.
In order to make the aforementioned objects, features and advantages of the embodiments of the present application more comprehensible, embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
Fig. 1 is a block diagram illustrating an import system of feature data (or a management and control system of feature data;
fig. 2 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;
fig. 3 is a flowchart illustrating a method for importing feature data according to an embodiment of the present disclosure;
fig. 4 is a schematic diagram illustrating feature import in a feature data import method provided in an embodiment of the present application;
fig. 5 is a flowchart illustrating a method for managing and controlling feature data according to an embodiment of the present application;
fig. 6 is a schematic structural diagram illustrating an apparatus for importing feature data according to an embodiment of the present application;
fig. 7 shows a schematic structural diagram of a feature data management and control device provided in an embodiment of the present application.
Detailed Description
In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are for illustrative and descriptive purposes only and are not used to limit the scope of protection of the present application. Additionally, it should be understood that the schematic drawings are not necessarily drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of the present application. It should be understood that the operations of the flow diagrams may be performed out of order, and steps without logical context may be performed in reverse order or simultaneously. One skilled in the art, under the guidance of this application, may add one or more other operations to, or remove one or more operations from, the flowchart.
In addition, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
In order to enable those skilled in the art to use the present disclosure, the following embodiments are given in conjunction with a specific application scenario "network appointment service feature platform". It will be apparent to those skilled in the art that the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the application. Although the present application is described primarily in the context of a network appointment payment scenario, it should be understood that this is only one exemplary embodiment. The method and the device can be applied to any other characteristic platform scene needing characteristic data collection. For example, the method and the device can be applied to different take-away ordering service characteristic platform scenes, shared economic service characteristic platform scenes (such as shared bicycle, shared charging pile and shared automobile), and the like. In the scene of the network car booking service characteristic platform, the method can be applied to the following vehicles: may include a taxi, a private car, a tailgating, a bus, an unmanned vehicle, or the like, or any combination thereof. The present application may also include any service system for collecting characteristic data, for example, a system for sending and/or receiving couriers, a service system for business transactions between buyers and sellers. Applications of the system or method of the present application may include web pages, plug-ins for browsers, client terminals, customization systems, internal analysis systems, or artificial intelligence robots, among others, or any combination thereof.
It should be noted that in the embodiments of the present application, the term "comprising" is used to indicate the presence of the features stated hereinafter, but does not exclude the addition of further features.
One aspect of the application relates to a feature data importing system, which first obtains feature metadata imported from a feature table to a metadata table by a user, wherein the feature table is a structured table formed by feature data collected by the user, the metadata table is a table corresponding to a template in a feature database, then, whether the user has import authority of the feature metadata is checked, if the user has the import authority, whether the feature metadata meets the requirement of the metadata table is checked, and if the feature metadata meets the requirement, the feature metadata is written into the metadata table. According to the embodiment of the application, the feature metadata are obtained from the feature table of the user, when the fact that the user has the import permission of the feature metadata is verified, and the feature metadata meet requirements of the metadata table, the feature metadata are written into the metadata table, and compared with the existing mode that the feature data are imported through manual input, the feature data import efficiency is greatly improved.
Another aspect of the present application relates to a feature data management and control system, which is applied to a feature platform server, where the feature platform server stores a feature database, and feature metadata in a feature metadata table in the feature database is imported by the system in the previous aspect, where the feature data management and control system first receives a feature data operation request of a user, where the feature data operation request carries an operation type; then searching the management authority of the user; and responding to the operation corresponding to the operation type if the searched management authority comprises the operation authority corresponding to the operation type. On the basis of feature data import, the embodiment realizes a management and control mode when a user extracts feature data, searches the feature data management authority corresponding to the user according to the operation request of the user, and responds to different operations based on different management authorities.
It is noted that, before the present application is filed, for the problem of importing feature data in a feature platform, the currently known and disclosed solutions are mainly: the characteristic table is newly built on the characteristic platform by workers, and data is manually input into the characteristic table, so that the method is complex in operation, easy to make mistakes and low in working efficiency. The feature data importing system provided by the application can acquire the feature metadata from the feature table of the user, and write the feature metadata into the feature metadata table when the user is verified to have the importing authority of the feature metadata and the feature metadata meet the requirements of the feature metadata table.
Fig. 1 is a block diagram of an import system of feature data (or a management and control system of feature data) according to some embodiments of the present application. The feature data import system may include one or more of a feature platform server 110, a network 120, a user terminal 130, and a database 140, and the feature platform server 110 may include a processor therein that performs instruction operations.
In some embodiments, the feature platform server 110 may be a single server or a group of servers. The set of servers can be centralized or distributed (e.g., the servers 110 can be a distributed system). In some embodiments, the feature platform server 110 may be local or remote with respect to the user terminal. For example, feature platform server 110 may access information and/or data stored in user terminal 130, or database 140, or any combination thereof, via network 120. As another example, the feature platform server 110 may be directly connected to at least one of the user terminal 130 and the database 140 to access stored information and/or data. In some embodiments, feature platform server 110 may be implemented on a cloud platform; by way of example only, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud (community cloud), a distributed cloud, an inter-cloud, a multi-cloud, and the like, or any combination thereof.
In some embodiments, the feature platform server 110 may include a processor. The processor may process information and/or data related to the user request to perform one or more of the functions described herein. In some embodiments, a processor may include one or more processing cores (e.g., a single-core processor (S) or a multi-core processor (S)). Merely by way of example, a Processor may include a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), an Application specific Instruction Set Processor (ASIP), a Graphics Processing Unit (GPU), a Physical Processing Unit (PPU), a Digital Signal Processor (DSP), a Field Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), a controller, a microcontroller Unit, a Reduced Instruction Set computer (Reduced Instruction Set Computing, RISC), a microprocessor, or the like, or any combination thereof.
In some embodiments, the network 120 may be any type of wired or wireless network, or combination thereof. Merely by way of example, Network 120 may include a wired Network, a Wireless Network, a fiber optic Network, a telecommunications Network, an intranet, the internet, a Local Area Network (LAN), a Wide Area Network (WAN), a Wireless Local Area Network (WLAN), a Metropolitan Area Network (MAN), a Wide Area Network (WAN), a Public Switched Telephone Network (PSTN), a bluetooth Network, a ZigBee Network, a Near Field Communication (NFC) Network, or the like, or any combination thereof. In some embodiments, network 120 may include one or more network access points. For example, network 120 may include wired or wireless network access points, such as base stations and/or network switching nodes, through which one or more components of an import system of feature data (or a governing system of feature data) may connect to network 120 to exchange data and/or information.
In some embodiments, user terminal 130 may comprise a mobile device, a tablet computer, a laptop computer, or a built-in device in a motor vehicle, etc., or any combination thereof. In some embodiments, the mobile device may include a smart home device, a wearable device, a smart mobile device, a virtual reality device, an augmented reality device, or the like, or any combination thereof.
Database 140 may store data and/or instructions. In some embodiments, database 140 may store data obtained from user terminal 130. In some embodiments, database 140 may store data and/or instructions for the exemplary methods described herein. In some embodiments, the database 140 may include mass storage, removable storage, volatile Read-write Memory, or Read-Only Memory (ROM), among others, or any combination thereof. By way of example, mass storage may include magnetic disks, optical disks, solid state drives, and the like; removable memory may include flash drives, floppy disks, optical disks, memory cards, zip disks, tapes, and the like; volatile read-write memory may include Random Access Memory (RAM); the RAM may include Dynamic RAM (DRAM), Double data Rate Synchronous Dynamic RAM (DDR SDRAM); static RAM (SRAM), Thyristor-Based Random Access Memory (T-RAM), Zero-capacitor RAM (Zero-RAM), and the like. By way of example, ROMs may include Mask Read-Only memories (MROMs), Programmable ROMs (PROMs), Erasable Programmable ROMs (PERROMs), Electrically Erasable Programmable ROMs (EEPROMs), compact disk ROMs (CD-ROMs), digital versatile disks (ROMs), and the like. In some embodiments, database 140 may be implemented on a cloud platform. By way of example only, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, across clouds, multiple clouds, or the like, or any combination thereof.
In some embodiments, a database 140 may be connected to the network 120 to communicate with one or more components in the import system of feature data (or the governing system of feature data) 100. One or more components in the import system of feature data (or the governing system of feature data) may access data or instructions stored in database 140 via network 120.
Fig. 2 is a method for importing feature data according to an embodiment of the present application, where the method is applied to a feature platform server, and specifically includes the following steps:
in step S202, feature metadata imported from the feature table to the metadata table by the user is acquired.
The feature table is a structured table formed by feature data collected by a user, and the metadata table is a table corresponding to a template in the feature database. The user may be an operator inside a company or a person who performs data analysis or model building, and in the specific implementation, the user may complete importing the feature data metadata through a visual interface. For example, a user may directly drag feature data from the feature table to a newly created metadata table in the feature platform server, or may first generate an intermediate table and then import the intermediate table, that is, import the data into an existing metadata table in the feature platform server.
Step S204, checking whether the user has the import authority of the feature metadata.
In specific implementation, the data in some metadata tables in the feature platform server has been assigned with an authority, and only for some users who have authority to view the data, new data can be imported, and unauthorized users cannot import the data, so that after the feature platform server obtains the feature metadata exported from the user feature table, the import authority of the feature metadata is further verified, and specific verification methods are various, and the verification of whether the import authority of the feature metadata exists can be performed by other authority servers, or can be performed by a server of the feature platform itself, which is not specifically limited herein.
Step S206, if the user has import authority, whether the feature metadata meets the requirement of the metadata table is checked.
When a user imports data, besides checking whether the user has import permission, the imported feature metadata also needs to be checked whether to meet the requirement of a metadata table, and if not, the data cannot be imported into the metadata table of the platform. Specific metadata table requirements include, but are not limited to: partitioning requirements for the feature table, country code requirements for the feature table with country code, and data uniqueness requirements.
In step S208, if the feature metadata meets the requirement, the feature metadata is written into the metadata table.
When the feature metadata which the user wants to import is verified to meet the requirement of the metadata table, the feature platform server writes the feature metadata which is obtained at the beginning into the metadata table which is appointed by the user, wherein the feature metadata comprises a newly-built metadata table or an existing metadata table.
According to the feature data importing method provided by the embodiment of the application, the feature metadata can be acquired from the feature table of the user, when the condition that the user has the importing authority of the feature metadata and the feature metadata meet the requirements of the metadata table is verified, the feature metadata is written into the metadata table, and compared with the existing mode of importing the feature data through manual input, the feature data importing efficiency is greatly improved.
As a preferred mode of the embodiment of the application, the user feature table is a Hive table, the Hive is a data warehouse tool based on Hadoop, the structured data file can be mapped into a database table, a simple sql query function is provided, and the sql statement can be converted into a MapReduce task for running. Hive stores metadata in a database, such as MySQL, derby. The metadata in the Hive table comprises the name of the table, the column and the partition of the table and the attributes of the column and the partition, the attributes of the table, the directory where the data of the table is located and the like.
The step of acquiring the feature metadata imported from the feature table to the metadata table by the user may include the following steps: monitoring whether a user drags the feature data in the feature table; and if so, acquiring the feature metadata of the dragged feature data and a metadata table corresponding to the feature metadata.
In specific implementation, an operator selects feature data to be imported from the Hive table, drags the feature data to the metadata table of the feature platform, and when the feature platform server monitors the dragging operation, further obtains the metadata of the dragged feature data and the metadata table to be imported, so that the metadata of the dragged feature data is finally imported to the metadata table of the feature platform. The metadata table in the feature platform can be a MySQL metadata table, and can also be other types of metadata tables.
As shown in fig. 3, the operator selects feature data Af1 and Af2 in Hive table a, the analyst selects feature data Bf1 in Hive table B, the modeler selects feature data Cf3 in Hive table C, and if the import is successful, the MySQL metadata table in the final feature platform includes: af1, Af2, Bf1 and Cf 3.
The above process of obtaining the metadata table corresponding to the feature metadata includes the following steps: if a new command of a user is received, a metadata table is newly established for the feature metadata; and if an adding instruction of a user is received, searching a metadata table matched with the characteristic metadata, and taking the searched metadata table as the metadata table corresponding to the characteristic metadata.
Taking the network appointment service feature platform as an example, an operator, a data analyzer or a modeling person first collects a large amount of business data, such as data of passengers and drivers in business orders every day, and the driver data includes: the name, age, departure time, registration time, address and the like, a Hive feature table is formed by data collected on the same day, then data needing to be imported into a feature platform are selected from the Hive feature table, and the data are imported into a metadata table in the feature platform in a dragging mode. For example, data for express service lines, taxi service lines and special car service lines are stored in a metadata table stored in a feature platform server, and data which a user wants to import is express data, the user can directly select an adding instruction to enable the server to search a metadata table matched with the feature metadata, so that the user can import express data into the searched metadata table, if the user wants to import data of a new service line, such as data of a tailwind service line, a new establishing instruction can be sent to the server to enable the server to establish a metadata table corresponding to the tailwind service line on a feature platform, and then the data which the user wants to import is dragged into the newly established metadata table.
In order to ensure security, a user is prevented from uploading data or changing data at will, when the user imports feature data, it is necessary to check whether the user has import permission, specifically, the process of checking whether the user has import permission of the feature metadata includes the following steps: sending a query request to an authority server; the query request carries an account of the user and an identifier of the metadata table; if receiving prompt information with permission returned by the permission server, determining that the user has the import permission of the characteristic metadata; the prompt information with the authority is generated when the authority server finds that the account of the user corresponds to the metadata table.
In specific implementation, each user has an account number for logging in the feature platform, a plurality of databases are correspondingly arranged under each account number, a plurality of metadata tables are stored in each database, different tables in different databases with operation authority can be operated through the account number login, and under the condition of authorization, data of the different account numbers can be mutually accessed. When the user imports the feature data, the feature platform server can automatically verify whether the user has import authority through another independent authority server and return a verification result. Specifically, after the feature platform server obtains feature metadata imported from a feature table to a metadata table by a user, a query request is automatically sent to the authority server; the query request carries an account of the user and an identifier of the metadata table; the authority server inquires the data of the metadata table through the account number of the user and the identifier of the metadata table, if the data can be inquired, the authority server indicates that the user has the import authority of the characteristic metadata, and returns the authorized prompt information to the characteristic platform server; otherwise, a prompt message without permission is returned to the feature platform server, that is, the user cannot import the feature data.
After the user is verified to have the import authority of the feature metadata, whether the feature metadata meets the requirement of a metadata table is further verified, wherein the requirement of the metadata table comprises the following steps: partitioning requirements for the feature table, country code requirements for the feature table with country code, and data uniqueness requirements.
Such as: the concept of partitions in the Hive table needs to check whether each partition is normally produced, and whether each partition is normally produced or not has a flag file. In addition, a company has many foreign services, and for international reasons, it is necessary to determine whether the Hive table corresponding to a foreign service includes a country code and whether the included international code is consistent with a standard country code, so as to determine whether the feature table with the country code satisfies the requirement of the country code. In addition, because the data in the user characteristic table are all data after rough machining, and each passenger or driver ID corresponds to a unique piece of data every day, whether the characteristic metadata meets the data uniqueness requirement is determined by judging whether each ID corresponds to only one piece of data. The data after rough machining can be better used for modeling or analysis and the like.
Fig. 4 is a characteristic data management and control method provided in an embodiment of the present application, where the method is applied to a characteristic platform server, where a characteristic database is stored in the characteristic platform server, and characteristic metadata in a characteristic metadata table in the characteristic database is imported by using the above-mentioned feature data importing method, where the characteristic data management and control method specifically includes the following steps:
step S402, receiving a characteristic data operation request of a user, wherein the characteristic data operation request carries an operation type.
Step S404, searching the management authority of the user.
Step S406, if the found management authority includes an operation authority corresponding to the operation type, responding to the operation corresponding to the operation type.
After the user imports the feature data into the metadata table of the feature platform by the import method of the feature data provided by the embodiment, the user can also perform operations of viewing, extracting or examining and approving the data according to the management authority of the user on the data, so that the user can conveniently use a large amount of data to perform modeling or big data analysis and the like. Specifically, a user first sends a feature data operation request to a feature platform server, where the request carries an operation type, such as: viewing, extracting or approving operations. And after receiving the characteristic data operation request of the user, the characteristic platform server searches the corresponding management authority of the user based on the operation type, and responds to the operation corresponding to the operation type if the searched management authority comprises the operation authority corresponding to the operation type.
On the basis of feature data import, the embodiment realizes a management and control mode when a user extracts feature data, searches the feature data management authority corresponding to the user according to the operation request of the user, and responds to different operations based on different management authorities.
The process of responding to the operation corresponding to the operation type if the searched management authority contains the operation authority corresponding to the operation type comprises the following steps:
(1) and if the searched management authority is the viewing authority and the operation type is the viewing operation, displaying the data corresponding to the characteristic data operation request.
(2) And if the searched management authority is the extraction authority or the approval authority and the operation type is the viewing operation or the extraction operation, providing data corresponding to the characteristic data operation request.
(3) And if the searched management authority is the approval authority and the operation type is the approval operation, distributing the viewing authority or the extraction authority of the data corresponding to the characteristic data operation request for the user corresponding to the approval operation.
The viewing authority refers to the authority that a user who possesses the viewing authority can only view the characteristic data possessed by the user; the extraction authority means that a user who has the extraction authority can check the owned feature data, and meanwhile, when the features are extracted and used, the feature data do not need to pass through a feature responsible person, namely, the examination and approval of the user to which the feature data belong; the approval authority means that a user who owns the approval authority can check the owned feature data, and can even own the approval authority of the feature data without being approved by a feature responsible person when the feature data are extracted and used. Therefore, the levels of the viewing right, the extraction right and the approval right are sequentially increased, and the latter includes the former.
The extraction operation refers to that according to the feature data selected by the user, the feature platform server writes the feature data into a Hive temporary table with custom TTL (Time-to-Live) by executing an automatically generated SQL statement, and then the user can use the Hive temporary table to calculate query data according to own services. The examination and approval operation means that the user can determine whether to pass the examination and approval according to the sensitivity and confidentiality of the feature data, namely, the security level label printed on the feature data by the server in advance according to security regulations.
There are two cases of automatic approval, the first case where the user selects a time period when extracting feature data, and if the features are extracted again next time and within this time period, no approval is required. For example, three features of F1, F2 and F3 in the time period of 2018/11/01-2018/11/14 are provided for the first time, three features of F2, F3 and F4 in the time period of 2018/11/01-2018/11/10 are extracted for the next time, so that F2 and F3 are automatically approved, and F4 needs to be approved because F4 is not extracted before or is used out of date; in the second case, the extraction of non-confidential features by high value users (alternatively referred to as VIP users, or high credit users) may be automatically approved by the feature platform server. The factors of the evaluation index of the high-value user are as follows: firstly, the characteristics extracted on the platform are used by the user for model, operation activity or data analysis, and the characteristics are fed back to the real effectiveness benefit index of the system (for example, 10000 new users are pulled in, the AUC of the model is improved by 5%, and the like); secondly, extracting the frequency of the features by using a system; thirdly, using the characteristics on the system, and feeding back the number of the new characteristics of the system after the new characteristics are generated after the characteristics are applied to the service.
The control method of the characteristic data further comprises the following steps: distributing the management authority of the feature metadata in the metadata table according to a preset authority dimension; wherein, the preset authority dimension comprises: feature type, user type, region scope.
If the preset authority dimension comprises the characteristic type, acquiring a security label corresponding to the characteristic type of the current metadata table; and assigning management authority of the feature metadata in the metadata table according to the security label. And if the preset authority dimension comprises the user type, distributing the management authority of the characteristic metadata in the metadata table according to the management level and/or the value level in the user type. If the preset authority dimension comprises an area range, dividing the feature metadata in the metadata table according to the area range, and distributing management authority to the divided feature metadata.
From a data security perspective, rights assignment takes into account the following factors: feature type, user type, area scope. The person responsible for the characteristics, i.e., the user, can assign the management authority (e.g., viewing authority, extraction authority, approval authority) to the characteristic data owned by the user according to the characteristic type (e.g., the characteristic in the aspect of vehicle performance, the characteristic in the aspect of driver driving habit, the characteristic in the aspect of driving track, the characteristic in the aspect of order receiving quantity or money amount, etc.), the user type (e.g., the ordinary user, the high-value user, etc.), the area range (e.g., the first-line city, the second-line city or the city name, etc.).
The feature responsible person allocates the feature set F1 to the user set U1, and simultaneously allocates the region scope set C1, and the management authority V, so that the user of the user set U1 can only have the management authority V in the management authority set V on the feature platform for the feature data in the feature set F1 in the region scope set C1.
The management of the feature data by the existing feature platform is usually based on the management of the whole table, that is, once a certain user has the right to use a certain table, all the feature data in the table are publicly available to the user, so if some sensitive data exists in the table, the data leakage is caused, and the security of the feature data management is poor.
In this embodiment, the metadata table is based on the management of the feature fields, for example, in the Hive table, for the fields of the basic data types (e.g., int, bigint, default, string, etc.), the granularity of management is these fields; for complex data types such as struct and map, each element in the struct type needs to be managed as a feature field, and each Key in the map type needs to be managed as a feature field.
For example, there are 10 feature data in a table, and the feature platform server may be allocated to only 2 of the users, and the other 8 users may still not be visible. However, the conventional management method is accurate to the Hive table, for example, if a user is authorized with some features in one table, the feature field of the whole table is authorized to the user. Thus, if a table contains confidential information such as bank cards, identification numbers, etc., the two fields that should not be authorized are authorized because the authority of the table is authorized.
When the user uses the feature data, the three-dimensional data can be used based on the feature dimension, the time dimension and the crowd/city dimension.
The feature platform almost has all feature data of a company to which the platform belongs, and the feature dimension means that the feature data which can be extracted is a subset of the feature data owned by the feature platform. The crowd/city dimension means that if a user already owns a batch of crowd (such as passenger ID, driver ID, etc.), and the batch of crowd can be dispersed in each city where business is developed, or can be a full number of users in some cities, the user can upload data of the batch of crowd to a platform, and the feature data obtained by the end user is only the feature data of the batch of crowd; if the user does not have crowd information, feature data of some cities or all cities can be obtained by selecting the cities from the feature platform.
The time dimension refers to a time range in which extraction needs to be selected when the feature data is extracted, wherein the selectable earliest extraction time refers to the time corresponding to the earliest created feature in the feature data selected by the user. Since all feature data does not exist at the time if an earlier time is selected, the data finally acquired by the user is empty (or has no data), which causes misunderstanding for the user and waste of cluster resources. With respect to the selectable latest extraction time, there is theoretically no limit, including that a day in the future may also be used as a cut-off time for extraction. In this case, the data source needs to ensure that the data can be produced all the time, otherwise, if the characteristic data is not produced on time, the user may not obtain the corresponding data in the next day.
The time range is selected in the following three cases:
(1) if the time range selected by the user is a certain time period in the history, the feature platform allocates feature data of the time period to the user at one time.
(2) If the user-selected time range is a time period in the future, the feature platform will assign feature data to the user sequentially day by day when the time reaches the time period.
(3) If the user selects a time period starting from a certain day of the history and ending on a certain day in the future, the feature platform allocates feature data of the day selected by the history to the user first, and then allocates feature data of the day to the user day by day until the extraction deadline is reached.
Fig. 5 is a block diagram illustrating an importing apparatus of feature data according to some embodiments of the present application, where the function implemented by the importing apparatus of feature data corresponds to the steps performed by the above method. The device may be understood as the server or the processor of the server, or may be understood as a component that is independent of the server or the processor and implements the functions of the present application under the control of the server, and as shown in the figure, the importing device of the feature data may include: a data fetch module 502, a first check module 504, a second check module 506, and a data write module 508.
The data obtaining module 502 is configured to obtain feature metadata imported from a feature table to a metadata table by a user, where the feature table is a structured table formed by feature data collected by the user, and the metadata table is a table corresponding to a template in a feature database; a first checking module 504, configured to check whether a user has an import right of the feature metadata; a second checking module 506, configured to check whether the feature metadata meets the requirement of the metadata table if the user has the import permission; a data writing module 508, configured to write the feature metadata into the metadata table if the feature metadata meets the requirement.
In the embodiment of the present application, first, the data obtaining module 502 obtains feature metadata from a feature table of a user, the feature table is a structured table formed by feature data collected by the user, the metadata table is a table corresponding to a template in a feature database, and then, when it is verified that the user has an import authority of the feature metadata and the feature metadata meets requirements of the metadata table, the first verification module 504 and the second verification module 506 write the feature metadata into the metadata table through the data writing module 508, which greatly improves the efficiency of importing the feature data compared to the existing manner of importing the feature data through manual input.
In specific implementation, the data obtaining module 502 specifically includes: the monitoring module and the data acquisition sub-module. The monitoring module is used for monitoring whether a user drags the feature data in the feature table; and the data acquisition submodule is used for acquiring the feature metadata of the dragged feature data and the metadata table corresponding to the feature metadata when the dragging operation of the user on the feature data in the feature table is monitored.
In specific implementation, the data obtaining sub-module 502 is further configured to: if a new command of a user is received, a metadata table is newly established for the feature metadata; and if an adding instruction of a user is received, searching a metadata table matched with the characteristic metadata, and taking the searched metadata table as the metadata table corresponding to the characteristic metadata.
In specific implementation, the first checking module 504 is further configured to: sending a query request to an authority server; the query request carries an account of the user and an identifier of the metadata table; if receiving prompt information with permission returned by the permission server, determining that the user has the import permission of the characteristic metadata; the prompt information with the authority is generated when the authority server finds that the account of the user corresponds to the metadata table.
In a possible embodiment, the requirement of the metadata table includes: partitioning requirements for the feature table, country code requirements for the feature table with country code, and data uniqueness requirements.
In one possible embodiment, the feature table is a Hive table.
Fig. 6 is a characteristic data management and control apparatus according to an embodiment of the present application, where the apparatus is applied to a characteristic platform server, where a characteristic database is stored in the characteristic platform server, and characteristic metadata in a characteristic metadata table in the characteristic database is imported by using the characteristic data importing apparatus, and the characteristic data management and control apparatus includes: a request receiving module 602, a permission lookup module 604, and an operation response module 606.
The request receiving module 602 is configured to receive a feature data operation request of a user; wherein the characteristic data operation request carries an operation type; the authority searching module 604 is used for searching the management authority of the user; an operation response module 606, configured to respond to the operation corresponding to the operation type if the found management authority includes the operation authority corresponding to the operation type.
The feature management and control device provided by this embodiment realizes a management and control mode when a user extracts feature data, searches for a feature data management authority corresponding to the user for an operation request of the user, and responds to different operations based on different management authorities, so that the security of data use can be improved.
In specific implementation, the operation response module 606 is further configured to: if the searched management authority is the viewing authority and the operation type is the viewing operation, displaying data corresponding to the characteristic data operation request; if the searched management authority is an extraction authority or an approval authority and the operation type is a viewing operation or an extraction operation, providing data corresponding to the characteristic data operation request; and if the searched management authority is the approval authority and the operation type is the approval operation, distributing the viewing authority or the extraction authority of the data corresponding to the characteristic data operation request for the user corresponding to the approval operation.
In a possible implementation manner, the apparatus for managing and controlling feature data further includes: the authority distribution module 608 is configured to distribute management authorities of the feature metadata in the metadata table according to a preset authority dimension; wherein, the preset authority dimension comprises: feature type, user type, region scope.
In specific implementation, the permission assignment module 608 is further configured to: if the preset authority dimension comprises the characteristic type, acquiring a security label corresponding to the characteristic type of the current metadata table; and assigning management authority of the feature metadata in the metadata table according to the security label.
In specific implementation, the permission assignment module 608 is further configured to: and if the preset authority dimension comprises the user type, distributing the management authority of the characteristic metadata in the metadata table according to the management level and/or the value level in the user type.
In specific implementation, the permission assignment module 608 is further configured to: if the preset authority dimension comprises an area range, dividing the feature metadata in the metadata table according to the area range, and distributing management authority to the divided feature metadata.
Further, the present embodiment also provides an electronic device, including: the electronic equipment comprises a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, when the electronic equipment runs, the processor and the storage medium are communicated through the bus, and the processor executes the machine-readable instructions to execute the steps of the feature data importing method or the feature data managing and controlling method.
For ease of understanding, fig. 7 illustrates a schematic diagram of exemplary hardware and software components of an electronic device 700 that may implement the concepts of the present application, according to some embodiments of the present application. For example, the processor 720 may be used on the electronic device 700 and to perform functions in the present application.
The electronic device 700 may be a general-purpose computer or a special-purpose computer, both of which may be used to implement the method of identifying abnormal driving behavior of the present application. Although only a single computer is shown, for convenience, the functions described herein may be implemented in a distributed fashion across multiple similar platforms to balance processing loads.
For example, electronic device 700 may include a network port 710 connected to a network, one or more processors 720 for executing program instructions, a communication bus 730, and a different form of storage medium 740, such as a disk, ROM, or RAM, or any combination thereof. Illustratively, the computer platform may also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The method of the present application may be implemented in accordance with these program instructions. The electronic device 700 also includes an Input/Output (I/O) interface 750 between the computer and other Input/Output devices (e.g., keyboard, display screen).
For ease of illustration, only one processor is depicted in the electronic device 700. However, it should be noted that the electronic device 700 in the present application may also comprise multiple processors, and thus the steps performed by one processor described in the present application may also be performed by multiple processors in combination or individually. For example, if the processor of the electronic device 700 performs step a and step B, it should be understood that step a and step B may also be performed by two different processors together or performed separately in one processor. For example, a first processor performs step a and a second processor performs step B, or the first processor and the second processor perform steps a and B together.
According to another aspect of the present application, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of any of the feature data importing methods or the feature data managing method described above.
The modules may be connected or in communication with each other via a wired or wireless connection. The wired connection may include a metal cable, an optical cable, a hybrid cable, etc., or any combination thereof. The wireless connection may comprise a connection over a LAN, WAN, bluetooth, ZigBee, NFC, or the like, or any combination thereof. Two or more modules may be combined into a single module, and any one module may be divided into two or more units.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to corresponding processes in the method embodiments, and are not described in detail in this application. In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical division, and there may be other divisions in actual implementation, and for example, a plurality of modules or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or modules through some communication interfaces, and may be in an electrical, mechanical or other form.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.
In addition, in the description of the embodiments of the present application, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meaning of the above terms in the present application can be understood in a specific case by those of ordinary skill in the art.
In the description of the present application, it is noted that the terms "first", "second", and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present application, and are used for illustrating the technical solutions of the present application, but not limiting the same, and the scope of the present application is not limited thereto, and although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope disclosed in the present application; such modifications, changes or substitutions do not depart from the spirit and scope of the exemplary embodiments of the present application, and are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (26)

1. The method for importing the feature data is applied to a feature platform server and comprises the following steps:
acquiring feature metadata imported from a feature table to a metadata table by a user, wherein the feature table is a structured table formed by the feature data acquired by the user, and the metadata table is a table corresponding to a template in a feature database;
checking whether the user has the import authority of the feature metadata;
if the user has import authority, checking whether the feature metadata meets the requirement of the metadata table;
and if the characteristic metadata meets the requirement, writing the characteristic metadata into the metadata table.
2. The method of claim 1, wherein the step of obtaining feature metadata imported by a user from a feature table into a metadata table comprises:
monitoring whether a user drags the feature data in the feature table;
and if so, acquiring the feature metadata of the dragged feature data and a metadata table corresponding to the feature metadata.
3. The method according to claim 2, wherein the step of obtaining the metadata table corresponding to the feature metadata comprises:
if a new command of the user is received, a metadata table is newly established for the feature metadata;
and if an adding instruction of the user is received, searching a metadata table matched with the feature metadata, and taking the searched metadata table as the metadata table corresponding to the feature metadata.
4. The method of claim 1, wherein the step of verifying whether the user has the import right of the feature metadata comprises:
sending a query request to an authority server; the query request carries the account of the user and the identifier of the metadata table;
if receiving prompt information with permission returned by the permission server, determining that the user has the import permission of the feature metadata; the permission prompt information is generated when the permission server finds that the account of the user corresponds to the metadata table.
5. The method of claim 1, wherein the requirements of the metadata table include: a partition requirement for the feature table, a country code requirement for the feature table with a country code, and a data uniqueness requirement.
6. The method of claim 1, wherein the feature table is a Hive table.
7. A method for managing and controlling feature data is applied to a feature platform server, the feature platform server stores a feature database, and feature metadata in a feature metadata table in the feature database is imported by the method of any one of claims 1 to 6, and the method for managing and controlling feature data comprises the following steps:
receiving a characteristic data operation request of a user; wherein the characteristic data operation request carries an operation type;
searching the management authority of the user;
and responding to the operation corresponding to the operation type if the searched management authority comprises the operation authority corresponding to the operation type.
8. The method according to claim 7, wherein if the found management authority includes the operation authority corresponding to the operation type, the step of responding to the operation corresponding to the operation type includes:
if the searched management authority is a viewing authority and the operation type is a viewing operation, displaying data corresponding to the characteristic data operation request;
if the searched management authority is an extraction authority or an approval authority and the operation type is a viewing operation or an extraction operation, providing data corresponding to the characteristic data operation request;
and if the searched management authority is an approval authority and the operation type is an approval operation, allocating a viewing authority or an extraction authority of the data corresponding to the characteristic data operation request to the user corresponding to the approval operation.
9. The method of claim 7, further comprising:
distributing the management authority of the feature metadata in the metadata table according to a preset authority dimension; wherein the preset authority dimension comprises: feature type, user type, region scope.
10. The method of claim 9, wherein the step of assigning the management authority of the feature metadata in the metadata table according to a preset authority dimension comprises:
if the preset authority dimension comprises the characteristic type, acquiring a security label corresponding to the characteristic type of the current metadata table;
and distributing the management authority of the feature metadata in the metadata table according to the security label.
11. The method of claim 9, wherein the step of assigning the management authority of the feature metadata in the metadata table according to a preset authority dimension comprises:
and if the preset authority dimension comprises the user type, distributing the management authority of the feature metadata in the metadata table according to the management level and/or the value level in the user type.
12. The method of claim 9, wherein the step of assigning the management authority of the feature metadata in the metadata table according to a preset authority dimension comprises:
if the preset authority dimension comprises an area range, dividing the feature metadata in the metadata table according to the area range, and distributing management authority to the divided feature metadata.
13. An importing apparatus of feature data, the apparatus being applied to a feature platform server, and comprising:
the data acquisition module is used for acquiring feature metadata imported from a feature table to a metadata table by a user, wherein the feature table is a structured table formed by the feature data acquired by the user, and the metadata table is a table corresponding to a template in a feature database;
the first checking module is used for checking whether the user has the import permission of the feature metadata;
the second checking module is used for checking whether the characteristic metadata meets the requirement of the metadata table or not if the user has the import permission;
and the data writing module is used for writing the characteristic metadata into the metadata table if the characteristic metadata meets the requirements.
14. The apparatus of claim 13, wherein the data acquisition module comprises:
the monitoring module is used for monitoring whether a user drags the feature data in the feature table;
and the data acquisition submodule is used for acquiring the feature metadata of the dragged feature data and the metadata table corresponding to the feature metadata when the dragging operation of the user on the feature data in the feature table is monitored.
15. The apparatus of claim 14, wherein the data acquisition sub-module is further configured to:
if a new command of the user is received, a metadata table is newly established for the feature metadata;
and if an adding instruction of the user is received, searching a metadata table matched with the feature metadata, and taking the searched metadata table as the metadata table corresponding to the feature metadata.
16. The apparatus of claim 13, wherein the first checking module is further configured to:
sending a query request to an authority server; the query request carries the account of the user and the identifier of the metadata table;
if receiving prompt information with permission returned by the permission server, determining that the user has the import permission of the feature metadata; the permission prompt information is generated when the permission server finds that the account of the user corresponds to the metadata table.
17. The apparatus of claim 13, wherein the requirements of the metadata table comprise: a partition requirement for the feature table, a country code requirement for the feature table with a country code, and a data uniqueness requirement.
18. The apparatus of claim 13, wherein the feature table is a Hive table.
19. A management and control device for feature data, wherein the device is applied to a feature platform server, the feature platform server stores a feature database, and feature metadata in a feature metadata table in the feature database is imported by the device according to any one of claims 13 to 18, and the management and control device comprises:
the request receiving module is used for receiving a characteristic data operation request of a user; wherein the characteristic data operation request carries an operation type;
the authority searching module is used for searching the management authority of the user;
and the operation response module is used for responding to the operation corresponding to the operation type if the searched management authority contains the operation authority corresponding to the operation type.
20. The apparatus of claim 19, wherein the operation response module is further configured to:
if the searched management authority is a viewing authority and the operation type is a viewing operation, displaying data corresponding to the characteristic data operation request;
if the searched management authority is an extraction authority or an approval authority and the operation type is a viewing operation or an extraction operation, providing data corresponding to the characteristic data operation request;
and if the searched management authority is an approval authority and the operation type is an approval operation, allocating a viewing authority or an extraction authority of the data corresponding to the characteristic data operation request to the user corresponding to the approval operation.
21. The apparatus of claim 19, further comprising:
the authority distribution module is used for distributing the management authority of the feature metadata in the metadata table according to a preset authority dimension; wherein the preset authority dimension comprises: feature type, user type, region scope.
22. The apparatus of claim 21, wherein the permission assignment module is further configured to:
if the preset authority dimension comprises the characteristic type, acquiring a security label corresponding to the characteristic type of the current metadata table;
and distributing the management authority of the feature metadata in the metadata table according to the security label.
23. The apparatus of claim 21, wherein the permission assignment module is further configured to:
and if the preset authority dimension comprises the user type, distributing the management authority of the feature metadata in the metadata table according to the management level and/or the value level in the user type.
24. The apparatus of claim 21, wherein the permission assignment module is further configured to:
if the preset authority dimension comprises an area range, dividing the feature metadata in the metadata table according to the area range, and distributing management authority to the divided feature metadata.
25. An electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, when an electronic device runs, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to execute the steps of the method for importing the feature data according to any one of claims 1 to 6 or the method for managing the feature data according to any one of claims 7 to 12.
26. A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when executed by a processor, the computer program performs the steps of the method for importing feature data according to any one of claims 1 to 6 or the method for managing and managing feature data according to any one of claims 7 to 12.
CN201811478228.6A 2018-12-04 2018-12-04 Feature data importing method, feature data managing and controlling method, feature data importing device, feature data managing and controlling device and electronic equipment Pending CN111274326A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811478228.6A CN111274326A (en) 2018-12-04 2018-12-04 Feature data importing method, feature data managing and controlling method, feature data importing device, feature data managing and controlling device and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811478228.6A CN111274326A (en) 2018-12-04 2018-12-04 Feature data importing method, feature data managing and controlling method, feature data importing device, feature data managing and controlling device and electronic equipment

Publications (1)

Publication Number Publication Date
CN111274326A true CN111274326A (en) 2020-06-12

Family

ID=71001445

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811478228.6A Pending CN111274326A (en) 2018-12-04 2018-12-04 Feature data importing method, feature data managing and controlling method, feature data importing device, feature data managing and controlling device and electronic equipment

Country Status (1)

Country Link
CN (1) CN111274326A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115438024A (en) * 2022-08-23 2022-12-06 光大环境科技(中国)有限公司 Method, device and system for importing data into middle station, electronic equipment and storage medium
CN115510121A (en) * 2022-10-08 2022-12-23 上海数禾信息科技有限公司 Method, device and equipment for managing business form data and readable storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017023388A2 (en) * 2015-07-07 2017-02-09 Private Machines Inc. Secure data management system and method
CN107679422A (en) * 2017-10-25 2018-02-09 厦门市美亚柏科信息股份有限公司 Role-security management method, terminal device and storage medium based on various dimensions
CN108319864A (en) * 2018-01-17 2018-07-24 链家网(北京)科技有限公司 A kind of information inspection control method and device
CN108595597A (en) * 2018-04-19 2018-09-28 平安科技(深圳)有限公司 Data lead-in method, device, computer equipment and storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017023388A2 (en) * 2015-07-07 2017-02-09 Private Machines Inc. Secure data management system and method
CN107679422A (en) * 2017-10-25 2018-02-09 厦门市美亚柏科信息股份有限公司 Role-security management method, terminal device and storage medium based on various dimensions
CN108319864A (en) * 2018-01-17 2018-07-24 链家网(北京)科技有限公司 A kind of information inspection control method and device
CN108595597A (en) * 2018-04-19 2018-09-28 平安科技(深圳)有限公司 Data lead-in method, device, computer equipment and storage medium

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115438024A (en) * 2022-08-23 2022-12-06 光大环境科技(中国)有限公司 Method, device and system for importing data into middle station, electronic equipment and storage medium
CN115510121A (en) * 2022-10-08 2022-12-23 上海数禾信息科技有限公司 Method, device and equipment for managing business form data and readable storage medium
CN115510121B (en) * 2022-10-08 2024-01-05 上海数禾信息科技有限公司 List data management method, device, equipment and readable storage medium

Similar Documents

Publication Publication Date Title
CN107230008B (en) Risk information output and risk information construction method and device
WO2019161778A1 (en) Systems and methods for data storage and querying
CN109614238B (en) Target object identification method, device and system and readable storage medium
CN105095970A (en) Execution method and system of third-party application
CN113821777B (en) Authority control method and device, computer equipment and storage medium
CN105989275B (en) Method and system for certification
CN111382956A (en) Enterprise group relationship mining method and device
CN111400474A (en) Intelligent customer service question and answer method and device for government affair service
CN108681674B (en) Report module creating method and device, computer device and storage medium
CN111966866A (en) Data asset management method and device
DE102012220390A1 (en) Use the geographic location to determine element and territory data for provision to a computing device
CN111274326A (en) Feature data importing method, feature data managing and controlling method, feature data importing device, feature data managing and controlling device and electronic equipment
CN111694615A (en) Data configuration method, device, equipment and storage medium
CN110135815A (en) Travel order monitoring method, device, computer equipment and storage medium
US20230101037A1 (en) Determining optimized parking based on user preferences
CN112037032A (en) Method and device for managing limit based on knowledge graph
WO2019206134A1 (en) Methods and systems for order allocation
CN108663062B (en) Path planning method and system
CN114511235A (en) Process evaluation method and system
US10650337B2 (en) Technical field service inventory management
CN114219023A (en) Data clustering method and device, electronic equipment and readable storage medium
CN111489214A (en) Order allocation method, condition setting method and device and electronic equipment
CN115563117B (en) Multi-tenant management application based on SaaS technology and implementation method thereof
CN111865558A (en) Service data processing method and device, electronic equipment and storage medium
CN115640457A (en) Information management method, apparatus, device, medium, and program product

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination