CN105843899B - A kind of big data automation analytic method for simplifying programming and system - Google Patents

A kind of big data automation analytic method for simplifying programming and system Download PDF

Info

Publication number
CN105843899B
CN105843899B CN201610167763.4A CN201610167763A CN105843899B CN 105843899 B CN105843899 B CN 105843899B CN 201610167763 A CN201610167763 A CN 201610167763A CN 105843899 B CN105843899 B CN 105843899B
Authority
CN
China
Prior art keywords
hive
input file
data
java
anonymous class
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201610167763.4A
Other languages
Chinese (zh)
Other versions
CN105843899A (en
Inventor
尤海浪
唐勇
陈杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xuan Cai Interactive Network Science And Technology Ltd
Original Assignee
Xuan Cai Interactive Network Science And Technology Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xuan Cai Interactive Network Science And Technology Ltd filed Critical Xuan Cai Interactive Network Science And Technology Ltd
Priority to CN201610167763.4A priority Critical patent/CN105843899B/en
Publication of CN105843899A publication Critical patent/CN105843899A/en
Application granted granted Critical
Publication of CN105843899B publication Critical patent/CN105843899B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25Integrating or interfacing systems involving database management systems
    • G06F16/258Data format conversion from or to a database
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/283Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Machine Translation (AREA)

Abstract

The present invention proposes a kind of big data automation analytic method for simplifying programming and system.Cumbersome parsing is needed for the programming of existing big data, causing to there are problems that data parsing is error-prone causes dirty data and parses, propose by way of automating parsing formatting and nonformatted data, the anonymous class objects of Java, and record Hive tables and the corresponding relation of anonymous class field in systems will be converted into Hive tables per data.Anonymous class is matched according to the name variable and Hive literary name name sections of IField member variable, and corresponding literary name segment value is directly assigned for the member variable matched.While big data processing holding data flexibility is carried out, the speed and accuracy of data parsing are improved.

Description

A kind of big data automation analytic method for simplifying programming and system
Technical field
Field, more particularly to a kind of big data process object neutralizing analysis based on Hadoop are parsed the present invention relates to data Method and system.
Background technology
Hadoop is a distributed system architecture developed by Apache funds club, and user can not know about In the case of distributed low-level details, distributed program is developed.The power of cluster is made full use of to carry out high-speed computation and storage.
HDFS(Hadoop Distributed File System)Number is stored for providing file system for Hadoop According to, the characteristics of HDFS has high fault tolerance, and be designed to be deployed on cheap hardware;And it provides high-throughput (high throughput)Carry out the data of access application, being adapted to those has super large data set(large data set) Application program.HDFS is relaxed(relax)POSIX requirement, can be accessed in the form of streaming(streaming access) Data in file system.
MapReduce is a kind of programming model, for large-scale dataset(More than 1TB)Concurrent operation.Concept " Map (Mapping)" and " Reduce(Reduction)".It is very easy to programming personnel will not distributed parallel programming in the case of, will The program of oneself is operated in distributed system.MapReduce related algorithms are had been realized in Hadoop system, are implemented Positioned at org.apache.hadoop.mapred.
Hive is a Tool for Data Warehouse based on Hadoop, the data file of structuring can be mapped as into a number According to storehouse table, and simple sql query functions are provided, sql sentences can be converted to MapReduce tasks and run.Its is excellent Point is that learning cost is low, simple MapReduce statistics can be quickly realized by class SQL statement, it is not necessary to develop specially MapReduce is applied, and is very suitable for the statistical analysis of data warehouse.
Framework is handled as a general big data, Hadoop can easily handle the structuring text more than 1T very much Notebook data, and in traditional server log, most of data(Access time, reference address, user id etc.)Will loading Stored into Hive in the way of table.
The research that inventor passes through to existing MapReduce calculations finds that existing MapReduce processing modes are deposited In problems with:
1) data of simple text structure are dealt with and unfriendly for developer, such as:Type, form Deng conversion.
2) file data parsing work is coupled with business logic processing, causes program complexity to ramp, and increase There is the probability of mistake.
3) change of data structure directly results in program and largely changed.
Numerous and diverse Data Matching and parsing undoubtedly improves the cost of learning curve and exploitation.
The content of the invention
In order to solve the above technical problems, this application provides a kind of automation that Hive table data are parsed into data object Data in Hive tables are parsed into Java anonymous class by analytic method and system, with more preferable processing business logic.The application's Technical scheme is specific as follows:
A kind of big data automation analytic method for simplifying programming, it is characterised in that this method comprises the following steps:
Step 1:The input file of MapReduce tasks is predefined;And register anonymous class in Map classes;Wherein, The input file includes Hive list files;
Step 2:In the Map stages, judge to detect whether input file is hidden with predefined Hive table objects and registration Name class object matches;If so, then performing step 3;Otherwise, terminate;
Step 3:Input file is read, the Hive table objects matched according to input file are parsed to input file;
Step 4:Input file after being parsed according to Hive table objects is mapped to the member that@IField are marked in anonymous class Variable.
Further, in step 1:
The input file to MapReduce tasks carries out predefined include:
The Hive table names and Hive list files of the needs processing of this subtask are preset before the submission of MapReduce tasks Location;
The anonymous class of being registered in Map classes includes:Anonymous class is mapped to containing the@IField member variables marked The corresponding field of Hive table objects;
Wherein, the corresponding word of title and the Hive table objects containing the@IField member variables marked of the anonymous class The title of section is identical.
Further, in step 2:
In the Map stages, input file and file address are carried out with predefined Hive table names and Hive list files address Matching;
Meanwhile, by input file and file address and the member's change marked containing IField of the anonymous class object of registration The title of amount is matched with the field name of the Hive tables of input file.
Further, in step 3:
The parsing includes:
The data structure for the Hive table objects that the text data of input file is matched according to the input file first is entered Row parses and generates Hive table objects;
Again by the member variable of the Hive table objects of generation and the IField marks of anonymous class object by Data Matching and Binding, is converted to the data type consistent with the type of anonymous class object.
Further, in step 4:
Each Hive literary name sections member's change marked by@IField corresponding with anonymous class that input file is parsed Amount is compared, and will compare successful Hive literary names segment value and carries out data type conversion according to the type of member variable and be assigned to The member variable.
Further, before step 1, it is further comprising the steps of:
Hive list files are loaded, and define the text data format of input file;
Hive table objects and anonymous class object are generated according to the Hive list files of loading, the anonymous class object is used to map The attribute information of the data structure of the Hive table objects, the attribute information includes the title, type, Yi Jimo of each field Recognize value.)
A kind of big data automation resolution system for simplifying programming, it is characterised in that the system includes:
Default unit, is predefined for the input file to MapReduce tasks;And registered in Map classes anonymous Class;Wherein, the input file includes Hive list files;
Matching unit, in the Map stages, judge detection input file whether with predefined Hive table objects and note The anonymous class match objects of volume;If so, then being parsed to input file;Otherwise, terminate;
Resolution unit, for reading input file, the Hive table objects matched according to input file enter to input file Row parsing;
Map unit, is marked for the input file after being parsed according to Hive table objects to be mapped into@IField in anonymous class The member variable of note.
Further, the default unit also includes:
Predefined module, for presetting the Hive tables that this subtask needs to handle before the submission of MapReduce tasks Name and Hive list files address;
Anonymous class Registering modules, for anonymous class to be mapped into Hive tables pair containing the@IField member variables marked The corresponding field of elephant;
Wherein, the corresponding word of title and the Hive table objects containing the@IField member variables marked of the anonymous class The title of section is identical.
Further, the matching unit also includes:
Hive table matching modules, in the Map stages, by input file and file address and predefined Hive table names and Hive list files address is matched;
Anonymous class matching module, for containing input file and file address and the anonymous class object registered The title of the member variable of IField marks is matched with the field name of the Hive tables of input file.
Further, the resolution unit also includes:
Hive table parsing modules, for the Hive tables pair for being matched the text data of input file according to the input file The data structure of elephant is parsed and generates Hive table objects;
Anonymous class parsing module, for the member for marking the Hive table objects of generation with the IField of anonymous class object Variable is converted to the data type consistent with the type of anonymous class object by Data Matching and binding.
Further, the map unit also includes:
Comparing module, each Hive literary name sections for input file to be parsed are corresponding with anonymous class by@ The member variable of IField marks is compared, and will compare successful Hive literary names segment value according to the type of member variable and enters line number Changed according to type and be assigned to the member variable.
From the above-mentioned technical proposal of the application, the text data object neutralizing analysis method and system that the application is provided, Anonymous class in the Hive tables of pre-defined MapReduce inputs and list file address, registration Map classes, anonymous class@IField Match somebody with somebody the information such as Property Name, type, the default value of each field, and progress is parsed with reflecting automatically when reading input data Penetrate, the application has directly bound the Java anonymous objects of data using these, solves existing MapReduce processing modes Present in problem.
Brief description of the drawings
, below will be to embodiment or existing in order to illustrate more clearly of the embodiment of the present application or technical scheme of the prior art The accompanying drawing to be used needed for having technology description is briefly described, it should be apparent that, drawings discussed below is only this Shen Please described in some embodiments, for those of ordinary skill in the art, on the premise of not paying creative work, also Other accompanying drawings can be obtained according to these accompanying drawings.
Fig. 1 is the flow chart that analytic method is automated according to a kind of big data for simplifying programming of the embodiment of the present application;
Fig. 2 is the detail flowchart that analytic method is automated according to the big data of the embodiment of the present application;
Fig. 3 is the Hive tables and anonymous class corresponding relation exemplary plot according to the embodiment of the present application;
Fig. 4 is the system architecture that resolution system is automated according to a kind of big data for simplifying programming of the embodiment of the present application Figure.
Embodiment
Specific embodiments of the present invention described further below:
A kind of big data automation analytic method for simplifying programming, before the analytic method is performed, to MapReduce Input data source Hive tables to be predefined, the predefined process is:This is first set before the submission of MapReduce tasks Subtask needs Hive tables to be processed and the corresponding data address of table, and anonymous class is registered in Map classes(The anonymous class contains@ The member variable that IField is explained will actively be mapped to Hive table corresponding fields);After above-mentioned predefined process terminates, system MapReduce tasks and relevant parameter will be submitted automatically, and perform the automatic analytic method automatically during Map, including:
Input data and its file address according to the predefined Hive tables and table corresponding data address to the Map stages Matched, while the matching anonymous class object consistent with this definition data structure;
The data of input file are read, and this article notebook data is parsed automatically according to the Hive tables matched;
The each literary name section parsed is compared with the member variables identified of@IField in anonymous class, compared into Literary name segment value after work(will carry out data type conversion according to the type of member variable and be assigned to the member variable;
Above-mentioned method, it is preferred that Hive table names and table data file are preset before MapReduce task starts Location, the setting also implies the data form of the definition to text data format, the Hive table names and table data file simultaneously It is completely one-to-one.
Above-mentioned method, it is preferred that include during being parsed to described table data:By the input data solution Analyse into table data, then identification field explained with anonymous class IField and match and data binding, finally will convert into it is described The consistent data type of anonymous Class Type.
A kind of big data automation resolution system for simplifying programming, including:Default unit, matching unit, resolution unit And map unit.
Wherein:The default unit is used to predefine the relevant information in MapReduce input datas source, to anonymity The member variable of class is set;The predefined process is:Hive table names and list file address are set in task in advance, Anonymous class is bound in Map classes.
The file address that the matching unit is used for the foundation input of Map stages matches Hive table objects, Ran Hou first Matched with anonymous class object, only three all correspond on, can just perform following resolving.
The resolution unit is used for the defined input data for reading the default unit, and according to your matching unit The table data association message mixed is parsed to input data;
The map unit is used for the table object information matched according to the matching unit and anonymous category information, will IField explain member variable be compared with table object field, when comparing successfully according to the types of variables translation resolution after Literary name segment value simultaneously be assigned to the variable;
Above-mentioned system, it is preferred that the matching unit is noted according in the default default Hive tables of unit and Map classes The matching of the anonymous class of volume, the anonymous class is used for the attribute information for mapping the list data structure.
Above-mentioned system, it is preferred that the process that the resolution unit is parsed to the input data includes:Will be described The text data of input file is parsed and generates table object automatically according to Hive tables structure.
Above-mentioned system, it is preferred that the map unit is mapped described table object and anonymous class object one by one, The value of table object is automatically mapped to the member variable and translation type of anonymous class object in the way of IField.
The embodiment of the present application provides a kind of big data automation analytic method for simplifying programming and system, by Hive tables In the anonymous class objects of Java, and record Hive tables and the corresponding relation of anonymous class field in systems are converted into per data.Hide Name class is matched according to the name variable and Hive literary name name sections of IField member variable, for the member matched Variable directly assigns corresponding literary name segment value.The embodiment of the present application directly sets up corresponding Java type simultaneously using these data Getter, setter method of defined attribute, so just can directly quote the anonymous object class in follow-up processing routine Member variable value.
Above is the core concept of the application, in order that those skilled in the art more fully understand application scheme.Under The accompanying drawing that face will be combined in the embodiment of the present application, the technical scheme in the embodiment of the present application is clearly and completely described, and shows So, described embodiment is only some embodiments of the present application, rather than whole embodiments.Based on the reality in the application Example is applied, the every other embodiment that those of ordinary skill in the art are obtained under the premise of creative work is not made all should When the scope for belonging to the application protection.
A kind of big data for simplifying programming automates analytic method before this method is performed disclosed in the embodiment of the present application, Input data to MapReduce is predefined, after predefined process terminates, and the Map processes of MapReduce programs will be automatic Perform analytic method.
A kind of big data for simplifying programming automates flow chart such as Fig. 1 institutes of analytic method disclosed in the embodiment of the present application Show, including:
Step S101:The input file of MapReduce tasks is carried out according to described Hive tables and table data address pre- If, while registering anonymous class in Map classes.
Wherein, the member variable and table object field name that@IField are identified on the anonymous class are consistent.
Step S102:Read above-mentioned input data source, and match Hive table objects and anonymous class object, and according to Hive table objects are parsed to above-mentioned input data.
Step S103:Data after above-mentioned table object is parsed are mapped to the member variable that@IField are identified in anonymous class.
Need to illustrate above step is:
The member variable of anonymous class@IField marks must be included in the literary name section name in Hive tables, if do not wrapped It is contained in Hive literary name sections name, the member variable will be sky.
The data read in from Hive tables must be on same HDFS with MapReduce programs.
Unified automatic mapping class is defined, all anonymous class are required for inheriting such, and such realizes general data Operation, such as obtain the value of specific field, obtain corresponding table object etc..
A kind of detail flowchart for the big data automation analytic method for simplifying programming disclosed in the embodiment of the present application is such as Shown in Fig. 2, including:
Step S201:Automatic analytic method starts;
Step S202:MapReduce loads Hive table data;
Step S203:In Map procedure initialization methods, the anonymous class object of generation Hive table objects and registration;
Step S204:Detection judges whether current input file can match Hive tables and anonymous class;If so, then performing step Rapid S205;Otherwise, call and step S208 is performed after the map methods that Map carries;
Step S205:Detection judges currently whether there is input data;If so, then performing step S206;Otherwise, step is performed S208;
Step S206:Parse input data;
Step S207:Hive tables data are mapped to anonymous class, anonymous class process methods are then called, and return to S205;
Step S208:Automatic analytic method terminates
Hive tables disclosed in the embodiment of the present application are with anonymous class corresponding relation exemplary plot as shown in figure 3, corresponding Hive tables table Entitled dim_game, field includes:Unique number of playing gid, game name gname, tri- fields of type of play gametype. The anonymous class of corresponding conversion such as Fig. 3 upper rights, sample data is shown in below Fig. 3.
A kind of system construction drawing for the big data automation resolution system for simplifying programming disclosed in the embodiment of the present application is such as Shown in Fig. 4, including default cell S 301, matching unit S302, resolution unit S303 and map unit S304;
Wherein:Default cell S 301 is used to predefine MapReduce input data, that is, specifies and load those Hive table data, while registering anonymous class in Map classes.
Matching unit S302 is used for the anonymous internal class object pair according to the default default Hive tables of cell S 301 and registration The input file address in Map stages is matched.
Resolution unit S303 is used to read the input data that the default units of S301 are defined, and text data format is carried out certainly Dynamic parsing.
Map unit S304 is used for the automatic assignment for the member variable that anonymous class@IField are identified.

Claims (8)

1. a kind of big data automation analytic method for simplifying programming, it is characterised in that this method comprises the following steps:
Step 1:The input file of MapReduce tasks is predefined;And Java anonymous class is registered in Map classes;Wherein, The input file includes Hive list files;
Step 2:In the Map stages, judge to detect whether input file is hidden with the Java of predefined Hive table objects and registration Name class object matches;If so, then performing step 3;Otherwise, terminate;
Step 3:Input file is read, the Hive table objects matched according to input file are parsed to input file;
Step 4:Input file after being parsed according to Hive table objects is mapped to the member that@IField are marked in Java anonymous class Variable.
2. according to the method described in claim 1, it is characterised in that in step 1:
The input file to MapReduce tasks carries out predefined include:
Hive table names and Hive list files address that this subtask needs to handle are preset before the submission of MapReduce tasks;
The Java anonymous class of being registered in Map classes includes:Java anonymous class is reflected containing the@IField member variables marked It is mapped to the corresponding field of Hive table objects;
Wherein, the corresponding word of title and the Hive table objects containing the@IField member variables marked of the Java anonymous class The title of section is identical.
3. method according to claim 2, it is characterised in that in step 3:
The parsing includes:
The data structure for the Hive table objects that the text data of input file is matched according to the input file first is solved Analyse and generate Hive table objects;
Again by the member variable of the IField marks of the anonymous class object of the Hive table objects of generation and Java by Data Matching and Binding, is converted to the data type consistent with the type of the anonymous class objects of Java.
4. method according to claim 3, it is characterised in that in step 4:
Each Hive literary name sections member's change marked by@IField corresponding with Java anonymous class that input file is parsed Amount is compared, and will compare successful Hive literary names segment value and carries out data type conversion according to the type of member variable and be assigned to The member variable.
5. a kind of big data automation resolution system for simplifying programming, it is characterised in that the system includes:
Default unit, is predefined for the input file to MapReduce tasks;And it is anonymous that Java is registered in Map classes Class;Wherein, the input file includes Hive list files;
Matching unit, in the Map stages, judge to detect input file whether with predefined Hive table objects and registration Java anonymous class match objects;If so, then being parsed to input file;Otherwise, terminate;
Resolution unit, for reading input file, the Hive table objects matched according to input file are solved to input file Analysis;Map unit, is marked for the input file after being parsed according to Hive table objects to be mapped into@IField in Java anonymous class Member variable.
6. system according to claim 5, it is characterised in that the default unit also includes:
Predefined module, the Hive table names handled for presetting this subtask to need before the submission of MapReduce tasks and Hive list files address;
Java anonymous class Registering modules, for Java anonymous class to be mapped into Hive containing the@IField member variables marked The corresponding field of table object;
Wherein, the corresponding word of title and the Hive table objects containing the@IField member variables marked of the Java anonymous class The title of section is identical.
7. system according to claim 6, it is characterised in that the resolution unit also includes:
Hive table parsing modules, for the Hive table objects that are matched the text data of input file according to the input file Data structure is parsed and generates Hive table objects;
Java anonymous class parsing modules, for mark the IField of the Hive table objects of generation and the anonymous class objects of Java Member variable is converted to the data type consistent with the type of the anonymous class objects of Java by Data Matching and binding.
8. system according to claim 7, it is characterised in that the map unit also includes:
Comparing module, each Hive literary name sections for input file to be parsed are corresponding with Java anonymous class by@ The member variable of IField marks is compared, and will compare successful Hive literary names segment value according to the type of member variable and enters line number Changed according to type and be assigned to the member variable.
CN201610167763.4A 2016-03-23 2016-03-23 A kind of big data automation analytic method for simplifying programming and system Active CN105843899B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610167763.4A CN105843899B (en) 2016-03-23 2016-03-23 A kind of big data automation analytic method for simplifying programming and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610167763.4A CN105843899B (en) 2016-03-23 2016-03-23 A kind of big data automation analytic method for simplifying programming and system

Publications (2)

Publication Number Publication Date
CN105843899A CN105843899A (en) 2016-08-10
CN105843899B true CN105843899B (en) 2017-07-18

Family

ID=56583064

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610167763.4A Active CN105843899B (en) 2016-03-23 2016-03-23 A kind of big data automation analytic method for simplifying programming and system

Country Status (1)

Country Link
CN (1) CN105843899B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107766381A (en) * 2016-08-22 2018-03-06 北京京东尚科信息技术有限公司 Data query method, system and electronic equipment
CN106569868B (en) * 2016-11-09 2019-12-31 腾讯音乐娱乐(深圳)有限公司 Gradle-based compiling optimization method and device
CN107562562B (en) * 2017-09-08 2021-06-29 郑州云海信息技术有限公司 Method and system for setting extra information describing fault event
CN107704625B (en) * 2017-10-30 2021-01-15 锐捷网络股份有限公司 Method and device for field matching
CN109376154B (en) * 2018-10-26 2020-11-10 杭州玳数科技有限公司 Data reading and writing method and data reading and writing system

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102880503B (en) * 2012-08-24 2015-04-15 新浪网技术(中国)有限公司 Data analysis system and data analysis method
US9031925B2 (en) * 2012-12-13 2015-05-12 Sap Se Hadoop access via hadoop interface services based on function conversion
CN103942099B (en) * 2014-04-30 2017-11-28 广州唯品会网络技术有限公司 Executing tasks parallelly method and device based on Hive
CN104992379A (en) * 2015-07-10 2015-10-21 贵州广思信息网络有限公司 Dynamic text exchange format management method of big data of power industry

Also Published As

Publication number Publication date
CN105843899A (en) 2016-08-10

Similar Documents

Publication Publication Date Title
CN105843899B (en) A kind of big data automation analytic method for simplifying programming and system
CN109558525B (en) Test data set generation method, device, equipment and storage medium
CN106970929B (en) Data import method and device
CN104572122A (en) Software application data generating device and method
EP2626831A1 (en) Tag Inheritance
CN110019111B (en) Data processing method, data processing device, storage medium and processor
US20140201214A1 (en) Creating a file descriptor independent of an open operation
US20150199408A1 (en) Systems and methods for a high speed query infrastructure
CN111966760B (en) Test data generation method and device based on Hive data warehouse
CN112732567A (en) Mock data testing method and device based on ip, electronic equipment and storage medium
CN113177090A (en) Data processing method and device
CN103647775A (en) Method and apparatus for registration in webpage
CN110888972A (en) Sensitive content identification method and device based on Spark Streaming
CN109189688B (en) Test case script generation method and device and electronic equipment
CN111767533A (en) Offline mode user authorization method, device, electronic device and storage medium
CN111125216B (en) Method and device for importing data into Phoenix
CN117033249A (en) Test case generation method and device, computer equipment and storage medium
US10339151B2 (en) Creating federated data source connectors
CN114968725A (en) Task dependency relationship correction method and device, computer equipment and storage medium
CN115114297A (en) Data lightweight storage and search method and device, electronic equipment and storage medium
CN114547206A (en) Data synchronization method and data synchronization system
CN114003661A (en) Offline data entry method and device, electronic equipment and storage medium
CN113111065A (en) Database creation method and device, computer equipment and storage medium
CN111078905A (en) Data processing method, device, medium and equipment
CN116755684B (en) OAS Schema generation method, device, equipment and medium

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant