CN109522319A - Data mapping method and device, electronic equipment, storage medium based on DSL - Google Patents
Data mapping method and device, electronic equipment, storage medium based on DSL Download PDFInfo
- Publication number
- CN109522319A CN109522319A CN201811339286.0A CN201811339286A CN109522319A CN 109522319 A CN109522319 A CN 109522319A CN 201811339286 A CN201811339286 A CN 201811339286A CN 109522319 A CN109522319 A CN 109522319A
- Authority
- CN
- China
- Prior art keywords
- data
- instruction
- dsl
- demapping
- mapping
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
Present disclose provides a kind of data mapping method based on DSL and device, electronic equipment, computer readable storage mediums, belong to technical field of data processing.This method comprises: obtaining one or more raw data tables；DSL is provided and instructs the page, the data demapping instruction for being directed to the raw data table is received by the DSL instruction page；The data demapping instruction is parsed according to structure description rule, and generates the data map statement based on database language；The raw data table is mapped as application data sheet according to the data map statement.The disclosure can reduce the communication cost of data engineering teacher and domain expert, improve the completion quality of data mapping, and can preferably realize application demand.
Description
Technical field
This disclosure relates to technical field of data processing more particularly to a kind of data mapping method and device, electricity based on DSL
Sub- equipment, computer readable storage medium.
Background technique
With the construction of information-based development and public information system, in each professional domain, data processing and management
Business all play an increasingly important role.By taking medicine and hygiene fields as an example, data management and big data technology are in clinic
Data, the research and development of medicine enterprise, gene studies, health control etc. played an important role.One of them element task is will to cure
The original medical data of institute's information system output carry out the processing such as conversion, arrangement and conclusion appropriate, to be mapped as being suitble to respectively answering
With the data mode of scene.
Since the complexity of medical data is higher, highly professional, the medicine that data engineering teacher does not have profession generally is known
Know, it is difficult to the mappings work of complete independently medical data.Existing solution majority is to require data engineering teacher and medicine special
Family's hand-in-glove, to develop corresponding data mapping operating system.However this method be strongly dependent on communication as a result, due to
The completion quality of the various problems being difficult to avoid that in communication, data mapping is usually lower.Same problem is also prevalent in it
In the data mapping method of his professional domain.
It is therefore desirable to propose a kind of new data mapping method.
It should be noted that information is only used for reinforcing the reason to the background of the disclosure disclosed in above-mentioned background technology part
Solution, therefore may include the information not constituted to the prior art known to persons of ordinary skill in the art.
Summary of the invention
Being designed to provide for the disclosure is a kind of based on DSL (Domain Specific Language, the specific language in field
Speech) data mapping method and device, electronic equipment, computer readable storage medium, and then overcome at least to a certain extent
Existing data mapping method is strongly depend on the problem of data engineering teacher and domain expert's communication.
Other characteristics and advantages of the disclosure will be apparent from by the following detailed description, or partially by the disclosure
Practice and acquistion.
According to one aspect of the disclosure, a kind of data mapping method based on DSL is provided, comprising: obtain one or more
A raw data table；DSL is provided and instructs the page, the number for being directed to the raw data table is received by the DSL instruction page
According to demapping instruction；The data demapping instruction is parsed according to structure description rule, and generates the data based on database language and reflects
Penetrate sentence；The raw data table is mapped as application data sheet according to the data map statement.
In a kind of exemplary embodiment of the disclosure, the data demapping instruction includes at least one in following grammer
Kind: Join, for the field in the raw data table to be associated；Filter, for filtering in the raw data table
Data；Mapping, the mapping for specifying the field in the raw data table to the field in the application data sheet are closed
System；Group, for summarizing the field in the application data sheet.
In a kind of exemplary embodiment of the disclosure, the database language includes SparkSQL (Spark structure query
Language)；It is described that the data demapping instruction is parsed according to structure description rule, and generate the data mapping based on database language
Sentence includes: the FROM clause that the instruction in the data demapping instruction comprising Join grammer is converted to SparkSQL；By institute
State the WHERE clause that the instruction in data demapping instruction comprising Filter grammer is converted to SparkSQL；The data are mapped
Instruction in instruction comprising Mapping grammer is converted to the SELECT clause of SparkSQL；It will be wrapped in the data demapping instruction
The instruction of the grammer containing Group is converted to the GROUP BY clause of SparkSQL；It is added on the head of switched clause
The INSERT clause of SparkSQL, and whole clauses are merged into the data map statement.
In a kind of exemplary embodiment of the disclosure, further includes: after providing the DSL instruction page, in the DSL
It instructs and a function edit control is presented in the page；The function edit instruction generation received according to the function edit control is made by oneself
Adopted function (User Defined Function, UDF).
In a kind of exemplary embodiment of the disclosure, the custom function includes that array aggregate function, structural body are poly-
Close function.
In a kind of exemplary embodiment of the disclosure, further includes: the custom function is added to the structure and is retouched
It states in rule.
In a kind of exemplary embodiment of the disclosure, the DSL instruction page is provided, instructs the page to receive by the DSL
The data demapping instruction for being directed to the raw data table includes: the offer DSL instruction page, and in the DSL instruction page
Sample data table and sample demapping instruction are presented in face；The modification operation for being directed to the sample demapping instruction is received, is in simultaneously
The existing modified sample data table；It is grasped according to the sample demapping instruction and the modification for being directed to the sample demapping instruction
Make, generates the data demapping instruction.
According to one aspect of the disclosure, a kind of data mapping unit based on DSL is provided, comprising: data acquisition module,
For obtaining one or more raw data tables；Command reception module is instructed for providing the DSL instruction page by the DSL
The page receives the data demapping instruction for being directed to the raw data table；Statement converter modular, for according to instruction transformation rule
The data demapping instruction is parsed, and generates the data map statement based on database language；Target mapping block is used for basis
The raw data table is mapped as application data sheet by the data map statement.
According to one aspect of the disclosure, a kind of electronic equipment is provided, comprising: processor；And memory, for storing
The executable instruction of the processor；Wherein, the processor is configured to above-mentioned to execute via the executable instruction is executed
Method described in any one.
According to one aspect of the disclosure, a kind of computer readable storage medium is provided, computer program is stored thereon with,
The computer program realizes method described in above-mentioned any one when being executed by processor.
The exemplary embodiment of the disclosure has the advantages that
On the one hand, by the design of DSL, the rule of data structure description is formed, so that domain expert is not having data
In the case where handling professional knowledge, the simple data demapping instruction of grammer can be write, reduces data engineering teacher and domain expert
Communication cost, and reduce communication effectiveness it is bad to data mapping scheme complete quality influence；On the other hand, have it is perfect
Structure description rule DSL on the basis of, the overall process of the present exemplary embodiment, in addition to data demapping instruction needs manual compiling
Outside, other steps can automatically complete, and so as to which more work are concentrated on writing for data demapping instruction, be conducive to
It preferably realizes that data are mapped in the demand on application, while reducing human cost.
It should be understood that above general description and following detailed description be only it is exemplary and explanatory, not
The disclosure can be limited.
Detailed description of the invention
The drawings herein are incorporated into the specification and forms part of this specification, and shows the implementation for meeting the disclosure
Example, and together with specification for explaining the principles of this disclosure.It should be evident that the accompanying drawings in the following description is only the disclosure
Some embodiments for those of ordinary skill in the art without creative efforts, can also basis
These attached drawings obtain other attached drawings.
Fig. 1 schematically shows a kind of system architecture diagram of the data mapping method using the present exemplary embodiment；
Fig. 2 schematically shows a kind of flow chart of data mapping method in the present exemplary embodiment；
Fig. 3 schematically shows a kind of sub-process figure of data mapping method in the present exemplary embodiment；
Fig. 4 schematically shows a kind of structural block diagram of data mapping unit in the present exemplary embodiment；
Fig. 5 schematically shows a kind of electronic equipment for realizing the above method in the present exemplary embodiment；
Fig. 6 schematically shows a kind of computer-readable storage medium for realizing the above method in the present exemplary embodiment
Matter.
Specific embodiment
Example embodiment is described more fully with reference to the drawings.However, example embodiment can be with a variety of shapes
Formula is implemented, and is not understood as limited to example set forth herein；On the contrary, thesing embodiments are provided so that the disclosure will more
Fully and completely, and by the design of example embodiment comprehensively it is communicated to those skilled in the art.Described feature, knot
Structure or characteristic can be incorporated in any suitable manner in one or more embodiments.
The exemplary embodiment of the disclosure provides firstly a kind of data mapping method based on DSL.Data mapping refers to
From fixed format and in numerous and jumbled initial data, the processing such as data extraction, conversion, cleaning, merging is carried out according to demand, to obtain
Desired application data.The present embodiment can be applied to the data mapping of medical field, also can be applied to other field, such as
The mapping of electric power monitoring data, climate monitoring data, raw financial data etc..
Fig. 1 shows a kind of system architecture that can apply the present embodiment.The system 100 includes application layer 101, mapping clothes
Business device 102, internal database 103 and external data base 104.Wherein, application layer 101 can be the terminal or service of application side
Device can be interacted by specific client software or browser with mapping server 102；Mapping server 102 can be with
The data mapping method for executing the present embodiment, obtains initial data from internal database 103, is mapped as using data and stores
Into internal database 103, when application layer 101 is initiated to request, mapping server 102 can be transferred from internal data 103
Application layer 101 is arrived through completing the application data of mapping and feeding back；Internal database 103 can store initial data and application respectively
Data, initial data is from external data base 104；External data base 104 can be the HIS of hospital or medical institutions
(Hospital InformationSytem, hospital information system) database, is also possible to the specialized database of other field.
Since expert data is usually directed to huge data volume, and versions of data is more, and internal database 103 can use HDFS (base
In the distributed file system of Hadoop) or MongoDB distributed framework.
System 100 can use the body frame structure of B/S (Browser/Server, browser/server), such as can use
A kind of Jquery (simple JavaScript frame)+Php (common Web development language)+NginX (Reverse Proxy)+
The framework of MongoDB (database schema) a kind of.System 100 can also use C/S (Client/Server, client/service
Device) framework, or the Web development frame based on J2EE (Java platform for being suitable for enterprise host grade calculating type) etc., this reality
It applies example and this is not specially limited.
Based on above description, the data mapping method of the present embodiment can be applied particularly to mapping server 102.
It should be appreciated that the number of each section is merely exemplary in Fig. 1, according to actual needs, application layer 101 may include
Any number of application terminal or server, mapping server 102 are also possible to the server of any number of server composition
Cluster, internal database 103 or external data base 104 also may include any number of database node.
It 2 is described further with reference to the accompanying drawing, refering to what is shown in Fig. 2, the data mapping method of the present embodiment may include
Following steps:
Step S21 obtains one or more raw data tables.
It is illustrated by taking medical data as an example.Have stronger relational between medical data, therefore the database of HIS is big
It is mostly relevant database, initial data therein can be stored as the form of table, i.e. raw data table.Raw data table can
It can be one with the raw data table in the HIS of one or more hospitals, each HIS, be also possible to multiple and more
A raw data table can also have certain association or nest relation, and the present embodiment is not specially limited this.
It, can also be one or more original from the acquisition of one or more data sources using similar method for other field
Expert data table.
Step S22 provides the DSL instruction page, and the data for being directed to the raw data table are received by the DSL instruction page
Demapping instruction.
Wherein, DSL instructs the page can be by DSL for developer's input data demapping instruction, data demapping instruction
It writes.DSL is that have the computer design language of corresponding ability to express for a certain specific area, compared to programming language or
Data base query language etc., DSL typically are provided with simpler structure and grammer, in form closer to pseudocode or natural language
Speech.In the present embodiment, DSL is the data structure description language designed for specific area, can be grasped simultaneously by domain expert
It uses, therefore the demand that domain expert can map according to data, corresponding data mapping is write in the DSL instruction page and is referred to
It enables.
Step S23 parses data demapping instruction according to structure description rule, and generates the data based on database language and reflect
Penetrate sentence.
The syntax rule of DSL is usually write in DSL design process, in the present embodiment, since DSL is used for expert data
Structure mapping operation is carried out, the syntax rule of DSL can be structure description rule, i.e. grammer in DSL is mainly structure description
Grammer, with structure operation logic.The type of structure operation is less, and logical description is simpler, therefore the number based on DSL
It is relatively easy to parse according to demapping instruction, such as the grammer of DSL can be carried out with the grammer of database language corresponding, data are mapped
The sentence of instruction is converted to the data map statement in database language with same syntax of figs function, can also be mapped according to data
The syntactic structure of instruction, parsing obtains abstract syntax tree, then abstract syntax tree is converted to corresponding data map statement etc..
In one exemplary embodiment, database language may include SparkSQL, HiveQL (Hive query language) and
At least one of ImpalaSQL (Impala Structured Query Language) is related to the type of internal database or framework.
Raw data table is mapped as application data sheet according to data map statement by step S24.
Can be with installation data library manager on mapping server, the data map statement based on database language can be by
Data base administrator is identified and is executed, so as to carry out corresponding structure behaviour to raw data table according to data demapping instruction
Make, be applied tables of data.
In the above-mentioned methods, on the one hand, by the design of DSL, the rule of data structure description is formed, so that domain expert
In the case where not having data processing arts knowledge, the simple data demapping instruction of grammer can be write, reduces data engineering
The communication cost of teacher and domain expert, and reduce the bad influence that quality is completed to data mapping scheme of communication effectiveness；Another party
Face, on the basis of having the DSL of perfect structure description rule, the overall process of the present exemplary embodiment, in addition to data mapping refers to
Order needs outside manual compiling, other steps can automatically complete, and refers to so as to which more work are concentrated on data mapping
What is enabled writes, and is conducive to preferably realize that data are mapped in the demand on application, while reducing human cost.
In one exemplary embodiment, data demapping instruction may include at least one of following grammer: Join,
Filter, Mapping, Group.DSL can use JSON (JavaScript object numbered musical notation) format, and the structure of each grammer describes
Rule is as follows:
Join, for the field in raw data table to be associated；It can specify table alias, the associated field of table, table
Associated mode, such as left connection, right connection.Syntax example is as follows:
The example indicates first to carry out the field Field_B of the field Field_A of table Table_A and table Table_B right
Then the field Field_C of middle table and Table_C after connection is carried out left connection by connection.
Filter, for screening the data in raw data table；Raw data table can be done before association basic clear
Work is washed, unwanted data are filtered, syntax example is as follows:
The example indicates that field Field_A is not empty in screening table Table_A, and the field Field_B value of table Table_B
For 1 data.
Mapping, for specifying the mapping relations of field of the field in raw data table into application data sheet, grammer
Example is as follows:
Table structure after mapping can also be propped up with the mixed structure of body support structure, array and structural body and array
Hold multi-layer mapping etc..
Group, for summarizing the field in application data sheet, syntax example is as follows:
By Join, Filter, Mapping, the grammer of Group can carry out most structure operation, and lead to
Above example is crossed it is found that the form and structure description rule of the included grammer of data demapping instruction are all relatively simple, is easy to field
The understanding and use of expert.
Based on above-mentioned grammer and its structure description rule, in one exemplary embodiment, database language may include
SparkSQL, step S23 may comprise steps of:
It will include that the instruction of Join grammer is converted to the FROM clause of SparkSQL in data demapping instruction；
It will include that the instruction of Filter grammer is converted to the WHERE clause of SparkSQL in data demapping instruction；
It will include that the instruction of Mapping grammer is converted to the SELECT clause of SparkSQL in data demapping instruction；
It will include that the instruction of Group grammer is converted to the GROUP BY clause of SparkSQL in data demapping instruction；
In the INSERT clause of the head of switched clause addition SparkSQL, and whole clauses are merged into data and are reflected
Penetrate sentence.
Wherein, the instruction in data demapping instruction comprising Join grammer is parsed, the association of raw data table can be indicated
For a directed acyclic graph, extreme saturation then is carried out to directed acyclic graph, the field between connected applications tables of data, which is associated with, closes
System generates the FROM clause in Spark SQL.
Parse include in data demapping instruction Filter grammer instruction, according to the data screening condition of definition and can patrol
It collects relationship (relationships such as "AND", "or" between each screening conditions), generates the WHERE clause in SparkSQL.
Instruction comprising Mapping grammer in parsing data demapping instruction, the basic structure of available application data sheet,
And the field of the corresponding raw data table of each field and raw data table, generate the SELECT clause in Spark SQL.
The instruction in data demapping instruction comprising Group grammer is parsed, the field of available application data sheet, which polymerize, closes
System, generates the GROUP BY clause of SparkSQL, and rewrite SELECT clause.
The clause that above step generation can finally be summarized adds INSERT clause on head, and generation can completely transport
Data map statement of the row in Spark.
The design of DSL inevitably has certain limitation, complicated, the personalized demand in face of application layer, in fact it could happen that language
Situations such as method is insufficient, so that it cannot write out suitable data demapping instruction.In consideration of it, in one exemplary embodiment, number
According to mapping method can with the following steps are included:
After the DSL instruction page is provided, a function edit control is presented in the DSL instruction page；
Custom function is generated according to the function edit instruction that function edit control receives.
Wherein, custom function can be the grammer of new structure operation, be also possible to the rule that data are calculated or converted,
The present embodiment is not specially limited this.
In one exemplary embodiment, complicated structural body, array or structural body and the mixed structure of array are difficult to pass through
Group grammer direct polymerization, therefore the interior polymeric of field can be realized by writing specific custom function, accordingly
, custom function may include array aggregate function, structural body aggregate function.
It should be noted that the function edit instruction in function edit control needs the syntax rule according to database language
It writes, major part database language all supports custom function at present.
After editing custom function, custom function can also be added in structure description rule, make customized letter
Number becomes the normative grammars of DSL, with realize the update of DSL with it is perfect, write need to meet subsequent more, more complicated instruction
It asks.
In one exemplary embodiment, data demapping instruction is write in order to further facilitate domain expert, with reference to Fig. 3 institute
Show, step S22 can be realized by following steps:
Step S31 provides the DSL instruction page, and presentation sample data table and sample mapping refer in the DSL instruction page
It enables；
Step S32 receives the modification operation for being directed to sample demapping instruction, while modified sample data table is presented；
Step S33 is operated according to sample demapping instruction and the modification for being directed to sample demapping instruction, is generated data mapping and is referred to
It enables.
Wherein, sample demapping instruction can be considered that the model of a data demapping instruction, sample data table are corresponding application
Tables of data.Based on sample demapping instruction, domain expert is not necessarily to write data demapping instruction from zero, modifies in sample demapping instruction
Table name, field name etc., it can obtain data demapping instruction, while can also be referring to modified sample data table, with detection
Whether the content of modification is suitable.The present embodiment can further decrease the learning cost of domain expert, and data demapping instruction
It writes that mode is simpler, friendly, is conducive to the popularization and application of the present embodiment.
The exemplary embodiment of the disclosure additionally provides a kind of data mapping unit based on DSL, refering to what is shown in Fig. 4, should
Device 40 may include: data acquisition module 41, for obtaining one or more raw data tables；Command reception module 42 is used
The page is instructed in providing DSL, the data demapping instruction for being directed to raw data table is received by the DSL instruction page；Sentence conversion
Module 43 for parsing data demapping instruction according to instruction transformation rule, and generates the data based on database language and maps language
Sentence；Target mapping block 44, for raw data table to be mapped as application data sheet according to data map statement.
In one exemplary embodiment, data demapping instruction may include at least one of following grammer: Join is used for
Field in raw data table is associated；Filter, for filtering the data in raw data table；Mapping, for referring to
Determine the mapping relations of field of the field in raw data table into application data sheet；Group, for will be in application data sheet
Field is summarized.
In one exemplary embodiment, database language can be SparkSQL；Statement converter modular can be used for count
According to including that the instruction of Join grammer is converted to the FROM clause of SparkSQL in demapping instruction, will include in data demapping instruction
The instruction of Filter grammer is converted to the WHERE clause of SparkSQL, will include the finger of Mapping grammer in data demapping instruction
The SELECT clause for being converted to SparkSQL is enabled, the instruction in data demapping instruction comprising Group grammer is converted to
The GROUP BY clause of SparkSQL, and in the INSERT clause of the head of switched clause addition SparkSQL, and will
Whole clauses merge into data map statement.
In one exemplary embodiment, command reception module can also include: function edit cell, for providing DSL
After instructing the page, a function edit control is presented in the DSL instruction page, and compile according to the function that function edit control receives
It collects instruction and generates custom function.
In one exemplary embodiment, above-mentioned custom function may include array aggregate function, structural body aggregate function.
In one exemplary embodiment, function edit cell can be also used for for custom function being added to structure description rule
In then.
In one exemplary embodiment, command reception module may include: sample display unit, for providing DSL instruction
The page, and sample data table and sample demapping instruction is presented in the DSL instruction page；Receiving unit is modified, is directed to for receiving
It is operated in the modification of sample demapping instruction, while modified sample data table is presented；Instruction generation unit, for according to sample
Demapping instruction and the modification operation for being directed to sample demapping instruction, generate data demapping instruction.
The detail of above each module/unit has been described in detail in the embodiment of method part, therefore no longer superfluous
It states.
The exemplary embodiment of the disclosure additionally provides a kind of electronic equipment that can be realized the above method.
Person of ordinary skill in the field it is understood that various aspects of the disclosure can be implemented as system, method or
Program product.Therefore, various aspects of the disclosure can be with specific implementation is as follows, it may be assumed that complete hardware embodiment, complete
The embodiment combined in terms of full Software Implementation (including firmware, microcode etc.) or hardware and software, can unite here
Referred to as circuit, " module " or " system ".
The electronic equipment 500 of this exemplary embodiment according to the disclosure is described referring to Fig. 5.What Fig. 5 was shown
Electronic equipment 500 is only an example, should not function to the embodiment of the present disclosure and use scope bring any restrictions.
As shown in figure 5, electronic equipment 500 is showed in the form of universal computing device.The component of electronic equipment 500 can wrap
It includes but is not limited to: at least one above-mentioned processing unit 510, at least one above-mentioned storage unit 520, the different system components of connection
The bus 530 of (including storage unit 520 and processing unit 510), display unit 540.
Wherein, storage unit is stored with program code, and program code can be executed with unit 510 processed, so that processing is single
Member 510 executes the step described in above-mentioned " illustrative methods " part of this specification according to the various illustrative embodiments of the disclosure
Suddenly.For example, processing unit 510 can execute step S21~S24 shown in Fig. 2, can also execute step S31 shown in Fig. 3~
S33 etc..
Storage unit 520 may include the readable medium of volatile memory cell form, such as Random Access Storage Unit
(RAM) 521 and/or cache memory unit 522, it can further include read-only memory unit (ROM) 523.
Storage unit 520 can also include program/utility 524 with one group of (at least one) program module 525,
Such program module 525 includes but is not limited to: operating system, one or more application program, other program modules and
It may include the realization of network environment in program data, each of these examples or certain combination.
Bus 530 can be to indicate one of a few class bus structures or a variety of, including storage unit bus or storage
Cell controller, peripheral bus, graphics acceleration port, processing unit use any bus structures in a variety of bus structures
Local bus.
Electronic equipment 500 can also be with one or more external equipments 700 (such as keyboard, sensing equipment, bluetooth equipment
Deng) communication, can also be enabled a user to one or more equipment interact with the electronic equipment 500 communicate, and/or with make
Any equipment (such as the router, modulation /demodulation that the electronic equipment 500 can be communicated with one or more of the other calculating equipment
Device etc.) communication.This communication can be carried out by input/output (I/O) interface 550.Also, electronic equipment 500 can be with
By network adapter 560 and one or more network (such as local area network (LAN), wide area network (WAN) and/or public network,
Such as internet) communication.As shown, network adapter 560 is communicated by bus 530 with other modules of electronic equipment 500.
It should be understood that although not shown in the drawings, other hardware and/or software module can not used in conjunction with electronic equipment 500, including but not
Be limited to: microcode, device driver, redundant processing unit, external disk drive array, RAID system, tape drive and
Data backup storage system etc..
Through the above description of the embodiments, those skilled in the art is it can be readily appreciated that example described herein is implemented
Mode can also be realized by software realization in such a way that software is in conjunction with necessary hardware.Therefore, according to the disclosure
The technical solution of embodiment can be embodied in the form of software products, which can store non-volatile at one
Property storage medium (can be CD-ROM, USB flash disk, mobile hard disk etc.) in or network on, including some instructions are so that a calculating
Equipment (can be personal computer, server, terminal installation or network equipment etc.) is executed according to the exemplary implementation of the disclosure
The method of example.
The exemplary embodiment of the disclosure additionally provides a kind of computer readable storage medium, and being stored thereon with can be realized
The program product of this specification above method.In some possible embodiments, various aspects of the disclosure can also be realized
For a kind of form of program product comprising program code, when program product is run on the terminal device, program code is used for
Execute terminal device described in above-mentioned " illustrative methods " part of this specification according to the various exemplary embodiment party of the disclosure
The step of formula.
It is produced refering to what is shown in Fig. 6, describing the program according to the exemplary embodiment of the disclosure for realizing the above method
Product 600, can be using portable compact disc read only memory (CD-ROM) and including program code, and can set in terminal
It is standby, such as run on PC.However, the program product of the disclosure is without being limited thereto, in this document, readable storage medium storing program for executing can
With to be any include or the tangible medium of storage program, the program can be commanded execution system, device or device use or
Person is in connection.
Program product can be using any combination of one or more readable mediums.Readable medium can be readable signal Jie
Matter or readable storage medium storing program for executing.Readable storage medium storing program for executing for example can be but be not limited to electricity, magnetic, optical, electromagnetic, infrared ray or partly lead
System, device or the device of body, or any above combination.More specific example (the non exhaustive column of readable storage medium storing program for executing
Table) it include: the electrical connection with one or more conducting wires, portable disc, hard disk, random access memory (RAM), read-only storage
Device (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc read only memory (CD-
ROM), light storage device, magnetic memory device or above-mentioned any appropriate combination.
Computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal,
In carry readable program code.The data-signal of this propagation can take various forms, including but not limited to electromagnetic signal,
Optical signal or above-mentioned any appropriate combination.Readable signal medium can also be any readable Jie other than readable storage medium storing program for executing
Matter, the readable medium can send, propagate or transmit for by instruction execution system, device or device use or and its
The program of combined use.
The program code for including on readable medium can transmit with any suitable medium, including but not limited to wirelessly, have
Line, optical cable, RF etc. or above-mentioned any appropriate combination.
Can with any combination of one or more programming languages come write for execute the disclosure operation program
Code, programming language include object oriented program language-Java, C++ etc., further include conventional process
Formula programming language-such as " C " language or similar programming language.Program code can be calculated fully in user
It executes in equipment, partly execute on a user device, executing, as an independent software package partially in user calculating equipment
Upper part executes on a remote computing or executes in remote computing device or server completely.It is being related to remotely counting
In the situation for calculating equipment, remote computing device can pass through the network of any kind, including local area network (LAN) or wide area network
(WAN), it is connected to user calculating equipment, or, it may be connected to external computing device (such as utilize ISP
To be connected by internet).
In addition, above-mentioned attached drawing is only the schematic theory of the processing according to included by the method for disclosure exemplary embodiment
It is bright, rather than limit purpose.It can be readily appreciated that the time that above-mentioned processing shown in the drawings did not indicated or limited these processing is suitable
Sequence.In addition, be also easy to understand, these processing, which can be, for example either synchronously or asynchronously to be executed in multiple modules.
It should be noted that although being referred to several modules or list for acting the equipment executed in the above detailed description
Member, but this division is not enforceable.In fact, according to an exemplary embodiment of the present disclosure, above-described two or
More multimode or the feature and function of unit can embody in a module or unit.Conversely, above-described one
A module or the feature and function of unit can be to be embodied by multiple modules or unit with further division.
Those skilled in the art after considering the specification and implementing the invention disclosed here, will readily occur to its of the disclosure
His embodiment.This application is intended to cover any variations, uses, or adaptations of the disclosure, these modifications, purposes or
Adaptive change follow the general principles of this disclosure and including the undocumented common knowledge in the art of the disclosure or
Conventional techniques.The description and examples are only to be considered as illustrative, and the true scope and spirit of the disclosure are by claim
It points out.
It should be understood that the present disclosure is not limited to the precise structures that have been described above and shown in the drawings, and
And various modifications and changes may be made without departing from the scope thereof.The scope of the present disclosure is only limited by the attached claims.
Claims (10)
1. a kind of data mapping method based on DSL characterized by comprising
Obtain one or more raw data tables；
DSL is provided and instructs the page, is referred to by the data mapping that the DSL instructs page reception to be directed to the raw data table
It enables；
The data demapping instruction is parsed according to structure description rule, and generates the data map statement based on database language；
The raw data table is mapped as application data sheet according to the data map statement.
2. the method according to claim 1, wherein the data demapping instruction include in following grammer at least
It is a kind of:
Join, for the field in the raw data table to be associated；
Filter, for filtering the data in the raw data table；
Mapping, for specifying the field in the raw data table to the mapping relations of the field in the application data sheet；
Group, for summarizing the field in the application data sheet.
3. according to the method described in claim 2, it is characterized in that, the database language includes SparkSQL；The basis
Structure description rule parses the data demapping instruction, and generates the data map statement based on database language and include:
It will include that the instruction of Join grammer is converted to the FROM clause of SparkSQL in the data demapping instruction；
It will include that the instruction of Filter grammer is converted to the WHERE clause of SparkSQL in the data demapping instruction；
It will include that the instruction of Mapping grammer is converted to the SELECT clause of SparkSQL in the data demapping instruction；
It will include that the instruction of Group grammer is converted to the GROUP BY clause of SparkSQL in the data demapping instruction；
In the INSERT clause of the head of switched clause addition SparkSQL, and whole clauses are merged into the data and are reflected
Penetrate sentence.
4. the method according to claim 1, wherein further include:
After the DSL instruction page is provided, a function edit control is presented in the DSL instruction page；
Custom function is generated according to the function edit instruction that the function edit control receives.
5. according to the method described in claim 4, it is characterized in that, the custom function includes array aggregate function, structure
Body aggregate function.
6. according to the method described in claim 4, it is characterized by further comprising:
The custom function is added in the structure description rule.
7. instructing the page by the DSL the method according to claim 1, wherein providing DSL instructs the page
It receives and is directed to the data demapping instruction of the raw data table and includes:
The DSL instruction page is provided, and sample data table and sample demapping instruction is presented in the DSL instruction page；
The modification operation for being directed to the sample demapping instruction is received, while the modified sample data table is presented；
It is operated according to the sample demapping instruction and the modification for being directed to the sample demapping instruction, generates the data mapping and refer to
It enables.
8. a kind of data mapping unit based on DSL characterized by comprising
Data acquisition module, for obtaining one or more raw data tables；
Command reception module instructs page reception to be directed to the original number for providing the DSL instruction page by the DSL
According to the data demapping instruction of table；
Statement converter modular for parsing the data demapping instruction according to instruction transformation rule, and generates and is based on database language
The data map statement of speech；
Target mapping block, for the raw data table to be mapped as application data sheet according to the data map statement.
9. a kind of electronic equipment characterized by comprising
Processor；And
Memory, for storing the executable instruction of the processor；
Wherein, the processor is configured to require 1-7 described in any item via executing the executable instruction and carry out perform claim
Method.
10. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program
Claim 1-7 described in any item methods are realized when being executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811339286.0A CN109522319A (en) | 2018-11-12 | 2018-11-12 | Data mapping method and device, electronic equipment, storage medium based on DSL |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811339286.0A CN109522319A (en) | 2018-11-12 | 2018-11-12 | Data mapping method and device, electronic equipment, storage medium based on DSL |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109522319A true CN109522319A (en) | 2019-03-26 |
Family
ID=65773903
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811339286.0A Pending CN109522319A (en) | 2018-11-12 | 2018-11-12 | Data mapping method and device, electronic equipment, storage medium based on DSL |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109522319A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110175031A (en) * | 2019-04-25 | 2019-08-27 | 平安科技（深圳）有限公司 | Sql like language changes into the method, apparatus, computer equipment and storage medium of DSL language |
| CN110427352A (en) * | 2019-06-05 | 2019-11-08 | 福建奇点时空数字科技有限公司 | A kind of data self mapping method based on R2RML standard |
| CN112015831A (en) * | 2020-09-09 | 2020-12-01 | 深圳市绿联科技有限公司 | Method, device and equipment for operating relational database based on C language |
| CN113742407A (en) * | 2021-08-31 | 2021-12-03 | 浙江惠瀜网络科技有限公司 | Data conversion method and device |
| CN117075967A (en) * | 2023-10-16 | 2023-11-17 | 西安羚控电子科技有限公司 | Ground station automation instruction generation method, data rendering method and system |
| US11886396B2 (en) | 2021-11-13 | 2024-01-30 | Tata Consultancy Services Limited | System and method for learning-based synthesis of data transformation rules |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101105814A (en) * | 2007-09-11 | 2008-01-16 | 金蝶软件（中国）有限公司 | Method and device for converting Script language to SQL language |
| US20080307430A1 (en) * | 2007-06-08 | 2008-12-11 | Friedlander Robert R | System and method for a multiple disciplinary normalization of source for metadata integration with etl processing layer of complex data across multiple claim engine sources in support of the creation of universal/enterprise healthcare claims record |
| CN102841889A (en) * | 2011-06-20 | 2012-12-26 | 中兴通讯股份有限公司 | Method and device for achieving efficient database access based on ORM (object relational mapping) architecture |
| CN106951305A (en) * | 2017-03-24 | 2017-07-14 | 南京大学 | It is a kind of based on method of the model conversion by QVT R language generation midCore scripts |
| CN106970802A (en) * | 2017-04-25 | 2017-07-21 | 北京航天飞行控制中心 | The method and device of integrated programming script in the language-specific of field |
| CN107391529A (en) * | 2017-03-28 | 2017-11-24 | 阿里巴巴集团控股有限公司 | A kind of method and device for realizing Object Relation Mapping ORM |
-
2018
- 2018-11-12 CN CN201811339286.0A patent/CN109522319A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080307430A1 (en) * | 2007-06-08 | 2008-12-11 | Friedlander Robert R | System and method for a multiple disciplinary normalization of source for metadata integration with etl processing layer of complex data across multiple claim engine sources in support of the creation of universal/enterprise healthcare claims record |
| CN101105814A (en) * | 2007-09-11 | 2008-01-16 | 金蝶软件（中国）有限公司 | Method and device for converting Script language to SQL language |
| CN102841889A (en) * | 2011-06-20 | 2012-12-26 | 中兴通讯股份有限公司 | Method and device for achieving efficient database access based on ORM (object relational mapping) architecture |
| CN106951305A (en) * | 2017-03-24 | 2017-07-14 | 南京大学 | It is a kind of based on method of the model conversion by QVT R language generation midCore scripts |
| CN107391529A (en) * | 2017-03-28 | 2017-11-24 | 阿里巴巴集团控股有限公司 | A kind of method and device for realizing Object Relation Mapping ORM |
| CN106970802A (en) * | 2017-04-25 | 2017-07-21 | 北京航天飞行控制中心 | The method and device of integrated programming script in the language-specific of field |
Non-Patent Citations (1)
| Title |
|---|
| 李金龙: ""面向数据集成的非文本DSL与SQL互译系统的设计与实现"", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110175031A (en) * | 2019-04-25 | 2019-08-27 | 平安科技（深圳）有限公司 | Sql like language changes into the method, apparatus, computer equipment and storage medium of DSL language |
| CN110175031B (en) * | 2019-04-25 | 2023-08-18 | 平安科技（深圳）有限公司 | Method, device, computer equipment and storage medium for converting SQL language into DSL language |
| CN110427352A (en) * | 2019-06-05 | 2019-11-08 | 福建奇点时空数字科技有限公司 | A kind of data self mapping method based on R2RML standard |
| CN112015831A (en) * | 2020-09-09 | 2020-12-01 | 深圳市绿联科技有限公司 | Method, device and equipment for operating relational database based on C language |
| CN113742407A (en) * | 2021-08-31 | 2021-12-03 | 浙江惠瀜网络科技有限公司 | Data conversion method and device |
| CN113742407B (en) * | 2021-08-31 | 2024-04-30 | 浙江惠瀜网络科技有限公司 | Data conversion method and device |
| US11886396B2 (en) | 2021-11-13 | 2024-01-30 | Tata Consultancy Services Limited | System and method for learning-based synthesis of data transformation rules |
| CN117075967A (en) * | 2023-10-16 | 2023-11-17 | 西安羚控电子科技有限公司 | Ground station automation instruction generation method, data rendering method and system |
| CN117075967B (en) * | 2023-10-16 | 2024-01-30 | 西安羚控电子科技有限公司 | Ground station automation instruction generation method, data rendering method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109522319A (en) | Data mapping method and device, electronic equipment, storage medium based on DSL | |
| US11100154B2 (en) | Data integration tool | |
| US8473521B2 (en) | Ontology guided reference data discovery | |
| Brown et al. | The Architecture of Open Source Applications: Elegance, Evolution, and a Few Fearless Hacks | |
| CN104145257B (en) | For the lsa cache cloud service of connected equipment | |
| US8751558B2 (en) | Mashup infrastructure with learning mechanism | |
| CN106201642B (en) | A kind of template automation compiling loading method and system based on NODEJS | |
| CN102609255B (en) | A kind of generation method of Spring IOC configuration file | |
| CN105308565B (en) | The parallel compilation of source code of depth | |
| JP7350923B2 (en) | Deep learning framework operator registration method, equipment, device and storage medium | |
| US10803083B2 (en) | System and method of generating platform-agnostic abstract syntax tree | |
| US9110659B2 (en) | Policy to source code conversion | |
| CN105493074B (en) | Method for generating the help of Dynamic Customization context-sensitive | |
| CN109948151A (en) | The method for constructing voice assistant | |
| WO2019232980A1 (en) | Node configuration method and apparatus, computer readable storage medium, and electronic device | |
| US20120166977A1 (en) | User programming access to data model in user interface design | |
| WO2021070012A1 (en) | Ontology-based query routing for distributed knowledge bases | |
| CN112711581A (en) | Medical data verification method and device, electronic equipment and storage medium | |
| CN108388423A (en) | A kind of IOS data objects conversion method and device | |
| WO2021070013A1 (en) | Ontology-based data storage for distributed knowledge bases | |
| CN108388623A (en) | ER relationships generation method, device, computer equipment and storage medium | |
| US8538976B2 (en) | Abstract data model extension through managed physical statement structures | |
| CN102520963A (en) | Method and system for realizing service logic processing based on strongly-typed relation mapping | |
| US10505873B2 (en) | Streamlining end-to-end flow of business-to-business integration processes | |
| Barrasa et al. | Building Knowledge Graphs |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190326 |
|
| RJ01 | Rejection of invention patent application after publication |