CN115994142B - Data development method based on data center - Google Patents
Data development method based on data center Download PDFInfo
- Publication number
- CN115994142B CN115994142B CN202211604285.0A CN202211604285A CN115994142B CN 115994142 B CN115994142 B CN 115994142B CN 202211604285 A CN202211604285 A CN 202211604285A CN 115994142 B CN115994142 B CN 115994142B
- Authority
- CN
- China
- Prior art keywords
- data
- development
- field
- fields
- sentences
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 25
- 230000002194 synthesizing effect Effects 0.000 claims abstract description 23
- 230000015572 biosynthetic process Effects 0.000 claims abstract description 4
- 238000003786 synthesis reaction Methods 0.000 claims abstract description 4
- 230000000875 corresponding effect Effects 0.000 claims description 18
- 230000002596 correlated effect Effects 0.000 claims description 15
- 238000007405 data analysis Methods 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000004148 unit process Methods 0.000 abstract 1
- 230000001360 synchronised effect Effects 0.000 description 5
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 238000007792 addition Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data development method based on a data center, in particular to the technical field of computers, wherein the data generated in the business handling process of an enterprise project is collected and stored in a database through the data center, and a judgment and synthesis unit processes data processing sentences submitted by all developers at the same moment, so that the condition that a plurality of developers modify different field data of the same row in the data table at the same moment due to row-level locks of the data table can only be executed, and the development efficiency is low is avoided; the judging and synthesizing unit performs identical matching on fields contained in the data processing statement at the same moment, deletes the fields meeting the conditions in the original data processing statement, and splices the subsequent data processing statement to generate a new data processing statement, so that the situation that the fields with strong relevance are repeatedly modified by multiple persons to cause data disorder is avoided, and the safety of data is ensured.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a data development method based on a data center.
Background
The data center is used for collecting, storing and processing a large amount of data through a data technology, and unifying standards and calibers, and is used for enterprises, the enterprises collect the data generated in the enterprise project business handling process through the data center, generate data tables of different projects according to the project business types, and manage the data tables according to project type sub-libraries, so that developers of the enterprises can develop the data conveniently;
the existing data development method based on the data center is based on a development platform, an enterprise internal developer operates data in a data table in a data base based on the development platform inputting sql statement, however, if a plurality of developers operate the same data in the data table at the same time, the line-level lock mechanism of the data table only allows one line of data to operate at the same time, other sql statements operating on the data are blocked, and as projects are larger, the service scope is wider and wider, the relation between different services is finer and finer, and the different developers need to modify a certain data in one line of data in the same data table in the process of developing the data, however, the line-level lock of the data table seriously affects the efficiency of the developer on the data development, so that the whole project development efficiency is reduced;
in order to solve the above problems, the present invention proposes a solution.
Disclosure of Invention
The invention aims to provide a data development method based on a data center, and aims to solve the problems that in the prior art, different developers need to modify a certain data in one row of data in the same data table in the process of developing the data because of row-level locks in the data table, but the overall project development efficiency is reduced because the row-level locks in the data table seriously influence the development efficiency of the developers on the data.
The aim of the invention can be achieved by the following technical scheme:
a data development method based on a data center includes the following steps:
step one: the data center station collects data generated in the business handling process of the enterprise project and generates a data table of different businesses of the project according to the business type of the project, wherein business data of the enterprise comprise order data, client data, commodity data and supply chain data;
step two: the data analysis module analyzes the data tables of all project services of the enterprise to generate field association information data of all the data tables, wherein the field association information data of one data table comprises an association information list of all the fields in the data table;
step three: the development end comprises a storage module, an authentication module, a data processing module and a plurality of development modules, wherein the authentication module authenticates the identity of a developer in the enterprise to generate a pre-selection instruction and transmits the pre-selection instruction to the development modules;
step four: the development module comprises a preloading unit and a development unit, wherein the development module receives a preselected instruction transmitted by the authentication module, connects databases required by current developers for data development, acquires field associated information data of all data tables of the databases, and loads the field associated information data into a memory;
the development unit acquires a data development statement of the developer according to a certain acquisition mode and transmits the data development statement to the data processing module;
step five: the data processing module receives the data development sentences transmitted by the development units and then operates the data in the database, the data processing module comprises a judging and synthesizing unit and an operating unit, if the data processing module receives only one data development sentence transmitted by the development unit at the same time, the data processing module transmits the data development sentences transmitted by the development unit to the operating unit, and the operating unit receives the data development sentences transmitted by the data processing module and executes the data development sentences;
if the data processing module receives the data development sentences transmitted by the development units at the same moment, the data processing module transmits the data development sentences to the judging and synthesizing unit, and the judging and synthesizing unit judges the data development sentences transmitted by the data processing module according to a certain judging step after receiving the data development sentences to generate homologous data of all relevant tables at the moment.
Further, the data development statement of the developer is obtained by two ways: the developer obtains the data development statement of the developer in two ways: one is that the development unit acquires a data development statement which is newly input by a developer until the current moment after the developer inputs an active submission instruction, and the other is that the development unit detects that the developer is inputting; and acquiring a data development statement newly input by a developer at the current moment when no input is performed in W later time, wherein W is a preset time threshold.
Further, the data development statement specifically refers to SQL statements, and one data development statement represents one SQL statement.
Further, the operation unit includes an sql actuator.
Further, the specific determination step of the determination synthesis unit determining to generate all the table homology data related to the moment is as follows:
s11: taking a data development statement at the moment as an example, sequentially acquiring a table name, a field name and a field value contained in the data development statement and generating comparison information data of the data development statement according to the table name, the field name and the field value;
s12: according to S11, obtaining the comparison information data of all the data development sentences at the moment and synchronously comparing the comparison information data:
if the table names and the field names are consistent in the comparison information data of all the data development sentences at the moment, the comparison information data are marked as homologous information data, the data tables with the same names in the comparison information data are marked as related tables, and the fields with the same names are marked as related fields;
s13: taking a related table as an example, acquiring all data development sentences with table names being the related table and field names being consistent, wherein the data development sentences are marked as R1, R2, rr and R is more than or equal to 1;
acquiring field association information data of the correlation table;
s14: taking the data development statement R1 as an example, if the sql statement contains other fields except relevant fields, marking the other fields except relevant fields contained in the sql statement as fields to be relevant;
otherwise, do not do any treatment;
s15: according to S14, all fields to be correlated in the data development sentences R1, R2, R, and R are sequentially acquired, and homologous data of the correlation table are generated according to the fields to be correlated and the correlation fields of the correlation table;
s16: according to S13 to S15, the homologous data of all the correlation tables at that time are calculated and acquired.
Furthermore, the judging and synthesizing unit splices the data development sentences according to the homologous data of all tables in the data development sentences at the moment, and the specific steps are as follows:
s21: taking homologous data of a correlation table as an example, acquiring all relevant fields U1, U2, U.S., U is more than or equal to 1, and fields V1, V2, vv, V is more than or equal to 1 in the data of the correlation table;
s22: acquiring relevant fields U1, U2, and fields in a Uu associated information list in relevant field associated information data of the relevant table, carrying out homologous matching on the relevant fields with fields V1, V2, and Vv to be relevant, and acquiring all non-modifiable fields:
s221: taking the field V1 to be correlated as an example, if the field V1 to be correlated is the same as the fields in the U1, U2, the U.S. and Uu associated information list, the field V1 to be correlated is recalibrated to be an unmodified field, otherwise, no processing is performed;
s222: sequentially carrying out identical matching on the fields V1, V2, V, U1, U2, U and Uu associated information list according to S221 to obtain all non-modifiable fields;
the judging and synthesizing unit performs non-modifiable field retrieval on all the data development sentences containing the related table at the moment, performs non-modifiable field retrieval on the data development sentences with the table names of the related table in all the data development sentences at the moment, deletes all languages operating on the non-modifiable field, and returns that the current moment of the field is non-modifiable;
the judging and synthesizing unit splices the deleted data development sentences to generate new data development sentences and transmits the new data development sentences to the operation unit, and the operation unit executes the data development sentences transmitted by the judging and synthesizing unit after receiving the data development sentences.
Further, the specific steps of the data analysis unit for analyzing and generating all table field associated information data are as follows:
s31: firstly, selecting one database in enterprise data development as a to-be-developed library, selecting one data table contained in the to-be-developed library as a to-be-developed table, and acquiring all fields in the to-be-developed table, wherein the marks are A1, A2, aa and a is more than or equal to 1;
s32: dividing development sections, dividing a development period into B development sections with equal time length, and marking the B development sections of the development period as B1, B2, bb and more than or equal to 1;
s33: acquiring all data development sentences which are marked as C1, C2, cc and C is more than or equal to 1 and are input by a developer for development of a to-be-developed table in a development period development section B1;
s34: according to a certain calculation step, the cascade duty ratio H1 of the field A1 and the field A2 in the to-be-published list in b development sections of one development period is calculated and obtained;
s35: generating an association list J1 of a field A1 in a to-be-developed table according to a certain generation step;
s36: according to S35, obtaining a determined final association list, where the final association list includes association list J1 of field A1, association list J2 of field A2, association list Ja of field Aa;
s37: calculating and obtaining a union interval [ Hmax, phi 1] of a to-be-published table;
s371: acquiring the maximum value Hmax and the minimum value Hmin of the homonymy duty ratios corresponding to all the association fields in the association lists J1, J2;
s372: calculating and obtaining a judgment limit value phi of the co-linked duty ratio by using a formula phi=hmax- λ1× (Hmax-Hmin), wherein λ1 is a preset duty ratio coefficient;
s38: generating all associated information lists N1, N2, and the third and the fourth lists N of the to-be-published tables according to a certain generation step k/2 ；
S39: acquiring all data tables contained in a to-be-developed library, marking the data tables as P1, P2, and P, wherein P is more than or equal to 1, sequentially selecting the data tables as P1, P2, P as to-be-developed tables according to S31 to S38, and generating field related information data of all the data tables in the to-be-developed library;
s310: according to S31 to S39, a database of all the items of the enterprise is obtained, and field association information data of all the data tables in the corresponding database is obtained.
Further, the storage module comprises a database storage unit and an information storage unit, wherein the information storage unit stores field association information data of all data tables in all databases of the enterprise.
The invention has the beneficial effects that:
(1) The invention collects the data generated in the business handling process of the enterprise project through the data center table and stores the data according to different sub-libraries of the project, and sets the judging and synthesizing unit to process the data processing statement submitted by all developers at the same moment, so that on one hand, a plurality of developers are prevented from modifying different field data of the same row in the data table at the same moment, only one developer's data processing statement can be executed at the moment due to the row-level lock of the data table, and the data processing statements of other developers wait for response, thereby causing the occurrence of low development efficiency;
(2) According to the invention, all the enterprise data tables are analyzed through the data analysis module, field related information data of all the enterprise data tables are generated, the judgment and synthesis unit performs identical matching on fields contained in data processing sentences submitted at the same time, the fields meeting the conditions are deleted in the original data processing sentences, the subsequent data processing sentences are spliced to generate new data processing sentences, the occurrence of data disorder caused by repeated modification of fields with strong relevance by multiple persons is avoided, and the safety of data is ensured.
Drawings
The invention is further described below with reference to the accompanying drawings.
Fig. 1 is a system block diagram of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
As shown in fig. 1, a data development method based on a data center is performed by a data development system based on the data center, wherein the system comprises the data center, a development end and a data analysis module;
the data center is used for collecting data generated in the business handling process of the enterprise project, generating a data table of different businesses of the project according to the business type of the project and transmitting the data table to the development terminal; in this embodiment, the business data of the enterprise includes order data, customer data, commodity data, and supply chain data;
the development end comprises a storage module, an authentication module, a data processing module and a plurality of development modules, wherein the development end receives data tables of all projects of different services transmitted by a data center station and then transmits the data tables to the storage module, and the storage module comprises a database storage unit and an information storage unit;
the storage module receives the data tables of all the different projects of the enterprise transmitted by the storage module and then transmits the data tables of all the different projects, and the database storage unit correspondingly establishes a database of the project according to the types of the project after receiving the data tables of all the different projects of the enterprise transmitted by the storage module, wherein the data tables of all the projects are stored in the database of one project, and in the embodiment, the database refers to a mysql database;
the authentication module is used for carrying out development authority authentication on developers in the enterprise, and after the development authority authentication is passed, the authentication module generates a pre-selection instruction and transmits the pre-selection instruction to the development module;
the development module is used for carrying out data development on developers in an enterprise and comprises a preloading unit and a development unit, and one development module is used for carrying out data development on one developer in the enterprise;
the development module receives the pre-selection instruction transmitted by the authentication module and then transmits the pre-selection instruction to the pre-loading unit, the pre-loading unit establishes connection with a database required by the current developer for data development after receiving the pre-selection instruction transmitted by the development module and acquires field related information data of all data tables of the database, and the pre-loading unit loads the field related information data of all data tables in the database to a memory and then generates an instruction to be developed and transmits the instruction to the development unit;
the development unit acquires the data development statement of the developer according to a specific acquisition mode after receiving the instruction to be developed transmitted by the preloading unit and transmits the data development statement to the data processing module, wherein the specific acquisition mode is two, one is that the development unit acquires the data development statement which is newly input by the developer until the current moment after the developer inputs the active submitting instruction, and the other is that the development unit detects that the developer is inputting'; acquiring a data development statement newly input by a developer at the current moment when no input is performed in the post W time; in this embodiment, the data development statement specifically refers to a set of SQL statements input by a developer, and in this embodiment, the end of the set of SQL statements is "used; dividing, wherein W is a preset time threshold;
the data processing module comprises a judging and synthesizing unit and an operating unit, the data processing module is used for operating data in a database according to data development sentences of a plurality of development units, if the data processing module receives only one data development sentence transmitted by one development unit at the same time, the data processing module transmits the data development sentence transmitted by the development unit to the operating unit, the operating unit comprises an sql executor, and the sql executor executes the data development sentence transmitted by the data processing module after the operating unit receives the data development sentence;
if the data processing module receives the data development sentences transmitted by the development units at the same moment, the data processing module transmits the data development sentences to the judging and synthesizing unit, and the judging and synthesizing unit judges the data development sentences transmitted by the data processing module at the same moment according to a certain judging step after receiving the data development sentences transmitted by the data processing module, wherein the specific judging step is as follows:
s11: taking a data development statement at the moment as an example, sequentially acquiring a table name, a field name and a field value contained in the data development statement and generating comparison information data of the data development statement according to the table name, the field name and the field value;
s12: according to S11, obtaining the comparison information data of all the data development sentences at the moment and synchronously comparing the comparison information data:
if the table names and the field names are consistent in the comparison information data of all the data development sentences at the moment, the comparison information data are marked as homologous information data, the data tables with the same names in the comparison information data are marked as related tables, and the fields with the same names are marked as related fields;
s13: taking a related table as an example, acquiring all data development sentences with table names being the related table and field names being consistent, wherein the data development sentences are marked as R1, R2, rr and R is more than or equal to 1;
acquiring field association information data of the correlation table;
s14: taking the data development statement R1 as an example, if the sql statement contains other fields except relevant fields, marking the other fields except relevant fields contained in the sql statement as fields to be relevant;
otherwise, do not do any treatment;
s15: according to S14, all fields to be correlated in the data development sentences R1, R2, R, and R are sequentially acquired, and homologous data of the correlation table are generated according to the fields to be correlated and the correlation fields of the correlation table;
s16: according to S13 to S15, calculating and acquiring homologous data of all relevant tables at the moment;
the judging and synthesizing unit splices the data development sentences according to the homologous data of all tables in the data development sentences at the moment, and the specific steps are as follows:
s21: taking homologous data of a correlation table as an example, acquiring all relevant fields U1, U2, U.S., U is more than or equal to 1, and fields V1, V2, vv, V is more than or equal to 1 in the data of the correlation table;
s22: acquiring relevant fields U1, U2, and fields in a Uu associated information list in relevant field associated information data of the relevant table, carrying out homologous matching on the relevant fields with fields V1, V2, and Vv to be relevant, and acquiring all non-modifiable fields:
s221: taking the field V1 to be correlated as an example, if the field V1 to be correlated is the same as the fields in the U1, U2, the U.S. and Uu associated information list, the field V1 to be correlated is recalibrated to be an unmodified field, otherwise, no processing is performed;
s222: sequentially carrying out identical matching on the fields V1, V2, V, U1, U2, U and Uu associated information list according to S221 to obtain all non-modifiable fields;
the judging and synthesizing unit performs non-modifiable field retrieval on all the data development sentences containing the related table at the moment, performs non-modifiable field retrieval on the data development sentences with the table names of the related table in all the data development sentences at the moment, deletes all languages operating on the non-modifiable field, and returns that the current moment of the field is non-modifiable;
the judging and synthesizing unit splices the deleted data development sentences to generate new data development sentences and transmits the new data development sentences to the operation unit, and the operation unit executes the data development sentences transmitted by the judging and synthesizing unit after receiving the data development sentences;
the data analysis module is used for analyzing the operation of the enterprise developer on the data table in the database, and the specific analysis steps are as follows:
s31: firstly, selecting one database in enterprise data development as a to-be-developed library, selecting one data table in the to-be-developed library as a to-be-developed table, and acquiring all fields in the to-be-developed table, wherein the marks are A1, A2, aa and a is more than or equal to 1;
s32: dividing development sections, dividing a development period into B development sections with equal time length, and marking the B development sections of the development period as B1, B2, bb and more than or equal to 1;
s33: according to a certain calculation step, the cascade duty ratio G1 of the field A1 and the field A2 in the to-be-published table in b development sections of one development period is calculated and obtained, and the steps are as follows:
s331: acquiring all data development sentences which are marked as C1, C2, cc and C is more than or equal to 1 and developed by a developer aiming at a to-be-developed table in a development period development section B1;
s332: acquiring all data processing sentences containing a field A1 in data development sentences C1, C2, and Cc in a development section of a development period B1, and calibrating the data processing sentences as SQL sentences to be analyzed;
s333: acquiring the quantity D1 of SQL sentences to be analyzed, which simultaneously contain the fields A1 and A2, in all the SQL sentences to be analyzed;
s334: using the formulaCalculating and obtaining the synchronous duty ratio E1 of the field A1 and the field A2 in a development period development section B1, wherein alpha is the total number of SQL sentences to be analyzed;
s335: according to S331 to S334, the synchronous duty ratios E1, E2, and Ec of the field A1 and the field A2 in the to-be-published list in the development period b are calculated and obtained;
s336: according to the formulaCalculating and obtaining the discrete value F of the synchronization ratio of the field A1 and the field A2 in the to-be-opened table in the development period b, and adding F to the discrete value FF1 is subjected to size comparison, if F is more than or equal to F1, corresponding Ee values are sequentially deleted according to the sequence of |Ee-E| from large to small, discrete values F of the residual Ee values are calculated, and F1 are subjected to size comparison again until F<F1, wherein F1 is a preset threshold value, and E is the average value of the synchronous duty ratio of the field A1 and the field A2 calculated by the reference and discrete values in the development period b;
the average value of the synchronous duty ratios of the field A1 and the field A2 calculated by the reference and discrete values in the development period b is recalibrated to be the cascade duty ratio, and the average value is marked as G1;
s347: according to the formulaCalculating to obtain the discrete value G of the synchronization ratio of the field A1 and the field A2 in the to-be-opened table in the development period b, comparing the sizes of G and G1, if G is more than or equal to G1, deleting the corresponding Fg values in sequence from large to small according to the absolute value of Fg-F, calculating the discrete value G of the residual Fg values, and comparing the sizes of G and G1 again until G is larger than or equal to G<G1, wherein G1 is a preset threshold value, and F is the average value of the synchronization ratio of the field A1 and the field A2 calculated by the reference and discrete values in the development period b;
the average value of the synchronous ratio of the field A1 and the field A2 calculated by the reference and discrete values in the development period b is recalibrated to be the cascade duty ratio, and is marked as H1;
s35: according to a certain generation step, generating a correlation list J1 of a field A1 in a to-be-developed table, wherein the specific judgment generation step is as follows:
s351: creating an empty association list J1 of a field A1 in the to-be-developed table, namely, an association list J1= [ ];
the cascade duty ratio H1, H2, and h.i. h.t of the field A1 and the field A2 in the b development periods of t development periods are obtained by calculation according to S32 to S34, in this embodiment, t development periods are t development periods back from the current development period to the past, in this embodiment, 1 development period is 1 day, and one development period is 1 hour;
s352: using the formulaCalculating to obtain discrete values I of cascade duty ratios of fields A1 and A2 in t development periods b, comparing the sizes of the I and the I1, if I is more than or equal to I1, deleting corresponding Hi values in sequence from big to small according to I Hi-H I, calculating discrete values I of the residual Hi, and comparing the sizes of the I and the I1 again until I is larger than or equal to I1<I1, wherein I1 is a preset threshold value, H is a field A1 and a field A2 cascade duty ratio average value calculated by reference and discrete values in b development sections of t development periods, and the field A1 and the field A2 cascade duty ratio average value are recalibrated to be identical-cascade duty ratio;
s353: comparing H with eta, if H is larger than or equal to eta, judging that the field A1 and the field A2 of the table to be developed have an association relation, and generating a corresponding association field A according to the field A2 1-2 The associated field A 1-2 The subscript "1" refers to field A1, the subscript "2" refers to field A2, and field A will be associated 1-2 Added to association list J1 of field A1;
otherwise, judging that the field A1 and the field A2 of the to-be-published table have no association relationship, wherein eta is a preset threshold;
s354: according to S34 to S353, performing association field judgment on the field A1 and the field A2 of the table to be developed, and acquiring an association list J1 of the final field A1;
s36: according to S35, obtaining a determined final association list, where the final association list includes association list J1 of field A1, association list J2 of field A2, association list Ja of field Aa;
s37: calculating and obtaining a union interval [ Hmax, phi 1] of a to-be-published table;
s371: acquiring the maximum value Hmax and the minimum value Hmin of the homonymy duty ratios corresponding to all the association fields in the association lists J1, J2;
s372: calculating and obtaining a judgment limit value phi of the co-linked duty ratio by using a formula phi=hmax- λ1× (Hmax-Hmin), wherein λ1 is a preset duty ratio coefficient;
s38: generating all associated information lists N1, N2, and the third and the fourth lists N of the to-be-published tables according to a certain generation step k/2 ；
S381: acquiring all associated fields with the same-linked duty ratio in a same-linked interval [ Hmax, phi ] and sequentially recalibrating the associated fields into cascade fields according to the sequence from the same-linked duty ratio to the smaller one, wherein the cascade fields are marked as K1, K2, kk and K is more than or equal to 1 and less than or equal to a;
s382: taking a cascading field K1 as an example, acquiring a first subscript K1 and a second subscript K2 of an original associated field corresponding to the cascading field K1, wherein K1 and K2 are respectively the numerical values before and after the associated field subscript "-";
s383: according to S382, the first subscript and the second subscript of the original associated field corresponding to the cascade field K2, the..and the Kk are sequentially acquired: k3 and k4, k5 and k6, k-1 and kk;
s384: if the fields corresponding to the first subscript k2 exist in the first subscripts k3, k5, and kk-1, taking the first same subscript as the first subscript corresponding to the field k2, marking as Kl 1, and obtaining a second subscript of the associated field to which the Kl 1 belongs, marking as Kl2 according to the sequence of the first subscript k3, k5, and kk-1;
obtaining fields corresponding to K1, K2 and kl2, namely M1, M2 and Ml2, and generating a related information list N1 according to the combination of the fields M1, M2 and Ml2, wherein N1= [ M1, M2, ml2];
otherwise, fields corresponding to K1 and K2 are obtained and marked as O1 and O2, and a related information list N1 is generated according to the combination of the fields O1 and O2, and then N1= [01, O2];
s385: the first subscript and the second subscript of the original associated field corresponding to the cascade field: deleting the first subscripts corresponding to the fields in the associated information list N1 in k1 and k2, k3 and k4, k5 and k6, and generating all associated information lists N2, and N according to S384 k/2 ；
The data analysis module is used for analyzing the data according to all the associated information lists N1, N2, the number of the user and the number of the user k/2 Generating field association information data of a table to be developed;
s39: acquiring all data tables contained in a to-be-developed library, marking the data tables as P1, P2, and P, wherein P is more than or equal to 1, sequentially selecting the data tables as P1, P2, P as to-be-developed tables according to S31 to S38, and generating field related information data of all the data tables in the to-be-developed library;
s310: according to S31 to S39, acquiring databases of all projects of the enterprise and acquiring field association information data of all data tables in the corresponding databases;
the data analysis module transmits field association information data of the data tables in the databases of all the projects of the enterprise to the development end, the development end receives the field association information data of the data tables in the databases of all the projects of the enterprise and transmits the field association information data to the storage module, and the storage module receives the field association information data of the data tables in the databases of all the projects of the enterprise and transmits the field association information data to the information storage unit for permanent storage.
In the description of the present specification, the descriptions of the terms "one embodiment," "example," "specific example," and the like, mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The foregoing is merely illustrative and explanatory of the invention, as various modifications and additions may be made to the particular embodiments described, or in a similar manner, by those skilled in the art, without departing from the scope of the invention or exceeding the scope of the invention as defined in the claims.
The foregoing describes one embodiment of the present invention in detail, but the description is only a preferred embodiment of the present invention and should not be construed as limiting the scope of the invention. All equivalent changes and modifications within the scope of the present invention are intended to be covered by the present invention.
Claims (8)
1. The data development method based on the data center is characterized by comprising the following steps of:
step one: the data center station collects data generated in the business handling process of the enterprise project and generates a data table of different businesses of the project according to the business type of the project, wherein business data of the enterprise comprise order data, client data, commodity data and supply chain data;
step two: the data analysis module analyzes the operation of the enterprise developer on the data tables in the database to generate field association information data of all the data tables in all the databases of the enterprise, wherein the field association information data of one data table comprises an association information list of all the fields in the data table;
step three: the development end comprises a storage module, an authentication module, a data processing module and a plurality of development modules, wherein the authentication module authenticates the identity of a developer in the enterprise to generate a pre-selection instruction and transmits the pre-selection instruction to the development modules;
step four: the development module comprises a preloading unit and a development unit, wherein the development module receives a preselected instruction transmitted by the authentication module, connects databases required by current developers for data development, acquires field associated information data of all data tables of the databases, and loads the field associated information data into a memory;
the development unit acquires a data development statement of the developer according to a certain acquisition mode and transmits the data development statement to the data processing module;
step five: the data processing module receives the data development sentences transmitted by the development units and then operates the data in the database, the data processing module comprises a judging and synthesizing unit and an operating unit, if the data processing module receives only one data development sentence transmitted by the development unit at the same time, the data processing module transmits the data development sentences transmitted by the development unit to the operating unit, and the operating unit receives the data development sentences transmitted by the data processing module and executes the data development sentences;
if the data processing module receives the data development sentences transmitted by the development units at the same moment, the data processing module transmits the data development sentences to the judging and synthesizing unit, and the judging and synthesizing unit judges the data development sentences transmitted by the data processing module according to a certain judging step after receiving the data development sentences to generate homologous data of all relevant tables at the moment.
2. The data development method based on the data center as claimed in claim 1, wherein the developer obtains the data development statement of the developer in two ways: one is that the development unit acquires a data development statement which is newly input by a developer until the current moment after the developer inputs an active submission instruction, and the other is that the development unit detects that the developer is inputting; and acquiring a data development statement newly input by a developer at the current moment when no input is performed in W later time, wherein W is a preset time threshold.
3. The data development method based on the data center as claimed in claim 1, wherein the data development statement specifically refers to SQL statement, one of the data development statement represents an SQL statement, and one of the data development statement ends is used "; the "end" represents the end.
4. A data development method based on a data center as claimed in claim 1, wherein the operation unit includes an sql executor.
5. The data development method according to claim 1, wherein the specific determination step of the determination synthesis unit determining to generate all relevant table source data at the time is as follows:
s11: taking a data development statement at the moment as an example, sequentially acquiring a table name, a field name and a field value contained in the data development statement and generating comparison information data of the data development statement according to the table name, the field name and the field value;
s12: according to S11, obtaining the comparison information data of all the data development sentences at the moment and synchronously comparing the comparison information data:
if the table names and the field names are consistent in the comparison information data of all the data development sentences at the moment, the comparison information data are marked as homologous information data, the data tables with the same names in the comparison information data are marked as related tables, and the fields with the same names are marked as related fields;
s13: taking a related table as an example, acquiring all data development sentences with table names being the related table and field names being consistent, wherein the data development sentences are marked as R1, R2, rr and R is more than or equal to 1;
acquiring field association information data of the correlation table;
s14: taking the data development statement R1 as an example, if the sql statement contains other fields except relevant fields, marking the other fields except relevant fields contained in the sql statement as fields to be relevant;
otherwise, do not do any treatment;
s15: according to S14, all fields to be correlated in the data development sentences R1, R2, R, and R are sequentially acquired, and homologous data of the correlation table are generated according to the fields to be correlated and the correlation fields of the correlation table;
s16: according to S13 to S15, the homologous data of all the correlation tables at that time are calculated and acquired.
6. The data development method based on the data center as claimed in claim 5, wherein the decision synthesizing unit splices the data development sentences according to the homologous data of all tables in the data development sentences at the moment, and the specific steps are as follows:
s21: taking homologous data of a correlation table as an example, acquiring all relevant fields U1, U2, U.S., U is more than or equal to 1, and fields V1, V2, vv, V is more than or equal to 1 in the data of the correlation table;
s22: acquiring relevant fields U1, U2, and fields in a Uu associated information list in relevant field associated information data of the relevant table, carrying out homologous matching on the relevant fields with fields V1, V2, and Vv to be relevant, and acquiring all non-modifiable fields:
s221: taking the field V1 to be correlated as an example, if the field V1 to be correlated is the same as the fields in the U1, U2, the U.S. and Uu associated information list, the field V1 to be correlated is recalibrated to be an unmodified field, otherwise, no processing is performed;
s222: sequentially carrying out identical matching on the fields V1, V2, V, U1, U2, U and Uu associated information list according to S221 to obtain all non-modifiable fields;
the judging and synthesizing unit performs non-modifiable field retrieval on all the data development sentences containing the related table at the moment, performs non-modifiable field retrieval on the data development sentences with the table names of the related table in all the data development sentences at the moment, deletes all languages operating on the non-modifiable field, and returns that the current moment of the field is non-modifiable;
the judging and synthesizing unit splices the deleted data development sentences to generate new data development sentences and transmits the new data development sentences to the operation unit, and the operation unit executes the data development sentences transmitted by the judging and synthesizing unit after receiving the data development sentences.
7. The data development method based on the data center as claimed in claim 1, wherein the specific steps of the data analysis unit for analyzing and generating all table field associated information data are as follows:
s31: firstly, selecting one database in enterprise data development as a to-be-developed library, selecting one data table contained in the to-be-developed library as a to-be-developed table, and acquiring all fields in the to-be-developed table, wherein the marks are A1, A2, aa and a is more than or equal to 1;
s32: dividing development sections, dividing a development period into B development sections with equal time length, and marking the B development sections of the development period as B1, B2, bb and more than or equal to 1;
s33: acquiring all data development sentences which are marked as C1, C2, cc and C is more than or equal to 1 and are input by a developer for development of a to-be-developed table in a development period development section B1;
s34: according to a certain calculation step, the cascade duty ratio H1 of the field A1 and the field A2 in the to-be-published list in b development sections of one development period is calculated and obtained;
s35: generating an association list J1 of a field A1 in a to-be-developed table according to a certain generation step;
s36: according to S35, obtaining a determined final association list, where the final association list includes association list J1 of field A1, association list J2 of field A2, association list Ja of field Aa;
s37: calculating and obtaining a union interval [ Hmax, phi 1] of a to-be-published table;
s371: acquiring the maximum value Hmax and the minimum value Hmin of the homonymy duty ratios corresponding to all the association fields in the association lists J1, J2;
s372: calculating and obtaining a judgment limit value phi of the co-linked duty ratio by using a formula phi=hmax- λ1× (Hmax-Hmin), wherein λ1 is a preset duty ratio coefficient;
s38: generating all associated information lists N1, N2, and the third and the fourth lists N of the to-be-published tables according to a certain generation step k/2 ；
S39: acquiring all data tables contained in a to-be-developed library, marking the data tables as P1, P2, and P, wherein P is more than or equal to 1, sequentially selecting the data tables as P1, P2, P as to-be-developed tables according to S31 to S38, and generating field related information data of all the data tables in the to-be-developed library;
s310: according to S31 to S39, a database of all the items of the enterprise is obtained, and field association information data of all the data tables in the corresponding database is obtained.
8. The data development method based on the data center as claimed in claim 1, wherein the storage module includes a database storage unit and an information storage unit, and the information storage unit stores field-related information data of all data tables in all databases of the enterprise.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211604285.0A CN115994142B (en) | 2022-12-13 | 2022-12-13 | Data development method based on data center |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211604285.0A CN115994142B (en) | 2022-12-13 | 2022-12-13 | Data development method based on data center |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115994142A CN115994142A (en) | 2023-04-21 |
| CN115994142B true CN115994142B (en) | 2024-04-02 |
Family
ID=85989777
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211604285.0A Active CN115994142B (en) | 2022-12-13 | 2022-12-13 | Data development method based on data center |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115994142B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105912634A (en) * | 2016-04-05 | 2016-08-31 | 扬州大学 | Software code search oriented query statement regenerating method |
| CN108304463A (en) * | 2017-12-26 | 2018-07-20 | 中国广核电力股份有限公司 | A kind of data managing method and its database application component for database |
| CN111382170A (en) * | 2018-12-29 | 2020-07-07 | 北京亿阳信通科技有限公司 | Automatic statement conversion method and device |
| CN112434059A (en) * | 2021-01-26 | 2021-03-02 | 腾讯科技（深圳）有限公司 | Data processing method, data processing device, computer equipment and storage medium |
| CN115455091A (en) * | 2022-09-22 | 2022-12-09 | 金篆信科有限责任公司 | Data generation method and device, electronic equipment and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100558952B1 (en) * | 2005-05-12 | 2006-03-10 | 김길웅 | Method for automation of software manufacturing process based on graphic user interface design, and computer readable medium having thereon computer executable instruction for performing the same |
| CN110443059A (en) * | 2018-05-02 | 2019-11-12 | 中兴通讯股份有限公司 | Data guard method and device |
-
2022
- 2022-12-13 CN CN202211604285.0A patent/CN115994142B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105912634A (en) * | 2016-04-05 | 2016-08-31 | 扬州大学 | Software code search oriented query statement regenerating method |
| CN108304463A (en) * | 2017-12-26 | 2018-07-20 | 中国广核电力股份有限公司 | A kind of data managing method and its database application component for database |
| CN111382170A (en) * | 2018-12-29 | 2020-07-07 | 北京亿阳信通科技有限公司 | Automatic statement conversion method and device |
| CN112434059A (en) * | 2021-01-26 | 2021-03-02 | 腾讯科技（深圳）有限公司 | Data processing method, data processing device, computer equipment and storage medium |
| CN115455091A (en) * | 2022-09-22 | 2022-12-09 | 金篆信科有限责任公司 | Data generation method and device, electronic equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 基于Json的小型异构数据库同步策略研究;黄志;李涛;宋瑶;苏传程;;《气象研究与应用》;20200315(第1期);50-55 * |
| 基于STM32的外加式电梯语音交互按键系统;贾志成 等;《吉林大学学报（信息科学版）》;20220412;第40卷(第2期);313-318 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115994142A (en) | 2023-04-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8316292B1 (en) | Identifying multiple versions of documents | |
| McColl et al. | A new parallel algorithm for connected components in dynamic graphs | |
| US20060218160A1 (en) | Change control management of XML documents | |
| US20150026230A1 (en) | System and Method for Performing Distributed Asynchronous Calculations in a Networked Environment | |
| CN103514223A (en) | Data synchronism method and system of database | |
| US20070005546A1 (en) | Attribute engine | |
| US20050060345A1 (en) | Methods and systems for using XML schemas to identify and categorize documents | |
| CN112015741A (en) | Method and device for storing massive data in different databases and tables | |
| US11086906B2 (en) | System and method for reconciliation of data in multiple systems using permutation matching | |
| CN106716420A (en) | Message matching method, message matching device, computer program product and electronic equipment | |
| Amiri et al. | Permissioned blockchains: Properties, techniques and applications | |
| CN115994142B (en) | Data development method based on data center | |
| US7225198B2 (en) | Data compiling method | |
| US20010032077A1 (en) | Compare | |
| Lukács | neatStats: An R package for a neat pipeline from raw data to reportable statistics in psychological science | |
| CN112579604A (en) | Test system number making method, device, equipment and storage medium | |
| CN111858534A (en) | Ordering method for increasing large data volume of logs | |
| US8037109B2 (en) | Generation of repeatable synthetic data | |
| CN116467219A (en) | Test processing method and device | |
| Huang et al. | Simultaneous selection and incorporation of consistent external aggregate information | |
| Compton et al. | Intelligent validation and routing of electronic forms in a distributed workflow environment | |
| CN111506578B (en) | Service data verification method, device, equipment and storage medium | |
| US20230421398A1 (en) | System and method for inserting transactions into transaction pools | |
| CN117149100B (en) | Data storage method and device | |
| Daviaud et al. | The Big-O Problem for Max-Plus Automata is Decidable (PSPACE-Complete) |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |