CN118092908A - Application program generation method and device based on large language model - Google Patents
Application program generation method and device based on large language model Download PDFInfo
- Publication number
- CN118092908A CN118092908A CN202410458660.8A CN202410458660A CN118092908A CN 118092908 A CN118092908 A CN 118092908A CN 202410458660 A CN202410458660 A CN 202410458660A CN 118092908 A CN118092908 A CN 118092908A
- Authority
- CN
- China
- Prior art keywords
- model
- design
- training
- component
- fine tuning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000013461 design Methods 0.000 claims abstract description 193
- 230000006870 function Effects 0.000 claims abstract description 38
- 238000012549 training Methods 0.000 claims description 91
- 238000002372 labelling Methods 0.000 claims description 30
- 238000011156 evaluation Methods 0.000 claims description 18
- 239000013598 vector Substances 0.000 claims description 15
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 230000008014 freezing Effects 0.000 claims description 6
- 238000007710 freezing Methods 0.000 claims description 6
- 238000011161 development Methods 0.000 abstract description 7
- 238000012938 design process Methods 0.000 abstract description 4
- 238000009966 trimming Methods 0.000 abstract 2
- 238000009825 accumulation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Landscapes
- Stored Programmes (AREA)
Abstract
The invention discloses an application program generation method and device based on a large language model, and relates to the technical field of application development. Aiming at the problems that a user needs to relearn for a low-code platform and the steps of a complex design process are complicated, the method comprises the following steps: defining a form generation specification and a component generation specification; trimming the large language model by utilizing a trimming data set conforming to the specification, thereby obtaining a structural design model with structural design capability and a form design model with form design capability; carrying out semantic analysis on system function requirement description of a user by utilizing a structural design model and a form design model, and converting the requirement description into a data structure conforming to a generation specification; the data conforming to the generation specification is converted into a real form and components using a low code generator, thereby generating a usable application. The invention can effectively save labor cost.
Description
Technical Field
The invention relates to the technical field of application development, in particular to an application program generation method and device based on a large language model.
Background
The low-code development platform is a software development tool based on a graphical interface and a templated design, and can quickly construct an application program by means of dragging components, configuration parameters and the like, so that the development period is greatly shortened, the development cost is reduced, and the development efficiency is improved.
There are many low-code platforms on the market at present, and these platforms all provide visual design tools and templated design modes, so that enterprise applications can be quickly built. Meanwhile, the platforms also provide rich component libraries, so that a user can conveniently deploy and maintain application programs.
But now the low code platform still has the following drawbacks:
1. the user needs a certain learning cost to be familiar with the operation of the low code platform.
2. In the face of more complex user demands, users need to design the form by themselves and draw the form step by step, and the step is time-consuming and not convenient enough.
The large language model refers to a deep learning model trained using a large amount of text data, and the meaning of natural language text can be well understood. The fine tuning of the large language model refers to optimizing the performance of the model by performing supervised training on specific tasks on the basis of the pre-trained model, and the fine tuning method can enable the model to better adapt to task requirements of specific fields.
Disclosure of Invention
Aiming at the needs and the shortcomings of the prior art development, the invention provides a large language model-based application program generation method and device, which are used for solving the problems that a user needs to relearn a low-code platform and steps of a complex design process are complicated, and effectively saving the labor cost of the user.
In a first aspect, the present invention provides a method for generating an application program based on a large language model, which solves the above technical problems by adopting the following technical scheme:
An application program generation method based on a large language model, comprising the following steps:
s1, defining a form generation specification and a component generation specification;
s2, fine tuning is carried out on the large language model by utilizing a fine tuning data set which accords with the specification, so that a structural design model with structural design capability and a form design model with form design capability are obtained;
S3, carrying out semantic analysis on the system function requirement description of the user by utilizing the structural design model and the form design model, and converting the requirement description into a data structure conforming to the generation specification;
S4, converting the data conforming to the generation specification into a real form and a component by using a low-code generator, so as to generate a usable application program.
Alternatively, the step S1 is performed,
The defined form generation specification is a json data structure and comprises a name field and a description field, wherein the name field is defined as a form name, and the description field is defined as a form function description;
The defined component generation specification is a json data structure and comprises a name field and a type field, wherein the name field is defined as a component name, and the type field is defined as a component type; the value of type belongs to one of input, number, textarea, checkbox, fileupload, select, where input means a single line of text, number means a number selector, textarea means a multiple line of text, checkbox means a check box, fileupload means a file upload, and select means a drop down selection box.
Further optionally, step S2 is performed to perform fine tuning on the large language model using a fine-tuning dataset that meets the specification, the process involving three parts of a fine-tuning dataset, a model trainer, and a model evaluator, wherein:
S2.1, a fine tuning data set F= { D, T } comprises a structural design model fine tuning data set D= { D 1,d2,d3...dn } and a form design model fine tuning data set T= { T 1,t2,t3...tn},dn and T n which are single pieces of fine tuning data, a structural design model is obtained after a large language model is subjected to fine tuning training of the structural design model fine tuning data set D, and a form design model is obtained after the form design model fine tuning data set T is subjected to fine tuning; the fine tuning data in the fine tuning data set consists of a system demand description text and a labeling label, wherein the demand description text is a real system design demand text, the labeling label is a real design result of an application program corresponding to demand description, and the labeling label of the structural design model fine tuning data set D is required to meet a form generation specification;
s2.2, the model training device is used for updating the model structure by utilizing the fine adjustment data set, so that the model has the system design capability, and the training process is as follows:
S2.1.1, preparing a candidate prompt word set P= { P 1,p2,p3...pk }, enabling labeling labels of the candidate prompt word set P to conform to component generation specifications, setting a requirement description text of fine adjustment data as x n, and enabling an original weight of a pre-training model to be W;
S2.1.2, polling a candidate prompt word set P, and splicing vectors converted from P n through the encoding into tokens and the vectors converted from x n through embedding, wherein the vectors are used as input of model training;
s2.1.3, update tokens of p n only per training round until training loss convergence no longer drops;
s2.1.4, initializing two low-rank matrices B and a;
s2.1.5, training and updating model weights through a fine adjustment data set, specifically freezing an original weight W of a pre-training model, and updating only weights of low-rank matrixes B and A, wherein a training model weight updating value delta W=BA of each round;
S2.1.6, after training, changing the model weight into:
W` = W + ΔW；
S2.3, a model evaluator is used for evaluating the trained model, and evaluating the model design capacity by comparing the design result and the real label of the model, wherein the evaluation score n of a single sample has the following calculation formula:
formula (a),
The formula (b),
Formula (c),
Wherein,Representing the generation of the correct component,/>Representing the component that generated the error,/>Representing components which are in the labeling data but are not actually generated by the model;
The evaluation score S of the final model was calculated as follows:
The formula (d),
After training is completed, the model after training is evaluated by a model evaluator, a completion threshold T is set according to the requirement, and training can be completed when the evaluation score S is greater than the threshold T, otherwise, the fine adjustment data set is supplemented to continue training.
Further optionally, the step S3 involved specifically comprises the following operations:
S3.1, acquiring a user system function requirement I;
S3.2, the user system function requirement I is transmitted into a structural design model, the structural design model generates a form list M consisting of at least one form name according to the user system function requirement I by virtue of the semantic understanding capability of the structural design model, and the output format is required to meet the form generation specification;
S3.3, inputting a form list M and a user system function requirement I into a form design model, wherein the form design model respectively outputs a design result T= [ T 1,t2,...,tn ] of each form, and T n represents the design result of each form;
S3.4, the form list M and the design result T of each form are final output of the system design module, and the form list M and the design result T of each form are summarized and input into a low code generator.
Optionally, step S4 is executed, where the low code generator is used to convert the data meeting the generation specification into a real form and a component, so as to generate a usable application program, which specifically includes:
Converting the data conforming to the form generation specification into an actual application form list;
Converting the data conforming to the component generation specification into actual components of each form, wherein the specific conversion rules of the components are as follows: the type value is input converted into a single line text component, the type value is select converted into a drop-down box selector component, the type value is textarea converted into a multiple line text component, the type value is checkbox converted into a check box component, the type value is fileupload converted into a file upload component, and the type value is number converted into a digital component.
In a second aspect, the present invention provides an application program generating device based on a large language model, which solves the technical problems as follows:
an application generation device based on a large language model, comprising:
the specification definition module is used for defining a form generation specification and a component generation specification;
the model training module is used for carrying out fine adjustment on the large language model by utilizing the fine adjustment data set which accords with the specifications, so as to obtain a structural design model with structural design capability and a form design model with form design capability;
The system design module comprises a structural design model and a form design model, and is used for carrying out semantic analysis on the system function requirement description of the user and converting the requirement description into a data structure conforming to the generation specification;
The system generation module is used as a low-code generator and is used for converting data conforming to the generation specification into a real form and a component so as to generate a usable application program.
Optionally, the form generation specification defined by the related specification definition module is a json data structure, and comprises a name field and a description field, wherein the name field is defined as a form name, and the description field is defined as a form function description;
The component generation specification defined by the specification definition module is a json data structure and comprises a name field and a type field, wherein the name field is defined as a component name, and the type field is defined as a component type; the value of type belongs to one of input, number, textarea, checkbox, fileupload, select, where input means a single line of text, number means a number selector, textarea means a multiple line of text, checkbox means a check box, fileupload means a file upload, and select means a drop down selection box.
Further optionally, the model training module involved includes three parts, namely a fine tuning data set, a model trainer and a model evaluator, wherein:
(1) The fine tuning data sets F= { D and T } comprise a structural design model fine tuning data set D= { D 1,d2,d3...dn } and form design model fine tuning data sets T= { T 1,t2,t3...tn},dn and T n which are single pieces of fine tuning data, the structural design model is obtained after the large language model is subjected to fine tuning training of the structural design model fine tuning data set D, and the form design model is obtained after the form design model fine tuning data set T is subjected to fine tuning; the fine tuning data in the fine tuning data set consists of a system demand description text and a labeling label, wherein the demand description text is a real system design demand text, the labeling label is a real design result of an application program corresponding to demand description, and the labeling label of the structural design model fine tuning data set D is required to meet a form generation specification;
(2) The model training device is used for updating the model structure by utilizing the fine adjustment data set, so that the model has the system design capability, and the training process is as follows:
(2.1) preparing a candidate prompt word set P= { P 1,p2,p3...pk }, enabling labeling labels of the candidate prompt word set P to conform to component generation specifications, setting a requirement description text of fine adjustment data as x n, and enabling an original weight of a pre-training model to be W;
(2.2) polling the candidate prompt word set P, and splicing vectors converted from tokens converted from P n through encodings and embedding converted from x n, wherein the vectors are used as input of model training;
(2.3) updating only tokens of p n per round of training until training loss convergence no longer drops;
(2.4) initializing two low rank matrices B and a;
(2.5) training the updated model weight through the fine-tuning data set, specifically freezing the original weight W of the pre-trained model, and updating only the weights of the low-rank matrixes B and A, wherein the updated value delta W=BA of the trained model weight of each round;
(2.6) changing the model weight after training is completed to be:
W` = W + ΔW；
(3) The model evaluator is used for evaluating the trained model, and evaluating the model design capacity by comparing the design result and the real label of the model, and the evaluation score n of a single sample has the following calculation formula:
formula (a),
The formula (b),
Formula (c),
Wherein,Representing the generation of the correct component,/>Representing the component that generated the error,/>Representing components which are in the labeling data but are not actually generated by the model;
The evaluation score S of the final model was calculated as follows:
The formula (d),
After training is completed, the model after training is evaluated by a model evaluator, a completion threshold T is set according to the requirement, and training can be completed when the evaluation score S is greater than the threshold T, otherwise, the fine adjustment data set is supplemented to continue training.
Further optionally, the system design module involved converts the demand description into a data structure compliant with the generation specification by:
(1) Collecting a user system function requirement I;
(2) The user system function requirement I is transmitted into a structural design model, the structural design model generates a form list M composed of at least one form name according to the user system function requirement I by means of semantic understanding capability of the structural design model, and an output format is required to meet form generation specifications;
(3) Inputting a form list M and a user system function requirement I into a form design model, wherein the form design model respectively outputs a design result T= [ T 1,t2,...,tn ] of each form, and T n represents the design result of each form;
(4) The form list M and the design result T of each form are the final output of the system design module, and the form list M and the design result T of each form are summarized and input to the system generation module.
Optionally, the specific steps involved in the system generation module to generate the usable application program are as follows:
Converting the data conforming to the form generation specification into an actual application form list;
Converting the data conforming to the component generation specification into actual components of each form, wherein the specific conversion rules of the components are as follows: the type value is input converted into a single line text component, the type value is select converted into a drop-down box selector component, the type value is textarea converted into a multiple line text component, the type value is checkbox converted into a check box component, the type value is fileupload converted into a file upload component, and the type value is number converted into a digital component.
Compared with the prior art, the method and the device for generating the application program based on the large language model have the beneficial effects that:
The invention utilizes the fine-tuning data set which accords with the specification to carry out fine tuning on the large language model to obtain the structural design model with structural design capability and the form design model with form design capability, and further directly converts the natural language description of the system function design requirement into the usable application program page, thereby being used for solving the problems that a user needs to learn again for a low-code platform and has complicated steps for a complex design process, and effectively saving the labor cost of the user.
Drawings
FIG. 1 is a flow chart of a method according to a first embodiment of the invention;
FIG. 2 is a block diagram of a module connection according to a second embodiment of the present invention;
FIG. 3 is a flowchart of the system design module according to the second embodiment of the present invention.
Detailed Description
In order to make the technical scheme, the technical problems to be solved and the technical effects of the invention more clear, the technical scheme of the invention is clearly and completely described below by combining specific embodiments.
Embodiment one:
Referring to fig. 1, this embodiment proposes an application program generating method based on a large language model, which includes the following steps:
S1, defining a form generation specification and a component generation specification.
The defined form generation specification is a json data structure and comprises a name field and a description field, wherein the name field is defined as a form name, and the description field is defined as a form function description.
The defined component generation specification is a json data structure and comprises a name field and a type field, wherein the name field is defined as a component name, and the type field is defined as a component type; the value of type belongs to one of input, number, textarea, checkbox, fileupload, select, where input means a single line of text, number means a number selector, textarea means a multiple line of text, checkbox means a check box, fileupload means a file upload, and select means a drop down selection box.
S2, fine tuning is carried out on the large language model by utilizing a fine tuning data set which accords with the specification, so that a structural design model with structural design capability and a form design model with form design capability are obtained;
the process of fine tuning a large language model involves three parts, a fine tuning dataset, a model trainer, and a model evaluator, wherein:
S2.1, a fine tuning data set F= { D, T } comprises a structural design model fine tuning data set D= { D 1,d2,d3...dn } and a form design model fine tuning data set T= { T 1,t2,t3...tn},dn and T n which are single pieces of fine tuning data, a structural design model is obtained after a large language model is subjected to fine tuning training of the structural design model fine tuning data set D, and a form design model is obtained after the form design model fine tuning data set T is subjected to fine tuning; the fine tuning data in the fine tuning data set consists of a system demand description text and a labeling label, wherein the demand description text is a real system design demand text, the labeling label is a real design result of an application program corresponding to demand description, and the labeling label of the structural design model fine tuning data set D is required to meet a form generation specification.
Table 1 below illustrates a single structural design model trim data d n.
Table 1:
table 2 below illustrates exemplary single form design model trim data t n.
Table 2:
s2.2, the model training device is used for updating the model structure by utilizing the fine adjustment data set, so that the model has the system design capability, and the training process is as follows:
S2.1.1, preparing a candidate prompt word set P= { P 1,p2,p3...pk }, enabling labeling labels of the candidate prompt word set P to conform to component generation specifications, setting a requirement description text of fine adjustment data as x n, and enabling an original weight of a pre-training model to be W;
S2.1.2, polling a candidate prompt word set P, and splicing vectors converted from P n through the encoding into tokens and the vectors converted from x n through embedding, wherein the vectors are used as input of model training;
s2.1.3, update tokens of p n only per training round until training loss convergence no longer drops;
s2.1.4, initializing two low-rank matrices B and a;
s2.1.5, training and updating model weights through a fine adjustment data set, specifically freezing an original weight W of a pre-training model, and updating only weights of low-rank matrixes B and A, wherein a training model weight updating value delta W=BA of each round;
S2.1.6, after training, changing the model weight into:
W` = W + ΔW；
S2.3, a model evaluator is used for evaluating the trained model, and evaluating the model design capacity by comparing the design result and the real label of the model, wherein the evaluation score n of a single sample has the following calculation formula:
formula (a),
The formula (b),
Formula (c),
Wherein,Representing the generation of the correct component,/>Representing the component that generated the error,/>Representing components which are in the labeling data but are not actually generated by the model;
The evaluation score S of the final model was calculated as follows:
The formula (d),
After training is completed, the model after training is evaluated by a model evaluator, a completion threshold T is set according to the requirement, and training can be completed when the evaluation score S is greater than the threshold T, otherwise, the fine adjustment data set is supplemented to continue training.
S3, carrying out semantic analysis on the system function requirement description of the user by utilizing the structural design model and the form design model, and converting the requirement description into a data structure conforming to the generation specification, wherein the method specifically comprises the following operations:
S3.1, acquiring a user system function requirement I;
S3.2, the user system function requirement I is transmitted into a structural design model, the structural design model generates a form list M consisting of at least one form name according to the user system function requirement I by virtue of the semantic understanding capability of the structural design model, and the output format is required to meet the form generation specification;
S3.3, inputting a form list M and a user system function requirement I into a form design model, wherein the form design model respectively outputs a design result T= [ T 1,t2,...,tn ] of each form, and T n represents the design result of each form;
S3.4, the form list M and the design result T of each form are final output of the system design module, and the form list M and the design result T of each form are summarized and input into a low code generator.
When steps S3.1-S3.4 are performed, let i= "design an accumulation fund management system for me", m= [ { "name": "principal information form", "description": "record employee principal information" }, { "name": "principal payment record form", "description": "record employee principal payment record form" }, { "name": employee information form "," description ": record employee principal information" }, then t n = { "type": input "," name ": name" }, { "type": select "," name ": identity", "options": personal "," unit "}, { type": date "," name ": year of birth" }, { "type": number "}, {" type ": type" }, { "type": input "," name ": principal ratio" }, { "type": radio ": whether or not", "name": in "is }" "payment }", and } ] are in the order of no.
S4, converting the data conforming to the generation specification into a real form and a component by using a low-code generator so as to generate a usable application program, wherein the method specifically comprises the following steps of:
Converting the data conforming to the form generation specification into an actual application form list;
Converting the data conforming to the component generation specification into actual components of each form, wherein the specific conversion rules of the components are as follows: the type value is input converted into a single line text component, the type value is select converted into a drop-down box selector component, the type value is textarea converted into a multiple line text component, the type value is checkbox converted into a check box component, the type value is fileupload converted into a file upload component, and the type value is number converted into a digital component.
Embodiment two:
referring to fig. 2, this embodiment proposes an application generating device based on a large language model, which includes:
the specification definition module is used for defining a form generation specification and a component generation specification;
the model training module is used for carrying out fine adjustment on the large language model by utilizing the fine adjustment data set which accords with the specifications, so as to obtain a structural design model with structural design capability and a form design model with form design capability;
The system design module comprises a structural design model and a form design model, and is used for carrying out semantic analysis on the system function requirement description of the user and converting the requirement description into a data structure conforming to the generation specification;
The system generation module is used as a low-code generator and is used for converting data conforming to the generation specification into a real form and a component so as to generate a usable application program.
In this embodiment, the form generation specification defined by the specification definition module is a json data structure, and includes a name field and a description field, where the name field is defined as a form name, and the description field is defined as a form function description. The component generation specification defined by the specification definition module is a json data structure and comprises a name field and a type field, wherein the name field is defined as a component name, and the type field is defined as a component type; the value of type belongs to one of input, number, textarea, checkbox, fileupload, select, where input means a single line of text, number means a number selector, textarea means a multiple line of text, checkbox means a check box, fileupload means a file upload, and select means a drop down selection box.
In this embodiment, the model training module includes three parts, namely a fine tuning data set, a model trainer, and a model evaluator, wherein:
(1) The fine tuning data sets F= { D and T } comprise a structural design model fine tuning data set D= { D 1,d2,d3...dn } and form design model fine tuning data sets T= { T 1,t2,t3...tn},dn and T n which are single pieces of fine tuning data, the structural design model is obtained after the large language model is subjected to fine tuning training of the structural design model fine tuning data set D, and the form design model is obtained after the form design model fine tuning data set T is subjected to fine tuning; the fine tuning data in the fine tuning data set consists of a system demand description text and a labeling label, wherein the demand description text is a real system design demand text, the labeling label is a real design result of an application program corresponding to demand description, and the labeling label of the structural design model fine tuning data set D is required to meet a form generation specification;
(2) The model training device is used for updating the model structure by utilizing the fine adjustment data set, so that the model has the system design capability, and the training process is as follows:
(2.1) preparing a candidate prompt word set P= { P 1,p2,p3...pk }, enabling labeling labels of the candidate prompt word set P to conform to component generation specifications, setting a requirement description text of fine adjustment data as x n, and enabling an original weight of a pre-training model to be W;
(2.2) polling the candidate prompt word set P, and splicing vectors converted from tokens converted from P n through encodings and embedding converted from x n, wherein the vectors are used as input of model training;
(2.3) updating only tokens of p n per round of training until training loss convergence no longer drops;
(2.4) initializing two low rank matrices B and a;
(2.5) training the updated model weight through the fine-tuning data set, specifically freezing the original weight W of the pre-trained model, and updating only the weights of the low-rank matrixes B and A, wherein the updated value delta W=BA of the trained model weight of each round;
(2.6) changing the model weight after training is completed to be:
W` = W + ΔW；
(3) The model evaluator is used for evaluating the trained model, and evaluating the model design capacity by comparing the design result and the real label of the model, and the evaluation score n of a single sample has the following calculation formula:
formula (a),
The formula (b),
Formula (c),
Wherein,Representing the generation of the correct component,/>Representing the component that generated the error,/>Representing components which are in the labeling data but are not actually generated by the model;
The evaluation score S of the final model was calculated as follows:
The formula (d),
After training is completed, the model after training is evaluated by a model evaluator, a completion threshold T is set according to the requirement, and training can be completed when the evaluation score S is greater than the threshold T, otherwise, the fine adjustment data set is supplemented to continue training.
Referring to fig. 3, in this embodiment, the system design module converts the requirement description into a data structure conforming to the generation specification by:
(1) Collecting a user system function requirement I;
(2) The user system function requirement I is transmitted into a structural design model, the structural design model generates a form list M composed of at least one form name according to the user system function requirement I by means of semantic understanding capability of the structural design model, and an output format is required to meet form generation specifications;
(3) Inputting a form list M and a user system function requirement I into a form design model, wherein the form design model respectively outputs a design result T= [ T 1,t2,...,tn ] of each form, and T n represents the design result of each form;
(4) The form list M and the design result T of each form are the final output of the system design module, and the form list M and the design result T of each form are summarized and input to the system generation module.
In this embodiment, the specific steps of the system generation module to generate the usable application program are as follows:
Converting the data conforming to the form generation specification into an actual application form list;
Converting the data conforming to the component generation specification into actual components of each form, wherein the specific conversion rules of the components are as follows: the type value is input converted into a single line text component, the type value is select converted into a drop-down box selector component, the type value is textarea converted into a multiple line text component, the type value is checkbox converted into a check box component, the type value is fileupload converted into a file upload component, and the type value is number converted into a digital component.
In summary, the application program generating method and device based on the large language model can solve the problems that a user needs to learn again for a low-code platform and steps are complicated for a complex design process, and effectively saves the labor cost of the user.
The foregoing has outlined rather broadly the principles and embodiments of the present invention in order that the detailed description of the invention may be better understood. Based on the above-mentioned embodiments of the present invention, any improvements and modifications made by those skilled in the art without departing from the principles of the present invention should fall within the scope of the present invention.
Claims (10)
1. An application program generating method based on a large language model is characterized by comprising the following steps:
s1, defining a form generation specification and a component generation specification;
s2, fine tuning is carried out on the large language model by utilizing a fine tuning data set which accords with the specification, so that a structural design model with structural design capability and a form design model with form design capability are obtained;
S3, carrying out semantic analysis on the system function requirement description of the user by utilizing the structural design model and the form design model, and converting the requirement description into a data structure conforming to the generation specification;
S4, converting the data conforming to the generation specification into a real form and a component by using a low-code generator, so as to generate a usable application program.
2. The method for generating an application program based on a large language model as claimed in claim 1, wherein the step S1 is performed,
The defined form generation specification is a json data structure and comprises a name field and a description field, wherein the name field is defined as a form name, and the description field is defined as a form function description;
The defined component generation specification is a json data structure and comprises a name field and a type field, wherein the name field is defined as a component name, and the type field is defined as a component type; the value of type belongs to one of input, number, textarea, checkbox, fileupload, select, where input means a single line of text, number means a number selector, textarea means a multiple line of text, checkbox means a check box, fileupload means a file upload, and select means a drop down selection box.
3. The method for generating a large language model based application program according to claim 2, wherein step S2 is performed to perform fine tuning on the large language model using a fine tuning dataset that meets specifications, the process involving three parts of a fine tuning dataset, a model trainer, and a model evaluator, wherein:
S2.1, a fine tuning data set F= { D, T } comprises a structural design model fine tuning data set D= { D 1,d2,d3...dn } and a form design model fine tuning data set T= { T 1,t2,t3...tn},dn and T n which are single pieces of fine tuning data, a structural design model is obtained after a large language model is subjected to fine tuning training of the structural design model fine tuning data set D, and a form design model is obtained after the form design model fine tuning data set T is subjected to fine tuning; the fine tuning data in the fine tuning data set consists of a system demand description text and a labeling label, wherein the demand description text is a real system design demand text, the labeling label is a real design result of an application program corresponding to demand description, and the labeling label of the structural design model fine tuning data set D is required to meet a form generation specification;
s2.2, the model training device is used for updating the model structure by utilizing the fine adjustment data set, so that the model has the system design capability, and the training process is as follows:
S2.1.1, preparing a candidate prompt word set P= { P 1,p2,p3...pk }, enabling labeling labels of the candidate prompt word set P to conform to component generation specifications, setting a requirement description text of fine adjustment data as x n, and enabling an original weight of a pre-training model to be W;
S2.1.2, polling a candidate prompt word set P, and splicing vectors converted from P n through the encoding into tokens and the vectors converted from x n through embedding, wherein the vectors are used as input of model training;
s2.1.3, update tokens of p n only per training round until training loss convergence no longer drops;
s2.1.4, initializing two low-rank matrices B and a;
s2.1.5, training and updating model weights through a fine adjustment data set, specifically freezing an original weight W of a pre-training model, and updating only weights of low-rank matrixes B and A, wherein a training model weight updating value delta W=BA of each round;
S2.1.6, after training, changing the model weight into:
W` = W + ΔW；
S2.3, a model evaluator is used for evaluating the trained model, and evaluating the model design capacity by comparing the design result and the real label of the model, wherein the evaluation score n of a single sample has the following calculation formula:
formula (a),
The formula (b),
Formula (c),
Wherein,Representing the generation of the correct component,/>Representing the component that generated the error,/>Representing components which are in the labeling data but are not actually generated by the model;
The evaluation score S of the final model was calculated as follows:
The formula (d),
After training is completed, the model after training is evaluated by a model evaluator, a completion threshold T is set according to the requirement, and training can be completed when the evaluation score S is greater than the threshold T, otherwise, the fine adjustment data set is supplemented to continue training.
4. The method for generating an application program based on a large language model according to claim 3, wherein said step S3 comprises the following operations:
S3.1, acquiring a user system function requirement I;
S3.2, the user system function requirement I is transmitted into a structural design model, the structural design model generates a form list M consisting of at least one form name according to the user system function requirement I by virtue of the semantic understanding capability of the structural design model, and the output format is required to meet the form generation specification;
S3.3, inputting a form list M and a user system function requirement I into a form design model, wherein the form design model respectively outputs a design result T= [ T 1,t2,...,tn ] of each form, and T n represents the design result of each form;
S3.4, the form list M and the design result T of each form are final output of the system design module, and the form list M and the design result T of each form are summarized and input into a low code generator.
5. The method for generating an application program based on a large language model according to claim 1, wherein the step S4 is performed to convert data conforming to the generation specification into a real form and a component by using a low code generator, thereby generating a usable application program, and specifically comprises:
Converting the data conforming to the form generation specification into an actual application form list;
Converting the data conforming to the component generation specification into actual components of each form, wherein the specific conversion rules of the components are as follows: the type value is input converted into a single line text component, the type value is select converted into a drop-down box selector component, the type value is textarea converted into a multiple line text component, the type value is checkbox converted into a check box component, the type value is fileupload converted into a file upload component, and the type value is number converted into a digital component.
6. An application generation device based on a large language model, comprising:
the specification definition module is used for defining a form generation specification and a component generation specification;
the model training module is used for carrying out fine adjustment on the large language model by utilizing the fine adjustment data set which accords with the specifications, so as to obtain a structural design model with structural design capability and a form design model with form design capability;
The system design module comprises a structural design model and a form design model, and is used for carrying out semantic analysis on the system function requirement description of the user and converting the requirement description into a data structure conforming to the generation specification;
The system generation module is used as a low-code generator and is used for converting data conforming to the generation specification into a real form and a component so as to generate a usable application program.
7. The large language model based application program generating device according to claim 6, wherein the form generating specification defined by the specification definition module is a json data structure, and comprises a name field and a description field, wherein the name field is a form name, and the description field is a form function description;
The component generation specification defined by the specification definition module is a json data structure and comprises a name field and a type field, wherein the name field is defined as a component name, and the type field is defined as a component type; the value of type belongs to one of input, number, textarea, checkbox, fileupload, select, where input means a single line of text, number means a number selector, textarea means a multiple line of text, checkbox means a check box, fileupload means a file upload, and select means a drop down selection box.
8. The large language model based application generation device of claim 7, wherein the model training module comprises three parts, namely a fine tuning data set, a model trainer and a model evaluator, wherein:
(1) The fine tuning data sets F= { D and T } comprise a structural design model fine tuning data set D= { D 1,d2,d3...dn } and form design model fine tuning data sets T= { T 1,t2,t3...tn},dn and T n which are single pieces of fine tuning data, the structural design model is obtained after the large language model is subjected to fine tuning training of the structural design model fine tuning data set D, and the form design model is obtained after the form design model fine tuning data set T is subjected to fine tuning; the fine tuning data in the fine tuning data set consists of a system demand description text and a labeling label, wherein the demand description text is a real system design demand text, the labeling label is a real design result of an application program corresponding to demand description, and the labeling label of the structural design model fine tuning data set D is required to meet a form generation specification;
(2) The model training device is used for updating the model structure by utilizing the fine adjustment data set, so that the model has the system design capability, and the training process is as follows:
(2.1) preparing a candidate prompt word set P= { P 1,p2,p3...pk }, enabling labeling labels of the candidate prompt word set P to conform to component generation specifications, setting a requirement description text of fine adjustment data as x n, and enabling an original weight of a pre-training model to be W;
(2.2) polling the candidate prompt word set P, and splicing vectors converted from tokens converted from P n through encodings and embedding converted from x n, wherein the vectors are used as input of model training;
(2.3) updating only tokens of p n per round of training until training loss convergence no longer drops;
(2.4) initializing two low rank matrices B and a;
(2.5) training the updated model weight through the fine-tuning data set, specifically freezing the original weight W of the pre-trained model, and updating only the weights of the low-rank matrixes B and A, wherein the updated value delta W=BA of the trained model weight of each round;
(2.6) changing the model weight after training is completed to be:
W` = W + ΔW；
(3) The model evaluator is used for evaluating the trained model, and evaluating the model design capacity by comparing the design result and the real label of the model, and the evaluation score n of a single sample has the following calculation formula:
formula (a),
The formula (b),
Formula (c),
Wherein,Representing the generation of the correct component,/>Representing the component that generated the error,/>Representing components which are in the labeling data but are not actually generated by the model;
The evaluation score S of the final model was calculated as follows:
The formula (d),
After training is completed, the model after training is evaluated by a model evaluator, a completion threshold T is set according to the requirement, and training can be completed when the evaluation score S is greater than the threshold T, otherwise, the fine adjustment data set is supplemented to continue training.
9. The large language model based application generation apparatus of claim 8, wherein the system design module converts the requirement description into a data structure conforming to the generation specification by:
(1) Collecting a user system function requirement I;
(2) The user system function requirement I is transmitted into a structural design model, the structural design model generates a form list M composed of at least one form name according to the user system function requirement I by means of semantic understanding capability of the structural design model, and an output format is required to meet form generation specifications;
(3) Inputting a form list M and a user system function requirement I into a form design model, wherein the form design model respectively outputs a design result T= [ T 1,t2,...,tn ] of each form, and T n represents the design result of each form;
(4) The form list M and the design result T of each form are the final output of the system design module, and the form list M and the design result T of each form are summarized and input to the system generation module.
10. The large language model based application program generating apparatus as claimed in claim 7, wherein the specific steps of the system generating module generating the usable application program are as follows:
Converting the data conforming to the form generation specification into an actual application form list;
Converting the data conforming to the component generation specification into actual components of each form, wherein the specific conversion rules of the components are as follows: the type value is input converted into a single line text component, the type value is select converted into a drop-down box selector component, the type value is textarea converted into a multiple line text component, the type value is checkbox converted into a check box component, the type value is fileupload converted into a file upload component, and the type value is number converted into a digital component.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410458660.8A CN118092908A (en) | 2024-04-17 | 2024-04-17 | Application program generation method and device based on large language model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410458660.8A CN118092908A (en) | 2024-04-17 | 2024-04-17 | Application program generation method and device based on large language model |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118092908A true CN118092908A (en) | 2024-05-28 |
Family
ID=91160006
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410458660.8A Pending CN118092908A (en) | 2024-04-17 | 2024-04-17 | Application program generation method and device based on large language model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118092908A (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116149631A (en) * | 2023-01-05 | 2023-05-23 | 三峡高科信息技术有限责任公司 | Method for generating Web intelligent form based on natural language |
| CN116860221A (en) * | 2023-07-28 | 2023-10-10 | 金现代信息产业股份有限公司 | Intelligent recommendation method and system for low-code data model |
| CN117215551A (en) * | 2023-08-30 | 2023-12-12 | 慧之安信息技术股份有限公司 | LLM-based low-code application development method and system |
| CN117272959A (en) * | 2023-09-07 | 2023-12-22 | 浪潮软件股份有限公司 | Method and system for generating form low codes based on BERT model |
| CN117331561A (en) * | 2023-11-28 | 2024-01-02 | 之江实验室 | Intelligent low-code page development system and method |
| CN117492736A (en) * | 2023-10-31 | 2024-02-02 | 慧之安信息技术股份有限公司 | Low-code platform construction method and system based on large model |
| CN117637175A (en) * | 2023-11-27 | 2024-03-01 | 上海交通大学医学院附属瑞金医院 | Large model medical training data generation method and system based on multistage semantics |
| US20240111498A1 (en) * | 2023-12-12 | 2024-04-04 | Intel Corporation | Apparatus, Device, Method and Computer Program for Generating Code using an LLM |
| CN117875288A (en) * | 2024-01-12 | 2024-04-12 | 泛微网络科技股份有限公司 | Form page generation method and device and electronic equipment |
-
2024
- 2024-04-17 CN CN202410458660.8A patent/CN118092908A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116149631A (en) * | 2023-01-05 | 2023-05-23 | 三峡高科信息技术有限责任公司 | Method for generating Web intelligent form based on natural language |
| CN116860221A (en) * | 2023-07-28 | 2023-10-10 | 金现代信息产业股份有限公司 | Intelligent recommendation method and system for low-code data model |
| CN117215551A (en) * | 2023-08-30 | 2023-12-12 | 慧之安信息技术股份有限公司 | LLM-based low-code application development method and system |
| CN117272959A (en) * | 2023-09-07 | 2023-12-22 | 浪潮软件股份有限公司 | Method and system for generating form low codes based on BERT model |
| CN117492736A (en) * | 2023-10-31 | 2024-02-02 | 慧之安信息技术股份有限公司 | Low-code platform construction method and system based on large model |
| CN117637175A (en) * | 2023-11-27 | 2024-03-01 | 上海交通大学医学院附属瑞金医院 | Large model medical training data generation method and system based on multistage semantics |
| CN117331561A (en) * | 2023-11-28 | 2024-01-02 | 之江实验室 | Intelligent low-code page development system and method |
| US20240111498A1 (en) * | 2023-12-12 | 2024-04-04 | Intel Corporation | Apparatus, Device, Method and Computer Program for Generating Code using an LLM |
| CN117875288A (en) * | 2024-01-12 | 2024-04-12 | 泛微网络科技股份有限公司 | Form page generation method and device and electronic equipment |
Non-Patent Citations (1)
| Title |
|---|
| 冯辉宗;朱澎;蒋建春;: "基于RTW的驱动代码快速生成技术研究", 计算机应用与软件, no. 03, 15 March 2016 (2016-03-15) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111368545A (en) | Named entity identification method and device based on multi-task learning | |
| CN114816497B (en) | Link generation method based on BERT pre-training model | |
| CN113157919B (en) | Sentence text aspect-level emotion classification method and sentence text aspect-level emotion classification system | |
| CN116842964A (en) | Business process generation method and system based on semantic analysis | |
| CN112000770A (en) | Intelligent question and answer oriented sentence-to-sentence matching method based on semantic feature map | |
| CN110866115A (en) | Sequence labeling method, system, computer equipment and computer readable storage medium | |
| CN115062070A (en) | Question and answer based text table data query method | |
| CN117236677A (en) | RPA process mining method and device based on event extraction | |
| CN115423118A (en) | Method, system and device for fine tuning of pre-training language model | |
| CN112100486A (en) | Deep learning recommendation system and method based on graph model | |
| CN116881641A (en) | Pre-training model adjustment method and device, storage medium and computing equipment | |
| CN117634867A (en) | RPA flow automatic construction method and system combining large language model and reinforcement learning | |
| CN117852593A (en) | Compression method for distilling perception mixing precision quantification | |
| CN116244484B (en) | Federal cross-modal retrieval method and system for unbalanced data | |
| CN110705279A (en) | Vocabulary selection method and device and computer readable storage medium | |
| CN117236335A (en) | Two-stage named entity recognition method based on prompt learning | |
| CN118092908A (en) | Application program generation method and device based on large language model | |
| CN116861877A (en) | Template construction method, device, equipment and storage medium based on reinforcement learning | |
| CN116777646A (en) | Artificial intelligence-based risk identification method, apparatus, device and storage medium | |
| CN114997920A (en) | Method for generating advertisement file, device, equipment, medium and product thereof | |
| CN112364662A (en) | Intention identification method based on neural network and electronic device | |
| CN113010687A (en) | Exercise label prediction method and device, storage medium and computer equipment | |
| CN117829149B (en) | Language model hybrid training method and device, electronic equipment and storage medium | |
| CN116932037B (en) | Software system configuration generation method and system based on generation type large model | |
| CN117272979B (en) | Unsupervised sentence representation method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |