CN116841525A

CN116841525A - Computer drawing method, device, electronic equipment and readable storage medium

Info

Publication number: CN116841525A
Application number: CN202310927280.XA
Authority: CN
Inventors: 隗刚; 孙士欣; 孙敏杰; 郑旺旺; 刘海平; 张泽众
Original assignee: Beijing Daoheng Software Co ltd
Current assignee: Beijing Daoheng Software Co ltd
Priority date: 2023-07-26
Filing date: 2023-07-26
Publication date: 2023-10-03

Abstract

The application discloses a computer drawing method, a computer drawing device, an electronic device and a readable storage medium. The method comprises the following steps: the method comprises the steps of obtaining a natural language instruction indicating drawing design, inputting the natural language instruction into a pre-trained large language model LLM, obtaining a script language instruction output by the LLM, wherein the script language instruction is a script language instruction of a drawing application program, and training data of the LLM comprises: the script language instruction with the corresponding relation and the natural language instruction indicating the drawing design are analyzed through the script engine, and the graphic engine of the drawing application program is called to draw according to the analysis result. The application converts the natural language instruction into the script language instruction by using the LLM, is convenient for the script analysis engine to call the graphic engine to draw according to the script language instruction, does not need a user to grasp a large number of drawing commands and shortcut keys, can directly draw through the natural language, and effectively improves the intelligent degree of CAD software.

Description

Computer drawing method, device, electronic equipment and readable storage medium

Technical Field

The present application relates to the field of computers, and more particularly, to a computer drawing method, apparatus, electronic device, and readable storage medium.

Background

CAD (Computer Aided Design ) is a design task that uses a computer and its graphics devices to assist users in performing the design task. In the CAD software, a user can draw a drawing through interaction modes such as a keyboard and a mouse, but in the existing interaction modes, the user needs to master a large number of drawing commands and shortcut keys to quickly finish the drawing, and even if the function of AI aided design is provided in the existing CAD software, the drawing is only aided and suggested mainly, and the drawing of the user is not directly assisted. Therefore, the current CAD software is not highly intelligent.

Disclosure of Invention

In view of the above, the present application provides a computer drawing method, apparatus, electronic device and readable storage medium for solving the problem of low intelligentization degree of CAD software.

In order to achieve the above object, the following solutions have been proposed:

a computer mapping method, the method comprising:

acquiring a natural language instruction indicating drawing design;

inputting the natural language instruction into a pre-trained large language model LLM, obtaining a script language instruction output by the LLM, wherein the script language instruction is a script language instruction of a drawing application program, and training data of the LLM comprises: script language instructions with corresponding relations and natural language instructions indicating drawing designs;

and analyzing the script language instruction through a script engine, and calling a graphic engine of the drawing application program to draw according to an analysis result.

Optionally, the parsing, by the script engine, the scripting language instruction, and calling, according to a result of the parsing, a graphics engine of the drawing application program to draw, including:

the script engine divides a target part from the script language instruction through lexical analysis;

the script engine converts the script language instruction of the target part into a target byte instruction;

and the script engine calls a target call interface of the graphic engine corresponding to the target byte instruction through a virtual machine to draw.

Optionally, before the inputting the natural language instruction into the pre-trained large language model LLM, the method further comprises:

if the natural language instruction is a natural language instruction in a voice format, converting the natural language instruction in the voice format into a natural language instruction in a text format through a voice conversion technology.

Optionally, the training process of the LLM includes:

acquiring training data, wherein the training data comprises script language instructions with corresponding relations and natural language instructions for indicating drawing design;

and performing fine tuning training on the LLM by using the training data.

Optionally, the performing fine tuning training on the LLM using the training data includes:

setting a training environment of the LLM and acquiring initial parameters of the LLM;

under the training environment, updating the initial parameters by using the training data to obtain training parameters conforming to the training data;

and determining the training parameters as current parameters of the LLM.

A computer graphics apparatus, the apparatus comprising:

the acquisition unit is used for acquiring a natural language instruction indicating drawing design;

the conversion unit is used for inputting the natural language instruction into a pre-trained large language model LLM to obtain a script language instruction output by the LLM, wherein the script language instruction is a script language instruction of a drawing application program, and training data of the LLM comprises: script language instructions with corresponding relations and natural language instructions indicating drawing designs;

and the drawing unit is used for analyzing the script language instruction, and calling a graphic engine of the drawing application program to draw according to the analysis result.

Optionally, the drawing unit includes:

a dividing subunit for dividing a target portion from the scripting language instruction by lexical analysis;

an instruction conversion subunit, configured to convert the scripting language instruction of the target portion into a target byte instruction;

and the interface calling subunit is used for calling a target calling interface of the graphic engine corresponding to the target byte instruction through the virtual machine to draw.

Optionally, before the conversion unit inputs the natural language instruction into the pre-trained large language model LLM, the apparatus further comprises:

if the natural language instruction is a natural language instruction in a voice format, triggering a text unit;

the text unit is used for converting the natural language instruction in the voice format into the natural language instruction in the text format.

An electronic device includes a memory and a processor;

the memory is used for storing programs;

the processor is configured to execute the program to implement each step of any one of the computer graphics methods described above.

A readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of any of the computer mapping methods described above.

The application provides a computer drawing method, a computer drawing device, electronic equipment and a readable storage medium. According to the method, the artificial intelligent large language model LLM is used for converting the natural language instruction into the script language instruction, so that the script analysis engine can conveniently call the graphic engine to draw according to the script language instruction, a user does not need to grasp a large number of drawing commands and shortcut keys, drawing can be directly carried out through the natural language, and the intelligent degree of CAD software is effectively improved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present application, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic flow chart of a computer drawing method according to an embodiment of the application;

FIG. 2 is a flowchart of another computer graphics method according to an embodiment of the present application;

fig. 3 is a schematic structural diagram of an image interaction interface according to an embodiment of the present application;

fig. 4 is a schematic structural diagram of a computer drawing device according to an embodiment of the present application;

fig. 5 is a block diagram of a hardware structure of an electronic device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

As shown in fig. 1, an embodiment of the present application provides a computer mapping method, which may include:

s10, acquiring a natural language instruction indicating drawing design.

The natural language can be a language form of daily communication and ideation of human beings, and can be a spoken language or a written language. The drawing design can refer to drawing content in the computer aided design, and optionally, the drawing design can be that 'please place the power transmission tower pile at intervals of two kilometers', and the span distance is increased by one and five times when meeting a river in the middle. Naturally, natural language instructions may not require a standard format and the drawing design may be freely described according to the language habits of the user. The embodiment can speak the drawing design through the voice of the user or input the drawing design in a text window by the user, thereby obtaining a natural language instruction indicating the drawing design.

S11, inputting natural language instructions into a pre-trained large language model LLM to obtain script language instructions output by the LLM, wherein the script language instructions are script language instructions of a drawing application program, and training data of the LLM comprise: script language instructions with correspondence and natural language instructions indicating a drawing design.

LLM is an artificial intelligence model, among other things, aimed at understanding and generating human language. LLM can perform a wide range of tasks including text summarization, translation, emotion analysis, and the like. The core idea is to learn the mode and language structure of natural language through large-scale unsupervised training, which can simulate the human language cognition and generation process to a certain extent. After training and fine-tuning the LLM through proprietary data, the LLM can be enhanced for generating proprietary instructions so as to serve a specific task designed in advance. In this embodiment, the script language instruction with a corresponding relationship and the natural language instruction indicating the drawing design may be used to perform fine tuning training on the LLM, so that the input of the LLM is a natural language instruction and the output is a script language instruction. The scripting language instructions may be those of a drawing application and the scripting engine may parse, for example, the natural language instructions are "draw a 100 cm long red line", then the scripting language instructions are "Color rgb (255, 0); line (200 ), (280, 260) ".

S12, analyzing the script language instruction through the script engine, and calling a graphic engine of the drawing application program to draw according to the analysis result.

The script engine can be a software module which is programmed and developed according to actual needs. The script engine in this embodiment may be dedicated to a business scenario associated with a computer-aided design, and may include many instructions associated with the computer-aided design, such as: drawing lines, drawing dots, printing, setting color styles, and the like. In this embodiment, the script engine may parse the script language instruction into the basic machine language, that is, a combination of 0 and 1, where different combinations of 0 and 1 correspond to different drawing instructions, for example 0011 is a drawing line, and 0101 is a drawing circle. According to the embodiment, the functional interfaces of the corresponding graphic engines can be called through the basic machine language analyzed by the script engine, and different functional interfaces can correspond to different drawing modes.

The embodiment of the application provides a computer drawing method, which uses an artificial intelligent large language model LLM to convert natural language instructions into script language instructions, is convenient for a script analysis engine to call a graphic engine to draw according to the script language instructions, does not need a user to grasp a large number of drawing commands and shortcut keys, can directly draw through natural language, and effectively improves the intelligent degree of CAD software.

In another computer graphics method provided according to an embodiment of the present application, step S12 shown in fig. 1 may include steps one to three:

step one: the script engine divides a target part from the script language instruction through lexical analysis;

step two: the script engine converts the script language instruction of the target part into a target byte instruction;

step three: the script engine calls a target call interface of the graphic engine corresponding to the target byte instruction through the virtual machine to draw.

The embodiment can divide the script language instruction through lexical analysis. The lexical analysis may scan the characters from left to right, recognize individual words and determine the type of word. After dividing the scripting language instruction, the embodiment can convert the scripting language instruction of each part into the basic machine language by syntax analysis, namely, the combination of 0 and 1, and of course, the combination of 0 and 1 needs to meet the instruction set. The virtual machine may be a complete computer system running in a completely isolated environment with complete hardware system functionality emulated by software. After the basic machine language of each part is obtained, the interface of the graphic engine can be called through the virtual machine, specifically, the virtual machine can call the specific functional interface of the graphic engine corresponding to the basic machine language according to the basic machine language of each part, so that drawing can be conveniently performed in the interactive image interface displayed to the user.

As shown in fig. 2, in another computer graphics method provided according to an embodiment of the present application, before the natural language instruction is input into the pre-trained large language model LLM in step S11 shown in fig. 1, the method may further include step S13:

step S13, if the natural language instruction is a natural language instruction in a voice format, converting the natural language instruction in the voice format into a natural language instruction in a text format through a voice conversion technology.

Because the input and output of the LLM are text, the embodiment can convert the natural language instruction in the voice format into the natural language instruction in the text format by the voice conversion technology for the natural language instruction in the voice format, thereby being convenient for the LLM to recognize. In addition, the natural language instruction in the text format can be input through the added text input window and drawn in the image interaction interface. As shown in FIG. 3, the region A is a natural language instruction output window, the region B is a drawing region, and the region C is a text input window or a voice input window. The user may manually input a drawing design or a voice input drawing design in the C area, display a natural language instruction in a final text format in the a area, and draw an image conforming to the natural language instruction in the B area. As shown in fig. 3, if the natural language command in the area a is "draw a black line", a black line is drawn in the area B.

In another computer graphics method provided by the embodiment of the present application, the training process of LLM includes steps four and five:

step four: acquiring training data, wherein the training data comprises script language instructions with corresponding relations and natural language instructions for indicating drawing design;

step five: fine tuning training is performed on the LLM using the training data.

In order to enable the LLM to serve instruction conversion in the present embodiment, the present embodiment may perform fine-tuning training on the LLM by using a script language instruction including a correspondence relationship and a natural language instruction indicating a drawing design. Fine tuning is a pointer to "Fine-tuning" the pre-trained LLM large language model. Fine tuning is a technique used in natural language processing to adapt a pre-trained language model to a particular task or domain. The basic idea of fine tuning may be to employ a pre-trained language model that has been trained on a large number of texts, and then to continue training it on a small scale of task-specific texts.

In another computer graphics method provided by an embodiment of the present application, the fifth step may include steps six to eight:

step six: setting training environment of LLM and obtaining initial parameters of LLM;

step seven: under a training environment, updating initial parameters by using training data to obtain training parameters conforming to the training data;

step eight: the training parameters are determined as current parameters of the LLM.

Wherein, the LLM may be considered as a model with multiple layers, and the embodiment may update parameters of a later layer of the initial LLM with training data, so that the parameters of the later layer conform to the training data. Specifically, initial parameters of a later layer in the LLM are obtained, training is carried out on the LLM by using training data, after each training, the parameters are reversely updated through a loss function, and finally training parameters conforming to the training data are obtained.

Corresponding to the computer drawing method provided by the embodiment of the application, the embodiment of the application also provides a computer drawing device.

As shown in fig. 4, an embodiment of the present application further provides a computer drawing device, which may include:

an acquisition unit 100 for acquiring a natural language instruction indicating a drawing design;

the conversion unit 110 is configured to input a natural language instruction into a pre-trained large language model LLM, obtain a scripting language instruction output by the LLM, where the scripting language instruction is a scripting language instruction of a drawing application program, and training data of the LLM includes: script language instructions with corresponding relations and natural language instructions indicating drawing designs;

the drawing unit 120 is configured to parse the scripting language instruction, and call a graphics engine of the drawing application program to draw according to the parsing result.

In another embodiment of the present application, the drawing unit 120 may include:

a dividing subunit for dividing the target portion from the scripting language instruction by lexical analysis;

In another computer graphics apparatus provided according to an embodiment of the present application, before the conversion unit 110 inputs the natural language instruction into the pre-trained large language model LLM, the apparatus may further include:

and the text unit is used for converting the natural language instruction in the voice format into the natural language instruction in the text format.

In another computer graphics apparatus provided according to an embodiment of the present application, the training unit of the LLM in the conversion unit 110 may include:

the training data acquisition subunit is used for acquiring training data, wherein the training data comprises script language instructions with corresponding relations and natural language instructions for indicating drawing design;

and the fine tuning training subunit is used for carrying out fine tuning training on the LLM by using the training data.

In another embodiment of the present application, the fine tuning training subunit may include:

the parameter setting subunit is used for setting the training environment of the LLM and acquiring initial parameters of the LLM;

the parameter updating subunit is used for updating the initial parameters by using the training data under the training environment to obtain training parameters conforming to the training data;

and the parameter determining subunit is used for determining the training parameter as the current parameter of the LLM.

As shown in fig. 5, an embodiment of the present application provides an electronic device 70 comprising at least one processor 701, and at least one memory 702 and bus 703 connected to the processor 701; wherein, the processor 701 and the memory 702 complete communication with each other through the bus 703; the processor 701 is configured to call program instructions in the memory 702 to perform the computer graphics method described above. The electronic device 70 herein may be a server, PC, PAD, cell phone, etc.

The embodiment of the application also provides a readable storage medium, on which a computer program is stored, which when executed by a processor, implements the steps of any of the computer mapping methods described above.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In one typical configuration, the device includes one or more processors (CPUs), memory, and a bus. The device may also include input/output interfaces, network interfaces, and the like.

The memory may include volatile memory, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM), in a computer readable storage medium, the memory including at least one memory chip. Memory is an example of a computer-readable medium.

Computer-readable storage media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises an element.

In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for the device embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments in part.

The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and variations of the present application will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the application are to be included in the scope of the claims of the present application.

Claims

1. A computer mapping method, the method comprising:

acquiring a natural language instruction indicating drawing design;

2. The method according to claim 1, wherein the parsing the scripting language instructions by the scripting engine, and calling a graphics engine of the drawing application to draw according to the parsing result, comprises:

3. The method of claim 1, wherein prior to said inputting the natural language instructions into the pre-trained large language model LLM, the method further comprises:

4. The method of claim 1, wherein the training process of LLM comprises:

and performing fine tuning training on the LLM by using the training data.

5. The method of claim 4, wherein the fine-tuning training the LLM using the training data comprises:

and determining the training parameters as current parameters of the LLM.

6. A computer graphics apparatus, the apparatus comprising:

7. The apparatus of claim 6, wherein the drawing unit comprises:

8. The apparatus of claim 6, wherein before the conversion unit inputs the natural language instructions into a pre-trained large language model LLM, the apparatus further comprises:

9. An electronic device comprising a memory and a processor;

the memory is used for storing programs;

the processor for executing the program to implement the steps of the computer mapping method as claimed in any one of claims 1-5.

10. A readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the steps of the computer mapping method according to any of claims 1-5.