CN116912413A - Model generation method, model generation device, computer device, and storage medium - Google Patents
Model generation method, model generation device, computer device, and storage medium Download PDFInfo
- Publication number
- CN116912413A CN116912413A CN202310877924.9A CN202310877924A CN116912413A CN 116912413 A CN116912413 A CN 116912413A CN 202310877924 A CN202310877924 A CN 202310877924A CN 116912413 A CN116912413 A CN 116912413A
- Authority
- CN
- China
- Prior art keywords
- model
- scene
- information
- description information
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 59
- 238000003860 storage Methods 0.000 title claims abstract description 15
- 238000010276 construction Methods 0.000 claims abstract description 10
- 238000009877 rendering Methods 0.000 claims description 20
- 238000004590 computer program Methods 0.000 claims description 13
- 238000013507 mapping Methods 0.000 claims description 11
- 238000004378 air conditioning Methods 0.000 claims 1
- 238000011161 development Methods 0.000 abstract description 25
- 238000005516 engineering process Methods 0.000 description 19
- 238000004519 manufacturing process Methods 0.000 description 15
- 238000013473 artificial intelligence Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 7
- 230000007613 environmental effect Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 230000000007 visual effect Effects 0.000 description 5
- ATJFFYVFTNAWJD-UHFFFAOYSA-N Tin Chemical compound [Sn] ATJFFYVFTNAWJD-UHFFFAOYSA-N 0.000 description 4
- 238000005265 energy consumption Methods 0.000 description 4
- 239000000463 material Substances 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 229910000679 solder Inorganic materials 0.000 description 4
- 238000013519 translation Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000007639 printing Methods 0.000 description 3
- 238000005476 soldering Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000010147 laser engraving Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000012800 visualization Methods 0.000 description 2
- 239000003086 colorant Substances 0.000 description 1
- 238000004040 coloring Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003032 molecular docking Methods 0.000 description 1
- 230000037361 pathway Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 229910052705 radium Inorganic materials 0.000 description 1
- HCWPIIXVSYCSAN-UHFFFAOYSA-N radium atom Chemical compound [Ra] HCWPIIXVSYCSAN-UHFFFAOYSA-N 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 239000004984 smart glass Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/30—Creation or generation of source code
- G06F8/34—Graphical or visual programming
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/005—General purpose rendering architectures
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Graphics (AREA)
- General Engineering & Computer Science (AREA)
- Geometry (AREA)
- Processing Or Creating Images (AREA)
Abstract
The application discloses a model generation method, a model generation device, a computer device and a nonvolatile computer readable storage medium. The method comprises the steps of obtaining scene description information, wherein the scene description information at least comprises environment information, equipment information in a scene and layout information; obtaining a target model file corresponding to the scene description information in a preset model library, wherein the model library comprises a plurality of model files, and the model files are generated according to physical entities existing in a preset scene; and analyzing the scene description information based on a preset large language model, and constructing a three-dimensional scene model by combining the target model file. The method can realize the universality of the 3D model, can quickly realize the construction of the three-dimensional scene by analyzing the scene description by using the large language model, is time-saving and labor-saving, and has lower requirements on the development personnel due to the fact that personnel only need to carry out the scene description, and development cost is reduced.
Description
Technical Field
The present application relates to the field of three-dimensional modeling technology, and more particularly, to a model generating method, a model generating apparatus, a computer device, and a non-volatile computer-readable storage medium.
Background
Under the continuous development of new-generation industries and information technologies such as cloud computing, big data, internet of things, artificial intelligence (Artificial Intelligence, AI) and the like, digital transformation becomes an effective way for promoting innovation processes in the main countries of the world. Industrial digital twinning is an important technology and provides powerful support for realizing industrial digital transformation. The industrial digital twin technology fully utilizes data such as a physical model, sensor updating, operation history and the like, integrates simulation processes of multiple disciplines, multiple physical quantities and multiple scales, and completes mapping in a virtual space, thereby reflecting the full life cycle process of corresponding entity equipment. The development of industrial digital twin three-dimensional (3D) scenes is a key task for the visualization part in industrial digital twin applications.
However, when 3D scene development is performed at present, for different projects, professionals are required to match the projects to perform development of the 3D model, visualization is realized through programming, the developed 3D scene is generally only suitable for the corresponding projects, and under the condition that a plurality of projects exist simultaneously, the development needs to be performed respectively, so that the 3D scene development is time-consuming and labor-consuming, and has higher requirements on the developers, and development cost is increased.
Disclosure of Invention
The embodiment of the application provides a model generation method, a model generation device, computer equipment and a nonvolatile computer readable storage medium, which can realize the universality of a 3D model, and can quickly realize the construction of a three-dimensional scene by analyzing scene description by using a large language model, so that time and labor are saved, personnel only need to perform scene description, the requirements on the development personnel are low, and the development cost is reduced.
The model generation method of the embodiment of the application comprises the steps of obtaining scene description information, wherein the scene description information at least comprises environment information, equipment information in a scene and layout information; obtaining a target model file corresponding to the scene description information in a preset model library, wherein the model library comprises a plurality of model files, and the model files are generated according to physical entities existing in a preset scene; and analyzing the scene description information based on a preset large language model, and constructing a three-dimensional scene model by combining the target model file.
In some embodiments, the obtaining the object model file corresponding to the scene description information in the preset model library includes: acquiring a first physical entity in the environment information and a second physical entity in the equipment information, wherein the target model file comprises a first target model file corresponding to the first physical entity and a second target model file corresponding to the second physical entity; and acquiring the first target model file and the second target model file.
In some embodiments, the model file is stored in a preset database, the address of the model file and the name of the corresponding physical entity are stored in a preset mapping table, and the obtaining the first target model file and the second target model file includes: and searching a first target address corresponding to the first physical entity and a second target address corresponding to the second physical entity in the mapping table.
In some embodiments, the analyzing the scene description information based on the preset large language model and constructing a three-dimensional scene model in combination with the target model file includes: analyzing the scene description information based on the large language model, and generating a renderable scene code by combining the target model file; rendering is carried out according to the scene codes so as to generate the three-dimensional scene model.
In some embodiments, the scene code includes azimuth information of each model object, color information of each model object, and an address of the target model file corresponding to each model object, and the rendering according to the scene code to generate the three-dimensional scene model includes: and loading the target model file from the address of the target model file corresponding to each model object through a browser, and rendering each model object according to the azimuth information and the color information of each model object so as to generate and display the three-dimensional scene model.
In some embodiments, the model generation method further comprises: displaying an input box and acquiring feedback information input by the input box; analyzing the feedback information based on a preset large language model, and adjusting the scene code according to the feedback information; or, according to the feedback information, adjusting the scene description information; and analyzing the adjusted scene description information based on a preset large language model to regenerate the scene code.
In some embodiments, the scene code is generated based on a codebase of a preset three-dimensional drawing frame.
In some embodiments, the acquiring scene description information includes: displaying a description information input interface, wherein the input interface comprises a plurality of input boxes, and the input boxes respectively correspond to the input of the environment information, the input of the equipment information and the input of the layout information; and acquiring input information of a plurality of input boxes to generate the scene description information.
In some embodiments, the model generation method further comprises: associating each model object in the three-dimensional scene model with corresponding equipment in the scene; and controlling the display content of the model object corresponding to the equipment according to the operation information of the equipment.
The model generating device comprises a first acquisition module, a second acquisition module and a construction module. The first acquisition module is used for acquiring scene description information, and the scene description information at least comprises environment information, equipment information in a scene and layout information; the second obtaining module is used for obtaining a target model file corresponding to the scene description information in a preset model library, the model library comprises a plurality of model files, and the model files are generated according to physical entities existing in a preset scene; the construction module is used for analyzing the scene description information based on a preset large language model and constructing a three-dimensional scene model by combining the target model file.
The computer device according to an embodiment of the present application includes a processor, a memory, and a computer program, wherein the computer program is stored in the memory and executed by the processor, and the computer program includes instructions for executing the model generating method according to any one of the above embodiments.
The non-transitory computer readable storage medium of the embodiment of the present application includes a computer program that, when executed by a processor, causes the processor to execute the model generation method of any of the above embodiments.
The model generating method, the model generating device, the computer equipment and the computer readable storage medium of the embodiment of the application acquire scene description information describing a scene, wherein the scene description information comprises environment information (such as wall related information, ceiling related information and the like), equipment information in the scene (such as related information of equipment (such as electric appliances and cameras and the like) in the scene) and layout information (such as the layout of the wall, the equipment and the like).
The environment information and the equipment information both comprise physical entities (such as walls, equipment and the like) needing to establish a 3D model, and it can be understood that for a preset scene (such as a factory scene of a specific industry), after the factory is established, the environment information is determined, and the equipment in the preset scene is basically determined, so that the 3D model of each physical entity in the preset scene can be established in advance, the 3D model can form a model file, and a plurality of model files form a model library; and after the scene description information is acquired, the target model file corresponding to the physical entity described in the scene description information can be quickly acquired from the model library.
After scene description information and corresponding target model files are obtained, the scene model is analyzed based on a preset large language model, so that information such as physical entities required by scene construction, how to perform layout among the physical entities and the like can be obtained, and the three-dimensional scene model can be quickly constructed by combining the target model files, so that time and labor are saved, personnel only need to perform scene description, requirements on developing personnel are low, and development cost is reduced.
Additional aspects and advantages of embodiments of the application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of embodiments of the application.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
FIG. 1 is a schematic illustration of an application scenario of a model generation method of some embodiments of the present application;
FIG. 2 is a flow diagram of a model generation method of some embodiments of the application;
FIG. 3 is a schematic view of a scenario of a model generation method of some embodiments of the present application;
FIG. 4 is a flow diagram of a model generation method of some embodiments of the application;
FIG. 5 is a flow diagram of a model generation method of some embodiments of the application;
FIG. 6 is a schematic view of a scenario of a model generation method of some embodiments of the present application;
FIG. 7 is a block diagram of a model generation apparatus according to some embodiments of the present application;
FIG. 8 is a schematic structural diagram of a computer device in accordance with certain embodiments of the present application;
FIG. 9 is a schematic diagram of the connection state of a non-transitory computer readable storage medium and a processor of some embodiments of the application.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are exemplary only for explaining the embodiments of the present application and are not to be construed as limiting the embodiments of the present application.
To facilitate an understanding of the application, the following terms used in connection with the application will be explained:
1. artificial intelligence (Artificial Intelligence, AI), is a theory, method, technique, and application system that simulates, extends, and extends human intelligence using a digital computer or a machine controlled by a digital computer, senses the environment, obtains knowledge, and uses the knowledge to obtain optimal results. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence, i.e. research on design principles and implementation methods of various intelligent machines, enables the machines to have functions of sensing, reasoning and decision. The artificial intelligence technology is a comprehensive subject, and relates to the technology with wide fields, namely the technology with a hardware level and the technology with a software level. Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions. The technical scheme provided by the embodiment of the application mainly relates to natural language processing technology in artificial intelligence, machine learning/deep learning and the like.
2. A large language model (Large Language Model, LLM), which refers to a deep learning model trained using large amounts of text data, can generate natural language text or understand the meaning of language text. The large language model can process various natural language tasks such as text parsing, classification, question-answering, dialogue, etc., or generate text from parsed contents, etc., and particularly shows good performance in the field of code generation. LLM is an important pathway to artificial intelligence.
3. JS (JavaScript) programming language is a lightweight, interpreted or just-in-time programming language with functional preference. Although it is a scripting language that is famous for developing Web pages, it is also used in many non-browser environments, javaScript is based on a prototype-programmed, multi-paradigm dynamic scripting language, and supports object-oriented, command-style, declarative, functional programming paradigms. Through JS programming, 3D scene development can be realized.
The current industrial digital twin 3D scene development mainly comprises two steps: firstly, the development and modeling of the 3D model, and secondly, the selection of a visual computing engine and the loading rendering of the model.
The development and modeling of a 3D model mainly completes the geometric modeling of a physical entity related to a 3D scene, and is generally manufactured by 3D model design software such as 3dmax, blender and the like, and model files are typically generated, such as: common model file formats such as obj, mtl, etc.;
For the selection and model loading of visual computing engines, there are currently two techniques: (1) Visual computing engines based on Server-Client (C/S) technology, such as Unity and Unreal, mainly based on c# to program and realize the design and development of 3D scenes; (2) WebGL computational engines based on Browser/Server (B/S) technology, such as thread. Js, etc. computational frameworks, develop scenes based on manual JS (JavaScript) programming.
However, for each 3D scene development project, the corresponding 3D model file needs to be manufactured first, then programming development is performed manually, and the 3D scene model developed for the specific project has the following drawbacks:
(1) Lack of versatility. The result of the development of the existing 3D scene model is only aimed at a certain project, and the developed 3D scene model cannot be suitable for multiplexing of a plurality of projects;
(2) The scene development cost is high. The requirements of the designer are required to cooperate with the developer, the requirement communication cost is high, the development efficiency is low, and the requirement of rapid delivery of the industrial digital twin scene is difficult to meet;
(3) The scene design difficulty is high. Scene development requires a great deal of knowledge of computer graphics, such as 3D computing geometry, with a high technical threshold for developers.
In order to solve the technical problems, an embodiment of the present application provides a model generating method.
An application scenario of the technical solution of the present application is described first, as shown in fig. 1, which is a schematic application scenario diagram of a model generating method provided by an embodiment of the present application, where the application scenario relates to a terminal device 110 and a server 120, and the terminal device 110 may communicate with the server 120.
Fig. 1 illustrates one terminal device 110 and one server 120, and may actually include other numbers of terminal devices and servers, as embodiments of the application are not limited in this respect.
In some implementations, the server 120 in fig. 1 may be an independent physical server, or may be a server cluster or a distributed system formed by a plurality of physical servers, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms. The embodiments of the present application are not limited in this regard.
In some implementations, the terminal device 110 as shown in fig. 1 may be installed with an application client that, when running in the terminal device 110, may interact with the server 120. The client may specifically include, for example: browser clients, vehicle clients, smart home clients, game clients, multimedia clients (e.g., video clients), social clients, and information-based clients (e.g., news clients).
Optionally, in the embodiment of the present application, the terminal device 110 may be a device with rich man-machine interaction modes, capability of accessing to the internet, typically carrying various operating systems, and relatively strong processing capability. The terminal device 110 may be a smart phone, smart glasses, a handheld terminal, a smart television, a tablet computer, a vehicle-mounted terminal, etc., but is not limited thereto.
In one implementation manner, the server 120 and the terminal device 110 may perform the model generation method provided by the embodiment of the present application in an interactive manner, or the terminal device 110 or the server 120 may perform the model generation method provided by the embodiment of the present application.
The model generation method of the present application will be described in detail below:
referring to fig. 2, an embodiment of the present application provides a model generating method, which includes:
step 011: acquiring scene description information, wherein the scene description information at least comprises environment information, equipment information in a scene and layout information;
in particular, since a large language model can understand text, a specific case of a three-dimensional scene needs to be described before constructing a three-dimensional scene model, thereby forming scene description information.
For a three-dimensional scene, environmental information is generally included, where the environmental information may include physical entities other than electrical appliances in the scene (e.g., non-electrical objects such as walls), the environmental information may include wall information in the environment (e.g., wall shape, size, color, wall surface pattern, etc.), ceilings (ceiling shape, color, pattern, etc.), background colors of the environment as a whole, and the like.
For the digital twin model, the model is required to be used as a mapping of the device in the 3D scene to reflect the full life cycle of the device, such as the real-time running state of the device, can be embodied in the corresponding digital twin model, so the scene description information also includes the device information required to establish the digital twin model, such as a camera, a production line machine and the like in the scene.
For the three-dimensional scene, after the environment information and the device information are determined, how the physical entities and the devices in the environment information are placed is determined, so that layout information is also required in the scene description information, and the layout information includes the placement direction (such as the placement direction and the placement position) of the physical entities in the environment information and the placement direction of the devices.
Optionally, the scene description information may further include other needed information, and if the indication information needs to be displayed, the indication information may be displayed through a virtual display frame, so that an information display area (such as a specific area of a wall or any other area capable of being displayed in a 3D scene) where information needs to be displayed in the environment may be added to the scene description information.
Alternatively, referring to fig. 3, the scene description information may be acquired by displaying a description information input interface T, where the input interface includes an input box K through which the scene description information may be input. Alternatively, the input boxes K may be plural, and the plural input boxes K correspond to the input of the environment information, the input of the device information, and the input of the layout information, respectively; input information of a plurality of input boxes K is acquired to generate scene description information.
Specifically, one or more input boxes for inputting environment information, device information, and layout information are displayed through a visual interface. For example, the visual interface displays 3 input boxes for inputting environment information, device information and layout information, respectively. After the user finishes inputting, the scene description information is obtained.
Step 012: obtaining a target model file corresponding to scene description information in a preset model library, wherein the model library comprises a plurality of model files, and the model files are generated according to physical entities existing in a preset scene;
specifically, after the scene description information is acquired, for the subsequent modeling, a 3D model corresponding to the physical entities in the scene description information needs to be acquired, so that information (such as a wall name and a device name) describing names of the physical entities in the scene description information can be acquired, and then a target model file corresponding to each physical entity is queried in a preset model library, wherein the target model file is used as a basic file of the subsequent modeling.
Alternatively, a first physical entity in the environment information and a second physical entity in the device information may be acquired, and then a first target model file corresponding to the first physical entity and a second target model file corresponding to the second physical entity are acquired in the model library, so that target model files corresponding to the respective physical entities in the scene description information are acquired.
It can be understood that the actual target model file is only required to be acquired when the model is subsequently constructed, so that the address of the target model file is only required to be acquired when the target model file is acquired before the model is constructed, and the address of the target model file is conveniently referenced when the subsequent large language model generates the code text. And then when the 3D rendering is carried out on the code text, loading the target model file through the address of the target model file and rendering. Therefore, before the model is built, only the address of the target model file is acquired, so that the data reading quantity is reduced, and the model generation efficiency is improved.
Optionally, after the 3D model of each physical entity in the preset scene is established, the obtained model file is stored in the database, so as to obtain the address of the model file, and the address of the model file and the name of the corresponding physical entity are stored in the preset mapping table, so that the mapping relation between the address of the model file and the name of the corresponding physical entity is established.
When the address of the target model file is obtained, the address corresponding to the name of each physical entity (such as the first physical entity contained in the environment information and the second physical entity contained in the equipment information) contained in the scene description information is searched in the mapping table, so that the address of the target model file (such as the first target address corresponding to the first physical entity and the second target address corresponding to the second physical entity) can be obtained, and the address of the target model file is the storage address for storing the target model file, and the target model file can be read through the address of the target model file.
It can be understood that, for the preset scene, after the factory is established, the environmental information is determined, and the equipment in the preset scene is basically determined, so that the 3D model of each physical entity in the preset scene can be established in advance, the 3D model can form a model file, and a plurality of model files form a model library. Thus, for a preset scene, no matter how many items exist in the follow-up, the model library can be shared, so that the 3D model is universal.
Step 013: and analyzing scene description information based on a preset large language model, and constructing a three-dimensional scene model by combining the target model file.
Specifically, after the scene description information is acquired, the scene description information can be analyzed by using a large language model to understand the 3D scene, a code text which can be rendered into the 3D scene is output after the scene description information is understood, the code text only needs to record the address of the acquired target model file, and the target model file can be quickly rendered according to the code text to construct the 3D scene model.
Because the large language model can analyze scene description information and automatically generate a renderable code text based on the analyzed content, a developer is not required to manually program to generate the code text, and a designer is not required to communicate with the developer in a demand manner, and only the designer is required to provide scene description files for the large language model, so that communication cost is not required, labor cost of the developer is saved, development efficiency is high, and quick delivery of the 3D scene model can be realized. In addition, because the manual programming of a developer is not needed, the developer is not required to be familiar with a great deal of computer graphics knowledge, such as 3D calculation geometric knowledge, and the technical threshold requirement on the developer is low.
The model generating method of the embodiment of the application obtains scene description information describing a scene, wherein the scene description information comprises environment information (such as wall related information, ceiling related information and the like), equipment information in the scene (such as related information of equipment (such as electric appliances and cameras and the like) in the scene) and layout information (such as the layout of the wall, the equipment and the like).
The environment information and the equipment information both comprise physical entities (such as walls, equipment and the like) needing to establish a 3D model, and it can be understood that for a preset scene (such as a factory scene of a specific industry), after the factory is established, the environment information is determined, and the equipment in the preset scene is basically determined, so that the 3D model of each physical entity in the preset scene can be established in advance, the 3D model can form a model file, and a plurality of model files form a model library; and after the scene description information is acquired, the target model file corresponding to the physical entity described in the scene description information can be quickly acquired from the model library.
After scene description information and corresponding target model files are obtained, the scene model is analyzed based on a preset large language model, so that information such as physical entities required by scene construction, how to perform layout among the physical entities and the like can be obtained, and the three-dimensional scene model can be quickly constructed by combining the target model files, so that time and labor are saved, personnel only need to perform scene description, requirements on developing personnel are low, and development cost is reduced.
Referring to fig. 4, in certain embodiments, step 013: analyzing scene description information based on a preset large language model, and constructing a three-dimensional scene model by combining a target model file, wherein the method comprises the following steps:
Step 0131: analyzing scene description information based on a large language model, and generating a renderable scene code by combining a target model file;
step 0132: rendering is performed according to the scene codes to generate a three-dimensional scene model.
Specifically, the large language model can analyze the scene description information to determine the size, color, orientation, etc. of each physical entity, and in order to realize 3D rendering, programming is required based on the analyzed information to generate a scene code conforming to a preset three-dimensional drawing frame, and finally, a client such as a browser runs the scene code to realize 3D rendering and display a 3D scene.
When generating the scene code, in addition to programming the information of the size, color, azimuth and the like of each physical entity, in order to load the target model file in the subsequent 3D rendering, the address corresponding to the target model file is required to be referenced in programming, so that the scene code is generated. That is, each physical entity has a corresponding model object in the 3D scene, and the scene code includes not only the azimuth information and the color information of each model object, but also the address of the target model file corresponding to the model object.
And when 3D rendering is carried out, loading the target model file from the address of the target model file corresponding to each model object through a browser, and rendering each model object according to the azimuth information and the color information of each model object so as to generate and display a three-dimensional scene model. Therefore, each physical entity in the actual scene is correspondingly presented in the three-dimensional scene model according to the actual arrangement, and the generation of the three-dimensional scene model is realized.
Alternatively, the scene code of the large language model is generated based on a code library of a preset three-dimensional drawing frame. For example, the preset three-dimensional drawing frame is a WebGL frame (i.e. Web Graphics Library), the WebGL frame is a 3D drawing protocol, and WebGL can provide hardware 3D accelerated rendering for HTML5 Canvas, so that Web developers can smoothly display 3D scenes and models in a browser by means of a system graphics card. The application uses the thread. Js as a code base to generate codes, the thread. Js is a javascript base of WebGL, the Java script base encapsulates the access details of WebGL, the thread. Js codes (namely scene codes) generated by the large language model can be loaded and rendered in a browser to generate a three-dimensional scene model, and compared with a calculation engine based on C/S, such as Unity and Unreal, the application has lower resource consumption for deploying a server, better universality and better support for different operating system platforms.
Referring to fig. 5, in some embodiments, the model generating method further includes:
step 014: displaying an input box and acquiring feedback information input by the input box;
step 015: analyzing feedback information based on a preset large language model, and adjusting scene codes according to the feedback information; or, according to the feedback information, adjusting scene description information; and analyzing the adjusted scene description information based on the preset large language model to regenerate the scene code.
Specifically, since scene description information is not accurate enough and even description is wrong, or in the case that a large language model deviates from understanding of some descriptions, a finally generated three-dimensional scene model may not meet the requirements of a user, and the three-dimensional model needs to be adjusted. Or the user is dissatisfied with the effect after seeing the three-dimensional scene model, and may need to adjust the model.
The user inputs feedback information through an input box of the feedback information, after the feedback information is obtained, the feedback information can be analyzed based on a preset large language model, and a scene code can be adjusted according to the feedback information; or, according to the feedback information, adjusting scene description information; and analyzing the adjusted scene description information based on the preset large language model to regenerate the scene code.
For example, the azimuth error of the device a is detected, at this time, the feedback information can be input through the input box of the feedback information on the display interface of the final three-dimensional scene model, the feedback information describes the azimuth error of the device a and describes the correct azimuth of the device a, so that after the feedback information is analyzed by the large language model, the scene code can be adjusted according to the analysis content of the feedback information, and then the scene code is rendered again based on the scene code, thereby generating the three-dimensional scene model with the correct azimuth adjustment of the device a.
For another example, after the feedback information is obtained, the large language model can adjust the scene description information according to the analysis content of the feedback information after analyzing the feedback information, for example, the description of the equipment A in the scene description information is replaced by the description of the equipment A in the feedback information, then the scene code is regenerated based on the adjusted scene description information, and then the three-dimensional scene model with the correct orientation adjustment of the equipment A is generated based on the regenerated scene code again for rendering.
In some implementations, to generate a digital twin model, each model object in the three-dimensional scene model and a corresponding device in the scene may be associated; and controlling the display content of the model object corresponding to the equipment according to the operation information of the equipment.
Specifically, the method and the device are mainly used for generating the 3D scene model in the digital twin model, and in order to realize digital twin in the follow-up, the device and the 3D scene model are required to be associated, so that a model object corresponding to the device in the 3D scene model can display information related to the working condition of the corresponding device in real time. After each model object in the 3D scene model is associated with the corresponding equipment, the display content of the corresponding model object can be controlled according to the running information of the equipment, for example, for the model object of a camera, the display content of the model object of the camera can be controlled according to the image data acquired by the camera in real time, and a user can see the video of the actual scene shot by the camera by clicking the model object of the camera. In this way, mapping of the device to model objects of the virtual space (i.e., the 3D scene model) can be achieved, enabling digital twinning.
Referring to fig. 6, in one example, a surface assembly technology production line (SurfaceMountTechnology, SMT) 3D scene model of a factory is generated.
1. The scene description information is as follows:
(1) The scene comprises 4 SMT production lines, the production lines are distributed at equal intervals, the left side of each production line is marked with characters, and the production lines are respectively from bottom to top: a first production line, a second production line, a third production line and a fourth production line;
(2) Wherein each SMT production line equipment includes from right to left: the method comprises the following steps of plate blanking, AOI detection, a connection table, a reflow soldering furnace, a translation machine, a chip mounter, a solder paste detection machine, a tin printing machine, a laser engraving machine and plate blanking;
(3) The background of the product line environment is brown blue, the 3D environment is a closed space, the floor S1 is black, no ceiling exists, cameras S2 are arranged on the upper side wall and the lower side wall, the cameras are arranged above the walls, a product line information prompt frame S3 is arranged on the right side wall, four square display boards S4 are arranged on the walls, and the dimensions of data display on the display boards S4 are as follows: good product number, single product energy consumption, yield, today's load and total energy consumption; the display text is white, and the frame is gray.
It can be understood that the environmental information includes "the environment background of the production line is brown blue, the 3D environment is a closed space, the floor color is black, no ceiling", the equipment information includes "the lower trigger, automatic optical detector (Automated Optical Inspection, AOI), docking station, reflow oven, translation machine, chip mounter, solder paste detector, tin printer, laser engraving machine, upper trigger, camera", the layout information includes "the layout is arranged according to equidistant between the production lines", "each SMT production line equipment includes from right to left: lower trigger, AOI detects, the platform of plugging into, reflow soldering stove, translation machine, chip mounter, solder paste detects the machine, the tin printing machine, radium carving machine, go up trigger "," on the left, there is the camera on higher authority and the lower wall, be located the top of wall "," on the right wall have the line information prompt box to lay out four square show cards on the wall ", and the instruction information includes" every line left mark characters, from bottom to top is respectively: the dimensions of the data display on the display board are as follows: good product number, single product energy consumption, yield, today's load and total energy consumption; the display text is white, and the frame is gray. "
2. The large language model analyzes scene description information, and combines the addresses of target model files corresponding to physical entities (such as a wall, a floor, a lower plate machine, an automatic optical detector, a connection platform, a reflow soldering furnace, a translation machine, a chip mounter, a solder paste detector, a tin printing machine, a laser carving machine, an upper plate machine, a camera and the like) to generate the scene code. For brevity, the scene code of the floor is taken as an example, and the principles of other physical entities are similar and will not be repeated.
The scene code of the floor is as follows:
"javascript// generate js code for a factory SMT production line 3D scene based on three. Js;
v/create scene = new three. Scene ();
the address varMeshpath= "D: \s10.obj" of the target model file corresponding to the definition floor;
v/create floor object var floor=new three
Creating a grid object representing the floor new three.plangegeometry (100 );
the method comprises the steps of (1) creating a plane geometry, wherein the width and the height are 100,new THREE.MeshLambertMaterial ({// creating a material, coloring by using Lambert, and the color is black color 0x000000; the material is set to be rendered to be double-sided, namely, the front side and the back side are rendered, and the side is THREE.double side);
Addition of floors to scene. Add (floor)'
3. The browser runs the scene code, performs 3D rendering to generate and display a 3D scene model, as shown in fig. 6, which is the finally generated 3D scene model.
Referring to fig. 7, in order to facilitate better implementation of the model generating method according to the embodiment of the present application, the embodiment of the present application further provides a model generating apparatus 10. The model generating apparatus 10 may include a first acquisition module 11, a second acquisition module 12, and a build model 13. The first obtaining module 11 is configured to obtain scene description information, where the scene description information includes at least environment information, device information in a scene, and layout information; the second obtaining module 12 is configured to obtain a target model file corresponding to the scene description information in a preset model library, where the model library includes a plurality of model files, and the plurality of model files are generated according to physical entities existing in the preset scene; the construction module 13 is configured to parse the scene description information based on a preset large language model, and construct a three-dimensional scene model in combination with the target model file.
The first obtaining module 11 is specifically configured to display a description information input interface, where the input interface includes a plurality of input boxes, and the plurality of input boxes respectively correspond to input of environment information, input of device information, and input of layout information; input information of a plurality of input boxes is acquired to generate scene description information.
The second obtaining module 12 is specifically configured to obtain a first physical entity in the environmental information and a second physical entity in the device information, where the object model file includes a first object model file corresponding to the first physical entity and a second object model file corresponding to the second physical entity; and acquiring a first target model file and a second target model file.
The second obtaining module 12 is specifically further configured to search the mapping table for a first target address corresponding to the first physical entity and a second target address corresponding to the second physical entity.
The construction module 13 is specifically configured to parse the scene description information based on the large language model, and generate a renderable scene code in combination with the target model file; rendering is performed according to the scene codes to generate a three-dimensional scene model.
The construction module 13 is specifically further configured to load the target model file from the address of the target model file corresponding to each model object through the browser, and render each model object according to the azimuth information and the color information of each model object, so as to generate and display a three-dimensional scene model.
The model generating device 10 further comprises a display module 14 and an adjustment module 15. The display module 14 is used for displaying an input box and acquiring feedback information input by the input box; the adjustment module 15 is configured to parse feedback information based on a preset large language model, and adjust a scene code according to the feedback information; or, according to the feedback information, adjusting scene description information; and analyzing the adjusted scene description information based on the preset large language model to regenerate the scene code.
The model generating apparatus 10 is described above in connection with the accompanying drawings from the viewpoint of functional modules, which may be implemented in hardware, or in instructions in software, or in a combination of hardware and software modules. Specifically, each step of the method embodiment in the embodiment of the present application may be implemented by an integrated logic circuit of hardware in a processor and/or an instruction in software form, and the steps of the method disclosed in connection with the embodiment of the present application may be directly implemented as a hardware encoding processor or implemented by a combination of hardware and software modules in the encoding processor. Alternatively, the software modules may be located in a well-established storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, and the like. The storage medium is located in a memory, and the processor reads information in the memory, and in combination with hardware, performs the steps in the above method embodiments.
Referring to fig. 8, a computer device 100 according to an embodiment of the present application includes a processor 20, a memory 30, and a computer program, wherein the computer program is stored in the memory 30 and executed by the processor 20, and the computer program includes instructions for executing the model generating method according to any of the above embodiments.
Alternatively, the computer device 100 may be any device having image processing capabilities, such as a server or terminal device (e.g., a cell phone, tablet, display device, notebook, smart watch, head display device, game console, etc.).
Optionally, the computer device 100 may also be a combined system of a terminal device and a server, where the terminal device is used to obtain scene description information, the server is used to store a model library, and the large language model may be deployed to the server, where after the terminal device obtains the scene description information, the terminal device sends the scene code to the server, and after the server obtains the address of the corresponding object model file, the scene code is generated based on the large language model, and sent to the terminal device, and then the browser of the terminal device runs the scene code to implement 3D rendering to generate and display the 3D scene model.
It should be appreciated that the various components in the computer device 100 are connected by a bus system that includes a power bus, a control bus, and a status signal bus in addition to a data bus.
Referring to fig. 9, an embodiment of the present application further provides a computer readable storage medium 300, on which a computer program 310 is stored, where the computer program 310, when executed by the processor 320, implements the steps of the model generating method according to any of the foregoing embodiments, which are not described herein for brevity.
In the description of the present specification, reference to the terms "certain embodiments," "in one example," "illustratively," and the like, means that a particular feature, structure, material, or characteristic described in connection with the embodiments or examples is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and further implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.
While embodiments of the present application have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the application, and that variations, modifications, alternatives and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the application.
Claims (12)
1. A model generation method, characterized by comprising:
acquiring scene description information, wherein the scene description information at least comprises environment information, equipment information in a scene and layout information;
obtaining a target model file corresponding to the scene description information in a preset model library, wherein the model library comprises a plurality of model files, and the model files are generated according to physical entities existing in a preset scene;
and analyzing the scene description information based on a preset large language model, and constructing a three-dimensional scene model by combining the target model file.
2. The method for generating a model according to claim 1, wherein the obtaining the object model file corresponding to the scene description information in the preset model library includes:
acquiring a first physical entity in the environment information and a second physical entity in the equipment information, wherein the target model file comprises a first target model file corresponding to the first physical entity and a second target model file corresponding to the second physical entity;
And acquiring the first target model file and the second target model file.
3. The method of generating a model according to claim 2, wherein the model file is stored in a preset database, an address of the model file and a name of a corresponding physical entity are stored in a preset mapping table, and the acquiring the first target model file and the second target model file includes:
and searching a first target address corresponding to the first physical entity and a second target address corresponding to the second physical entity in the mapping table.
4. The method of generating a model according to claim 1, wherein the parsing the scene description information based on the preset large language model and constructing a three-dimensional scene model in combination with the object model file includes:
analyzing the scene description information based on the large language model, and generating a renderable scene code by combining the target model file;
rendering is carried out according to the scene codes so as to generate the three-dimensional scene model.
5. The model generation method according to claim 1, wherein the scene code includes azimuth information of each model object, color information of each model object, and an address of the target model file corresponding to each model object, and wherein the rendering according to the scene code to generate the three-dimensional scene model includes:
And loading the target model file from the address of the target model file corresponding to each model object through a browser, and rendering each model object according to the azimuth information and the color information of each model object so as to generate and display the three-dimensional scene model.
6. The model generation method according to claim 4, characterized by further comprising:
displaying an input box and acquiring feedback information input by the input box;
analyzing the feedback information based on a preset large language model, and adjusting the scene code according to the feedback information; or, according to the feedback information, adjusting the scene description information; and analyzing the adjusted scene description information based on a preset large language model to regenerate the scene code.
7. The model generation method according to any one of claims 4 to 6, wherein the scene code is generated based on a code base of a preset three-dimensional drawing frame.
8. The model generation method according to claim 1, wherein the acquiring scene description information includes:
displaying a description information input interface, wherein the input interface comprises a plurality of input boxes, and the input boxes respectively correspond to the input of the environment information, the input of the equipment information and the input of the layout information;
And acquiring input information of a plurality of input boxes to generate the scene description information.
9. The model generation method according to claim 1, characterized by further comprising:
associating each model object in the three-dimensional scene model with corresponding equipment in the scene;
and controlling the display content of the model object corresponding to the equipment according to the operation information of the equipment.
10. A model generation apparatus, characterized in that the model generation apparatus comprises:
the first acquisition module is used for acquiring scene description information, wherein the scene description information at least comprises environment information, equipment information in a scene and layout information;
the second acquisition module is used for acquiring a target model file corresponding to the scene description information in a preset model library, wherein the model library comprises a plurality of model files, and the model files are generated according to physical entities existing in a preset scene;
the construction module is used for analyzing the scene description information based on a preset large language model and constructing a three-dimensional scene model by combining the target model file.
11. A computer device, comprising:
a processor, a memory; a kind of electronic device with high-pressure air-conditioning system
A computer program, wherein the computer program is stored in the memory and executed by the processor, the computer program comprising instructions for performing the model generation method of any of claims 1 to 9.
12. A non-transitory computer readable storage medium containing a computer program which, when executed by a processor, causes the processor to perform the model generation method of any of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310877924.9A CN116912413A (en) | 2023-07-17 | 2023-07-17 | Model generation method, model generation device, computer device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310877924.9A CN116912413A (en) | 2023-07-17 | 2023-07-17 | Model generation method, model generation device, computer device, and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116912413A true CN116912413A (en) | 2023-10-20 |
Family
ID=88352572
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310877924.9A Pending CN116912413A (en) | 2023-07-17 | 2023-07-17 | Model generation method, model generation device, computer device, and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116912413A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117576359A (en) * | 2024-01-16 | 2024-02-20 | 北京德塔精要信息技术有限公司 | 3D model construction method and device based on Unity webpage platform |
-
2023
- 2023-07-17 CN CN202310877924.9A patent/CN116912413A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117576359A (en) * | 2024-01-16 | 2024-02-20 | 北京德塔精要信息技术有限公司 | 3D model construction method and device based on Unity webpage platform |
| CN117576359B (en) * | 2024-01-16 | 2024-04-12 | 北京德塔精要信息技术有限公司 | 3D model construction method and device based on Unity webpage platform |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116912413A (en) | Model generation method, model generation device, computer device, and storage medium | |
| WO2007149425A2 (en) | System and method for generating a dynamic electrical wiring diagram including an internal box layout structure | |
| CN111970503B (en) | Three-dimensional method, device and equipment for two-dimensional image and computer readable storage medium | |
| CN103914876A (en) | Method and apparatus for displaying video on 3D map | |
| CN112732255A (en) | Rendering method, device, equipment and storage medium | |
| CN112734896A (en) | Environment shielding rendering method and device, storage medium and electronic equipment | |
| Gimeno et al. | Multiuser augmented reality system for indoor exhibitions | |
| US11341716B1 (en) | Augmented-reality system and method | |
| CN112702643A (en) | Bullet screen information display method and device and mobile terminal | |
| CN108597025B (en) | Artificial intelligence-based virtual reality-oriented rapid model construction method and device | |
| CN113094287A (en) | Page compatibility detection method, device, equipment and storage medium | |
| CN114463104B (en) | Method, apparatus, and computer-readable storage medium for processing VR scene | |
| CN115847384A (en) | Mechanical arm safety plane information display method and related product | |
| US11710039B2 (en) | Systems and methods for training image detection systems for augmented and mixed reality applications | |
| CN113289337B (en) | UE-based dynamic generation method for simulated global minimap | |
| CN112115680B (en) | Font splitting method and device, computer equipment and computer readable storage medium | |
| CN114307158A (en) | Three-dimensional virtual scene data generation method and device, storage medium and terminal | |
| CN113742507A (en) | Method for three-dimensionally displaying an article and associated device | |
| CN112233208B (en) | Robot state processing method, apparatus, computing device and storage medium | |
| Girašek et al. | Visualization of temperature fields distribution on power module within web interface | |
| CN113010627A (en) | City information model service method | |
| CN110889157A (en) | Wind power plant layout visualization display method and related device | |
| CN112307378B (en) | Network view display method | |
| CN112354188B (en) | Image processing method and device of virtual prop, electronic equipment and storage medium | |
| CN219875853U (en) | 3D scanning handheld terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |