CN118132059A - Graphical programming method, device, equipment and storage medium - Google Patents

Abstract

The application discloses a graphical programming method, a graphical programming device, graphical programming equipment and a graphical programming storage medium, and relates to the technical field of computers. The method comprises the following steps: displaying a code editing interface of the graphical programming tool, wherein the code editing interface comprises a plurality of preset graphical codes; responding to the editing operation of a first graphical code in a plurality of preset graphical codes, displaying the edited first graphical code, wherein the edited first graphical code is used for adding at least one object into a first formation; responding to the editing operation of a second graphical code in a plurality of preset graphical codes, and displaying the edited second graphical code; and running a graphical programming result to control at least one object in the first formation, wherein the graphical programming result comprises the edited first graphical code and the edited second graphical code. The method improves programming efficiency.

Description

Graphical programming method, device, equipment and storage medium

Technical Field

The present application relates to the field of computer technology, and in particular to a graphical programming method, apparatus, device and storage medium.

Background technique

With the development of computer technology, the technology of using graphical building blocks for graphical programming has become increasingly mature. Graphical building blocks can also be understood as graphical codes, which encapsulate programming languages. Users can implement graphical programming by dragging graphical building blocks.

In the related art, when multiple objects need to be controlled separately, graphical programming is generally performed for each object separately. Even if the actions to be performed by multiple objects are the same, that is, when multiple objects are controlled to implement the same control logic (such as all need to move 5 meters to the left), each object needs to be programmed separately.

Therefore, in the above-mentioned related technologies, when controlling multiple objects to implement the same control logic, it is still necessary to program the multiple objects separately, that is, repeated programming occurs, resulting in low programming efficiency.

Summary of the invention

The embodiments of the present application provide a graphical programming method, apparatus, device and storage medium, which can improve the efficiency of graphical programming. The technical solutions proposed in this application are as follows:

According to one aspect of an embodiment of the present application, a graphical programming method is provided, the method comprising:

Displaying a code editing interface of a graphical programming tool, wherein the code editing interface includes a plurality of preset graphical codes, and the graphical codes are used to process one or more objects;

In response to an editing operation on a first graphical code among the plurality of preset graphical codes, displaying the edited first graphical code, wherein the first graphical code is used to add at least one object to the same formation, and the edited first graphical code is used to add at least one object to the first formation;

In response to an editing operation on a second graphical code among the plurality of preset graphical codes, displaying the edited second graphical code, wherein the second graphical code is used to uniformly control at least one object in the same formation, and the edited second graphical code is used to uniformly control at least one object in the first formation;

The graphical programming result is run to control at least one object in the first formation, wherein the graphical programming result includes the edited first graphical code and the edited second graphical code.

According to one aspect of an embodiment of the present application, a graphical programming device is provided, the device comprising:

An interface display module, used to display a code editing interface of a graphical programming tool, wherein the code editing interface includes a plurality of preset graphical codes, and the graphical codes are used to process one or more objects;

a graphical code display module, configured to display an edited first graphical code in response to an edit operation on a first graphical code among the plurality of preset graphical codes, wherein the first graphical code is used to add at least one object to the same formation, and the edited first graphical code is used to add at least one object to the first formation;

The graphical code display module is further used to display the edited second graphical code in response to an editing operation on a second graphical code among the plurality of preset graphical codes, wherein the second graphical code is used to uniformly control at least one object in the same formation, and the edited second graphical code is used to uniformly control at least one object in the first formation;

The running module is used to run the graphical programming result to control at least one object in the first formation, and the graphical programming result includes the edited first graphical code and the edited second graphical code.

According to one aspect of an embodiment of the present application, a computer device is provided, the computer device comprising a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above-mentioned graphical programming method.

According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is loaded and executed by a processor to implement the above-mentioned graphical programming method.

According to one aspect of an embodiment of the present application, a computer program product is provided. The computer program product includes a computer program. The computer program is loaded and executed by a processor to implement the above-mentioned graphical programming method.

The technical solution provided in the embodiments of the present application can bring the following beneficial effects:

By editing the first graphical code in the code editing interface of the graphical programming tool, at least one object can be added to the first formation. By editing the second graphical code, the edited second graphical code is used to uniformly control at least one object in the first formation. Running the graphical programming result including the edited first graphical code and the edited second graphical code can achieve unified control of at least one object added to the first formation. The technical solution provided by the embodiment of the present application uses the first graphical code to compile objects that need to execute the same code logic into the first formation, so that at least one object added to the first formation shares the same edited second graphical code, without the need for repeated programming, thereby improving programming efficiency.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of an implementation environment of a solution provided by an embodiment of the present application;

FIG2 is an interface diagram of a graphical programming method provided by the related art;

FIG3 is an interface diagram of a graphical programming method provided by the related art;

FIG4 is an interface diagram of a graphical programming method provided by the related art;

FIG5 is an interface diagram of a graphical programming method provided by the related art;

FIG6 is an interface diagram of a graphical programming method provided by the related art;

FIG7 is an interface diagram of a graphical programming method provided by the related art;

FIG8 is an interface diagram of a graphical programming method provided by an embodiment of the present application;

FIG9 is an interface diagram of a graphical programming method provided by another embodiment of the present application;

FIG10 is a schematic diagram of a virtual scene interface provided by an embodiment of the present application;

FIG11 is a schematic diagram of a code editing interface provided by an embodiment of the present application;

FIG12 is a flowchart of a graphical programming method provided by one embodiment of the present application;

FIG13 is an interface diagram of a graphical programming method provided by another embodiment of the present application;

FIG14 is an interface diagram of a graphical programming method provided by another embodiment of the present application;

FIG15 is a flowchart of a graphical programming method provided by another embodiment of the present application;

FIG16 is a schematic diagram of an object selection method provided by an embodiment of the present application;

FIG17 is a schematic diagram of an object selection method provided by another embodiment of the present application;

FIG18 is an interface diagram of a graphical programming method provided by another embodiment of the present application;

FIG19 is an interface diagram of a graphical programming method provided by another embodiment of the present application;

FIG20 is a block diagram of an object status updating method provided by an embodiment of the present application;

FIG21 is a block diagram of an object status updating method provided by another embodiment of the present application;

FIG22 is a block diagram of an object status updating method provided by another embodiment of the present application;

FIG23 is a schematic diagram of code operation provided by an embodiment of the present application;

FIG24 is a schematic diagram of interface switching provided by an embodiment of the present application;

FIG25 is a flowchart of a graphical programming method provided by another embodiment of the present application;

FIG26 is a schematic diagram of a function panel interface provided by an embodiment of the present application;

FIG27 is a schematic diagram of a function panel interface provided by another embodiment of the present application;

FIG28 is a flowchart of a method for executing a graphical code according to an embodiment of the present application;

FIG29 is a flowchart of a method for executing a graphical code according to another embodiment of the present application;

FIG30 is a flowchart of a method for executing a graphical code according to another embodiment of the present application;

FIG31 is a schematic diagram of a compilation process provided by an embodiment of the present application;

FIG32 is a schematic diagram of a component structure provided by an embodiment of the present application;

FIG33 is a schematic diagram of a code provided by an embodiment of the present application;

FIG34 is a block diagram of a graphical programming device provided by one embodiment of the present application;

FIG35 is a block diagram of a graphical programming device provided by another embodiment of the present application;

FIG36 is a structural block diagram of a computer device provided in one embodiment of the present application.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present application more clear, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

Before introducing the technical solution of the present application, some terms involved in the present application are explained. The following related explanations can be combined arbitrarily with the technical solution of the embodiment of the present application as optional solutions, and they all belong to the protection scope of the embodiment of the present application. The embodiment of the present application includes at least part of the following contents.

Graphical programming: The programming language is encapsulated in graphical building blocks, and programming can be completed by dragging the building blocks to build. It is generally used for programming learning among teenagers or beginners.

IDE (Integrated Development Environment): used to provide a development environment for applications, generally including tools such as code editors, compilers, debuggers, and graphical user interfaces.

Virtual scene: A mode in a graphical programming tool that displays a virtual scene in non-running static and running states.

Code editing: A mode in a graphical programming tool that displays graphical building block codes and allows users to add, delete, write and run codes.

Please refer to FIG1 , which shows a schematic diagram of a solution implementation environment provided by an embodiment of the present application. The solution implementation environment may include: a terminal device 10 and a server 20 .

The terminal device 10 includes, but is not limited to, mobile phones, tablet computers, intelligent voice interaction devices, game consoles, wearable devices, multimedia playback devices, PCs (Personal Computers), vehicle-mounted terminals, smart home appliances and other electronic devices. The client of the target application can be installed in the terminal device 10. Optionally, the target application can be an application that needs to be downloaded and installed, or it can be a click-to-use application, which is not limited in the embodiments of the present application.

In an embodiment of the present application, the above-mentioned target application may be an application with programming functions, such as an integrated development environment provided in the target application, in which the user can write code, compile code and run code. Exemplarily, the target application includes a graphical programming tool. Exemplarily, the graphical programming tool provides a plurality of preset graphical codes (i.e., graphical building blocks), and the target application may encapsulate the programming language in the graphical code, so that the graphical code is presented to the user in the form of building blocks or other graphics, and the user can drag these graphical codes to implement programming. Exemplarily, the target application may directly obtain the encapsulated graphical code and directly provide it to the user for use. The present application does not limit the specific functions of the target application. The types of the target application may be test applications, development applications, game applications, virtual reality (VR) applications, augmented reality (AR) applications, etc., and the embodiments of the present application do not limit the specific categories of the target application. In other embodiments, the target application may be considered as a separate functional module, such as being implemented as one of the functional modules in the application to be developed. Exemplarily, a client of the target application program is running in the terminal device 10 .

The server 20 is used to provide background services for the client of the target application in the terminal device 10. For example, the server 20 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, but is not limited to these.

The terminal device 10 and the server 20 can communicate with each other via a network, which can be a wired network or a wireless network.

In the method provided in the embodiment of the present application, the execution subject of each step may be a computer device. The computer device may be any electronic device with data storage and processing capabilities. For example, the computer device may be the terminal device 10 in FIG. 1 , or may be the server 20.

The following is a brief description of the virtual drone as an example. With the development of artificial intelligence, programming methods based on graphical programming tools are becoming increasingly mature. The graphical programming tools provided by some applications do not rely on the hardware environment, only require the network, and are highly standardized.

However, in the related art, when it comes to the need to group multiple objects to achieve an overall consistent operating effect, the code can only be copied and written for different objects separately. The process is cumbersome and repetitive, and it will also increase the number of building block codes, which is not conducive to improving the efficiency of building block code writing.

In some embodiments, as shown in FIG. 2 , a user interface 200 provided by a graphical programming tool in the related art includes a building block area, an editing area, a stage area, and a material area. The interface layout is relatively fixed. The user selects building blocks in the building block area according to his needs, splices building blocks in the editing area, and selects materials from the material area. After that, click the run button to preview the effect in the stage area. Of course, if you want to achieve the same running effect for different objects, you need to switch different objects in the actual programming process to add and write codes respectively.

In some embodiments, as shown in FIG3 , when you want to control object 300, you need to assemble blocks in the editing area to obtain block code 301 for controlling object 300. In some embodiments, as shown in FIG4 , when you want to control object 400, you need to assemble blocks in the editing area to obtain block code 401 for controlling object 400. Even if you need to control object 300 and object 400 to achieve the same operating effect, in the actual coding process, you need to switch between different objects to add and write code respectively.

In some embodiments, as shown in FIG5 , another graphical programming tool in the related art provides a user interface 500 including a stage area, a material area, a block preset area, and a block editing area. The interface layout is relatively fixed, and the user selects blocks in the block preset area according to their needs, splices blocks in the block editing area, and after selecting materials from the material area, clicks the run button to preview the effect in the stage area. Of course, if you want to achieve the same running effect for different objects, you also need to switch different objects in the actual programming process to add and write codes separately.

In some embodiments, as shown in FIG6 , when you want to control object 600, you need to assemble blocks in the editing area to obtain a block code 601 for controlling object 600. In some embodiments, as shown in FIG7 , when you want to control object 700, you need to assemble blocks in the editing area to obtain a block code 701 for controlling object 700. Even if you want to control object 600 and object 700 to achieve the same operating effect, in the actual coding process, you need to switch between different objects to add and write codes respectively.

In the above-mentioned related technologies, it is impossible to use only one set of building block codes to realize the control of objects in multiple stage areas. If you want to achieve an overall unified operating effect for multiple objects during the code writing process, you need to repeatedly write multiple sets of codes, and the process is cumbersome and complicated. In addition, when viewing and modifying the code for different objects, you need to switch back and forth to different objects, which greatly reduces the efficiency of debugging (viewing and modifying code).

The technical solution provided in the embodiment of the present application innovatively provides building block codes in the building block preset area that can simultaneously control multiple drones to join a formation. Users can use the building blocks to add multiple drones to a formation according to actual needs, and use a set of codes to complete the flight control of the entire drone formation, thereby improving the efficiency of code writing, thereby realizing group control, interconnected performances, and formation flight of virtual drones.

This application provides a set of building blocks to simultaneously control the flight states of multiple virtual drones, such as formation control, interconnected performances, and formation flying. Users only need to add a set of building block codes and modify the parameters of the building block codes to formulate the code to a formation composed of multiple virtual drones, thereby simplifying the code writing process and improving writing efficiency. Avoid the tedious operation of switching back and forth between different objects during debugging (viewing and modifying building block codes) to improve debugging efficiency. In the process of adjusting parameters, the interactive form supports the use of box selection to improve the efficiency of selecting and grouping multiple drones.

Before introducing the specific method of the present application, the application scenario of the graphical programming method applied to the graphical programming tool in the present application is first exemplified. Of course, it should be known that the following application scenario is only an example and is not limited to this.

In some embodiments, the graphical programming method and code execution method in the embodiments of the present application can be used in the field of UGC (User generated content). Exemplarily, UGC means that users can display or provide their original content to other users. Exemplarily, UGC can be used in the field of games, animation production, self-media, etc. Exemplarily, only taking UGC in the field of games as an example, the graphical programming method mentioned in this application and the following code execution method are specifically described. Exemplarily, the target application is an application that allows users to customize the game, such as the target application provides an interface that allows users to customize the game, which is called a game customization interface. In the game customization interface, multiple preset graphical codes are displayed, and the graphical codes are used to process one or more objects. Of course, the objects here include but are not limited to any controllable objects, such as the objects in the virtual scene, such as virtual animals, virtual plants, virtual cars, virtual buildings, etc., which are objects that may appear in the game that the user wants to customize. In some embodiments, in response to an editing operation on a first graphical code among a plurality of preset graphical codes, the edited first graphical code is displayed, the first graphical code is used to add at least one object to the same formation, and the edited first graphical code is used to add at least one object to the first formation. Exemplarily, multiple virtual stars are organized into the same formation to control their moving direction. Exemplarily, multiple virtual plants are organized into the same formation to control their growth direction and growth speed. In some embodiments, in response to an editing operation on a second graphical code among a plurality of preset graphical codes, the edited second graphical code is displayed, the second graphical code is used to uniformly control at least one object in the same formation, and the edited second graphical code is used to uniformly control at least one object in the first formation. Exemplarily, the user can achieve unified control of each object in the formation by editing the second graphical code. In some embodiments, after the user edits the first graphical code and the second graphical code, the graphical programming result including the edited first graphical code and the edited second graphical code is run to control at least one object in the first formation. For example, when the user clicks "Run", the following code execution method is executed, and the virtual scene constructed by the user can be seen, and the objects in the virtual scene move, such as a team of virtual cars on the street driving in the direction defined by the user, and multiple virtual birds in the sky flying along the tracks defined by the user. Users can construct game scenes, game levels, etc. through graphical programming.

In some embodiments, the graphical programming method in the embodiments of the present application is used for animation development. In the target application, the developer adds a new scene and multiple animation objects, configures each animation object, and uses a graphical building block code to control these animation objects, including forming a formation, and uses a second graphical code to control the animation objects in the formation. Exemplarily, a preview interface is also provided in the graphical programming tool, in which the control process of each animation object can be previewed while running the graphical programming result. Exemplarily, when the animation presented in the preview interface meets expectations, the previewed animation is directly used as the generated animation.

In other embodiments, the graphical programming method in the embodiments of the present application is used to control virtual drones. Exemplarily, the target application provides a virtual drone flight environment and drones. Exemplarily, the combined drone formation is controlled by graphical building block codes to realize dynamic changes such as interconnected performances and flights of the formation. Exemplarily, a building block code (i.e., a second graphical code) that can simultaneously control multiple drones to join the formation is provided in the building block preset area. Users can use the building block to add multiple drones to the formation according to actual use, and use a set of codes to complete the flight control of the entire drone formation, thereby improving the efficiency of writing code, thereby realizing formation control, interconnected performances, formation flight, etc. of virtual drones.

In some embodiments, the graphical programming method in the embodiments of the present application is used to control actual objects. Exemplarily, the specific parameters of multiple actual objects (such as multiple real toys, drones, light bulbs), etc. are imported into the target application, and the multiple actual objects are bound to the terminal device where the target application is located. Exemplarily, the combination of the multiple actual object formations is controlled by graphical building block codes to realize the dynamic changes of the interconnected performances, flights, etc. of the formations. Exemplarily, multiple actual objects are added to the formation according to the second graphical code. When the second graphical code is run, control signals are sent to multiple actual objects to control the actual objects to perform corresponding actions, thereby realizing the use of a set of codes to complete the joint control of multiple actual objects and improve the efficiency of programming.

The following takes a virtual drone as an example to illustrate the specific application scenarios of the present application in combination with the accompanying drawings.

In some embodiments, as shown in Figure 8, a user download interface 800 is provided in the target application. When the user clicks to purchase the target application, the programming function provided by the target application can be used. Of course, it can also be provided to users for free download without the need for users to purchase. After the user has the authority to use the target application, a function selection interface 801 is displayed, and the user can select simulation programming 802 in the function selection interface 801. After the user selects simulation programming 802, a user login interface 803 is displayed. The user can choose different accounts to log in to the target application. After the user logs in to the target application, a programming mode selection interface 804 is displayed, in which the user can select graphical programming or other programming modes, which is not limited in this application.

In some embodiments, as shown in FIG9 , after the user selects graphical programming, an object quantity selection interface 900 is displayed, and the user can configure the number of objects by himself, such as configuring only one drone. Exemplarily, after the user configures the number of objects, a scene selection interface 901 is displayed, and the user can configure the scene where the object is located by himself, such as configuring the scene to be xx smart city. After the user configures the object and scene, a virtual scene interface 902 is displayed, and the virtual scene and the object 903 created in the scene are displayed in the virtual scene interface.

In some embodiments, as shown in Figure 10, the user interface provided by the graphical programming tool includes a virtual scene interface 1000. In some embodiments, the virtual scene interface 1000 includes a scene area, a navigation toolbar, a map navigation area, and a function panel area.

In some embodiments, as shown in Figure 11, the user interface provided for the graphical programming tool also includes a code editing interface 1100. In some embodiments, the code editing interface 1100 includes a building block preset area, a building block editing area, a navigation toolbar, a map navigation area, and a function panel area.

In some embodiments, the user can drag and drop the preset building block code in the code editing interface 1100 and edit it to obtain the edited building block code. When the edited building block code is run, the control process of the object in the virtual scene can be seen in the virtual scene interface 1000.

In the technical solution provided by the embodiment of the present application, a first graphical code and a second graphical code are preset in the building block preset area, and at least one object is added to the same formation through the first graphical code, and unified control of at least one object added to the same formation is achieved through the second graphical code. This avoids repeated code writing and speeds up code writing efficiency.

The above is the relevant introduction related to the present application. The graphical programming method proposed in the embodiment of the present application is explained below in combination with specific embodiments.

Please refer to Figure 12, which shows a flowchart of a graphical programming method provided by an embodiment of the present application. The execution subject of each step of the method can be the terminal device 10 introduced above, or the client of the target application running on the terminal device 10, or the server 20. In the following method embodiment, for the convenience of description, only the execution subject of each step is introduced as a "computer device". The method can include at least one of the following steps (1210-1240).

Step 1210, displaying a code editing interface of a graphical programming tool, wherein the code editing interface includes a plurality of preset graphical codes, and the graphical codes are used to process one or more objects.

In some embodiments, the graphical programming tool is an application program or a functional module in an application program that provides a graphical programming function. Exemplarily, the graphical programming tool is the above-mentioned target application program. Exemplarily, the graphical programming tool is a graphical programming module in the above-mentioned target application program. Exemplarily, the graphical programming tool provides a graphical programming function.

In some embodiments, the code editing interface of the graphical programming tool is an interface provided by the graphical programming tool for editing code (including graphical code). Exemplarily, as shown in Figure 13, the code editing interface can be interpreted broadly, that is, the complete user interface 1300 is considered to be the code editing interface, and the complete user interface 1300 includes at least one of the building block preset area 1303, the function panel area, the navigation toolbar, and the map navigation area in addition to the building block editing area 1304. Of course, the code editing interface proposed in this application can also be interpreted in a narrow sense, that is, only including the building block editing area 1304. At this time, the preset building block code is not displayed in the building block editing area 1304, but is displayed in the building block preset area 1303.

In some embodiments, the code editing interface includes multiple preset graphical codes. Exemplarily, multiple preset graphical codes are displayed in the building block preset area 1303. The graphical code in the embodiment of the present application refers to a code displayed in the form of a graphic, and the graphical code encapsulates a programming language. Different graphical codes encapsulate different programming languages, that is, different control logics for objects.

In some embodiments, the graphical code includes at least two categories. The first type of graphical code is used to control a single object, and can only control a single object. The second type of graphical code is used to control a formation, which includes at least one object. When there are multiple objects in the formation, the graphical code can control multiple objects in the formation. Exemplarily, the preset graphical code may include at least one graphical code of the second type, and of course, in addition to the graphical code of the second type, it may also include at least one graphical code of the first type. The embodiments of the present application do not limit the types of preset graphical codes.

The objects in the embodiments of the present application include but are not limited to virtual objects or real objects. When the object is a real object, the object can be an active or inactive object in the real world, such as a real drone, a light bulb, a toy car, etc. When the object is a virtual object, the object can be an active or inactive object in a virtual scene, such as a virtual drone, a virtual light bulb, a virtual car, etc. In some embodiments, when the object is a virtual object, the virtual object can be created by a user. When the object is a real object, the graphical programming tool can be connected to at least one real object, and when the graphical programming result is run using the graphical programming tool, a control signal can be sent to at least one real object to control the at least one real object.

Step 1220, in response to an editing operation on a first graphical code among multiple preset graphical codes, display the edited first graphical code, the first graphical code is used to add at least one object to the same formation, and the edited first graphical code is used to add at least one object to the first formation.

In some embodiments, in response to the operation of selecting at least one graphical code in the building block preset area, multiple selected graphical codes are displayed in the building block editing area 1304. Exemplarily, the code editing interface includes a first graphical code 1301 and a second graphical code 1302. Exemplarily, the first graphical code 1301 and the second graphical code 1302 can be edited respectively to obtain the edited first graphical code and the edited second graphical code.

Of course, when the default first graphical code meets the user's needs, the first graphical code may not be further edited. For example, if the first graphical code defaults to adding drone 1 to formation 1, and the user's needs are also to add drones to formation 1, the first graphical code may not be edited.

In some embodiments, the first graphical code is a graphical code for adding at least one object to the same formation. Exemplarily, the editing operation on the first graphical code includes at least one of an editing operation on an object configuration area in the first graphical code and an editing operation on a formation configuration area in the first graphical code. In some embodiments, as shown in FIG. 14 , the editing operation on the first graphical code 1400 can be considered to include at least one of an editing operation on an object configuration area 1401 in the first graphical code and an editing operation on a formation configuration area 1402 in the first graphical code. Exemplarily, the first graphical code displays the text “UAV (No. 1) joins formation (1)”, wherein the position of the brackets allows the user to edit, that is, not only the object to be added to the formation can be selected, but also the formation to be added can be selected. Exemplarily, the editing operation on the object configuration area is used to configure the object to be added to the formation, and the editing operation on the formation configuration area is used to configure the formation to be added.

In some embodiments, the editing operation on the first graphical code can be considered as an operation to edit the first graphical code. This application does not limit the specific operation type of the operation, such as the operation is at least one of a click operation, a long press operation, a sliding operation, an input operation, etc.

In some embodiments, the edited first graphical code includes identification information of the selected objects. For example, if object 1 and object 2 are selected to join formation 1, 1 and 2 are displayed in the edited first graphical code.

Step 1230, in response to an editing operation on a second graphical code among multiple preset graphical codes, display the edited second graphical code, the second graphical code is used to uniformly control at least one object in the same formation, and the edited second graphical code is used to uniformly control at least one object in the first formation.

In some embodiments, the second graphical code is a graphical code for unified control of at least one object in the first formation. Exemplarily, the editing operation on the second graphical code includes at least one of an editing operation on a formation configuration area in the second graphical code and an editing operation on a parameter configuration area in the second graphical code. In some embodiments, as shown in FIG13 , the second graphical code is displayed with the text “Set the flight speed of formation (1) to (x) meters per second”, wherein the position of the brackets allows the user to edit, that is, not only the formation can be configured, such as configuring the formation to formation 1, but also the control parameters of the graphical code can be configured, such as configuring the flight speed to 10 meters per second. Exemplarily, the editing operation on the formation configuration area is used to configure the formation controlled by the graphical code, and the editing operation on the parameter configuration area is used to configure the control parameters of the graphical code.

In some embodiments, the edited first graphical code sets object 1, object 2, and object 3 to join formation 1, and the edited second graphical code sets formation 1 to move 5m to the left. Then, when the edited first graphical code and the edited second graphical code are run, the edited second graphical code can uniformly control object 1, object 2, and object to move 5m to the left.

Step 1240 , running the graphical programming result to control at least one object in the first formation, the graphical programming result including the edited first graphical code and the edited second graphical code.

In some embodiments, before step 1240, the step further includes displaying a graphical programming result. Exemplarily, the edited graphical code displayed in the building block editing area is called a graphical programming result.

In some embodiments, the graphical programming result includes the edited first graphical code and the edited second graphical code. Of course, the graphical programming result may also include the edited third graphical code. The edited third graphical code may be the graphical code of the first type mentioned above, or the graphical code of the second type mentioned above, that is, the edited third graphical code may control one or more objects. That is, each object in the embodiment of the present application is allowed to be organized into formations and controlled together, or may be controlled individually, and the present application does not limit this. It should be understood that each object in the present application is a singleton, that is, each object independently becomes an instance, because it is impossible to group and control multiple objects together in the relevant technology, but can only be controlled separately. The technical solution provided in the embodiment of the present application can not only control a single object, but also control multiple formations at the same time, and uniformly control different objects in each formation, which is conducive to improving the diversity and flexibility of object control.

In some embodiments, a run control is displayed. In response to an operation on the run control, the graphical programming result is run. In some embodiments, as shown in FIG. 13 , in response to an operation on the run control 1305 , the graphical programming result in the building block editing area 1304 is run to control at least one object in the formation 1 .

The controls in the embodiments of the present application are all user interface (UI) controls. UI controls are any visual controls or elements that can be seen on the user interface of an application, such as pictures, input boxes, text boxes, buttons, labels and other controls. Some of these UI controls respond to user operations, such as run controls, and run graphical programming results. The UI controls involved in the embodiments of the present application include, but are not limited to: run controls.

The technical solution provided by the embodiment of the present application can add at least one object to the first formation by editing the first graphical code in the code editing interface of the graphical programming tool, and by editing the second graphical code, the edited second graphical code is used to uniformly control at least one object in the first formation, and the graphical programming result including the edited first graphical code and the edited second graphical code is run to achieve unified control of at least one object added to the first formation. The technical solution provided by the embodiment of the present application can add objects that need to execute the same code logic to the first formation through the first graphical code, so that at least one object added to the first formation shares the same edited second graphical code, without the need for repeated programming, thereby improving programming efficiency.

Please refer to Figure 15, which shows a flowchart of a graphical programming method provided by another embodiment of the present application. The execution subject of each step of the method can be the terminal device 10 introduced above, or the client of the target application running on the terminal device 10, or the server 20. In the following method embodiment, for the convenience of description, only the execution subject of each step is introduced as a "computer device". The method can include at least one of the following steps (1510-1550).

Step 1510, displaying a code editing interface of a graphical programming tool, wherein the code editing interface includes a plurality of preset graphical codes, and the graphical codes are used to process one or more objects.

Step 1520 , in response to a trigger operation on the object configuration area in the first graphical code, an object selection interface is displayed, wherein a plurality of elements are displayed in the object selection interface, and each element corresponds to an object.

In some embodiments, the first graphical code includes an object configuration area, and the object configuration area is used to configure at least one object added to the same formation.

In some embodiments, the object configuration area is a sub-area of the displayed first graphical code. The present application does not limit the display position of the object configuration area in the user interface, such as the object configuration area is in the middle of the first graphical code.

In some embodiments, the object selection interface is displayed in full screen or non-full screen. Exemplarily, when the object selection interface is displayed in non-full screen, as shown in FIG14 , in response to the object configuration area 1401 in the first graphical code, the object selection interface 1403 is displayed.

In some embodiments, the object selection interface includes multiple elements, each element corresponds to an object. Exemplarily, each element corresponds to the number of an object. For example, the objects created in the virtual scene are numbered to obtain numbers 1 to N, where N is a positive integer. Number 1 corresponds to object 1, number 2 corresponds to object 2, and so on. In other embodiments, each element corresponds to a thumbnail of an object. This form is suitable for situations where the appearance of each object is quite different. When the appearance of the object is quite different, the object can be represented by a thumbnail of the object, such as using a head picture of each object to represent the object. Exemplarily, the object selection interface includes a head picture of the object person, a head picture of the object puppy, a head picture of the object kitten, and so on. The user can determine the corresponding object by directly selecting the corresponding picture.

Of course, this application also does not limit the arrangement order and arrangement position of the multiple elements in the object selection interface. For example, they can be arranged from top to bottom in ascending order of numbers, or from left to right in the chronological order of object creation.

Step 1530 , in response to a selection operation on at least one first element among the multiple elements, displaying the edited first graphical code, and adding an object corresponding to the at least one first element to a first formation.

In some embodiments, the first element is the element to which the selection operation points. Exemplarily, the selection operation for at least one first element among the multiple elements is an operation of selecting the at least one first element. The present application does not limit the operation type of the operation, such as the selection operation is a click operation, a long press operation, a sliding operation, etc.

In some embodiments, the selection operation includes a click operation on each first element. That is, the user only needs to directly click on the element that he wants to add to the formation, and when the user wants to cancel the selection, he can directly click again. In some embodiments, as shown in 1600 of Figure 16, when selecting an element, the element can be directly clicked, and when the element is canceled, as shown in 1610 of Figure 16, the element is clicked again to cancel the selection.

In some embodiments, the selection operation is a box selection operation starting from the first position and ending at the second position, and the element included in the box selection area constructed by the first position and the second position is the first element. In some embodiments, a long press and slide from the first position to the second position forms a rectangular box for the box selection area. When the user wants to cancel the selection, just long press the box again. In some embodiments, as shown in 1700 of Figure 17, when selecting an element, you can directly long press the element to select it (that is, the box selection operation). When you deselect an element, as shown in 1710 of Figure 17, long press the selected element again to deselect the element.

In some embodiments, as shown in sub-picture a of FIG. 18 , a user can edit a graphical code in the code editing interface. As shown in sub-picture b of FIG. 18 , a user drags a plurality of graphical codes from the block preset area to the code editing area. As shown in sub-picture c of FIG. 18 , a user can edit the first graphical code 1800 therein. As shown in sub-picture d of FIG. 19 , a user can select a plurality of first elements after triggering the display object selection interface 1801. As shown in sub-picture e of FIG. 19 , after exiting the editing of the first graphical code, the edited first graphical code 1900 is displayed.

The technical solution provided by the embodiment of the present application provides two methods for selecting elements. For the first method, the selection operation is a click operation, that is, a click operation is performed once for all elements pointed to by objects that want to be included in the same formation. This method ensures the accuracy of the selection and avoids selection errors. For the second method, the selection operation is a box selection operation, that is, a large number of elements can be selected at one time. This method ensures the efficiency of the selection. When a large number of objects need to be selected to join the formation, it is faster to use the second method.

In some embodiments, in response to a trigger operation on an object configuration area in a first graphical code, states corresponding to a plurality of elements are determined according to a correspondence between elements and objects; when an element corresponds to an object created in a virtual scene, the element is in an unselected state; and when an element does not correspond to an object created in a virtual scene, the element is in an uncreated state.

Exemplarily, M elements are preset, that is, a maximum of M objects are allowed to appear in the scene, where M is a positive integer. When no more than M objects are created in the scene, there is an element without a corresponding object, that is, it is in an uncreated state.

In some embodiments, for the mth element among the multiple elements, when the mth element is selected and the state of the mth element is an uncreated state, the state of the mth element is not changed; when the mth element is selected and the state of the mth element is an unselected state, the state of the mth element is changed to a selected state, where m is a positive integer. In some embodiments, when the mth element is deselected and the state of the mth element is a selected state, the state of the mth element is changed to an unselected state.

Exemplarily, the mth element is clicked, and if the element is in an unselected state, it is determined that the element is in a selected state. Exemplarily, the element is clicked again, and the element changes from a selected state to an unselected state.

Exemplarily, the mth element is long-pressed to be selected, and if the element is in an unselected state, it is determined that the element is in a selected state. Exemplarily, the element is long-pressed again to be selected, and the element changes from a selected state to an unselected state.

Exemplarily, click or long press to select the mth element. If the element is in an uncreated state, the state of the element remains unchanged.

Please refer to Figure 20, which shows a block diagram of an object state update method provided by an embodiment of the present application. As shown in 2000 of Figure 20, taking the object as a drone as an example, click on the drone parameters (i.e., the object configuration area) in the drone formation related building block (i.e., the first graphical code), and the drone formation selection drone pop-up box (i.e., the object selection interface) pops up, where the interface of the extended pop-up box is used. Create a custom input type FieldFormationDronePopup inherited from the Blockly.FieldTextInput class, in which the click event is processed and the drone selection pop-up box (referred to as the pop-up box) is displayed. Assuming that the maximum number of drones supported in the scene is M, the pop-up box is fixed with a total of M numbers from 1 to M, representing drones 1 to M created in the order of creation. Each number has three states, namely, uncreated, unselected, and selected. The uncreated state indicates that there is no drone with this serial number in the scene. If the number of drones in the scene is n (n<M), then n+1 to M are uncreated states; the unselected state indicates that the drone with this serial number has not been selected; the selected state indicates that the drone with this serial number has been selected.

Please refer to Figure 21, which shows a block diagram of an object state update method provided by another embodiment of the present application. As shown in 2100 of Figure 21, taking the object as a drone as an example, for the click operation, if the serial number (that is, the above-mentioned element) is currently in an uncreated state, then the serial number cannot be clicked, does not respond to the click operation, and remains in an uncreated state; if the serial number is currently in an unselected state, then after clicking the serial number, the serial number is changed to a selected state, and at the same time, the drone entry corresponding to the serial number in the function panel area is highlighted and flashed to prompt the user, and the drone logo corresponding to the map navigation area is also highlighted to prompt the user; if the serial number is currently in a selected state, then after clicking the serial number, the serial number is changed to an unselected state, and at the same time, the drone entry corresponding to the serial number in the function panel area is highlighted and flashed in another way (distinguished from the above-mentioned selected state) to prompt the user, and the drone logo corresponding to the map navigation area is also highlighted in another way (distinguished from the above-mentioned selected state) to prompt the user.

Please refer to Figure 22, which shows a block diagram of an object state update method provided by another embodiment of the present application. As shown in 2200 of Figure 22, taking the object as a drone as an example, the state of the serial number (that is, the above-mentioned element) outside the box (referring to the box of the selection area) is not changed. For all serial numbers in the box, the following operations are performed. If the serial number is currently in an uncreated state, then the serial number does not change its state and remains in an uncreated state; if the serial number is currently in an unselected state, then the serial number is changed to a selected state, and at the same time, the drone entry corresponding to the serial number in the function panel area is highlighted and flashed to prompt the user, and the drone logo corresponding to the map navigation area is also highlighted to prompt the user; if the serial number is currently in a selected state, then the serial number is changed to an unselected state, and at the same time, the drone entry corresponding to the serial number in the function panel area is highlighted and flashed in another way (distinguished from the above-mentioned selected state) to prompt the user, and the drone logo corresponding to the map navigation area is also highlighted in another way (distinguished from the above-mentioned selected state) to prompt the user.

In some embodiments, the first element in the selected state is displayed differently from other elements in the unselected state. In some embodiments, the display color, brightness, grayscale, etc. of the first element in the selected state and other elements in the unselected state are different.

In some embodiments, the object corresponding to the first element in the selected state and the objects corresponding to other elements in the unselected state are displayed separately in the virtual scene. In some embodiments, when the user selects a target element in the object selection interface, the object corresponding to the target element in the selected state is highlighted in the virtual scene, while the objects pointed to by the unselected elements are not highlighted. That is, if the user selects several elements, the objects corresponding to these elements are highlighted in the virtual scene, and the highlight display here also includes flashing display, yellow display, etc.

In some embodiments, the mark of the object corresponding to the first element in the selected state in the map navigation interface is displayed separately from the marks of the objects corresponding to other elements in the unselected state in the map navigation interface, wherein the map navigation interface displays a map of the virtual scene and the marks of each created object in the virtual scene in the map. In some embodiments, the map navigation interface here corresponds to the above-mentioned map navigation area. The mark of the object corresponding to the first element in the selected state in the map navigation interface is highlighted, and the highlighting here also includes flashing display, yellow display, etc. The mark in the embodiment of the present application is also used to indicate the object, such as the mark is at least one of the head picture, number, and logo of the object.

In some embodiments, the identification of the object corresponding to the first element in the selected state in the function panel interface is displayed separately from the identification of the objects corresponding to other elements in the unselected state in the function panel interface, wherein the function panel interface is used to display the configuration information corresponding to each created object in the virtual scene, and the configuration information includes the identification of the object. In some embodiments, the function panel interface here corresponds to the above-mentioned function panel area. The identification of the object corresponding to the first element in the selected state in the function panel interface is highlighted, and the highlighting here also includes flashing display, yellow display, etc.

In some embodiments, the above-mentioned several distinctions are displayed differently.

The technical solution provided in the embodiment of the present application allows users to clearly know which objects are selected by distinguishing the selected objects from other objects in the map navigation interface, function panel interface, and object selection interface during the object selection process, thereby more intuitively observing whether the selection is wrong, so that users can make timely modifications.

In some embodiments, the triggering operation on the object configuration area in the first graphical code and the selection operation on at least one first element among the multiple elements are two-step operations, or they can be a one-step sliding operation without releasing the hand. When the triggering operation on the object configuration area in the first graphical code and the selection operation on at least one first element among the multiple elements are a one-step sliding operation without releasing the hand, for example, the triggering operation on the object configuration area in the first graphical code is long pressed to call out the object selection interface, and without releasing the hand, elements to be added to the same formation can be continuously selected on the object interface to reduce the operation steps.

In some embodiments, at least one object joining the same formation is configured using the method shown in step 1520 and step 1530. In other embodiments, identification information of at least one object joining the same formation can be directly input in the object configuration area, and at least one object joining the first formation is determined based on the input identification information of the at least one object. The input can be text input or voice input.

The above steps 1520 and 1530 illustrate a method for configuring at least one object to join the same formation. In other embodiments, the following steps 1560 (not shown in the figure) and 1570 (not shown in the figure) are also used to configure at least one object to join the same formation.

Step 1560, in response to a trigger operation on the object configuration area in the first graphical code, a virtual scene interface in an editing state is displayed, wherein the virtual scene interface displays a virtual scene and objects created in the virtual scene, and different objects have different positions in the virtual scene.

Exemplarily, objects in the virtual scene interface in the editing state are allowed to be edited, which is distinguished from the previous virtual scene interface.

Exemplarily, similar to the object selection interface 1403 in FIG. 14 , the virtual scene interface in the editing state is displayed in a non-full screen format.

Exemplarily, the virtual scene interface in the editing state is displayed in full screen form. That is, in response to the triggering operation on the object configuration area in the first graphical code, the virtual scene interface in the editing state is displayed in full screen.

Step 1570 , in response to a selection operation on at least one first object among the created objects, displaying the edited first graphical code, and adding the at least one first object to the first formation.

In some embodiments, the selection operation for at least one first object among the created objects is an operation of selecting the at least one object. In some embodiments, the selection operation includes a click operation for each object. In some embodiments, the selection operation is a box selection operation starting from the third position and ending at the fourth position, and the objects included in the box selection area constructed by the third position and the fourth position are the selected objects. This is similar to the above, and reference is made to the above explanations, and no further elaboration is given.

In some embodiments, the edited first graphical code includes identification information of the selected objects. For example, if object 1 and object 2 are selected to join formation 1, identification 1 and identification 2 are displayed in the edited first graphical code.

The technical solution provided in the embodiment of the present application allows the user to directly select objects in the virtual scene to join the formation, making the selection of objects more intuitive and less prone to errors, which is conducive to improving programming efficiency.

The above embodiment specifically introduces how to configure the object selected to join the formation. Of course, the formation pointed to by the first graphical code can also be configured.

In some embodiments, the first graphical code also includes a formation configuration area, and the formation configuration area is used to configure a formation to which at least one object joins.

In some embodiments, in response to a trigger operation on a formation configuration area in a first graphical code, a formation selection interface is displayed, in which multiple formation numbers are displayed, and each formation number corresponds to a formation; in response to a selection operation on a first formation number among multiple formation numbers, an edited first graphical code is displayed, and the formation corresponding to the first formation number is the first formation.

In some embodiments, the triggering operation for the formation configuration area in the first graphical code and the selection operation for the first formation number among multiple formation numbers are two-step operations, or they can be a one-step sliding operation without letting go. When the triggering operation for the formation configuration area in the first graphical code and the selection operation for the first formation number among multiple formation numbers are a one-step sliding operation without letting go, for example, long pressing the triggering operation of the formation configuration area in the first graphical code to call out the formation selection interface, without letting go, continue to slide to the first formation number on the object interface to reduce the number of operation steps.

In other embodiments, the identification information of the formation to be joined can be directly input in the formation configuration area, and the formation to be pointed to is determined according to the input identification information of the formation. The input can be text input or voice input.

Step 1540, in response to an editing operation on a second graphical code among multiple preset graphical codes, display the edited second graphical code, the second graphical code is used to uniformly control at least one object in the same formation, and the edited second graphical code is used to uniformly control at least one object in the first formation.

Step 1550 , running the graphical programming result to control at least one object in the first formation, the graphical programming result including the edited first graphical code and the edited second graphical code.

In some embodiments, the graphical programming result is run to display the control process of at least one object in the first formation in the virtual scene. If the graphical programming result requires that formation 1 be controlled to fly 10 meters to the left, and formation 1 includes object 1 and object 2, then an animation of controlling both object 1 and object 2 to fly 10 meters to the left is displayed in the virtual scene. In some embodiments, as shown in FIG23 , in response to an operation on the running control 2301, the graphical programming result is run to display the control process 2302 of at least one object in the first formation in the virtual scene.

The technical solution provided in the embodiment of the present application can provide the running results to the user in an animated manner for preview, which is conducive to the user to quickly discover problems in the graphical programming results, so as to quickly modify them and improve programming efficiency.

In some embodiments, at least one of a scene control and a code editing control is displayed; the scene control is used to switch the display of a virtual scene interface, which includes a virtual scene and objects created in the virtual scene; the code editing control is used to switch the display of a code editing interface. The graphical programming result is run, and in response to a trigger operation on the scene control, the virtual scene interface is displayed, and the control process of at least one object in the first formation in the virtual scene is displayed in the virtual scene interface; in response to a trigger operation on the code editing control, the code editing interface is redisplayed, and the code editing interface includes the graphical programming result.

In some embodiments, as shown in FIG. 24 , a scene control 2401 and a code editing control 2402 are displayed. In response to a trigger operation on the scene control 2401, a virtual scene interface 2400 is displayed. In response to a trigger operation on the code editing control 2402, a code editing interface 2404 is displayed. That is, through the scene control and the code editing control, the switching between the virtual scene interface and the code editing interface can be realized. Of course, the two can also be displayed on the same screen, such as displaying the virtual scene interface on the left and the code editing interface on the right. Of course, the code editing interface can also be displayed in full screen, and the virtual scene interface can be displayed in a picture-in-picture format.

In some embodiments, in response to a formation setting operation for a target object among objects created in a virtual scene interface, a formation selection interface is displayed, wherein the formation selection interface includes multiple formation numbers, each formation number corresponding to a formation; in response to a selection operation for a second formation number among multiple formation numbers, the target object is determined to join the formation corresponding to the second formation number. In this way, the formation of an object can be modified directly in the virtual scene interface, such as long pressing the target object in the virtual scene interface to display a formation selection interface; in response to a selection operation for a second formation number among multiple formation numbers, the target object is determined to join the formation corresponding to the second formation number. By directly modifying the formation in the scene and combining the code with the scene, it is beneficial to improve the flexibility of code editing and improve programming efficiency.

Exemplarily, modifications to the formation in interfaces other than the code editing interface will be synchronized to the edited first graphical code and the edited second graphical code, so that the scene and the code remain synchronized to avoid errors.

The technical solution provided in the embodiment of the present application switches the display of the virtual scene interface and the code editing interface or displays them on the same screen, making it convenient for users to edit the code and view the control process of each object in the scene at the same time, which is conducive to improving programming efficiency.

Please refer to Figure 25, which shows a flowchart of a graphical programming method provided by another embodiment of the present application. The execution subject of each step of the method can be the terminal device 10 introduced above, or the client of the target application running on the terminal device 10, or the server 20. In the following method embodiment, for the convenience of description, only the execution subject of each step is introduced as a "computer device". The method can include at least one of the following steps (2510-2570).

Step 2510, in response to a triggering operation on an object creation control in a function panel interface, configuration information corresponding to a newly added object is displayed in the function panel interface, and the function panel interface is used to display configuration information corresponding to each created object in a virtual scene.

In some embodiments, in response to a triggering operation on an object creation control in a function panel interface, configuration information corresponding to a newly added object displayed in the function panel interface is initialized information, or in other words, default configuration information.

In some embodiments, in response to a triggering operation on an object creation control in the function panel interface, a newly added object is displayed in the virtual scene.

In some embodiments, as shown in FIG. 26 , in response to a trigger operation on an object creation control 2600 in a function panel interface, configuration information 2601 corresponding to a newly added object is displayed in the function panel interface.

Step 2520: In response to the modification operation on the configuration information corresponding to the newly added object, display the modified configuration information.

In other embodiments, in addition to modifying the configuration information corresponding to the newly added object, the configuration information corresponding to other created objects may also be modified.

In some embodiments, as shown in FIG. 27 , in response to a modification operation on configuration information 2701 corresponding to a created object, the modified configuration information is displayed.

Step 2530: Display the newly added object created based on the modified configuration information in the virtual scene.

In some embodiments, the newly added object is modified in the virtual scene based on the modified configuration information.

In some embodiments, an object 2702 corresponding to the modified configuration information is displayed in the virtual scene.

In some embodiments, the function panel interface includes a search input field.

Exemplarily, the search bar is used to search for a created object and view configuration information of the object.

In some embodiments, in response to an input operation on a search input bar, search information for a created object input in the search input bar is displayed; and configuration information corresponding to the object pointed to by the search information is displayed.

In some embodiments, the computer device matches the keywords in the search information with the configuration information of each created object according to the search information input by the user, and determines the created object with the highest matching degree as the object pointed to by the search information. Further, the configuration information corresponding to the object pointed to by the search information is displayed.

Considering that when a user has created a large number of objects, it is impossible to display all the configuration information of all the created objects in one interface, so the user needs to pull down or scroll to find the relevant object. Therefore, the technical solution provided by the embodiment of the present application provides a search bar to facilitate the user to quickly search for the relevant object, so as to quickly view the configuration information of the object and modify the configuration information, which is conducive to improving information processing efficiency.

In some embodiments, the function panel interface includes tabs corresponding to each created object.

The tabs in the embodiments of the present application should be interpreted broadly, that is, each object corresponds to a tab, and the tab is used to characterize a specific element of the object, such as a control, logo, thumbnail, etc. The reason why the tabs are displayed is that the tab area is small, and more tabs can be displayed in a smaller area, that is, the tabs corresponding to multiple created objects can be displayed in a small area. For example, M labels are displayed in the function panel interface, and each label corresponds to a created object.

In some embodiments, in response to the selection operation of the target tab in the tabs corresponding to each created object, the configuration information corresponding to the object pointed to by the target tab is displayed. Exemplarily, the configuration information of the object pointed to by the target tab is quickly displayed by selecting the target tab. This method improves the efficiency of information display.

In some embodiments, in response to a delete operation on a target tab, the object pointed to by the target tab is deleted from the virtual scene. Exemplarily, deleting the target tab can quickly delete the object pointed to by the tab. This method improves the efficiency of object deletion.

In some embodiments, in response to a formation setting operation for a target tab, the formation to which the object pointed to by the target tab joins is changed. Exemplarily, in response to a formation setting operation for a target tab, a formation selection interface is displayed, and the formation to which the object pointed to by the target tab joins is reselected on the formation selection interface. This method realizes quick configuration of formations, which is conducive to improving programming efficiency.

In some embodiments, the function panel interface also displays controls corresponding to each formation. Exemplarily, in response to a click operation on a target control in a control corresponding to each formation, the configured objects under the formation corresponding to the target control are displayed. Exemplarily, when the configured objects under the formation corresponding to the target control are displayed, all objects under the formation and other objects in the virtual scene are displayed separately. Exemplarily, objects in the formation corresponding to the target control can be added or deleted. Exemplarily, the objects under each formation can also be quickly viewed in the function panel interface, and the location of each object under the formation can also be viewed in the scene. In addition, objects can be quickly deleted or added, which is conducive to improving the flexibility and efficiency of programming.

Step 2540, displaying a code editing interface of the graphical programming tool, wherein the code editing interface includes a plurality of preset graphical codes, and the graphical codes are used to process one or more objects.

Step 2550, in response to an editing operation on a first graphical code among multiple preset graphical codes, display the edited first graphical code, the first graphical code is used to add at least one object to the same formation, and the edited first graphical code is used to add at least one object to the first formation.

Step 2560, in response to an editing operation on a second graphical code among multiple preset graphical codes, the edited second graphical code is displayed, the second graphical code is used to uniformly control at least one object in the same formation, and the edited second graphical code is used to uniformly control at least one object in the first formation.

Step 2570, running the graphical programming result to control at least one object in the first formation, the graphical programming result including the edited first graphical code and the edited second graphical code.

The above is an introduction to the graphical programming method. The following is a detailed explanation of how to run the graphical programming results after programming using the graphical programming tool in conjunction with the following embodiments.

Of course, it should be understood that the following graphical code execution method and the above-mentioned graphical programming method can be executed by the same computer device or by different computer devices. When executed by the same computer device, an application program that supports programming functions and code execution functions is running on the computer device. When executed by different computer devices, the first computer device executes the above-mentioned graphical programming method to obtain a graphical programming result, and the first computer device sends the graphical programming result to the second computer device, and the second computer device executes the graphical programming result. An application program that supports programming functions is running on the first computer device, and an application program that supports code execution functions is running on the second computer device.

It should also be noted that the edited first graphical code and the edited second graphical code included in the graphical programming result in the graphical programming method are considered to be the first graphical code and the second graphical code in the graphical programming result mentioned in the graphical code execution method described below. Since the following embodiment is the code execution side, the graphical code mentioned in the following embodiment is considered to be edited, that is, in the following embodiment, the first graphical code directly replaces the edited first graphical code, the second graphical code replaces the edited second graphical code, and the third graphical code is considered to be the edited third graphical code.

Please refer to Figure 28, which shows a flowchart of a graphical code execution method provided by an embodiment of the present application. The execution subject of each step of the method can be the terminal device 10 introduced above, or the client of the target application running on the terminal device 10, or the server 20. In the following method embodiment, for the convenience of description, only the execution subject of each step is introduced as a "computer device". The method can include at least one of the following steps (2810-2830).

Step 2810, obtaining a graphical programming result, the graphical programming result including a plurality of graphical codes executed in sequence, the graphical codes being used to process one or more virtual objects in the virtual scene.

In some embodiments, a graphical programming result is obtained, and the graphical programming result includes a plurality of graphical codes, which are graphical codes edited by a programmer. Exemplarily, the graphical codes in the graphical programming result are arranged in a certain order, such as from top to bottom, from left to right, and so on. Exemplarily, when the graphical programming result is executed, the graphical codes in the graphical programming result are executed in sequence. In some embodiments, each graphical code corresponds to a control code logic, that is, a programming language encapsulated in the graphical code. Exemplarily, different graphical codes correspond to different control code logics, that is, different programming languages encapsulated in the graphical code.

In some embodiments, the virtual scene is a virtual environment created (or provided) by the application program for the user. The virtual environment can be a simulation of the real world, a semi-simulated and semi-imaginary three-dimensional world, or a purely imaginary three-dimensional world. The virtual environment can be any one of a two-dimensional virtual environment, a 2.5-dimensional virtual environment, and a three-dimensional virtual environment.

In some embodiments, a virtual object is an movable object in a virtual environment. The movable object may be at least one of a virtual drone, a virtual character, a virtual animal, and a cartoon character. In some embodiments, when the virtual environment is a three-dimensional virtual environment, the virtual object may be a three-dimensional virtual model, and each virtual object has its own shape and volume in the three-dimensional virtual environment, occupying a portion of the space in the three-dimensional virtual environment. Optionally, the virtual object is a three-dimensional character constructed based on three-dimensional human skeleton technology, and the virtual object achieves different external images by wearing different skins. In some implementations, the virtual object may also be implemented using a 2.5-dimensional or 2-dimensional model, which is not limited in the embodiments of the present application.

Step 2820, executing the first graphical code included in the graphical programming result, and adding at least one virtual object in the virtual scene to the first formation.

In some embodiments, the first graphical code is a graphical code for adding at least one virtual object in a virtual scene to a first formation. In response to a user's editing operation on the first graphical code during the programming process, the edited first graphical code includes first configuration information. The first configuration information includes identification information of the first formation and identification information corresponding to at least one virtual object added to the first formation. In some embodiments, the identification information here is information used to characterize the formation or object, such as number, label, serial number, etc. As shown in Figure 19, the edited first graphical code 1900 shows that drones 1 and 2 have joined formation 1, then drones 1 and 2 and the formation are the first configuration information included in the edited first graphical code.

In some embodiments, when the code is executed, at least one virtual object in the virtual scene is added to the first formation according to the first configuration information included in the first graphical code.

Exemplarily, the identification information corresponding to at least one virtual object joining the first formation in the first configuration information is bound to the identification information of the first formation, that is, at least one virtual object joining the first formation can be quickly found based on the identification information of the first formation.

Exemplarily, a correspondence table between virtual objects and formations is established based on the identification information corresponding to at least one virtual object joining the first formation in the first configuration information and the identification information of the first formation. That is, according to the identification information of the first formation, at least one virtual object joining the first formation can be quickly found from the correspondence table.

Exemplarily, according to the identification information corresponding to at least one virtual object added to the first formation in the first configuration information and the identification information of the first formation, a dynamic link is established between the first formation and at least one virtual object added to the first formation. For example, a dynamic link is established between formation 1 and virtual object 1, a dynamic link is established between formation 1 and virtual object 2, and a dynamic link is established between formation 1 and virtual object 3. That is, when the second graphical code is executed, that is, when each object in the formation is controlled, the dynamic link of the first formation is used to link to the virtual object added to the first formation. This method directly constructs a dynamic link to find the virtual object added to the formation, which not only avoids the construction of a correspondence table or a binding relationship, but also enables rapid positioning to improve search efficiency.

Step 2830, executing the second graphical code included in the graphical programming result to uniformly control at least one virtual object added to the first formation.

In some embodiments, the second graphical code is a graphical code for unified control of each virtual object in the first formation. In response to the user's editing operation on the second graphical code during the programming process, the edited second graphical code includes second configuration information. The second configuration information includes identification information and control parameter information of the first formation.

The technical solution provided by the embodiment of the present application obtains the graphical programming result; executes the first graphical code included in the graphical programming result, thereby realizing the addition of at least one virtual object in the virtual scene to the first formation; and executes the second graphical code included in the graphical programming result, thereby realizing the unified control of at least one virtual object added to the first formation. The technical solution provided by the embodiment of the present application uses the first graphical code to organize objects that need to execute the same code logic into the first formation, so that at least one object added to the first formation shares a second graphical code, and the execution of the second graphical code can realize the unified control of at least one virtual object in the same formation, without the need to obtain repeated programming codes to control each virtual object separately, which is conducive to improving the execution efficiency of the code.

Please refer to Figure 29, which shows a flowchart of a graphical code execution method provided by another embodiment of the present application. The execution subject of each step of the method can be the terminal device 10 introduced above, or the client of the target application running on the terminal device 10, or the server 20. In the following method embodiment, for the convenience of description, only the execution subject of each step is introduced as a "computer device". The method can include at least one of the following steps (2910-2950).

Step 2910, obtaining a graphical programming result, where the graphical programming result includes a plurality of graphical codes executed in sequence, and the graphical codes are used to process one or more virtual objects in the virtual scene.

Step 2920: execute the first graphical code included in the graphical programming result to add at least one virtual object in the virtual scene to the first formation.

Step 2930, obtaining a control logic code corresponding to the second graphical code, where the control logic code refers to a programming language encapsulated in the second graphical code.

In some embodiments, the programming language encapsulated in the second graphical code is acquired to obtain the control code logic corresponding to the second graphical code.

Step 2940, determine at least one virtual object to join the first formation.

In some embodiments, since the identification information corresponding to at least one virtual object joining the first formation and the identification information of the first formation have been bound, at least one virtual object joining the first formation can be quickly found directly based on the identification information of the first formation.

Exemplarily, since a correspondence table is established between virtual objects and formations, at least one virtual object added to the first formation can be quickly found from the correspondence table directly based on the identification information of the first formation.

Exemplarily, since dynamic links have been established between the first formation and at least one virtual object that joins the first formation, the dynamic link of the first formation is directly used to link to at least one virtual object that joins the first formation. This method of directly building a dynamic link to find the virtual object that joins the formation can improve the search efficiency.

Step 2950: execute the control logic code to uniformly control at least one virtual object added to the first formation.

In some embodiments, after determining at least one virtual object, the control logic code is used to control the at least one virtual object separately, that is, the at least one virtual object is controlled separately at the same time. Exemplarily, the control code logic is a programming language that controls flying 1 meter to the left, and this programming language is used to control at least one virtual object in the formation.

In some embodiments, when controlling at least one virtual object, there is no order of precedence. Exemplarily, the target application includes multiple executors, and the sth executor is used to execute the control logic code to control the sth virtual object added to the first formation, where s is a positive integer. Multiple executors synchronously control virtual objects, which is conducive to speeding up processing speed and improving code execution efficiency.

In some embodiments, there is a second graphical code, which has an execution condition, that is, the second graphical code can be executed only when the virtual object satisfies the condition. The execution condition of the second graphical code is described below as an example.

In some embodiments, the above method further includes at least one of the following steps S1 to S3 (not shown in the figure).

Step S1, obtaining state information corresponding to at least one virtual object added to the first formation, where the state information is used to reflect the real-time state of the virtual object.

In some embodiments, the state information includes the current position information, color information, shape information, etc. of the virtual object. In some embodiments, before executing the second graphical code, it is determined whether at least one virtual object of the first formation meets the execution condition of the second graphical code.

Step S2, when the status information corresponding to at least one virtual object added to the first formation does not meet the execution condition of the second graphical code, continue to control the at least one virtual object added to the first formation until the status information corresponding to at least one virtual object added to the first formation meets the execution condition of the second graphical code.

Exemplarily, the execution condition of the second graphical code is that the longest distance between the two virtual objects joining the first formation does not exceed 5 meters, then the state information corresponding to at least one virtual object joining the first formation is obtained, and if it is determined that the longest distance between the two virtual objects joining the first formation is 3 meters, then the execution of the second graphical code is allowed at this time. Exemplarily, if it is determined that the longest distance between the two virtual objects joining the first formation is 6 meters, then the execution of the second graphical code is not allowed at this time. Exemplarily, when the execution of the second graphical code is not allowed, it is necessary to continue to control the at least one virtual object joining the first formation until the state information corresponding to the at least one virtual object joining the first formation meets the execution condition of the second graphical code.

Step S3, when the state information corresponding to the at least one virtual object added to the first formation meets the execution condition of the second graphical code, the control logic code is executed to uniformly control the at least one virtual object added to the first formation.

The above steps S1 to S3 introduce how to execute graphical codes simultaneously. Taking the graphical code as a building block code as an example, when a drone in the group executes a building block, if the current state does not meet the prerequisite for executing the building block, other drones need to wait for the drone to adjust its state to a state that satisfies the execution of the building block. The drone formation executor will check the status of all drones in the group. If it meets the requirements, the building block will be executed; if not, the drone that does not meet the execution of the building block will adjust its state, and the drone formation executor (see the explanation of the following embodiment) will check the status of all drones in the group again in the next main loop of the life cycle, and repeat the cycle until all drones meet the state of executing the building block. For example, there are 3 drones in drone formation A, namely 1, 2, and 3. To let formation A perform circular motion around a point p, it is necessary to wait for drones 1, 2, and 3 to spin to a direction perpendicular to the current position and the line connecting the center of the circle, and then perform circular motion together.

In some embodiments, the above method further includes at least one of the following steps S4 to S6 (not shown in the figure).

Step S4, for the kth virtual object among the at least one virtual object, after executing the control logic code to control the kth virtual object, determining that the kth virtual object is in a state where the execution of the second graphical code is completed, where k is a positive integer.

In some embodiments, a flag is set for all virtual objects, and when the kth virtual object is in a state where the execution of the second graphical code is completed, the value of the flag is a first value.

Exemplarily, when the control logic code is executed to control the kth virtual object, the value of the flag bit of the kth virtual object is changed from the second value to the first value. Exemplarily, the original value of the flag bit is the second value.

Step S5, when the control logic code is executed and the kth virtual object is not controlled, it is determined that the kth virtual object is in a state where the second graphical code has not been executed, and the first value represents the execution completion state.

In some embodiments, when the kth virtual object is in an incomplete execution state of the second graphical code, the value of the flag bit is a second numerical value, and the second numerical value represents the incomplete execution state.

Exemplarily, when the control logic code is executed and the kth virtual object is not controlled, the value of the flag bit of the kth virtual object is not changed. Exemplarily, the original value of the flag bit is the second value.

Step S6, traverse at least one virtual object added to the first formation, and when at least one virtual object added to the first formation is in a state where the execution of the second graphical code is completed, execute the third graphical code, where the third graphical code is the graphical code that follows the second graphical code in the graphical programming result.

In some embodiments, when the flag bits of at least one virtual object added to the first formation are all the first value, it is considered that all of the at least one virtual object added to the first formation is in a state where the execution of the second graphical code is completed, and the execution of the next graphical code of the second graphical code is allowed. In some embodiments, when one of the flag bits of at least one virtual object added to the first formation is the second value, it is considered that not all of the at least one virtual object added to the first formation is in a state where the execution of the second graphical code is completed, and the execution of the next graphical code of the second graphical code is not allowed.

The above steps S4 to S6 introduce how to determine whether the graphical code has been executed. When the building block starts to execute, a flag (completed) is set to false for all drones in the group. When the building block is executed, the value of the flag is changed to true. The drone formation executor checks the value of the flag of the drones in the group in each main loop of the life cycle. If the flag of any drone is false, it means that the execution of the building block of this group of drones has not been completed. Otherwise, it means that the building block of this group of drones has been executed and the subsequent building blocks can continue to be executed. For example, there are 3 drones in drone formation A, namely 1, 2, and 3. To move formation A to a certain point p, it is necessary to wait until drones 1, 2, and 3 have all flown to point p before executing the subsequent building blocks.

In some embodiments, the code type of the second graphical code is the first type or the second type.

In some embodiments, when the code type of the second graphical code is the first type, the control type of the second graphical code for at least one virtual object in the first formation is instantaneous control, and the instantaneous control refers to modifying the attributes of the virtual object, such as modifying the color of the virtual object, modifying the light of the virtual object, modifying the clothes of the virtual object, etc.

In some embodiments, when the code type of the second graphical code is the second type, the control type of the second graphical code for at least one virtual object in the first formation is continuous control, and continuous control refers to controlling the virtual object for a first duration, such as controlling the virtual object to fly left at speed a for 1 minute, controlling the virtual object to fly around point b for 5 seconds, controlling the virtual object to stay still for 1 hour, and so on.

In some embodiments, when the code type of the second graphical code is the second type, before executing the second graphical code, it is necessary to determine whether at least one virtual object joining the first formation meets the execution condition. Exemplarily, when the code type of the second graphical code is the second type, when executing the second graphical code, it is necessary to determine whether the execution of the second graphical code is completed.

In some embodiments, when the code type of the second graphical code is the second type, before executing the second graphical code, it is not necessary to determine whether the at least one virtual object joining the first formation meets the execution condition. Exemplarily, when the code type of the second graphical code is the second type, when executing the second graphical code, it is not necessary to determine whether the execution of the second graphical code is completed.

In some embodiments, when the code type of the second graphical code is the first type, before executing the second graphical code, it is necessary to determine whether at least one virtual object joining the first formation meets the execution condition. Exemplarily, when the code type of the second graphical code is the first type, when executing the second graphical code, it is necessary to determine whether the execution of the second graphical code is completed.

In some embodiments, when the code type of the second graphical code is the first type, before executing the second graphical code, it is not necessary to determine whether the at least one virtual object joining the first formation meets the execution condition. Exemplarily, when the code type of the second graphical code is the first type, when executing the second graphical code, it is not necessary to determine whether the execution of the second graphical code is completed.

In some embodiments, the drone formation blocks (i.e., graphical codes) can be divided into two categories according to the execution time of the blocks. The first category (i.e., the graphical codes of the first type mentioned above) are blocks that do not need to be executed for a period of time (such as setting properties such as speed or light, and the block is considered to be executed after the properties are modified), and no special processing is required; the second category (i.e., the graphical codes of the second type mentioned above) are blocks that need to be executed for a period of time (such as a drone flying forward at a speed of t for t time, and the block is considered to be executed after t time), and a group of drones executing blocks for a period of time requires special processing. Exemplarily, the special processing includes the above steps S1 to S6, that is, judging whether at least one virtual object joining the first formation meets the execution conditions and judging whether the graphical code has been executed.

The embodiment of the present application takes into account that the graphical code of instantaneous control generally only needs to modify the value of the corresponding position, and the possibility of error is small, so there is no need to judge whether the execution conditions are met and whether the execution is completed, while the graphical code of continuous control usually needs to be executed for a period of time, especially when there are many virtual objects in the formation, it is relatively easy to make mistakes, so for the graphical code of continuous control, it is necessary to judge whether the execution conditions are met and whether the execution is completed. This processing method is conducive to ensuring the smooth execution of the graphical code and reducing the possibility of errors.

The technical solution provided in the embodiment of the present application also provides an error correction function for the formation.

Exemplarily, after obtaining at least one virtual object added to the formation according to the first graphical code, the rationality of the at least one virtual object being included in the same team is judged. If the judgment result is unreasonable, the execution of the code is suspended, and a formation error correction prompt message is sent. The formation error correction prompt message is used to be displayed on the user interface for the user to view, so as to modify the formation in time. If the judgment result is reasonable, the formation error correction prompt message is not sent. Exemplarily, the formation error correction prompt message includes the status information of the unreasonable virtual object, and the unreasonable virtual object is a virtual object that is considered not to be included in the formation.

Exemplarily, the position information of each virtual object added to the formation is obtained. When there is a virtual object with a large position deviation, the judgment result is unreasonable. For example, if there are 10 virtual objects in a formation, the distance between 9 virtual objects is less than the threshold, and only the distance between the 10th virtual object and the 9 virtual objects is greater than the threshold, then it is considered that there is an error in the 10th virtual object, and the judgment result is unreasonable. When there is no virtual object with a large position deviation, the judgment result is reasonable.

Exemplarily, historical formation information (historical formations and objects that joined the formation) is obtained, and the current formation information is compared with the historical formation information. When the error is greater than a threshold, the result is judged to be unreasonable. Exemplarily, historical formation information (historical formations and objects that joined the formation) is obtained, and the current formation information is compared with the historical formation information. When the error is less than a threshold, the result is judged to be reasonable.

The technical solution provided by the embodiment of the present application is that each time a graphical programming result is obtained from the graphical programming result, when the first graphical code is detected according to the execution order, the rationality is first judged, and then the first graphical code is executed after it is reasonable. This method can avoid explicit errors in advance and report them to the user in time, which is conducive to the user to modify them in time, thereby improving the execution efficiency of the code.

Please refer to Figure 30, which shows a flowchart of a graphical code execution method provided by an embodiment of the present application. The execution subject of each step of the method can be the terminal device 10 introduced above, or the client of the target application running on the terminal device 10, or the server 20. In the following method embodiment, for the convenience of description, only the execution subject of each step is introduced as a "computer device". The method can include at least one of the following steps (3010-3030).

Step 3010, obtaining a graphical programming result, where the graphical programming result includes a plurality of graphical codes executed in sequence, and the graphical codes are used to process one or more virtual objects in the virtual scene.

Step 3020, based on the first configuration information, update the first correspondence table to obtain an updated first correspondence table, the first correspondence table is used to record the correspondence between the formation and the virtual object, and the updated first correspondence table records the correspondence between the identification information of the first formation and the identification information corresponding to at least one virtual object added to the first formation.

In some embodiments, when executing the graphical programming result, each time the first graphical code is encountered, the first correspondence table is updated once. When the first graphical code is encountered for the first time in executing the graphical programming result, the first correspondence table is created according to the first configuration information in the first graphical code. When the first graphical code is encountered again subsequently, only the first correspondence table needs to be updated.

In some embodiments, the first graphical code includes first configuration information, the first configuration information is used to configure virtual objects included in the first formation, and the first configuration information includes identification information of the first formation and identification information corresponding to at least one virtual object added to the first formation.

In some embodiments, before step 3030, the second graphical code is compiled, and if the compilation is successful, the compiled control logic code corresponding to the second graphical code is obtained, and the compiled control logic code is executed to control at least one virtual object added to the first formation, and the control logic code refers to the programming language encapsulated in the second graphical code; if the compilation is unsuccessful, a modification prompt information is sent, and the modification prompt information is used to indicate that the second graphical code is edited. Exemplarily, the modification prompt information includes text prompt information, voice broadcast information, picture prompt information, etc.

In some embodiments, when executing the code, the programming language encapsulated in the graphical code needs to be compiled to obtain the compiled code that can be executed. For example, the graphical code is translated into C# code that can be run by Unity. In some embodiments, in addition to compiling the second graphical code, other graphical codes in the graphical programming result also need to be compiled.

In some embodiments, as shown in 3100 of FIG. 31 , the user operates the graphical code, operates the graphical code (building block) in the building block editing interface (including dragging, combining, etc.), and clicks the run control to compile the graphical code. After the compiler compiles, if there is an error, it will prompt that there is a problem with the code, and ask the user to modify it and try again; if there is no error, the code will start to run.

Step 3030: execute the second graphical code included in the graphical programming result to uniformly control at least one virtual object added to the first formation.

In some embodiments, the second graphical code includes second configuration information, the second configuration information is used to indicate the first formation and parameters for controlling the first formation, and the second configuration information includes identification information and control parameter information of the first formation.

In some embodiments, the identification information of the first formation is used to search for at least one virtual object added to the first formation from the updated first correspondence table, and the control parameter information is used to modify the original logic code corresponding to the second graphical code to obtain the control logic code corresponding to the second graphical code, where the control logic code refers to the programming language encapsulated in the second graphical code.

In some embodiments, the identification information here is information used to characterize the formation, such as number, label, serial number, etc. In some embodiments, the second graphical code is "Formation 1 flies 1 meter to the left", then the identification information of the first formation included in the second configuration information is "Formation 1", and the control parameter information is "left, 1 meter". Exemplarily, the original control code logic corresponding to the second graphical code is consistent, that is, the programming language encapsulated in the second graphical code is consistent, then whether flying left or right, only the control parameters are different, and the original control code logic is consistent, so you only need to modify the control parameters in the programming language to get the control logic code corresponding to the second graphical code.

In some embodiments, the control parameter information in the second configuration information is used to modify the programming language encapsulated in the second graphical code to obtain a modified programming language, and the modified programming language is used to control at least one virtual object added to the first formation.

In some embodiments, third configuration information is obtained, and the third configuration information is used to configure the virtual object created in the virtual scene. The third configuration information includes identification information of the created virtual object and instance information of the created virtual object, and the instance information is used to indicate configuration information corresponding to the virtual object.

Exemplarily, the user can create a new virtual object in the graphical programming tool, and can also modify the configuration information of the new virtual object, so the instance information of the virtual object is determined according to the last saved created virtual object and its corresponding configuration information.

In some embodiments, the second correspondence table is updated according to the third configuration information to obtain an updated second correspondence table, wherein the second correspondence table is used to record the correspondence between the virtual object and the instance information, and the instance information is used to provide configuration information corresponding to the virtual object when executing the second graphical code to control the virtual object.

Exemplarily, the second correspondence table is constructed according to the instance information of the original created virtual object.

In some embodiments, for a first virtual object among the created virtual objects, when executing the second graphical code to control the first virtual object, instance information of the first virtual object is found from the updated second correspondence table according to identification information of the first virtual object.

Exemplarily, the computer device pre-stores an original second correspondence table, and the second correspondence table is used to record the correspondence between the virtual object and the instance information. That is, the virtual objects configured in the virtual scene are recorded in a table form. Exemplarily, the identification information of each virtual object corresponds to the instance information of the virtual object. Exemplarily, the instance information includes the configuration information of the virtual object, and the configuration information includes the position, color, light, etc. of the virtual object.

In some embodiments, the instance information of the first virtual object in the second correspondence table is updated according to the control result of the first virtual object.

Exemplarily, when the control logic code corresponding to the second graphical code is used to control the first virtual object, the virtual object is controlled based on the instance information of the virtual object. Therefore, when the control starts, the corresponding instance information needs to be found. When the control ends, the configuration information of the virtual object changes, and the changed configuration information needs to be used to update the instance information of the first virtual object.

In some embodiments, when the second graphical code is executed and the first virtual object is not controlled, the instance information of the first virtual object in the second correspondence table is not updated.

Exemplarily, when the second graphical code is executed and the first virtual object is not controlled, it means that the first virtual object is not controlled, and the configuration information is not changed, and there is no need to update the instance information of the first virtual object in the second correspondence table.

Still taking the virtual drone as an example, the core of code control of drone formation lies in the implementation at runtime. The drone formation runtime relies on the life cycle provided by Unity. In each main loop of the life cycle, the drone status is updated according to the attributes of each drone (attributes refer to the basic attributes of the drone, such as speed and direction. The motion of the drone in this frame is calculated according to the attributes, and the position, orientation and other states of the drone are updated). The target application of drone formation runtime includes two parts: formation drone manager and drone formation executor.

Please refer to Figure 32, which shows a schematic diagram of the component structure provided by an embodiment of the present application. As shown in 3200 of Figure 32, the formation drone manager class is a singleton (a singleton is a creational design pattern that ensures that a class has only one instance and provides a global node for accessing the instance), and provides the following functions to the outside, including adding a new drone, deleting a drone, modifying drone information, obtaining a drone instance through the drone's identification information (a drone instance refers to the specific implementation of a drone object), adding a drone to a formation, deleting a drone from a formation, and obtaining a list of a group of drone identification information through the formation's identification information. All operations of the user on the drone in the function panel interface are managed by the formation drone manager. Exemplarily, the formation drone manager maintains each virtual object in the formation and the instance information of each virtual object. Exemplarily, the formation drone manager is used to dynamically update at least one of the above-mentioned first correspondence table and the above-mentioned second correspondence table.

The drone formation executor class is created and dynamically mounted by the drone formation runtime class. It is responsible for executing the lifecycle state management of the user code. The drone formation executor class inherits from the Unity script base class Monobehaviour and is driven by the Mono script lifecycle function to decide whether to execute the next operation of the user code, thereby driving the drone formation runtime to execute the next operation and update the status of all drones. In other words, the drone formation executor is used to execute the control logic code corresponding to each graphical code to control at least one virtual object in the formation, update its properties or control its movement.

In some embodiments, when the edited graphical code is compiled, the graphical code for controlling a single virtual object includes the identification information of the drone, and the graphical code for controlling a formation (including the first graphical code and the second graphical code) includes the identification information of the formation. Taking the virtual object as a drone as an example, the drone formation manager has a correspondence table between the identification information of the drone and the instance information of the drone. The instance information of the corresponding drone can be obtained through the identification information of the drone, and the drone can be controlled after the instance information of the drone is obtained.

As shown in FIG33 , the edited graphical code 3300 is used to control a single drone, and the control logic code 3301 corresponding to the edited graphical code 3300 includes the identification information (droneID) of the single drone. The edited first graphical code 3302 is used to add the virtual object drone to a formation, and the control logic code 3303 corresponding to the edited first graphical code 3302 includes the identification information (droneID) of the drone and the identification information (formationID) of the added formation.

The technical solution provided by the embodiment of the present application uses the first configuration information to construct a first correspondence table, that is, to establish a correspondence between a formation and a virtual object, and constructs a second correspondence table according to the third configuration information, that is, a correspondence between a virtual object and an instance. The identification information of the virtual object joining the formation can be found from the first object relationship table through the second configuration information, and the instance information of the virtual object joining the formation can be found by searching the second correspondence table, thereby realizing control over the virtual object. The technical solution provided by the embodiment of the present application can realize rapid search for virtual objects, which is conducive to realizing unified control over virtual objects in the formation, thereby improving code execution efficiency.

The following are device embodiments of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

Please refer to FIG34 , which shows a block diagram of a graphical programming device provided by an embodiment of the present application. As shown in FIG34 , the device 3400 may include: an interface display module 3410 , a graphical code display module 3420 , and an operation module 3430 .

The interface display module 3410 is used to display the code editing interface of the graphical programming tool, wherein the code editing interface includes a plurality of preset graphical codes, and the graphical codes are used to process one or more objects;

A graphical code display module 3420, configured to display an edited first graphical code in response to an edit operation on a first graphical code among the plurality of preset graphical codes, wherein the first graphical code is used to add at least one object to the same formation, and the edited first graphical code is used to add at least one object to the first formation;

The graphical code display module 3420 is further configured to, in response to an editing operation on a second graphical code among the plurality of preset graphical codes, display an edited second graphical code, wherein the second graphical code is used to uniformly control at least one object in the same formation, and the edited second graphical code is used to uniformly control at least one object in the first formation;

The running module 3430 is used to run the graphical programming result to control at least one object in the first formation, and the graphical programming result includes the edited first graphical code and the edited second graphical code.

In some embodiments, the first graphical code includes an object configuration area, and the object configuration area is used to configure at least one object to be added to the same formation.

In some embodiments, the graphical code display module 3420 is used to display an object selection interface in response to a trigger operation on the object configuration area in the first graphical code, wherein the object selection interface displays multiple elements, each element corresponding to an object.

The graphical code display module 3420 is used to display the edited first graphical code in response to a selection operation on at least one first element among the multiple elements, and the object corresponding to the at least one first element is added to the first formation.

In some embodiments, the selection operation includes a click operation on each of the first elements.

In some embodiments, the selection operation is a box selection operation starting from the first position and ending at the second position, and the element included in the box selection area constructed by the first position and the second position is the first element.

In some embodiments, the first element in a selected state is displayed separately from other elements in an unselected state; or, the object corresponding to the first element in a selected state is displayed separately from the objects corresponding to other elements in an unselected state in the virtual scene; or, the mark of the object corresponding to the first element in a selected state in a map navigation interface is displayed separately from the marks of the objects corresponding to other elements in an unselected state in the map navigation interface, wherein the map navigation interface displays a map of the virtual scene and the marks of each created object in the virtual scene in the map; or, the identifier of the object corresponding to the first element in a selected state in a function panel interface is displayed separately from the identifier of the object corresponding to other elements in an unselected state in the function panel interface, wherein the function panel interface is used to display configuration information corresponding to each created object in the virtual scene, and the configuration information includes the identifier of the object.

In some embodiments, as shown in FIG. 35 , the device further includes a state adjustment module 3440 .

In some embodiments, the state adjustment module 3440 is used to respond to a trigger operation on the object configuration area in the first graphical code, and determine the states corresponding to the multiple elements according to the correspondence between the elements and the objects. When the element corresponds to an object created in the virtual scene, the element is in an unselected state; when the element does not correspond to an object created in the virtual scene, the element is in an uncreated state.

The state adjustment module 3440 is also used for, for the mth element among the multiple elements, when the mth element is selected and the state of the mth element is an uncreated state, not changing the state of the mth element; and when the mth element is selected and the state of the mth element is an unselected state, changing the state of the mth element to a selected state, where m is a positive integer.

The state adjustment module 3440 is further configured to change the state of the mth element to an unselected state when the mth element is deselected and the state of the mth element is a selected state.

In some embodiments, the first graphical code includes an object configuration area, and the object configuration area is used to configure at least one object to be added to the same formation.

In some embodiments, the graphical code display module 3420 is used to display a virtual scene interface in an editing state in response to a trigger operation on the object configuration area in the first graphical code, wherein the virtual scene interface displays a virtual scene and objects created in the virtual scene, and different objects have different positions in the virtual scene.

In some embodiments, the graphical code display module 3420 is used to display the edited first graphical code in response to a selection operation on at least one first object among the created objects, and the at least one first object is added to the first formation.

In some embodiments, the first graphical code also includes a formation configuration area, and the formation configuration area is used to configure the formation that the at least one object joins.

In some embodiments, the graphical code display module 3420 is also used to display a formation selection interface in response to a trigger operation on the formation configuration area in the first graphical code, wherein multiple formation numbers are displayed in the formation selection interface, and each formation number corresponds to a formation.

The graphical code display module 3420 is further used to display the edited first graphical code in response to a selection operation on a first formation number among the multiple formation numbers, and the formation corresponding to the first formation number is the first formation.

In some embodiments, the interface display module 3410 is also used to respond to a trigger operation on an object creation control in a function panel interface, and display configuration information corresponding to a newly added object in the function panel interface. The function panel interface is used to display configuration information corresponding to each created object in the virtual scene, and the configuration information includes an identifier of the object.

The interface display module 3410 is further configured to display the modified configuration information in response to a modification operation on the configuration information corresponding to the newly added object.

The interface display module 3410 is further configured to display the newly added object created based on the modified configuration information in the virtual scene.

In some embodiments, the function panel interface includes a search input field.

In some embodiments, the interface display module 3410 is further configured to display search information for the created object input in the search input bar in response to an input operation on the search input bar.

The interface display module 3410 is also used to display the configuration information corresponding to the object pointed to by the search information.

In some embodiments, the function panel interface includes tabs corresponding to each created object.

In some embodiments, the interface display module 3410 is also used to display configuration information corresponding to the object pointed to by the target tab in response to a selection operation of the target tab in the tabs corresponding to the respective created objects; or, in response to a deletion operation on the target tab, delete the object pointed to by the target tab from the virtual scene; or, in response to a formation setting operation on the target tab, change the formation to which the object pointed to by the target tab joins.

In some embodiments, the running module 3430 is used to run the graphical programming result to display the control process of at least one object in the first formation in the virtual scene.

In some embodiments, the interface display module 3410 is also used to display at least one of a scene control and a code editing control; the scene control is used to switch display of a virtual scene interface, the virtual scene interface includes a virtual scene and objects created in the virtual scene, and the code editing control is used to switch display of the code editing interface.

The interface display module 3410 is also used to run the graphical programming results, and in response to a triggering operation on the scene control, display the virtual scene interface, and display the control process of at least one object in the first formation in the virtual scene in the virtual scene interface.

The interface display module 3410 is further used to redisplay the code editing interface in response to a triggering operation on the code editing control, wherein the code editing interface includes the graphical programming result.

In some embodiments, the interface display module 3410 is also used to display a formation selection interface in response to a formation setting operation for a target object among the created objects in the virtual scene interface, wherein the formation selection interface includes multiple formation numbers, each formation number corresponding to a formation; in response to a selection operation for a second formation number among the multiple formation numbers, determine that the target object joins the formation corresponding to the second formation number.

It should be noted that the device provided in the above embodiment, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the content structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

Please refer to Figure 36, which shows a block diagram of a computer device 3600 provided in one embodiment of the present application. The computer device 3600 can be any electronic device with data calculation, processing and storage capabilities. The computer device 3600 can be used to implement the graphical programming method provided in the above embodiment.

Typically, the computer device 3600 includes: a processor 3601 and a memory 3602 .

The processor 3601 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 3601 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). The processor 3601 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 3601 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 3601 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

The memory 3602 may include one or more computer-readable storage media, which may be non-transitory. The memory 3602 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 3602 is used to store a computer program, which is configured to be executed by one or more processors to implement the above-mentioned graphical programming method.

Those skilled in the art will appreciate that the structure shown in FIG. 36 does not limit the computer device 3600 and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

In an exemplary embodiment, a computer-readable storage medium is also provided, in which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned graphical programming method is implemented. Optionally, the computer-readable storage medium may include: ROM (Read-Only Memory), RAM (Random Access Memory), SSD (Solid State Drives) or an optical disk, etc. Among them, the random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

In an exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program, the computer program being stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the above-mentioned graphical programming method.

It should be noted that the relevant data collection and processing in this application should be strictly in accordance with the requirements of relevant national laws and regulations when applied in examples, and the informed consent or separate consent of the personal information subject should be obtained, and subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.

It should be understood that the "multiple" mentioned in this article refers to two or more. "And/or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and/or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character "/" generally indicates that the objects associated with each other are in an "or" relationship. In addition, the step numbers described in this article only illustrate a possible execution sequence between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to the diagram. The embodiments of the present application are not limited to this.

The above description is only an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims (18)

1. A graphical programming method, characterized in that the method comprises: Displaying a code editing interface of a graphical programming tool, wherein the code editing interface includes a plurality of preset graphical codes, and the graphical codes are used to process one or more objects; In response to an editing operation on a first graphical code among the plurality of preset graphical codes, displaying the edited first graphical code, wherein the first graphical code is used to add at least one object to the same formation, and the edited first graphical code is used to add at least one object to the first formation; In response to an editing operation on a second graphical code among the plurality of preset graphical codes, displaying the edited second graphical code, wherein the second graphical code is used to uniformly control at least one object in the same formation, and the edited second graphical code is used to uniformly control at least one object in the first formation; The graphical programming result is run to control at least one object in the first formation, wherein the graphical programming result includes the edited first graphical code and the edited second graphical code. 2. The method according to claim 1, characterized in that the first graphical code includes an object configuration area, and the object configuration area is used to configure at least one object added to the same formation; In response to an editing operation on a first graphical code among the plurality of preset graphical codes, displaying the edited first graphical code comprises: In response to a trigger operation on the object configuration area in the first graphical code, an object selection interface is displayed, wherein a plurality of elements are displayed in the object selection interface, each element corresponding to an object; In response to a selection operation on at least one first element among the plurality of elements, the edited first graphical code is displayed, and an object corresponding to the at least one first element is added to the first formation. 3 . The method according to claim 2 , wherein the selection operation comprises a click operation on each of the first elements. 4. The method according to claim 2 is characterized in that the selection operation is a box selection operation starting from the first position and ending at the second position, and the element included in the box selection area constructed by the first position and the second position is the first element. 5. The method according to claim 2, characterized in that: The first element in the selected state is displayed separately from other elements in the unselected state; or, The object corresponding to the first element in the selected state and the objects corresponding to other elements in the unselected state are displayed separately in the virtual scene; or, The mark of the object corresponding to the first element in the selected state in the map navigation interface is displayed separately from the marks of the objects corresponding to other elements in the unselected state in the map navigation interface, wherein the map navigation interface displays a map of the virtual scene and the marks of each created object in the virtual scene in the map; or The identifier of the object corresponding to the first element in a selected state in the function panel interface is displayed separately from the identifiers of the objects corresponding to other elements in an unselected state in the function panel interface, wherein the function panel interface is used to display the configuration information corresponding to each created object in the virtual scene, and the configuration information includes the identifier of the object. 6. The method according to claim 2, characterized in that the method further comprises: In response to a trigger operation on the object configuration area in the first graphical code, determining states corresponding to the plurality of elements according to a correspondence between elements and objects, wherein when the element corresponds to an object that has been created in the virtual scene, the element is in an unselected state, and when the element does not correspond to an object that has been created in the virtual scene, the element is in an uncreated state; For an mth element among the multiple elements, when the mth element is selected and the state of the mth element is an uncreated state, the state of the mth element is not changed; when the mth element is selected and the state of the mth element is an unselected state, the state of the mth element is changed to a selected state, where m is a positive integer; When the m-th element is deselected and the state of the m-th element is a selected state, the state of the m-th element is changed to an unselected state. 7. The method according to claim 1, characterized in that the first graphical code includes an object configuration area, and the object configuration area is used to configure at least one object added to the same formation; In response to an editing operation on a first graphical code among the plurality of preset graphical codes, displaying the edited first graphical code comprises: In response to a trigger operation on the object configuration area in the first graphical code, a virtual scene interface in an editing state is displayed, wherein the virtual scene interface displays a virtual scene and objects created in the virtual scene, and different objects have different positions in the virtual scene; In response to a selection operation on at least one first object among the created objects, the edited first graphical code is displayed, and the at least one first object is added to the first formation. 8. The method according to claim 2 or 7, characterized in that the first graphical code further includes a formation configuration area, and the formation configuration area is used to configure the formation to which the at least one object joins; the method further includes: In response to a trigger operation on the formation configuration area in the first graphical code, a formation selection interface is displayed, wherein a plurality of formation numbers are displayed in the formation selection interface, and each formation number corresponds to a formation; In response to a selection operation on a first formation number among the multiple formation numbers, the edited first graphical code is displayed, and the formation corresponding to the first formation number is the first formation. 9. The method according to claim 1, characterized in that the method further comprises: In response to a trigger operation on an object creation control in a function panel interface, configuration information corresponding to a newly added object is displayed in the function panel interface, wherein the function panel interface is used to display configuration information corresponding to each created object in a virtual scene; In response to a modification operation on the configuration information corresponding to the newly added object, displaying the modified configuration information; The newly added object created based on the modified configuration information is displayed in the virtual scene. 10. The method according to claim 9, characterized in that the function panel interface includes a search input bar; the method further comprises: In response to an input operation on the search input bar, displaying search information for the created object input in the search input bar; Display the configuration information corresponding to the object pointed to by the search information. 11. The method according to claim 9, characterized in that the function panel interface includes tabs corresponding to each created object; the method further comprises: In response to a selection operation of a target tab in the tabs corresponding to the created objects, displaying configuration information corresponding to the object pointed to by the target tab; or In response to a deletion operation on the target tab, the object pointed to by the target tab is deleted from the virtual scene; or, In response to a team setting operation for the target tab, the team to which the object pointed to by the target tab joins is changed. 12. The method according to claim 1, wherein the step of running the graphical programming result to control at least one object in the first formation comprises: The graphical programming result is run to display the control process of at least one object in the first formation in the virtual scene. 13. The method according to claim 12, characterized in that the method further comprises: Display at least one of a scene control and a code editing control; the scene control is used to switch and display a virtual scene interface, the virtual scene interface includes a virtual scene and objects created in the virtual scene, and the code editing control is used to switch and display the code editing interface; The step of running the graphical programming result to display the control process of at least one object in the first formation in a virtual scene includes: Running the graphical programming result, in response to a triggering operation on the scene control, displaying the virtual scene interface, and displaying in the virtual scene interface a control process of at least one object in the first formation in the virtual scene; The method further comprises: In response to a triggering operation on the code editing control, the code editing interface is redisplayed, and the code editing interface includes the graphical programming result. 14. The method according to claim 13, characterized in that the method further comprises: In response to a formation setting operation for a target object among the objects created in the virtual scene interface, displaying a formation selection interface, wherein the formation selection interface includes a plurality of formation numbers, each formation number corresponding to a formation; In response to a selection operation on a second formation number among the multiple formation numbers, it is determined that the target object joins the formation corresponding to the second formation number. 15. A graphical programming device, characterized in that the device comprises: An interface display module, used to display a code editing interface of a graphical programming tool, wherein the code editing interface includes a plurality of preset graphical codes, and the graphical codes are used to process one or more objects; a graphical code display module, configured to display an edited first graphical code in response to an edit operation on a first graphical code among the plurality of preset graphical codes, wherein the first graphical code is used to add at least one object to the same formation, and the edited first graphical code is used to add at least one object to the first formation; The graphical code display module is further configured to, in response to an editing operation on a second graphical code among the plurality of preset graphical codes, display an edited second graphical code, wherein the second graphical code is used to uniformly control at least one object in the same formation, and the edited second graphical code is used to uniformly control at least one object in the first formation; The running module is used to run the graphical programming result to control at least one object in the first formation, and the graphical programming result includes the edited first graphical code and the edited second graphical code. 16. A computer device, characterized in that the computer device comprises a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the graphical programming method according to any one of claims 1 to 14. 17. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, and the computer program is loaded and executed by a processor to implement the graphical programming method according to any one of claims 1 to 14. 18. A computer program product, characterized in that the computer program product comprises a computer program, the computer program is stored in a computer-readable storage medium, and a processor reads and executes the computer program from the computer-readable storage medium to implement the graphical programming method according to any one of claims 1 to 14.