CN109240678B

CN109240678B - Code generation method and device

Info

Publication number: CN109240678B
Application number: CN201710551695.6A
Authority: CN
Inventors: 朱霞; 代培
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2017-07-07
Filing date: 2017-07-07
Publication date: 2022-07-15
Anticipated expiration: 2037-07-07
Also published as: CN109240678A

Abstract

The application provides a code generation method and a device, and the method can comprise the following steps: displaying a target interface determined by a user; according to the position of a user control icon on the target interface, interface elements at corresponding positions in the target interface are displayed in a distinguishing mode; when the selection operation of the user control icon on the interface element is detected, displaying a corresponding interactive window, wherein the interactive window comprises alternative operation options aiming at the selected interface element; when the selection operation of the user control icon on the alternative operation option is detected, generating a code for implementing corresponding operation on the selected interface element; and executing corresponding operation aiming at the selected interface element so as to update the target interface.

Description

Code generation method and device

Technical Field

The present application relates to the field of computer technologies, and in particular, to a code generation method and apparatus.

Background

The computer implements corresponding functions by executing the codes. In other words, the user can write appropriate code to cause the computer to realize the corresponding functions based on the code. The "user" herein is not often a general user, but should be a professional user, such as a programmer, who has at least one computer language. Therefore, for an ordinary user, it is often difficult to implement function control or function extension of a computer by writing codes, which causes higher learning cost and use threshold for the user.

Disclosure of Invention

In view of this, the present application provides a code generation method and apparatus, which can implement automatic code generation.

In order to achieve the above purpose, the present application provides the following technical solutions:

according to a first aspect of the present application, a code generation method is provided, including:

displaying a target interface determined by a user;

according to the position of a user control icon on the target interface, displaying interface elements at corresponding positions in the target interface in a distinguishing manner;

when the selection operation of the user control icon on the interface element is detected, displaying a corresponding interactive window, wherein the interactive window comprises alternative operation options aiming at the selected interface element;

when the selection operation of the user control icon on the alternative operation option is detected, generating a code for implementing corresponding operation on the selected interface element;

and executing corresponding operation aiming at the selected interface element so as to update the target interface.

According to a second aspect of the present application, there is provided a code generation method including:

displaying alternative operation options aiming at the selected interface elements according to the interface elements selected by the user on the target interface;

And generating a code for implementing corresponding operation according to the operation option selected by the user.

According to a third aspect of the present application, there is provided a code generation method, including:

displaying corresponding alternative operation options aiming at interface elements contained in the target interface;

According to a fourth aspect of the present application, there is provided a code generation apparatus comprising:

the interface display unit is used for displaying a target interface determined by a user;

the distinguishing display unit is used for distinguishing and displaying interface elements at corresponding positions in the target interface according to the positions of the user control icons on the target interface;

the window display unit is used for displaying a corresponding interactive window when the selection operation of the user control icon on the interface element is detected, wherein the interactive window comprises alternative operation options aiming at the selected interface element;

the code generation unit is used for generating a code for implementing corresponding operation on the selected interface element when the selection operation of the user control icon on the alternative operation option is detected;

and the operation execution unit executes corresponding operation aiming at the selected interface element so as to update the target interface.

According to a fifth aspect of the present application, there is provided a code generation apparatus comprising:

the option display unit is used for displaying alternative operation options aiming at the selected interface elements according to the interface elements selected by the user on the target interface;

and the code generating unit is used for generating codes for implementing corresponding operations according to the operation options selected by the user.

According to a sixth aspect of the present application, there is provided a code generation apparatus comprising:

the option display unit is used for displaying corresponding alternative operation options aiming at the interface elements contained in the target interface;

According to a seventh aspect of the present application, a task execution method is provided, including:

acquiring an operation task aiming at a target interface, wherein the operation task is generated according to an interface element selected in the target interface by a user in advance and an operation mode aiming at the interface element;

and executing the operation task to implement the operation based on the operation mode on the interface element in the target interface.

According to an eighth aspect of the present application, there is provided a task execution device including:

The acquisition unit is used for acquiring an operation task aiming at a target interface, wherein the operation task is generated according to an interface element selected in the target interface by a user in advance and an operation mode aiming at the interface element;

and the execution unit executes the operation task to implement the operation based on the operation mode on the interface element in the target interface.

According to the technical scheme, the interface elements selected by the user are identified, the alternative operation options aiming at the selected interface elements are provided for the user, the selection operation of the user on the alternative operation options is identified, the corresponding codes can be automatically generated, the user does not need to master computer language and programming skills, the codes can be automatically generated, the code generation process is greatly simplified, and the learning cost of the user is reduced.

Drawings

Fig. 1 is a schematic diagram of an architecture of a code generation system according to an exemplary embodiment of the present application.

Fig. 2A is a flowchart of a code generation method according to an exemplary embodiment of the present application.

Fig. 2B is a flowchart of another code generation method provided in an exemplary embodiment of the present application.

Fig. 3 is a flowchart of a task execution method according to an exemplary embodiment of the present application.

Fig. 4 is a schematic diagram of an operation interface for automatically generating code according to an exemplary embodiment of the present application.

FIG. 5 is a schematic diagram of an input target interface provided by an exemplary embodiment of the present application.

FIG. 6 is a schematic diagram of a capture objects interface provided by an exemplary embodiment of the present application.

FIG. 7 is a diagram illustrating a code block for automatically generating an open web page according to an exemplary embodiment of the present application.

Fig. 8 is a schematic diagram illustrating a prompt for an input box according to an exemplary embodiment of the present application.

FIG. 9 is a diagram illustrating a manner of selecting an input box according to an exemplary embodiment of the present application.

FIG. 10 is a diagram illustrating an automatic generation of a code block for filling in the content of an input box according to an exemplary embodiment of the present application.

FIG. 11 is a schematic diagram of automatically generating a code block for opening a web page and filling in the contents of an input box in an exemplary embodiment of the present application.

Fig. 12 is a schematic diagram illustrating a prompt for a search button according to an exemplary embodiment of the present application.

Fig. 13 is a schematic diagram illustrating a manner of selecting an operation of a search button according to an exemplary embodiment of the present application.

FIG. 14 is a block diagram of code for automatically generating clicked on web page elements, according to an exemplary embodiment of the present application.

Fig. 15 is a schematic diagram illustrating a prompt for a search result according to an exemplary embodiment of the present application.

FIG. 16 is a diagram illustrating a manner of selecting an operation of a search button provided by an exemplary embodiment of the present application.

Fig. 17 is a schematic diagram of a code block for automatically generating an acquisition element list according to an exemplary embodiment of the present application.

Fig. 18 is a schematic diagram of another code block for automatically generating an acquisition element list according to an exemplary embodiment of the present application.

Fig. 19 is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present application.

Fig. 20 is a block diagram of a code generation apparatus according to an exemplary embodiment of the present application.

Fig. 21 is a block diagram of another code generation apparatus provided in an exemplary embodiment of the present application.

Fig. 22 is a block diagram of another code generation apparatus provided in an exemplary embodiment of the present application.

Fig. 23 is a schematic structural diagram of another electronic device according to an exemplary embodiment of the present application.

Fig. 24 is a block diagram of a task execution device according to an exemplary embodiment of the present application.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings in which the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the application, as detailed in the appended claims.

According to the method and the device, the interface elements selected by the user are identified, the alternative operation options aiming at the selected interface elements are provided for the user, and the selection operation of the user on the alternative operation options is identified, so that the corresponding codes can be automatically generated, the user does not need to master computer language and programming skills, the codes can be automatically generated, the code generation process is greatly simplified, and the learning cost of the user is reduced.

In an embodiment, the technical solution of the present application may be applied to an electronic device used by a user, for example, the electronic device may include a mobile phone, a PC, a tablet device, a notebook computer, a handheld computer (PDAs), a wearable device (such as smart glasses, a smart watch, and the like), and the present application is not limited thereto. During operation, the electronic device may run a client-side program of an application to implement the code generation function of the present application.

In another embodiment, the technical solution of the present application may be applied to a server, where the server cooperates with an electronic device used by a user, and the server automatically generates a corresponding code based on a local operation performed by the user on the electronic device. For example, fig. 1 is a schematic architecture diagram of a code generation system provided in an exemplary embodiment of the present application, which may include a server 11, a network 12, an electronic device 13, and the like.

The server 11 may be a physical server comprising an independent host, or the server 11 may be a virtual server carried by a cluster of hosts, or the server 11 may be a cloud server. During operation, the server 11 may run a server-side program of an application to implement the code generation function of the present application.

The electronic device 13 may include types such as: a mobile phone, a PC, a tablet device, a notebook computer, a pda (Personal Digital Assistants), a wearable device (such as smart glasses, a smart watch, etc.), etc., which are not limited in this application. In the operation process, the electronic device 13 may collect operation data of the user and upload the operation data to the server 11, so that the server 11 may implement the code generation function. The data collected by the electronic device 13 may include various forms, such as image information (based on a form of a picture or a video, etc.) displayed on the electronic device 13, background running data of the electronic device 13, etc., and practically any manner that enables the server 11 to obtain operation data of the user may be applied thereto, and the disclosure is not limited thereto.

And the network 12 for interaction between the server 11 and the electronic device 13 may include various types of wired or wireless networks. In one embodiment, the Network 12 may include the Public Switched Telephone Network (PSTN) and the Internet.

For further explanation of the present application, the following examples are provided:

fig. 2A is a flowchart of a code generation method according to an exemplary embodiment of the present application. As shown in FIG. 2A, the method may be applied to many types of devices, such as servers or electronic devices local to the user; the method may comprise the steps of:

step 202A, according to the interface element selected by the user in the target interface, displaying the alternative operation options for the selected interface element.

In an embodiment, the target interface may include a preset interface provided by the device applying the embodiment. In another embodiment, the target interface may include a user-specified interface, such as a web page, an explorer interface, a document interface, etc., which is not limited in this application.

In an embodiment, a target interface may be presented to a user by the device applying the embodiment, where the target interface includes at least one interface element for the user to select; for example, when the device applying the embodiment is a server, a server running on the server may send a target interface to a local electronic device of a user for displaying, or when the device applying the embodiment is a local electronic device of a user, a client running on the electronic device may open and display the target interface. In another embodiment, the target interface may be exposed independently of the server or the client, for example, the target interface may be opened through a local browser, a resource manager, or the like.

In one embodiment, the position of a user control icon on the target interface can be obtained; when the interface element exists at the position of the target interface, the interface element at the position is displayed in a distinguishing way; and when the user manipulation icon is detected to select the interface element at the position, determining the interface element at the position as the selected interface element. By the aid of the interface elements, the user can accurately know the interface elements on the target interface, and the user can conveniently select the interface elements to automatically generate corresponding codes.

And step 204A, generating a code for implementing corresponding operation according to the operation option selected by the user.

In an embodiment, the corresponding operation content may not be required for the operation option selected by the user. For example, when the interface element is a virtual key in the target interface, if the operation option is "click", the corresponding operation content does not exist; and when the interface element is an input box in the target interface, assuming that the operation option is "input", determining the input content of the "input" operation according to the corresponding operation content.

In an embodiment, when the operation option has corresponding operation content, the operation content may be a default operation content configured in advance.

In an embodiment, when the operation option has corresponding operation content, the operation content set by the user for the selected operation option can be acquired; and then, generating a code for implementing corresponding operation according to the operation options selected by the user and the set operation content.

In one embodiment, the operation object of the operation includes at least one of: and the selected interface element and other interface elements belonging to the same category as the selected interface element in the target interface. For example, when the interface element is text in the target interface, assuming that the operation option is "collect one/more sets of data", the operation object may be selected text, or the operation object may be selected text and other similar text that is not selected in the target interface.

In an embodiment, the device applying the embodiment may execute the operation on the selected interface element to update the target interface, so that the user can further select the interface element and the operation option for the interface element, and thus by continuously cycling and repeating the above operations, a series of operations on the target interface is implemented, and a corresponding series of codes are generated.

In an embodiment, a plurality of interface elements respectively selected by a user in the target interface and corresponding multi-section codes can be obtained; and then, sequentially assembling the multiple sections of codes according to the selection sequence of the multiple interface elements by the user.

Fig. 2B is a flowchart of another code generation method provided in an exemplary embodiment of the present application. As shown in FIG. 2B, the method may be applied to many types of devices, such as servers or electronic devices local to the user; the method may comprise the steps of:

step 202B, for the interface elements included in the target interface, displaying corresponding alternative operation options.

In an embodiment, under the condition that the user has not performed any operation, the alternative operation options corresponding to each interface element can be displayed on the target interface, so that on one hand, the user can accurately know the interface elements on the target interface, on the other hand, the user can conveniently select the operation options and simultaneously complete the selection of the corresponding interface elements, and the user operation is facilitated to be simplified.

And step 204B, generating a code for implementing corresponding operation according to the operation option selected by the user.

In an embodiment, when the operation option has corresponding operation content, the operation content may be a default operation content that is preconfigured.

In one embodiment, the operation object of the operation includes at least one of: and the selected interface element and other interface elements in the target interface which belong to the same category as the selected interface element. For example, when the interface element is text in the target interface, assuming that the operation option is "collect one/more sets of data", the operation object may be selected text, or the operation object may be selected text and other similar text that is not selected in the target interface.

Fig. 3 is a flowchart of a task execution method according to an exemplary embodiment of the present application. As shown in FIG. 3, the method may be applied to many types of devices, such as servers or electronic devices local to the user; the method may comprise the steps of:

step 302, obtaining an operation task for a target interface, where the operation task is generated according to an interface element selected in the target interface by a user in advance and an operation mode for the interface element.

In one embodiment, the operation tasks are composed of corresponding codes, and the codes can be automatically generated through the code generation scheme provided by the application without the need for a user to master computer languages and programming skills. The code generation scheme may refer to an embodiment such as that shown in fig. 2A or fig. 2B, or other embodiments provided in this application, which is not limited in this application.

And step 304, executing the operation task to implement the operation based on the operation mode on the interface element in the target interface.

In one embodiment, by generating and executing the operation task, the operation related to the target interface can be automatically and repeatedly implemented based on the operation task without manually implementing the operation by the user, thereby greatly simplifying the operation of the user and contributing to the improvement of the operation efficiency.

In an embodiment, when the target interface includes one or more interface elements, the operation task may be used to perform an operation based on the operation mode on a single interface element.

In an embodiment, when the target interface includes a plurality of interface elements, the operation task may be configured to perform an operation based on the operation manner on the plurality of interface elements in a batch manner.

In an embodiment, the interface elements included in the target interface may be of the same type, and the same operation manner may be adopted for the operation of the interface elements. In other embodiments, operations in different operation modes can be also adopted for different interface elements of the same type.

In an embodiment, a plurality of interface elements included in the target interface may belong to a plurality of types, and different operation modes may be adopted for different types of interface elements. In other embodiments, the same operation mode may be used for different types of interface elements.

For convenience of understanding, the following describes the technical solution of the present application in detail with reference to the interface display change during the user operation.

Fig. 4 is a schematic diagram of an operation interface for automatically generating a code according to an exemplary embodiment of the present application. A client may be run on the electronic device used by the user and may provide an operator interface 40 as shown in fig. 4 for automatically generating code.

In one embodiment, the operation interface 40 may include a first area 41 and a second area 42, the first area 41 may include an "open target interface" option 411, a "capture current interface" option 412, and the like, for determining a target interface, and the second area 42 is used for presenting the target interface and automatically generating corresponding codes based on the user's operation on the target interface.

FIG. 5 is a schematic diagram of an input target interface provided by an exemplary embodiment of the present application. When it is detected that the user selects the "open target interface" option 411 (e.g., the "open target interface" option 411 is clicked by the cursor 43), the client may provide an address input window 411A as shown in fig. 5 for the user to input an access address of the target interface, such as a website of a certain web page, a storage path of a certain file, and the like.

FIG. 6 is a schematic diagram of a capture objects interface provided by an exemplary embodiment of the present application. When it is detected that the user selects the "Capture Current interface" option 412 (e.g., the "Capture Current interface" option 412 is clicked on by the cursor 43), the client may automatically capture the interface opened in the second area 42 (i.e., the current interface) as the target interface. Such as when the user has entered the access address http:// www.xxxxxx.com in the address bar 421 at the top of the second area 42 and opened the corresponding interface 44, the client can automatically capture the interface 44 as the target interface by triggering the "capture current interface" option 412 without the user having to repeatedly enter the access address of the target interface through the address entry window 411A as shown in fig. 5.

FIG. 7 is a diagram illustrating an automatic generation of a code block for an open web page according to an exemplary embodiment of the present application. After the target interface is determined based on the above embodiment, the client may automatically generate a code for implementing an operation of "opening a web page" according to the operation process, the operation result, and the like of the user, and the code may be used to automatically open the target interface during the operation process. To facilitate the user's understanding that the client has generated the corresponding code, the corresponding processing logic 45 may be exposed in the first area 41, such as the processing logic 45 in fig. 7 only includes an "open web page" step 451, corresponding to the code described above for implementing the "open web page" operation; meanwhile, when the processing logic 45 includes several steps, the code corresponding to each step may be referred to as a code block, and the code blocks corresponding to all the steps may be combined into a set of codes for implementing a series of operations that the user wishes to perform.

In an embodiment, for the automatically generated code corresponding to the step 451 of "opening a web page" or any other step, the user may adjust the interface element, the adopted operation mode, and the like for a certain step, so that the client may adjust the step, and further update the corresponding code block for the adjusted step. In another embodiment, the user may edit the code block corresponding to the step.

In one embodiment, when the user triggers the "Capture Current interface" option 412, the second area 42 may maintain the presentation of the interface 44 when transitioning to the state shown in FIG. 7, since the user needs to open the interface 44 in the second area 42 in advance. When the user triggers the "open target interface" option 411, the user does not need to open the interface 44 in the second area 42 in advance, and in order to facilitate the user to perform an operation on the interface 44 in a subsequent process, the client may automatically perform an operation of "opening a web page" in addition to automatically generating a code corresponding to the "open web page" step 451, so that the interface 44 is displayed in the second area 42.

In an embodiment, the user may trigger the "start recording" option 413 shown in fig. 7, so that during the operation of the interface 44 by the user, the client may automatically recognize the interface element in the interface 44, assist the user in completing the triggering operation on the corresponding interface element for the interface element triggered by the user, and automatically generate a corresponding code by the client.

Fig. 8 is a schematic diagram illustrating a prompt for an input box according to an exemplary embodiment of the present application. When the option 413 shown in fig. 7 is triggered, the presentation content in the option 413 may be changed from "start recording" to "stop recording" shown in fig. 8, and when the option 413 shown in fig. 8 is triggered, the presentation content in the option 413 may be changed from "stop recording" to "start recording", so as to implement switching of the recording function.

After the recording function is turned on, when the user moves the cursor 43 in the interface 44, the client determines the interface element at the position of the cursor 43 and prompts the user for the existence of the interface element. For example, as shown in fig. 8, when the cursor 43 is located at the Input box 441, the client may prompt the user by presenting the Input box 441 with other interface elements in a distinguishing manner (e.g., changing the color of the presentation, etc.), presenting the corresponding interface element ID (e.g., "Input # kw.s _ ipt") in the prompt box 441A, and the specification of the presentation area (e.g., "500 × 22", that is, 500 pixels in the horizontal direction and 22 pixels in the vertical direction): there is an interface element (i.e., input box 441) where the cursor 43 is located.

FIG. 9 is a schematic diagram illustrating a manner of selecting an operation for an input box according to an exemplary embodiment of the present application. When detecting that the user clicks the cursor 43 to select the input box 441, the client may present an operation selection window 46 shown in fig. 9, where the operation selection window 46 may include several operation options 461, and the operation options 461 correspond to operation modes that may be implemented for the input box 441, such as "input", "click", "obtain text of an element", "capture one/more sets of data", "mouse-in", and so on, for selection by the user.

Assuming that the user selects the "enter" operation, the client may further present a content input box 462 in the operation selection window 46 for the user to fill in the content input box 462 with the input content of the "enter" operation; for example, the user may fill out a "code stack" or any other content.

Fig. 10 is a diagram illustrating an automatic generation of a code block for filling in the content of an input box according to an exemplary embodiment of the present application. The client may automatically generate a code block corresponding to the "fill input box content" step 452 shown in fig. 10 according to the interface element selected by the user (e.g., input box 441), the operation mode for the interface element (e.g., "input" and the input content is "code stack"), and the like, and at this time, the processing logic 45 may include the "open web page" step 451 and the "fill input box content" step 452 performed in sequence.

In one embodiment, the user may adjust the execution order among the steps included in the processing logic 45, so that the client correspondingly adjusts the execution order among the corresponding code blocks, and the generation order of the code blocks corresponding to the steps is not necessarily consistent.

In one embodiment, in addition to generating the code block corresponding to the "fill input box content" step 452, the client may also automatically perform the "fill input box content" operation, so that the interface 44 is updated accordingly. As shown in fig. 10, by performing an "enter" operation on the input box 441, the "stack of codes" filled by the user can be automatically filled in the input box 441 without manual filling by the user.

In one embodiment, the "record" function described above is only applied to the user's operation of the input box 441, and the user needs to open the interface 44 through the "open target interface" option 411, the "capture current interface" 412, and the like, and prompt the client to generate the code block corresponding to the "open web page" step 451.

In another embodiment, a "record" function may be applied to cause the client to generate the code patch corresponding to the "open web page" step 451. For example, fig. 11 is a schematic diagram of automatically generating a code block for opening a web page and filling in an input box content according to an exemplary embodiment of the present application, and after detecting that the option 413 displayed as "start recording" is triggered, the user may determine, through the interface 44 opened in the second area 42, that the user wants to perform an "input" operation on the input box 441 in the interface 44 and the input content is "code stack" based on the operation procedure in the above-described embodiment shown in fig. 8 to 10, i.e., selecting the input box 441 and selecting an operation manner through the operation selection window 46, and the like, that the client automatically determines that the corresponding processing logic 45 includes an "open web page" step 451 and an "fill in input box content" step 452, and generates a code block corresponding to the "open web page" step 451 and the "fill in input box content" step 452, respectively.

Fig. 12 is a diagram illustrating prompting of a search button according to an exemplary embodiment of the present application. As shown in fig. 12, the client may prompt the user when detecting that the cursor 43 is located at the search button 442; similar to the prompt for the Input box 441 in fig. 8, the client may display the search button 442 differently from other interface elements (e.g., change the display color, etc.), display the corresponding interface element ID (e.g., "Input # sub.bg. s _ btn") and the specification of the display area (e.g., "100 × 34", i.e., 100 pixels in the horizontal direction and 34 pixels in the vertical direction) in the prompt box 442A, and so on.

Fig. 13 is a schematic diagram illustrating a manner of selecting an operation of a search button according to an exemplary embodiment of the present application. When it is detected that the user clicks the cursor 43 to select the search button 442, the client may present the operation selection window 46 shown in fig. 13, and the operation selection window 46 may include a plurality of operation options 461, and the operation options 461 correspond to operation modes that can be implemented for the search button 442, such as "clicking", "obtaining text of an element", "capturing one or more sets of data", "mouse moving in", and the like, for selection by the user.

FIG. 14 is a block diagram of code for automatically generating clicked web page elements according to an exemplary embodiment of the present application. Assuming that the user selects the "click" operation, the client may automatically generate a code block corresponding to the "click web element" step 453 shown in fig. 14 according to the interface element selected by the user (e.g., the search button 442), the operation mode (e.g., the "click" operation) for the interface element, and the like, and at this time, the processing logic 45 may include the "open web page" step 451, the "fill input box content" step 452, and the "click web element" step 453, which are sequentially performed.

In one embodiment, in addition to generating the code block corresponding to the "click on web page element" step 453, the client may also automatically perform the "click on web page element" operation to enable the corresponding update to be performed by the interface 44. For example, as shown in fig. 14, by performing a "click" operation on the search button 442, the interface 44 can be updated from the search interface shown in fig. 13 to a search result interface, which includes search results 471, 472, 473, and the like, obtained by searching with "code stack" as a keyword, without manually triggering the search button 442 shown in fig. 13 by the user.

Fig. 15 is a schematic diagram illustrating a prompt for a search result according to an exemplary embodiment of the present application. As shown in fig. 15, when cursor 43 is detected at search result 471, the client may prompt the user; similar to the prompt for the input box 441 in fig. 8, the client may display the search result 471 differently from other interface elements (e.g., change the display color, etc.), display the corresponding interface element ID (e.g., "a") and the specification of the display area (e.g., "165 × 17", that is, 165 pixels in the horizontal direction and 17 pixels in the vertical direction) in the prompt box 471A, and so on.

FIG. 16 is a diagram illustrating a manner of selecting an operation of a search button provided by an exemplary embodiment of the present application. When detecting that the user clicks the cursor 43 to select the search result 471, the client may present an operation selection window 46 shown in fig. 16, where the operation selection window 46 may include several operation options 461, and the operation options 461 correspond to operation manners that may be implemented for the search result 471, such as "click", "obtain text of an element", "capture one/more sets of data", "mouse-in", and so on, for selection by the user.

Assuming that the user selects the "gather one or more sets of data" operation, the client may further present a data list 463 in the operation selection window 46, where the data list 463 includes an interface element, i.e., a search result 471, selected by the user in the embodiment shown in fig. 15, and similar elements corresponding to the search result 471 in the interface 44, such as a search result 472, a search result 473, and the like, such as "9 similar elements found" shown in fig. 16, indicating that 9 search results are included in the interface 44.

Fig. 17 is a schematic diagram of a code block for automatically generating an acquisition element list according to an exemplary embodiment of the present application. When the user does not select the "gather similar elements" option 464 included in the operation selection window 46 in fig. 16, which indicates that the user only wishes to gather the interface element selected in the embodiment shown in fig. 15, i.e., the search result 471, the client may automatically generate the code block corresponding to the "get element list" step 454 shown in fig. 17, and at this time, the processing logic 45 may include an "open web page" step 451, a "fill input box content" step 452, a "click on web page element" step 453, and a "get element list" step 454, which are sequentially performed.

Fig. 18 is a schematic diagram of another code block for automatically generating an acquisition element list according to an exemplary embodiment of the present application. When the user selects the "gather similar elements" option 464 included in the operation selection window 46 in fig. 16, which indicates that the user wishes to gather the interface elements selected in the embodiment shown in fig. 15, that is, the search result 471 and similar elements thereof, the client may automatically generate code blocks corresponding to the "get element list" step 454 and the "loop element list" step 455 shown in fig. 17, so as to loop-acquire the interface elements that are the same as or similar to the search result 471 in the interface 44 (for example, all the interface elements listed in the data list 463 shown in fig. 16 may be finally acquired), and at this time, the processing logic 45 may include an "open web page" step 451, an "fill input box content" step 452, a "click web page element" step 453, a "get element list" step 454, and a "loop element list" step 455, which are sequentially executed.

In summary, according to the application, by identifying the interface elements and displaying the alternative operation options of the selected interface elements to the user, the codes for executing the corresponding operations can be automatically generated according to the operation options selected by the user, the user does not need to know the computer language and the programming skills, and only needs to know the operation functions and the processing logic to be realized, so that the code generation process is greatly simplified, and the code generation efficiency is improved.

FIG. 19 shows a schematic block diagram of an electronic device according to an example embodiment of the present application. Referring to fig. 19, at the hardware level, the electronic device includes a processor 1902, an internal bus 1904, a network interface 1906, a memory 1908 and a non-volatile memory 1910, although it may also include hardware required by other services. The processor 1902 reads a corresponding computer program from the non-volatile memory 1910 into the memory 1908 and then runs, forming a code generation apparatus on a logical level. Of course, besides the software implementation, the present application does not exclude other implementations, such as logic devices or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, and may also be hardware or logic devices.

Referring to fig. 20, in a software implementation, the code generation apparatus may include:

an interface display unit 2001 which displays a target interface determined by a user;

the difference display unit 2002 is configured to perform difference display on interface elements at corresponding positions in the target interface according to positions of the user control icons on the target interface;

a window displaying unit 2003, configured to display a corresponding interactive window when the selection operation of the user control icon on the interface element is detected, where the interactive window includes an alternative operation option for the selected interface element;

The code generation unit 2004 is used for generating a code for implementing corresponding operation on the selected interface element when the selection operation of the user control icon on the alternative operation option is detected;

the operation executing unit 2005 is configured to execute a corresponding operation on the selected interface element to update the target interface.

Referring to fig. 21, in another software implementation, the code generating apparatus may include:

an option display unit 2101 configured to display alternative operation options for a selected interface element according to the interface element selected by a user on a target interface;

the code generation unit 2102 generates a code for performing a corresponding operation according to the operation option selected by the user.

Optionally, the target interface includes: a preset interface or a user-specified interface.

Optionally, the method further includes:

the interface display unit 2103 displays a target interface, wherein the target interface comprises at least one interface element for selection by a user.

Optionally, the method further includes:

a position acquiring unit 2104 that acquires a position of the user manipulation icon on the target interface;

a difference display unit 2105, configured to perform difference display on the interface element at the position when the interface element exists at the position in the target interface;

An element selecting unit 2106, configured to determine, when a selection operation of the user manipulation icon on the interface element at the position is detected, the interface element at the position as the selected interface element.

In the alternative,

further comprising: a content acquisition unit 2107 for acquiring operation contents set by the user for the selected operation option;

the code generation unit 2102 is specifically configured to: and generating a code for implementing corresponding operation according to the operation options selected by the user and the set operation content.

Optionally, the operation object of the operation includes at least one of: and the selected interface element and other interface elements belonging to the same category as the selected interface element in the target interface.

Optionally, the method further includes:

an operation executing unit 2108, configured to execute the operation on the selected interface element to update the target interface.

Optionally, the method further includes:

an obtaining unit 2109, configured to obtain a plurality of interface elements respectively selected by a user in the target interface and corresponding multiple sections of codes;

the assembling unit 2110 is used for sequentially assembling the multiple sections of codes according to the selection sequence of the multiple interface elements by the user.

Referring to fig. 22, in another software implementation, the code generating apparatus may include:

an option display unit 2201, which displays corresponding alternative operation options for the interface elements included in the target interface;

code generation section 2202 generates a code for executing a corresponding operation based on the operation option selected by the user.

FIG. 23 shows a schematic block diagram of an electronic device according to an example embodiment of the present application. Referring to fig. 23, in the hardware level, the electronic device includes a processor 2302, an internal bus 2304, a network interface 2306, a memory 2308 and a nonvolatile memory 2310, but may also include hardware required for other services. The processor 2302 reads a corresponding computer program from the nonvolatile memory 2310 into the memory 2308 and then runs, thereby forming a task execution device on a logical level. Of course, besides the software implementation, the present application does not exclude other implementations, such as logic devices or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, and may also be hardware or logic devices.

Referring to fig. 24, in a software implementation, the task performing device may include:

An obtaining unit 2401, configured to obtain an operation task for a target interface, where the operation task is generated according to an interface element selected in the target interface by a user in advance and an operation mode for the interface element;

the execution unit 2402 executes the operation task to perform an operation based on the operation mode on the interface element in the target interface.

Optionally, when the target interface includes a plurality of interface elements, the operation task is configured to perform operations based on the operation manner on the plurality of interface elements in batch.

The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. A typical implementation device is a computer, which may take the form of a personal computer, laptop computer, cellular telephone, camera phone, smart phone, personal digital assistant, media player, navigation device, email messaging device, game console, tablet computer, wearable device, or a combination of any of these devices.

In a typical configuration, a computer includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.

The foregoing description of specific embodiments has been presented for purposes of illustration and description. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.

The above description is only a preferred embodiment of the present application and should not be taken as limiting the present application, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method of code generation, comprising:

displaying a target interface determined by a user; and circularly executing the following operations to obtain an operation task aiming at the target interface, wherein the operation task comprises codes generated by circularly executing the following operations:

according to the position of a user control icon on the target interface, interface elements at corresponding positions in the target interface are displayed in a distinguishing mode;

2. A method of code generation, comprising:

circularly executing the following operations to obtain an operation task aiming at the target interface, wherein the operation task comprises codes generated by circularly executing the following operations:

displaying alternative operation options aiming at the selected interface element according to the interface element selected by the user on the target interface;

Generating a code for implementing corresponding operation according to the operation option selected by the user;

and executing the operation aiming at the selected interface element to update the target interface.

3. The method of claim 2, wherein the target interface comprises: a preset interface or a user-specified interface.

4. The method of claim 2, further comprising:

and displaying a target interface, wherein the target interface comprises at least one interface element for a user to select.

5. The method of claim 2, further comprising:

acquiring the position of a user control icon on the target interface;

when the interface element exists at the position of the target interface, the interface element at the position is displayed in a distinguishing way;

when the user manipulation icon is detected to select the interface element at the position, determining the interface element at the position as the selected interface element.

6. The method of claim 2,

further comprising: acquiring operation contents set for the selected operation options by the user;

generating a code for implementing corresponding operation on the selected interface element according to the operation option selected by the user, wherein the code comprises the following steps: and generating codes for implementing corresponding operations according to the operation options selected by the user and the set operation contents.

7. The method of claim 2, wherein the operation object of the operation comprises at least one of: and the selected interface element and other interface elements belonging to the same category as the selected interface element in the target interface.

8. The method of claim 2, further comprising:

acquiring a plurality of interface elements respectively selected by a user in the target interface and corresponding multi-section codes;

and sequentially assembling the multiple sections of codes according to the selection sequence of the multiple interface elements by the user.

9. A code generation method, comprising:

displaying corresponding alternative operation options aiming at the interface elements contained in the target interface;

10. A code generation apparatus, comprising:

the interface display unit is used for displaying a target interface determined by a user; and circularly executing the following operations to obtain an operation task aiming at the target interface, wherein the operation task comprises codes generated by circularly executing the following operations:

11. A code generation apparatus, comprising:

the option display unit circularly executes the following operations to obtain an operation task aiming at the target interface, wherein the operation task comprises codes generated by circularly executing the following operations: displaying alternative operation options aiming at the selected interface element according to the interface element selected by the user on the target interface;

the code generating unit is used for generating codes for implementing corresponding operations according to the operation options selected by the user;

And the operation execution unit executes the operation aiming at the selected interface element so as to update the target interface.

12. The apparatus of claim 11, wherein the target interface comprises: a preset interface or a user-specified interface.

13. The apparatus of claim 11, further comprising:

and the interface display unit is used for displaying a target interface, and the target interface comprises at least one interface element for a user to select.

14. The apparatus of claim 11, further comprising:

the position acquisition unit is used for acquiring the position of the user control icon on the target interface;

the difference display unit is used for performing difference display on the interface element at the position when the interface element exists at the position of the target interface;

and the element selection unit is used for determining the interface element at the position as the selected interface element when the selection operation of the user control icon on the interface element at the position is detected.

15. The apparatus of claim 11,

further comprising: the content acquisition unit is used for acquiring the operation content set for the selected operation option by the user;

The code generation unit is specifically configured to: and generating codes for implementing corresponding operations according to the operation options selected by the user and the set operation contents.

16. The apparatus according to claim 11, wherein the operation object of the operation includes at least one of: and the selected interface element and other interface elements in the target interface which belong to the same category as the selected interface element.

17. The apparatus of claim 11, further comprising:

the acquisition unit is used for acquiring a plurality of interface elements respectively selected by a user in the target interface and corresponding multi-section codes;

and the assembling unit is used for sequentially assembling the plurality of sections of codes according to the selection sequence of the plurality of interface elements by the user.

18. A code generation apparatus, comprising:

the option display unit circularly executes the following operations to obtain an operation task aiming at the target interface, wherein the operation task comprises codes generated by circularly executing the following operations: displaying corresponding alternative operation options aiming at interface elements contained in the target interface;

And the operation execution unit is used for executing the operation aiming at the selected interface element so as to update the target interface.

19. A method of task execution, comprising:

acquiring an operation task aiming at a target interface, wherein the operation task is generated according to an interface element selected in the target interface by a user in advance and an operation mode aiming at the interface element, and the operation task comprises a code generated by circularly executing the following operations:

executing corresponding operation aiming at the selected interface element to update the target interface;

20. The method according to claim 19, wherein when the target interface contains a plurality of interface elements, the operation task is used for implementing operation based on the operation mode on the plurality of interface elements in batches.

21. A task execution apparatus, comprising:

the acquisition unit is used for acquiring an operation task aiming at a target interface, wherein the operation task is generated according to an interface element selected in the target interface by a user in advance and an operation mode aiming at the interface element, and the operation task comprises a code generated by circularly executing the following operations:

22. The apparatus according to claim 21, wherein when the target interface includes a plurality of interface elements, the operation task is configured to perform the operation based on the operation manner on the plurality of interface elements in batch.