CN112016889A - Process construction method and device, electronic equipment and storage medium - Google Patents
Process construction method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112016889A CN112016889A CN202010859299.1A CN202010859299A CN112016889A CN 112016889 A CN112016889 A CN 112016889A CN 202010859299 A CN202010859299 A CN 202010859299A CN 112016889 A CN112016889 A CN 112016889A
- Authority
- CN
- China
- Prior art keywords
- target
- voice
- input event
- determining
- historical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 153
- 230000008569 process Effects 0.000 title claims abstract description 132
- 238000010276 construction Methods 0.000 title claims abstract description 90
- 230000009471 action Effects 0.000 claims abstract description 48
- 238000013507 mapping Methods 0.000 claims description 20
- 238000004590 computer program Methods 0.000 claims description 13
- 238000012545 processing Methods 0.000 claims description 8
- 238000001514 detection method Methods 0.000 claims description 5
- 238000009432 framing Methods 0.000 claims description 5
- 238000007781 pre-processing Methods 0.000 claims description 4
- 238000013473 artificial intelligence Methods 0.000 abstract description 14
- 230000006870 function Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000012546 transfer Methods 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 238000013500 data storage Methods 0.000 description 3
- VYZAMTAEIAYCRO-UHFFFAOYSA-N Chromium Chemical compound [Cr] VYZAMTAEIAYCRO-UHFFFAOYSA-N 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
- G06Q10/103—Workflow collaboration or project management
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/02—Feature extraction for speech recognition; Selection of recognition unit
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
- G10L15/18—Speech classification or search using natural language modelling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
- H04L9/3236—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using cryptographic hash functions
- H04L9/3239—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using cryptographic hash functions involving non-keyed hash functions, e.g. modification detection codes [MDCs], MD5, SHA or RIPEMD
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Strategic Management (AREA)
- Human Resources & Organizations (AREA)
- Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Entrepreneurship & Innovation (AREA)
- Computational Linguistics (AREA)
- Multimedia (AREA)
- Acoustics & Sound (AREA)
- Human Computer Interaction (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- Operations Research (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- Tourism & Hospitality (AREA)
- Quality & Reliability (AREA)
- Theoretical Computer Science (AREA)
- Marketing (AREA)
- Data Mining & Analysis (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention relates to the technical field of artificial intelligence, and provides a process construction method, which comprises the following steps: receiving an operation input event, wherein the operation input event comprises a target operation action and a target input value; acquiring a focus coordinate of the operation input event; determining a target environment according to the focus coordinates; determining element type information corresponding to the focus coordinate according to a preset interface of the target environment; determining a target operation object according to the element type information; combining the operation input event and the target operation object to obtain an operation step; receiving an input process construction voice; and constructing a target process according to the process construction voice and the operation steps. The invention also provides a flow construction device, electronic equipment and a medium. The invention can ensure the accuracy of the operation of artificial intelligence.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a process construction method and device, electronic equipment and a storage medium.
Background
At present, with the development of artificial intelligence technology, it is possible to record human operations and generate format data that can be recognized by a computer, and automatically execute these operations by a program according to the format data, thereby realizing that artificial intelligence automatically completes a part of operations. However, in practice, it is found that if various operations are to be implemented by artificial intelligence, multiple recordings need to be performed, which is very inconvenient, and the function of logic judgment cannot be implemented, so that an error occurs in the operation.
Therefore, how to construct format data to ensure the accuracy of the operation of artificial intelligence is a technical problem to be solved.
Disclosure of Invention
In view of the above, it is necessary to provide a process building method capable of ensuring the accuracy of the artificial intelligence operation.
A first aspect of the present invention provides a process construction method, including:
receiving an operation input event, wherein the operation input event comprises a target operation action and a target input value;
acquiring a focus coordinate of the operation input event;
determining a target environment according to the focus coordinates;
determining element type information corresponding to the focus coordinate according to a preset interface of the target environment;
determining a target operation object according to the element type information;
combining the operation input event and the target operation object to obtain an operation step;
receiving an input process construction voice;
and constructing a target process according to the process construction voice and the operation steps.
In a possible implementation manner, after determining the target operation object according to the element class information, the method further includes:
acquiring a historical operation input event adjacent to the operation input event, wherein the historical operation input event comprises a historical operation action and a historical input value;
determining a historical operation object corresponding to the historical operation input event;
judging whether the target operation action is consistent with the historical operation action or not, and judging whether the target operation object is consistent with the historical operation object or not;
if the target operation action is consistent with the historical operation action and the target operation object is consistent with the historical operation object, splicing the target input value and the historical input value to obtain a spliced input value;
the combining the operation input event and the target operation object to obtain the operation step comprises:
and combining the target operation object, the target operation action and the splicing input value to obtain an operation step.
In one possible implementation manner, the operation steps are multiple, and the constructing a target flow according to the flow and the operation steps include:
constructing voice according to the process, determining a target step from a plurality of operation steps, and determining a target logic object from a preset logic object library;
and combining the target step and the target logic object into a target process.
In one possible implementation, after the combining the target step and the target logical object into a target flow, the method further includes:
acquiring first format data corresponding to the target step and acquiring second format data corresponding to the target logical object;
generating format data of the target process according to the first format data and the second format data;
and uploading the format data to a block chain.
In a possible implementation manner, after the process of receiving the input constructs a voice, the method further includes:
preprocessing the process construction voice to obtain a voice to be recognized;
carrying out endpoint detection on the voice to be recognized to obtain the voice content of the voice to be recognized;
framing the voice content and extracting voice characteristics;
inputting the voice features into a trained process construction voice recognition model to obtain process construction characters;
the constructing of the target process according to the process constructing voice and the operating steps comprises:
and constructing a target flow according to the flow construction characters and the operation steps.
In a possible implementation manner, after the determining an operation action corresponding to the operation input event and combining the target operation object and the operation action to obtain the operation step and before the receiving the input flow-structured speech, the flow-structured method further includes:
generating a descriptive text corresponding to the operation step;
and outputting the description words.
In a possible implementation manner, the operation step has a mapping relationship with the description text, and the description text has a mapping relationship with the format data of the operation step, where the mapping relationship includes key-value pair mapping.
A second aspect of the present invention provides a flow construction apparatus, including:
the device comprises a receiving module, a processing module and a processing module, wherein the receiving module is used for receiving an operation input event, and the operation input event comprises a target operation action and a target input value;
the acquisition module is used for acquiring the focus coordinate of the operation input event;
the determining module is used for determining a target environment according to the focus coordinate;
the determining module is further configured to determine, according to a preset interface of the target environment, element class information corresponding to the focus coordinate;
the determining module is further configured to determine a target operation object according to the element class information;
the combination module is used for combining the operation input event and the target operation object to obtain an operation step;
the receiving module is also used for receiving the input process construction voice;
and the construction module is used for constructing the voice according to the process, constructing the target process according to the operation steps and constructing the target process.
A third aspect of the present invention provides an electronic device, which includes a processor and a memory, wherein the processor is configured to implement the flow construction method when executing a computer program stored in the memory.
A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the flow construction method.
By the technical scheme, the method and the device can determine the information such as the coordinates, the environment, the operation objects and the like input by operation, then construct the voice according to the process, select the corresponding operation objects and the logic objects, construct the target process, namely combine a plurality of operations and logic judgments together, and can ensure the accuracy of the operation of artificial intelligence.
Drawings
FIG. 1 is a flow chart of a preferred embodiment of a process building method disclosed in the present invention.
FIG. 2 is a functional block diagram of a preferred embodiment of a process building apparatus according to the present disclosure.
Fig. 3 is a schematic structural diagram of an electronic device implementing a flow construction method according to a preferred embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.
The flow construction method of the embodiment of the invention is applied to the electronic equipment, and can also be applied to a hardware environment formed by the electronic equipment and a server connected with the electronic equipment through a network, and the server and the electronic equipment execute together. Networks include, but are not limited to: a wide area network, a metropolitan area network, or a local area network.
A server may refer to a computer system that provides services to other devices (e.g., electronic devices) in a network. A personal computer may also be called a server if it can externally provide a File Transfer Protocol (FTP) service. In a narrow sense, a server refers to a high-performance computer, which can provide services to the outside through a network, and has higher requirements on stability, security, performance and the like compared with a common personal computer, so that hardware such as a CPU, a chipset, a memory, a disk system and the like is different from that of the common personal computer.
The electronic device is a device capable of automatically performing numerical calculation and/or information processing according to a preset or stored instruction, and the hardware thereof includes, but is not limited to, a microprocessor, an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), an embedded device, and the like. The electronic device may also include a network device and/or a user device. The network device includes, but is not limited to, a single network device, a server group consisting of a plurality of network devices, or a Cloud Computing (Cloud Computing) based Cloud consisting of a large number of hosts or network devices, wherein the Cloud Computing is one of distributed Computing, and is a super virtual computer consisting of a group of loosely coupled computers. The user device includes, but is not limited to, any electronic product that can interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device, for example, a personal computer, a tablet computer, a smart phone, a Personal Digital Assistant (PDA), or the like.
Referring to fig. 1, fig. 1 is a flowchart illustrating a preferred embodiment of a process construction method according to the present invention. The order of the steps in the flowchart may be changed, and some steps may be omitted. The execution subject of the flow construction method may be an electronic device.
S11, receiving an operation input event, wherein the operation input event comprises a target operation action and a target input value.
Wherein the operation input event may include a keyboard input and a mouse input.
In the embodiment of the invention, the keyboard input event and the mouse input event of the operating system can be hooked through HOOK technology (HOOK), so that the accuracy of the detected input event is ensured.
The hook may be a segment of a program that handles messages and is used to hang the hook on the system through a system call. Whenever a particular message is sent, the hook program captures the message before the destination window is reached, i.e. the hook function gets control. At this time, the hook function may process the message, may continue to transfer the message without processing, or may forcibly end the transfer of the message.
And S12, acquiring the focus coordinate of the operation input event.
The focus coordinate may be a coordinate of a focus of the mouse.
And S13, determining a target environment according to the focus coordinates.
In the embodiment of the present invention, the focus coordinate may be transmitted to an automation interface of an operating system (for example, microsoft automation interface), and it may be determined whether a target environment of a current form is an IE, google (Chrome), a windows client program, or the like.
And S14, determining element type information corresponding to the focus coordinate according to a preset interface of the target environment.
In the embodiment of the present invention, different environments (IE, google, windows client program, etc.) are provided with corresponding Application Programming Interfaces (APIs), which can obtain class information (class) of an element of a current focus coordinate, and can call an Interface provided by the template environment to obtain the element class information of the current focus coordinate.
And S15, determining a target operation object according to the element class information.
The target operation object may include a text box, a drop-down box, a login button, and the like.
In the embodiment of the present invention, the element class information and the target operation object have a mapping relationship, and a corresponding target operation object may be determined through the element class information.
And S16, combining the operation input event and the target operation object to obtain an operation step.
Wherein the operation action may include, but is not limited to, clicking, inputting characters, and the like.
As an optional implementation manner, after the step S15, the method further includes:
acquiring a historical operation input event adjacent to the operation input event, wherein the historical operation input event comprises a historical operation action and a historical input value;
determining a historical operation object corresponding to the historical operation input event;
judging whether the target operation action is consistent with the historical operation action or not, and judging whether the target operation object is consistent with the historical operation object or not;
if the target operation action is consistent with the historical operation action and the target operation object is consistent with the historical operation object, splicing the target input value and the historical input value to obtain a spliced input value;
the combining the operation input event and the target operation object to obtain the operation step comprises:
and combining the target operation object, the target operation action and the splicing input value to obtain an operation step.
In this optional implementation, the historical operation object may be an operation object corresponding to a last input event. For example, if the current keyboard input object and the last input object are the same, the previous input word and the current input word may be spliced, and the current input word may not be newly created as a new step, otherwise, multiple operation steps for the object may be generated (for example, inputting "123" in a text box, which may result in three operation steps, the first operation step "input 1 in the text box", the second operation step "input 2 in the text box", the third operation step "input 3 in the text box"), and after the splicing, only one operation step "input 123 in the text box" may be generated.
As an optional implementation manner, after the determining an operation action corresponding to the operation input event, and combining the target operation object and the operation action, and obtaining the operation step, and before the receiving the input flow-building voice, the flow-building method further includes:
generating a descriptive text corresponding to the operation step;
and outputting the description words.
In this alternative embodiment, a descriptive text corresponding to the operation step may be generated, such as: step 1, inputting 123 in a text box; step 2, click the login button, etc. The descriptive characters can be output to a screen of equipment for display, and a user can select which operation steps are used for constructing the flow according to the displayed descriptive characters, so that the problem that the flow construction fails due to the fact that nonexistent operation steps are selected in the construction flow is avoided, and the success rate of the flow construction is improved.
The operation step and the description characters have a mapping relation, the description characters and the format data of the operation step have a mapping relation, and the mapping relation comprises key-value pair mapping.
And S17, receiving the input flow to construct the voice.
In this embodiment of the present invention, the process construction voice may be used to instruct to construct a specific process, for example, the process construction voice may be: and adding a switch, if a is larger than b, executing the step one, otherwise, executing the step two.
As an optional implementation manner, after the process of receiving the input constructs the speech, the method further includes:
preprocessing the process construction voice to obtain a voice to be recognized;
carrying out endpoint detection on the voice to be recognized to obtain the voice content of the voice to be recognized;
framing the voice content and extracting voice characteristics;
inputting the voice features into a trained process construction voice recognition model to obtain process construction characters;
the constructing of the target process according to the process constructing voice and the operating steps comprises:
and constructing a target flow according to the flow construction characters and the operation steps.
In this optional embodiment, a pre-trained process construction language model may be used to analyze the text content of the process construction language, that is, the process construction words, and then each operation step may be constructed into a target process according to the mapping relationship between the process construction words and each operation step.
The trained flow-established speech recognition model can be composed of a general speech recognition model and an operation scene speech recognition model, the general speech recognition model can be obtained by training natural language samples, the operation scene speech recognition model can be obtained by training speech samples corresponding to professional terms of AI-RPA operation scenes, the principle of the language model is to calculate the probability of a certain word appearing under the condition that each word in front of the certain word appears, and then the word with the maximum probability at each position is selected to form a sentence.
S18, constructing voice according to the process, and constructing a target process according to the operation steps.
In the embodiment of the present invention, the process construction voice may be analyzed, the operation steps corresponding to the process construction language may be determined, and the corresponding operation steps may be combined to construct the target process.
Specifically, there are a plurality of operation steps, and the constructing a target flow according to the flow construction voice and the operation steps includes:
constructing voice according to the process, determining a target step from a plurality of operation steps, and determining a target logic object from a preset logic object library;
and combining the target step and the target logic object into a target process.
In this alternative embodiment, a target operation step corresponding to the process building voice may be determined from a plurality of recorded operation steps, a target logical object corresponding to the process building voice may be determined from a preset logical object library, and the target operation step and the target logical object may be combined into a target process.
Various logic objects can be stored in the logic object library in advance, and the logic objects can include but are not limited to judgment logic objects, circulation logic objects and the like.
As an optional implementation, after the target step and the target logical object are combined into a target flow, the method further includes:
acquiring first format data corresponding to the target step and acquiring second format data corresponding to the target logical object;
generating format data of the target process according to the first format data and the second format data;
and uploading the format data to a block chain.
In this optional implementation manner, the first format data corresponding to the target step of the target process and the second format data corresponding to the target logical object may constitute the format data of the target process. The target process can be run by a program of the equipment, and the operation of artificial intelligence is realized.
Wherein the first format data and the second format data may be data that can be recognized and executed by a computer program. The first format data corresponding to the target step can be acquired by software recording, and the second format data corresponding to the target logical object can be stored in a database in advance and called when needed.
The corresponding digest information is obtained based on the format data, and specifically, the digest information is obtained by hashing the format data, for example, using the sha256s algorithm. Uploading summary information to the blockchain can ensure the safety and the fair transparency of the user. The user equipment can download the summary information from the blockchain to verify whether the format data is tampered.
The blockchain referred to in this example is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm, and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
In the method flow described in fig. 1, information such as coordinates, environment, and operation objects of operation input may be determined, then a voice is constructed according to the flow, a corresponding operation object is selected, and a target flow is constructed, that is, a plurality of operations and logical judgment in the language are combined together, so that the accuracy of the operation of artificial intelligence may be ensured.
FIG. 2 is a functional block diagram of a preferred embodiment of a process building apparatus according to the present disclosure.
Referring to fig. 2, the flow building apparatus 20 is operated in an electronic device. The flow construction device 20 may include a plurality of functional modules composed of program code segments. The program code of each program segment in the flow construction apparatus 20 can be stored in a memory and executed by at least one processor to perform some or all of the steps of the flow construction method described in fig. 1.
In this embodiment, the flow construction device 20 may be divided into a plurality of functional modules according to the functions performed by the flow construction device. The functional module may include: the device comprises a receiving module 201, an obtaining module 202, a determining module 203, a combining module 204 and a constructing module 205. The module referred to herein is a series of computer program segments capable of being executed by at least one processor and capable of performing a fixed function and is stored in memory.
The receiving module 201 is configured to receive an operation input event, where the operation input event includes a target operation action and a target input value.
Wherein the operation input event may include a keyboard input and a mouse input.
In the embodiment of the invention, the keyboard input event and the mouse input event of the operating system can be hooked through HOOK technology (HOOK), so that the accuracy of the detected input event is ensured.
The hook may be a segment of a program that handles messages and is used to hang the hook on the system through a system call. Whenever a particular message is sent, the hook program captures the message before the destination window is reached, i.e. the hook function gets control. In this case, the hook function may process (change) the message, may continue to transfer the message without processing, or may forcibly end the transfer of the message.
An obtaining module 202, configured to obtain a focus coordinate of the operation input event.
The focus coordinate may be a coordinate of a focus of the mouse.
And the determining module 203 is used for determining the target environment according to the focus coordinate.
In the embodiment of the present invention, the focus coordinate may be transmitted to an automation interface of an operating system (for example, microsoft automation interface), and it may be determined whether a target environment of a current form is an IE, google (Chrome), a windows client program, or the like.
The determining module 203 is further configured to determine, according to a preset interface of the target environment, element class information corresponding to the focus coordinate.
In the embodiment of the present invention, different environments (IE, google, windows client program, etc.) are provided with corresponding Application Programming Interfaces (APIs), which can obtain class information (class) of an element of a current focus coordinate, and can call an Interface provided by the template environment to obtain the element class information of the current focus coordinate.
The determining module 203 is further configured to determine a target operation object according to the element class information.
The target operation object may include a text box, a drop-down box, a login button, and the like.
In the embodiment of the present invention, the element class information and the target operation object have a mapping relationship, and a corresponding target operation object may be determined through the element class information.
And the combining module 204 is configured to combine the operation input event and the target operation object to obtain an operation step.
Wherein the operation action may include, but is not limited to, clicking, inputting characters, and the like.
The receiving module 201 is further configured to receive an input process-constructed voice.
In this embodiment of the present invention, the process construction voice may be used to instruct to construct a specific process, for example, the process construction voice may be: and adding a switch, if a is larger than b, executing the step one, otherwise, executing the step two.
A building module 205, configured to build a target process according to the process building voice and the operation steps.
In the embodiment of the present invention, the process construction voice may be analyzed, the operation steps corresponding to the process construction language may be determined, and the corresponding operation steps may be combined to construct the target process.
As an optional implementation manner, the obtaining module 202 is further configured to, after the determining module 203 determines the target operation object according to the element class information, obtain a historical operation input event adjacent to the operation input event, where the historical operation input event includes a historical operation action and a historical input value;
the determining module 203 is further configured to determine a historical operation object corresponding to the historical operation input event;
the flow construction apparatus 20 may further include:
the judging module is used for judging whether the target operation action is consistent with the historical operation action or not and judging whether the target operation object is consistent with the historical operation object or not;
and the splicing module is used for splicing the target input value and the historical input value to obtain a spliced input value if the target operation action is consistent with the historical operation action and the target operation object is consistent with the historical operation object.
The mode of combining the operation input event and the target operation object to obtain the operation step specifically comprises the following steps:
and combining the target operation object, the target operation action and the splicing input value to obtain an operation step.
In this optional implementation, the historical operation object may be an operation object corresponding to a last input event. For example, if the current keyboard input object and the last input object are the same, the previous input word and the current input word may be spliced, and the current input word may not be newly created as a new step, otherwise, multiple operation steps for the object may be generated (for example, inputting "123" in a text box, which may result in three operation steps, the first operation step "input 1 in the text box", the second operation step "input 2 in the text box", the third operation step "input 3 in the text box"), and after the splicing, only one operation step "input 123 in the text box" may be generated.
As an optional implementation manner, there are a plurality of operation steps, and the constructing module 205 constructs the speech according to the process and the operation steps, and the manner of constructing the target process specifically includes:
constructing voice according to the process, determining a target step from a plurality of operation steps, and determining a target logic object from a preset logic object library;
and combining the target step and the target logic object into a target process.
In this alternative embodiment, a target operation step corresponding to the process building voice may be determined from a plurality of recorded operation steps, a target logical object corresponding to the process building voice may be determined from a preset logical object library, and the target operation step and the target logical object may be combined into a target process.
Various logic objects can be stored in the logic object library in advance, and the logic objects can include but are not limited to judgment logic objects, circulation logic objects and the like.
As an optional implementation manner, the obtaining module 202 is further configured to, after the combining module 204 combines the target step and the target logical object into a target flow, obtain first format data corresponding to the target step and obtain second format data corresponding to the target logical object;
the flow construction apparatus 20 may further include:
the first generating module is used for generating format data of the target process according to the first format data and the second format data;
and the uploading module is used for uploading the format data to a block chain.
In this optional implementation manner, the first format data corresponding to the target step of the target process and the second format data corresponding to the target logical object may constitute the format data of the target process. The target process can be run by a program of the equipment, and the operation of artificial intelligence is realized.
Wherein the first format data and the second format data may be data that can be recognized and executed by a computer program. The first format data corresponding to the target step can be acquired by software recording, and the second format data corresponding to the target logical object can be stored in a database in advance and called when needed.
The corresponding digest information is obtained based on the format data, and specifically, the digest information is obtained by hashing the format data, for example, using the sha256s algorithm. Uploading summary information to the blockchain can ensure the safety and the fair transparency of the user. The user equipment can download the summary information from the blockchain to verify whether the format data is tampered.
The blockchain referred to in this example is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm, and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
As an optional implementation manner, the process building apparatus 20 may further include:
the preprocessing module is configured to preprocess the process construction voice after the receiving module 201 receives the input process construction voice, and obtain a voice to be recognized;
the detection module is used for carrying out endpoint detection on the voice to be recognized to obtain the voice content of the voice to be recognized;
the framing module is used for framing the voice content and extracting voice characteristics;
and the input module is used for inputting the voice characteristics into the trained process construction voice recognition model to obtain process construction characters.
The construction module 205 constructs the speech according to the process and the operation steps, and the method for constructing the target process specifically includes:
and constructing a target flow according to the flow construction characters and the operation steps.
In this optional embodiment, a pre-trained process construction language model may be used to analyze the text content of the process construction language, that is, the process construction words, and then each operation step may be constructed into a target process according to the mapping relationship between the process construction words and each operation step.
The trained flow-established speech recognition model can be composed of a general speech recognition model and an operation scene speech recognition model, the general speech recognition model can be obtained by training natural language samples, the operation scene speech recognition model can be obtained by training speech samples corresponding to professional terms of AI-RPA operation scenes, the principle of the language model is to calculate the probability of a certain word appearing under the condition that each word in front of the certain word appears, and then the word with the maximum probability at each position is selected to form a sentence.
As an optional implementation manner, the process building apparatus 20 may further include:
a second generating module, configured to determine, by the combining module 204, an operation action corresponding to the operation input event, combine the target operation object and the operation action, and generate a descriptive text corresponding to the operation step after the operation step is obtained and before the receiving module 201 receives the input flow building voice;
and the output module is used for outputting the description words.
In this alternative embodiment, a descriptive text corresponding to the operation step may be generated, such as: step 1, inputting 123 in a text box; step 2, click the login button, etc. The descriptive characters can be output to a screen of equipment for display, and a user can select which operation steps are used for constructing the flow according to the displayed descriptive characters, so that the problem that the flow construction fails due to the fact that nonexistent operation steps are selected in the construction flow is avoided, and the success rate of the flow construction is improved.
The operation step and the description characters have a mapping relation, the description characters and the format data of the operation step have a mapping relation, and the mapping relation comprises key-value pair mapping.
In the flow construction apparatus 20 depicted in fig. 2, information such as coordinates, environment, operation objects, etc. of the operation input may be determined, then a voice is constructed according to the flow, a corresponding operation object is selected, and a target flow is constructed, that is, a plurality of operations and logical judgment in the language are combined together, so that the accuracy of the operation of the artificial intelligence may be ensured.
As shown in fig. 3, fig. 3 is a schematic structural diagram of an electronic device implementing a flow construction method according to a preferred embodiment of the invention. The electronic device 3 comprises a memory 31, at least one processor 32, a computer program 33 stored in the memory 31 and executable on the at least one processor 32, and at least one communication bus 34.
Those skilled in the art will appreciate that the schematic diagram shown in fig. 3 is merely an example of the electronic device 3, and does not constitute a limitation of the electronic device 3, and may include more or less components than those shown, or combine some components, or different components, for example, the electronic device 3 may further include an input/output device, a network access device, and the like.
The electronic device 3 may also include, but is not limited to, any electronic product that can interact with a user through a keyboard, a mouse, a remote controller, a touch panel, or a voice control device, for example, a Personal computer, a tablet computer, a smart phone, a Personal Digital Assistant (PDA), a game console, an Internet Protocol Television (IPTV), a smart wearable device, and the like. The Network where the electronic device 3 is located includes, but is not limited to, the internet, a wide area Network, a metropolitan area Network, a local area Network, a Virtual Private Network (VPN), and the like.
The at least one Processor 32 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a transistor logic device, a discrete hardware component, etc. The processor 32 may be a microprocessor or the processor 32 may be any conventional processor or the like, and the processor 32 is a control center of the electronic device 3 and connects various parts of the whole electronic device 3 by various interfaces and lines.
The memory 31 may be used to store the computer program 33 and/or the module/unit, and the processor 32 may implement various functions of the electronic device 3 by running or executing the computer program and/or the module/unit stored in the memory 31 and calling data stored in the memory 31. The memory 31 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to the use of the electronic device 3, and the like. In addition, the memory 31 may include volatile and non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one disk storage device, a Flash memory device, and so forth.
With reference to fig. 1, the memory 31 in the electronic device 3 stores a plurality of instructions to implement a flow construction method, and the processor 32 can execute the plurality of instructions to implement:
receiving an operation input event, wherein the operation input event comprises a target operation action and a target input value;
acquiring a focus coordinate of the operation input event;
determining a target environment according to the focus coordinates;
determining element type information corresponding to the focus coordinate according to a preset interface of the target environment;
determining a target operation object according to the element type information;
combining the operation input event and the target operation object to obtain an operation step;
receiving an input process construction voice;
and constructing a target process according to the process construction voice and the operation steps.
In an alternative embodiment, after determining the target operand according to the element class information, the processor 32 may execute the plurality of instructions to implement:
acquiring a historical operation input event adjacent to the operation input event, wherein the historical operation input event comprises a historical operation action and a historical input value;
determining a historical operation object corresponding to the historical operation input event;
judging whether the target operation action is consistent with the historical operation action or not, and judging whether the target operation object is consistent with the historical operation object or not;
if the target operation action is consistent with the historical operation action and the target operation object is consistent with the historical operation object, splicing the target input value and the historical input value to obtain a spliced input value;
the combining the operation input event and the target operation object to obtain the operation step comprises:
and combining the target operation object, the target operation action and the splicing input value to obtain an operation step.
Specifically, the processor 32 may refer to the description of the relevant steps in the embodiment corresponding to fig. 1 for a specific implementation method of the instruction, which is not described herein again.
In the electronic device 3 described in fig. 3, information such as coordinates, environment, and operation objects of the operation input may be determined, then a voice is constructed according to the process, a corresponding operation object is selected, and a target process is constructed, that is, a plurality of operations and logical judgment in the language are combined together, so that the accuracy of the operation of the artificial intelligence may be ensured.
The integrated modules/units of the electronic device 3 may be stored in a computer-readable storage medium if they are implemented in the form of software functional units and sold or used as separate products. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program code may be in source code form, object code form, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), etc.
In the embodiments provided in the present invention, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned. Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the system claims may also be implemented by one unit or means in software or hardware. The terms second, etc. are used to denote names, but not any particular order.
Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims (10)
1. A process construction method, characterized in that the process construction method comprises:
receiving an operation input event, wherein the operation input event comprises a target operation action and a target input value;
acquiring a focus coordinate of the operation input event;
determining a target environment according to the focus coordinates;
determining element type information corresponding to the focus coordinate according to a preset interface of the target environment;
determining a target operation object according to the element type information;
combining the operation input event and the target operation object to obtain an operation step;
receiving an input process construction voice;
and constructing a target process according to the process construction voice and the operation steps.
2. The process construction method according to claim 1, wherein after determining the target operation object according to the element class information, the process construction method further comprises:
acquiring a historical operation input event adjacent to the operation input event, wherein the historical operation input event comprises a historical operation action and a historical input value;
determining a historical operation object corresponding to the historical operation input event;
judging whether the target operation action is consistent with the historical operation action or not, and judging whether the target operation object is consistent with the historical operation object or not;
if the target operation action is consistent with the historical operation action and the target operation object is consistent with the historical operation object, splicing the target input value and the historical input value to obtain a spliced input value;
the combining the operation input event and the target operation object to obtain the operation step comprises:
and combining the target operation object, the target operation action and the splicing input value to obtain an operation step.
3. The process construction method according to claim 1, wherein there are a plurality of said operation steps, said constructing a speech according to said process and said operation steps, constructing a target process comprises:
constructing voice according to the process, determining a target step from a plurality of operation steps, and determining a target logic object from a preset logic object library;
and combining the target step and the target logic object into a target process.
4. The process building method according to claim 3, wherein after said combining said target step and said target logical object into a target process, said process building method further comprises:
acquiring first format data corresponding to the target step and acquiring second format data corresponding to the target logical object;
generating format data of the target process according to the first format data and the second format data;
and uploading the format data to a block chain.
5. The process construction method according to claim 1, wherein after the process construction voice receiving the input, the process construction method further comprises:
preprocessing the process construction voice to obtain a voice to be recognized;
carrying out endpoint detection on the voice to be recognized to obtain the voice content of the voice to be recognized;
framing the voice content and extracting voice characteristics;
inputting the voice features into a trained process construction voice recognition model to obtain process construction characters;
the constructing of the target process according to the process constructing voice and the operating steps comprises:
and constructing a target flow according to the flow construction characters and the operation steps.
6. The process construction method according to any one of claims 1 to 5, wherein after the step of determining the operation action corresponding to the operation input event and combining the target operation object and the operation action to obtain the operation object and before the step of receiving the input process construction voice, the process construction method further comprises:
generating a descriptive text corresponding to the operation step;
and outputting the description words.
7. The process construction method according to claim 6, wherein the operation step has a mapping relationship with the descriptor, and the descriptor has a mapping relationship with format data of the operation step, and the mapping relationship includes key-value pair mapping.
8. A process building apparatus, characterized in that the process building apparatus comprises:
the device comprises a receiving module, a processing module and a processing module, wherein the receiving module is used for receiving an operation input event, and the operation input event comprises a target operation action and a target input value;
the acquisition module is used for acquiring the focus coordinate of the operation input event;
the determining module is used for determining a target environment according to the focus coordinate;
the determining module is further configured to determine, according to a preset interface of the target environment, element class information corresponding to the focus coordinate;
the determining module is further configured to determine a target operation object according to the element class information;
the combination module is used for combining the operation input event and the target operation object to obtain an operation step;
the receiving module is also used for receiving the input process construction voice;
and the construction module is used for constructing the voice according to the process, constructing the target process according to the operation steps and constructing the target process.
9. An electronic device, characterized in that the electronic device comprises a processor and a memory, the processor being configured to execute a computer program stored in the memory to implement the flow construction method according to any one of claims 1 to 7.
10. A computer-readable storage medium storing at least one instruction which, when executed by a processor, implements the flow construction method of any one of claims 1 through 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010859299.1A CN112016889A (en) | 2020-08-24 | 2020-08-24 | Process construction method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010859299.1A CN112016889A (en) | 2020-08-24 | 2020-08-24 | Process construction method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112016889A true CN112016889A (en) | 2020-12-01 |
Family
ID=73505766
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010859299.1A Pending CN112016889A (en) | 2020-08-24 | 2020-08-24 | Process construction method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112016889A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114926151A (en) * | 2022-06-21 | 2022-08-19 | 中关村科学城城市大脑股份有限公司 | RPA flow automatic generation method and device based on reinforcement learning |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109032568A (en) * | 2018-07-13 | 2018-12-18 | 孟智平 | Generation method, terminal device and the storage medium of programmed logic |
| CN110851108A (en) * | 2019-11-18 | 2020-02-28 | 北京小米移动软件有限公司 | Electronic equipment operation method and device, electronic equipment and storage medium |
| US20200118545A1 (en) * | 2018-10-15 | 2020-04-16 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method and system for building speech recognizer, and speech recognition method and system |
| CN111459460A (en) * | 2020-03-19 | 2020-07-28 | 佛山青藤信息科技有限公司 | Service data processing method and system |
| CN111553655A (en) * | 2020-04-27 | 2020-08-18 | 平安医疗健康管理股份有限公司 | Method and system for showing handling history |
-
2020
- 2020-08-24 CN CN202010859299.1A patent/CN112016889A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109032568A (en) * | 2018-07-13 | 2018-12-18 | 孟智平 | Generation method, terminal device and the storage medium of programmed logic |
| US20200118545A1 (en) * | 2018-10-15 | 2020-04-16 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method and system for building speech recognizer, and speech recognition method and system |
| CN110851108A (en) * | 2019-11-18 | 2020-02-28 | 北京小米移动软件有限公司 | Electronic equipment operation method and device, electronic equipment and storage medium |
| CN111459460A (en) * | 2020-03-19 | 2020-07-28 | 佛山青藤信息科技有限公司 | Service data processing method and system |
| CN111553655A (en) * | 2020-04-27 | 2020-08-18 | 平安医疗健康管理股份有限公司 | Method and system for showing handling history |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114926151A (en) * | 2022-06-21 | 2022-08-19 | 中关村科学城城市大脑股份有限公司 | RPA flow automatic generation method and device based on reinforcement learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7316453B2 (en) | Object recommendation method and device, computer equipment and medium | |
| CN111739016B (en) | Target detection model training method and device, electronic equipment and storage medium | |
| CN111797351A (en) | Page data management method and device, electronic equipment and medium | |
| US11741177B2 (en) | Entity validation of a content originator | |
| CN112132032A (en) | Traffic sign detection method and device, electronic equipment and storage medium | |
| CN111986194A (en) | Medical annotation image detection method and device, electronic equipment and storage medium | |
| CN112053143B (en) | Fund routing method, apparatus, electronic device and storage medium | |
| WO2023109631A1 (en) | Data processing method and apparatus, device, storage medium, and program product | |
| CN112001159B (en) | Document generation method and device, electronic equipment and storage medium | |
| CN112395886A (en) | Similar text determination method and related equipment | |
| CN114035827A (en) | Application program updating method, device, equipment and storage medium | |
| CN111858891A (en) | Question-answer library construction method and device, electronic equipment and storage medium | |
| WO2021237414A1 (en) | Information pushing method and apparatus, electronic device, and storage medium | |
| CN112990625A (en) | Method and device for allocating annotation tasks and server | |
| CN111128233A (en) | Recording detection method and device, electronic equipment and storage medium | |
| CN114116108A (en) | Dynamic rendering method, device, equipment and storage medium | |
| CN111968624B (en) | Data construction method, device, electronic equipment and storage medium | |
| CN112582073B (en) | Medical information acquisition method, device, electronic equipment and medium | |
| CN112016889A (en) | Process construction method and device, electronic equipment and storage medium | |
| CN112542172A (en) | Communication auxiliary method, device, equipment and medium based on online conference | |
| CN112184059A (en) | Scoring analysis method and device, electronic equipment and storage medium | |
| CN115334698B (en) | Construction method, device, terminal and medium of target 5G safety network of target range | |
| CN112383819B (en) | Video frame extraction method and related equipment | |
| CN114820409A (en) | Image anomaly detection method and device, electronic device and storage medium | |
| CN112528647A (en) | Similar text generation method and device, electronic equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |