CN108845788B - Analog manipulation method and portable terminal - Google Patents
Analog manipulation method and portable terminal Download PDFInfo
- Publication number
- CN108845788B CN108845788B CN201810596582.2A CN201810596582A CN108845788B CN 108845788 B CN108845788 B CN 108845788B CN 201810596582 A CN201810596582 A CN 201810596582A CN 108845788 B CN108845788 B CN 108845788B
- Authority
- CN
- China
- Prior art keywords
- input
- user
- page
- overlay
- target application
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Disclosed are an analog manipulation method and a portable terminal. The analog manipulation method comprises the following steps: receiving a voice input from a user; recognizing the voice input and obtaining a corresponding simulation operation instruction; starting a target application based on the instruction; generating an overlay page to overlay at least a portion of a display interface of the target application; and automatically performing a series of operations to implement the instructions.
Description
Technical Field
The present invention relates to the technical field of information processing, and more particularly, to the technical field of virtual assistant and simulation manipulation.
Background
The application of the voice-controlled mobile Virtual assistants (Virtual assistants VAs) enters the mass market across technology cracks, and helps users to search, make calls, send short messages and other functions through voice commands. The simulation operation is used as a new operation mode of a virtual assistant and starts to be displayed on a recent intelligent mobile terminal. The implementation principle of the method is that based on the barrier-free function provided by the Android operating system, page jumping and touch screen operation of a user are simulated, and the final form is represented as an operation flow indicated by a voice instruction of the user. For example, if a user wants to send a WeChat red envelope of five money to a WeChat, in general, the user needs to perform a series of operations of clicking on the WeChat icon to open a WeChat application, entering an address book to find the WeChat, clicking on the head portrait of the WeChat to enter a dialog box, then clicking on the red envelope and inputting the amount. However, in the case of applying the analog manipulation, the user only needs to input a voice command of "WeChat Red envelope for five money of the invention", and the above-described series of operations can be automatically performed without the user's participation. That is, by simulating the manipulation, an operation by a user touch click is simulated with an instruction automatically executed.
However, the current implementation scheme has a problem in that a series of operation processes automatically performed in response to a voice instruction of a user cannot be completed within the VA application interaction, and there are generally a plurality of page jumps and the like, such as jumping from a main interface to a wechat start interface, jumping from the wechat start interface to an address book interface, and jumping from the address book interface to a contact dialog box. Frequent, rapid and visual changes of the interactive interface easily cause dazzling feeling, and seriously affect the interactive experience of the user. In addition, during the simulation manipulation, there may be a case where the process is terminated due to the user mistakenly touching the screen.
Disclosure of Invention
In view of the above, it is desirable to provide a new simulation manipulation mode ui (user interface) interaction mode suitable for VA to solve the complex process of multiple page jumps and optimize the interaction experience of the user.
According to an aspect of the present invention, there is provided an analog manipulation method, including: receiving a voice input from a user; recognizing the voice input and obtaining a corresponding simulation operation instruction; starting a target application based on the instruction; generating an overlay page to overlay at least a portion of a display interface of the target application; and automatically performing a series of operations to implement the instructions.
According to another aspect of the present invention, there is provided a portable terminal including: voice input means for receiving a voice input from a user; the recognition device is used for recognizing the voice input and obtaining a corresponding simulation operation instruction; executing means for starting a target application and automatically executing a series of operations to implement the instruction based on the instruction; and the overlay page generating device is used for generating an overlay page to cover at least part of the display interface of the target application.
In the simulation control method and the portable terminal, the covering page covers at least part of the display interface of the target application, so that the UI interface of the simulation operation can be normalized and unified, and the condition that the UIs are disordered and inconsistent when different applications are controlled is avoided. In addition, the simulation operation process is completed through a friendly and stable UI display interface, so that dazzling discomfort can be avoided, and more attractive visual experience is provided for users. In addition, the processing termination caused by mistaken touch of the screen in the simulation operation process of the user can be avoided. In addition, the covering page is partially transparent or the progress state and the result are displayed on the covering page, so that a user can know the actual execution progress state and the result in real time in the simulation operation process, and the blind feeling of the user on the real-time state and the blind feeling of whether the execution result is successful or not are avoided.
Drawings
Fig. 1 is a flowchart illustrating a procedure of a simulation manipulation method according to a first embodiment of the present invention;
fig. 2 is a flowchart illustrating a procedure of a simulation manipulation method according to a second embodiment of the present invention;
fig. 3 is a flowchart illustrating a procedure of a simulation manipulation method according to a third embodiment of the present invention;
fig. 4 is a functional block diagram illustrating the configuration of a portable terminal according to a first embodiment of the present invention;
fig. 5 is a functional block diagram illustrating a configuration of a portable terminal according to a second embodiment of the present invention; and
fig. 6 is a functional block diagram illustrating a configuration of a portable terminal according to a third embodiment of the present invention.
Detailed Description
Various preferred embodiments of the present invention will be described below with reference to the accompanying drawings. The following description with reference to the accompanying drawings is provided to assist in understanding the exemplary embodiments of the invention as defined by the claims and their equivalents. It includes various specific details to assist understanding, but they are to be construed as merely illustrative. Accordingly, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Also, in order to make the description clearer and simpler, a detailed description of functions and configurations well known in the art will be omitted.
First, an analog manipulation method according to a first embodiment of the present invention will be described with reference to fig. 1. As shown in fig. 1, the analog manipulation method includes the following steps.
First, in step S101, a voice input from a user is received.
Then, in step S102, the voice input is recognized and a corresponding instruction of the simulated operation class is obtained, for example, a WeChat red envelope for five money to the invention.
Next, in step S103, based on the instruction, a target application, for example, a wechat application, is started. Of course, there may be more than one target application corresponding to the user's voice input.
Then, in step S104, an overlay page is generated to overlay at least a portion of the display interface of the target application. The overlay page is a UI interface overlaid on top of other applications and interfaces, which becomes the visible interface for the user.
During the simulation operation, the covering page is always stably displayed above other jumping pages. Therefore, by the simulation control method according to the first embodiment of the present invention, the coverage page of simulation control can be normalized and unified, and the situation of confusion and inconsistency of UI interfaces when different applications are controlled is avoided. The process of simulating control is completed through friendly and stable covering pages, so that dazzling discomfort is avoided, and more attractive visual experience is provided for users. In addition, as the user can not operate other covered applications and interfaces by clicking the covering page, the processing termination caused by mistakenly touching the screen by the user in the process of simulating the operation can be avoided by the covering page.
Here, the overlay page may be opaque, thereby completely isolating the user from frequent, rapid, but visually visible UI changes, avoiding discomfort.
Or, alternatively, the overlay page may also be partially transparent. The partially transparent covering page can greatly relieve the dazzling discomfort, and meanwhile, a user can visually see the page turning process, so that the execution state and the progress of the simulation manipulation process can be known.
For example, the overlay page may be a screenshot image of the current operating interface. This is done for the purpose of enabling the user to understand the current simulated manipulation progress while avoiding visual discomfort and termination of processing caused by the user touching the screen by mistake. Of course, if the screen capture frequency is high, the visual discomfort of the user increases, and only the technical effect of avoiding the false touch can be achieved. Therefore, more preferably, the screen capture frequency may be set to be lower, so that dazzling can be avoided to some extent, while achieving the effect of the user to know the progress of the treatment and to avoid misoperation.
For another example, the overlay page may only overlay an input area on the application interface to avoid a malfunction.
As another example, different overlay pages may also be automatically adapted based on different applications currently executing. Also, the transparency level of the overlay page may be adjusted when different applications are executed. Alternatively, different images, text messages, or audio messages may be displayed on the overlay interface.
Then, in step S105, a series of operations to realize the instruction are automatically performed.
For example, in the case where the analog manipulation corresponding to the voice instruction of the user involves the input of intermediate information, the analog manipulation method will need to determine the intermediate input information. Therefore, the step S105 of automatically performing a series of operations to implement the instruction may further include: acquiring an input area in the target application; the received voice input is analyzed and it is determined whether it contains input information.
When the voice input includes input information input to the corresponding input area, that is, when a user inputs a command of "WeChat Red envelope for five money WeChat Ming", the input information (five money) included therein is extracted, and the extracted input information is input to the corresponding input area, and the subsequent operation is performed.
On the other hand, when the voice input does not include the input information input to the corresponding input area, for example, when the user inputs a voice command of "red packet for WeChat", the voice information is output to prompt the user that the current flow needs to be subjected to intermediate input. For example, when the page state is covered, VA may prompt the user in a tts (text To speech) manner that the current flow needs an intermediate input (please say the red packet amount). After the TTS prompts the user, ASR (automatic Speech recognition) is automatically activated to recognize the intermediate input information (the amount of the red packet) input by the user, and ASR states (a recording state, a real-time volume, a voice detection state, and the like) are appropriately displayed on the overlay page. Then, a subsequent operation is performed based on the identified intermediate input information.
After the series of operations is completed, the simulation manipulation process is ended. The process proceeds to step S106. In step S106, the execution result (success or failure) of the simulation manipulation is analyzed, and the result information is presented to the user in an appropriate manner. For example, textual information indicating success or failure may be displayed on the overlay page. Alternatively, the user may be notified of the execution result of the simulation manipulation in the form of audio information. Finally, in step S107, after a delay of a predetermined time period has elapsed, the overlay page is closed, and the normal UI interface is restored.
Next, a simulation manipulation method according to a second embodiment of the present invention will be described with reference to fig. 2. As shown in fig. 2, the analog manipulation method according to the second embodiment includes the following steps.
Similarly to the first embodiment, first, in step S201, a voice input from a user is received.
Then, in step S202, the voice input is recognized and a corresponding instruction of the simulated operation class is obtained, for example, a WeChat red envelope for five money to the invention.
Next, in step S203, based on the instruction, a target application, for example, a wechat application, is started.
Then, in step S204, an overlay page is generated to overlay at least a portion of the display interface of the target application.
Next, unlike the first embodiment, in step S205, a series of operations to implement the instruction are automatically executed while determining a real-time state of the series of operations, and the real-time state is displayed on the overlay page.
Therefore, in the second embodiment, in addition to the technical effect of the first embodiment, the progress state and the result of the simulation operation and control execution can be better presented to the user in real time, so that the user can know the progress state and the result of the actual execution in real time in the simulation operation and control process, and the blind feeling of the user on the real-time state and the blind feeling of whether the execution result is successful or not are avoided.
Wherein the real-time status comprises at least one of: target application information of current operation, current real-time operation progress, multi-round interactive intermediate input flow state and final operation result.
For example, the currently operating target application information may include, but is not limited to, an application name, an icon, and the like. The current real-time operation progress may include, but is not limited to, the total number of operation steps, the current operation step number, the current operation step name or brief description, the operation progress percentage, and the like. The multi-round interactive intermediate input flow state may include, but is not limited to, simple descriptions and prompts of the required intermediate input (e.g., please say the amount of the red envelope), Automatic Speech Recognition (ASR) state (e.g., recording state, real-time volume, Speech detection state, etc.), ASR results, and the like. The final operation result may be success or failure.
After the series of operations is completed, the simulation manipulation process is ended. The process proceeds to step S106. In step S106, the execution result (success or failure) of the simulation manipulation is analyzed, and the result information is presented to the user in an appropriate manner. For example, textual information indicating success or failure may be displayed on the overlay page. Alternatively, the user may be notified of the execution result of the simulation manipulation in the form of audio information. Finally, in step S107, after a delay of a predetermined time period has elapsed, the overlay page is closed, and the normal UI interface is restored.
Next, an analog manipulation method according to a third embodiment of the present invention will be described with reference to fig. 3. As shown in fig. 3, the analog manipulation method includes the following steps.
Similarly to the first embodiment, first, in step S301, a voice input from a user is received.
Then, in step S302, the voice input is recognized and a corresponding instruction of the simulated operation class is obtained, for example, a WeChat red envelope for five money to the invention.
Next, in step S303, based on the instruction, a target application, for example, a wechat application, is started. Of course, there may be more than one target application corresponding to the user's voice input.
Then, in step S304, an overlay page is generated to overlay at least a portion of the display interface of the target application. The overlay page is a UI interface overlaid on top of other applications and interfaces, which becomes the visible interface for the user.
Unlike the first embodiment, next, in step S305, it is determined whether or not manual input by the user is required. If it is determined yes in step S305, the process proceeds to step S306. In step S306, the overlay page is adjusted such that at least one input area of the target application interface is exposed in the overlay page. Then, the process proceeds to step S307. In step S307, a series of operations to implement the instruction are automatically performed. If no is determined in step S306, no adjustment is made to the overlay page, and the process proceeds directly to step S307.
That is, in the third embodiment, a hole may be dug in the overlay page to expose an area in the application interface that is overlaid thereunder where input can be made, so that when a user is required to make a conventional touch input during the simulated manipulation, input can be made through the exposed area.
It should be noted here that in fig. 3, the step of determining whether or not the manual input by the user is required is shown as a step performed once before step S307. Alternatively, however, it is also possible during the execution of step S307, namely: in the process of automatically performing the series of operations, the determination as to whether or not manual input by the user is required is performed every predetermined period of time. And if the manual input is needed, adjusting the coverage page, otherwise, continuously executing the series of operations.
After the series of operations is completed, the simulation manipulation process is ended. The process proceeds to step S308. In step S308, the execution result (success or failure) of the simulation manipulation is analyzed, and the result information is presented to the user in an appropriate manner. Finally, in step S309, after a delay of a predetermined time period elapses, the overlay page is closed, and the normal UI interface is restored.
Next, a portable terminal according to a first embodiment of the present invention will be described with reference to fig. 4. As shown in fig. 4, the portable terminal 400 includes: speech input means 401, recognition means 402, execution means 403 and overlay page generation means 404.
The voice input device 401 receives a voice input from a user. For example, the voice input device 401 is a microphone.
The recognition means 402 is used to recognize the speech input and obtain corresponding instructions of the analog operation type, for example, a WeChat red envelope for five money to the invention.
The execution means 403 is used to launch a target application, such as a WeChat application, based on the instruction. Of course, there may be more than one target application corresponding to the user's voice input. And, the executing device 403 is also used for automatically executing a series of operations to implement the instruction.
The overlay page generating device 404 is configured to generate an overlay page to overlay at least a portion of the display interface of the target application. The overlay page is a UI interface overlaid on top of other applications and interfaces, which becomes the visible interface for the user.
During the simulation operation, the covering page is always stably displayed above other jumping pages. Therefore, by the portable terminal according to the first embodiment of the present invention, the overlay page for simulation operation can be normalized and unified, and the situation that the UI interfaces are disordered and inconsistent when different applications are operated is avoided. The process of simulating control is completed through friendly and stable covering pages, so that dazzling discomfort is avoided, and more attractive visual experience is provided for users. In addition, as the user can not operate other covered applications and interfaces by clicking the covering page, the processing termination caused by mistakenly touching the screen by the user in the process of simulating the operation can be avoided by the covering page.
Here, the overlay page may be opaque, thereby completely isolating the user from frequent, rapid, but visually visible UI changes, avoiding discomfort.
Or, alternatively, the overlay page may also be partially transparent. The partially transparent covering page can greatly relieve the dazzling discomfort, and meanwhile, a user can visually see the page turning process, so that the execution state and the progress of the simulation manipulation process can be known.
For example, the overlay page may be a screenshot image of the current operating interface. This is done for the purpose of enabling the user to understand the current simulated manipulation progress while avoiding visual discomfort and termination of processing caused by the user touching the screen by mistake. Of course, if the screen capture frequency is high, the visual discomfort of the user increases, and only the technical effect of avoiding the false touch can be achieved. Therefore, more preferably, the screen capture frequency may be set to be lower, so that dazzling can be avoided to some extent, while achieving the effect of the user to know the progress of the treatment and to avoid misoperation.
For another example, the overlay page may only overlay an input area on the application interface to avoid a malfunction.
As another example, different overlay pages may also be automatically adapted based on different applications currently executing. Also, the transparency level of the overlay page may be adjusted when different applications are executed. Alternatively, different images, text messages, or audio messages may be displayed on the overlay interface.
For example, in the case where the analog manipulation corresponding to the voice instruction of the user involves the input of intermediate information, the analog manipulation method will need to determine the intermediate input information. Therefore, the executing device 403 may further include: an obtaining unit 4031, configured to obtain an input area in the target application; an analyzing unit 4032, configured to analyze the received voice input and determine whether the received voice input contains input information.
When the input information input to the corresponding input area is included in the voice input, that is, when the user inputs an instruction of "a voice for a WeChat Red envelope for five money of the invention", the analysis unit 4032 extracts the input information (five money) included therein. And the execution device 403 further includes an execution unit 4033 for inputting the extracted input information into the corresponding input area and performing subsequent operations.
On the other hand, when the voice input does not include the input information input to the corresponding input area, for example, when the user inputs a voice instruction of "red envelope for WeChat": and the prompting unit is used for outputting voice information to prompt a user that the current process needs to be subjected to intermediate input. For example, when the page state is covered, VA may prompt the user in a tts (text To speech) manner that the current flow needs an intermediate input (please say the red packet amount). After the TTS prompts the user, ASR (automatic Speech recognition) is automatically activated to recognize the intermediate input information (the amount of the red packet) input by the user, and ASR states (a recording state, a real-time volume, a voice detection state, and the like) are appropriately displayed on the overlay page. Then, a subsequent operation is performed based on the identified intermediate input information.
After the series of operations is completed, the simulation manipulation process is ended. The execution means 403 is further configured to analyze the execution result (success or failure) of the simulation manipulation and present the result information to the user in an appropriate manner. For example, textual information indicating success or failure may be displayed on the overlay page. Alternatively, the user may be notified of the execution result of the simulation manipulation in the form of audio information. And, the executing device 403 is further configured to close the overlay page and return to the normal UI interface after a delay of a predetermined time period.
Next, a portable terminal according to a second embodiment of the present invention will be described with reference to fig. 5. As shown in fig. 5, the portable terminal 500 includes: a voice input means 501, a recognition means 502, an execution means 503 and an overlay page generation means 504. Unlike the first embodiment, the portable terminal 500 further includes: a monitoring device 505 and a control device 505.
The voice input device 501 receives a voice input from a user. For example, the voice input device 501 is a microphone.
The recognition means 502 is used to recognize the speech input and obtain corresponding instructions of the analog operation type, for example, a WeChat red envelope for five money to the invention.
The execution means 503 is used to launch a target application, such as a WeChat application, based on the instruction. Of course, there may be more than one target application corresponding to the user's voice input. And, the executing device 503 is also used for automatically executing a series of operations for realizing the instruction.
The overlay page generating device 504 is configured to generate an overlay page to overlay at least a portion of the display interface of the target application. The overlay page is a UI interface overlaid on top of other applications and interfaces, which becomes the visible interface for the user.
The monitoring means 505 is used to determine the real-time status of the series of operations.
The control device 506 is used for displaying the real-time status on the overlay page.
Therefore, in the second embodiment, in addition to the technical effect of the first embodiment, the progress state and the result of the simulation operation and control execution can be better presented to the user in real time, so that the user can know the progress state and the result of the actual execution in real time in the simulation operation and control process, and the blind feeling of the user on the real-time state and the blind feeling of whether the execution result is successful or not are avoided.
Wherein the real-time status comprises at least one of: target application information of current operation, current real-time operation progress, multi-round interactive intermediate input flow state and final operation result.
For example, the currently operating target application information may include, but is not limited to, an application name, an icon, and the like. The current real-time operation progress may include, but is not limited to, the total number of operation steps, the current operation step number, the current operation step name or brief description, the operation progress percentage, and the like. The multi-round interactive intermediate input flow state may include, but is not limited to, simple descriptions and prompts of the required intermediate input (e.g., please say the amount of the red envelope), Automatic Speech Recognition (ASR) state (e.g., recording state, real-time volume, Speech detection state, etc.), ASR results, and the like. The final operation result may be success or failure.
After the series of operations is completed, the simulation manipulation process is ended. The execution means 503 is further configured to analyze the execution result (success or failure) of the simulation manipulation and present the result information to the user in an appropriate manner. For example, textual information indicating success or failure may be displayed on the overlay page. Alternatively, the user may be notified of the execution result of the simulation manipulation in the form of audio information. And, the executing device 503 is further configured to close the overlay page after a delay of a predetermined time period, and return to the normal UI interface.
Finally, a portable terminal according to a third embodiment of the present invention will be described with reference to fig. 6. As shown in fig. 6, the portable terminal 600 includes: speech input means 601, recognition means 602, execution means 603 and overlay page generation means 604.
The voice input device 601 receives a voice input from a user. For example, the voice input device 601 is a microphone.
The recognition means 602 is used to recognize the voice input and obtain corresponding instructions of the analog operation type, for example, a WeChat red envelope for five money to the invention.
The executing means 603 is configured to launch a target application, such as a WeChat application, based on the instruction. Of course, there may be more than one target application corresponding to the user's voice input. And, the executing device 603 is further configured to automatically execute a series of operations to implement the instruction.
The overlay page generating device 604 is configured to generate an overlay page to overlay at least a portion of the display interface of the target application. The overlay page is a UI interface overlaid on top of other applications and interfaces, which becomes the visible interface for the user.
In the third embodiment, the overlay page generating means 604 further comprises: a determination unit 6041 for determining whether or not manual input by the user is required; and an adjusting unit 6042 configured to, when a manual input by a user is required, adjust the overlay page such that at least one input area of the target application interface is exposed in the overlay page.
After the series of operations is completed, the simulation manipulation process is ended. The execution means 603 is further configured to analyze the execution result (success or failure) of the simulation manipulation and present the result information to the user in an appropriate manner. For example, textual information indicating success or failure may be displayed on the overlay page. Alternatively, the user may be notified of the execution result of the simulation manipulation in the form of audio information. And, the executing device 603 is further configured to close the overlay page and restore to the normal UI interface after a delay of a predetermined time period.
Hereinabove, the analog manipulation method and the portable terminal according to the first to third embodiments of the present invention are described in detail with reference to fig. 1 to 6, respectively. Here, it is to be noted that, although the above embodiments are separately described, those skilled in the art will understand that, in practice, the above three embodiments may be combined with each other. For example, in the third embodiment, it is also possible to determine a real-time status of a series of operations to implement the instruction while automatically performing the series of operations, and display the real-time status on the overlay page. As another example, the various implementations of the overlay page described in the first embodiment may also be applied to the second and third embodiments as well. Of course, any other combination of the above embodiments as would occur to one skilled in the art is also intended to be included within the scope of the present invention.
In the simulation control method and the portable terminal, the covering page covers at least part of the display interface of the target application, so that the UI interface of the simulation operation can be normalized and unified, and the condition that the UIs are disordered and inconsistent when different applications are controlled is avoided. In addition, the simulation operation process is completed through a friendly and stable UI display interface, so that dazzling discomfort can be avoided, and more attractive visual experience is provided for users. In addition, the processing termination caused by mistaken touch of the screen in the simulation operation process of the user can be avoided. In addition, the covering page is partially transparent or the progress state and the result are displayed on the covering page, so that a user can know the actual execution progress state and the result in real time in the simulation operation process, and the blind feeling of the user on the real-time state and the blind feeling of whether the execution result is successful or not are avoided.
It should be noted that, in the present specification, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Finally, it should be noted that the series of processes described above includes not only processes performed in time series in the order described herein, but also processes performed in parallel or individually, rather than in time series.
Through the above description of the embodiments, those skilled in the art will clearly understand that the present invention may be implemented by software plus a necessary hardware platform, and may also be implemented by software entirely. With this understanding in mind, all or part of the technical solutions of the present invention that contribute to the background can be embodied in the form of a software product, which can be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and includes instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods according to the embodiments or some parts of the embodiments of the present invention.
The present invention has been described in detail, and the principle and embodiments of the present invention are explained herein by using specific examples, which are only used to help understand the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. An analog manipulation method, comprising:
receiving a voice input from a user;
recognizing the voice input and obtaining a corresponding simulation operation instruction;
starting a target application based on the instruction;
generating a covering page to cover at least part of a display interface of the target application, wherein the covering page is stably displayed above other jump pages all the time in the process of simulation operation, and when a user needs to perform touch input in the process of simulation operation, holes are dug in the covering page to expose an area which can be input in the application interface covered under the covering page; and
automatically performing a series of operations to implement the instructions;
wherein the method further comprises: and after the series of operations are completed, displaying the execution result of the simulation operation on the overlay page, and closing the overlay page after a delay of a preset time period.
2. The analog manipulation method of claim 1, further comprising:
determining a real-time status of the series of operations; and
displaying the real-time status on the overlay page.
3. The simulated steering method of claim 2, wherein the real-time conditions include at least one of:
target application information of current operation, current real-time operation progress, multi-round interactive intermediate input flow state and final operation result.
4. The simulated manipulation method of claim 1, wherein the step of automatically performing a series of operations to implement the instructions further comprises:
acquiring an input area in the target application;
analyzing the received voice input and extracting input information contained therein; and
and inputting the extracted input information into the corresponding input area, and executing subsequent operation.
5. The simulated manipulation method of claim 4, wherein the step of automatically performing a series of operations to implement the instructions further comprises:
and when the voice input does not contain the input information input to the corresponding input area, outputting the voice information to prompt a user that the current process needs to be subjected to intermediate input.
6. A portable terminal, comprising:
voice input means for receiving a voice input from a user;
the recognition device is used for recognizing the voice input and obtaining a corresponding simulation operation instruction;
executing means for starting a target application and automatically executing a series of operations to implement the instruction based on the instruction; and
the overlay page generating device is used for generating an overlay page to cover at least part of the display interface of the target application, wherein in the process of simulation operation and control, the overlay page is always stably displayed above other jump pages, and when touch input of a user is required in the process of simulation operation and control, holes are dug in the overlay page to expose areas which can be input in the application interface covered under the overlay page;
the overlay page generation device is further configured to display an execution result of the simulation operation on the overlay page after a series of operations of the execution device are completed, and close the overlay page after a delay of a predetermined time period elapses.
7. The portable terminal according to claim 6, further comprising:
monitoring means for determining a real-time status of the series of operations; and
and the control device is used for displaying the real-time state on the overlay page.
8. The portable terminal of claim 7, wherein the real-time status includes at least one of:
target application information of current operation, current real-time operation progress, multi-round interactive intermediate input flow state and final operation result.
9. The portable terminal according to claim 6, wherein the executing means further comprises:
an acquisition unit, configured to acquire an input area in the target application;
an analyzing unit for analyzing the received voice input and extracting input information contained therein; and
and the execution unit is used for inputting the extracted input information into the corresponding input area and executing subsequent operation.
10. The portable terminal according to claim 9, wherein the executing means further comprises:
and the prompting unit is used for outputting the voice information when the voice input does not contain the input information input to the corresponding input area so as to prompt a user that the current process needs to be subjected to intermediate input.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810596582.2A CN108845788B (en) | 2018-06-11 | 2018-06-11 | Analog manipulation method and portable terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810596582.2A CN108845788B (en) | 2018-06-11 | 2018-06-11 | Analog manipulation method and portable terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108845788A CN108845788A (en) | 2018-11-20 |
| CN108845788B true CN108845788B (en) | 2021-09-14 |
Family
ID=64211561
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810596582.2A Active CN108845788B (en) | 2018-06-11 | 2018-06-11 | Analog manipulation method and portable terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108845788B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110837334B (en) * | 2019-11-04 | 2022-03-22 | 北京字节跳动网络技术有限公司 | Method, device, terminal and storage medium for interactive control |
| CN113641280A (en) * | 2021-07-14 | 2021-11-12 | 阿里巴巴（中国）有限公司 | Equipment control method, control method and equipment in barrier-free mode |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106157955A (en) * | 2015-03-30 | 2016-11-23 | 阿里巴巴集团控股有限公司 | A kind of sound control method and device |
| CN106777184A (en) * | 2016-12-22 | 2017-05-31 | 北京猎豹移动科技有限公司 | A kind of page loading method, device and electronic equipment |
| CN107102796A (en) * | 2017-04-20 | 2017-08-29 | 三星电子（中国）研发中心 | A kind of message treatment method and device |
| CN108038150A (en) * | 2017-11-30 | 2018-05-15 | 广州市动景计算机科技有限公司 | Webpage deployment method, device and terminal device |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9965133B1 (en) * | 2011-07-22 | 2018-05-08 | Ntrepid Corporation | Application for assisting in conducting covert cyber operations |
| CN105447041A (en) * | 2014-09-02 | 2016-03-30 | 阿里巴巴集团控股有限公司 | Webpage processing method and device |
| CN105786395B (en) * | 2016-04-07 | 2019-04-09 | 广州华多网络科技有限公司 | A kind of live streaming method for switching between, apparatus and system based on mobile terminal |
| US20170316363A1 (en) * | 2016-04-28 | 2017-11-02 | Microsoft Technology Licensing, Llc | Tailored recommendations for a workflow development system |
| CN106372251B (en) * | 2016-09-28 | 2020-03-03 | 北京京东尚科信息技术有限公司 | Method and device for returning page display position |
| CN107657159A (en) * | 2017-08-31 | 2018-02-02 | 珠海市魅族科技有限公司 | A kind of method for secret protection and device, terminal and readable storage medium storing program for executing |
| CN107704233B (en) * | 2017-09-29 | 2021-06-15 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN107861857B (en) * | 2017-11-09 | 2021-04-13 | 上海触乐信息科技有限公司 | Screen virtual shade control method and device in terminal equipment and terminal equipment |
| CN108022586B (en) * | 2017-11-30 | 2019-10-18 | 百度在线网络技术（北京）有限公司 | Method and apparatus for controlling the page |
-
2018
- 2018-06-11 CN CN201810596582.2A patent/CN108845788B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106157955A (en) * | 2015-03-30 | 2016-11-23 | 阿里巴巴集团控股有限公司 | A kind of sound control method and device |
| CN106777184A (en) * | 2016-12-22 | 2017-05-31 | 北京猎豹移动科技有限公司 | A kind of page loading method, device and electronic equipment |
| CN107102796A (en) * | 2017-04-20 | 2017-08-29 | 三星电子（中国）研发中心 | A kind of message treatment method and device |
| CN108038150A (en) * | 2017-11-30 | 2018-05-15 | 广州市动景计算机科技有限公司 | Webpage deployment method, device and terminal device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108845788A (en) | 2018-11-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109300347B (en) | Dictation auxiliary method based on image recognition and family education equipment | |
| US8560326B2 (en) | Voice prompts for use in speech-to-speech translation system | |
| JP7203865B2 (en) | Multimodal interaction between users, automated assistants, and other computing services | |
| CN106648535A (en) | Live client voice input method and terminal device | |
| JP2013068952A (en) | Consolidating speech recognition results | |
| CN105895093A (en) | Voice information processing method and device | |
| CN111627436B (en) | Voice control method and device | |
| US12107995B2 (en) | Objective training and evaluation | |
| JP2005321817A (en) | Method and apparatus for obtaining combining information from speech signals for adaptive interaction in teaching and testing | |
| US20150039307A1 (en) | Interfacing device and method for supporting speech dialogue service | |
| CN111901665B (en) | Teaching resource playing method and device and storage medium | |
| CN108845788B (en) | Analog manipulation method and portable terminal | |
| CN112151034B (en) | Voice control method and device of equipment, electronic equipment and storage medium | |
| CN111343028A (en) | Distribution network control method and device | |
| KR102527107B1 (en) | Method for executing function based on voice and electronic device for supporting the same | |
| US8019591B2 (en) | Rapid automatic user training with simulated bilingual user actions and responses in speech-to-speech translation | |
| CN111841007A (en) | Game control method, device, equipment and storage medium | |
| JP2007187939A (en) | English conversation learning apparatus | |
| JP6166831B1 (en) | Word learning support device, word learning support program, and word learning support method | |
| KR100898104B1 (en) | Learning system and method by interactive conversation | |
| CN110603536A (en) | Teaching and assessment of spoken skills through fine-grained evaluation of human speech | |
| CN111090383B (en) | Instruction identification method and electronic equipment | |
| CN111091821B (en) | Control method based on voice recognition and terminal equipment | |
| US20220319516A1 (en) | Conversation method, conversation system, conversation apparatus, and program | |
| CN112380871A (en) | Semantic recognition method, apparatus, and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |