CN110795175A - Method and device for analog control of intelligent terminal and intelligent terminal - Google Patents

Method and device for analog control of intelligent terminal and intelligent terminal Download PDF

Info

Publication number
CN110795175A
CN110795175A CN201810872236.2A CN201810872236A CN110795175A CN 110795175 A CN110795175 A CN 110795175A CN 201810872236 A CN201810872236 A CN 201810872236A CN 110795175 A CN110795175 A CN 110795175A
Authority
CN
China
Prior art keywords
node
text information
current control
control node
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201810872236.2A
Other languages
Chinese (zh)
Inventor
李兰
赵志宝
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
TCL Corp
TCL Research America Inc
Original Assignee
TCL Research America Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by TCL Research America Inc filed Critical TCL Research America Inc
Priority to CN201810872236.2A priority Critical patent/CN110795175A/en
Publication of CN110795175A publication Critical patent/CN110795175A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/0483—Interaction with page-structured environments, e.g. book metaphor
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/26—Speech to text systems
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G10L25/54—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for retrieval
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/223—Execution procedure of a spoken command

Abstract

The embodiment of the invention discloses a method and a device for analog control of an intelligent terminal and the intelligent terminal, wherein the method comprises the following steps: acquiring character information; sending a triggering instruction according to the text information so as to trigger an interface retrieval process of the intelligent terminal; traversing each control node of the current interface through an interface retrieval process to acquire node character information of the current control node; matching the text information with the node text information of the current control node according to a preset matching rule, and judging whether the matching is successful; and when the character information is successfully matched with the node character information of the current control node, acquiring the node attribute information of the current control node, and performing simulation control operation on the current control node according to the node attribute information of the current control node. The embodiment of the invention can improve the convenience of intelligent terminal control, is convenient for disabled people to use and improves the user experience.

Description

Method and device for analog control of intelligent terminal and intelligent terminal
Technical Field
The invention belongs to the technical field of terminals, and particularly relates to a method and a device for analog control of an intelligent terminal, the intelligent terminal and a computer readable storage medium.
Background
With the continuous development and progress of terminal technology, the application of the intelligent terminal is more and more extensive.
At present, the control mode of an intelligent terminal is generally touch and click, for example, a mobile phone terminal. For some special intelligent terminals, for example, intelligent televisions, it is generally operated and clicked through a remote controller. The existing intelligent terminal control methods all need manual control of a user, but under the condition that both hands of some users are inconvenient or for disabled people, the existing intelligent terminal control methods are low in convenience and low in user experience.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method and an apparatus for controlling an intelligent terminal in an analog manner, an intelligent terminal, and a computer-readable storage medium, so as to control the intelligent terminal in an analog manner, improve convenience of controlling the intelligent terminal, and improve user experience.
A first aspect of an embodiment of the present invention provides a method for analog control of an intelligent terminal, including:
acquiring character information;
sending a triggering instruction according to the text information so as to trigger an interface retrieval process of the intelligent terminal;
traversing each control node of the current interface through the interface retrieval process to acquire node character information of the current control node;
matching the text information with the node text information of the current control node according to a preset matching rule, and judging whether the matching is successful;
and when the character information is successfully matched with the node character information of the current control node, acquiring the node attribute information of the current control node, and performing simulation control operation on the current control node according to the node attribute information of the current control node.
Optionally, the matching the text information with the node text information of the current control node according to a preset matching rule, and determining whether the matching is successful includes:
comparing the text information with the node text information of the current control node, and judging whether the text information is consistent with the node text information of the current control node;
when the text information is consistent with the node text information of the current control node, matching is successful;
when the text information is inconsistent with the node text information of the current control node, judging whether the same proportion between the text information and the node text information is larger than or equal to a preset threshold value or not;
when the same proportion is larger than or equal to the preset threshold value, the matching is successful;
when the same proportion is smaller than the preset threshold value, the matching is failed;
or
Comparing the text information with the node text information of the current control node, and judging whether the text information is consistent with the node text information of the current control node;
when the text information is consistent with the node text information of the current control node, matching is successful;
when the text information is inconsistent with the node text information of the current control node, the matching is failed;
or
Judging whether the same proportion between the text information and the node text information is greater than or equal to a preset threshold value or not;
when the same proportion is larger than or equal to the preset threshold value, the matching is successful;
and when the same proportion is smaller than the preset threshold value, the matching is failed.
Optionally, performing simulation control operation on the current control node according to the node attribute information of the current control node, including:
judging whether the current control node is a clickable node or not according to the node type information of the current control node;
when the current control node is a clickable node, sending a click event to the current control node so as to perform simulated click operation on the current control node;
when the current control node is not a clickable node, sending a touch event to the current control node according to the interface position information of the current control node so as to perform simulated touch operation on the current control node;
the node attribute information comprises the node type information and the interface position information, and the node type comprises clickable nodes and non-clickable nodes.
Optionally, when the current control node is a clickable node, after sending a click event to the current control node to perform a simulated click operation on the current control node, the method further includes:
judging whether the click is successful;
when the click is successful, ending the process;
and when the click fails, sending the touch event to the current control node according to the interface position information of the current control node so as to perform simulated touch operation on the current control node.
Optionally, after the acquiring the text information, the method further includes:
judging whether the control operation corresponding to the text information belongs to a preset simple operation category or not according to the text information; wherein the preset simple operation categories include up, down, left, right, previous page, next page, home page, return, and back;
if the control operation corresponding to the text information belongs to the preset simple operation category, performing corresponding control operation according to the file information;
and if the click operation corresponding to the text information does not belong to the preset simple operation category, the step of sending a trigger instruction according to the text information so as to trigger an interface retrieval process of the intelligent terminal is carried out.
Optionally, according to the text information, a triggering instruction is issued to trigger an interface retrieval process of the intelligent terminal, including:
and when the text information is received, sending an application notification message to trigger an inaccessible event callback process of the intelligent terminal through the application notification message.
Optionally, the acquiring text information includes:
receiving voice input by a user;
processing the voice to obtain the text information;
or
Receiving a broadcast message including text parameters;
and acquiring the text information according to the broadcast message.
A second aspect of the embodiments of the present invention provides an apparatus for analog control of an intelligent terminal, including:
the acquisition module is used for acquiring the text information;
the triggering module is used for sending a triggering instruction according to the text information so as to trigger an interface retrieval process of the intelligent terminal;
the node character information acquisition module is used for traversing each control node of the current interface through the interface retrieval process to acquire the node character information of the current control node;
the matching module is used for matching the text information with the node text information of the current control node according to a preset matching rule and judging whether the matching is successful or not;
and the simulation control module is used for acquiring the node attribute information of the current control node when the character information is successfully matched with the node character information of the current control node, and performing simulation control operation on the current control node according to the node attribute information of the current control node.
A third aspect of embodiments of the present invention provides an intelligent terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the method according to any one of the first aspect when executing the computer program.
A fourth aspect of embodiments of the present invention provides a computer-readable storage medium, in which a computer program is stored, which, when executed by a processor, performs the steps of the method according to any one of the above-mentioned first aspects.
Compared with the prior art, the embodiment of the invention has the following beneficial effects:
according to the embodiment of the invention, the acquired text information is matched with the text information of the control node on the intelligent terminal interface, and after the matching is successful, the control node is subjected to simulation control operation according to the node attribute information, so that the intelligent terminal is controlled through the text information without manually operating the intelligent terminal by a user, the control convenience of the intelligent terminal is improved, the use of disabled persons is facilitated, and the user experience is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1 is a schematic flowchart of a method for analog control of an intelligent terminal according to an embodiment of the present invention;
fig. 2 is a schematic flowchart of step S104 according to an embodiment of the present invention;
fig. 3 is a schematic flowchart of step S105 according to an embodiment of the present invention;
fig. 4 is a schematic flow chart of another implementation of a method for analog control of an intelligent terminal according to an embodiment of the present invention;
fig. 5 is a schematic block diagram of a structure of an apparatus for analog control of an intelligent terminal according to an embodiment of the present invention;
fig. 6 is a schematic diagram of an intelligent terminal according to an embodiment of the present invention.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
In order to explain the technical means of the present invention, the following description will be given by way of specific examples.
Example one
Referring to fig. 1, a schematic flow chart of a method for analog control of an intelligent terminal according to an embodiment of the present invention is shown, where the method includes the following steps:
and step S101, acquiring character information.
It should be noted that the text information may be input externally, may be actively read by the intelligent terminal, or may be extracted from the received broadcast command. When the text information is input externally, it may be specifically text information obtained according to a voice input by a user. That is, a user inputs voice, and the intelligent terminal converts the voice of the user into Text through a Speech To Text (STT) technology To obtain Text information. That is, the user can control the intelligent terminal through voice.
The text information can be actively read from a file stored in advance by the intelligent terminal, namely, a text edited in advance is stored in the file, and the intelligent terminal can actively read the file so as to obtain corresponding text information. The text information can also be extracted from the received broadcast command containing the text parameters by the intelligent terminal.
Because the text information can be acquired through various ways, the technical scheme of the embodiment of the invention can have a plurality of application scenes. For example, the embodiment of the invention can be specifically applied to a scene of voice control of the intelligent terminal, that is, a user can simulate the operation of the intelligent terminal through voice, so that the convenience of the user is improved, the use of disabled people is facilitated, and the user experience is improved; the method can also be specifically applied to the scenes of intelligent terminal testing, specifically pressure testing, smoking testing and the like, the input text information and commands are edited in advance through the script file, the text information can be obtained by reading the file, and the operation of a user and a tester on the intelligent terminal is simulated, so that the testing resources can be saved, and the convenience and the efficiency of the testing are improved.
Of course, the obtaining approach of the text information is not limited to the above mentioned manner, and the application scenario of the technical solution of the embodiment of the present invention is not limited to the above mentioned voice-controlled intelligent terminal and the intelligent terminal test, which is not limited herein.
In some embodiments of the present invention, the step, namely the process of acquiring the text information, may specifically be: receiving voice input by a user; and processing the voice to obtain character information. When receiving the voice of the user, the intelligent terminal can process the voice through an STT technology and convert the voice of the user into corresponding text information. In other embodiments of the present invention, the step, namely the process of acquiring the text information, may also specifically be: receiving a broadcast message including text parameters; and acquiring the text information according to the broadcast message. I.e. by sending a broadcast message, the corresponding text information is entered. Of course, the corresponding text information can also be obtained by reading the file including the text information. The obtaining mode of the text information can be selected according to actual requirements and actual application scenes.
And S102, sending a trigger instruction according to the text information to trigger an interface retrieval process of the intelligent terminal.
It should be noted that the intelligent terminal may specifically be a mobile phone, a smart television, a tablet, an intelligent wearable device, or other terminal devices. The intelligent terminal is based on an Android system, generally, an accessible service (accessibiservie) without barrier function exists in the Android system, correspondingly, an inaccessible event (onassibivent) callback method exists, and the onassibivent callback method can be used for retrieving a current interface of the intelligent terminal and searching each node on the interface and the attribute of each node. The interface retrieval process is specifically an onaccessibirityevent callback method.
The interface may refer to a user interface of the intelligent terminal, or may refer to an interface of an application in the intelligent terminal. That is, analog control can be performed on a certain application interface of the intelligent terminal, and also analog control can be performed on a user interface of the intelligent terminal, so that analog clicking can be realized on all applications installed on the intelligent terminal.
Since the onaccessibiteyEvent callback method is called by the system, and is automatically called only when the current interface changes or is at intervals, namely, the system calls the onaccessibiteyEvent callback method untimely. In order to ensure that the onaccestibiteevent callback method can be called in time for interface retrieval after the text information is input, the onaccessibiteevent callback method can be automatically triggered by sending a trigger instruction after the text information is acquired.
The trigger instruction may specifically be an application notification message, that is, a system notification message is used to trigger an onaccestibiteeveevent callback process. In the onaccestibitevent callback flow, it can filter the event type by the parameter event, so that only the message notification that the event is of the type "android. Of course, the triggering instruction may also be embodied as other instructions, and is not limited herein.
In some embodiments of the present invention, the specific process of the step of sending the trigger instruction according to the text information to trigger the interface retrieval process of the intelligent terminal may be, for example: and when the text information is received, sending an application notification message to trigger an inaccessible event callback process of the intelligent terminal through the application notification message. That is, when receiving the text message, the application notification message is sent to actively trigger the onaccessibirityevent callback process so as to retrieve the control node of the current interface.
Step S103, traversing each control node of the current interface through the interface retrieval process, and acquiring the node character information of the current control node.
Specifically, after the onaccestibiteyevent callback process is triggered, the onaccestibiteyevent callback process sequentially traverses each control node on the current interface to obtain the node text information of the currently traversed node.
And step S104, matching the text information with the node text information of the current control node according to a preset matching rule, and judging whether the matching is successful.
It should be noted that the preset matching rule may include one or more of an exact matching rule and a fuzzy matching rule, and the exact matching rule may be expressed as that when the node text information and the text information are completely the same, the matching is successful, otherwise, the matching is failed; the fuzzy matching rule can show that when the same proportion of the node text information and the text information reaches a certain threshold value, the matching is successful, otherwise, the matching is failed. The same ratio of the node character information to the character information refers to a ratio of the same number of the input characters to the node characters to the total number of the node characters, for example, the character information is "movie", the node character information is "hot movie", since the two characters of the character information and the node character information are the same, and the total number of the node characters is 4, the same ratio is 50%.
The preset matching rules can simultaneously comprise an accurate matching rule and a fuzzy matching rule, and at the moment, the accurate matching can be firstly carried out, and then the fuzzy matching is carried out after the accurate matching fails; it is also possible to include only one of the exact match and fuzzy match rules, i.e. to only make exact matches or fuzzy matches.
In specific application, the preset matching rule can be selected according to actual requirements. Generally, the matching accuracy can be improved by performing the precise matching first and then performing the fuzzy matching after the matching fails, and the user experience can also be improved to a certain extent.
And S105, when the character information is successfully matched with the node character information of the current control node, acquiring the node attribute information of the current control node, and performing simulation control operation on the current control node according to the node attribute information of the current control node.
It should be noted that the node attribute information may include, but is not limited to, interface location information of a node, node type information, and the like. And the interface position information may be specifically coordinate information of the control node on the interface. The node types of the control can be divided into clickable nodes and non-clickable nodes.
Specifically, after the text information of the node is successfully matched with the text information, the onaccesssitivyevent call-back process is used for obtaining node attribute information such as node type information, interface position information and the like of the current control node, and then, a click event of the control node is triggered, so that the click operation of a person is simulated. And traversing the next control node after the matching fails, acquiring the text information of the node, matching according to the text information of the node, and repeating the steps until the matching is successful or the control node of the current interface is traversed.
It should be noted that the above-mentioned analog control operation may include, but is not limited to, simulating click, simulating touch, simulating double click, simulating scrolling, trackball, moving focus up, down, left, right, and the like. Generally, most of the control operations of the intelligent terminal can be realized through click operations and touch operations, and therefore, the simulation control operations generally include simulation of click operations and simulation of touch operations. Of course, other analog control operations may be included.
It can be understood that the above-mentioned simulated click operation refers to triggering a node click event through text information to simulate an operation of a human clicking the control node through a finger, a touch pen, or the like, and actually the human does not click the control node, but the intelligent terminal simulates the human click operation. Similarly, the simulation of the touch operation refers to the operation of touching the control node by a human through a finger, a touch pen, and the like by sending a touch event, and the human does not actually touch the control node. The same is true for other analog control operations such as simulating double-click, simulating scrolling, etc.
In a specific application, the above-described simulation control operation may be expressed in different implementations according to different node types. Specifically, the simulated click operation can be realized through a node click function, namely, a node performance section, provided by the accessitivyservice in the Android system, and the simulated touch operation can also be performed on the control node according to the coordinate position of the control node through clicking built to a driving layer, namely, a method for receiving an input event of the Android system is called.
In the embodiment, the acquired text information is matched with the text information of the control node on the interface of the intelligent terminal, after the matching is successful, the control node is subjected to simulation control operation according to the node attribute information, the intelligent terminal is controlled through the text information, the intelligent terminal is not required to be manually operated by a user, the convenience of intelligent terminal control is improved, the use of disabled people is facilitated, and the user experience is improved.
Example two
The preset matching rule may include one or more of an exact matching rule and a fuzzy matching rule, and in a specific application, the preset matching rule may be determined according to an actual application scenario and an actual requirement. The following describes a specific process of three different preset matching rules.
Based on the first embodiment, in some embodiments of the present invention, referring to a flowchart of step S104 shown in fig. 2, in step S104, that is, the specific process of matching the text information with the node text information of the current control node according to the preset matching rule and determining whether the matching is successful may be:
step S201, comparing the text information with the node text information of the current control node, and judging whether the text information is consistent with the node text information of the current control node. When consistent, the match is successful. If not, the process proceeds to step S202.
Step S202, whether the same ratio between the text information and the node text information is larger than or equal to a preset threshold value or not is judged. And when the same proportion is larger than or equal to a preset threshold value, the matching is successful. And when the same proportion is smaller than a preset threshold value, the matching fails.
Specifically, the precise matching is performed first, and after the precise matching fails, the fuzzy matching is performed. Namely, judging whether the text information is completely the same as the node text information of the current control node, if so, successfully matching, otherwise, entering fuzzy matching; in fuzzy matching, whether the same proportion between the node character information and the character information reaches a preset threshold value or not is judged, if the same proportion is larger than the preset threshold value, matching is successful, otherwise, matching is failed.
It should be noted that the preset threshold may be set according to actual requirements, and in general, the number of the characters of the interface control node on the intelligent terminal is small, and is usually 2, 3, or 4, so that the preset threshold may be set to be 50% or 75% of the number of the characters in the node character information.
For example, the intelligent terminal is an intelligent television, the text information input by the user to the intelligent television by voice is 'movie and television', the text information of a node of an APP in the intelligent television is 'hot movie and television', and the preset threshold is 50%. At this moment, the smart television firstly judges whether the 'movie' and the 'hot movie' are completely the same, so that the 'movie' and the 'hot movie' are not completely the same, and further judges the same proportion between the 'movie' and the 'hot movie', because two characters of the 'movie' of the character information and the node character information are the same, the total number of the node characters is 4, at this moment, the same proportion is 50%, the same proportion reaches a preset threshold value, and at this moment, the matching is successful.
Based on the first embodiment, in another embodiment of the present invention, in the step S104, that is, the step of matching the text information with the node text information of the current control node according to the preset matching rule, and a specific process of determining whether matching is successful may be: comparing the text information with the node text information of the current control node, and judging whether the text information is consistent with the node text information of the current control node; when the matching is consistent, the matching is successful; when there is an inconsistency, the match fails. Namely, whether the text information is completely the same as the node text information of the current control node is judged, if so, the matching is successful, otherwise, the matching is failed.
For example, the intelligent terminal is an intelligent television, the text information input by the user to the intelligent television by voice is 'movie and television', and the text information of a node of an APP in the intelligent television is 'hot movie and television'. And the intelligent television judges whether the 'movie' and the 'hot movie' are completely the same or not, and the 'movie' and the 'hot movie' are not completely the same, so that the matching fails.
Based on the first embodiment, in further embodiments of the present invention, in the step S104, that is, the step of matching the text information with the node text information of the current control node according to the preset matching rule, and a specific process of determining whether the matching is successful may be: judging whether the same ratio between the text information and the node text information is greater than or equal to a preset threshold value or not; when the same proportion is larger than or equal to a preset threshold value, matching is successful; and when the same proportion is smaller than a preset threshold value, the matching fails. Namely, the same proportion between the text information and the node text information is judged, if the same proportion reaches a preset threshold value, the matching is successful, otherwise, the matching is failed.
For example, the intelligent terminal is an intelligent television, the text information input by the user to the intelligent television by voice is 'movie and television', the text information of a node of an APP in the intelligent television is 'hot movie and television', and the preset threshold is 50%. Judging the same proportion between the 'movie & TV' and the 'popular movie & TV', wherein the two characters of the 'movie & TV' of the character information and the node character information are the same, the total number of the node characters is 4, at the moment, the same proportion is 50%, and the matching is successful if the same proportion reaches a preset threshold value.
In practical application, only the fuzzy matching rule is utilized, so that the application control displayed on the voice operation television of most users can be met, and therefore, in the application scene of the voice operation smart television, only the fuzzy matching process can be carried out. In other application scenarios, the selection may be performed according to actual requirements, and is not limited herein.
EXAMPLE III
After the matching is successful, the simulation control operation is carried out on the current control node. The specific modes of the simulation control can be two, and one of the click modes is selected for clicking according to the node attributes of the control nodes. The procedure of the analog control will be described below.
Based on any of the above embodiments, in some embodiments of the present invention, referring to a flowchart of step S105 shown in fig. 3, in step S105, in time, when the matching is successful, the node attribute information of the current control node is obtained, and according to the node attribute information of the current control node, a specific process of performing the simulation control operation on the current control node may be, for example:
and S301, when the matching is successful, obtaining the node attribute information of the current control node through the onAccessabilityEvent callback process. The node attribute information may include, but is not limited to, node type information and interface location information, and the node type includes clickable nodes and non-clickable nodes.
It should be noted that the node type information refers to information that characterizes whether the node type is a clickable node or an clickable node. The interface position information refers to coordinate information of the control node on the interface of the intelligent terminal.
Step S302, judging whether the current control node is a clickable node or not according to the node type information of the current control node.
And step S303, when the current control node is the clickable node, sending a click event to the current control node so as to perform simulated click operation on the current control node.
And S304, when the current control node is not the clickable node, sending a touch event to the current control node according to the interface position information of the current control node so as to perform simulated touch operation on the current control node.
It can be understood that the clickable node may simulate clicking through a node clicking function, i.e., node. The click event registered by the clickable node is a click event. Therefore, when the widget is a clickable node, a click event can be sent through the node click function provided by the accessitivyservice, and the widget node click is triggered. And the non-clickable node needs to be built in a drive layer to click, and specifically, an input event receiving method hidden in an Android system can be called.
After the simulation control operation is successful, the input text information can be cleared in order to ensure the correctness of the processing flow. And when the current control node is a clickable node, whether the click is successful can be judged, and when the click is failed, a touch event is sent to the current control node, so that the touch operation simulation of the control node can be realized.
Therefore, in the step S303, that is, after the sending a click event to the current control node to perform the simulated click operation on the current control node when the current control node is the clickable node, the method may further include:
step S3031, judging whether the click is successful; when the click is successful, ending the process; and when the click fails, sending a touch event to the current control node according to the interface position information of the current control node so as to perform simulated click operation on the current control node. Namely, if the click fails, the interface position information of the control node is acquired, and then the touch event is sent to the node, so that the simulation control operation of the control node is realized.
It can be seen that by generating a click event or a touch event, the simulated click operation of most control nodes can be guaranteed. Although the simulated click and the simulated touch may cover the control operations of most applications, these two control methods may also fail, and at this time, other control operations may be added to the known interface, such as double-click, scroll, track ball, etc. Referring to fig. 4, another implementation flow diagram of a method for analog controlling an intelligent terminal according to an embodiment of the present invention is shown, where the method includes the following steps:
and step S401, acquiring character information.
In specific application, the method for analog control of the intelligent terminal can be realized through an Android application. Therefore, before obtaining the text information, a service, called a SeeService, may be created first, where the SeeService inherits the accessitivyservice provided by the Android system, and after the inheritance, the onaccessitivevevent callback method may be prompted to be added. The SeeServer can realize the function of searching the text information of the nodes on the interface. And as the accessibility service needs to be used, the barrier-free function of the Android system is needed, and all barrier-free permissions of SeeService need to be opened.
It can be understood that the method for controlling the intelligent terminal in a simulation mode can be achieved through one Android application, the interior of the application can be controlled through simulated clicking, and all applications of the intelligent terminal can also be controlled. When controlling a certain application, the control node may apply some buttons, labels, scroll bars, etc. on the interface, and when controlling all applications, the control node may be each APP installed on the interface of the intelligent terminal.
And step S402, performing character preprocessing operation on the character information.
It should be noted that the word preprocessing operations may include, but are not limited to, converting words into lower case, removing spaces, removing punctuation marks, converting numbers all into upper case, and the like. The characters are preprocessed, so that the success rate of subsequent fuzzy matching is higher. However, the object of the embodiment of the present invention can be achieved without performing a preprocessing operation on the text information.
Step S403, judging whether the control operation corresponding to the text information belongs to a preset simple operation category or not according to the text information; wherein the preset simple operation categories include up, down, left, right, previous page, next page, home page, return, and back. If yes, simple operation is carried out according to the file information; if not, step S404.
It can be understood that the text information may correspond to some simple operations, and the simple operations may be directly operated without interface control.
The preset simple operation types can include two types, one type can include "to home", "return" and "back", which can be operated through the access availability service; another category may include "up," "down," "left," "right," "previous page," and "next page" operations. Since the key values of all keys of the Android system are fixed, the operations can be established to a driving layer to send key event triggers without interface operation. When the input text information corresponds to up, down, left and right movement operation, a focus movement key value can be sent to simulate up, down, left and right key operation of a remote controller; of course, in some cases, it is also possible to simulate a sending fixed key such as a home key, a return key, a confirm key, etc. to simulate a corresponding control operation. It is to be understood that the preset simple operation category described above is not limited to only the above-mentioned operation.
That is, according to whether the character information is matched with the operations, if the character information can be matched with the operations, the operations are directly carried out without interface control. If the match is not found, the next step is entered.
It should be noted that the purpose of the embodiment of the present invention can also be achieved by directly controlling through an interface without determining whether the operation belongs to the simple operation category.
And S404, sending a trigger instruction according to the text information to trigger an interface retrieval process of the intelligent terminal.
And S405, traversing each control node of the current interface through an interface retrieval process.
It is understood that before searching, in order to avoid the situation that the system or other applications also send notification messages and the text information is not empty, which causes processing flow errors, it can be determined whether the text information is null. After the characters are input once, the character information can be directly emptied, and the input character object is set to null, so that the error processing flow caused by non-self active notification is avoided.
When traversing the interface, the method can be performed by an onaccessibityEvent callback method, and all nodes and child nodes corresponding to the nodes of the interface are traversed, which can be specifically realized by a circular traversal method.
And S406, judging whether the current control node has characters, if so, entering the step S407, and if not, traversing the next node.
It can be understood that the node text information can also be directly obtained without judging whether the control node has the text or not.
And step S407, acquiring the node character information of the current control node, and performing character preprocessing operation on the acquired node character information.
It should be noted that the word preprocessing operations may include converting words into lower case words, removing spaces, removing punctuation marks, converting numbers all into upper case letters, etc. The characters are preprocessed, so that the success rate of subsequent fuzzy matching is higher. However, the object of the embodiment of the present invention can be achieved without performing a preprocessing operation on the text information.
And step S408, accurately matching the text information with the node text information of the current control node according to an accurate matching rule, judging whether the matching is successful, if the matching is failed, entering step S409, and if the matching is successful, entering step S410.
Step S409, judging whether the same proportion of the text information and the node text information is greater than or equal to a preset threshold value or not according to a fuzzy matching rule, if so, successfully matching, entering step S410, if not, failing to match, traversing the next control node, and returning to step S406, namely, obtaining the node text information of the current control node; and matching the text information with the node text information of the current control node according to a preset matching rule, and judging whether the matching is successful or not until all the control nodes are traversed or the matching is successful.
And step S410, judging whether the current control node is a clickable node, if so, entering step S411, and if not, entering step S413.
And S411, sending a click event to the current control node, and performing simulated clicking.
And step S412, judging whether the click is successful, if so, ending the process, and if not, entering step S413.
And step S413, acquiring the position information of the current control node, sending a touch event, and performing simulated touch.
It should be noted that the same or similar parts of this embodiment as those of other embodiments may be referred to other embodiments, and are not repeated herein.
In the embodiment, the acquired text information is matched with the text information of the control node on the interface of the intelligent terminal, after the matching is successful, the control node is subjected to simulation control operation according to the node attribute information, the intelligent terminal is controlled through the text information, the intelligent terminal is not required to be manually operated by a user, the convenience of intelligent terminal control is improved, the use of disabled people is facilitated, and the user experience is improved.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
EXAMPLE five
Referring to fig. 5, a schematic block diagram of a structure of an apparatus for analog-controlling an intelligent terminal according to an embodiment of the present invention is shown, where the apparatus may include:
an obtaining module 51, configured to obtain text information;
the triggering module 52 is configured to send a triggering instruction according to the text information to trigger an interface retrieval process of the intelligent terminal;
a node text information obtaining module 53, configured to traverse each control node of the current interface through the interface retrieval process, and obtain node text information of the current control node;
the matching module 54 is configured to match the text information with the node text information of the current control node according to a preset matching rule, and determine whether matching is successful;
and the simulation control module 55 is configured to, when matching is successful, obtain node attribute information of the current control node, and perform simulation control operation on the current control node according to the node attribute information of the current control node.
In some embodiments of the present invention, the matching module may include:
the first judgment unit is used for comparing the text information with the node text information of the current control node and judging whether the text information is consistent with the node text information of the current control node; when the matching is consistent, the matching is successful;
the second judgment unit is used for judging whether the same proportion between the character information and the node character information is greater than or equal to a preset threshold value or not when the character information and the node character information are inconsistent; when the same proportion is larger than or equal to a preset threshold value, matching is successful; when the same proportion is smaller than a preset threshold value, the matching is failed;
or
The third judging unit is used for comparing the character information with the node character information of the current control node and judging whether the character information is consistent with the node character information of the current control node; when the matching is consistent, the matching is successful; if not, the matching is failed;
or
A fourth judging unit, configured to judge whether the same ratio between the text information and the node text information is greater than or equal to a preset threshold; when the same proportion is larger than or equal to a preset threshold value, matching is successful; and when the same proportion is smaller than a preset threshold value, the matching fails.
In some embodiments of the present invention, the analog control module may include:
the fifth judging unit is used for judging whether the current control node is a clickable node or not according to the node type information of the current control node;
the first click unit is used for sending a click event to the current control node when the current control node is a clickable node so as to carry out simulated click operation on the current control node;
the second clicking unit is used for sending a touch event to the current control node according to the interface position information of the current control node when the current control node is not a clickable node so as to simulate touch operation on the current control node;
the node attribute information comprises node type information and interface position information, and the node type comprises clickable nodes and non-clickable nodes.
In some embodiments of the present invention, the apparatus may further include:
the first judgment module is used for judging whether the click is successful; when the click is successful, ending the process; and when the click fails, sending a touch event to the current control node according to the interface position information of the current control node so as to perform simulated touch operation on the current control node.
In some embodiments of the present invention, the apparatus may further include:
the second judgment module is used for judging whether the control operation corresponding to the character information belongs to the preset simple operation category or not according to the character information; the preset simple operation categories comprise upward, downward, leftward, rightward, previous page, next page, homepage, return and backward; if so, performing corresponding control operation according to the file information; if not, the step of sending a triggering instruction according to the text information to trigger the interface retrieval process of the intelligent terminal is carried out.
In some embodiments of the present invention, the triggering module may include:
and the triggering unit is used for sending out an application notification message when the text message is received so as to trigger the callback process of the inaccessible event of the intelligent terminal through the application notification message.
In some embodiments of the present invention, the obtaining module may include:
a first receiving unit for receiving a voice input by a user;
the voice processing unit is used for processing the voice to obtain character information;
or
A second receiving unit for receiving a broadcast message including a text parameter;
and the broadcast message processing unit is used for acquiring the text information according to the broadcast message.
In the embodiment, the acquired text information is matched with the text information of the control node on the interface of the intelligent terminal, after the matching is successful, the control node is subjected to simulation control operation according to the node attribute information, the intelligent terminal is controlled through the text information, the intelligent terminal is not controlled manually by a user, the convenience of intelligent terminal control is improved, the use of disabled people is facilitated, and the user experience is improved.
EXAMPLE six
Fig. 6 is a schematic diagram of an intelligent terminal according to an embodiment of the present invention. As shown in fig. 6, the intelligent terminal 6 of this embodiment includes: a processor 60, a memory 61 and a computer program 62,. The processor 60, when executing the computer program 62, implements the steps in the various method embodiments of the analog control intelligent terminal method described above, such as the steps S101 to S105 shown in fig. 1. Alternatively, the processor 60, when executing the computer program 62, implements the functions of the modules/units in the above-mentioned device embodiments, such as the functions of the modules 51 to 55 shown in fig. 5.
Illustratively, the computer program 62 may be partitioned into one or more modules, units, which are stored in the memory 61 and executed by the processor 60 to implement the present invention. The one or more modules or units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program 62 in the intelligent terminal 6. For example, the computer program 62 may be divided into an acquisition module, a trigger module, a node text information acquisition module, and a click module, and each module has the following specific functions:
the acquisition module is used for acquiring the text information;
the triggering module is used for sending a triggering instruction according to the text information so as to trigger an interface retrieval process of the intelligent terminal;
the node character information acquisition module is used for traversing each control node of the current interface through the interface retrieval process to acquire the node character information of the current control node;
the matching module is used for matching the text information with the node text information of the current control node according to a preset matching rule and judging whether the matching is successful or not;
and the simulation control module is used for acquiring the node attribute information of the current control node when the matching is successful, and performing simulation control operation on the current control node according to the node attribute information of the current control node.
The intelligent terminal 6 can be a desktop computer, a notebook, a palm computer, an intelligent television, a mobile phone, an intelligent wearable device and the like. The intelligent terminal may include, but is not limited to, a processor 60, a memory 61. Those skilled in the art will appreciate that fig. 6 is merely an example of a smart terminal 6 and does not constitute a limitation of the smart terminal 6 and may include more or less components than those shown, or some components in combination, or different components, for example, the smart terminal may also include input-output devices, network access devices, buses, etc.
The Processor 60 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 61 may be an internal storage unit of the intelligent terminal 6, such as a hard disk or a memory of the intelligent terminal 6. The memory 61 may also be an external storage device of the intelligent terminal 6, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are equipped on the intelligent terminal 6. Further, the memory 61 may also include both an internal storage unit and an external storage device of the smart terminal 6. The memory 61 is used for storing the computer program and other programs and data required by the intelligent terminal. The memory 61 may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus, intelligent terminal, and method may be implemented in other ways. For example, the above-described embodiments of the apparatus or the smart terminal are merely illustrative, and for example, the division of the modules or units is only one logical function division, and there may be other division manners in actual implementation, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated modules or units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.

Claims (10)

1. A method for analog control of an intelligent terminal is characterized by comprising the following steps:
acquiring character information;
sending a triggering instruction according to the text information so as to trigger an interface retrieval process of the intelligent terminal;
traversing each control node of the current interface through the interface retrieval process to acquire node character information of the current control node;
matching the text information with the node text information of the current control node according to a preset matching rule, and judging whether the matching is successful;
and when the character information is successfully matched with the node character information of the current control node, acquiring the node attribute information of the current control node, and performing simulation control operation on the current control node according to the node attribute information of the current control node.
2. The method of claim 1, wherein the matching the text information with the node text information of the current control node according to a preset matching rule, and determining whether the matching is successful comprises:
comparing the text information with the node text information of the current control node, and judging whether the text information is consistent with the node text information of the current control node;
when the text information is consistent with the node text information of the current control node, matching is successful;
when the text information is inconsistent with the node text information of the current control node, judging whether the same proportion between the text information and the node text information is larger than or equal to a preset threshold value or not;
when the same proportion is larger than or equal to the preset threshold value, the matching is successful;
when the same proportion is smaller than the preset threshold value, the matching is failed;
or
Comparing the text information with the node text information of the current control node, and judging whether the text information is consistent with the node text information of the current control node;
when the text information is consistent with the node text information of the current control node, matching is successful;
when the text information is inconsistent with the node text information of the current control node, the matching is failed;
or
Judging whether the same proportion between the text information and the node text information is greater than or equal to a preset threshold value or not;
when the same proportion is larger than or equal to the preset threshold value, the matching is successful;
and when the same proportion is smaller than the preset threshold value, the matching is failed.
3. The method of claim 1, wherein performing a simulation control operation on the current control node based on the node attribute information of the current control node comprises:
judging whether the current control node is a clickable node or not according to the node type information of the current control node;
when the current control node is a clickable node, sending a click event to the current control node so as to perform simulated click operation on the current control node;
when the current control node is not a clickable node, sending a touch event to the current control node according to the interface position information of the current control node so as to perform simulated touch operation on the current control node;
the node attribute information comprises the node type information and the interface position information, and the node type comprises clickable nodes and non-clickable nodes.
4. The method of claim 3, after sending a click event to the current control node to perform a simulated click operation on the current control node when the current control node is a clickable node, further comprising:
judging whether the click is successful;
when the click is successful, ending the process;
and when the click fails, sending the touch event to the current control node according to the interface position information of the current control node so as to perform simulated touch operation on the current control node.
5. The method of claim 1, wherein after said obtaining text information, further comprising:
judging whether the control operation corresponding to the text information belongs to a preset simple operation category or not according to the text information; wherein the preset simple operation categories include up, down, left, right, previous page, next page, home page, return, and back;
if the control operation corresponding to the text information belongs to the preset simple operation category, performing corresponding control operation according to the file information;
and if the click operation corresponding to the text information does not belong to the preset simple operation category, the step of sending a trigger instruction according to the text information so as to trigger an interface retrieval process of the intelligent terminal is carried out.
6. The method of claim 1, wherein sending a triggering instruction according to the text message to trigger an interface retrieval process of the intelligent terminal comprises:
and when the text information is received, sending an application notification message to trigger an inaccessible event callback process of the intelligent terminal through the application notification message.
7. The method of claim 1, wherein said obtaining textual information comprises:
receiving voice input by a user;
processing the voice to obtain the text information;
or
Receiving a broadcast message including text parameters;
and acquiring the text information according to the broadcast message.
8. An apparatus for analog control of an intelligent terminal, comprising:
the acquisition module is used for acquiring the text information;
the triggering module is used for sending a triggering instruction according to the text information so as to trigger an interface retrieval process of the intelligent terminal;
the node character information acquisition module is used for traversing each control node of the current interface through the interface retrieval process to acquire the node character information of the current control node;
the matching module is used for matching the text information with the node text information of the current control node according to a preset matching rule and judging whether the matching is successful or not;
and the simulation control module is used for acquiring the node attribute information of the current control node when the character information is successfully matched with the node character information of the current control node, and performing simulation control operation on the current control node according to the node attribute information of the current control node.
9. An intelligent terminal, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.
CN201810872236.2A 2018-08-02 2018-08-02 Method and device for analog control of intelligent terminal and intelligent terminal Pending CN110795175A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810872236.2A CN110795175A (en) 2018-08-02 2018-08-02 Method and device for analog control of intelligent terminal and intelligent terminal

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810872236.2A CN110795175A (en) 2018-08-02 2018-08-02 Method and device for analog control of intelligent terminal and intelligent terminal

Publications (1)

Publication Number Publication Date
CN110795175A true CN110795175A (en) 2020-02-14

Family

ID=69425869

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810872236.2A Pending CN110795175A (en) 2018-08-02 2018-08-02 Method and device for analog control of intelligent terminal and intelligent terminal

Country Status (1)

Country Link
CN (1) CN110795175A (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112286486A (en) * 2020-12-30 2021-01-29 智道网联科技(北京)有限公司 Operation method of application program on intelligent terminal, intelligent terminal and storage medium
CN112530424A (en) * 2020-11-23 2021-03-19 北京小米移动软件有限公司 Voice processing method and device, electronic equipment and storage medium
CN112530433A (en) * 2020-12-01 2021-03-19 杭州灵伴科技有限公司 Universal voice instruction generation method and device and augmented reality display equipment
CN112559112A (en) * 2021-02-25 2021-03-26 智道网联科技(北京)有限公司 Interface node positioning method and device
CN112634896A (en) * 2020-12-30 2021-04-09 智道网联科技(北京)有限公司 Operation method of application program on intelligent terminal and intelligent terminal
WO2022001013A1 (en) * 2020-06-28 2022-01-06 广州橙行智动汽车科技有限公司 Voice interaction method, vehicle, server, system, and storage medium
CN114048726A (en) * 2022-01-13 2022-02-15 北京中科汇联科技股份有限公司 Computer graphic interface interaction method and system

Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120123781A1 (en) * 2010-11-11 2012-05-17 Park Kun Touch screen device for allowing blind people to operate objects displayed thereon and object operating method in the touch screen device
CN103985382A (en) * 2014-05-23 2014-08-13 国家电网公司 Power grid dispatching auxiliary interactive method based on speech recognition technology
CN104142793A (en) * 2014-07-26 2014-11-12 珠海市君天电子科技有限公司 Method and device for simulating click on terminal control
CN104156305A (en) * 2013-05-15 2014-11-19 腾讯科技（深圳）有限公司 Application program testing method and device
CN104503779A (en) * 2014-12-09 2015-04-08 北京奇虎科技有限公司 Application program operation method and device
CN104899003A (en) * 2015-06-12 2015-09-09 广州视源电子科技股份有限公司 Terminal control method and system
CN105161106A (en) * 2015-08-20 2015-12-16 深圳Tcl数字技术有限公司 Voice control method of intelligent terminal, voice control device and television system
CN105513594A (en) * 2015-11-26 2016-04-20 许传平 Voice control system
CN105701398A (en) * 2015-12-30 2016-06-22 北京金山安全软件有限公司 Method and device for opening auxiliary function permission and electronic equipment
CN105869643A (en) * 2016-06-06 2016-08-17 青岛海信移动通信技术股份有限公司 Terminal control method based on voice and voice control device
CN106030489A (en) * 2015-05-08 2016-10-12 上海欧拉网络技术有限公司 User interface implementation method and apparatus for mobile device
CN106101789A (en) * 2016-07-06 2016-11-09 深圳Tcl数字技术有限公司 The voice interactive method of terminal and device
CN106411883A (en) * 2016-09-28 2017-02-15 北京奇虎科技有限公司 Method for saving flow and flow-saving video playing system
CN106454463A (en) * 2016-11-03 2017-02-22 Tcl集团股份有限公司 Control method and device based on television
CN106486118A (en) * 2016-09-30 2017-03-08 北京奇虎科技有限公司 A kind of sound control method of application and device
CN107154262A (en) * 2017-05-31 2017-09-12 北京安云世纪科技有限公司 A kind of voice operating method, device and mobile terminal
CN107608652A (en) * 2017-08-28 2018-01-19 三星电子（中国）研发中心 A kind of method and apparatus of Voice command graphical interfaces
CN107608609A (en) * 2016-07-11 2018-01-19 阿里巴巴集团控股有限公司 A kind of event object sending method and device
CN107632850A (en) * 2016-07-19 2018-01-26 百度在线网络技术（北京）有限公司 A kind of method and apparatus for obtaining application controls information

Patent Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120123781A1 (en) * 2010-11-11 2012-05-17 Park Kun Touch screen device for allowing blind people to operate objects displayed thereon and object operating method in the touch screen device
CN104156305A (en) * 2013-05-15 2014-11-19 腾讯科技（深圳）有限公司 Application program testing method and device
CN103985382A (en) * 2014-05-23 2014-08-13 国家电网公司 Power grid dispatching auxiliary interactive method based on speech recognition technology
CN104142793A (en) * 2014-07-26 2014-11-12 珠海市君天电子科技有限公司 Method and device for simulating click on terminal control
CN104503779A (en) * 2014-12-09 2015-04-08 北京奇虎科技有限公司 Application program operation method and device
CN106030489A (en) * 2015-05-08 2016-10-12 上海欧拉网络技术有限公司 User interface implementation method and apparatus for mobile device
CN104899003A (en) * 2015-06-12 2015-09-09 广州视源电子科技股份有限公司 Terminal control method and system
CN105161106A (en) * 2015-08-20 2015-12-16 深圳Tcl数字技术有限公司 Voice control method of intelligent terminal, voice control device and television system
CN105513594A (en) * 2015-11-26 2016-04-20 许传平 Voice control system
CN105701398A (en) * 2015-12-30 2016-06-22 北京金山安全软件有限公司 Method and device for opening auxiliary function permission and electronic equipment
CN105869643A (en) * 2016-06-06 2016-08-17 青岛海信移动通信技术股份有限公司 Terminal control method based on voice and voice control device
CN106101789A (en) * 2016-07-06 2016-11-09 深圳Tcl数字技术有限公司 The voice interactive method of terminal and device
CN107608609A (en) * 2016-07-11 2018-01-19 阿里巴巴集团控股有限公司 A kind of event object sending method and device
CN107632850A (en) * 2016-07-19 2018-01-26 百度在线网络技术（北京）有限公司 A kind of method and apparatus for obtaining application controls information
CN106411883A (en) * 2016-09-28 2017-02-15 北京奇虎科技有限公司 Method for saving flow and flow-saving video playing system
CN106486118A (en) * 2016-09-30 2017-03-08 北京奇虎科技有限公司 A kind of sound control method of application and device
CN106454463A (en) * 2016-11-03 2017-02-22 Tcl集团股份有限公司 Control method and device based on television
CN107154262A (en) * 2017-05-31 2017-09-12 北京安云世纪科技有限公司 A kind of voice operating method, device and mobile terminal
CN107608652A (en) * 2017-08-28 2018-01-19 三星电子（中国）研发中心 A kind of method and apparatus of Voice command graphical interfaces

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022001013A1 (en) * 2020-06-28 2022-01-06 广州橙行智动汽车科技有限公司 Voice interaction method, vehicle, server, system, and storage medium
CN112530424A (en) * 2020-11-23 2021-03-19 北京小米移动软件有限公司 Voice processing method and device, electronic equipment and storage medium
CN112530433A (en) * 2020-12-01 2021-03-19 杭州灵伴科技有限公司 Universal voice instruction generation method and device and augmented reality display equipment
CN112530433B (en) * 2020-12-01 2023-11-07 杭州灵伴科技有限公司 General voice instruction generation method and device and augmented reality display device
CN112286486A (en) * 2020-12-30 2021-01-29 智道网联科技(北京)有限公司 Operation method of application program on intelligent terminal, intelligent terminal and storage medium
CN112634896A (en) * 2020-12-30 2021-04-09 智道网联科技(北京)有限公司 Operation method of application program on intelligent terminal and intelligent terminal
CN112559112A (en) * 2021-02-25 2021-03-26 智道网联科技(北京)有限公司 Interface node positioning method and device
CN114048726A (en) * 2022-01-13 2022-02-15 北京中科汇联科技股份有限公司 Computer graphic interface interaction method and system
CN114048726B (en) * 2022-01-13 2022-04-08 北京中科汇联科技股份有限公司 Computer graphic interface interaction method and system

Similar Documents

Publication Publication Date Title
CN110795175A (en) Method and device for analog control of intelligent terminal and intelligent terminal
CN111260545B (en) Method and device for generating image
CN107437416A (en) A kind of consultation service processing method and processing device based on speech recognition
CN109561271B (en) Method for guiding terminal operation, first terminal and second terminal
CN106126424B (en) A kind of visual automated testing method of script, device and mobile terminal
US20170242832A1 (en) Character editing method and device for screen display device
CN112527962A (en) Intelligent response method and device based on multi-mode fusion, machine readable medium and equipment
CN112597065B (en) Page testing method and device
CN108958503A (en) input method and device
CN105488370B (en) A kind of unlocking method and device of child mode
CN111899859A (en) Surgical instrument counting method and device
CN112286485B (en) Method and device for controlling application through voice, electronic equipment and storage medium
CN112597066B (en) Page testing method and device
CN112634896B (en) Operation method of application program on intelligent terminal and intelligent terminal
CN110427277B (en) Data verification method, device, equipment and storage medium
CN110046237B (en) Conversational interactive processing method and device
CN114764556A (en) Data processing method, data processing device, computer equipment and storage medium
CN106330864A (en) Verification information processing methods, apparatuses and system
CN111221723A (en) Test record generation method and device, storage medium and electronic equipment
CN110209242A (en) Push button function binding, push button function call method, device and projection control equipment
CN111966734A (en) Data processing method and electronic equipment of spreadsheet combined with RPA and AI
CN110221756A (en) Method, apparatus, terminal and the storage medium of data are applied in processing
CN111459917B (en) Knowledge base management method, device and processing equipment
CN116303101B (en) Test case generation method, device and equipment
CN113220307B (en) Optimization method and device for code coverage analysis and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20200214

RJ01 Rejection of invention patent application after publication