CN112286487B - Voice guidance operation method and device, electronic equipment and storage medium - Google Patents
Voice guidance operation method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN112286487B CN112286487B CN202011600793.2A CN202011600793A CN112286487B CN 112286487 B CN112286487 B CN 112286487B CN 202011600793 A CN202011600793 A CN 202011600793A CN 112286487 B CN112286487 B CN 112286487B
- Authority
- CN
- China
- Prior art keywords
- voice
- application program
- interface
- information
- information set
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention provides a voice guidance operation method, a voice guidance operation device, electronic equipment and a storage medium, wherein the method comprises the following steps: receiving a voice operation instruction of an application program on the vehicle-mounted intelligent terminal; the voice operation instruction is pre-configured to have a corresponding relation with an executable operation function on an interface of the application program, and the executable operation function comprises random operation which can be executed by a current interface of the application program in a touch mode; determining an executable operation function and random operation which can be executed in a touch mode in an application program according to the voice operation instruction; and executing the operation instruction corresponding to the voice operation instruction in the application program according to the executable operation function and the random operation which can be executed in a touch mode. When the method is applied to the existing intelligent terminal, the voice guide operation of different application programs in the background of the intelligent terminal can be realized only by starting the method at the mobile terminal without independently integrating a voice operation tool kit for each application in advance.
Description
Technical Field
The present invention relates to the field of voice control technologies, and in particular, to a voice guidance operation method and apparatus, an electronic device, and a storage medium.
Background
At present, an Application (APP) running on a smart mobile terminal generally realizes interaction by clicking a touch screen, and specifically includes: the user sends out a command through manual operation on the control interface, the intelligent mobile terminal receives the command and then responds, the response result is fed back to the user in a visual mode, and the user performs corresponding operation according to the seen interface.
However, there are some application scenarios where the user manually issues a command, and there is operational inconvenience in viewing the response of the mobile terminal through the eyes. One of the most typical application scenarios is when the user is in a driving situation. Obviously, the user needs both hands to hold the steering wheel, and the user's eyes need to be absorbed in the road conditions, and under the condition that eyes and both hands are not free, voice control becomes the direction that vehicle control system researched and developed.
When the application running on the intelligent mobile terminal is controlled based on a voice mode, two aspects of technologies are involved, one is that after a user command is sent out through voice, the intelligent terminal recognizes the user voice command; secondly, how the intelligent terminal informs the user of feedback based on the recognition of the voice command. Research on the voice recognition problem is very extensive, and the feedback of the intelligent terminal based on the voice command or the voice-guided operation of the intelligent terminal is more remarkable.
Existing applications voice-guided operations generally require that an SDK (Software Development Kit) for voice operations be integrated inside the application to be controlled to implement voice-guided operations. However, for most current applications, the SDK for voice operation is not integrated in advance, so that a technical problem to be solved is still needed for a person skilled in the art to perform voice guidance operation across different APPs in an intelligent terminal.
Disclosure of Invention
The invention provides a voice guide operation method, a voice guide operation device, electronic equipment and a storage medium, which are used for solving the defect that voice guide operation is limited in an application program in the prior art and a voice operation toolkit needs to be integrated in advance and realizing voice guide operation of different application programs in an intelligent terminal.
The invention provides a voice guidance operation method, which comprises the following steps: receiving a voice operation instruction of an application program on the vehicle-mounted intelligent terminal; the voice operation instruction is pre-configured to have a corresponding relation with an executable operation function on the interface of the application program, and the executable operation function comprises random operation which can be executed by a current interface of the application program in a touch mode; determining the executable operation function and the random operation which can be executed in a touch mode in the application program according to the voice operation instruction; and executing an operation instruction corresponding to the voice operation instruction in the application program according to the executable operation function and the random operation which can be executed in a touch mode.
According to the voice guidance operation method provided by the invention, the voice operation instruction is obtained through the following steps: under the condition that the interface of an application program changes, collecting an interface information set corresponding to the interface with the changed application program; matching a pre-stored search characteristic information set associated with the application program with the interface information set, and calculating the similarity; determining search characteristic information matched with the interface change in the search characteristic information set based on the similarity; recording at least one guiding operation corresponding to the searching characteristic information matched with the interface change in feedback information; wherein the booting operation is a pre-association setting; and acquiring the voice operation instruction by utilizing voice synthesis based on the feedback information.
According to the voice guidance operation method provided by the invention, the interface information set is a text information set and/or a picture information set; the text information set comprises at least one text information; the picture information set includes at least one picture information.
According to the voice guidance operation method provided by the invention, the step of collecting the interface information set corresponding to the interface changed by the application program comprises the following steps: creating a text information data linked list and a picture information data linked list; scanning an interface contained in the application program, and respectively writing corresponding data into the text information data linked list and the picture information data linked list based on a scanning result; determining the text information set based on data stored in the text information data linked list; and determining the picture information set based on the data stored in the picture information data linked list.
According to the voice guidance operation method provided by the invention, the search feature information set is obtained by the following steps: when an application program is opened, identifying a name identifier corresponding to the application program; transmitting the name identifier to a cloud server, and requesting search characteristic information associated with the name identifier from the cloud server; receiving search characteristic information sent by the cloud server, and downloading the search characteristic information to a local mobile terminal; the searching characteristic information is preset in the cloud server and comprises target text information, target image information, an application program name identifier and at least one guiding operation corresponding to the application program name identifier; and screening the target text information and the target image information which are associated with the application program name identification in the search characteristic information to generate the search characteristic information set.
According to the voice guidance operation method provided by the present invention, recording the at least one guidance operation corresponding to the search feature information matching the interface change in the feedback information includes: creating a feedback information data linked list; determining search characteristic information of which the similarity of an interface information set corresponding to the interface change exceeds a preset threshold; storing the search characteristic information in the feedback information data linked list; acquiring the length of the feedback information data linked list; and randomly extracting a text character string corresponding to the search characteristic information in the feedback information data linked list in the guiding operation to generate feedback information.
According to the voice guidance operation method provided by the invention, the change of the interface of the application program comprises the following steps: click, slide, or window switch.
In a second aspect, the present invention also provides a voice guidance operating device, including: the device comprises a receiving module, a confirming module and an executing module. The receiving module is used for receiving a voice operation instruction of an application program on the vehicle-mounted intelligent terminal; the voice operation instruction is pre-configured to have a corresponding relation with an executable operation function on the interface of the application program, and the executable operation function comprises random operation which can be executed by a current interface of the application program in a touch mode; the confirming module is used for confirming the executable operation function and the random operation which can be executed in a touch mode in the application program according to the voice operation instruction; the execution module is used for executing the operation instruction corresponding to the voice operation instruction in the application program according to the executable operation function and the random operation which can be executed in a touch mode.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the steps of any one of the voice guidance operation methods.
The invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method of voice-guided operation as described in any of the above
According to the voice guidance operation method, the voice guidance operation device, the electronic equipment and the storage medium, the executable operation function and the random operation which can be executed in a touch mode are determined by receiving the voice operation instruction of the application program on the vehicle-mounted intelligent terminal, and the operation instruction corresponding to the voice operation instruction is executed in the application program according to the executable operation function and the random operation which can be executed in the touch mode.
Therefore, the method and the device realize the feedback based on voice for the change of the interactive function presented by the change of the interactive interface of the application program, overcome the defect that the voice guide operation is limited to the self expansion and modification mode of the application program in the prior art, when the method and the device are applied to the prior intelligent terminal, the voice guide operation of different application programs in the background of the intelligent terminal can be realized only by starting the operation of the method and the device on the mobile terminal without independently integrating the voice operation kit for each application in advance, and the method and the device are convenient, practical and easy to popularize.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a flow chart of a voice guidance operation method provided by the present invention;
FIG. 2 is a schematic flow chart of a voice operation instruction in the voice guidance operation method according to the present invention;
fig. 3 is a schematic diagram of a collection flow of a text information set and a picture information set in the voice guidance operation method provided by the present invention;
fig. 4 is a schematic diagram illustrating a flow of acquiring a pre-stored search feature information set associated with an application program in the voice guidance operation method provided by the present invention;
fig. 5 is a schematic flow chart of generating voice feedback in the voice guidance operation method provided by the present invention;
FIG. 6 is a schematic structural diagram of a voice guidance operating device provided by the present invention;
fig. 7 is a schematic structural diagram of a voice operation instruction generating module connected to a receiving module of the voice guidance operating device according to the present invention;
fig. 8 is a schematic structural diagram of a collection unit of a voice operation instruction generation module in the voice guidance operation device provided by the invention;
fig. 9 is a schematic structural diagram of a computing unit of a voice operation instruction generating module in the voice guidance operating device provided by the invention;
fig. 10 is a schematic structural diagram of a guiding operation corresponding unit of a voice operation instruction generating module in the voice guiding operation device provided by the invention;
fig. 11 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The voice guidance operation method of the present invention is described below with reference to fig. 1 to 4.
It should be noted that, in the voice guidance operation method of the present invention, an execution main body of the method may be a terminal device, and specifically may be a processing system of the terminal device, or a plug-in loaded in the terminal device and used for implementing voice control, where the terminal device may be a smart phone, a tablet computer, a vehicle-mounted control device, and the present invention is not limited thereto.
Referring to fig. 1, fig. 1 is a schematic flow chart of a voice guidance operation method provided by the present invention, including the following steps:
and step 110, receiving a voice operation instruction of an application program on the vehicle-mounted intelligent terminal.
The voice operation instruction is pre-configured to have a corresponding relation with an executable operation function on the interface of the application program, and the executable operation function comprises random operation which can be executed by a current interface of the application program in a touch mode;
and step 120, determining the executable operation function and the random operation which can be executed in a touch mode in the application program according to the voice operation instruction.
And step 130, executing an operation instruction corresponding to the voice operation instruction in the application program according to the executable operation function and the random operation executable in the touch mode.
The method includes the steps of receiving a voice operation instruction of an application program on the vehicle-mounted intelligent terminal, determining an executable operation function and random operation capable of being executed in a touch mode, and executing the operation instruction corresponding to the voice operation instruction in the application program according to the executable operation function and the random operation capable of being executed in the touch mode.
Therefore, the method and the device realize the feedback based on voice for the change of the interactive function presented by the change of the interactive interface of the application program, overcome the defect that the voice guide operation is limited to the self expansion and modification mode of the application program in the prior art, when the method and the device are applied to the prior intelligent terminal, the voice guide operation of different application programs in the background of the intelligent terminal can be realized only by starting the operation of the method and the device on the mobile terminal without independently integrating the voice operation kit for each application in advance, and the method and the device are convenient, practical and easy to popularize. Referring to fig. 2, fig. 2 is a schematic view illustrating a flow of a voice operation instruction in the voice guidance operation method provided by the present invention, including the following steps:
and step 250, acquiring the voice operation instruction by utilizing voice synthesis based on the feedback information.
Next, each step will be explained.
With respect to step 210.
In this step, the meaning of the application program is that the application program is not the application of the voice guidance operation of the present invention; that is, applications other than voice guidance operations may be considered as application programs of the present application. In specific implementation, the number of the monitored applications is not limited, and may be one or more.
The "change" in this embodiment may be that App is clicked, slid, and switched through a window, which is not limited in the present invention. And triggering information collection of the interface of the current application program every time the interface of the application program changes.
The "monitoring" here may be implemented in actual operation as follows: and refreshing the interface of the application program regularly or irregularly, and collecting an interface information set corresponding to the changed interface under the condition that the interface information is changed.
In one embodiment, the interface information set may be a text information set or a picture information set, and of course, the interface information set may also be a text information set and a picture information set. The text information set may include one text information or a plurality of text information. The picture information set may include one picture information or may include a plurality of picture information.
As can be seen from the above description, for an interface of an application, the interface information set may be continuously changed over time, and the specific content may be that the text of the interface is changed or that the picture of the interface is changed.
In a preferred embodiment, the collection of the text information set and the picture information set may be performed as follows, referring to fig. 3, fig. 3 is a flow chart of collecting the text information set and the picture information set in the voice guidance operation method provided by the present invention, and the flow chart includes the following steps:
step 320, scanning an interface included in the application program, and writing corresponding data into the text information data linked list and the picture information data linked list respectively based on a result obtained by the scanning;
It can be seen that through the above steps 310 to 340, the collection of the text information set and the picture information set is completed. It should be noted that, this embodiment only provides a collection method of text information and picture information, and the present invention is not limited to this, and other collection methods of text information and picture information are also within the protection scope of the present invention.
Step 220 is further described below.
And step 220, matching the pre-stored search characteristic information set associated with the application program with the interface information set, and calculating the similarity between the elements.
The following describes the search characteristic information set, the matching between the search characteristic information set and the interface information set, the calculation method of the similarity, and how the search characteristic information set is stored in advance one by one.
1) Searching feature information sets
Each of the applications installed in the mobile terminal has a plurality of attributes, and a name identifier of the application, text information associated with the application, image information, and text of at least one guidance operation corresponding to the application name identifier may be used as the search feature information.
2) Acquisition of search feature information set
When the application program is opened in the mobile terminal, the application program is identified, and the text information set, the image information set and the text operation text associated with the application program are searched according to the name identification of the application program. The feature information set in this step is searched, that is, the feature information set includes a text information set and an image information set.
Referring to fig. 4, fig. 4 is a flowchart of a step of acquiring a pre-stored search feature information set associated with an application program according to an embodiment of the voice guidance operation method of the present invention, where the step includes:
And step 430, receiving the search characteristic information sent by the cloud server, and downloading the search characteristic information to the local mobile terminal.
The searching characteristic information is preset in the cloud server and comprises target text information, target image information, an application program name identifier and at least one guiding operation corresponding to the application program name identifier;
As can be seen from the above steps 410 to 440, when the application is not opened, the mobile terminal does not locally store the search feature information set related to the application, which is to call the cloud server for the associated information when the application is opened. When the current application is closed, the relevant associated information may be cleared.
3) Matching of search characteristic information set and interface information set
For example, in one embodiment, the search feature information set F includes a text information set T and an image information set P.
I.e., F = { T, P }
Wherein the text information set is T = { T1, T2, T3, T4}
The image information set is P = { P1, P2, P3}
At the present moment, through the control, the interface of discovery application APP changes, and through the scanning, the interface information set that the application APP that collects changes the interface and corresponds is A:
in this embodiment, the set of interface information is a set of image information, a = { a1, a2}
The operation of matching the search characteristic information set F with the interface information set a is specifically as follows: matching of the image information set of P = { P1, P2, P3} with the interface information set of a, including the following matching pairs:
（a1，p1），（a1，p2），（a1，p3），（a2，p1），（a2，p2），
（a2，p3）
of course, the above embodiment only provides matching when the interface information set is a set of image information, and in specific implementation, the interface information set may also be a text information set. The matching manner is similar to the above matching, and detailed description is not given.
4) Similarity calculation mode
Upon completion of step S120, the set of search characteristic information associated with the obtained application is matched against the set of interface information, i.e., similar to the matches of (a 1, p 1), (a 1, p 2), (a 1, p 3), (a 2, p 1), (a 2, p 2), (a 2, p 3). The next operation is to calculate the similarity of these matches.
In the present embodiment, each element in the set of image information P = { P1, P2, P3} is matched with each element in the set of interface information a = { a1, a2} that has changed, and a is a set of image information. At the moment, the similarity can be calculated through an openCV universal matching recognition algorithm. If the changed interface information set is a text information set, the similarity can be calculated through a character string acquaintance algorithm.
Step 230 is explained below.
In this step, the corresponding search feature information when the current application interface changes is determined based on the similarity obtained in step 220.
A more preferable rule is that the search feature information having the highest similarity in the search feature information set is determined as the search feature information matching the interface change.
Continuing with step 120, in one embodiment, the similarity is calculated by openCV universal matching recognition algorithm, and the similarity corresponding to each matching pair is:
S（a1，p1）=s1
S（a1，p2）=s2
S（a1，p3）=s3
S（a2，p1）=s4
S（a2，p2）=s5
S（a2，p3）=s6
and if s4 is the maximum, determining that the search characteristic information corresponding to the current application interface change is p 1.
For example, in one embodiment, referring to fig. 5, recording at least one guidance operation corresponding to the search feature information matching the interface change to the feedback information may include the steps of:
and 550, randomly extracting a text character string corresponding to the search characteristic information in the feedback information data linked list in the guiding operation to generate feedback information.
And step 250, acquiring a voice operation instruction by utilizing voice synthesis based on the feedback information.
In the embodiment, an interface information set is obtained by monitoring the change of any application program interface except the interface information set, similarity matching is performed between the interface information set and a pre-agreed search characteristic information set, search characteristic information corresponding to the interface change is determined, then guidance operation agreed by the association of the search characteristic information is used as feedback information, and voice synthesis is utilized to send out voice.
An example of a voice guidance operation method is given below, which is an application program that is resident in the background of the terminal device in implementation and may be called Service, and the example performs landing on the method of the above embodiment from the perspective of computer software implementation. It should be noted that the following example is only one possible implementation of the program for implementing the method of the present invention, and the present invention is not limited to the following implementation.
The following description is made in connection with several links of the Service in the process of implementing the Service through computer software:
1. the Service memory creates and caches a textCacheList, an imageCacheList, a searchTextList, a searchImageList datalink, and a feedbackList.
2. The Service monitors changes on any other third party App interface except the Service, the changes include that the App is clicked, slid, window switching and the like, and the Service is triggered to scan the third party App interface with the View nodes once when other App interfaces change.
After the Service scans other App, data collection is carried out on all views on an App interface, the interface comprises n views, 1 NodeInfo data object is created for each View, the NodeInfo data object stores Text information of the View, Rect (Left, Right, Top, Bottom) information of the View and cache information of the View, if the searched View does not contain the Text information, the NodeInfo is added into the 1 st ImageCacheList linked list, and if the Text contains the information, the NodeInfo is added into the 1 st textCacheList linked list.
It should be noted that an array object (tentatively named as PolicyCaches, and an internal object in the array is named as PolicyCache) is preset in the cloud server, the PolicyCache is composed of 1 text data targettext, one image data targetImage, a packagenammer string of a third party App (each App corresponds to a unique packageName), and a feedBacks array, and the feedBacks includes a plurality of chinese texts.
Therefore, when a user opens an App on the terminal device, the Service can identify the packageName of the current App, transmit the packageName to the cloud server as a parameter, download the PolicyCache object data with the same packageName as the transmitted packageName from the policycachs array in the server to the local, add the targetText in the PolicyCache data object into the searchTextList linked list in 1, add the targetImage into the searchImageList without being empty, add the targetText and the targetImage of the PolicyCache into a mutex, and only one is not empty and the other is empty.
3. After a user opens a certain App on terminal equipment, the Service establishes textCacheList, ImageCacheList, searchTextList and searchImageList, then starts to search, searches in the textCacheList by using the targetText of the PolicyCache object in the searchTextList in a character string recognition algorithm, searches for the PolicyCache object with the highest matching degree and adds the PolicyCache object into the feedback List; and similarly, searching in the targeted image reimage cachelist of the PolicyCache object in the searchImageList in an openCV universal matching recognition algorithm, and adding the PolicyCache with the highest searching matching degree into the feedbackList linked list.
4. The feedbackList is the link table data searched finally, the length feedbackjdength of the feedbackList is obtained, a Random policy cache object in the feedbackList is obtained by using new Random (). nextInt (feedbackjdengh), and a text character string is randomly extracted from a feedbackarray of the PolicyCache by using a Random algorithm to be used as the final threadfeedbackText.
5. The Service server has a voice synthesis function and synthesizes final voice feedback by using the speackfeed BackText.
It should be noted that the above 5 steps are not steps that need to be completely processed in each voice guidance operation.
During the Service operation, step 3, step 4 and step 5 are typically performed. And when step 3 is executed, the data link lists textCacheList and ImageCacheList are determined by the establishment of the NodeInfo data object in step 2, and searchTextList and searchImageList are established according to the cloud servers searchTextList and searchImageList. The data link lists textCacheList and Imagecachelist trigger page scanning, searching and operation once each time the interface of the application is refreshed, and the textCacheList and the Imagecachelist are reestablished every time the Service services; the searchTextList and the searchImageList are established when the App is opened, and are cleared when the App is quitted.
Through the above description, it can be seen that when the user opens the third party App on the terminal device, the Service uses the data preset by the cloud server and the view data of the current App page to prompt the user and real-time voice prompt, so as to prompt the user to operate.
The voice guidance operation device provided by the present invention is described below with reference to fig. 6 to 9, and the voice guidance operation device described below and the voice guidance operation method described above may be referred to in correspondence with each other.
Referring to fig. 6, fig. 6 is a schematic structural diagram of a voice guidance operating device provided in the present invention, including: a receiving module 61, a confirming module 62 and an executing module 63.
The receiving module 61 is used for receiving a voice operation instruction of an application program on the vehicle-mounted intelligent terminal; the voice operation instruction is pre-configured to have a corresponding relation with an executable operation function on the interface of the application program, and the executable operation function comprises random operation which can be executed by a current interface of the application program in a touch mode;
the confirming module 62 is configured to determine the executable operation function and the random operation executable by the touch manner in the application program according to the voice operation instruction;
the execution module 63 is configured to execute an operation instruction corresponding to the voice operation instruction in the application program according to the executable operation function and the random operation that can be executed in a touch manner.
The method includes the steps of receiving a voice operation instruction of an application program on the vehicle-mounted intelligent terminal, determining an executable operation function and random operation capable of being executed in a touch mode, and executing the operation instruction corresponding to the voice operation instruction in the application program according to the executable operation function and the random operation capable of being executed in the touch mode.
Therefore, the method and the device realize the feedback based on voice for the change of the interactive function presented by the change of the interactive interface of the application program, overcome the defect that the voice guide operation is limited to the self expansion and modification mode of the application program in the prior art, when the method and the device are applied to the prior intelligent terminal, the voice guide operation of different application programs in the background of the intelligent terminal can be realized only by starting the operation of the method and the device on the mobile terminal without independently integrating the voice operation kit for each application in advance, and the method and the device are convenient, practical and easy to popularize.
Referring to fig. 7, in an embodiment, a voice operation instruction generating module is further connected to the receiving module 61, and the voice operation instruction generating module includes: a collection unit 601, a calculation unit 602, a matching unit 603, a guidance operation correspondence unit 604, and a synthesis unit 605.
The collection unit 601 is configured to collect, when an interface of an application program changes, an interface information set corresponding to the interface where the application program changes; the calculating unit 602 is configured to match a pre-stored search feature information set associated with the application program with the interface information set, and calculate a similarity; the matching unit 603 is configured to determine, based on the similarity, search feature information that matches the interface change in the search feature information set; the guiding operation corresponding unit 604 is configured to record at least one guiding operation corresponding to the search feature information matching the interface change in the feedback information; wherein the booting operation is a pre-association setting. The synthesis unit 605 is configured to obtain the voice operation instruction by voice synthesis based on the feedback information.
The voice guidance operation device provided by this embodiment obtains an interface information set by monitoring changes of any application program interface except for the interface, matches the interface information set with a search feature information set agreed in advance in a similarity manner, determines search feature information corresponding to the interface changes, and then takes a guidance operation agreed by the association of the search feature information as feedback information to generate voice by using voice synthesis.
In one embodiment, the interface information set is a text information set and/or a picture information set; the text information set comprises at least one text information; the picture information set includes at least one picture information.
Also, in one embodiment, referring to fig. 8, the collection unit 601 may further include: a creation unit 6011, a scanning unit 6012, a text information set generation unit 6013, and a picture information set generation unit 6014.
The creating unit 6011 is configured to create a text information data linked list and a picture information data linked list; the scanning unit 6012 is configured to scan an interface included in the application program, and based on a result obtained by the scanning, write corresponding data into the text information data linked list and the picture information data linked list, respectively; a text information set generating unit 6013 is configured to determine the text information set based on data stored in the text information data linked list; the picture information set generating unit 6014 is configured to determine the picture information set based on the data stored in the picture information data linked list.
Referring to fig. 9, in an embodiment, fig. 9 is a schematic structural diagram of a downloading subunit in a computing unit of a voice operation instruction generating module in the voice guidance operating apparatus provided in the present invention, where the downloading subunit includes: a recognition portion 6021, a request portion 6022, a reception portion 6023, and a screening portion 6024.
The identification part 6021 is used for identifying the name identifier corresponding to the application program when the application program is opened. The request portion 6022 transfers the name identification to the cloud server, and requests the search feature information associated with the name identification from the cloud server. The receiving part 6023 receives the search characteristic information sent by the cloud server and downloads the search characteristic information to the local mobile terminal; the searching characteristic information is preset in the cloud server and comprises target text information, target image information, an application program name identifier and at least one guiding operation corresponding to the application program name identifier. The filtering portion 6024 is configured to filter the target text information and the target image information associated with the application name identifier from the search feature information, and generate the search feature information set.
Referring to fig. 10, fig. 10 is a schematic structural diagram of a guiding operation corresponding unit of a voice operation instruction generating module in the voice guiding operation device provided by the present invention, and the guiding operation corresponding unit includes: a data link list creation section 6041, a search characteristic information determination section 6042, a storage section 6043, a length acquisition section 6044, and a feedback information generation section 6045.
Wherein, the data link table creation part 6041 creates a feedback information data link table; a search characteristic information specifying unit 6042 configured to specify search characteristic information in which the similarity of the interface information set corresponding to the interface change exceeds a preset threshold; a storage part 6043 that stores the search characteristic information in the feedback information data link table; a length acquiring unit 6044 configured to acquire the length of the feedback information data link table; the feedback information generating unit 6045 randomly extracts a text string corresponding to the guidance operation for the search feature information in the feedback information data link table, and generates feedback information.
Fig. 11 illustrates a physical structure diagram of an electronic device, and as shown in fig. 11, the electronic device may include: a processor (processor)1110, a communication Interface (Communications Interface)1120, a memory (memory)1130, and a communication bus 1140, wherein the processor 1110, the communication Interface 1120, and the memory 1130 communicate with each other via the communication bus 1140. Processor 1110 may invoke logic instructions in memory 1130 to perform a voice-guided operation method comprising:
monitoring an interface of an application program, and collecting an interface information set corresponding to the interface with the change of the application program under the condition of the change;
matching a pre-stored search characteristic information set associated with the application program with the interface information set, and calculating the similarity between elements;
determining search characteristic information matched with the interface change in the search characteristic information set based on the similarity;
recording at least one guiding operation corresponding to the searching characteristic information matched with the interface change in feedback information; the guiding operation corresponding to the searching characteristic information is preset in a correlated mode;
based on the feedback information, speech is uttered by speech synthesis to form speech feedback.
In addition, the logic instructions in the memory 1130 may be implemented in software functional units and stored in a computer readable storage medium when sold or used as a stand-alone product. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the method provided by the above methods to perform a voice-guided operation, the method comprising: receiving a voice operation instruction of an application program on the vehicle-mounted intelligent terminal; the voice operation instruction is pre-configured to have a corresponding relation with an executable operation function on the interface of the application program, and the executable operation function comprises random operation which can be executed by a current interface of the application program in a touch mode; determining the executable operation function and the random operation which can be executed in a touch mode in the application program according to the voice operation instruction; and executing an operation instruction corresponding to the voice operation instruction in the application program according to the executable operation function and the random operation which can be executed in a touch mode.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor is implemented to perform the methods provided above to perform voice guidance operations, the method comprising: receiving a voice operation instruction of an application program on the vehicle-mounted intelligent terminal; the voice operation instruction is pre-configured to have a corresponding relation with an executable operation function on the interface of the application program, and the executable operation function comprises random operation which can be executed by a current interface of the application program in a touch mode; determining the executable operation function and the random operation which can be executed in a touch mode in the application program according to the voice operation instruction; and executing an operation instruction corresponding to the voice operation instruction in the application program according to the executable operation function and the random operation which can be executed in a touch mode.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (9)
1. A method of voice-guided operation, comprising:
receiving a voice operation instruction of an application program on the vehicle-mounted intelligent terminal; the voice operation instruction is pre-configured to have a corresponding relation with an executable operation function on the interface of the application program, and the executable operation function comprises random operation which can be executed by a current interface of the application program in a touch mode;
determining the random operation which can be executed in a touch mode in the application program according to the voice operation instruction;
according to the random operation which can be executed in a touch mode, executing an operation instruction corresponding to the voice operation instruction in the application program;
the voice operation instruction is obtained through the following steps:
under the condition that the interface of an application program changes, collecting an interface information set corresponding to the interface with the changed application program;
matching a pre-stored search characteristic information set associated with the application program with the interface information set, and calculating the similarity;
determining search characteristic information matched with the interface change in the search characteristic information set based on the similarity;
recording at least one guiding operation corresponding to the searching characteristic information matched with the interface change in feedback information; wherein the booting operation is a pre-association setting;
and acquiring the voice operation instruction by utilizing voice synthesis based on the feedback information.
2. The voice-guided operation method according to claim 1,
the interface information set is a text information set and/or a picture information set;
the text information set comprises at least one text information;
the picture information set includes at least one picture information.
3. The voice-guided operation method according to claim 2, wherein the collecting of the interface information set corresponding to the interface in which the application program has changed comprises the steps of:
creating a text information data linked list and a picture information data linked list;
scanning an interface contained in the application program, and respectively writing corresponding data into the text information data linked list and the picture information data linked list based on a scanning result;
determining the text information set based on data stored in the text information data linked list;
and determining the picture information set based on the data stored in the picture information data linked list.
4. The voice-guided operation method according to any one of claims 1 to 3, characterized in that the search feature information set is obtained by:
when an application program is opened, identifying a name identifier corresponding to the application program;
transmitting the name identifier to a cloud server, and requesting search characteristic information associated with the name identifier from the cloud server;
receiving search characteristic information sent by the cloud server, and downloading the search characteristic information to a local mobile terminal; the searching characteristic information is preset in the cloud server and comprises target text information, target image information, an application program name identifier and at least one guiding operation corresponding to the application program name identifier;
and screening the target text information and the target image information which are associated with the application program name identification in the search characteristic information to generate the search characteristic information set.
5. The voice-guided operation method according to claim 4, wherein the recording of the at least one guidance operation corresponding to the search feature information matching the interface change in feedback information includes:
creating a feedback information data linked list;
determining search characteristic information of which the similarity of an interface information set corresponding to the interface change exceeds a preset threshold;
storing the search characteristic information in the feedback information data linked list;
acquiring the length of the feedback information data linked list;
and randomly extracting a text character string corresponding to the search characteristic information in the feedback information data linked list in the guiding operation to generate feedback information.
6. The voice-guided operation method according to claim 1,
the changing of the interface of the application program comprises the following steps: click, slide, or window switch.
7. A voice-guidance operation device characterized by comprising:
the receiving module is used for receiving a voice operation instruction of an application program on the vehicle-mounted intelligent terminal; the voice operation instruction is pre-configured to have a corresponding relation with an executable operation function on the interface of the application program, and the executable operation function comprises random operation which can be executed by a current interface of the application program in a touch mode;
the confirming module is used for confirming the random operation which can be executed in a touch mode in the application program according to the voice operation instruction;
the execution module is used for executing the operation instruction corresponding to the voice operation instruction in the application program according to the random operation which can be executed in a touch mode;
the voice operation instruction generating module is used for generating a voice operation instruction and comprises the following units:
the device comprises a collecting unit, a judging unit and a judging unit, wherein the collecting unit is used for collecting an interface information set corresponding to an interface with a changed application program under the condition that the interface of the application program is changed;
the computing unit is used for matching a pre-stored search characteristic information set associated with the application program with the interface information set and computing similarity;
the matching unit is used for determining search characteristic information matched with the interface change in the search characteristic information set based on the similarity;
the guiding operation corresponding unit is used for recording at least one guiding operation corresponding to the searching characteristic information matched with the interface change in the feedback information; wherein the booting operation is a pre-association setting;
and the synthesis unit is used for acquiring the voice operation instruction by utilizing voice synthesis based on the feedback information.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of voice guidance operation according to any of claims 1 to 6 are implemented when the program is executed by the processor.
9. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of voice-guided operation of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011600793.2A CN112286487B (en) | 2020-12-30 | 2020-12-30 | Voice guidance operation method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011600793.2A CN112286487B (en) | 2020-12-30 | 2020-12-30 | Voice guidance operation method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112286487A CN112286487A (en) | 2021-01-29 |
| CN112286487B true CN112286487B (en) | 2021-03-16 |
Family
ID=74426672
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011600793.2A Active CN112286487B (en) | 2020-12-30 | 2020-12-30 | Voice guidance operation method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112286487B (en) |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104599669A (en) * | 2014-12-31 | 2015-05-06 | 乐视致新电子科技（天津）有限公司 | Voice control method and device |
| CN106373570A (en) * | 2016-09-12 | 2017-02-01 | 深圳市金立通信设备有限公司 | Voice control method and terminal |
| EP3328097B1 (en) * | 2016-11-24 | 2020-06-17 | Oticon A/s | A hearing device comprising an own voice detector |
| US10169604B2 (en) * | 2016-12-13 | 2019-01-01 | International Business Machines Corporation | Method and system to prevent ultrasound data leaks in mobile devices |
| CN109102802B (en) * | 2017-06-21 | 2023-10-17 | 三星电子株式会社 | System for processing user utterances |
| CN108108142A (en) * | 2017-12-14 | 2018-06-01 | 广东欧珀移动通信有限公司 | Voice information processing method, device, terminal device and storage medium |
| CN112114770A (en) * | 2019-06-19 | 2020-12-22 | 百度在线网络技术（北京）有限公司 | Interface guiding method, device and equipment based on voice interaction |
| CN111061452A (en) * | 2019-12-17 | 2020-04-24 | 北京小米智能科技有限公司 | Voice control method and device of user interface |
| CN111968640A (en) * | 2020-08-17 | 2020-11-20 | 北京小米松果电子有限公司 | Voice control method and device, electronic equipment and storage medium |
| CN112040442B (en) * | 2020-08-21 | 2023-03-24 | 博泰车联网(南京)有限公司 | Interaction method, mobile terminal, vehicle-mounted terminal and computer-readable storage medium |
-
2020
- 2020-12-30 CN CN202011600793.2A patent/CN112286487B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN112286487A (en) | 2021-01-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11874904B2 (en) | Electronic device including mode for using an artificial intelligence assistant function of another electronic device | |
| CN108363811A (en) | Device identification method and device, electronic equipment, storage medium | |
| CN106649446B (en) | Information pushing method and device | |
| CN112463106A (en) | Voice interaction method, device and equipment based on intelligent screen and storage medium | |
| US10950240B2 (en) | Information processing device and information processing method | |
| CN109861851A (en) | A kind of distribution method, apparatus, storage medium and the mobile terminal of household appliance | |
| CA3166742A1 (en) | Method of generating text plan based on deep learning, device and electronic equipment | |
| CN108733666B (en) | Server information pushing method, terminal information sending method, device and system | |
| CN102999628A (en) | Search method and information search terminal | |
| CN111479250A (en) | File sharing method, device and system and terminal equipment | |
| KR102205686B1 (en) | Method and apparatus for ranking candiate character and method and device for inputting character | |
| CN112286487B (en) | Voice guidance operation method and device, electronic equipment and storage medium | |
| CN111225115B (en) | Information providing method and device | |
| CN113766504A (en) | Communication connection method, device, server, terminal device, system and medium | |
| WO2018145574A1 (en) | Information processing method and device, terminal, server and storage medium | |
| CN108509442B (en) | Search method and apparatus, server, and computer-readable storage medium | |
| CN112491940B (en) | Request forwarding method and device of proxy server, storage medium and electronic equipment | |
| CN112331201A (en) | Voice interaction method and device, storage medium and electronic device | |
| KR101968287B1 (en) | Apparatus and method for providing transaction of an intellectual property service | |
| CN110442806A (en) | The method and apparatus of image for identification | |
| CN106254575B (en) | A kind of method and apparatus of determining user identifier | |
| CN111625746B (en) | Application page display method, system, electronic device and storage medium | |
| CN113421565A (en) | Search method, search device, electronic equipment and storage medium | |
| CN106453573A (en) | Method and system for processing CGI request in HTTP server | |
| CN111078215A (en) | Software product application method and device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |