CN111385633A - Resource searching method based on voice, intelligent terminal and storage medium - Google Patents
Resource searching method based on voice, intelligent terminal and storage medium Download PDFInfo
- Publication number
- CN111385633A CN111385633A CN201811609705.8A CN201811609705A CN111385633A CN 111385633 A CN111385633 A CN 111385633A CN 201811609705 A CN201811609705 A CN 201811609705A CN 111385633 A CN111385633 A CN 111385633A
- Authority
- CN
- China
- Prior art keywords
- search
- voice
- interface
- playing application
- video playing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/432—Content retrieval operation from a local storage medium, e.g. hard-disk
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/232—Content retrieval operation locally within server, e.g. reading video streams from disk arrays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440236—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/443—OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/8166—Monomedia components thereof involving executable data, e.g. software
- H04N21/8173—End-user applications, e.g. Web browser, game
Abstract
The invention discloses a resource searching method based on voice, an intelligent terminal and a storage medium, wherein the method comprises the following steps: receiving voice input by a user, identifying voice content, and searching video information corresponding to the voice content in a video resource library corresponding to the current video playing application; if the video information is not found and the system is monitored to have the third-party video playing application installed, analog search is carried out in the installed third-party video playing application according to the voice content so as to find the video information corresponding to the voice content. According to the invention, through recognizing the voice of the user, the resources required by the user are searched in the current video playing application and the third-party video playing application in a combined manner, so that abundant video resources are provided for the user, and the success rate of recommending the film and television resources is improved.
Description
Technical Field
The invention relates to the technical field of computer application, in particular to a resource searching method based on voice, an intelligent terminal and a storage medium.
Background
The existing operation method of intelligent voice control in the aspects of movies and television based on the Android platform mainly comprises the steps of converting input voice content into text content after voice recognition, requesting the text content to a cloud end, and judging the corresponding video information found by a resource library by the cloud end to return to a terminal. For example, the user inputs' Yanxi attack, the system inputs and recognizes the voice and converts the voice into characters, and after the video information corresponding to the character content is requested from the cloud, the returned result is displayed on the terminal in an interface form for the user to check.
At present, an intelligent platform device based on an Android system can install a video playing application in a system in advance, the above scenario is that a server for storing a video resource corresponding to a preinstalled video playing application of the system is queried, if a user installs a third-party video playing application, the server for storing the video resource corresponding to input voice content in the preinstalled video playing application of the system is not found, but the server for storing the video resource corresponding to the voice content in the installed third-party video playing application is provided, that is, an operation method of intelligent voice control in the aspect of the video is poor in compatibility, and the user is inconvenient to use.
Accordingly, the prior art is yet to be improved and developed.
Disclosure of Invention
The invention provides a resource searching method based on voice, an intelligent terminal and a storage medium, aiming at the defects of the prior art, and the resource searching method, the intelligent terminal and the storage medium combine the current video playing application and the third-party video playing application to search resources required by a user by identifying the voice of the user, provide abundant video resources for the user and improve the recommendation success rate of movie and television resources.
The technical scheme adopted by the invention for solving the technical problem is as follows:
a voice-based resource searching method, wherein the voice-based resource searching method comprises the following steps:
receiving voice input by a user, identifying voice content, and searching video information corresponding to the voice content in a video resource library corresponding to the current video playing application;
if the video information is not found and the system is monitored to have the third-party video playing application installed, analog search is carried out in the installed third-party video playing application according to the voice content so as to find the video information corresponding to the voice content.
The resource searching method based on voice, wherein if the third-party video playing application is not found and the system is monitored to be installed, the method further comprises the following steps of performing simulated search according to the voice content in the installed third-party video playing application to find the video information corresponding to the voice content:
if the video information is searched, displaying the video information in an interface mode;
and if the video information is not searched, searching in the next installed third-party video playing application until all the third-party video playing applications installed by the system are traversed.
The voice-based resource searching method includes receiving a voice input by a user, recognizing a voice content, and searching for video information corresponding to the voice content in a video resource library corresponding to a current video playing application, where the searching includes:
receiving voice input by a user, and converting the voice into text contents after voice recognition;
searching in a corresponding video resource library in the current video playing application according to the text content;
and if the corresponding video information is found in the movie and television resource library, displaying the video information to the user in the form of an interface.
The resource searching method based on voice, wherein if the resource searching method based on voice is not found and it is monitored that the system has installed the third-party video playing application, performing analog search according to the voice content in the installed third-party video playing application to find the video information corresponding to the voice content specifically includes:
if the resources required by the user are not found in the video resource library corresponding to the current video playing application according to the voice content, judging whether the system is provided with a third-party video playing application or not;
when the system is judged to be provided with the third-party video playing application, the third-party video playing application is analyzed, and a main interface of the third-party video playing application is recorded and stored;
and performing simulated click search in the third-party video playing application main interface to enter a search interface, and simulating search according to the text content in the search interface.
The voice-based resource searching method, wherein if the video information is searched, displaying the video information in an interface mode specifically comprises:
if the resources required by the user are found in the installed third-party video playing application through simulation search, searching the corresponding nodes of the search result according to the input text content on the search page of the third-party video playing application and clicking;
storing the video information of the search result, displaying the video information to a user in an interface mode and ending the current search process;
and when the video information clicked on the interface by the user is received, jumping to a playing interface of a third-party video playing application for playing through the prestored detailed information of the video.
The resource searching method based on the voice is characterized in that the simulation click search is carried out in the third-party video playing application main interface to enter the search interface, and the simulation search according to the text content in the search interface specifically comprises the following steps:
starting a main interface of a third-party video playing application;
traversing each node on the main interface according to the text content to be searched;
if the searched text content on the node is the text content to be searched, judging that the current node is the search button on the main interface;
acquiring the position of a current character node and sending a touch event click;
and entering a search interface to finish the action of simulating click search on the main interface.
The resource searching method based on voice, wherein if the resource required by the user is found through the simulated search in the installed third-party video playing application, the searching for the corresponding node of the search result according to the input text content on the search page of the third-party video playing application and clicking specifically comprises:
acquiring a third-party video playing application installed in the system, and performing simulation search in the third-party video playing application;
simulating click search on a main interface in the third-party video playing application to enter a search interface of the application;
converting voice content input by a user into the first letter of each Chinese character to obtain a search content first letter list, sequentially obtaining one first letter, searching for a matched key on a search keyboard and triggering and clicking until all letters are input in a simulated mode;
searching for the voice content input by the user according to the search interface, wherein the node character content is searched for by the search interface;
and acquiring the position of the text node, sending a touch event click, entering a detail interface for searching the video, acquiring the detail information of the video and storing the current interface.
The voice-based resource searching method, wherein the voice-based resource searching method further comprises:
simulating and starting a main interface of the third-party video playing application, entering a search interface of the third-party video playing application according to the search word, and entering a detail interface of the searched video according to a search result of the voice content input by the user.
An intelligent terminal, wherein the intelligent terminal comprises: a memory, a processor, and a speech-based resource search program stored on the memory and executable on the processor, the speech-based resource search program when executed by the processor implementing the steps of the speech-based resource search method as described above.
A storage medium, wherein the storage medium stores a voice-based resource search program, which when executed by a processor implements the steps of the voice-based resource search method as described above.
The invention discloses a resource searching method based on voice, an intelligent terminal and a storage medium, wherein the method comprises the following steps: receiving voice input by a user, identifying voice content, and searching video information corresponding to the voice content in a video resource library corresponding to the current video playing application; if the video information is not found and the system is monitored to have the third-party video playing application installed, performing analog search in the installed third-party video playing application according to the voice content to find the video information corresponding to the voice content; if the video information is searched, displaying the video information in an interface mode; and if the video information is not searched, searching in the next installed third-party video playing application until all the third-party video playing applications installed by the system are traversed. According to the invention, through recognizing the voice of the user, the resources required by the user are searched in the current video playing application and the third-party video playing application in a combined manner, so that abundant video resources are provided for the user, and the success rate of recommending the film and television resources is improved.
Drawings
FIG. 1 is a flow chart of a preferred embodiment of the voice-based resource searching method of the present invention;
FIG. 2 is a flowchart of step S10 in the preferred embodiment of the method for searching resources based on voice;
FIG. 3 is a flowchart of step S20 in the preferred embodiment of the method for searching resources based on voice;
FIG. 4 is a flowchart illustrating steps subsequent to step S20 in the preferred embodiment of the method for searching resources based on speech;
FIG. 5 is a schematic diagram illustrating a main interface of a third-party video playing application according to a preferred embodiment of the method for searching resources based on voice;
FIG. 6 is a schematic diagram of a search interface entered from FIG. 5 in a preferred embodiment of the speech-based resource search method of the present invention;
FIG. 7 is a diagram illustrating search results obtained from the search of FIG. 6 in accordance with a preferred embodiment of the speech-based resource search method of the present invention;
FIG. 8 is a flowchart of a preferred embodiment of the method for searching for a video in a third party video playback class application host interface;
FIG. 9 is a control flow diagram illustrating a complete video search by smart voice in accordance with a preferred embodiment of the present invention;
fig. 10 is a schematic operating environment diagram of an intelligent terminal according to a preferred embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer and clearer, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As shown in fig. 1, the resource searching method based on voice according to the preferred embodiment of the present invention includes the following steps:
step S10, receiving the voice input by the user, recognizing the voice content, and searching the video information corresponding to the voice content in the corresponding video resource library in the current video playing application.
Please refer to fig. 2, which is a flowchart of step S10 in the method for searching resources based on speech according to the present invention.
As shown in fig. 2, the step S10 includes:
s11, receiving voice input by a user, and converting the voice into text content after voice recognition;
s12, searching in a corresponding video resource library in the current video playing application according to the text content;
and S13, if the corresponding video information is found in the movie and television resource library, displaying the video information to the user in the form of an interface.
Specifically, the Android platform-based application installation modes can be divided into two types, one is interface installation, and the other is interface-free installation; the interface installation is an installation application interface which can be seen by user operation in actual use and popped up by the system, and the interface-free installation is that a developer installs the interface through a command. The invention mainly aims at installing third-party application in the using process of a user, and is the first installation mode. The interface installation mode is the interface of PackageInstalleraActivity in the Android native system, and the source code path is as follows:
packages/apps/PackageInstaller/src/com/android/packageinstaller/PackageInstallerActivity.java
the invention sets and monitors whether the user installs the third party application or not by modifying the PackageInstallerachoice and the onCreate function in the PackageInstallractity.
After receiving voice input by a user and recognizing voice content, searching in a corresponding video resource library in a current video playing application according to the voice content, wherein the current video playing application comprises a video playing application pre-installed in a system, that is, resources required by the user can be searched in the corresponding video resource library in the video playing application pre-installed in the system preferentially.
And step S20, if the video information is not found and the system is monitored to have the third-party video playing application installed, performing analog search in the installed third-party video playing application according to the voice content to find the video information corresponding to the voice content.
Please refer to fig. 3, which is a flowchart of step S20 in the method for searching resources based on speech according to the present invention.
As shown in fig. 3, the step S20 includes:
s21, if the resources needed by the user are not found in the movie and television resource library corresponding to the current video playing application according to the voice content, judging whether the system is provided with the third-party video playing application;
s22, when the system is judged to be provided with the third-party video playing application, the third-party video playing application is analyzed, and a main interface of the third-party video playing application is recorded and stored;
and S23, performing simulated click search in the third-party video playing application main interface to enter a search interface, and simulating the search according to the text content in the search interface.
That is to say, if the resources required by the user are not found in the video resource library corresponding to the current video playing application according to the voice content, the search may be continued in the installed third-party video playing application, and when the system installs a plurality of third-party video playing applications, the search sequence may be performed sequentially from high to low according to the frequency of using the applications by the user, for example, the a application pre-installed in the system is not found, and meanwhile, the system also installs the B application, the C application, and the D application (third-party video playing application), and then the search may be performed sequentially from high to low according to the frequency of using the user within a period of time by the B application, the C application, and the D application, so as to avoid confusion of the search.
Specifically, in the starting process of the Android system, an application management service (app management service) is started, and the service is responsible for scanning a specific directory in the system, finding out application files therein, that is, files with apk as suffix, analyzing the files to obtain related information of the application, and storing the related information in member variables of the files.
In the present invention, if the installation application is monitored, the analytic apk interface of the PackageManagerService is called to obtain and store each interface, i.e. Activity, of the installation application, wherein the path of the source code file of the PackageManagerService is as follows:
frameworks/base/services/java/com/android/server/PackageManagerService.java
as is well known, each Android application is launched by clicking an icon of the application on a main interface (i.e., Launcher), so that the application can be entered, the interface displayed first when the application is opened is the main interface of the application, and in the process of developing the application, the main interface is set to be a filter which is set for Activity in a configuration file Android manifest.
<intent-filter>
<action android:name="android.intent.action.MAIN" /><category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
Therefore, when the application is analyzed and installed, when an Activity node in a configuration file android manifest.
Each interface of the Android application corresponds to an Activity, the Activity is one of four most commonly used components (Activity, service, content providing and broadcasting receivers) which are the most basic in a system (such as an open source operating system such as an Android system), and in the system, the Activity is a component responsible for interaction with a user, provides a human-computer interaction interface for the user, and is widely applied to various applications; in the object-oriented programming language, a human-computer interaction interface provided by an Activity corresponds to an Activity instance, and if an installed application carries a playing interface, the playing interface corresponds to the Activity.
The application based on the Android system is to play the video by calling an Android player (MediaPlayer), and of course, the playing video is hosted in a playing interface, that is, there is a related call of a player (MediaPlayer) interface in the playing interface.
The method can obtain the smili code of the Android application program through the disassembling tool, and the apktool tool is adopted in the method and can disassemble apk to obtain resource files such as the smili code, the picture, the XML configuration file and the like of the application program. After the apk file is decompiled by using the apktool, a smali file is generated under a decompiled engineering directory, all the decompiled smali files are stored in the smali file, the files can generate corresponding directories according to the hierarchical structure of the program package, and all classes in the program can generate independent smali files under the corresponding directories. For convenience of description, the following are exemplified:
assuming that one Activity name is com.sunxz.test.mainActivlty, com \ sunxz \ test \ directory structures are sequentially generated under a smali directory, and then a mainActivlty.smali file is generated under the directory.
Traversing and analyzing a smali code file carrying an Activity name in a smali folder according to the obtained interface (i.e. Activity) name of the application, taking the above mainactivlty.
.class public Lcom/sunxz/test/MainActivlty;
.super Landroid/app/Activlty;
.source "MainActivlty.java"
#virutal methods
.method protected onCreate(Landroid/os/Bundle;)V
.locals 3
.parameter "savedInstanceState".prologue
.line 14
invoke-super{p0,p1},Landroid/app/Activity;->onCreate(Landroid/os/Bundle;)V
.line 15
const/high 16 v2, 0x7f03
invoke-virtual {p0,v2}, Lcom/sunxz/test/MainActivlty;->setContentView(I)V
Wherein the first row ". class" instruction specifies the class name of the current class; the second line ". super" instruction specifies the parent class of the current class; the third line ". source" instruction specifies the source file name of the current class; # virtual methods is a method declaration instruction, parameter is a parameter instruction, prologue is a code start instruction, and invoke-virtual is a method call instruction.
As described above, there is a related call of the player (MediaPlayer) interface in the play interface. Its smali pseudo code is as follows:
invoke-virtual {v0, v1},
Landroid/widget/VideoView;->setOnErrorListener(Landroid/media/MediaPlayer$OnErrorListener;) V
// call setOrorListener on v0 with parameter OnerrorListener stored in v1
Here, the playback monitor interface of the Android player (MediaPlayer) is called.
Therefore, if the fact that the smali code file carries the Activity name in the smali folder is checked to contain the Landoroid/media/MediaPlayer keywords when the smali code file carrying the Activity name in the smali folder is traversed and analyzed according to all interface (namely Activity) names of the installed application, the interface is judged to be the playing interface, and the application is confirmed to belong to the video playing application.
The step S20 is followed by the following steps: if the video information is searched, displaying the video information in an interface mode;
and if the video information is not searched, searching in the next installed third-party video playing application until all the third-party video playing applications installed by the system are traversed.
Referring to fig. 4, a specific process of displaying the video information in an interface manner if the video information is searched is shown as a flowchart of displaying the video information in the interface manner to the user in the voice-based resource searching method provided by the present invention.
As shown in fig. 4, the step of displaying the video information in an interface manner if the video information is searched includes:
s31, if the resources needed by the user are found in the installed third-party video playing application through simulation search, searching the corresponding nodes of the search result according to the input text content on the search page of the third-party video playing application and clicking;
s32, storing the video information of the search result, displaying the video information to the user in an interface mode and ending the current search process;
and S33, when the video information on the user click interface is received, jumping to the playing interface of the third-party video playing application for playing through the pre-stored detail information of the video.
Specifically, in general, a main interface of the video playing application has a search button, and a user searches for a video of interest through the search button. For convenience of description, taking a certain video application (a type of third-party video playing application) as an example, the main interface is shown in fig. 5, a search button can be seen, which is located at the upper left corner of the figure, and when a user wants to search for a video that is of interest to the user, the user can click the search button to enter the search interface. Continuing with the video application as an example, click the search button on the home page of fig. 5 to enter the search interface of the video application, as shown in fig. 6. The user displays the search result on the right side of fig. 6 by inputting the name of the video in which the user is interested. Assuming that the user wants to search for "Yanxi Gao", the initial letter "YXGL" is entered in the input box of FIG. 6 for each phonetic symbol of "Yanxi Gao", and if the corresponding video is searched, the search result is as shown in FIG. 7.
Furthermore, in the existing operation method of the Android platform-based intelligent voice control in the aspect of movies and television, the voice content is converted into text content after being subjected to voice recognition according to the input voice content, the text content requests for a corresponding movie and television resource repository in the current video playing application to search, and the resource repository returns corresponding video information to the user after finding the corresponding video information. For example, the user inputs' Yanxi attack, the system inputs and recognizes the voice and converts the voice into characters, and after the video information corresponding to the character content is requested from the cloud, the returned result is displayed on the terminal in an interface form for the user to check. Because the server of the current (including system preinstalled) video playing application request generally makes a request according to the cloud corresponding to the license plate side corresponding to the system, if the voice content input by the user is not requested at the cloud, but the video corresponding to the voice content is stored in the server of the movie and television resource corresponding to the installed third-party video playing application, in order to improve the success rate of resource search, what you see is what you get service is provided herein, and the server is used for performing simulated click search in the main interface of the third-party video playing application installed in the system to enter a search interface, and simulating search according to the voice content in the search interface. Simulating the above-described video search in the third-party video playing application, where the WYSIWYG control flow is shown in fig. 8, the steps are as follows:
step S101, inputting characters (the characters refer to the character content after the voice content is converted);
step S102, inheriting the service of the accessitiveService;
the accessibility service is Google, and provides barrier-free auxiliary service for users, the accessibility service runs in the background and can receive some events sent by the system, such as focus change, input content change, button click and the like, and the service can request to acquire the current active window and search the content in the window; that is, any change in the interface will generate an event and the system will notify the accessitivetService.
Step S103, character preprocessing;
step S104, sending a notice and triggering an onaccesssitivyEvent callback;
taking an Android platform as an example, for example, each application interface installed on a mobile phone of a user corresponds to an Activity in the Android platform, the actually written interface of a developer is actually inherited from the Activity, the Activity is a base class provided by the Android platform for the developer to write the interface, the base class has a plurality of functions, one of the functions is a start function, and the function is triggered when the interface is started; the interfaces developed by actual developers are different, and the developers inherit the Activity to rewrite the starting function when developing the interfaces of the developers. Respective initialization tasks are realized in the starting function, and the starting function is called back; meaning that when the interface is started, the function is triggered; the service obtained in the invention, namely the service inherited from the accessitivyservice (corresponding to the service in the Android platform), the accessitivyservice also has a plurality of functions, the functions comprise onassitiveevent functions, the triggering (called back) of the functions is to receive certain system notification, and the system notification can be packaged and sent through the Android native interface.
S105, traversing each node of the interface;
step S106, judging whether the node has characters, if so, executing step 107, otherwise, executing step 108;
step S107, matching the node characters with the input content, and executing step S109;
step S108, judging whether the node traversal is finished, if so, executing step S111, otherwise, returning to execute step S105;
step S109, judging whether the node characters are successfully matched with the input content, if so, executing step S110, otherwise, executing step S111;
step S110, acquiring the position of a character node and sending touch event click;
and step S111, ending.
Further, as described above, if it is detected that the application installed in the system is a third-party video playing application and the main interface thereof is acquired, the step of simulating click search in the main interface is as follows, and for convenience of description, the following description is continued by taking the video application in fig. 5 as an example:
s1, starting a main interface of the application, wherein the main interface can be acquired by the method;
s2, traversing each node on the main interface by means of the WYSIWYG service according to the text content of the search;
s3, if the word content on the searched node is 'search', judging that the node is a search button on the main interface;
s4, executing the step S110 in the control flow, namely acquiring the position of the character node and sending touch event click;
and S5, entering a search interface at the moment, and finishing the action of simulating click search on the main interface.
Further, as described above, if the voice content input by the user is not requested by the server requested by the current video playing application (which is explained below by using a video playing application pre-installed in the system), but the video corresponding to the voice content is stored by the server corresponding to the video resource in the third-party video playing application installed by the user, for such a scenario, the control steps of the intelligent voice on the video resource are as follows, and the flowchart can refer to fig. 9, and the steps are as follows:
step S201, inputting voice content;
step S202, recognizing voice content;
step S203, inputting text contents corresponding to the voice contents, such as inputting a Yanxi strategy;
step S204, searching in a video playing application pre-installed in the system;
step S205, judging whether video resources required by the user are searched in the video playing application pre-installed in the system, if so, executing step S211, otherwise, executing step S206;
step S206, traversing all video application lists in the system, namely traversing all third-party video playing applications installed in the system;
step S207, acquiring one of the video type lists (all third-party video playing type applications installed in the system) and simulating and searching in the video type list;
the method comprises the steps of inputting a search interface of an application according to a click simulation search of a main interface in a video playing application by the method, generally, inputting the search interface, enabling a focus to be located on an alphanumeric disc, converting voice content input by a user into the initial of each Chinese character to obtain a search content initial list, sequentially obtaining one initial by means of a service which is obtained by what you see, searching for a matched key on a search keyboard according to the service which is obtained by what you see, and triggering the click until all letters are input in a simulation mode.
Step S208, according to the service of the WYSIWYG, the text content of the searching node of the interface is the voice content input by the user, if the user inputs 'Yangxi Gao', the operation of the WYSIWYG is executed on the interface, the input content is 'Yangxi Gao', if the input content is matched, the video is searched, the step S210 is executed, otherwise, the step S209 is skipped;
step S209, judging whether all the video application lists are traversed completely, if so, executing step S212, otherwise, returning to execute step S207;
step S210, exiting traversal;
s211, acquiring a touch event click sent by the position of the character node, entering a detail interface for searching the video, acquiring the detail information of the video and storing the interface; acquiring videos corresponding to voice contents input by a user, which are searched by a server side not searching for a video playing application pre-installed in a system but stored in corresponding video resources in a third-party video playing application installed by the user; finally, the returned result is displayed on the terminal in an interface form for the user to check;
and step S212, ending.
In particular, in the present invention, the third-party video playing application started according to the WYSIWYG service simulation is started, the search interface of the application is entered according to the word of "search", and the detailed interface of the searched video is entered according to the search result of the voice content input by the user, which are all performed in a non-interface manner.
The method comprises the steps of firstly, increasing monitoring whether the third-party video playing application is installed or not in the system, analyzing and installing the application to record the main interface and storing the main interface when the third-party video playing application is monitored, secondly, increasing the WYSIWYG service in the system, performing simulated click search in the main interface of the third-party video playing application installed in the system to enter a search interface, and simulating search according to voice content in the search interface.
In actual use, when a user inputs voice content, the voice content is firstly searched by a server side of corresponding video resource storage in a video playing application pre-installed in the system, if the third-party video playing application is not found, judging whether the system is provided with the third-party video playing application, if the third-party video playing application is provided, simulating search in the provided third-party video playing application according to the voice content, if the search is finished, searching a search result corresponding node according to the input voice content through a search page of the WYSIWYG service in the video playing application, clicking, storing the video information of the search result, finally displaying the video information to the user in an interface mode and finishing the current search process, and if not, searching in the next installed third-party video playing application until all the video playing applications installed by the system are traversed.
The method comprises the steps of simulating search and searching a search result through a WYSIWYG service and clicking, entering a detail interface (such as a detail page of the Yanxi strategy) of a searched video, acquiring and storing detail information of the searched video, wherein the interface displayed to a user is a result searched by a voice control application, is arranged in an interface of an application of the user, is displayed to the user in an interface mode, and is played by jumping to a playing interface of the application (a third-party video playing type application) through the previously stored detail information of the video when the user clicks the video on the interface.
Further, as shown in fig. 10, based on the above-mentioned voice-based resource searching method, the present invention also provides an intelligent terminal (e.g. an intelligent television) correspondingly, where the intelligent terminal includes a processor 10, a memory 20 and a display 30. Fig. 10 shows only some of the components of the smart terminal, but it is to be understood that not all of the shown components are required to be implemented, and that more or fewer components may be implemented instead.
The memory 20 may be an internal storage unit of the intelligent terminal in some embodiments, such as a hard disk or a memory of the intelligent terminal. The memory 20 may also be an external storage device of the Smart terminal in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the Smart terminal. Further, the memory 20 may also include both an internal storage unit and an external storage device of the smart terminal. The memory 20 is used for storing application software installed in the intelligent terminal and various data, such as program codes of the installed intelligent terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, the memory 20 stores a speech-based resource search program 40, and the speech-based resource search program 40 can be executed by the processor 10 to implement the speech-based resource search method of the present application.
The processor 10 may be, in some embodiments, a Central Processing Unit (CPU), a microprocessor or other data Processing chip, and is configured to execute program codes stored in the memory 20 or process data, such as executing the voice-based resource searching method.
The display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch panel, or the like in some embodiments. The display 30 is used for displaying information at the intelligent terminal and for displaying a visual user interface. The components 10-30 of the intelligent terminal communicate with each other via a system bus.
In one embodiment, the following steps are implemented when the processor 10 executes the speech-based resource search program 40 in the memory 20:
receiving voice input by a user, identifying voice content, and searching video information corresponding to the voice content in a video resource library corresponding to the current video playing application;
if the video information is not found and the system is monitored to have the third-party video playing application installed, performing analog search in the installed third-party video playing application according to the voice content to find the video information corresponding to the voice content;
if the video information is searched, displaying the video information in an interface mode;
and if the video information is not searched, searching in the next installed third-party video playing application until all the third-party video playing applications installed by the system are traversed.
The receiving of the voice input by the user and the recognition of the voice content, and the searching of the video information corresponding to the voice content in the video resource library corresponding to the current video playing application specifically includes:
receiving voice input by a user, and converting the voice into text contents after voice recognition;
searching in a corresponding video resource library in the current video playing application according to the text content;
and if the corresponding video information is found in the movie and television resource library, displaying the video information to the user in the form of an interface.
If the video information corresponding to the voice content is not found and the system is monitored to have the third-party video playing application installed, performing analog search in the installed third-party video playing application according to the voice content to find the video information corresponding to the voice content specifically includes:
if the resources required by the user are not found in the video resource library corresponding to the current video playing application according to the voice content, judging whether the system is provided with a third-party video playing application or not;
when the system is judged to be provided with the third-party video playing application, the third-party video playing application is analyzed, and a main interface of the third-party video playing application is recorded and stored;
and performing simulated click search in the third-party video playing application main interface to enter a search interface, and simulating search according to the text content in the search interface.
If the video information is searched, displaying the video information in an interface mode specifically comprises the following steps:
if the resources required by the user are found in the installed third-party video playing application through simulation search, searching the corresponding nodes of the search result according to the input text content on the search page of the third-party video playing application and clicking;
storing the video information of the search result, displaying the video information to a user in an interface mode and ending the current search process;
and when the video information clicked on the interface by the user is received, jumping to a playing interface of a third-party video playing application for playing through the prestored detailed information of the video.
The simulation click search in the third-party video playing application main interface and the simulation search in the search interface according to the text content specifically comprise:
starting a main interface of a third-party video playing application;
traversing each node on the main interface according to the text content to be searched;
if the searched text content on the node is the text content to be searched, judging that the current node is the search button on the main interface;
acquiring the position of a current character node and sending a touch event click;
and entering a search interface to finish the action of simulating click search on the main interface.
If the resources required by the user are found through simulation search in the installed third-party video playing application, finding the corresponding nodes of the search result according to the input text content on the search page of the third-party video playing application and clicking specifically comprises the following steps:
acquiring a third-party video playing application installed in the system, and performing simulation search in the third-party video playing application;
simulating click search on a main interface in the third-party video playing application to enter a search interface of the application;
converting voice content input by a user into the first letter of each Chinese character to obtain a search content first letter list, sequentially obtaining one first letter, searching for a matched key on a search keyboard and triggering and clicking until all letters are input in a simulated mode;
searching for the voice content input by the user according to the search interface, wherein the node character content is searched for by the search interface;
and acquiring the position of the text node, sending a touch event click, entering a detail interface for searching the video, acquiring the detail information of the video and storing the current interface.
The operation of simulating and starting the main interface of the third-party video playing application, entering the search interface of the third-party video playing application according to the search word and entering the detailed interface of the searched video according to the search result of the voice content input by the user is carried out in an interface-free mode.
The present invention also provides a storage medium, wherein the storage medium stores a voice-based resource search program, and the voice-based resource search program implements the steps of the voice-based resource search method as described above when executed by a processor.
In summary, the present invention provides a resource search method based on voice, an intelligent terminal and a storage medium, wherein the method includes: receiving voice input by a user, identifying voice content, and searching video information corresponding to the voice content in a video resource library corresponding to the current video playing application; if the video information is not found and the system is monitored to have the third-party video playing application installed, performing analog search in the installed third-party video playing application according to the voice content to find the video information corresponding to the voice content; if the video information is searched, displaying the video information in an interface mode; and if the video information is not searched, searching in the next installed third-party video playing application until all the third-party video playing applications installed by the system are traversed. According to the invention, through recognizing the voice of the user, the resources required by the user are searched in the current video playing application and the third-party video playing application in a combined manner, so that abundant video resources are provided for the user, and the success rate of recommending the film and television resources is improved.
Of course, it will be understood by those skilled in the art that all or part of the processes of the methods of the above embodiments may be implemented by a computer program instructing relevant hardware (such as a processor, a controller, etc.), and the program may be stored in a computer readable storage medium, and when executed, the program may include the processes of the above method embodiments. The storage medium may be a memory, a magnetic disk, an optical disk, etc.
It is to be understood that the invention is not limited to the examples described above, but that modifications and variations may be effected thereto by those of ordinary skill in the art in light of the foregoing description, and that all such modifications and variations are intended to be within the scope of the invention as defined by the appended claims.
Claims (10)
1. A resource search method based on voice is characterized in that the resource search method based on voice comprises the following steps:
receiving voice input by a user, identifying voice content, and searching video information corresponding to the voice content in a video resource library corresponding to the current video playing application;
if the video information is not found and the system is monitored to have the third-party video playing application installed, analog search is carried out in the installed third-party video playing application according to the voice content so as to find the video information corresponding to the voice content.
2. The method according to claim 1, wherein if the third-party video playing application is not found and it is monitored that the system has installed the third-party video playing application, performing an analog search according to the voice content in the installed third-party video playing application to find the video information corresponding to the voice content further comprises:
if the video information is searched, displaying the video information in an interface mode;
and if the video information is not searched, searching in the next installed third-party video playing application until all the third-party video playing applications installed by the system are traversed.
3. The method of claim 2, wherein the receiving a voice input by a user and recognizing a voice content, and searching for video information corresponding to the voice content in a video resource library corresponding to a current video playing application specifically includes:
receiving voice input by a user, and converting the voice into text contents after voice recognition;
searching in a corresponding video resource library in the current video playing application according to the text content;
and if the corresponding video information is found in the movie and television resource library, displaying the video information to the user in the form of an interface.
4. The resource search method based on voice according to claim 3, wherein if the resource search method is not found and it is monitored that the system has installed the third-party video playing application, performing the analog search according to the voice content in the installed third-party video playing application to find the video information corresponding to the voice content specifically includes:
if the resources required by the user are not found in the video resource library corresponding to the current video playing application according to the voice content, judging whether the system is provided with a third-party video playing application or not;
when the system is judged to be provided with the third-party video playing application, the third-party video playing application is analyzed, and a main interface of the third-party video playing application is recorded and stored;
and performing simulated click search in the third-party video playing application main interface to enter a search interface, and simulating search according to the text content in the search interface.
5. The method according to claim 4, wherein the displaying the video information in an interface manner if the video information is searched comprises:
if the resources required by the user are found in the installed third-party video playing application through simulation search, searching the corresponding nodes of the search result according to the input text content on the search page of the third-party video playing application and clicking;
storing the video information of the search result, displaying the video information to a user in an interface mode and ending the current search process;
and when the video information clicked on the interface by the user is received, jumping to a playing interface of a third-party video playing application for playing through the prestored detailed information of the video.
6. The resource searching method based on the voice of claim 4, wherein the simulating click search in the third-party video playing application main interface enters a search interface, and the simulating search according to the text content in the search interface specifically comprises:
starting a main interface of a third-party video playing application;
traversing each node on the main interface according to the text content to be searched;
if the searched text content on the node is the text content to be searched, judging that the current node is the search button on the main interface;
acquiring the position of a current character node and sending a touch event click;
and entering a search interface to finish the action of simulating click search on the main interface.
7. The resource searching method based on voice of claim 5, wherein if the resource needed by the user is found through the simulated search in the installed third-party video playing application, finding the node corresponding to the search result according to the input text content on the search page of the third-party video playing application and clicking specifically comprises:
acquiring a third-party video playing application installed in the system, and performing simulation search in the third-party video playing application;
simulating click search on a main interface in the third-party video playing application to enter a search interface of the application;
converting voice content input by a user into the first letter of each Chinese character to obtain a search content first letter list, sequentially obtaining one first letter, searching for a matched key on a search keyboard and triggering and clicking until all letters are input in a simulated mode;
searching for the voice content input by the user according to the search interface, wherein the node character content is searched for by the search interface;
and acquiring the position of the text node, sending a touch event click, entering a detail interface for searching the video, acquiring the detail information of the video and storing the current interface.
8. The speech-based resource search method of claim 7, further comprising:
simulating and starting a main interface of the third-party video playing application, entering a search interface of the third-party video playing application according to the search word, and entering a detail interface of the searched video according to a search result of the voice content input by the user.
9. An intelligent terminal, characterized in that, intelligent terminal includes: a memory, a processor, and a speech-based resource search program stored on the memory and executable on the processor, the speech-based resource search program when executed by the processor implementing the steps of the speech-based resource search method according to any one of claims 1-8.
10. A storage medium storing a voice-based resource search program, the voice-based resource search program when executed by a processor implementing the steps of the voice-based resource search method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811609705.8A CN111385633B (en) | 2018-12-27 | 2018-12-27 | Resource searching method based on voice, intelligent terminal and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811609705.8A CN111385633B (en) | 2018-12-27 | 2018-12-27 | Resource searching method based on voice, intelligent terminal and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111385633A true CN111385633A (en) | 2020-07-07 |
| CN111385633B CN111385633B (en) | 2022-04-01 |
Family
ID=71216234
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811609705.8A Active CN111385633B (en) | 2018-12-27 | 2018-12-27 | Resource searching method based on voice, intelligent terminal and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111385633B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111968640A (en) * | 2020-08-17 | 2020-11-20 | 北京小米松果电子有限公司 | Voice control method and device, electronic equipment and storage medium |
| CN112423029A (en) * | 2020-10-28 | 2021-02-26 | 深圳Tcl新技术有限公司 | Program playing method, display device and computer readable storage medium |
| CN113207042A (en) * | 2021-04-30 | 2021-08-03 | 海信视像科技股份有限公司 | Media asset playing method and display equipment |
| CN113301395A (en) * | 2021-04-30 | 2021-08-24 | 当趣网络科技(杭州)有限公司 | Voice searching method combining user grades in video playing state |
| CN115456456A (en) * | 2022-09-29 | 2022-12-09 | 广州宏途数字科技有限公司 | Campus resource sharing system based on internet big data |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140088961A1 (en) * | 2012-09-26 | 2014-03-27 | International Business Machines Corporation | Captioning Using Socially Derived Acoustic Profiles |
| CN107371064A (en) * | 2017-07-03 | 2017-11-21 | 奇酷互联网络科技（深圳）有限公司 | Mobile terminal and its audio and video playing method and apparatus |
| CN109036417A (en) * | 2018-07-03 | 2018-12-18 | 百度在线网络技术（北京）有限公司 | Method and apparatus for handling voice request |
-
2018
- 2018-12-27 CN CN201811609705.8A patent/CN111385633B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140088961A1 (en) * | 2012-09-26 | 2014-03-27 | International Business Machines Corporation | Captioning Using Socially Derived Acoustic Profiles |
| CN107371064A (en) * | 2017-07-03 | 2017-11-21 | 奇酷互联网络科技（深圳）有限公司 | Mobile terminal and its audio and video playing method and apparatus |
| CN109036417A (en) * | 2018-07-03 | 2018-12-18 | 百度在线网络技术（北京）有限公司 | Method and apparatus for handling voice request |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111968640A (en) * | 2020-08-17 | 2020-11-20 | 北京小米松果电子有限公司 | Voice control method and device, electronic equipment and storage medium |
| CN112423029A (en) * | 2020-10-28 | 2021-02-26 | 深圳Tcl新技术有限公司 | Program playing method, display device and computer readable storage medium |
| CN113207042A (en) * | 2021-04-30 | 2021-08-03 | 海信视像科技股份有限公司 | Media asset playing method and display equipment |
| CN113301395A (en) * | 2021-04-30 | 2021-08-24 | 当趣网络科技(杭州)有限公司 | Voice searching method combining user grades in video playing state |
| CN115456456A (en) * | 2022-09-29 | 2022-12-09 | 广州宏途数字科技有限公司 | Campus resource sharing system based on internet big data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111385633B (en) | 2022-04-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111385633B (en) | Resource searching method based on voice, intelligent terminal and storage medium | |
| EP3605324B1 (en) | Application development method and tool, and storage medium thereof | |
| JP6195182B2 (en) | Application scenario identification method, power consumption management method, apparatus, and terminal device | |
| US9588752B2 (en) | Performing unattended software installation | |
| US20170289338A1 (en) | Enabling stateful dynamic links in mobile applications | |
| CN109074278B (en) | Validating stateful dynamic links in mobile applications | |
| WO2021017735A1 (en) | Smart contract formal verification method, electronic apparatus and storage medium | |
| US9524279B2 (en) | Help document animated visualization | |
| WO2013138919A1 (en) | Systems and methods for building a universal intelligent assistant with learning capabilities | |
| CN110263546B (en) | Method, device and equipment for carrying out security check on container mirror image | |
| CN111198730A (en) | Method, device, terminal and computer storage medium for starting sub-application program | |
| CN112149109A (en) | Modularized authority control management method and system | |
| KR20060047998A (en) | Method and system for embedding context information in a document | |
| CN113590327A (en) | Project resource detection method and device, electronic equipment and storage medium | |
| CN113132805B (en) | Playing control method, system, intelligent terminal and storage medium | |
| CN102855129A (en) | Method and system for automatically creating isolated processes | |
| CN113760763A (en) | Software testing method, device, server and system | |
| CN111385661B (en) | Method, device, terminal and storage medium for voice control of full screen playing | |
| WO2004029802A1 (en) | Method and computer system for dynamic data type enrichment | |
| CN108595656B (en) | Data processing method and system | |
| CN110825370B (en) | Mobile terminal application development method, device and system | |
| KR20080027293A (en) | Managing multiple languages in a data language | |
| CN112148450A (en) | Data processing method, device, equipment and storage medium | |
| Hoang | Finnish scanner mobile application | |
| CN117742836A (en) | Page processing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 516006 TCL science and technology building, No. 17, Huifeng Third Road, Zhongkai high tech Zone, Huizhou City, Guangdong Province Applicant after: TCL Technology Group Co.,Ltd. Address before: 516006 Guangdong province Huizhou Zhongkai hi tech Development Zone No. nineteen District Applicant before: TCL Corp. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |