Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
The following describes a program reminding method, device and system based on artificial intelligence according to an embodiment of the present invention with reference to the accompanying drawings.
Fig. 1 is a flowchart of a program reminding method based on artificial intelligence according to an embodiment of the invention. It should be noted that the program reminding method based on artificial intelligence in the embodiment of the present invention can be applied to the program reminding device based on artificial intelligence in the embodiment of the present invention. The program reminding device can be configured on a program playing terminal, and the program playing terminal can be an intelligent television or a digital television.
As shown in fig. 1, the artificial intelligence based program reminding method may include:
s110, acquiring voice information of the user, and identifying the voice information to generate corresponding text information.
For example, it is assumed that the program reminding method based on artificial intelligence according to the embodiment of the present invention can be applied to a program playing terminal, the program playing terminal can be included in a program reminding system based on artificial intelligence, the program reminding system can further include a remote controller corresponding to the program playing terminal, and the remote controller has a voice acquisition module for acquiring voice information input by a user and sending the voice information to the program playing terminal, so that the program playing terminal acquires the voice information of the user. In the embodiment of the present invention, the communication connection between the remote controller and the program playing terminal may be a bluetooth connection, a WiFi connection, or the like.
After the voice information of the user is acquired, the voice information can be recognized through a voice recognition technology, and corresponding text information is obtained. In this step, the process of recognizing the voice information by the voice recognition technology may be as follows: for example, as shown in fig. 6(a), firstly, feature extraction may be performed on the speech information, and then, the extracted audio features may be decoded by a specific decoder, so as to finally obtain a recognition result, where the recognition result is text information corresponding to the speech information. Wherein, in the decoding process of the decoder, an acoustic model, a language model and a pronunciation dictionary are used. The acoustic model is mainly used for converting audio features into syllables, the language model is used for converting syllables into texts, and the pronunciation dictionary provides a mapping table from syllables to texts. With X representing the input audio signal and W representing the text sequence, the speech recognition process solves the following problem:
after the conversion of the above formula (1), it is equivalent to the following formula (2):
the process of speech recognition is carried out in such a way that,in the above equation (2), the maximum P (X | W) and P (W) are obtained, the maximum P (X | W) can be obtained by an acoustic model, and the maximum P (W) can be obtained by a language model. Further, the above W*To finally identify the resulting text sequence.
And S120, analyzing the text information to generate user intention and program information aiming at the target program, wherein the program information comprises a program name and a viewing time.
Specifically, the text information may be parsed by a natural language processing technique to obtain a user intention and program information for the target program, the program information including at least a program name and a viewing time. The viewing time may be understood as a time that the user wants to view the target program, which is expressed in the user voice information, for example, the user voice information is: reminding me to watch the happy big book camp on the weekend, wherein the time that the user wants to watch the target program (namely the happy big book camp) is expressed in the voice information as the 'weekend', namely the watching time. The target program may be understood as a program that is ultimately reserved for the user.
It can be understood that natural language Query understanding is a key entry of intelligent conversational intelligence products; the method can uniformly express the output of the semantic understanding of the Query into three parts, namely Domain, Intent and Slot, wherein the Domain expresses the field of the Query, the Intent expresses the user intention of the current Query, and the Slot is a condition meeting the user intention, wherein the condition meeting the user intention can comprise the program name and the viewing time.
More specifically, the text information may be parsed by a natural language processing technique to obtain corresponding structured data, which may represent: "user intention + slot position structured data", in the present invention, the slot position may include two attributes of viewing time and program name.
Specifically, in an embodiment of the present invention, the specific implementation process of parsing the text information to generate the user intention and the program information for the target program may be as follows: the method comprises the steps of carrying out syntactic structure analysis on text information, carrying out semantic analysis based on words, field multi-classification identification based on topic models and intention multi-classification identification based on intention models on the text information subjected to the syntactic structure analysis to obtain user intentions aiming at a target program, and analyzing the text information subjected to the syntactic structure analysis based on a sequence labeling model to obtain a program name and watching time aiming at the target program.
In the embodiment of the present invention, the topic model, the intention model, and the sequence labeling model may be obtained by training in advance. For example, as shown in fig. 6(b), DNN machine learning technology may be adopted to combine the user sample with the universal big data features in combination with the internet big data, so as to train out a topic model, an intention model, and a sequence annotation model. In this step, the user intention may be obtained by transmitting the text information to the natural language processing service to perform syntactic structure analysis on the text information, respectively, and performing word-based semantic analysis, topic-model-based domain multi-classification recognition, intention multi-classification recognition based on the intention model on the text information subjected to the syntactic structure analysis, and obtaining a program name and viewing time for the target program based on the intention multi-classification recognition of the intention model.
For example, taking text information as "remind me to watch happy book of the weekend" as an example, the text information may be analyzed by a natural language processing technology to obtain corresponding structured data, that is: { Intent: reminder, { slot: time, value: this weekend }, { slot: program, value: happy big book camp } }.
Therefore, the text information is analyzed through the natural language processing technology, and the method has the following technical characteristics: the method comprises the steps of automatically learning and generalizing according to user requirements, adopting a DNN learning model with the strongest generalization capability at present, providing a small amount (such as 1k) of labeled data according to no application requirements, and introducing the universal characteristic of internet big data to enable the learning effect of small data to be better.
And S130, generating reservation reminding information aiming at the target program according to the program name and the watching time.
In an embodiment of the present invention, the playing date of the target program may be determined according to the viewing time, then the search may be performed in the video resource system according to the program name and the playing date, the playing channel information, the playing address information, and the starting time of the target program are matched, and finally, the reservation reminding information for the target program may be generated according to the playing channel information, the playing address information, and the starting time. Wherein, the playing date can be understood as the specific playing time of the target program.
For example, taking text information "remind me to watch happy book of this weekend" as an example, the name of the program is "happy book of this weekend", and the watching time is "this weekend", so that the playing date of the target program can be determined according to the watching time, for example, 1 month and 14 days in 2017, and then the search can be performed in the video resource system according to the name and the playing date of the program, and a unique result is matched: and 8, broadcasting the happy book at night by the Hunan satellite television, and establishing reservation reminding information aiming at the target program (namely the happy book at the festival) according to the broadcasting time point (8 points at 14 evening in 1 month and 14 days in 2017) of the matching result, broadcasting channel information (such as the Hunan satellite television) and broadcasting address information.
And S140, establishing a reminding timer according to the user intention and the watching time, and starting the reminding timer.
In one embodiment of the invention, it may be determined whether the user intent includes time information; if the user intention contains time information, a reminding timer is established according to the time information and the watching time; if the user intention does not contain time information, a reminding timer is established according to default time information and watching time, and the reminding timer is started when the reminding timer is established.
In the embodiment of the present invention, the user intention includes time information, which may be understood as how long the user has set a time to remind himself in advance according to the user's own needs, for example, a voice message "remind me 30 seconds before the business of happy big book is started", which may be analyzed by a natural language processing technique to obtain the user intention "remind me 30 seconds in advance", and the user intention includes time information, and a reminding timer may be established according to the time information and the viewing time. For another example, the voice message "remind me when running a happy big book", may be analyzed by a natural language processing technology to obtain the user intention "remind when running", and the user intention does not include time information, and a reminding timer may be established according to default time information and viewing time, where the default time information may be 1 minute, 30 seconds, and the like.
Therefore, the reminding reserved time can be obtained by analyzing the voice of the user, the function of customizing the reminding reserved time is realized, the personalized requirements of the user are met, and the user experience is improved.
And S150, when the reminding timer is finished, pushing the appointment reminding information to the user.
As an example, when the reminder timer is finished, the scheduled reminder information may be pushed to the user in different pushing manners according to different states of the program playing terminal. Specifically, in an embodiment of the present invention, it may be determined whether the program playing terminal is in an open state; if the program playing terminal is in the open state, skipping to a playing channel corresponding to the target program according to the playing channel information and the playing address information in the reservation reminding information; and if the program playing terminal is not in the open state, sending the reservation reminding information to the mobile terminal of the user. The program playing terminal may be a smart television or a digital television.
More specifically, after the reminding timer is started, the automatic operation of the background can be started, and when the reminding timer is finished, the appointment reminding information can be pushed to the user: when the program playing terminal is in an open state, the user can jump to a playing channel corresponding to the target program according to the playing channel information and the playing address information in the reservation reminding information, and simultaneously can pop up a prompt that 'you reserve a happy big book of Hunan-nan satellite television and can immediately jump to Hunan-satellite television'; when the program playing terminal is not in the open state, the reservation reminding information can be sent to the mobile terminal of the user so as to remind the user to watch happy book management. The method for sending the appointment reminding information to the mobile terminal of the user is not particularly limited, and for example, the appointment reminding information can be sent to the mobile terminal of the user in a short message mode, and can also be sent to the mobile terminal in a notification message pushing mode.
The program reminding method based on artificial intelligence of the embodiment of the invention generates corresponding text information by identifying the voice information of a user, analyzes the text information to generate user intention and program information aiming at a target program, generates reservation reminding information aiming at the target program according to the program name and the viewing time, establishes a reminding timer according to the user intention and the viewing time, starts the reminding timer, and pushes the reservation reminding information to the user when the reminding timer is finished. Realize the live broadcast reservation and the warning of on-demand resource update of TV end promptly through voice command, overturned traditional key remote controller input mode, directly express the reservation through pronunciation more effectively and remind the demand, it is simple more convenient during the operation, the practicality is stronger, the reserve time that simultaneously can self-defined warning, according to the custom of different crowds come in good time propelling movement to remind the notice, satisfy user's individualized demand, promoted user experience.
In order to further improve the user experience and facilitate the user to know whether the program reminder is successfully reserved, in an embodiment of the present invention, as shown in fig. 2, on the basis of fig. 1, while the reminder timer is started, the program reminding method based on artificial intelligence may further include:
and S210, generating prompt information and feeding the prompt information back to the user.
That is, while the reminder timer is started, a reminder message may be returned to the user, e.g., "good, i will remind you to watch happy book camp".
As an example, it is assumed that the program reminding method based on artificial intelligence according to the embodiment of the present invention is applied to a television, and the television can provide a human-computer interaction interface for a user. The man-machine interaction interface can display the recognized text information and can also display the prompt information fed back to the user. For example, as shown in fig. 3(a), taking a target program as a live program as an example, a text message "remind me 30 seconds before marketing and broadcasting for happy book" corresponding to a voice input by a user may be displayed in a human-computer interaction interface of the television end, while a reminder timer is started, a prompt message may be returned to the user and displayed in the human-computer interaction interface, for example, "good, jump to the south of the lake for you 30 seconds before broadcasting"; as shown in fig. 3(b), taking the target program as the on-demand program as an example, the man-machine interaction interface may display text information "notify me to watch when the second season of the western world is updated" corresponding to the voice input by the user, when the reminder timer is started, a prompt message may be returned to the user, and the prompt message is displayed in the man-machine interaction interface, and if "good", the episode will notify you at the first time when updated.
It can be understood that the way of feeding back the prompt information to the user is not limited to the text form, and the prompt information can be fed back to the user in a voice playing way, or the prompt information can be fed back to the user in a voice + text way, etc.
In order to improve the user experience and improve the accuracy of speech recognition, further, in an embodiment of the present invention, as shown in fig. 4, on the basis of fig. 2, before recognizing the speech information, the artificial intelligence based program reminding method may further include:
s410, determining surrounding scenes of the user, and determining a corresponding noise processing algorithm according to the surrounding scenes.
And S420, carrying out noise processing on the voice information according to the corresponding noise processing algorithm.
It can be understood that in the audio data acquisition process, the sound effect may vary due to differences in the performance of the device, the distance from the sound source to the device, and whether the device supports a single microphone or an array of microphones. Generally speaking, the higher the performance of the recording device, the shorter the distance from the sound source to the device, and the effective microphone array rather than a single microphone, the audio data with more complete and more favorable identification characteristics can be obtained. For example, to support far-field (e.g., greater than 5 meters) wake-up or identification, the performance of using a microphone array may be significantly better than single-microphone performance.
For this reason, the acquired voice information may have some problems and cannot be directly used for recognition. For example, in a scenario of hands-free or conference application, sound of a speaker may be fed back to a microphone many times, and at this time, Acoustic Echo may exist in voice information acquired by the microphone, and an Echo Cancellation (Acoustic Echo Cancellation, abbreviated as AEC in english) algorithm needs to be used for Echo Cancellation; for another example, the voice information collected in a specific environment (e.g. in a running vehicle) has a specific Noise, and then a Noise reduction (Noise Suppression, abbreviated as NS) algorithm needs to be performed on the voice information to eliminate the environmental Noise.
Therefore, before the voice information is identified, the surrounding scene of the user can be determined, the corresponding noise processing algorithm can be determined according to the surrounding scene, and the voice information can be subjected to noise processing according to the corresponding noise processing algorithm. Therefore, under different scene environments, the corresponding noise processing algorithm is used for carrying out noise processing on the voice information, and the accuracy of voice recognition can be greatly improved.
In summary, when a user wants to know the latest program or episode update information, only a voice input is performed through a voice acquisition module (e.g., a remote controller that can be disposed in a mobile phone or a television), for example, the voice input "remind me 30 seconds before the happy book is started", as shown in fig. 5, after receiving the voice information sent by the voice acquisition module, voice recognition and natural language processing may be sequentially performed on the voice information to analyze the user intention, the program name and the viewing time for a target program, then an index is established in a video resource system, the starting time, the playing channel information and the playing address information of the target program are matched, a reminding timer is established, the reminding timer is started, and when the reminding timer is finished, an accurate result of the search is pushed to the user. From this, realize the live broadcast reservation and the warning of broadcasting the resource update of TV end through voice command, the innovation point lies in subverting traditional button remote controller input mode, directly expresses the reservation through pronunciation more effectively and reminds the demand, and is simpler convenient during the operation, and the practicality is stronger. Meanwhile, the reminding reservation time can be customized, and the reminding notice can be pushed in due time according to habits of different people.
Corresponding to the program reminding methods based on artificial intelligence provided in the foregoing embodiments, an embodiment of the present invention further provides a program reminding device based on artificial intelligence, and since the program reminding device based on artificial intelligence provided in the embodiment of the present invention corresponds to the program reminding methods based on artificial intelligence provided in the foregoing embodiments, the implementation manner of the program reminding method based on artificial intelligence is also applicable to the program reminding device based on artificial intelligence provided in the embodiment, and is not described in detail in the embodiment. Fig. 7 is a schematic structural diagram of an artificial intelligence-based program reminder according to an embodiment of the present invention. It should be noted that the program reminding device based on artificial intelligence according to the embodiment of the present invention may be configured on a program playing terminal, and the program playing terminal may be an intelligent television or a digital television.
As shown in fig. 7, the artificial intelligence based program reminder may include: an obtaining module 710, a speech recognition module 720, a first generation module 730, a second generation module 740, a timer establishing module 750, and a pushing module 760.
Specifically, the obtaining module 710 is configured to obtain voice information of the user.
The speech recognition module 720 is used for recognizing the speech information to generate corresponding text information.
The first generating module 730 is configured to parse the text information to generate a user intention and program information for the target program, where the program information includes a program name and a viewing time. Specifically, in one embodiment of the present invention, as shown in fig. 8, the first generation module 730 may include: a first generating unit 731 and a second generating unit 732. The first generating unit 731 is configured to perform syntactic structure analysis on the text information, and perform semantic analysis based on words, domain multi-class recognition based on a topic model, and intention multi-class recognition based on an intention model on the text information after the syntactic structure analysis, so as to obtain the user intention for the target program. The second generating unit 732 is configured to analyze the text information after performing the syntactic structure analysis based on the sequence tagging model, and obtain a program name and a viewing time of the target program.
The second generating module 740 is configured to generate reservation reminding information for the target program according to the program name and the viewing time. Specifically, in an embodiment of the present invention, as shown in fig. 9, the second generating module 740 may include: a determination unit 741, a retrieval unit 742, and a generation unit 743. The determination unit 741 is configured to determine the playing date of the target program according to the viewing time. The retrieving unit 742 is configured to retrieve in the video resource system according to the program name and the playing date, and match the playing channel information, the playing address information, and the playing time of the target program. The generating unit 743 is configured to generate reservation reminding information for the target program according to the broadcast channel information, the broadcast address information, and the broadcast time.
The timer establishing module 750 is used for establishing a reminder timer according to the user's intention and the viewing time, and starting the reminder timer. Specifically, in one embodiment of the present invention, as shown in fig. 10, the timer establishing module 750 may include: a judging unit 751 and a establishing unit 752. The determination unit 751 is configured to determine whether time information is included in the user intention. The establishing unit 752 is configured to establish a reminder timer according to the time information and the viewing time when the user intention includes the time information, and establish the reminder timer according to default time information and viewing time when the user intention does not include the time information.
The pushing module 760 is configured to push the appointment reminding information to the user when the reminding timer is finished. As an example, as shown in fig. 11, the pushing module 760 may include: a judging unit 761, a jumping unit 762, and a transmitting unit 763. The determining unit 761 is used for determining whether the program playing terminal is in an on state. The jumping unit 762 is configured to jump to a playing channel corresponding to the target program according to the playing channel information and the playing address information in the reservation reminding information when the program playing terminal is in an on state. The sending unit 763 is configured to send the reservation reminding information to the mobile terminal of the user when the program playing terminal is not in an on state.
In order to further enhance the user experience and facilitate the user to know whether the program reminder is successfully reserved, in an embodiment of the present invention, as shown in fig. 12, the artificial intelligence based program reminder apparatus may further include: a third generation module 770. The third generating module 770 is configured to generate a prompt message while starting the reminder timer, and feed back the prompt message to the user.
In order to improve the user experience and improve the accuracy of speech recognition, further, in an embodiment of the present invention, as shown in fig. 13, the artificial intelligence based program reminder apparatus may further include: a determination module 780, and a noise processing module 790. The determining module 780 is configured to determine surrounding scenes of the user before identifying the voice information, and determine a corresponding noise processing algorithm according to the surrounding scenes. The noise processing module 790 is used for performing noise processing on the voice information according to a corresponding noise processing algorithm.
According to the program reminding device based on artificial intelligence, the voice information of a user is identified through the voice identification module to generate the corresponding text information, the first generation module analyzes the text information to generate the user intention and the program information aiming at a target program, the program information comprises a program name and watching time, the second generation module generates reservation reminding information aiming at the target program according to the program name and the watching time, the timer establishment module establishes a reminding timer according to the user intention and the watching time and starts the reminding timer, and the push module pushes the reservation reminding information to the user when the reminding timer is finished. Realize the live broadcast reservation and the warning of on-demand resource update of TV end promptly through voice command, overturned traditional key remote controller input mode, directly express the reservation through pronunciation more effectively and remind the demand, it is simple more convenient during the operation, the practicality is stronger, the reserve time that simultaneously can self-defined warning, according to the custom of different crowds come in good time propelling movement to remind the notice, satisfy user's individualized demand, promoted user experience.
In order to realize the embodiment, the invention further provides a program reminding system based on artificial intelligence.
Fig. 14 is a schematic structural diagram of an artificial intelligence-based program reminder system according to an embodiment of the present invention. As shown in fig. 14, the artificial intelligence based program reminder system may include: a voice acquisition module 10 and a program reminder 20. The voice collecting module 10 may be configured to collect voice information input by a user, and send the voice information to the program reminding device 20. The program reminder 20 is an artificial intelligence-based program reminder according to any of the above embodiments of the present invention.
In the description of the present invention, it is to be understood that the terms "first", "second" and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
In the description of the specification, reference to the description of the term "one embodiment", "some embodiments", "an example", "a specific example", or "some examples", etc., means that a particular feature or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.