CN117014668A - Display device and awakening method thereof - Google Patents

Display device and awakening method thereof Download PDF

Info

Publication number
CN117014668A
CN117014668A CN202310791851.1A CN202310791851A CN117014668A CN 117014668 A CN117014668 A CN 117014668A CN 202310791851 A CN202310791851 A CN 202310791851A CN 117014668 A CN117014668 A CN 117014668A
Authority
CN
China
Prior art keywords
wake
display device
user
volume
voice
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310791851.1A
Other languages
Chinese (zh)
Inventor
杨香斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hisense Visual Technology Co Ltd
Original Assignee
Hisense Visual Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hisense Visual Technology Co Ltd filed Critical Hisense Visual Technology Co Ltd
Priority to CN202310791851.1A priority Critical patent/CN117014668A/en
Publication of CN117014668A publication Critical patent/CN117014668A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L17/00—Speaker identification or verification techniques
    • G10L17/22—Interactive procedures; Man-machine interfaces
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L17/00—Speaker identification or verification techniques
    • G10L17/22—Interactive procedures; Man-machine interfaces
    • G10L17/24—Interactive procedures; Man-machine interfaces the user being prompted to utter a password or a predefined phrase
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42203—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS] sound input device, e.g. microphone
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/443—OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
    • H04N21/4436—Power management, e.g. shutting down unused components of the receiver
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/485—End-user interface for client configuration
    • H04N21/4852—End-user interface for client configuration for modifying audio parameters, e.g. switching between mono and stereo
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/223—Execution procedure of a spoken command
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Physics & Mathematics (AREA)
  • Signal Processing (AREA)
  • Acoustics & Sound (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

The application provides a display device and a wake-up method of the display device. And when the display equipment is in a bright screen state and is in a first state, the voice of the user is collected. Upon recognizing that the user's voice contains a wake-up word, the display device wakes up. The display device displays the popup window according to the first state or plays the wake-up cue. When the display device is in a bright screen state, if the display device is awakened in a first state, a user can be prompted to be awakened in a mode of displaying a popup window or playing an awakening prompt sound, and the reason that no sound is emitted is that the display device is in a mute mode currently, so that the experience of the user is improved.

Description

Display device and awakening method thereof
Technical Field
The present application relates to the field of display devices, and in particular, to a display device and a wake-up method for the display device.
Background
The display device refers to a terminal device capable of outputting a specific display screen, and may be a terminal device such as a smart television, a mobile terminal, a smart advertisement screen, and a projector. Taking intelligent electricity as an example, the intelligent television is based on the Internet application technology, has an open operating system and a chip, has an open application platform, can realize a bidirectional man-machine interaction function, and is a television product integrating multiple functions of video, entertainment, data and the like, and the intelligent television is used for meeting the diversified and personalized requirements of users.
The display device is configured with a far-field voice control function, wakes up after recognizing a voice of a Wake word (Wake word) input by a user, and plays a Wake-up cue. When the display device is in the mute mode, the speaker will not sound when the display device plays the wake-up cue due to the restriction of the mute mode. The user is likely to misunderstand that the display device is not responding because the user cannot perceive the wake-up cue. Even if the user perceives that the display device is awakened, the user can easily misunderstand that the display device has a problem and cannot make a sound, but cannot quickly find the reason for the problem.
Disclosure of Invention
The application provides a display device and a wake-up method of the display device, wherein when the display device is in a bright screen state, if the display device is waken up in a first state, a user can be prompted to wake up by displaying a popup window or playing a wake-up prompt tone, and the reason that no sound is emitted is that the display device is in a mute mode currently, so that the experience of the user is improved.
In a first aspect, the present application provides a display apparatus comprising:
a display configured to: in a bright screen state;
a speaker configured to: the method comprises the steps that the method is in a first state, wherein the first state is that the display equipment is in a mute mode or the display equipment is in an unmuted mode, and the current system volume is 0;
A sound collector configured to: collecting user voice;
a controller configured to:
when the fact that the user voice contains the awakening word is recognized, controlling the display to display a popup window according to the first state, wherein the popup window comprises first information used for prompting the display device to be awakened and in the mute mode.
In some embodiments of the application, the controller, prior to controlling the display to display a pop-up window according to the first state, is configured to:
calculating a wake-up value corresponding to the user voice, wherein the wake-up value is used for representing the similarity degree of the user voice and the standard voice of the wake-up word;
determining whether the user voice contains a wake-up word according to the wake-up value and the wake-up value threshold;
if the wake-up value is greater than or equal to the wake-up value threshold, determining that the user voice contains a wake-up word; and if the wake-up value is smaller than the wake-up value threshold, determining that the user voice does not contain wake-up words.
In some embodiments of the application, the controller calculates a wake-up value corresponding to the user speech configured to:
inputting the user voice into a model to obtain the score of each syllable in the user voice, wherein the model is used for calculating a wake-up value, and the score of each syllable is used for representing the similarity degree of syllables with corresponding bit sequences in standard voice of the wake-up word;
And calculating a wake-up value corresponding to the user voice according to the score of each syllable in the user voice by using the model.
In some embodiments of the present application, the controller, upon recognizing that the user speech includes a wake word, controls the display to display a pop-up window according to the first state, configured to:
controlling the display to display the popup window according to the difference value between the awakening value and the awakening value threshold value;
when the difference value is smaller than a first difference value threshold value, controlling the display to display a popup window of a first style; and when the difference value is greater than or equal to a second difference value threshold, controlling the display to display a popup window of a second style, wherein the first difference value threshold is smaller than the second difference value threshold, and the popup window of the second style is easier to be perceived by a user than the popup window of the first style.
In some embodiments of the application, the controller, when in the mute mode, controls the display to display a pop-up window, further configured to:
acquiring the current system volume;
and controlling the display to display second information in the popup window according to the current system volume, wherein the second information is used for prompting a user to adjust the system volume.
In some embodiments of the application, the controller controls the display to display second information in the popup window according to the current system volume, and is configured to:
when the current system volume is larger than or equal to a first volume threshold, the second information is used for prompting a user to turn down the system volume;
and when the current system volume is smaller than a second volume threshold, the second information is used for prompting the user to increase the system volume, and the first volume threshold is larger than the second volume threshold.
In some embodiments of the application, the controller is further configured to:
and when the display equipment is in a non-mute mode and the current system volume is 0, controlling the display to display prompt information, wherein the prompt information is used for prompting that the current system volume is 0.
In a second aspect, the present application also provides a display apparatus, including:
a display configured to: in a bright screen state;
a speaker configured to: in a mute mode;
a sound collector configured to: collecting user voice;
a controller configured to:
and when the user voice is recognized to contain the wake-up word, the mute mode is released, and after the mute mode is released, the audio to be played of the wake-up prompt tone is played at a first volume.
In a third aspect, the present application also provides a display apparatus, including:
a display configured to: in a bright screen state;
a speaker configured to: in a mute mode;
a sound collector configured to: collecting user voice;
a controller configured to:
and when the user voice is recognized to contain a wake-up word, adjusting to an incomplete wake-up state, releasing the mute mode, playing a prompt voice at a first volume after the mute mode is released, and readjusting to the mute mode after the prompt voice is played, wherein the prompt voice is used for prompting the user to release the mute mode and awaken the display device again, and in the incomplete wake-up state, the display device supports playing the prompt voice and does not support responding other far-field voice instructions of the user.
In a fourth aspect, the present application also provides a display apparatus, including:
a display configured to: in a bright screen state;
a speaker configured to: in a non-mute mode and the current system volume is 0;
a sound collector configured to: collecting user voice;
a controller configured to:
When the user voice is recognized to contain the wake-up word, the system volume is increased to a preset volume, and the audio to be played of the wake-up prompt tone is played at the preset volume.
In a fifth aspect, the present application also provides a display apparatus, including:
a display configured to: in a bright screen state;
a speaker configured to: in a non-mute mode and the current system volume is 0;
a sound collector configured to: collecting user voice;
a controller configured to:
when the user voice is recognized to contain a wake-up word, the voice is adjusted to an incomplete wake-up state, the system volume is increased to a preset volume, the prompt voice is played at the preset volume, and after the prompt voice is played, the system volume is adjusted to 0, the prompt voice is used for prompting the user to increase the system volume and wake up the display device again, wherein in the incomplete wake-up state, the display device supports playing the prompt voice and does not support responding other far-field voice instructions of the user.
In a sixth aspect, the present application further provides a wake-up method of a display device, which is applied to the display device according to any one of the first aspect, where the display device is in a bright screen state and a first state, and the first state is that the display device is in a mute mode or the display device is in a non-mute mode, and a current system volume is 0;
The method comprises the following steps:
collecting user voice;
and when the user voice is recognized to contain a wake-up word, displaying a popup according to the first state, wherein the popup comprises first information used for prompting the display equipment to be awakened and in the mute mode.
In a seventh aspect, the present application further provides a wake-up method of a display device, which is applied to the display device according to the second aspect, where the display device is in a bright screen state and a mute mode;
the method comprises the following steps:
collecting user voice;
and when the user voice is recognized to contain the wake-up word, the mute mode is released, and after the mute mode is released, the audio to be played of the wake-up prompt tone is played at a first volume.
In an eighth aspect, the present application further provides a wake-up method of a display device, which is applied to the display device according to the third aspect, where the display device is in a bright screen state and a mute mode;
the method comprises the following steps:
collecting user voice;
and when the user voice is recognized to contain a wake-up word, adjusting to an incomplete wake-up state, releasing the mute mode, playing a prompt voice at a first volume after the mute mode is released, and readjusting to the mute mode after the prompt voice is played, wherein the prompt voice is used for prompting the user to release the mute mode and awaken the display device again, and in the incomplete wake-up state, the display device supports playing the prompt voice and does not support responding other far-field voice instructions of the user.
In a ninth aspect, the present application further provides a wake-up method of a display device, which is applied to the display device according to the fourth aspect, where the display device is in a bright screen state and a non-mute mode, and the current system volume is 0;
the method comprises the following steps:
collecting user voice;
when the user voice is recognized to contain the wake-up word, the system volume is increased to a preset volume, and the audio to be played of the wake-up prompt tone is played at the preset volume.
In a tenth aspect, the present application further provides a wake-up method for a display device, which is applied to the display device in the fifth aspect, where the display device is in a bright screen state and a non-mute mode, and the current system volume is 0;
the method comprises the following steps:
collecting user voice;
when the user voice is recognized to contain a wake-up word, the voice is adjusted to an incomplete wake-up state, the system volume is increased to a preset volume, the prompt voice is played at the preset volume, and after the prompt voice is played, the system volume is adjusted to 0, the prompt voice is used for prompting the user to increase the system volume and wake up the display device again, wherein in the incomplete wake-up state, the display device supports playing the prompt voice and does not support responding other far-field voice instructions of the user.
And when the display equipment is in a bright screen state and is in a first state, the voice of the user is collected. Upon recognizing that the user's voice contains a wake-up word, the display device wakes up. The display device displays the popup window according to the first state or plays the wake-up cue. When the display device is in a bright screen state, if the display device is awakened in a first state, a user can be prompted to be awakened in a mode of displaying a popup window or playing an awakening prompt sound, and the reason that no sound is emitted is that the display device is in a mute mode currently, so that the experience of the user is improved.
Drawings
In order to more clearly illustrate the technical solution of the present application, the drawings that are needed in the embodiments will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.
FIG. 1 is a usage scenario of a display device according to an embodiment of the present application;
fig. 2 is a block diagram of a configuration of a control device in an embodiment of the present application;
fig. 3 is a configuration diagram of a display device in an embodiment of the present application;
FIG. 4 is a diagram illustrating an operating system configuration of a display device according to an embodiment of the present application;
FIG. 5 is a flowchart of a display device according to the embodiment of the present application for adjusting the volume of wake-up alert based on the signal-to-noise ratio of the user's voice and the current system volume;
FIG. 6 is a flowchart of a display device recognizing wake-up words in an embodiment of the present application;
FIG. 7 is a flowchart of a display device using a model to calculate a wake-up value for a user's voice in an embodiment of the present application;
FIG. 8 is a schematic diagram of a model for a display device to calculate wake-up values for user speech in an embodiment of the present application;
FIG. 9 is a schematic diagram of a model of a display device calculating a wake-up value of a user's voice in an embodiment of the present application;
FIG. 10 is a flow chart of a display device adjusting the amplitude of the audio of the wake-up cue according to the signal-to-noise ratio of the user's voice and the current system volume in an embodiment of the application;
FIG. 11 is a flow chart of a display device adjusting the amplitude of the audio of the wake-up cue according to the signal-to-noise ratio of the user's voice and the current system volume in an embodiment of the application;
FIG. 12 is a logic judgment diagram of the display device according to the signal-to-noise ratio of the user's voice and the current system volume to adjust the amplitude of the audio of the wake-up cue;
FIG. 13 is a schematic diagram of a pop-up window including information for prompting a user to adjust the system volume according to an embodiment of the present application;
FIG. 14 is a flowchart of a display device adjusting the volume of a wake-up alert tone according to the difference between the wake-up value and the wake-up value threshold and the current system volume in an embodiment of the present application;
FIG. 15 is a flowchart of a display device adjusting the amplitude of the audio of the wake-up cue according to the difference between the wake-up value and the wake-up value threshold and the current system volume in an embodiment of the present application;
FIG. 16 is a flow chart of a display device adjusting the amplitude of the audio of the wake-up cue according to the difference between the wake-up value and the wake-up value threshold and the current system volume in an embodiment of the present application;
FIG. 17 is a logic judgment diagram of the display device according to the difference between the wake-up value and the wake-up value threshold and the amplitude of the audio of the wake-up cue adjusted by the current system volume in the embodiment of the application;
FIG. 18 is a schematic view of a first type of pop-up window and a second type of pop-up window with different sizes according to an embodiment of the present application;
FIG. 19 is a schematic view of a first type of pop-up window and a second type of pop-up window in different positions according to an embodiment of the present application;
FIG. 20 is a schematic view of a first type of pop-up window and a second type of pop-up window with different sizes and different display positions according to an embodiment of the present application;
FIG. 21 is a flow chart of a display device according to the difference between the wake-up value and the wake-up value threshold, the signal to noise ratio of the user's voice, and the current system volume to adjust the volume of the wake-up alert sound in an embodiment of the present application;
FIG. 22 is a flow chart of a display device according to the difference between the wake-up value and the wake-up value threshold, the signal to noise ratio of the user's voice, and the current system volume to adjust the amplitude of the audio of the wake-up alert tone in an embodiment of the present application;
FIG. 23 is a flow chart of a display device according to the difference between the wake-up value and the wake-up value threshold, the signal to noise ratio of the user's voice, and the current system volume to adjust the amplitude of the audio of the wake-up alert tone in an embodiment of the present application;
FIG. 24 is a logic judgment diagram of the display device according to the difference between the wake-up value and the wake-up value threshold, the signal to noise ratio of the user's voice, and the current system volume to adjust the amplitude of the audio of the wake-up cue;
FIG. 25 is a flowchart of a wake-up response performed by a display device in a bright screen state and in a first state according to an embodiment of the present application;
FIG. 26 is a schematic diagram of a pop-up window including first information according to an embodiment of the present application;
FIG. 27 is a flowchart showing second information according to an embodiment of the present application;
FIG. 28 is a schematic view of a pop-up window including first information and second information according to an embodiment of the present application;
FIG. 29 is a flowchart of a display device in an embodiment of the application for unmuted mode;
FIG. 30 is a flowchart of a display device in an embodiment of the application for unmuted mode;
FIG. 31 is a schematic view of a pop-up window including third information according to an embodiment of the present application;
FIG. 32 is a flowchart of a wake-up response performed by a display device in a bright screen state and in a mute mode according to an embodiment of the present application;
FIG. 33 is a flowchart of a wake-up response performed by a display device in a bright screen state and in a non-mute mode when the current system volume is 0 according to an embodiment of the present application;
FIG. 34 is a flowchart of a display device responding to wake-up words when in a speaker mode and in a mute mode according to an embodiment of the present application;
FIG. 35 is a flowchart of a display device responding to wake-up words when in a speaker mode and in a non-mute mode and the current system volume is 0 according to an embodiment of the present application;
FIG. 36 is a schematic diagram showing a display device playing a wake-up alert and a first alert in an embodiment of the present application;
FIG. 37 is a schematic diagram of a display device playing wake-up alert tones, first alert messages, and second alert messages in an embodiment of the present application;
FIG. 38 is a flowchart of a display device responding to wake words when in a speaker mode and in a first state according to an embodiment of the present application;
FIG. 39 is a schematic diagram of a middle page and a popup window according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The embodiments described in the examples below do not represent all embodiments consistent with the application. Merely exemplary of systems and methods consistent with aspects of the application as set forth in the claims.
It should be noted that the brief description of the terminology in the present application is for the purpose of facilitating understanding of the embodiments described below only and is not intended to limit the embodiments of the present application. Unless otherwise indicated, these terms should be construed in their ordinary and customary meaning.
The terms first, second, third and the like in the description and in the claims and in the above-described figures are used for distinguishing between similar or similar objects or entities and not necessarily for describing a particular sequential or chronological order, unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances.
The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements is not necessarily limited to all elements explicitly listed, but may include other elements not expressly listed or inherent to such product or apparatus.
The display device provided by the embodiment of the application can have various implementation forms, for example, can be an intelligent television, a laser projection device, a display (monitor), an electronic whiteboard (electronic bulletin board), an electronic desktop (electronic table) and the like, and can also be a device with a display screen, such as a mobile phone, a tablet personal computer, an intelligent watch and the like. Fig. 1 and 2 are specific embodiments of a display device of the present application.
Fig. 1 is a schematic view of a usage scenario of a display device according to an embodiment. As shown in fig. 1, a user may operate the display apparatus 200 through the control apparatus 100, and the display apparatus 200 may acquire network data through the server 300 or acquire a live signal through a satellite. The user may also control the display device 200 by voice, for example, the user wakes up the display device 200 by issuing a wake word, and the waken display device 200 initiates far-field voice services in response to voice instructions issued by the user over a period of time.
Fig. 2 is a block diagram of the configuration of the control apparatus 100. In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the display device 200 may include at least one of infrared protocol communication or bluetooth protocol communication, and other short-range communication modes, and the display device 200 may be controlled by a wireless or wired mode. The user may control the display device 200 by inputting user instructions through keys on a remote control, voice input, control panel input, etc. The control device 100 may include a mute key for turning on a mute mode and an un-mute mode, wherein the mute mode is a mode in which the display device 200 does not change the amplitude of audio data, i.e., retains the current system volume, and does not transmit the decoded audio data to the speaker for playback after decoding the audio data. When the display apparatus 200 does not turn on the mute mode, the user inputs a control instruction to the display apparatus 200 based on the mute key, and the display apparatus 200 recognizes the control instruction as turning on the mute mode. The display apparatus 200 retains the current system volume in response to the control instruction and does not transmit the decoded audio data to the speaker after decoding the audio data. When the display apparatus 200 turns on the mute mode, the user inputs a control instruction to the display apparatus 200 based on the mute key, and the display apparatus 200 recognizes the control instruction as the unmuted mode. The display device 200 transmits the decoded audio data to the speaker in response to the control instruction, and plays the audio at the reserved current system volume. The control device 100 may also include a volume key for adjusting the volume at which audio is played. Wherein, if the user inputs an instruction to lower the volume based on the volume key, the display apparatus 200 reduces the amplitude of the decoded audio data after decoding the audio data in response to the instruction, and transmits the audio data with reduced amplitude to the speaker to play; if the user inputs an instruction to turn up the volume based on the volume key, the display apparatus 200 increases the amplitude of the decoded audio data after decoding the audio data and transmits the audio data with the increased amplitude to the speaker to play in response to the instruction. In some embodiments, if the user adjusts the volume to 0 through the volume key, the display apparatus 200 adjusts the amplitude of the decoded audio data to 0 after decoding the audio data, and transmits the audio data with the amplitude of 0 to the speaker to play.
Fig. 3 shows a block diagram of a configuration of the display device 200 in accordance with an exemplary embodiment.
The display apparatus 200 includes at least one of a modem 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface 280.
In some embodiments, the display apparatus 200 may establish transmission and reception of control signals and data signals with the control apparatus 100 or the server 300 through the communicator 220. In some embodiments, detector 230 includes a sound collector. The display device 200 may collect user voice through the sound collector and transmit the collected user voice to the controller 250 for processing. In some embodiments, the controller 250 and the modem 210 may be located in separate devices, i.e., the modem 210 may also be located in an external device to the main device in which the controller 250 is located, such as an external set-top box or the like. In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored on the memory. The controller 250 controls the overall operation of the display apparatus 200. In some embodiments, the controller 250 includes at least one of a central processing unit (Central Processing Unit, CPU), a video processor, an audio processor, a graphics processor (Graphics Processing Unit, GPU), RAM (Random Access Memory), a ROM (Read-Only Memory), a first to nth interface for input/output, a communication Bus (Bus), and the like. For example, the controller 250 may recognize the user's voice through the central processor and the audio processor, and if it is recognized that the user's voice includes a wake word, may process the audio of the wake-up cue, and transmit the processed audio of the wake-up cue to the audio output interface 270 for playing, such as through a speaker. In some embodiments, the display 260 includes a display screen component for presenting a picture, and a driving component for driving an image display, a component for receiving an image signal output from the controller 250, displaying video content, image content, and a menu manipulation interface, and a user manipulation UI interface. In some embodiments, the display 260 may be a liquid crystal display, an OLED display, a projection device, and a projection screen. In some embodiments, a user may input a user command through a graphical user interface (Graphic User Interface, GUI) displayed on the display 260, and the user input interface receives the user input command through the Graphical User Interface (GUI). Alternatively, the user may input the user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through the sensor to receive the user input command. In some embodiments, a "user interface" is a media interface for interaction and exchange of information between an application or operating system and a user that enables conversion between an internal form of information and a form acceptable to the user. A commonly used presentation form of a user interface is a Graphical User Interface (GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It may be an interface element such as an icon, a window, a control, etc. displayed in a display screen of the electronic device, where the control may include at least one of a visual interface element such as an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a Widget, etc.
Referring to fig. 4, in some embodiments, the operating system of the display device 200 is divided into four layers, an application layer (application layer), an application framework layer (Application Framework layer), a An Zhuoyun row (Android run) and a system library layer (system runtime layer), and a kernel layer, respectively. In some embodiments, at least one application program is running in the application program layer, where the application programs may be a Window (Window) program, a system setting program, or a clock program, etc. of an operating system, and the live application is used for playing live programs; or may be an application developed by a third party developer.
Based on the operating system shown in fig. 4, the display apparatus 200 determines an Audio Stream (Stream) type by an Audio management module (Audio Manager) in a frame layer, decodes Pulse-Code Modulation (PCM) data of the Audio, and delivers the decoded Audio to an Audio Track object created in a system runtime layer through an Audio Track interface corresponding to the Audio Stream type, and processes and outputs the Audio through the Audio Track object. When the Audio track object processes Audio, an Audio hardware device for outputting the Audio is determined through an Audio policy maker (Audio Policy Service) object, and the processed Audio is transmitted to the Audio hardware device through an Audio policy executor (Audio player) object, and the Audio is input through the Audio hardware device. The audio management module, the audio track object and the audio hardware equipment form an audio playing path, and the audio playing path corresponds to the audio stream type of the audio according to the process of playing the audio.
Audio played by the display device 200 may be classified into (Voice Call), music (Music), ring (Ring), alarm (Alarm), bluetooth physical link (Bluetooth Synchronous Connection Oriented), etc., according to scenes. In some embodiments, the display device 200 configures an audio Stream type, such as a Music type (stream_music). Therefore, the audio management module uniformly determines the audio of different scenes as the stream_music type, and invokes the audio track object to process and output the audio through the audio track interface corresponding to the stream_music type. That is, the audio of different scenes is played through the same playing path.
The display device 200 adjusts the volume through an audio management module, wherein the audio management module adjusts the volume of the corresponding playback path based on the audio stream type. For example, the audio management module adjusts the volume for the playback channel corresponding to the audio stream type by calling the adjust audio stream volume method (adjustStrea mVolume). In the case where the display apparatus 200 is configured with one audio stream type, the display apparatus 200 adjusts the volume for a playback channel corresponding to the audio stream type through the audio management module. For example, one type of audio Stream configured by the display apparatus 200 is the stream_music type, and the audio management module turns up the volume by calling the adjust Stream volume (audiomanager. Stream_ MUSIC, audioManager.ADJUST _ RAISE, audioManager.FLAG) or turns down the volume by calling the adjust Stream volume (audiomanager. Stream_ MUSIC, audioManager.ADJUST _ DECREASE, audioManager.FLAG). Since the audio of different scenes is played through the playing path, the display device 200 does not adjust the volume of the audio of different scenes respectively, but adjusts the volume of the audio of different scenes uniformly. At this time, the volume adjusted by the display apparatus 200 corresponds to the adjustment system volume.
The audio played by the display device 200 includes a wake-up cue. The display device 200 wakes up after recognizing that the user inputs a wake-up word and plays a wake-up cue to prompt the user to wake up. Based on the corresponding relation between the audio and the playing path and the corresponding relation between the playing path and the volume, the display device 200 plays the wake-up cue according to the system volume, if the system volume is large, the volume of the wake-up cue is large, and if the system volume is small, the volume of the wake-up cue is small.
When the user wakes up the display device 200 by far-field speech, the display device 200 will wake up in response to the wake-up word spoken by the user. In response to the display device 200 being woken up, the volume of the wake-up alert sound played will affect the user's experience of using the display device 200. In different scenes, such as the noisy level of the environment where the display device 200 is located, the working state where the display device 200 is located, etc., the response mode after the display device 200 is waken up, and the appropriate volume for playing the wake-up cue sound are different, and the display device 200 needs to adjust the response mode after the wake-up according to different scenes, and adjust the volume of the wake-up cue sound. The following embodiments are used to describe the manner in which the display device 200 adjusts the response after waking up and adjusts the volume of the wake-up cue sound in different scenarios.
The audio of the wake-up cue is played after the display device 200 is woken up. The effect of the user hearing the wake-up cue played by the display apparatus 200 may be affected by the degree of noise of the environment in which the display apparatus 200 is located and the current system volume of the display apparatus 200. For example, when the environment in which the display apparatus 200 is located is noisy, if the current system volume of the display apparatus 200 is small, the volume at which the display apparatus 200 plays the wake-up cue sound is also small. At this time, the user does not easily perceive the wake-up cue, resulting in the user misunderstanding that the display apparatus 200 is not woken up. When the environment where the display apparatus 200 is located is relatively quiet, if the current system volume of the display apparatus 200 is large, the volume of the display apparatus 200 playing the wake-up cue sound is also large. At this time, the wake-up prompt tone with a large volume is abrupt in the environment, and is easy to surprise the user.
In order to solve the above problem, the display device 200 may adjust the volume of the wake-up cue according to the flow shown in fig. 5, which includes the following steps:
s501, collecting user voice.
After the far-field voice function is activated, the display apparatus 200 captures user voice through the sound collector.
S502, when recognizing that the user voice contains wake-up words, acquiring the signal-to-noise ratio of the user voice and the current system volume.
The display device 200 may recognize whether the user voice includes the wake-up word according to the flow shown in fig. 6, which includes the following steps:
s601, calculating a wake-up value corresponding to the voice of the user.
The wake-up value is used to characterize how similar the user's voice is to the standard voice of the wake-up word. The wake-up value may be a value of 0-1, the higher the wake-up value of the user's voice, the higher the similarity of the user's voice to the standard voice of the wake-up word, and the higher the probability that the display device 200 will wake up. The lower the wake-up value of the user's voice, the lower the degree of similarity of the user's voice to the standard voice of the wake-up word, and the lower the probability that the display device 200 will wake up.
The display device 200 may calculate a wake-up value for the user's voice through the model. The model may be a neural network model, such as a convolutional neural network (Convolutional Neural Network, CNN) based model, a recurrent neural network (Recurrent Neural Network, RNN) based model, a deep neural network (Deep Neural Networks, DNN) based model. The model may be obtained in advance through standard voice training of the user voice sample and the wake-up word, and may be iteratively trained based on the accuracy of the user voice and the wake-up result actually collected by the display device 200, so as to perfect the accuracy of the model and the fit to the user.
The display device 200 may calculate the wake-up value of the user's voice using the model according to the flow shown in fig. 7, and the specific steps are as follows:
s701, inputting the user voice into a model to obtain the score of each syllable in the user voice.
S702, calculating a wake-up value corresponding to the user voice according to the score of each syllable in the user voice by using the model.
Taking the model shown in fig. 8, taking the wake-up word "sea letter small poly" as an example, exemplary descriptions will be made on steps S601-S602. The display device 200 inputs user voices into the model in real time, and the model recognizes each syllable in turn according to the received user voices. Wherein the model calculates a score for each syllable based on the decoding network of phoneme transition probabilities as the syllable is identified. The score of each syllable is used to characterize the similarity probability value, i.e., the degree of similarity, with the corresponding syllable of the bit in the standard speech of the wake word. For example, the model identifies the first of the user's voicesSyllables in the standard voice of the wake-up word are ranked in the first order and are "sea", and the model calculates the similarity probability value of "sea" in the user voice and "sea" in the standard voice of the wake-up word to be p 00 Obtaining the score corresponding to the sea in the voice of the user, namely p 00 . The process of calculating the scores of "letter", "small", "gather" in the user's voice by the model is similar to the process of calculating the score of "sea", and is not repeated here. The scores of the letter, the small and the poly are p respectively 11 、p 22 、p 33 . And the model calculates the wake-up value corresponding to the user voice according to the score of each syllable in the user voice. For example, the model may calculate a front-to-back transition probability value between each syllable and other syllables in the user's voice, and perform a weighted calculation according to the score of each syllable and the front-to-back transition probability value between the syllables, to obtain a weighted value, and use the weighted value as the wake-up value of the user's voice.
Taking the model shown in fig. 9, taking the wake-up word "sea letter small poly" as an example, exemplary descriptions will be made on steps S601-S602. The model includes an input layer, a hidden layer, and an output layer. The display device 200 inputs the user's voice in real time to an input layer of a model that divides the user's voice into n voice segments, such as voice segment x, at the input layer 1 -x n . Based on different training modes and training degrees of the model, the model divides the user voice into different voice fragments at the input layer. For example, the input layer may divide the user's speech into 4 speech segments by syllable, where x 1 Corresponding to "sea", x 2 Corresponds to "letter", x 3 Corresponding to "small", x 4 Corresponding to "poly". Classification-based decoding method for voice fragment x in hidden layer by model 1 -x n The probability values mapped to different targets are calculated, the targets of the hidden layer comprise 'sea message small gather' and other voices, namely the probability value P that the user voice is the sea message small gather can be obtained through the hidden layer 1 And a probability value P that the user's voice is other voice 2 . The hidden layer transmits the calculated probability value to the output layer for outputting. Wherein the user voice is the probability value P of the sea message gatherings 1 Can be used forAs a wake-up value for the user's voice.
The display device 200 may also calculate a wake-up value for the user's voice using other models, without limitation.
S602, determining whether the user voice contains a wake-up word according to the wake-up value and the wake-up value threshold.
The wake-up value threshold is the lowest wake-up value at which the user wakes up the display device 200 by voice. When the wake-up value of the user voice is greater than or equal to the wake-up value threshold, it is determined that the user voice contains a wake-up word, the display device 200 is woken up, and audio of the wake-up cue is played. And when the wake-up value of the user voice is smaller than the wake-up value threshold, determining that the user voice does not contain wake-up words, and enabling the display equipment not to be woken up.
When the wake-up value of the user voice is greater than or equal to the wake-up value threshold, that is, after the display device 200 determines to play the audio of the wake-up cue, the display device 200 obtains the signal-to-noise ratio of the user voice and the current system volume, so as to determine the volume of the wake-up cue to be played according to the signal-to-noise ratio of the user voice and the current system volume.
The signal-to-noise ratio of the user's voice is used to characterize the level of noise in the environment in which the display device 200 is located. The higher the signal-to-noise ratio of the user's voice, the lower the noise level of the environment in which the display device 200 is located, i.e., the quieter the environment in which the display device 200 is located. The lower the signal-to-noise ratio of the user's voice, the higher the degree of noise in the environment in which the display device 200 is located, i.e., the more noisy the environment in which the display device 200 is located.
The user voice acquired by the display device 200 includes a first voice and a second voice, the first voice is located before the second voice, the duration corresponding to the first voice and the second voice is equal, the first voice is an environmental sound without a wake-up word, and the second voice is a voice corresponding to the wake-up word. The display apparatus 200 obtains the signal-to-noise ratio of the user's voice by calculating the ratio of the volume or energy of the second voice and the first voice. Wherein the higher the ratio the higher the signal to noise ratio and the lower the ratio the lower the signal to noise ratio. The display device 200 may obtain the signal-to-noise ratio of the user's voice through a model, which may be the same model as the model for calculating the wake-up value, or may be a different model from the model for calculating the wake-up value.
The display device 200 may obtain the current system volume through a system standard interface, for example: int currentVolume = maudiomanager.
And S503, adjusting the amplitude of the audio of the wake-up cue according to the signal-to-noise ratio and the current system volume to obtain the audio to be played of the wake-up cue.
The volume of the wake-up cue is adjusted in combination with the level of noise of the environment in which the display device 200 is located and the current system volume of the display device 200. Based on the above analysis of the playback channels in which the display device 200 plays audio, in order not to affect the volume of other audio than the audio of the wake-up alert sound, the display device 200 does not make an adjustment to the volume of the playback channels (i.e., the system volume), but rather, by adjusting the amplitude of the audio of the wake-up alert sound, the effect of individually adjusting the volume of the wake-up alert sound is achieved. In the case where the system volume is the same, the smaller the amplitude of the audio of the wake-up cue, the smaller the volume of the wake-up cue output by the display apparatus 200, the larger the amplitude of the audio of the wake-up cue, and the larger the volume of the wake-up cue output by the display apparatus 200.
The display device 200 may adjust the amplitude of the audio of the wake-up cue according to the flow shown in fig. 10, specifically as follows:
S1001, acquiring the audio of the wake-up prompt tone.
The display device 200 may locally pre-store a wake-up cue audio, and the display device 200 may obtain the wake-up cue audio from the local storage. The display device 200 may not pre-store the audio of the wake-up cue, but acquire the audio of the wake-up cue from the server 300.
S1002, according to the signal-to-noise ratio and the current system volume, adjusting the amplitude of the audio of the wake-up cue to obtain the audio to be played of the wake-up cue.
The display device 200 dynamically adjusts the amplitude of the audio of the wake-up cue according to the signal-to-noise ratio and the current system volume, so as to obtain the audio to be played of the wake-up cue. For example, the amplitude of the audio of the wake-up cue is adjusted by adjusting the gain of the audio of the wake-up cue.
Based on the above-mentioned mode of adjusting the amplitude of the audio itself of the wake-up cue, it is convenient to set finer amplitude adjustment granularity, and the volume of the wake-up cue can be more accurate to suit the current environment.
The display device 200 may adjust the amplitude of the audio of the wake-up cue according to the procedure shown in fig. 11, specifically as follows:
s1101, determining the amplitude of the audio to be played of the wake-up cue according to the signal-to-noise ratio and the current system volume.
The display device 200 may locally pre-store a plurality of preset tones of wake-up cues, each corresponding to an amplitude, and the display device 200 may obtain pre-stored tones of wake-up cues having different amplitudes from the local storage. The display device 200 may not pre-store the audio of the wake-up cue, but may arrange a plurality of preset audio of the wake-up cue, each corresponding to one amplitude, at the server 300, and the display device 200 may obtain the pre-stored audio of the wake-up cue having different amplitudes from the server 300.
The different preset tones of the wake-up cue may be distinguished according to a specific amplitude of the amplitude, and the file name of the preset tone of the wake-up cue may include the specific amplitude, e.g. "wake-up cue-3 dB". Accordingly, the display apparatus 200 can recognize the corresponding amplitude based on the file name of the preset audio. The display device 200 may determine a specific magnitude of the amplitude of the audio to be played of the wake-up cue to be played based on the signal-to-noise ratio and the current system volume. For example, the signal-to-noise ratio and the current system volume have a correspondence with a specific amplitude of the audio to be played of the wake-up cue, and the display apparatus 200 may determine the specific amplitude based on the correspondence.
The different preset audios of the wake-up cue can be distinguished according to the amplitude, and the file name of the preset audios of the wake-up cue can comprise the amplitude, such as 'wake-up cue-small amplitude'. Accordingly, the display apparatus 200 can recognize the magnitude of the corresponding amplitude based on the file name of the preset audio. The display device 200 may determine the magnitude of the amplitude of the audio to be played of the wake-up cue to be played according to the signal-to-noise ratio and the current system volume. For example, the signal-to-noise ratio and the current system volume have a correspondence with the magnitude of the amplitude of the audio to be played of the wake-up cue, and the display apparatus 200 may determine the magnitude of the amplitude based on the correspondence.
S1102, according to the amplitude of the audio to be played of the wake-up cue, acquiring the preset audio of the wake-up cue with the corresponding amplitude.
After determining the amplitude of the audio to be played of the wake-up cue, the display apparatus 200 may acquire preset audio of the wake-up cue having the corresponding amplitude.
Based on the above-mentioned to wake up the audio frequency to be played amplitude direct acquisition prestored audio frequency to the mode of adjusting the audio frequency of wake up the audio frequency of cue, can obtain the audio frequency to be played of wake up the cue more fast, in order to improve the efficiency of playing the wake up the cue.
The process by which the display device 200 adjusts the amplitude of the audio of the wake-up cue based on the signal-to-noise ratio and the current system volume will be described in connection with fig. 12.
The display device 200 sets a first signal-to-noise ratio threshold and a second signal-to-noise ratio threshold, and a first volume threshold and a second volume threshold. Wherein the first signal-to-noise ratio threshold is greater than the second signal-to-noise ratio threshold and the first volume threshold is greater than the second volume threshold.
The display device 200 performs step S1201 to determine whether the signal-to-noise ratio of the user' S voice is greater than or equal to the first signal-to-noise ratio threshold.
When the signal-to-noise ratio of the user' S voice is greater than or equal to the first signal-to-noise ratio threshold, the display device 200 performs step S1202 to determine whether the current system volume is greater than or equal to the first volume threshold.
When the signal-to-noise ratio of the user' S voice is greater than or equal to the first signal-to-noise ratio threshold and the current system volume is greater than or equal to the first volume threshold, the display device 200 executes step S1203 to reduce the amplitude of the audio of the wake-up alert sound, and obtain the audio to be played of the wake-up alert sound.
The signal-to-noise ratio of the user's voice being greater than or equal to the first signal-to-noise ratio threshold indicates that the environment in which the display device 200 is located is less noisy, i.e., the environment in which the display device 200 is located is quieter. The current system volume is greater than or equal to the first volume threshold, which indicates that the display device 200 has a higher volume of the wake-up alert sound if the audio of the wake-up alert sound is directly played using the current system volume. At this time, the display device 200 plays the wake-up cue sound at a large volume in a relatively quiet environment, which is likely to cause a strong sense of intrusion to the user. To solve this problem, the amplitude of the audio of the wake-up cue is reduced to reduce the volume of the wake-up cue, thereby reducing the user's surprise.
When the signal-to-noise ratio of the user' S voice is greater than or equal to the first signal-to-noise ratio threshold and the current system volume is less than the first volume threshold, the display device 200 performs step S1204 without adjusting the amplitude of the audio of the wake-up cue.
The signal-to-noise ratio of the user's voice being greater than or equal to the first signal-to-noise ratio threshold indicates that the display device 200 is in a quieter environment. The current system volume is less than the first volume threshold, which indicates that if the display device 200 directly plays the audio of the wake-up alert sound using the current system volume, the volume of the wake-up alert sound is not too great. In this quieter environment, the wake-up cue of volume does not cause excessive intrusive feeling to the user. Accordingly, the display device 200 does not need to adjust the volume of the wake-up cue, and the display device 200 does not adjust the amplitude of the audio of the wake-up cue.
When the signal-to-noise ratio of the user voice is smaller than the first signal-to-noise ratio threshold, the display device 200 executes step S1205 to determine whether the signal-to-noise ratio of the user voice is greater than or equal to the second signal-to-noise ratio threshold.
When the signal-to-noise ratio of the user voice is smaller than the first signal-to-noise ratio threshold and the signal-to-noise ratio of the user voice is greater than or equal to the second signal-to-noise ratio threshold, the display device 200 performs step S1204 without adjusting the amplitude of the audio of the wake-up cue.
The signal-to-noise ratio of the user's voice is smaller than the first signal-to-noise ratio threshold and is greater than or equal to the second signal-to-noise ratio threshold, which indicates that the environment where the display device 200 is located is moderately noisy, and the display device 200 is considered to directly play the audio of the wake-up cue sound at the current system volume, in which the volume of the wake-up cue sound can play a role in prompting the user, and excessive intrusion feeling is not caused to the user. Accordingly, the display device 200 does not need to adjust the volume of the wake-up cue, and the display device 200 does not adjust the amplitude of the audio of the wake-up cue.
When the signal-to-noise ratio of the user' S voice is less than the second signal-to-noise ratio threshold, the display device 200 performs step S1206 to determine whether the current system volume is less than the second volume threshold.
When the signal-to-noise ratio of the user' S voice is less than the second signal-to-noise ratio threshold and the current system volume is greater than or equal to the second volume threshold, the display device 200 performs step S1204 without adjusting the amplitude of the audio of the wake-up cue.
The signal-to-noise ratio of the user's voice being less than the second signal-to-noise ratio threshold value indicates that the environment in which the display device 200 is located is noisy, i.e., the environment in which the display device 200 is located is noisy. The current system volume is greater than or equal to the second volume threshold, which indicates that the volume of the wake-up alert sound is not too small if the display device 200 directly plays the audio of the wake-up alert sound using the current system volume. At this time, the display device 200 plays the wake-up cue sound at the volume that is not too small in a noisy environment, and may also satisfy the user's perception of the wake-up cue sound. Accordingly, the display device 200 does not need to adjust the volume of the wake-up cue, and the display device 200 does not adjust the amplitude of the audio of the wake-up cue.
When the signal-to-noise ratio of the user voice is smaller than the second signal-to-noise ratio threshold and the current system volume is smaller than the second volume threshold, the display device 200 executes step S1207 to increase the amplitude of the audio of the wake-up cue to obtain the audio to be played of the wake-up cue.
The signal-to-noise ratio of the user's voice being less than the second signal-to-noise ratio threshold value indicates that the environment in which the display device 200 is located is noisy, i.e., the environment in which the display device 200 is located is noisy. The current system volume is less than the second volume threshold, which indicates that the display device 200 has a smaller volume of the wake-up alert sound if the audio of the wake-up alert sound is directly played using the current system volume. At this time, the display apparatus 200 plays the wake-up cue sound at a small volume in a noisy environment, which is likely to be undetectable to the user, thereby misunderstanding that the display apparatus 200 is not woken up. To solve this problem, the volume of the wake-up cue is increased by increasing the amplitude of the audio of the wake-up cue, thereby improving the perception of the wake-up cue by the user.
S504, playing the audio to be played of the wake-up cue at the current system volume.
After obtaining the audio to be played of the wake-up alert sound based on step S503, the display device 200 plays the audio to be played of the wake-up alert sound at the current system volume, where the volume of the wake-up alert sound is affected by the current system volume and the volume of the audio to be played of the wake-up alert sound itself.
When the current system volume is smaller than the third volume threshold, the display device 200 displays a popup window, and displays prompt information in the popup window, where the prompt information is used to prompt the user that the current system volume is too small, resulting in that the volume of the wake-up prompt tone is too small, and the system volume can be increased, where the third volume threshold is smaller than or equal to the second volume threshold. Taking the display device 200 as an example of a television, when the current system volume is less than the third volume threshold, the display device 200 may display a pop-up window 1301 as shown in (1) in fig. 13, where the pop-up window 1301 includes a prompt message, such as "the system volume is too low, and please turn up the system volume.
When the current system volume is greater than the fourth volume threshold, the display device 200 displays a popup window, and displays prompt information in the popup window, where the prompt information is used to prompt the user that the current system volume is too large, resulting in that the volume of the wake-up prompt tone is too large, and the system volume can be reduced, where the fourth volume threshold is greater than or equal to the first volume threshold. Taking the display device 200 as an example of a television, when the current system volume is less than the third volume threshold, the display device 200 may display a pop-up window 1302 as shown in (2) in fig. 13, where the pop-up window 1302 includes a prompt message, such as "system volume is too high, and please turn down the system volume.
The audio of the wake-up cue is played after the display device 200 is woken up. The effect of the user hearing the wake-up cue played by the display device 200 may be affected by whether the user really wants to wake up the display device 200 and the current system volume of the display device 200. If the user wakes up the display device 200 with a high probability of error, and the current system of the display device 200 has a high volume, the display device 200 plays the wake-up cue sound with a high volume, and the user can obviously perceive that the display device 200 is waken up. When the display device 200 is awakened by mistake, normal use of the display device 200 by a user is affected. If the user wakes up the display device 200 with a small probability of error and the current system volume of the display device 200 is small, the volume of the wake-up cue played by the display device 200 is also small, and the wake-up cue with the small volume makes it difficult for the user to perceive that the display device 200 is woken up. When the display apparatus 200 is not awakened by mistake, it is easy for the user to mistakenly consider that the display apparatus 200 is not awakened.
In order to solve the above problem, the display device 200 may adjust the volume of the wake-up cue according to the flow shown in fig. 14, which includes the following steps:
s1401, collecting user voice.
S1402, calculating the wake-up value corresponding to the user voice.
Steps S1401 to S1402 are similar to steps S501 and S601 and steps S701 to S702, and are not described here.
S1403, when the wake-up value is greater than or equal to the wake-up value threshold, the difference value between the wake-up value and the wake-up value threshold and the current system volume are obtained.
The difference between the wake-up value and the wake-up value threshold is used to characterize the probability that the display device 200 is awakened by mistake. The larger the difference between the wake-up value and the wake-up value threshold is, the smaller the probability that the display device 200 is awakened by mistake is, and the smaller the difference between the wake-up value and the wake-up value threshold is, the larger the probability that the display device 200 is awakened by mistake is.
The process of the display device 200 obtaining the current system volume may refer to step S502, which is not described herein.
And S1404, adjusting the amplitude of the audio of the wake-up cue according to the difference value between the wake-up value and the wake-up value threshold and the current system volume to obtain the audio to be played of the wake-up cue.
The display device 200 may adjust the amplitude of the audio of the wake-up cue according to the procedure shown in fig. 15, specifically as follows:
s1501, acquiring the audio of the wake-up cue.
Step S1501 may refer to step S1001, which is not described herein.
S1502, according to the difference value between the wake-up value and the wake-up value threshold and the current system volume, the amplitude of the audio of the wake-up cue is adjusted to obtain the audio to be played of the wake-up cue.
The display device 200 dynamically adjusts the amplitude of the audio of the wake-up cue according to the difference value between the wake-up value and the wake-up value threshold and the current system volume, so as to obtain the audio to be played of the wake-up cue. For example, the amplitude of the audio of the wake-up cue is adjusted by adjusting the gain of the audio of the wake-up cue.
Based on the above-mentioned mode of adjusting the amplitude of the audio itself of the wake-up cue, it is convenient to set finer amplitude adjustment granularity, and the volume of the wake-up cue can be more accurate to suit the current environment.
The display device 200 may adjust the amplitude of the audio of the wake-up cue according to the procedure shown in fig. 16, specifically as follows:
s1601, determining the amplitude of the audio to be played of the wake-up cue according to the difference value between the wake-up value and the wake-up value threshold and the current system volume.
The preset audio of the wake-up alert sound and the preset manner of the preset audio may refer to step S1001, and the display device 200 may determine the amplitude of the audio to be played of the wake-up alert sound to be played according to the difference between the wake-up value and the wake-up value threshold and the current system volume.
S1602, obtaining the preset audio of the wake-up cue with corresponding amplitude according to the amplitude of the audio to be played of the wake-up cue.
Step S1602 may refer to step S1102, which is not described herein.
The process by which the display device 200 adjusts the amplitude of the audio of the wake-up cue according to the difference between the wake-up value of the user's voice and the wake-up value threshold and the current system volume will be described with reference to fig. 17.
The display device 200 sets a first difference threshold and a second difference threshold, and a first volume threshold and a second volume threshold. The first difference threshold is smaller than the second difference threshold, and the first volume threshold is larger than the second volume threshold.
The display device 200 performs step S1701 to determine whether the difference between the wake-up value of the user voice and the wake-up value threshold is smaller than the first difference threshold.
When the difference is smaller than the first difference threshold, the display device 200 performs step S1702 to determine whether the current system volume is greater than the first volume threshold.
When the difference is smaller than the first difference threshold and the current system volume is larger than the first volume threshold, the display device 200 executes step S1703 to reduce the amplitude of the audio of the wake-up cue to obtain the audio to be played of the wake-up cue.
The difference being less than the first difference threshold indicates that the probability of the display device 200 being awakened by mistake is high. The current system volume is greater than the first volume threshold, which indicates that the display device 200 has a greater volume of the wake-up alert sound if the current system volume is used to directly play the audio of the wake-up alert sound. At this time, when the display device 200 is awakened by mistake with a high probability, the awakening prompt tone is played at a high volume, and a strong sense of intrusion is easily caused to the user. To solve this problem, the amplitude of the audio of the wake-up cue is reduced to reduce the volume of the wake-up cue, thereby reducing the disturbance to the user.
When the difference is smaller than the first difference threshold and the current system volume is smaller than or equal to the first volume threshold, the display device 200 performs step S1704 without adjusting the amplitude of the audio of the wake-up cue.
The difference being smaller than the first volume threshold indicates that the probability of the display device 200 being awakened by mistake is high. The current system volume is less than or equal to the first volume threshold, which indicates that if the display device 200 directly plays the audio of the wake-up alert sound using the current system volume, the volume of the wake-up alert sound is not excessively large. At this time, in the case where the display apparatus 200 is awakened by mistake with a high probability, the wake-up cue sound is played at a volume that is not excessively large, and the wake-up cue sound at the volume does not cause an excessive sense of intrusion to the user. Accordingly, the display device 200 does not need to adjust the volume of the wake-up cue, and the display device 200 does not adjust the amplitude of the audio of the wake-up cue.
When the difference is greater than or equal to the first difference threshold, the display apparatus 200 performs step S1705 to determine whether the difference is greater than or equal to the second difference threshold.
When the difference is greater than or equal to the first difference threshold and the difference is less than the second difference threshold, the display device 200 performs step S1704 without adjusting the amplitude of the audio of the wake-up cue.
The difference value is greater than or equal to the first difference threshold value and less than the second difference threshold value, and it may be considered that the display apparatus 200 may not easily accurately determine whether or not the awakened device meets the user intention within the threshold value interval. At this time, when it cannot be accurately determined whether the display apparatus 200 is awakened to meet the user's intention, the display apparatus 200 does not adjust the volume of the awakening alert sound, and the display apparatus 200 does not adjust the amplitude of the audio of the awakening alert sound, so as to reduce the data processing process.
When the difference is greater than or equal to the second difference threshold, the display device 200 performs step S1706 to determine whether the current system volume is less than the second volume threshold.
When the difference is greater than or equal to the second difference threshold and the current system volume is greater than or equal to the second volume threshold, the display device 200 performs step S1704 without adjusting the amplitude of the audio of the wake-up alert sound.
The difference being greater than or equal to the second difference threshold indicates that the probability of the display device 200 being awakened by mistake is low. The current system volume is greater than or equal to the second volume threshold, which indicates that if the display device 200 directly plays the audio of the wake-up alert sound using the current system volume, the volume of the wake-up alert sound is not too small. When the display device 200 is awakened by mistake with a small probability, the awakening cue sound is played at a volume that is not too small, and the awakening cue sound at the volume does not cause excessive sense of invasiveness to the user. Accordingly, the display device 200 does not need to adjust the volume of the wake-up cue, and the display device 200 does not adjust the amplitude of the audio of the wake-up cue.
When the difference is greater than or equal to the second difference threshold and the current system volume is less than the second volume threshold, the display device 200 executes step S1707 to increase the amplitude of the audio of the wake-up alert sound, and obtain the audio to be played of the wake-up alert sound.
The difference being greater than or equal to the second difference threshold indicates that the probability of the display device 200 being awakened by mistake is low. The current system volume is less than the second volume threshold, which indicates that the display device 200 has a smaller volume of the wake-up alert sound if the audio of the wake-up alert sound is directly played using the current system volume. When the display device 200 is awakened by mistake with a small probability, the awakening prompt tone is played at a small volume, which easily causes that the user cannot perceive the awakening prompt tone, and thus the display device 200 is mistakenly considered not to be awakened. To solve this problem, the volume of the wake-up cue is increased by increasing the amplitude of the audio of the wake-up cue, thereby improving the perception of the wake-up cue by the user.
S1405, controlling a loudspeaker to play the audio to be played of the wake-up cue at the current system volume.
After obtaining the audio to be played of the wake-up alert sound based on step S1404, the display device 200 plays the audio to be played of the wake-up alert sound at the current system volume, where the volume of the wake-up alert sound is affected by the current system volume and the volume of the audio to be played of the wake-up alert sound itself.
When the current system volume is smaller than the third volume threshold, the display device 200 displays a popup window, and displays prompt information in the popup window, where the prompt information is used to prompt the user that the current system volume is too small, resulting in that the volume of the wake-up prompt tone is too small, and the system volume can be increased, where the third volume threshold is smaller than or equal to the second volume threshold. The popup window displayed by the display device 200 may refer to the popup window 1301 shown in fig. 13 (1), which is not described herein.
When the current system volume is greater than the fourth volume threshold, the display device 200 displays a popup window, and displays prompt information in the popup window, where the prompt information is used to prompt the user that the current system volume is too large, resulting in that the volume of the wake-up prompt tone is too large, and the system volume can be reduced, where the fourth volume threshold is greater than or equal to the first volume threshold. The popup window displayed by the display device 200 may refer to the popup window 1302 shown in (2) of fig. 13, which is not described herein.
The display device 200 determines the style of the popup window displayed based on the difference between the wake-up value of the user's voice and the wake-up value threshold.
When the difference is less than the first difference threshold, the display device 200 displays a first style of pop-up window. When the difference is greater than or equal to the second difference threshold, the display device 200 displays a second pattern of popups. When the difference is greater than or equal to the first difference threshold and less than the second difference threshold, a pop-up window of the first style may be displayed, or a pop-up window of the second style may be displayed. Wherein the popup window of the second style is more perceptible to the user than the popup window of the first style.
For example, the second style of window is larger in size than the first style of window. Taking the display device 200 as a television, taking the example that the current system volume is smaller than the third volume threshold, the popup window of the first style displayed by the display device 200 may be the popup window 1801 shown in (1) in fig. 18, and the popup window of the second style displayed by the display device 200 may be the popup window 1802 shown in (2) in fig. 18. Both the pop-up window 1801 and the pop-up window 1802 include a prompt message, such as "system volume is too low, please turn up the system volume". The size of the pop-up window 1802 is larger than the size of the pop-up window 1801 to make the displayed pop-up window 1801 less noticeable to the user and to make the displayed pop-up window 1802 more noticeable to the user.
As another example, the second style of popup window is more visible than the first style of popup window. Taking the display device 200 as a television, taking the example that the current system volume is smaller than the third volume threshold, the popup window of the first style displayed by the display device 200 may be the popup window 1901 shown in (1) in fig. 19, and the popup window of the second style displayed by the display device 200 may be the popup window 1902 shown in (2) in fig. 19. Both popup 1901 and popup 1902 include prompts such as "system volume is too low, please turn up system volume". The size of the popup 1901 may be the same as the size of the popup 1902. The popup 1901 is displayed in the lower right corner of the screen and the popup 1902 is displayed in the center of the screen, so that the displayed popup 1901 is less noticeable to the user and the displayed popup 1902 is more noticeable to the user.
For another example, the second style of window is larger in size than the first style of window and the display position is more apparent. Taking the display device 200 as a television, taking the example that the current system volume is smaller than the third volume threshold, the popup window of the first style displayed by the display device 200 may be the popup window 2001 shown in (1) of fig. 20, and the popup window of the second style displayed by the display device 200 may be the popup window 2002 shown in (2) of fig. 20. Both popup 2001 and popup 2002 include prompts such as "system volume is too low, please turn up system volume". The size of the popup window 2001 may be smaller than the size of the popup window 2002. The popup 2001 is displayed in the lower right corner of the screen, and the popup 2002 is displayed in the center of the screen, so that the displayed popup 2001 is less noticeable to the user, and the displayed popup 2002 is more noticeable to the user.
The display device 200 may adjust the volume of the wake-up cue in combination with the probability of being awakened by mistake, the current system volume, and the degree of noise in the environment.
The display device 200 may adjust the volume of the wake-up alert sound according to the procedure shown in fig. 21, which includes the following steps:
s2101, collecting user voice.
S2102, calculating a wake-up value corresponding to the voice of the user.
Steps S2101-S2102 are similar to steps S501 and S601, and steps S701-S702, and are not described here.
S2103, when the wake-up value is larger than or equal to the wake-up value threshold, acquiring a difference value between the wake-up value and the wake-up value threshold, a signal-to-noise ratio of the user voice and the current system volume.
The difference between the wake-up value and the wake-up value threshold is used to characterize the probability that the display device 200 is awakened by mistake. The larger the difference between the wake-up value and the wake-up value threshold is, the smaller the probability that the display device 200 is awakened by mistake is, and the smaller the difference between the wake-up value and the wake-up value threshold is, the larger the probability that the display device 200 is awakened by mistake is.
The process of the display device 200 obtaining the signal-to-noise ratio of the user' S voice and the current system volume may refer to step S502, which is not described herein.
S2104, adjusting the amplitude of the audio of the wake-up cue according to the difference value of the wake-up value and the wake-up value threshold, the signal-to-noise ratio of the user voice and the current system volume, and obtaining the audio to be played of the wake-up cue.
The display device 200 may adjust the amplitude of the audio of the wake-up cue according to the procedure shown in fig. 22, specifically as follows:
s2201, acquiring the audio of the wake-up prompt tone.
Step S2201 may refer to step S1001, which is not described herein.
S2202, adjusting the amplitude of the audio of the wake-up cue according to the difference value between the wake-up value and the wake-up value threshold, the signal-to-noise ratio of the user voice and the current system volume, and obtaining the audio to be played of the wake-up cue.
The display device 200 dynamically adjusts the amplitude of the audio of the wake-up cue according to the difference value between the wake-up value and the wake-up value threshold, the signal-to-noise ratio of the user voice and the current system volume, and obtains the audio to be played of the wake-up cue. For example, the amplitude of the audio of the wake-up cue is adjusted by adjusting the gain of the audio of the wake-up cue.
Based on the above-mentioned mode of adjusting the amplitude of the audio itself of the wake-up cue, it is convenient to set finer amplitude adjustment granularity, and the volume of the wake-up cue can be more accurate to suit the current environment.
The display device 200 may adjust the amplitude of the audio of the wake-up cue according to the procedure shown in fig. 23, specifically as follows:
s2301, determining the amplitude of the audio to be played of the wake-up cue according to the difference between the wake-up value and the wake-up value threshold, the signal-to-noise ratio of the user voice and the current system volume.
The preset audio of the wake-up alert sound and the preset manner of the preset audio may refer to step S1101, and the display device 200 may determine the amplitude of the audio to be played of the wake-up alert sound to be played according to the difference between the wake-up value and the wake-up value threshold, the signal to noise ratio of the user voice, and the current system volume.
S2302, according to the amplitude of the audio to be played of the wake-up cue, obtaining the preset audio of the wake-up cue with the corresponding amplitude.
Step S2302 may refer to step S1102, which is not described herein.
The process by which the display device 200 adjusts the amplitude of the audio of the wake-up cue according to the difference between the wake-up value of the user's voice and the wake-up value threshold, the current system volume, and the signal-to-noise ratio of the user's voice will be described with reference to fig. 24.
The display device 200 sets a first difference threshold and a second difference threshold, a first signal-to-noise ratio threshold and a second signal-to-noise ratio threshold, and a first volume threshold and a second volume threshold. The first difference threshold is smaller than the second difference threshold, the first signal-to-noise ratio threshold is larger than the second signal-to-noise ratio threshold, and the first volume threshold is larger than the second volume threshold.
The display device 200 performs step S2401 to determine whether the difference between the wake-up value of the user' S voice and the wake-up value threshold is smaller than the first difference threshold.
When the difference is smaller than the first difference threshold, the display device 200 performs step S2402 to determine whether the signal-to-noise ratio of the user' S voice is greater than or equal to the first signal-to-noise ratio threshold.
When the signal-to-noise ratio of the user' S voice is greater than or equal to the first signal-to-noise ratio threshold, the display device 200 performs step S2403 to determine whether the current system volume is greater than or equal to the first volume threshold.
When the signal-to-noise ratio of the user voice is greater than or equal to the first signal-to-noise ratio threshold and the current system volume is greater than or equal to the first volume threshold, the display device 200 executes step S2404 to reduce the amplitude of the audio of the wake-up alert sound, and obtain the audio to be played of the wake-up alert sound.
The difference being less than the first difference threshold indicates that the probability of the display device 200 being awakened by mistake is high. The signal-to-noise ratio of the user's voice being greater than or equal to the first signal-to-noise ratio threshold indicates that the display device 200 is in a quieter environment. The current system volume is greater than or equal to the first volume threshold, which indicates that the display device 200 has a higher volume of the wake-up alert sound if the audio of the wake-up alert sound is directly played using the current system volume. In a quieter environment, the display device 200 plays the wake-up cue sound at a larger volume in the case of a high probability of being awakened by mistake, which is likely to cause a stronger sense of intrusion to the user. To solve this problem, the amplitude of the audio of the wake-up cue is reduced to reduce the volume of the wake-up cue, thereby reducing the user's surprise.
When the signal-to-noise ratio of the user' S voice is greater than or equal to the first signal-to-noise ratio threshold and the current system volume is less than the first volume threshold, the display apparatus 200 performs step S2405 without adjusting the amplitude of the audio of the wake-up cue.
The difference being less than the first difference threshold indicates that the probability of the display device 200 being awakened by mistake is high. The signal-to-noise ratio of the user's voice being greater than or equal to the first signal-to-noise ratio threshold indicates that the display device 200 is in a quieter environment. The current system volume is less than the first volume threshold, which indicates that if the display device 200 directly plays the audio of the wake-up alert sound using the current system volume, the volume of the wake-up alert sound is not too great. In a relatively quiet environment, the display device 200 plays the wake-up cue sound at a volume that is not too large in the case of a high probability of being awakened by mistake, and the sense of intrusion caused to the user is not too strong, so that the display device 200 does not need to adjust the volume of the wake-up cue sound.
When the signal-to-noise ratio of the user' S voice is smaller than the first signal-to-noise ratio threshold, the display device 200 performs step S2405 without adjusting the amplitude of the audio of the wake-up cue.
The signal-to-noise ratio of the user's voice being less than the first signal-to-noise ratio threshold indicates that the environment in which the display device 200 is located is not too quiet. In an environment where the display device 200 is not too quiet, if the audio of the wake-up cue is directly played at a large volume in the case where the display device 200 is erroneously waken up, the sense of intrusion caused to the user is not too strong, and thus, the display device 200 does not need to turn down the volume of the wake-up cue. If the audio of the wake-up cue is directly played at a small volume, the probability of affecting the user is low even if the user's perception of the wake-up cue is weak because of the large probability of being erroneously awakened, and thus, the display apparatus 200 does not need to turn up the volume of the wake-up cue. Accordingly, the display apparatus 200 does not need to adjust the volume of the wake-up cue.
When the difference is greater than or equal to the first difference threshold, the display apparatus 200 executes step S2406 to determine whether the difference is less than the second difference threshold.
When the difference is smaller than the second difference threshold, the display apparatus 200 performs step S2405 without adjusting the amplitude of the audio of the wake-up cue.
The difference value is greater than or equal to the first difference threshold value and less than the second difference threshold value, and it may be considered that the display apparatus 200 may not easily accurately determine whether or not the awakened device meets the user intention within the threshold value interval. At this time, when it cannot be accurately determined whether the display apparatus 200 is awakened to meet the user's intention, the display apparatus 200 does not adjust the volume of the awakening alert sound, and the display apparatus 200 does not adjust the amplitude of the audio of the awakening alert sound, so as to reduce the data processing process.
When the difference is greater than or equal to the second difference threshold, the display device 200 performs step S2407 to determine whether the signal-to-noise ratio of the user' S voice is greater than or equal to the second signal-to-noise ratio threshold.
When the signal-to-noise ratio of the user' S voice is smaller than the second signal-to-noise ratio threshold, the display device 200 executes step S2408 to determine whether the current system volume is smaller than the second volume threshold.
When the signal-to-noise ratio of the user voice is smaller than the second signal-to-noise ratio threshold and the current system volume is smaller than the second volume threshold, the display device 200 executes step S2409 to increase the amplitude of the audio of the wake-up cue to obtain the audio to be played of the wake-up cue.
The difference being greater than or equal to the second difference threshold indicates that the probability of the display device 200 being awakened by mistake is low. The signal-to-noise ratio of the user's voice being less than the second signal-to-noise ratio threshold value indicates that the environment in which the display device 200 is located is noisy, i.e., the environment in which the display device 200 is located is noisy. The current system volume is less than the second volume threshold, which indicates that the display device 200 has a smaller volume of the wake-up alert sound if the audio of the wake-up alert sound is directly played using the current system volume. In a noisy environment, when the display device 200 is awakened by mistake with a small probability, the awakening prompt tone is played at a small volume, which easily results in that the user cannot perceive the awakening prompt tone, and thus the display device 200 is mistakenly considered not to be awakened. To solve this problem, the volume of the wake-up cue is increased by increasing the amplitude of the audio of the wake-up cue, thereby improving the perception of the wake-up cue by the user.
When the signal-to-noise ratio of the user' S voice is less than the second signal-to-noise ratio threshold and the current system volume is greater than or equal to the second volume threshold, the display device 200 performs step S2405 without adjusting the amplitude of the audio of the wake-up cue.
The difference being greater than or equal to the second difference threshold indicates that the probability of the display device 200 being awakened by mistake is low. The signal-to-noise ratio of the user's voice being less than the second signal-to-noise ratio threshold value indicates that the environment in which the display device 200 is located is noisy, i.e., the environment in which the display device 200 is located is noisy. The current system volume is greater than or equal to the second volume threshold, which indicates that the volume of the wake-up alert sound is not too small if the display device 200 directly plays the audio of the wake-up alert sound using the current system volume. In a noisy environment, the display device 200 plays the wake-up cue sound at a volume that is not too small in the event of a small probability of being awakened by mistake, and it is considered that the user may also be made aware of the wake-up cue sound. Accordingly, the display device 200 does not need to adjust the volume of the wake-up cue, and the display device 200 does not adjust the amplitude of the audio of the wake-up cue.
When the signal-to-noise ratio of the user' S voice is greater than or equal to the second signal-to-noise ratio threshold, the display device 200 performs step S2405 without adjusting the amplitude of the audio of the wake-up cue.
The signal-to-noise ratio of the user's voice being greater than or equal to the second signal-to-noise ratio threshold indicates that the environment in which the display device 200 is located is not too noisy. In an environment that is not excessively noisy, if the display device 200 directly plays the audio of the wake-up cue at a relatively large volume under the condition that the probability of being awakened by mistake, the perception of the wake-up cue by the user can be enhanced due to the fact that the probability of being awakened by mistake is small. Thus, the display device 200 does not need to turn down the volume of the wake-up cue. If the audio of the wake-up prompt tone is directly played at a small volume, the environment is not too noisy, and the user can also feel the wake-up prompt tone. Thus, the display device 200 does not need to turn up the volume of the wake-up cue. Accordingly, the display apparatus 200 does not need to adjust the volume of the wake-up cue.
And S2105, controlling a loudspeaker to play the audio to be played of the wake-up prompt tone at the current system volume.
After obtaining the audio to be played of the wake-up cue based on step S2104, the display apparatus 200 plays the audio to be played of the wake-up cue at the current system volume, and the volume of the wake-up cue is affected by the current system volume and the volume of the audio to be played of the wake-up cue itself.
When the current system volume is smaller than the third volume threshold, the display device 200 displays a popup window, and displays prompt information in the popup window, where the prompt information is used to prompt the user that the current system volume is too small, resulting in that the volume of the wake-up prompt tone is too small, and the system volume can be increased, where the third volume threshold is smaller than or equal to the second volume threshold. The popup window displayed by the display device 200 may refer to the popup window 1301 shown in fig. 13 (1), which is not described herein.
When the current system volume is greater than the fourth volume threshold, the display device 200 displays a popup window, and displays prompt information in the popup window, where the prompt information is used to prompt the user that the current system volume is too large, resulting in that the volume of the wake-up prompt tone is too large, and the system volume can be reduced, where the fourth volume threshold is greater than or equal to the first volume threshold. The popup window displayed by the display device 200 may refer to the popup window 1302 shown in (2) of fig. 13, which is not described herein.
The display device 200 determines the style of the popup window displayed based on the difference between the wake-up value of the user's voice and the wake-up value threshold.
When the difference is less than the first difference threshold, the display device 200 displays a first style of pop-up window. When the difference is greater than or equal to the second difference threshold, the display device 200 displays a second pattern of popups. When the difference is greater than or equal to the first difference threshold and less than the second difference threshold, a pop-up window of the first style may be displayed, or a pop-up window of the second style may be displayed. Wherein the popup window of the second style is more perceptible to the user than the popup window of the first style. The first and second patterns of popup windows may refer to fig. 18, 19 and 20, and are not described herein.
The operating state of the display apparatus 200 includes a first state including the display apparatus 200 being in a mute mode and the display apparatus 200 being in an unmuted mode and the current system volume being 0. When the display apparatus 200 is in the first state, if the display apparatus 200 is awake, subject to the limitation of the first state, the speaker does not sound when the display apparatus 200 plays the awake alert sound. The user may easily misunderstand that the display device 200 is not responding because the user cannot perceive the wake-up cue. Even if the user perceives that the display device 200 is awake, the reason why the speaker cannot be quickly caused to be unvoiced is because it is in a mute mode or the current system volume is 0, so that the speaker cannot be quickly caused to be unvoiced.
In order to solve the above problem, the display device 200 may perform a wake-up response according to the flow shown in fig. 25, which includes the following specific steps:
s2501, when the screen is in a bright state and the screen is in a first state, collecting user voice.
S2502, when recognizing that the voice of the user contains the wake-up word, displaying a popup window according to the first state.
Step S2501 is similar to step S501, and the process of recognizing the wake-up word in step S2502 is similar to steps S601 to S602, and will not be repeated here.
The task manager of the display device 200 stores a volume status table for recording the current system volume, whether to turn on a mute mode, and the like. Wherein, the first flag bit in the volume state table is used for indicating whether to start the mute mode, for example, if the value on the first flag bit is 1, the mute mode is started; if the value on the first flag bit is 0, the mute mode is turned off. Determining whether to be in a mute mode by identifying the value on the first flag bit, e.g., if the value on the first flag bit is 1, then in the mute mode; if the value on the first flag bit is 0, then the non-silent mode is in.
The display apparatus 200 performs a process of acquiring audio to be played of the wake-up cue when the wake-up value is greater than or equal to the wake-up value threshold.
Wherein the display apparatus 200 may recognize whether to be in a mute mode before performing the audio to be played that acquires the wake-up cue. If the display device 200 recognizes that it is in the mute mode, the audio to be played to acquire the wake-up cue is not executed; if the display apparatus 200 recognizes that it is in the non-mute mode, audio to be played to acquire a wake-up cue is performed.
The display apparatus 200 may recognize whether to be in a mute mode in performing the process of acquiring the audio to be played of the wake-up cue. For example, the audio to be played for acquiring the wake-up cue and the recognition of whether or not in the mute mode are respectively performed in two parallel processes/threads. If the display device 200 recognizes that the audio is in the mute mode, terminating execution of the audio to be played for which the wake-up cue is acquired; if the display apparatus 200 recognizes that it is in the non-mute mode, audio to be played to acquire a wake-up cue is performed. Thus, continued occupation of computing resources of the display device 200 may be avoided when the silent mode is identified. And the audio to be played of the wake-up prompt tone can be obtained rapidly when the non-mute mode is identified.
The display apparatus 200 may recognize whether to be in the mute mode after performing the process of acquiring the audio to be played of the wake-up cue. Thus, when recognizing that the display device 200 is in the mute mode, once the mute mode is released, the audio to be played of the wake-up cue can be directly used after being awakened, so that the efficiency of playing the wake-up cue after the mute mode is released is improved.
If the display device 200 recognizes the wake word, a pop-up window is displayed to prompt the user that the user has been awakened through the pop-up window.
In some embodiments, when the display device 200 is in the mute mode, the display device 200 is in an awake state, displays a pop-up window, does not deactivate the mute mode, and does not adjust the system volume in response to the wake-up word. The display device 200 is in an awake state and may interact with a user in response to other far-field voice instructions of the user. The popup is used for prompting the user to wake up and is in a mute mode.
The popup includes prompt information including information (for convenience of distinction, simply referred to as first information) for prompting the display device 200 to be awakened and in a mute mode. Taking the display device 200 as an example of a television, the display device 200 may display a pop-up window 2601 as shown in fig. 26, where the pop-up window 2601 includes first information, such as "wake-up, currently in a mute mode". Thus, the display device 200 wakes up when in the mute mode, and although the wake-up cue sound played is silent, the user may be prompted through a pop-up window that the user has been woken up and that the user is currently in the mute mode, so that the user is informed that the wake-up is performed but the reason for not waking up the cue sound is in the mute mode.
The display device 200 determines the style of the popup window displayed based on the difference between the wake-up value of the user's voice and the wake-up value threshold. When the difference is less than the first difference threshold, the display device 200 displays a first style of pop-up window. When the difference is greater than or equal to the second difference threshold, the display device 200 displays a second pattern of popups. When the difference is greater than or equal to the first difference threshold and less than the second difference threshold, the display device 200 may display a first style of popup or may display a second style of popup. The popup window of the second style is more perceptible to the user than the popup window of the first style. The popup window of the first style and the popup window of the second style may refer to fig. 18, 19 and 20, except that the first information is displayed in the popup window, which is not described herein.
The display device 200 may also display a prompt message in the popup window according to the current system volume, and the prompt message further includes information (for convenience of distinction, simply referred to as second information) for prompting the user to adjust the system volume. The display device 200 may display the second information according to the flow shown in fig. 27, with the following specific steps:
s2701, the current system volume is acquired.
The current system volume is the system volume before the display apparatus 200 turns on the mute mode. The process of the display device 200 obtaining the current system volume may refer to step S502, which is not described herein.
S2702, displaying second information in the popup window according to the current system volume.
The display device 200 sets a first volume threshold and a second volume threshold, wherein the first volume threshold is greater than the second volume threshold.
And when the current system volume is greater than or equal to the first volume threshold, the second information is used for prompting the user to reduce the system volume.
The front system volume being greater than or equal to the first volume threshold indicates that the system volume of the display device 200 is large, and after unmuted, the display device 200 will play audio at this large volume, which is easily annoying to the user. To solve this problem, the display device 200 displays second information to prompt the user that the current system volume is large, and the system volume may be turned down. Taking the display device 200 as an example of a television, the display device 200 may display a pop-up window 2801 as shown in (1) in fig. 28, where the pop-up window 2801 includes first information, such as "wake-up, currently in mute mode", and the pop-up window 2801 further includes second information, such as "loud volume, and suggested to be turned down".
And when the current system volume is smaller than the second volume threshold, the second information is used for prompting the user to increase the system volume.
The front system volume being less than the second volume threshold indicates that the system volume of the display device 200 is less, and after de-muting, the display device 200 will play audio at the lower volume, which may not be easily perceived by the user. To solve this problem, the display device 200 displays the second information to prompt the user that the current system volume is small, and the system volume may be increased. Taking the display device 200 as an example of a television, the display device 200 may display a pop-up window 2802 as shown in (2) in fig. 28, where the pop-up window 2802 includes first information, such as "wake-up, currently in mute mode", and the pop-up window 2802 further includes second information, such as "low volume, and suggested to be turned up".
After the pop-up window is displayed, the display device 200 may play an audio play after releasing the mute mode.
The display device 200 may release the mute mode according to the flow shown in fig. 29, which is specifically as follows:
s2901, responding to a volume adjustment instruction of a user, removing the mute mode, and adjusting the current system volume to obtain the adjusted system volume.
The volume adjustment command may be an increase volume command and a decrease volume command. Taking the control device 100 as a remote controller for example, the user may input a volume adjustment instruction through a volume key on the remote controller.
The display device 200 delivers the decoded audio data to a speaker for playback in response to the volume adjustment instruction to cancel the mute mode. The display device 200 also adjusts the current system volume in response to the volume adjustment instruction. For example, if the volume adjustment command is an increase volume command, the system volume is turned up. If the volume adjustment command is a decrease volume command, the system volume is decreased.
S2902, playing the audio at the adjusted system volume.
After the display apparatus 200 releases the mute mode, the system volume is the adjusted system volume, and the display apparatus 200 plays audio at the adjusted system volume. For example, the current system volume is 2, the volume adjustment instruction is an increase volume instruction, such as volume increase 1, in response to which the display device 200 releases the mute mode and plays audio at volume 3 after releasing the mute mode.
After the display apparatus 200 releases the mute mode, the system volume is adjusted in response to the volume adjustment instruction input again by the user.
Based on the mode of eliminating the mute mode, a user does not need to input an additional instruction of eliminating the mute, the mute mode can be directly eliminated when the system volume is adjusted, and the user operation is more convenient and smoother.
The display device 200 may release the mute mode according to the flow shown in fig. 30, which is specifically as follows:
s3001, in response to a release instruction input by the user, releases the mute mode.
The release instruction is for instructing to release the mute mode. Taking the control device 100 as a remote controller for example, the user may input a control instruction based on a mute key on the remote controller, and the display device 200 determines that the control instruction is a cancel instruction when recognizing that it is in the mute mode in response to the control instruction. In response to the release instruction, the display device 200 delivers the decoded audio data to a speaker for playback to release the mute mode.
S3002, play audio at the current system volume.
After the mute mode is released, the display device 200 plays the audio at the current system volume. After the display apparatus 200 releases the mute mode, the system volume may be adjusted in response to a volume adjustment instruction input again by the user.
The display device 200 may also adjust the system volume in response to the volume adjustment instruction of the user before step S3001, to obtain the adjusted system volume. At this time, the display device 200 is still in the mute mode, and after the user system volume is adjusted, a release command is input, and after the mute is released, the display device 200 plays the audio at the current system volume, which is actually the adjusted system volume. Therefore, when the current system volume is too large or too small, the user can adjust the system volume to a proper volume before actually playing the audio to play the audio directly by using the proper volume after the display device 200 is unmuted, so as to ensure the listening quality of the user.
In other embodiments, when the display device 200 is in the mute mode, the display device 200 is in an incompletely awake state, displays a pop-up window, does not deactivate the mute mode, and does not adjust the system volume in response to the wake-up word. The incompletely awake state means that the display apparatus 200 starts a pop-up window function in response to an awake word, and after displaying the pop-up window, exits interaction with the user without responding to other far-field voice instructions of the user. The popup window is used for prompting the user to be in the mute mode currently and to wake up again after the mute mode is released. In this way, the silent mode set by the user can be prevented from being damaged, the user is prompted through the popup window with lower invasion feeling, the influence of the silent mode on the response of the display device 200 to the far-field voice command of the user is avoided by the mode that the user needs to wake up again after the silent mode is released, and the far-field voice interaction effect of the display device 200 and the user is improved.
In other embodiments, when the display apparatus 200 is in the mute mode, the display apparatus 200 is in the awake state in response to the wake-up word, automatically releases the mute mode, displays a pop-up window, and adjusts the system volume to a preset volume. The display device 200 is in an awake state and may interact with a user in response to other far-field voice instructions of the user. The popup is used to alert the user that he has been awakened and is in a silent mode, but is currently in an unmuted mode. The preset volume may be a default system volume, or may be a system volume stored when the mute mode is started. In this way, the display device 200 does not play the wake-up cue, but uses a pop-up window as a transition to cue that the user has been woken up so as not to surprise the user. Moreover, the display device 200 may continue to interact with the user without the user waking up again.
In some embodiments, when the display device 200 is in the non-mute state and the current system volume is 0, the display device 200 is in the awake state, displays a pop-up window, and does not adjust the system volume in response to the wake-up word. The popup is used to prompt the user that the user has been awakened and the system volume is 0. At this point, the display device 200 is in an awake state and may interact with the user in response to other far-field voice instructions of the user.
The popup includes prompt information including information for prompting the display device 200 to be awakened and the current system volume is 0 (for convenience of distinction, simply referred to as third information). Taking the display device 200 as an example of a television, the display device 200 may display a pop-up window 3101 as shown in fig. 31, where the pop-up window 3101 includes third information, such as "wake-up, and the current system volume is 0".
In other embodiments, when the display device 200 is in the non-mute state and the current system volume is 0, the display device 200 is in an incompletely awake state, displays a pop-up window, and does not adjust the system volume in response to the wake-up word. The incompletely awake state means that the display apparatus 200 starts a pop-up window function in response to an awake word, and after displaying the pop-up window, exits interaction with the user without responding to other far-field voice instructions of the user. The pop-up window is used for prompting the user that the system volume is 0 and that the user needs to wake up again after increasing the volume. In this way, the user can be prompted through the popup window with lower intrusion feeling, and the influence of the system volume of 0 on the response of the display device 200 to the far-field voice command of the user is avoided by the mode that the user needs to wake up again after the volume is increased, so that the far-field voice interaction effect of the display device 200 and the user is improved.
In other embodiments, when the display apparatus 200 is in the non-mute state and the current system volume is 0, the display apparatus 200 is in the wake-up state in response to the wake-up word, automatically increases the system volume to the preset volume, and displays the pop-up window. The pop-up window is used to prompt the user that the user has been awakened and the system volume is 0, but is currently increased to the first volume. The first volume may be a default system volume, or may be a system volume stored when the mute mode is started. At this point, the display device 200 is in an awake state and may interact with the user in response to other far-field voice instructions of the user. In this way, the display device 200 does not play the wake-up cue, but uses a pop-up window as a transition to cue that the user has been woken up so as not to surprise the user. Moreover, the display device 200 may continue to interact with the user without the user waking up again.
The display device 200 may also perform a wake-up response according to the flow shown in fig. 32, which includes the following specific steps:
s3201, when the user is in a bright screen state and in a mute mode, collecting user voice.
S3202, when recognizing that the user voice contains the wake-up word, the mute mode is released, and after the mute mode is released, the audio to be played of the wake-up prompt tone or the prompt voice is played at a first volume.
Step S3201 is similar to step S501, and the process of recognizing the wake-up word in step S3202 is similar to steps S601-S602, and will not be repeated here.
In some embodiments, when the display device 200 is in the mute mode, the display device 200 is in the awake state in response to the wake word, automatically releases the mute mode, and plays the wake-up cue at the first volume. The first volume may be a default system volume, or may be a system volume value stored when the mute mode is started. In this way, the display device 200 may interact well with the user through the wake-up alert tone, and the display device 200 may continue to interact with the user without the user waking up again.
In other embodiments, when the display apparatus 200 is in the mute mode, the display apparatus 200 is in an incompletely awake state in response to the wake word, releases the mute mode, and plays the alert voice at a preset volume after releasing the mute mode, and resumes the mute mode after playing the first alert voice to maintain the mute mode. The incomplete wake state means that the display device 200 starts the first alert voice play function in response to the wake word, exits the interaction with the user after playing the first alert voice, and does not respond to other far-field voice instructions of the user. The preset volume value may be a default system volume value or a system volume value stored when the mute mode is started. The prompt voice is used for prompting the user to be in the mute mode currently and to wake up again after the mute mode is released. In this way, the silent mode set by the user can be prevented from being destroyed, the user is prompted by the first prompt voice with stronger interaction, and the influence of the silent mode on the response of the display device 200 to the far-field voice instruction of the user is avoided by the mode that the user needs to wake up again after the silent mode is released, so that the far-field voice interaction effect of the display device 200 and the user is improved.
The display device 200 may turn up the system volume based on the signal-to-noise ratio of the user's voice. If the signal-to-noise ratio of the user voice is larger than the designated signal-to-noise ratio threshold, the up-regulation amplitude of the system volume is the first amplitude, and the first volume is smaller. If the signal-to-noise ratio of the user voice is smaller than the specified signal-to-noise ratio threshold, the system volume is increased by a second amplitude, the second amplitude is larger than the first amplitude, and the first volume is larger. The display device 200 may turn up the system volume based on the difference between the wake-up value of the user's voice and the wake-up value threshold. If the difference is larger than the specified difference threshold, the up-regulation amplitude of the system volume is the first amplitude, and the first volume is larger. If the signal-to-noise ratio of the user voice is smaller than the specified difference threshold, the system volume is increased by a second amplitude, the second amplitude is smaller than the first amplitude, and the first volume is smaller. The display device 200 may also turn up the system volume based on the signal-to-noise ratio of the user's voice and the difference between the wake-up value of the user's voice and the wake-up value threshold. If the difference is greater than the specified difference threshold and the signal-to-noise ratio of the user voice is less than the specified signal-to-noise ratio threshold, the system volume is increased by a first magnitude, and the first volume is larger. If the difference is smaller than the specified difference threshold and the signal-to-noise ratio of the user voice is greater than the specified signal-to-noise ratio threshold, the system volume is increased by a second magnitude which is smaller than the first magnitude, and the first volume is smaller.
The display device 200 may also perform a wake-up response according to the flow shown in fig. 33, which includes the following specific steps:
s3301, when the user is in a bright screen state and in a non-mute mode and the current system volume is 0, the user voice is collected.
S3302, when recognizing that the user voice contains the wake-up word, the system volume is increased to a preset volume, and the audio to be played of the wake-up prompt tone or the prompt voice is played at the preset volume.
Step S3301 is similar to step S501, and the process of recognizing the wake-up word in step S3302 is similar to steps S601-S602, and is not repeated here.
In some embodiments, when the display apparatus 200 is in the non-mute state and the current system volume is 0, the display apparatus 200 is in the wake-up state in response to the wake-up word, automatically increases the system volume to a preset volume, and plays the wake-up cue at the preset volume. The third volume may be a default system volume. In this way, the display device 200 may interact well with the user through the wake-up alert tone, and the display device 200 may continue to interact with the user without the user waking up again.
In other embodiments, when the display apparatus 200 is in the non-mute state and the current system volume is 0, the display apparatus 200 is in the incompletely awake state in response to the wake-up word, automatically increases the system volume to a preset volume, and plays the alert voice at the preset volume. The incomplete wake state means that the display device 200 starts a prompt voice play function in response to the wake word, and exits interaction with the user after playing the prompt voice, without responding to other far-field voice instructions of the user. The preset volume may be a default system volume. The prompt voice is used for prompting the user that the system volume is 0 and that the user needs to wake up again after increasing the volume. In this way, the user can be prompted by the first prompt voice with stronger interaction and the influence of the system volume of 0 on the response of the display device 200 to the far-field voice instruction of the user is avoided by the mode that the user needs to wake up again after the system volume is increased, so that the far-field voice interaction effect of the display device 200 and the user is improved.
The operating modes of the display device 200 include a speaker mode, in which the display is in a rest state while the display device 200 is in the speaker mode, but may play sound. When the display apparatus 200 is in the speaker mode, if the display apparatus 200 is in the first state when it is awakened, the user may easily misunderstand that the display apparatus 200 is not awakened because the awakening prompt tone is not played due to the restriction of the first state.
In order to solve the above problem, the display device 200 may respond to the wake-up word according to the flow shown in fig. 34, which is specifically as follows:
s3401, when the user is in a screen-off state and in a mute mode, collecting the voice of the user.
S3402, detecting whether the voice of the user comprises a wake-up word or not in a sound box mode when the voice of the user is recognized.
Step S3401 is similar to step S501, and the process of recognizing the wake-up word in step S3402 is similar to steps S601-S602, and is not repeated here.
The display device 200 may determine whether to be in the speaker mode by identifying the status of the audio service. Wherein if the audio service is active, it is in audio mode. If the audio service is off, then it is in a non-speaker mode. The display device 200 may also identify whether it is in the speaker mode according to the device status information, for example, the display device 200 obtains the device status information through the device management module, where the device status information includes a flag bit of the speaker mode. When the flag bit is provided with a number 1, the display device 200 is in the speaker mode, and when the flag bit is empty, or provided with a number 0, the display device 200 is in the non-speaker mode. The display device 200 may also identify whether in a speaker mode based on other means, without limitation.
The display apparatus 200 performs a process of acquiring audio to be played of the wake-up cue when the wake-up value is greater than or equal to the wake-up value threshold.
The display device 200 may identify whether to be in the speaker mode before performing the audio to be played that acquires the wake-up cue. If the display device 200 recognizes that it is in the non-sound box mode, the audio to be played to acquire the wake-up cue is not executed; if the display device 200 recognizes that it is in the speaker mode, audio to be played is performed to acquire a wake-up cue. Thus, it is possible to avoid occupying the computing resources of the display device 200 when in the non-speaker mode.
The display device 200 may recognize whether it is in the speaker mode during the execution of the audio to be played that acquires the wake-up cue. For example, the audio to be played for obtaining the wake-up cue and the recognition of whether in the speaker mode are performed in two parallel processes/threads, respectively. If the display device 200 recognizes that the mode is in the non-sound box mode, the execution of the audio to be played for obtaining the wake-up cue is terminated; if the display device 200 recognizes that it is in the speaker mode, audio to be played is performed to acquire a wake-up cue. Thus, continued occupation of computing resources of the display device 200 may be avoided upon recognition of a non-speaker mode. And the audio to be played of the wake-up prompt tone can be obtained rapidly when the sound box mode is identified.
The display device 200 may recognize whether it is in the speaker mode after performing the process of acquiring the audio to be played of the wake-up cue. Therefore, when the display device 200 recognizes that the non-sound box mode is in, once the operation mode capable of playing the wake-up prompt tone is entered, the audio to be played of the wake-up prompt tone can be directly used after the wake-up prompt tone is awakened, so that the efficiency of playing the wake-up prompt tone is improved.
S3403, if the voice box is in the voice box mode, keeping the screen-off state, releasing the mute mode, and after the mute mode is released, playing the audio to be played of the wake-up prompt tone at the first volume.
The display device 200 remains in the off-screen state and responds to the user's voice by playing a wake-up alert sound to alert the user that it has been woken up.
The display device 200 obtains the audio to be played of the wake-up cue before the audio to be played of the wake-up cue is played.
The display device 200 may obtain the audio to be played of the wake-up cue having a specified amplitude, the specified amplitude being less than the amplitude threshold. The display device 200 plays the audio to be played of the wake-up alert sound at the current system volume or the default system volume, and since the amplitude of the audio to be played of the wake-up alert sound is smaller than the amplitude threshold, the volume of the audio to be played of the wake-up alert sound at the current system volume or the default system volume is also smaller. The manner that the display device 200 directly obtains the audio to be played of the wake-up cue with the specified amplitude can effectively improve the efficiency of obtaining the audio to be played of the wake-up cue, and further improve the efficiency of playing the audio to be played of the wake-up cue.
The display device 200 may adjust the amplitude of the audio of the wake-up alert according to the first volume and the related parameters of the user voice, to obtain the audio to be played of the wake-up alert. Wherein the relevant parameters of the user voice comprise the signal-to-noise ratio of the user voice and/or the difference value between the wake-up value of the user voice and the wake-up value threshold. The display apparatus 200 responds to the wake-up of the user by playing the audio to be played of the wake-up cue having a moderate volume, while reducing the sense of intrusion to the user. The display device 200 dynamically obtains the audio to be played of the wake-up prompt tone according to the first volume and the related parameters of the user voice, so that the volume of the played wake-up prompt tone can be more fit with the user requirement, and the experience of the user is improved.
The display device 200 may also respond to the wake-up word according to the flow shown in fig. 35, which specifically includes the following steps:
s3501, when the user is in the screen-off state, and in the non-mute mode and the current system volume is 0, collecting the user voice.
S3502, when recognizing that the user voice contains the wake-up word, detecting whether the voice box is in the sound box mode or not.
Steps S3501 and S3502 are similar to steps S3401 and S3402, and are not repeated here.
S3503, if the voice box is in the voice box mode, the voice box is kept in the screen-off state, the system volume is increased to the second volume, and the audio to be played of the wake-up prompt tone is played at the second volume.
If in speaker mode, the display device 200 remains in the off-screen state and responds to the user's voice by playing a wake-up alert sound to alert the user that it has been woken up.
The display device 200 first increases the system volume to a second volume, which may be a default system volume.
In some embodiments, the second volume may be less than a preset volume threshold. The display device 200 may determine the second volume based on relevant parameters of the user's voice including a signal to noise ratio of the user's voice and/or a difference between a wake-up value of the user's voice and a wake-up value threshold. Since the second volume is less than the volume threshold, the system volume of the audio of the wake-up cue played by the display device 200 is also less. The display device 200 may reduce the sense of intrusion to the user while responding to the user's wake-up.
Based on steps S3401-S3403, or based on steps S3501-S3503, after playing the audio of the wake-up cue, the display device 200 also plays cue information including information for prompting the user to adjust the system volume (for convenience of distinction, simply referred to as first cue information) through a speaker according to the current system volume. The current system volume is the system volume of the audio playing the wake-up prompt tone.
When the current system volume is greater than or equal to a first volume threshold, the first prompt message is used for prompting the user to reduce the system volume. When the current system volume is smaller than the second volume threshold, the first prompt message is used for prompting the user to increase the system volume. Wherein the first volume threshold is greater than the second volume threshold. Taking the display device 200 as an example of a television, as shown in (1) in fig. 36, the display device 200 may play a wake-up cue if it is waken up while in the off-screen state and the first state. If the current system volume is greater than or equal to the first volume threshold, a first prompt message is also played, such as "system volume is too high, suggesting to turn down the system volume". The display device 200 may play a wake-up cue if it is waken while in the off-screen state and the first state, as shown in (2) of fig. 36. If the current system volume is less than the second volume threshold, a first prompt message is also played, such as "system volume is too low, and system volume is recommended to be increased".
After the display device 200 plays the audio of the wake-up alert sound, alert information may also be played, where the alert information includes information for alerting the user that the user is currently in the speaker mode (for convenience of distinction, simply referred to as second alert information). The prompt information played by the display device 200 may include a first prompt information and a second prompt information.
Taking the display device 200 as an example of a television, the display device 200 may play a wake-up cue if it is waken up while in the off-screen state and the first state as shown in (1) of fig. 37. If the current system volume is greater than or equal to the first volume threshold, a first prompt message is also played, such as "system volume is too high, suggesting to turn down the system volume". The display device 200 may also play a second alert message, such as "currently in speaker mode". The display device 200 may play a wake-up cue if it is waken while in the off-screen state and the first state, as shown in (2) of fig. 37. If the current system volume is less than the second volume threshold, a first prompt message is also played, such as "system volume is too low, and system volume is recommended to be increased". The display device 200 may also play a second alert message, such as "currently in speaker mode". Thus, the user can be made to quickly determine the reason why the display device 200 is currently in the off-screen state.
In some embodiments, if the display apparatus 200 is in the mute mode, the display apparatus 200 maintains the off-screen state in response to the wake word being in the incompletely awake state, releases the mute mode, and plays the alert voice at the first volume after the release of the mute mode, and resumes the mute mode after the play of the alert voice. The incomplete wake state means that the display device 200 starts the prompt voice play function in response to the wake word, and exits the interaction with the user after playing the prompt voice play function, without responding to other far-field voice instructions of the user. The first volume may be a default system volume or a system volume saved when the mute mode is initiated. The prompt voice is used for prompting the user to be in a sound box mode and in a mute mode, and the user needs to wake up again after the mute mode is released.
In other embodiments, if the display apparatus 200 is in the non-mute mode and the current system volume is 0, the display apparatus 200 maintains the off-screen state in response to the wake word being in the incompletely wake state, increases the system volume to the first volume, plays the alert voice at the first volume, and readjust the system volume to 0 after playing the alert voice. The incomplete wake state means that the display device 200 starts the prompt voice play function in response to the wake word, and exits the interaction with the user after playing the prompt voice play function, without responding to other far-field voice instructions of the user. The first volume may be a default system volume. The prompt voice is used for prompting the user to be in a sound box mode and in a non-mute mode but the current system volume is 0, and the user needs to wake up again after increasing the volume.
The display device 200 may also respond to the wake-up word according to the flow shown in fig. 38, which specifically includes the following steps:
s3801, collecting user voice when the user is in a screen-off state and in a first state.
S3802, detecting whether the voice of the user contains a wake-up word or not in a sound box mode when the voice of the user is recognized.
Step S3801 is similar to step S501, and the process of recognizing the wake-up word in step S3802 is similar to steps S601-S602, and is not repeated here.
S3803, if the sound box mode is adopted, the display is adjusted to be in a bright screen state, an intermediate page is displayed, and a popup window is displayed on the intermediate page according to the first state.
The display device 200 is in an awake state in response to the wake word. The display device 200 powers up the display and adjusts the display to a bright screen state. The display device 200 does not exit the speaker mode, and the display device 200 displays the intermediate page in order to simulate the off-screen state when in the speaker mode in the on-screen state. Referring to fig. 39, the display apparatus 200 displays the intermediate page 3901, the intermediate page 3901 being black, and when the intermediate page 3901 is displayed, it looks equivalent to being in a screen-off state from the appearance of the display apparatus 200. At this time, a popup window is displayed on the middle page. As shown in fig. 39, a pop-up window 3902 is displayed on the intermediate page 3901. The popup is used for prompting a user to wake up and be in a sound box mode and a first state. The popup window can be small in size and displayed at non-central positions such as the vertex angle and the frame of the display, so that the invasion feeling caused to the user is reduced on the basis of prompting the user.
In some embodiments, if the first state is that the display apparatus 200 is in the mute mode, the display apparatus 200 is adjusted to be in the bright screen state, the mute mode is released, the system volume is adjusted to a preset volume, an intermediate page is displayed, and a pop-up window is displayed on the intermediate page. The preset volume may be a default system volume or a system volume stored when the mute mode is turned on. The popup window is used to prompt the user that the speaker mode and the mute mode are currently in, but the mute mode is currently released.
In other embodiments, if the first state is that the display device 200 is in the non-mute mode and the current system volume is 0, the display device 200 is adjusted to be in the bright screen state, the system volume is adjusted to be the preset volume, an intermediate page is displayed, and a popup window is displayed on the intermediate page. The preset volume may be a default system volume. The popup is used to prompt the user that the speaker mode is currently in and the system volume is 0, but the volume has been increased currently.
When the intermediate page is displayed after the display device 200 is awakened, the display device 200 may respond to other far-field voice instructions from the user.
In some embodiments, the display device 200 exits the intermediate page and displays a page of video in response to a far field voice instruction from the user indicating to play the video. For example, the far-field voice instruction is "play movie a", and the display device 200 exits the intermediate page and displays the screen of movie a in response to the far-field voice instruction.
In other embodiments, the display device 200 remains displaying the intermediate page and plays the audio in response to a far-field voice command from the user indicating to play the audio. For example, the far-field voice instruction is "play song B", and the display device 200 keeps displaying the intermediate page and plays the audio of song B in response to the far-field voice instruction.
In other embodiments, the display device 200 adjusts the display to a off-screen state and plays the audio in response to a far-field voice command from the user indicating to play the audio. For example, the far-field voice command is "today's weather", and the display device 200 adjusts the display to the off-screen state and plays the weather-related audio in response to the far-field voice command.
In other embodiments, the display device 200 is in an incompletely awake state in response to the wake word.
If the first state is that the display device 200 is in the mute mode, the display device 200 adjusts to the bright screen state in response to the wake-up word being in the incompletely wake-up state, does not deactivate the mute mode, does not adjust the system volume, displays an intermediate page, and displays a popup window on the intermediate page. The incompletely awake state means that the display apparatus 200 starts a pop-up window function in response to an awake word, and after displaying the pop-up window, exits interaction with the user without responding to other far-field voice instructions of the user. The popup window is used for prompting a user to be in a sound box mode and a mute mode currently, and needs to wake up again after the mute mode is released.
If the first state is that the display device 200 is in the non-mute mode and the system volume is 0, the display device 200 adjusts to the bright screen state in response to the wake-up word being in the incompletely wake-up state, does not adjust the system volume, displays an intermediate page, and displays a popup window on the intermediate page. The incompletely awake state means that the display apparatus 200 starts a pop-up window function in response to an awake word, and after displaying the pop-up window, exits interaction with the user without responding to other far-field voice instructions of the user. The popup window is used for prompting the user that the user is in a sound box mode currently, the system volume is 0, and the user needs to wake up again after the volume is increased.
The foregoing description, for purposes of explanation, has been presented in conjunction with specific embodiments. However, the above discussion in some examples is not intended to be exhaustive or to limit the embodiments to the precise forms disclosed above. Many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the disclosure and to enable others skilled in the art to best utilize the embodiments.

Claims (16)

1. A display device, characterized by comprising:
a display configured to: in a bright screen state;
a speaker configured to: the method comprises the steps that the method is in a first state, wherein the first state is that the display equipment is in a mute mode or the display equipment is in an unmuted mode, and the current system volume is 0;
a sound collector configured to: collecting user voice;
a controller configured to:
when the fact that the user voice contains the awakening word is recognized, controlling the display to display a popup window according to the first state, wherein the popup window comprises first information used for prompting the display device to be awakened and in the mute mode.
2. The display device of claim 1, wherein the controller is configured to, prior to controlling the display to display a pop-up window according to the first state:
Calculating a wake-up value corresponding to the user voice, wherein the wake-up value is used for representing the similarity degree of the user voice and the standard voice of the wake-up word;
determining whether the user voice contains a wake-up word according to the wake-up value and the wake-up value threshold;
if the wake-up value is greater than or equal to the wake-up value threshold, determining that the user voice contains a wake-up word; and if the wake-up value is smaller than the wake-up value threshold, determining that the user voice does not contain wake-up words.
3. The display device of claim 2, wherein the controller calculates a wake-up value corresponding to the user voice configured to:
inputting the user voice into a model to obtain the score of each syllable in the user voice, wherein the model is used for calculating a wake-up value, and the score of each syllable is used for representing the similarity degree of syllables with corresponding bit sequences in standard voice of the wake-up word;
and calculating a wake-up value corresponding to the user voice according to the score of each syllable in the user voice by using the model.
4. The display device of claim 2, wherein the controller, upon recognizing that the user speech includes a wake word, controls the display to display a pop-up window according to the first state, configured to:
Controlling the display to display the popup window according to the difference value between the awakening value and the awakening value threshold value;
when the difference value is smaller than a first difference value threshold value, controlling the display to display a popup window of a first style; and when the difference value is greater than or equal to a second difference value threshold, controlling the display to display a popup window of a second style, wherein the first difference value threshold is smaller than the second difference value threshold, and the popup window of the second style is easier to be perceived by a user than the popup window of the first style.
5. The display device of any of claims 1-4, wherein the controller, when in the mute mode, controls the display to display a pop-up window, further configured to:
acquiring the current system volume;
and controlling the display to display second information in the popup window according to the current system volume, wherein the second information is used for prompting a user to adjust the system volume.
6. The display device of claim 5, wherein the controller, in accordance with the current system volume, controls the display to display second information within the pop-up window configured to:
when the current system volume is larger than or equal to a first volume threshold, the second information is used for prompting a user to turn down the system volume;
And when the current system volume is smaller than a second volume threshold, the second information is used for prompting the user to increase the system volume, and the first volume threshold is larger than the second volume threshold.
7. The display device of claim 1, wherein the controller is further configured to:
and when the display equipment is in a non-mute mode and the current system volume is 0, controlling the display to display prompt information, wherein the prompt information is used for prompting that the current system volume is 0.
8. A display device, characterized by comprising:
a display configured to: in a bright screen state;
a speaker configured to: in a mute mode;
a sound collector configured to: collecting user voice;
a controller configured to:
and when the user voice is recognized to contain the wake-up word, the mute mode is released, and after the mute mode is released, the audio to be played of the wake-up prompt tone is played at a first volume.
9. A display device, characterized by comprising:
a display configured to: in a bright screen state;
a speaker configured to: in a mute mode;
a sound collector configured to: collecting user voice;
A controller configured to:
and when the user voice is recognized to contain a wake-up word, adjusting to an incomplete wake-up state, releasing the mute mode, playing a prompt voice at a first volume after the mute mode is released, and readjusting to the mute mode after the prompt voice is played, wherein the prompt voice is used for prompting the user to release the mute mode and awaken the display device again, and in the incomplete wake-up state, the display device supports playing the prompt voice and does not support responding other far-field voice instructions of the user.
10. A display device, characterized by comprising:
a display configured to: in a bright screen state;
a speaker configured to: in a non-mute mode and the current system volume is 0;
a sound collector configured to: collecting user voice;
a controller configured to:
when the user voice is recognized to contain the wake-up word, the system volume is increased to a preset volume, and the audio to be played of the wake-up prompt tone is played at the preset volume.
11. A display device, characterized by comprising:
a display configured to: in a bright screen state;
A speaker configured to: in a non-mute mode and the current system volume is 0;
a sound collector configured to: collecting user voice;
a controller configured to:
when the user voice is recognized to contain a wake-up word, the voice is adjusted to an incomplete wake-up state, the system volume is increased to a preset volume, the prompt voice is played at the preset volume, and after the prompt voice is played, the system volume is adjusted to 0, the prompt voice is used for prompting the user to increase the system volume and wake up the display device again, wherein in the incomplete wake-up state, the display device supports playing the prompt voice and does not support responding other far-field voice instructions of the user.
12. A wake-up method of a display device, applied to the display device according to any one of claims 1 to 7, wherein the display device is in a bright screen state and a first state, the first state is that the display device is in a mute mode or the display device is in a non-mute mode and the current system volume is 0;
the method comprises the following steps:
collecting user voice;
and when the user voice is recognized to contain a wake-up word, displaying a popup according to the first state, wherein the popup comprises first information used for prompting the display equipment to be awakened and in the mute mode.
13. A wake-up method of a display device, applied to the display device of claim 8, wherein the display device is in a bright screen state and a mute mode;
the method comprises the following steps:
collecting user voice;
and when the user voice is recognized to contain the wake-up word, the mute mode is released, and after the mute mode is released, the audio to be played of the wake-up prompt tone is played at a first volume.
14. A wake-up method of a display device, applied to the display device of claim 9, wherein the display device is in a bright screen state and a mute mode;
the method comprises the following steps:
collecting user voice;
and when the user voice is recognized to contain a wake-up word, adjusting to an incomplete wake-up state, releasing the mute mode, playing a prompt voice at a first volume after the mute mode is released, and readjusting to the mute mode after the prompt voice is played, wherein the prompt voice is used for prompting the user to release the mute mode and awaken the display device again, and in the incomplete wake-up state, the display device supports playing the prompt voice and does not support responding other far-field voice instructions of the user.
15. A wake-up method of a display device, applied to the display device as claimed in claim 10, wherein the display device is in a bright screen state and in a non-mute mode, and the current system volume is 0;
the method comprises the following steps:
collecting user voice;
when the user voice is recognized to contain the wake-up word, the system volume is increased to a preset volume, and the audio to be played of the wake-up prompt tone is played at the preset volume.
16. A wake-up method of a display device, applied to the display device as claimed in claim 11, wherein the display device is in a bright screen state and in a non-mute mode, and the current system volume is 0;
the method comprises the following steps:
collecting user voice;
when the user voice is recognized to contain a wake-up word, the voice is adjusted to an incomplete wake-up state, the system volume is increased to a preset volume, the prompt voice is played at the preset volume, and after the prompt voice is played, the system volume is adjusted to 0, the prompt voice is used for prompting the user to increase the system volume and wake up the display device again, wherein in the incomplete wake-up state, the display device supports playing the prompt voice and does not support responding other far-field voice instructions of the user.
CN202310791851.1A 2023-06-29 2023-06-29 Display device and awakening method thereof Pending CN117014668A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310791851.1A CN117014668A (en) 2023-06-29 2023-06-29 Display device and awakening method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310791851.1A CN117014668A (en) 2023-06-29 2023-06-29 Display device and awakening method thereof

Publications (1)

Publication Number Publication Date
CN117014668A true CN117014668A (en) 2023-11-07

Family

ID=88566392

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310791851.1A Pending CN117014668A (en) 2023-06-29 2023-06-29 Display device and awakening method thereof

Country Status (1)

Country Link
CN (1) CN117014668A (en)

Similar Documents

Publication Publication Date Title
US11600265B2 (en) Systems and methods for determining whether to trigger a voice capable device based on speaking cadence
US11217240B2 (en) Context-aware control for smart devices
CN109545206B (en) Voice interaction processing method and device of intelligent equipment and intelligent equipment
CN109243431A (en) A kind of processing method, control method, recognition methods and its device and electronic equipment
CN107580113A (en) Reminding method, device, storage medium and terminal
KR20010113919A (en) Method of interacting with a consumer electronics system
US12062360B2 (en) Information processing device and information processing method
CN111833868A (en) Voice assistant control method, device and computer readable storage medium
CN115810356A (en) Voice control method, device, storage medium and electronic equipment
EP3793275B1 (en) Location reminder method and apparatus, storage medium, and electronic device
CN109195016B (en) Intelligent terminal equipment-oriented voice interaction method and terminal system for video barrage and intelligent terminal equipment
EP3407096B1 (en) Method and device for determining descriptive information of precipitation trend, and readable storage medium
CN113808584B (en) Voice wakeup method and device, electronic equipment and storage medium
EP3955099A1 (en) Method and device for controlling the operation mode of a terminal device, and storage medium
CN113066491A (en) Display device and voice interaction method
CN112929739A (en) Sound production control method and device, electronic equipment and storage medium
CN112017651B (en) Voice control method and device of electronic equipment, computer equipment and storage medium
CN109658924B (en) Session message processing method and device and intelligent equipment
CN117014668A (en) Display device and awakening method thereof
CN116978376A (en) Display device and device wake-up method
CN117809642A (en) Display device and volume adjustment method of wake-up prompt tone
CN117809676A (en) Display device and volume adjustment method of wake-up prompt tone
CN113038048B (en) Far-field voice awakening method and display device
CN112863511B (en) Signal processing method, device and storage medium
CN113079400A (en) Display device, server and voice interaction method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination