WO2023160087A1 - Procédé d'invite pour état de réponse d'instruction vocale et dispositif d'affichage - Google Patents

Procédé d'invite pour état de réponse d'instruction vocale et dispositif d'affichage Download PDF

Info

Publication number
WO2023160087A1
WO2023160087A1 PCT/CN2022/135427 CN2022135427W WO2023160087A1 WO 2023160087 A1 WO2023160087 A1 WO 2023160087A1 CN 2022135427 W CN2022135427 W CN 2022135427W WO 2023160087 A1 WO2023160087 A1 WO 2023160087A1
Authority
WO
WIPO (PCT)
Prior art keywords
brightness
screen
state
display device
indicator light
Prior art date
Application number
PCT/CN2022/135427
Other languages
English (en)
Chinese (zh)
Inventor
樊杰
贾桂丽
邵肖明
Original Assignee
海信视像科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202210186106.XA external-priority patent/CN114582343A/zh
Priority claimed from CN202210186905.7A external-priority patent/CN114495934A/zh
Application filed by 海信视像科技股份有限公司 filed Critical 海信视像科技股份有限公司
Publication of WO2023160087A1 publication Critical patent/WO2023160087A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue

Definitions

  • the embodiments of the present application relate to the technical field of display devices, and in particular to a method for prompting a voice command response state and a display device.
  • the display device can be configured with a voice function. After collecting the voice command input by the user, it responds to the voice command and executes an action that matches the user's voice intention.
  • the user can press the voice button on the remote control to input the voice command, and release the button when the voice command is input; for the far-field voice function, the display device is equipped with a far-field sound collector, and the user can use the keyword to wake up Voice application, for example, if the user says "Hi, XX", if XX matches the agreed keyword, the system wakes up the voice application, and the voice application immediately responds, such as "How can I help you?" After the voice answer, you can Recognize and respond to subsequent received voice commands, and the far-field voice function can realize voice control without remote control within a certain distance.
  • an indicator light assembly can be installed on the display device body.
  • the indicator light assembly includes several indicator lights. Different display effects of the indicator light assembly indicate different response states. This application considers how to adjust the brightness of the indicator light assembly .
  • the embodiment of the present application provides a display device, including: a display, used to display a user interface on the screen; a sound collector, used to collect voice commands; an indicator light component, used to indicate the response status of the voice commands, including at least one indication A light; a controller configured to: receive a voice command, monitor the response status of the voice command, and acquire screen brightness; control the indicator lamp assembly according to the screen brightness and the lighting effect matching the response status show.
  • An embodiment of the present application provides a method for prompting the response state of a voice command, the method comprising: receiving a voice command, monitoring the response state of the voice command, and acquiring screen brightness; The lighting effect controls the display of the indicator light assembly; the indicator light assembly is arranged on the display device to indicate the response state, and the indicator light assembly includes at least one indicator light.
  • FIG. 1 shows an operation scenario between a display device and a control device
  • FIG. 2 shows a hardware configuration block diagram of a control device 100
  • FIG. 3 shows a hardware configuration block diagram of a display device 200
  • FIG. 4 shows a software configuration diagram in a display device 200
  • Figure 5 shows a speech recognition network architecture diagram
  • Fig. 6 shows a schematic diagram of interaction of a user waking up a voice application in a display device in a far-field voice scene
  • FIG. 7 shows a schematic diagram of the display device UI prompting that the response state of the voice command 1 is the listening state when the voice application is woken up;
  • FIG. 8 shows a schematic diagram of the display device UI prompting that the response state of the voice command 1 is the thinking state when the voice application is woken up;
  • FIG. 9 shows a schematic diagram of the UI of the display device prompting that the response state of the voice command 1 is the response state after the wake-up is successful;
  • FIG. 10 exemplarily shows a schematic diagram showing that the UI of the display device prompts that the response state of the voice command 1 is a wake-up failure state
  • Fig. 11 shows a schematic diagram showing that the UI of the display device prompts that the execution of the voice search instruction is completed
  • Fig. 12 shows a schematic structural diagram of a display device with an indicator light assembly
  • Fig. 13 shows a schematic structural diagram of another display device with an indicator light assembly
  • Fig. 14 shows a schematic diagram of the indicator light component prompting that the response state of the voice command A is the listening state in the scene where the display is turned off;
  • Fig. 15 shows a schematic diagram of the indicator light component prompting that the response state of the voice command A is the thinking state in the scene where the display is set to be off;
  • Fig. 16 shows a schematic diagram of the indicator light component prompting that the response state of the voice command A is a wake-up failure state in the scene where the display is set to be off;
  • Fig. 17 shows a schematic diagram of the indicator light component prompting that the response state of the voice command A is the response state in the scene where the display is set to turn off the screen;
  • FIG. 18 shows a schematic diagram of the indicator light component prompting that the response state of the voice command B is the listening state after the screen is turned off and the voice application is successfully woken up;
  • Fig. 19 shows a schematic diagram of the indicator light assembly prompting that the response state of the voice command B is the voice recognition state
  • Fig. 20 shows a schematic diagram of the indicator light assembly prompting that the response state of the voice command B is the state of "starting playing";
  • FIG. 21 shows a schematic diagram of the indicator light assembly prompting that the response state of the voice command B is the start-up completion state
  • Fig. 22 shows a schematic diagram showing that the UI of the display device and the indicator light component synchronously prompt that the response state of the voice command B is the state of "starting playback";
  • Figure 23 shows a schematic diagram of the access path of the backlight setting page
  • Fig. 24 shows a schematic diagram of the backlight setting page in the backlight custom mode
  • Fig. 25 shows a schematic diagram of the backlight setting page in the backlight adaptive mode
  • Fig. 26 shows the state prompt control logic of the first indicator light assembly
  • Fig. 27 shows the state prompt control logic of the second indicator lamp assembly
  • Fig. 28 shows the flow chart of the prompting method of the first voice instruction response state
  • Fig. 29 shows a flow chart of the second method for prompting the voice command response state
  • Fig. 30 shows a flow chart of the prompting method of the third voice command response state
  • Fig. 31 shows the logic diagram of the status prompt of the indicator light assembly
  • Fig. 32 shows the flow chart of the prompting method of the first voice command response state
  • Fig. 33 shows the flowchart of the prompting method of the second voice instruction response state
  • Fig. 34 shows a flow chart of the third method for prompting the response status of voice commands.
  • the display device provided in the embodiments of the present application may have various implementation forms, for example, it may be a TV, a smart TV, a laser projection device, a monitor, an electronic bulletin board, an electronic table, etc.
  • Fig. 1 and Fig. 2 are a specific implementation manner of the display device of the present application.
  • Fig. 1 is a schematic diagram of an operation scene between a display device and a control device according to an embodiment. As shown in FIG. 1 , the user can operate the display device 200 through the smart device 300 or the control device 100 .
  • control device 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication, Bluetooth protocol communication, and other short-distance communication methods, and the display device 200 is controlled wirelessly or wiredly.
  • the user can control the display device 200 by inputting user commands through buttons on the remote control, voice input, control panel input, and the like.
  • the smart device 300 (such as a mobile terminal, a tablet computer, a computer, a notebook computer, etc.) can also be used to control the display device 200 .
  • the display device 200 is controlled using an application program running on the smart device.
  • the display device may not use the aforementioned smart device or control device to receive instructions, but may receive user control through touch or gesture.
  • the display device 200 can also be controlled in a manner other than the control device 100 and the smart device 300.
  • the module for obtaining voice commands configured inside the display device 200 can directly receive the user's voice command control , the user's voice command control can also be received through the voice control device installed outside the display device 200.
  • the display device 200 also performs data communication with the server 400 .
  • the display device 200 may be allowed to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks.
  • the server 400 may provide various contents and interactions to the display device 200 .
  • the server 400 may be one cluster, or multiple clusters, and may include one or more types of servers.
  • FIG. 2 shows a block diagram of the configuration of the control device 100 according to the exemplary embodiment.
  • the control device 100 includes a controller 110 , a communication interface 130 , a user input/output interface 140 , a memory, and a power supply.
  • the control device 100 can receive the user's input operation instruction, and convert the operation instruction into an instruction that the display device 200 can recognize and respond to, and play an intermediary role between the user and the display device 200 .
  • the display device 200 includes at least one of a tuner and demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface. kind.
  • the controller includes a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, a first interface to an nth interface for input/output.
  • the display 260 includes a display screen component for presenting a picture, and a drive component for driving image display, for receiving an image signal output from the controller, for displaying video content, image content and a menu control interface component, and a user control UI interface .
  • the display 260 can be a liquid crystal display, an OLED display, and a projection display, and can also be a projection device and a projection screen.
  • the communicator 220 is a component for communicating with external devices or servers according to various communication protocol types.
  • the communicator may include at least one of a Wifi module, a Bluetooth module, a wired Ethernet module and other network communication protocol chips or near field communication protocol chips, and an infrared receiver.
  • the display device 200 may establish transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220 .
  • the user interface can be used to receive control signals from the control device 100 (such as: an infrared remote controller, etc.).
  • the detector 230 is used to collect signals of the external environment or interaction with the outside.
  • the detector 230 includes a light receiver, which is used to collect a sensor (not shown in the figure) of ambient light intensity; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environmental scenes, attributes of the user, etc. or user interaction gestures, or, the detector 230 includes a sound collector, such as a microphone, for receiving external sound.
  • the external device interface 240 may include but not limited to the following: high-definition multimedia interface (HDMI), analog or data high-definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, etc. multiple interfaces. It may also be a composite input/output interface formed by the above-mentioned multiple interfaces.
  • HDMI high-definition multimedia interface
  • component analog or data high-definition component input interface
  • CVBS composite video input interface
  • USB USB input interface
  • RGB port etc. multiple interfaces.
  • It may also be a composite input/output interface formed by the above-mentioned multiple interfaces.
  • the tuner demodulator 210 receives broadcast TV signals through wired or wireless reception, and demodulates audio and video signals, such as EPG data signals, from multiple wireless or cable broadcast TV signals.
  • the controller 250 and the tuner-demodulator 210 may be located in different split devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box wait.
  • the controller 250 controls the work of the display device and responds to user operations through various software control programs stored in the memory.
  • the controller 250 controls the overall operations of the display device 200 . For example, in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.
  • the controller includes a central processing unit (Central Processing Unit, CPU), a video processor, an audio processor, a graphics processing unit (Graphics Processing Unit, GPU), RAM Random Access Memory, RAM), ROM (Read- Only Memory, ROM), at least one of the first interface to the nth interface for input/output, a communication bus (Bus), and the like.
  • CPU Central Processing Unit
  • video processor video processor
  • audio processor audio processor
  • graphics processing unit Graphics Processing Unit, GPU
  • RAM Random Access Memory
  • ROM Read- Only Memory
  • the user can input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI).
  • GUI graphical user interface
  • the user may input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.
  • the system is divided into four layers, from top to bottom are respectively the application (Applications) layer (abbreviated as “application layer”), application framework (Application Framework) layer (abbreviated as “framework layer”) "), Android runtime (Android runtime) and system library layer (referred to as “system runtime layer”), and the kernel layer.
  • application layer application layer
  • application framework Application Framework
  • Android runtime Android runtime
  • system library layer system library layer
  • there is at least one application program running in the application program layer and these application programs can be window (Window) program, system setting program or clock program etc. that come with the operating system; they can also be developed by third-party developers. s application.
  • the application program packages in the application program layer are not limited to the above examples.
  • the framework layer provides application programming interfaces (Application Programming Interface, API) and programming frameworks for applications.
  • the application framework layer includes some predefined functions.
  • the application framework layer is equivalent to a processing center, which decides to make the applications in the application layer take actions.
  • the API interface Through the API interface, the application program can access the resources in the system and obtain the services of the system during execution.
  • the application framework layer includes managers (Managers), content providers (Content Provider) etc.
  • the manager includes at least one of the following modules: activity manager (Activity Manager) Interact with all activities running in the system; the Location Manager is used to provide system services or applications with access to the system location service; the Package Manager is used to retrieve the information currently installed on the device Various information related to the application package; Notification Manager (Notification Manager) is used to control the display and clearing of notification messages; Window Manager (Window Manager) is used to manage icons, windows, toolbars, wallpapers on the user interface and desktop widgets.
  • Activity Manager Activity Manager
  • the Location Manager is used to provide system services or applications with access to the system location service
  • the Package Manager is used to retrieve the information currently installed on the device Various information related to the application package
  • Notification Manager Notification Manager
  • Window Manager Window Manager
  • the activity manager is used to manage the life cycle of each application program and the general navigation back function, such as controlling the exit, opening, back, etc. of the application program.
  • the window manager is used to manage all window programs, such as obtaining the size of the display screen, judging whether there is a status bar, locking the screen, capturing the screen, controlling the change of the display window (such as reducing the display window, shaking the display, distorting the display, etc.), etc.
  • the system runtime layer provides support for the upper layer, that is, the framework layer.
  • the Android operating system will run the C/C++ library contained in the system runtime layer to realize the functions to be realized by the framework layer.
  • the kernel layer is a layer between hardware and software. As shown in Figure 4, the kernel layer at least includes at least one of the following drivers: audio driver, display driver, bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensors, etc.), and power drives, etc.
  • the kernel layer at least includes at least one of the following drivers: audio driver, display driver, bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensors, etc.), and power drives, etc.
  • This application involves voice control and interaction of display devices.
  • entities refer to things that exist objectively and can be distinguished from each other, including specific people, things, things, institutions, abstract concepts, etc.; It is a semantic network that can represent semantic relationships between entities.
  • entities are vertices or nodes, and relationships are edges.
  • the knowledge graph can be constructed in various ways, and the focus of the embodiment of the present application is not on how to construct the knowledge graph, so no detailed description will be given here.
  • the smart device is configured to receive input voice information and output a processing result of the voice information.
  • the speech recognition service device is an electronic device deployed with a speech recognition service
  • the semantic service device is an electronic device deployed with a semantic service
  • the business service device is an electronic device deployed with a business service.
  • the electronic device here may include a server, a computer, etc.
  • the speech recognition service, semantic service (also called a semantic engine) and business service here are web services that can be deployed on the electronic device, wherein the speech recognition service is used for audio Recognized as text, the semantic service is used for semantic analysis of the text, and the business service is used to provide specific services such as the weather query service of Moji Weather, the music query service of QQ Music, etc.
  • there may be multiple entity service devices deployed with different business services in the architecture shown in FIG. 5 or one or more functional services may be integrated in one or more entity service devices.
  • the following describes the process of processing voice information input to a smart device based on the architecture shown in FIG. process:
  • the smart device After the smart device receives the query sentence input by voice, it can upload the audio of the query sentence to the speech recognition service device, so that the speech recognition service device can recognize the audio as text through the speech recognition service and then return to the smart device.
  • the smart device before uploading the audio of the query sentence to the speech recognition service device, the smart device may perform denoising processing on the audio of the query sentence, where the denoising processing may include steps such as removing echo and environmental noise.
  • the smart device uploads the text of the query sentence recognized by the voice recognition service to the semantic service device, so that the semantic service device can perform semantic analysis on the text through the semantic service to obtain the business field and intention of the text.
  • the semantic service device sends a query command to the corresponding business service device according to the semantic analysis result of the text of the query statement, so as to obtain the query result given by the business service.
  • the smart device can obtain and output the query result from the semantic service device.
  • the semantic service device can also send the semantic analysis result of the query sentence to the smart device, so that the smart device can output the feedback sentence in the semantic analysis result.
  • FIG. 5 the architecture shown in FIG. 5 is only an example, and does not limit the protection scope of the present application. In the embodiment of the present application, other architectures may also be used to implement similar functions. For example, all or part of the three processes may be completed by a smart terminal, which will not be described in detail here.
  • the display device is an intelligent device that supports functions such as user interface display and voice, and can realize processing links such as collection, analysis and response of voice commands according to the architecture shown in Figure 5 .
  • the remote controller is equipped with a microphone, and the buttons on the remote controller body may include a voice button. When the user presses the voice button, the remote controller controls the microphone to start collecting the user's voice until the user releases the voice.
  • the microphone of the remote control will send the collected voice command to the display device, and the display device will send the voice command to a server such as Semantic Cloud, and the server will analyze the voice command and return the analysis result, and the display device will obtain the The user's voice intention, so as to respond to the voice command to perform the corresponding action.
  • the user presses the voice key and says "I want to watch video A”.
  • the display device knows that the user's voice intention is to watch the video, and the object is the video A, the display device starts the playback link of video A, and controls the display to play video A.
  • the display device can be configured with a far-field voice function.
  • the remote voice function means that the hardware of the display device is equipped with a sound collector, and the sound collector can collect voices within a certain distance (such as 3-5 meters).
  • the voice of the display device is configured with a voice application (such as a voice assistant, etc.) in the software.
  • the voice application is used to receive the voice commands collected by the voice collector, and send the voice commands to the server for analysis, and according to the analysis results returned by the server. Voice commands for action responses.
  • the far-field voice function can realize voice control without remote control operation within a preset distance, and realize pure voice interaction. Since the link of the remote control to transmit voice commands to the display device is cancelled, the delay of voice response can also be reduced to a certain extent.
  • the sound collector if the sound collector is always on, it will always collect sounds within a preset distance, which may not only cause user privacy security risks, but also cause the display device to perform invalid voice collection, analysis and response, such as user In the scene, there are only simple dialogues and exchanges, and no substantive operation instructions are issued to the display device. The process of collecting and identifying dialogue information by the display device is invalid, which leads to meaningless consumption of the underlying resources of the display device.
  • a sleep/wake mechanism is set for the voice application. When the voice application is in the sleep state, it will not respond to other voice commands except the wake-up command. If the user successfully wakes the voice application from the sleep state, Only the voice application will open the sound collector, so that the voice processing channel is turned on.
  • the display device can pre-agreed with the user a wake-up word for waking up the voice application, such as the inherent nickname of the voice assistant, or a user-defined vocabulary, etc.
  • the user speaks the voice command 1 as "Hi, XX", which is equivalent to the user saying "hello” to the voice application, and the voice collector sends the collected voice command 1 to the voice application, and the voice application sends the voice command 1 Send to the server in the cloud, and then receive the analysis result fed back by the server.
  • the voice application obtains the first keyword 61 as "XX" from the analysis result.
  • the wake-up fails, and the voice The application remains in the dormant state; if the voice application recognizes "XX" and the wake-up word matches, that is, the recognition voice command 1 belongs to the voice wake-up command, then the wake-up is successful, the voice application is restored from the dormant state to the awake state, and the collected data after the response is allowed Voice commands and execute actions/programs that match the user's voice intent.
  • the display device can simultaneously prompt the real-time response status of the voice command 1 on the current user interface.
  • FIG. 7 when the user inputs the voice command 1, it is prompted to be in the listening state, and the listening state is used to indicate that the sound collector is collecting the voice command that the user is inputting.
  • a state prompt control 700 is displayed on the user interface, and a state effect mark 701 and state information 702 matching the current voice response state are loaded and displayed in the state prompt control 700.
  • the state effect mark 701 is visualized by rendering effects such as animation and color
  • the state information 702 expresses different states intuitively with text content. For example, the example of the state effect mark 701 in the listening state in FIG. "Voice typing".
  • the voice command 1 after the user finishes inputting the voice command 1, it is necessary to analyze the voice command 1 and decide whether to wake up the voice application according to the analysis result, that is, send the voice command 1 in audio format to the server from the voice application until the voice command 1 is recognized according to the analysis result.
  • the wake-up is successful or the wake-up fails.
  • This process is named thinking state, which is equivalent to "thinking" whether to wake up the voice application.
  • the example of the status effect identifier 701 in the thinking state is " ⁇ ", where ⁇ is for example Appears in green, ⁇ in red, and the example of status information 702 is "waking up".
  • the voice application recognizes the first keyword correctly, the wake-up is successful, and it is necessary to "answer" the user's voice command 1, that is, it is necessary to prompt the user to be in the answering state, as shown in FIG. 9 , in the answering state
  • An example of the status effect logo 701 is " ⁇ ", where all ⁇ are green and blinking, and an example of the status information 702 is "What can I do for you?".
  • the voice application if it recognizes the first keyword incorrectly, it needs to prompt the user that it is in a wake-up failure state. As shown in FIG . ⁇ All of them are red and flashing, and the example of status information 702 is "wrong wake-up word, wake-up failed!". After viewing the status prompt, the user may choose to re-enter the wake-up command.
  • the voice application can immediately start the first timing mechanism when it is woken up successfully, and if the first timing time reaches the first preset duration, the voice application has not received any voice commands, and then switches back to sleep state; if the voice command 2 is received within the first preset duration, the timing will be cleared, and the voice command 2 will be directly recognized and responded without repeated wake-up. Whether other voice commands are received within the first preset duration, and so on, until the voice application finally returns to the dormant state.
  • each voice instruction may generally include a listening state, an action performing state, and a completion state.
  • the listening state is used to indicate that the display device is receiving voice input (that is, the sound collector is collecting voice information); the execution action state indicates that the voice command is being parsed, and the action/program that matches the voice command is started, for example, in the voice wake-up command
  • the state of executing the action is the aforementioned thinking state.
  • the voice search command carries the second keyword. For example, if the user says "Please search for Variety Show C", after the second keyword is recognized in the state of executing the action, "Variety Show C" will be displayed.
  • the device searches for a video source that matches Variety C; the completion status indicates that the voice command has been responded to and the specified action has been executed.
  • the completion status in the voice search command means that the search results are displayed
  • the completion status in the voice playback command means that the playback user specified
  • the completion status of the voice wake-up command is displayed as a wake-up failure status or a response status when the wake-up is successful.
  • the state prompt control 700 can prompt the response timeout state; if the response state of the voice command can reach the completion state within the second preset duration, then there is no response timeout problem, and the state prompt control 700 prompts the completion state .
  • the voice search command Take the voice search command as an example, as shown in Figure 11, after the voice application is successfully awakened, the user continues to speak the voice command 2 "Search for popular movies", and when the display device search is completed, refer to Figure 11, in the search page
  • the search results are displayed, and the example of the status effect logo 701 is " ⁇ ", where ⁇ is all rendered in green, and the example of the status information 702 is "the search has been completed, do you have any other requirements?".
  • the display status prompt control 700 is canceled.
  • the display device will monitor the change of the response state of the voice command, and adjust the state prompt control 700 synchronously.
  • the display effect of the state prompt control 700 can be adapted and flexibly configured, and is not limited to this Example of application.
  • the aforementioned embodiments are in the form of the status prompt control 700 in the user interface, prompting the user the real-time response status of the input voice command, which requires the user to face the TV directly and focus on the status prompt control 700 , and the screen of the display device is off in the standby state, STR state, etc., and the display device does not display any UI in the state of the screen off, so that the user cannot know the response state of the voice state.
  • an indicator light assembly 280 may be provided on the outer body of the display device, and the indicator light assembly 280 includes N indicator lights 281, where N ⁇ 1.
  • the indicator light 281 can use light-emitting elements such as LED lights, and can support multi-dimensional adjustments such as the brightness, color, and light effect of the light, so that the indicator light component 280 can present different voices in different voice response states. Lighting effect, the user can quickly recognize the voice response status indicated by observing the lighting effect of the indicator light assembly 280 .
  • the N indicator lights in the indicator light assembly 280 can be controlled independently.
  • the display device can maintain a state-lighting effect relationship list, and the state-lighting effect relationship list records the lighting effect of the indicator light component 280 corresponding to each voice response state, and the light effect specifies the indicator light component 280
  • the respective on-off status of the N indicator lights used to indicate whether to emit light
  • the color of the light the time node at which it is lit (that is, the light-emitting node)
  • the duration of the light The display device is configured with an initial state-lighting effect relationship list. If the display device supports user-defined lighting effects in each voice response state, the state-lighting effect relationship list can be updated synchronously according to the user's setting operation. If a voice response state is mapped to multiple lighting effects (i.e.
  • One mapping relationship that is, one voice response state only maps one lighting effect.
  • the controller 250 when the controller 250 detects that the response state of the current voice instruction is switched, it queries the target lighting effect corresponding to the switched response state according to the status lighting effect relationship list, and generates a control instruction 1 according to the target lighting effect. , and then send the control instruction 1 to the underlying LED driver module; the LED driver module responds to the control instruction 1, controls the lighting status of each indicator light in the component, and then controls the indicator light component 280 to display the target lighting effect.
  • the indicator light component 280 includes ⁇ LED1, LED2, LED3, LED4 ⁇ , a total of 4 indicator lights, the target lighting effect is to light up LED1 and LED3, and LED1 is green light, LED3 is red light, then the LED driver module controls LED2 And LED2 is turned off, LED1 is turned on and the light color is green, and LED3 is turned on and the light color is red, so that the indicator light assembly 280 presents the target light effect, ensuring the accuracy of the voice response status prompt.
  • the display and the indicator light assembly 280 are independently controlled, even if the display is turned off, it will not affect the light display of the indicator light assembly 280, so that the user can still use the indicator light assembly 280 to know when the screen is off.
  • Voice response status the indicator light component 280 is independent of the screen and the light is eye-catching. The user does not have to face the screen and focus on the status prompt control 700 in the interface, as long as the display device is within the visible range of the naked eye, the user can visually check it even out of the corner of the eye to the voice response state to improve user experience. In addition, the user can also check whether the voice command is abnormal through the indicator light component 280.
  • the indicator light component 280 prompts a response timeout state, it may be that the voice command cannot be correctly parsed, the voice command is invalid, or the display device cannot execute the voice action. For another example, if there is no state change after the indicator light assembly 280 prompts the listening state, it may be that the sound collector cannot pick up the sound correctly or the voice is too long.
  • the display device if the voice command is input when the display device is in the screen-off state, before waking up the screen, the display device is controlled to remain in the screen-off state, and the indicator light component 280 is controlled to prompt the current response state, that is, in the When the display device is in standby state, the screen is turned off, etc. without UI state prompts, the effective light prompt of the voice response state can still be realized through the indicator light component 280 .
  • the display device is currently in the off-screen state without any UI screen display, and the four indicator lights in the indicator light component are all off, and then the user says "Hi, XX",
  • the sound collector starts to collect the voice instruction A, and the controller controls the indicator light assembly 280 to prompt the listening state, and controls the display to remain in the off-screen state.
  • the light effect of the indicator light assembly 280 in the listening state is presented as " ⁇ ", and the small circle represents one indicator light, that is, four indicator lights are lit at the same time, and each ⁇ shows white light.
  • the controller detects that the response state of voice command A is switched from “listening” to “thinking”, and immediately controls the indicator light assembly 280 to prompt thinking. state, and the control display remains off.
  • the light effect of the indicator light assembly 280 in the thinking state is presented as " ⁇ " ⁇ " ⁇ ” ⁇ " ⁇ ” ⁇ " ⁇ ” ⁇ “ ⁇ ” ⁇ ..., in this cycle, it presents a marquee-like effect, wherein ⁇ is white light, and ⁇ is slightly grayed out.
  • the voice application sends the audio data of the voice command A to the server, and the server performs voice processing on the audio data to obtain the analysis information A, and returns the analysis information A to the display device; the display device Read the first keyword carried in the analysis information A as "XX", judge whether "XX" matches the agreed wake-up word, if "XX” does not match the wake-up word, as shown in Figure 16, then control the indicator light assembly 280 prompts that the wake-up failed, and the control display remains in the off-screen state.
  • the light effect of the indicator light assembly 280 in the wake-up failure state is to light up " ⁇ " at the same time, and each ⁇ appears to flash red.
  • the voice application is successfully woken up, as shown in FIG. 17 , and the indicator light component 280 is controlled to indicate the answering state, and the control display remains in the off-screen state.
  • the light effect of the indicator light assembly 280 in the listening state in FIG. 17 is to simultaneously light up " ⁇ ", and each ⁇ appears to be flashing green.
  • the display device remains in the off-screen state, the user continues to say "play movie M”, the voice collector starts to collect this voice command B, and the control The prompt of the controller control indicator light component 280 is switched from the answering state to the listening state, and the control display remains in the off-screen state.
  • the light effect of the indicator light assembly 280 in the listening state is " ⁇ ", that is, four indicator lights are turned on at the same time, and each ⁇ shows white light.
  • the controller when the sound collector finishes collecting the voice command B, the controller detects that the response state of the voice command B is switched from “listening" to "voice recognition”, and immediately controls the indicator light assembly 280 to prompt the voice recognition status, and controls The display remains off, and the voice application sends the audio data of the voice command B to the server, and the server performs voice processing on the audio data, obtains the analysis information B, and returns the analysis information B to the display device.
  • the light effect of the indicator light assembly 280 in the recognition state is presented as " ⁇ " ⁇ " ⁇ " ⁇ " ⁇ " ⁇ " ⁇ ” ⁇ ” ⁇ ..., in this cycle, it presents a marquee-like effect, in which ⁇ is white light, and ⁇ is slightly grayed out.
  • the display device receives and recognizes the analysis information B, learns that the user's voice intention is to play a video, and the video object is a movie M, then the prompt of the control indicator light assembly 280 is updated to the state of "starting to play", and at the same time, the display device is released.
  • the video player is started after the screen wakes up, and the video player is controlled to load the movie M according to the resource link, thereby completing the start of the movie M.
  • the light effect of the indicator light assembly 280 is " ⁇ ", that is, 4 indicator lights are lit at the same time, ⁇ is green light, ⁇ is red light, that is, it is green light , Alternating effect of red light.
  • the response state of the voice instruction B is switched to the completion state, and the prompt of the control indicator light assembly 280 is updated to the completion state synchronously.
  • the completion state can be specified. It is reflected in the state of "start broadcasting completed", as shown in the example in Figure 21, the light effect of the indicator light assembly 280 in the "start broadcasting completed” state is presented as " ⁇ ", that is, 4 indicator lights are lit at the same time, each ⁇ Green light is displayed.
  • the voice command is input when the display device is in the on-screen state, or the display device actively wakes up the screen from the off-screen state during the process of processing the voice command, referring to Figure 20 and Figure 21, only the indication can be controlled
  • the light component 280 prompts the current response status without displaying the status prompt control 700 , that is, the status prompt is no longer performed through the UI.
  • the indicator light component 280 and the status prompt control can be used The 700 provides simultaneous and synchronous prompts for the current response status, that is, dual status prompts are provided from the two display angles of the additional indicator light component and UI control.
  • the response state of the prompt voice command B is the state of "starting to play", as shown in Figure 22, the light effect of the indicator light assembly 280 in this state is " ⁇ ", that is, four lights are lit at the same time.
  • Indicator light ⁇ is green light
  • ⁇ is red light, that is, it presents the effect of alternating green light and red light
  • a status prompt control 700 is also displayed on the user interface, and an example of status effect identification 701 in this state is " ⁇ ⁇ ", where ⁇ is displayed in white, for example, and ⁇ is displayed in gray, that is, the effect of alternating light and dark.
  • the example of the status information 702 is "preparing to play".
  • the response status of the voice command B when the response status of the voice command B is updated to the completion status, it means that the response process of the voice command B reaches the end, and after the indicator light component 280 and the status prompt control 700 prompt the completion status, the first method It is to hide the state prompt control 700, and the N indicator lamps of the control indicator light assembly 280 are all extinguished; perhaps, the second method is to hide the state prompt control 700, but make the light assembly 280 temporarily keep the prompt to the completion state until If the next voice command C is received within the first preset duration, the status prompt control 700 will be displayed again, and the status prompts of the status prompt control 700 and the indicator light assembly 280 will be changed synchronously; or, the third way is to hide the status prompt control 700, but make the indicator light component 280 temporarily keep prompting the completed state until it is detected that the voice application enters the dormant state, and all the N indicator lights of the indicator light component 280 are automatically controlled to go out.
  • the user may not be able to know whether the voice application enters sleep and whether the voice application needs to be woken up before inputting the next voice command.
  • the purpose of temporarily keeping the light prompt of the indicator light assembly 280 is to implicitly remind the user that the voice application is still in the In the wake-up state, the user can directly input the next voice command C without inputting the voice wake-up command; when the indicator light assembly 280 is in the completely off state, it implicitly prompts the user that the voice application is currently in a dormant state, and the user needs to use it before the next voice interaction.
  • the voice application is woken up first, and the wake-up/sleeping state of the voice application is prompted to the user by using the on/off state of the indicator light component 280 .
  • the voice commands can be divided into different categories according to the operation intention, such as voice wake-up command, voice search command, voice play command, application start command, etc., for each type of voice command, a corresponding state can be set separately -
  • the lighting effect relationship list, the state-lighting effect relationship list records the lighting effects of the indicator light assembly 280 in different response states; or, each category of voice commands shares the same state-lighting effect relationship list; or, for similar categories and associated categories and/or the state-light effect relationship list corresponding to the voice command of the subordinate category, for example, the voice playback command is essentially a subcategory of the application startup command, so the voice playback command and the application startup command can share the same state - light list of valid relationships.
  • the light effect of the indicator light assembly 280 can be adaptively set according to the voice command category and the response status contained in each voice command category, and the embodiment of the present application is only exemplary.
  • the above embodiments have described in detail how to use the indicator light assembly 280 to prompt the response status of the voice command in the far-field voice scene, and mainly focus on the lighting effect of the indicator light assembly 280 in different response states.
  • the light brightness of the indicator light assembly 280 is generally set to a preset brightness, and the preset brightness is a fixed brightness value configured when the display device leaves the factory, so the brightness of the indicator light assembly 280 is fixed.
  • the indicator light assembly 280 may be relatively dark, causing the voice response status prompt to be inconspicuous, or even easily ignored by the user; if the environment where the display device is located is relatively dark, for example, the living room only If the display device is not turned on, the indicator light assembly 280 may be relatively bright, causing the brightness of the indicator light to be dazzling, especially unfriendly to users with eye diseases.
  • the present application realizes self-adaptive and dynamic control of the brightness of the indicator light assembly.
  • a display device can generally automatically adjust its own screen brightness according to the ambient light intensity, or users can artificially set the screen brightness according to their own habits and requirements,
  • the brightness of the screen can adapt to the brightness of the scene or meet the requirements of the user. Therefore, the brightness of the screen is used as a reference standard, and the brightness of the indicator light is adjusted to be consistent with the brightness of the screen.
  • the user can access the backlight setting page through the access path of Settings ⁇ Image ⁇ Backlight, so as to select the backlight adjustment mode.
  • At least two modes are provided in the backlight setting page, the first is the backlight custom mode, and the second is the backlight adaptive mode.
  • the page includes a backlight adjustment control 241 and a switch control 242 .
  • the switch control 242 has associated inquiry information 242A, and the inquiry information 242A is, for example, "whether to turn on the screen backlight to adapt to the light intensity of the environment", if the user turns off the switch control 242, the backlight custom mode can be selected, and the backlight adjustment control 241 can be accessed from Define the brightness threshold for setting the brightness of the screen; if the user turns on the switch control 242, that is, the switch control 242 is turned on, that is, the user selects the backlight adaptive mode.
  • the user turns off the switch control 242 and selects the backlight custom mode, and the backlight adjustment control 241 is used to allow the user to manually adjust the brightness threshold.
  • the backlight adjustment control 241 may include a backlight bar 241A and a punctuation point 241B.
  • the user can operate the punctuation point 241B to slide along the backlight bar 241A, assuming that the left end point of the backlight bar 241A is the lower limit value of the screen brightness, and the right end point of the backlight bar 241A is the upper limit value of the screen brightness.
  • the limit value, that is, the backlight bar 241A is the direction in which the brightness continuously increases from left to right.
  • the backlight adjustment control 241 also includes brightness indication information 241C.
  • the user can move the punctuation point 241B through remote control or touch operation, and the brightness indication information 241C is synchronously prompted
  • the current screen brightness for example, when the punctuation point 241B is moved to the position shown in Figure 24 on the backlight bar 241A, the brightness indication information 241C prompts that the screen brightness is 75, and the user stops moving the punctuation point 241B after visually observing that the backlight is adjusted to a brightness that he thinks is appropriate.
  • the screen brightness displayed by the brightness indication information 241C is the user-defined brightness threshold.
  • the backlight adjustment control 241 is not limited to the setting method in FIG. 24 , for example, the user can directly input a desired brightness threshold in the backlight adjustment control 241 .
  • the controller after the user selects the backlight custom mode and sets the brightness threshold, the controller generates the control instruction 2 according to the brightness threshold, and sends the control instruction 2 to the backlight control module in the display; the backlight control module responds to The control instruction 2 is to adjust the screen backlight to the brightness threshold.
  • the backlight customization mode no matter how the ambient light changes in the scene, as long as the user does not modify the brightness threshold, the screen brightness when the display is in the bright screen state will always remain unchanged at the current brightness threshold.
  • the hardware structure of the display device is further provided with an illumination sensor, and the illumination sensor is used to detect the intensity of ambient light.
  • the switch control 242 is in the off state, the brightness threshold can be obtained according to the backlight adjustment control 241, and the screen brightness adjustment is not interfered by the ambient light intensity, so the light sensor should be controlled to be turned off in the backlight custom mode, so as to avoid the invalid work of the light sensor , and also reduce the power consumption of the display device.
  • the switch control 242 in the backlight setting page is switched to an on state, and at the same time, the controller controls the light sensor to be turned on, so as to start detecting the ambient light intensity, so that according to the environment Adaptively match the light intensity and adjust the screen brightness.
  • an ambient light-backlight relationship list can be preset.
  • the ambient light-backlight relationship list records the screen brightness corresponding to different ambient light intensities.
  • the relationship between the ambient light intensity and the screen Brightness is generally positively correlated, that is, the brighter the environment, the higher the adaptive screen brightness, and the darker the environment, the lower the adaptive screen brightness.
  • the display device obtains the target screen brightness matching the ambient light intensity currently detected by the illumination sensor from the ambient light-backlight relationship list, generates the control instruction 3 according to the target screen brightness, and sends the control instruction 3 to Backlight control module: the backlight control module adjusts the screen backlight to the target screen brightness in response to the control instruction 3 .
  • the ambient light intensity will change with factors such as sunlight and light sources. For example, the ambient light will naturally darken after sunset, and the ambient light will increase after turning on the lighting when it is dark.
  • the control When the sensor detects changes in the ambient light intensity, it updates the screen brightness synchronously based on the ambient light-backlight relationship list, thereby realizing adaptive and dynamic adjustment of the screen brightness according to the ambient light intensity.
  • the backlight adjustment mode is not limited to the aforementioned backlight custom mode and backlight adaptive mode.
  • the screen backlight can be adjusted according to the brightness of the video picture, such as night vision mode or night scene shooting. If the video image is usually dark, the brightness of the screen is relatively increased. For example, if the brightness of the video image is high, the brightness of the screen can also be relatively lowered, so that the video image can be more comfortable for human eyes to watch.
  • the controller when the sound collector receives a voice command, the controller needs to monitor the response status of the voice command and obtain the screen brightness at the same time, and then control the indicator light assembly 280 according to the screen brightness and the lighting effect matching the current response status display, so that the indicator light component 280 not only prompts the response state, but also keeps the brightness of the indicator light consistent with the brightness of the screen and dynamically synchronizes, so that the brightness of the indicator light can be adjusted to an appropriate state, adapting to the light and dark of the scene or meeting the user's requirements, and realizing the indication Adaptive dynamic regulation of the brightness of the lamp assembly.
  • FIG. 26 illustrates a status prompt control logic of an indicator light component.
  • the indicator light component performs a status prompt, it includes two processing logics. The first is that the voice collector collects voice command A, and the controller simultaneously monitors The response status of the voice command A, the second is that the controller queries the current screen brightness, and then generates a control command according to the current response status and screen brightness, and sends the control command to the LED driver module; the LED driver module responds to the control command , to display according to the lighting effect indicated by the control command, and at the same time adjust the brightness of the luminous target indicator to be consistent with the brightness of the screen, thereby completing the prompt for the current response state of the voice command A.
  • FIG. 27 illustrates another status prompt control logic of an indicator light assembly, which mainly involves brightness adjustment in two angles.
  • the first angle is screen backlight adjustment, which is not limited to include backlight custom mode, backlight adaptive mode and The initial default mode, etc.
  • the controller generates and sends a control command 2 to the backlight control module according to the current brightness threshold, and the backlight control module responds to the control command 2 to adjust the display to meet the brightness threshold; in the backlight adaptive mode Among them, the controller matches the target screen brightness according to the ambient light intensity obtained from the light sensor, generates and sends a control command 3 to the backlight control module according to the target screen brightness, and the backlight control module adjusts the display to meet the target screen brightness in response to the control command 3; In the initial default mode, the controller generates and sends a control command 4 to the backlight control module according to the preset brightness of the display device factory configuration, and the backlight control module responds to the control command 4 to adjust the display to reach the preset brightness
  • the second angle is the brightness adjustment of the indicator light assembly
  • the sound collector receives the voice command A
  • the controller monitors the response status of the voice command A, and at the same time inquires about the current screen brightness;
  • the controller inquires about the voice command A
  • the target lighting effect corresponding to the current response state generates a control command 1 according to the target lighting effect and the current screen brightness, and sends the control command 1 to the LED driver module;
  • the LED driver module responds to the control command 1 and controls the indicator light
  • the component prompts the response state according to the target lighting effect, queries the target indicator light that is lit in the indicator light component preset for the target lighting effect, and adjusts the brightness of the target indicator light to be equal to the brightness of the screen.
  • the display state of the indicator light component is dynamically adjustable, and when the response state is switched, the light effect of the light component needs to be updated synchronously; in the backlight custom mode, it is detected that the brightness threshold has changed, or, in In the backlight adaptive mode, if the ambient light intensity is detected to change, the screen brightness will naturally change accordingly, and the light brightness of the indicator light component will also be linked with the screen brightness. When any of the response state and screen brightness changes, the display state of the indicator light component needs to be updated synchronously.
  • the brightness of the screen when the display device is on, the brightness of the screen is greater than zero, so after the brightness of the indicator light component and the screen are synchronized, the brightness of the light of the indicator light component is naturally visible.
  • the display device is in standby, hibernation, STR (Suspend to RAM, suspend to memory) and other states, the display device is turned off, that is, the brightness of the screen is zero, and the indicator light component must ensure that the light is visible before the voice can be activated. Effective reminders are provided in response to the state, so this application also considers how to adjust the brightness of the indicator light component when the display device is turned off.
  • the light sensor when the display device is switched from the on-screen state to the off-screen state, the light sensor may not be turned off, and the light sensor is still used to detect the light intensity of the external environment.
  • the controller When the first voice command is received in the off-screen state, if the first voice command is not used to wake up the screen, the controller will adaptively match the target screen brightness according to the currently detected ambient light intensity, which is different from that in the on-screen state. The controller only generates the control command 4 according to the brightness of the target screen and the response state, and sends the control command 4 to the LED driver module, and no longer sends any command to the backlight control module.
  • the LED driver module Adjust the target indicator light in the current response state to the target screen brightness, while the display remains in the off-screen state.
  • the light sensor when the display device switches from the screen-on state to the screen-off state, the light sensor can be turned off first, and then when the first voice command is received in the screen-off state , and then control the light sensor to turn on, and adaptively adjust the brightness of the target indicator light in the component according to the ambient light intensity and the response status of the voice command, while the display remains in the off-screen state.
  • the light sensor is turned off to avoid invalid detection of the sensor, reduce the power consumption of the display device, and turn on the light when the voice command input is received
  • the sensor realizes that when the display device is in the off-screen state, the brightness of the indicator light component can be adapted to the ambient light intensity without waking up the screen.
  • the controller queries the brightness threshold currently recorded by the system, generates control command 5 only according to the brightness threshold and response status, and sends the control command 5 to the LED driver module instead of Send any command to the backlight control module, that is, the display will remain in the off-screen state.
  • the LED driver module adjusts the target indicator light in the current response state of the component to the brightness threshold, and when switching the response state later, the indicator light component only changes the lighting effect, and the target indicator light in each response state
  • the brightness is always kept constant at the brightness threshold.
  • the display device in some scenarios, for example, if the display device is not equipped with a light sensor, or the light sensor is faulty and cannot sense light and detect the ambient light intensity, the display device itself does not support the backlight adaptive mode, and the user does not customize the brightness threshold. It will cause the indicator light component to have no brightness adjustment reference when performing status prompts; for another example, when the user does not enable the backlight adaptive mode, that is, when the switch control 242 is off, and does not customize the brightness threshold, it will also cause the indicator light component to There is no brightness adjustment reference when performing status prompts, thus causing the indicator lights to fail to display.
  • the display device can be configured with a fixed brightness value when it leaves the factory. In the scenario where the brightness threshold is set, the preset brightness is used as the brightness adjustment benchmark of the indicator light component by default.
  • the light sensor keeps In the off state, when the first voice command is received in the off-screen state, if the first voice command is not used to wake up the screen, the controller will query the preset brightness recorded in the system initial configuration file, and only according to the preset brightness and response The state generates a control command 6, and sends the control command 6 to the LED driver module without sending any command to the backlight control module, that is, the display keeps the screen off state.
  • the LED driver module adjusts the target indicator light in the current response state of the component to the preset brightness, and when the response state is switched later, the indicator light component only changes the lighting effect, and the target indicator in each response state
  • the brightness of the lamp is always kept constant at the preset brightness.
  • the display device needs to be in a bright screen state after booting, restarting or waking up the screen. If the display device does not support or enable the backlight adaptive mode, and the user has not set the brightness threshold in the backlight custom mode,
  • the screen display can also be adjusted by default according to the preset brightness.
  • the controller queries the preset brightness recorded in the system initial configuration file, generates the control command 7 according to the preset brightness, and sends the control command 7 to the backlight control module, and the backlight control module responds In the control command 7, the brightness of the screen is adjusted to a preset brightness.
  • the screen brightness after the screen brightness is adjusted to the preset brightness, if the user enables the backlight adaptive mode, the screen brightness will no longer maintain the preset brightness, but switch to adaptively adjust the screen brightness according to the ambient light intensity, or , if the user sets the brightness threshold, and the brightness threshold is not equal to the preset brightness, the screen brightness will be switched from the preset brightness to the brightness threshold. Simultaneously, the brightness of the indicator light will no longer maintain the preset brightness, but follow the screen brightness.
  • the second voice command is a voice command that needs to wake up the screen, for example, the user says "play video S", “Search Popular Movies", etc., all need to wake up the screen to start playing the video S visually and display the search results of popular movies and other content.
  • the display device sends the audio data of the second voice command to the semantic server, and the server performs analysis processing such as semantic recognition on the audio data, and converts the audio into readable information indicating the user's voice intention (that is, the final analysis result), and return the analysis result to the display device.
  • the collection of the second voice command by the display device corresponds to the listening state
  • the server’s analysis of the second voice command corresponds to the analysis state/speech processing state.
  • the display device Before the display device receives the analysis result, it cannot respond to the second voice command and execute the program in line with the user’s voice intention. Therefore, when the second voice command is in the listening state and analyzing state, the display device remains in the off-screen state, and executes the brightness control scheme of the indicator light component when the screen is off.
  • the display device after the display device receives the parsing result, it learns the user's voice intention. Taking the user's voice intention to search for popular movies as an example, the display device immediately wakes up the screen and automatically adjusts the brightness of the screen according to the current backlight adjustment mode.
  • the backlight adjustment mode is one of various preset modes such as backlight adaptive mode (adapting to ambient light intensity), backlight custom mode (brightness threshold), initial default mode (preset brightness), and the indicator light component prompts " Searching" state, and control the target indicator light in the component to glow according to the current screen brightness in the "searching" state.
  • the display device will display the search results of popular movies on the screen interface (the screen is already on at this time) , and control the indicator light component to switch to the state of prompting "search complete”.
  • the present application provides a first prompt method for voice command response status, the method is executed by a controller, and the controller is respectively connected to the display, the sound collector and the indicator light assembly, so Said method comprises the following procedural steps:
  • Step S281 receiving a voice command, monitoring the response status of the voice command, and acquiring screen brightness.
  • Step S282 controlling the display of the indicator light component according to the brightness of the screen and the lighting effect matching the response state.
  • the indicator light component prompts different response states through different lighting effects, and at the same time keeps the brightness of the indicator light consistent with the brightness of the screen, and the brightness of the indicator light changes synchronously with the brightness of the screen to realize the brightness adjustment of the indicator light component.
  • Adaptive dynamic regulation when the display device is in the bright screen state, the screen brightness can be adjusted in any one of the ways such as backlight adaptive mode (adapting to ambient light intensity), backlight custom mode (brightness threshold), initial default mode (preset brightness), etc. adjust.
  • the present application also provides a second prompting method for voice command response status, the method is executed by the controller, and includes the following procedural steps:
  • Step S291 query the current backlight adjustment mode in response to the device startup instruction.
  • the backlight adjustment mode is one of multiple preset value modes such as backlight adaptive mode (adaptation to ambient light intensity), backlight custom mode (brightness threshold), initial default mode (preset brightness).
  • Step S292 according to the current backlight adjustment mode, the screen brightness is automatically adjusted when the display is started.
  • Step S293 after the system startup is completed, a voice instruction is received.
  • Step S294 monitor the response state of the voice command, and obtain the lighting effect of the indicator light component corresponding to the current response state and the current screen brightness.
  • the lighting effect is used to define the opening and closing status, luminous color, luminous duration, dynamic lighting effect, etc. of each indicator light in the component.
  • Step S295 inquire about the target indicator lights that are turned on in the indicator light components that are preset for the lighting effects.
  • Step S296 controlling the indicator light component to display the light effect, and at the same time adjusting the brightness of the target indicator light to the current screen brightness.
  • the present application also provides a third prompting method for voice command response status, the method is executed by the controller, and includes the following procedural steps:
  • Step S301 in response to the screen-off instruction, turn off the backlight of the display.
  • Step S302 receiving a voice wake-up command, controlling the display to keep the screen off, and determining the target brightness according to the backlight adjustment mode used before the screen off.
  • the target brightness is the brightness adapted according to the ambient light intensity detected by the light sensor; if the backlight custom mode is used before the screen is turned off, the target brightness is is the current brightness threshold; if the initial default mode is used before the screen is turned off, the target brightness is the preset brightness of the display device factory configuration.
  • Step S303 controlling the indicator light component to prompt the response status of the voice wake-up instruction, and adjusting the lit target indicator light to the target brightness.
  • Step S304 after the voice application wakes up successfully, receiving the first voice command.
  • Step S305 before receiving the analysis information of the first voice command from the server, control the display to keep the screen off, control the indicator light component to prompt the response status of the first voice command, and the target indicator light to be lit Adjust to the target brightness.
  • Step S306 receiving the parsing information sent by the server.
  • Step S307 judging whether the screen needs to be woken up. If there is no need to wake up the screen, execute step S308; if it is necessary to wake up the screen, execute steps S309 to S3010.
  • Step S308 controlling the display to keep the screen off, continuing to control the indicator light component to prompt the subsequent response state of the first voice command, and adjusting the illuminated target indicator light to the target brightness.
  • Step S309 start the backlight of the display, and automatically adjust the brightness of the screen when the display is started according to the current backlight adjustment mode.
  • Step S3010 controlling the indicator light assembly to prompt the subsequent response status of the first voice command, and adjusting the illuminated target indicator light to be consistent with the brightness of the screen.
  • the backlight adjustment mode after the screen wakes up may be different from the previous backlight adjustment mode before the screen was turned off. Then continue to synchronize with the aforementioned target brightness.
  • the logic of the indicator light assembly as shown in FIG. 31 performs status prompts.
  • the display device receives a voice command A, and the voice command A is used to wake up the voice application.
  • the voice command A goes through three response states, and the response state A1 is Listening state (the indicator light component displays the light effect A1), the response state A2 is the thinking state (the light component switches to display the light effect A2), the response state A3 is the response state (the light component switches to display the light effect A3), the response state Indicates that the voice application wakes up successfully, the user continues to input the voice command B, and the voice command B goes through S response states B i , i is used to indicate the serial number of the response state of the voice command, 1 ⁇ i ⁇ S, and the indicator light is in the response state B i
  • the component displays the lighting effect B i , and so on, and uses the indicator light component to prompt the response status of other subsequent voice commands.
  • the present application provides a first prompt method for voice command response status, the method is executed by a controller, and the controller is respectively connected to the display, the sound collector and the indicator light assembly, so
  • the methods described include:
  • step S321 the voice command is received, and the response status of the voice command is monitored.
  • Step S322 controlling the indicator light component to display a light effect matching the response state.
  • the indicator light component prompts different response states through different lighting effects, and the light effect of the indicator light component changes synchronously with the switching of the response state, so as to realize the precise prompting of the voice response state through the indicator light component.
  • the user can quickly distinguish the voice response status by viewing the lighting effect of the indicator light component.
  • the indicator light component and the display are independently controlled, the status prompt of the indicator light component will not be affected when the display device is in standby and the display is off. With certain divergence characteristics, the user does not have to face the screen completely and focus on a certain position in the interface. As long as the display device is within the visible range of the naked eye, the user can even observe the light effect of the indicator light with peripheral vision, so as to know the current Response status for voice commands.
  • the present application also provides a second prompting method for voice command response status, the method is executed by the controller, and includes the following procedural steps:
  • step S331 the voice command is received, and the response status of the voice command is monitored.
  • Step S332 inquire about screen brightness.
  • Step S333 query the lighting effect corresponding to the response state.
  • the lighting effect is used to define the opening and closing status, lighting color, lighting node, lighting duration, dynamic lighting effect and other contents of each indicator light in the component.
  • Step S334 controlling the indicator light component to prompt the response state according to the light effect and the screen brightness.
  • the brightness of the screen can be automatically adjusted through any of the backlight adjustment modes such as the backlight adaptive mode (adapting to the ambient light intensity), the backlight custom mode (brightness threshold), the initial default mode (preset brightness), and so on. Keep the brightness of the indicator light consistent with the brightness of the screen, and when the brightness of the screen changes dynamically, the brightness of the indicator light will be linked accordingly.
  • This application can adjust the brightness of the indicator light to a suitable state, adapt to the light and shade of the scene or meet the user's requirements, and realize the adaptive dynamic regulation of the brightness of the indicator light component when the indicator light component performs status prompts, and the indicator light component does not need to display a fixed display Brightness improves user experience.
  • the present application also provides a second prompting method for voice command response status, the method is executed by the controller, and includes the following procedural steps:
  • Step S341 receiving a voice instruction.
  • Step S342 judging whether the display device is in a bright screen state. If the display device is in the on-screen state, execute steps S343 and S344; if the display device is in the off-screen state, execute step S345.
  • Step S343 monitoring the response status of the voice command, and inquiring about the brightness of the screen.
  • Step S344 according to the response state and the screen brightness, control the display of the indicator light component.
  • Step S345 determine the target brightness according to the backlight adjustment mode used before the screen is turned off.
  • the target brightness is the first brightness
  • the first brightness is a brightness value adapted according to the ambient light intensity detected by the illumination sensor
  • the target brightness is the second brightness
  • the second brightness is the brightness threshold of the screen set by the user
  • the target brightness is the third brightness
  • the third brightness is a default brightness configured by the factory of the display device.
  • Step S346, according to the response state and the target brightness, control the display of the indicator light component.
  • the indicator light assembly is set on the body of the display device.
  • the indicator light assembly may not be fixed on the display device.
  • the indicator light assembly is set as an independent structure for connection and disassembly.
  • the indicator light component can be inserted into a specific interface to establish a wired connection between the indicator light component and the display device; Just unplug it from the interface.
  • the specific interface may be an interface specially designed for adapting the indicator light assembly, or may also be a conventional interface such as USB.
  • a wireless connection between the indicator light component and the display device can also be established through a form such as a local area network or Bluetooth; when the user does not want to use the indicator light component, disconnect Turn on the wireless connection between the indicator light component and the display device.
  • the wireless connection mode between the indicator light assembly and the display device greatly reduces the position limitation of the indicator light assembly for status prompts, and the user can move the indicator light assembly arbitrarily within the wireless communication range. As long as the user observes the indicator light assembly , Even without looking at the display device, you can still accurately know the response status of the voice command, which improves the scene adaptability.
  • the indicator light assembly when it is not connected to a display device, it can also be used as an independent intelligent light source device.
  • a power interface is provided on the indicator light assembly. Realize the lighting function; for another example, set a switching button on the indicator light component, and the user can click the switching button to switch different lighting effects; another example, set a dimming button on the indicator light component, and the user clicks the dimming button to switch Adjust light levels, and more.
  • the indicator light component can also be configured with a voice function, and the user can input a voice command at the indicator light component end, for example, the user speaks the voice command C of "turn on the light", and the light indicator component sends the voice command C to the server , the server analyzes the voice command C, and after receiving the analysis information returned by the server, recognizes the analysis information and obtains the user's voice intention, and then performs the action of turning on the light.
  • the indicator light component can also use different lighting effects to prompt the response status of the voice command C. It should be noted that if the indicator light component can be used as an independent smart device, its software and hardware function configuration is not limited to the examples provided in this application.
  • the light reminder effect and brightness contrast of the indicator light assembly cannot be truly presented in the drawings.
  • the drawings and descriptions of the state reminder of the indicator light assembly in this application are only exemplary. Different types of voice commands and the types of response states contained in each voice command can be set according to actual applications, and the lighting effects corresponding to different response states can be flexibly set, so that users can have no doubts after viewing the light prompts of the indicator light components. to accurately identify the response status of the current voice command.
  • the backlight adjustment mode of the screen is not limited to the backlight adaptive mode, backlight custom mode and initial default mode in the example of this application. When the display device is on, no matter which backlight adjustment mode is enabled, the brightness of the indicator light only needs to follow the brightness of the screen. Yes, so that the indicator light can be illuminated with a suitable brightness to prevent the light from being too dim or too bright.
  • the present application also provides a non-volatile computer storage medium, which can store a program.
  • the execution of the program may include the program steps of the method for prompting the response state of the voice command configured by the aforementioned controller.
  • the computer storage medium may be a magnetic disk, an optical disk, a read-only memory (English: Read-Only Memory, referred to as ROM) or a random access memory (English: Random Access Memory, referred to as RAM), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

Un procédé d'invite pour un état de réponse d'une instruction vocale et un dispositif d'affichage. Le procédé consiste : à recevoir une instruction vocale, à surveiller un état de réponse de l'instruction vocale et à obtenir une luminosité d'écran (S281) ; et à commander l'écran d'un ensemble lampe d'indicateur (280) en fonction de la luminosité d'écran et d'un effet de lumière correspondant à l'état de réponse (S282), l'ensemble lampe d'indicateur (280) étant disposé sur un dispositif d'affichage (200) et utilisé pour indiquer l'état de réponse, l'ensemble lampe d'indicateur (280) comprenant en outre au moins une lampe d'indicateur (281).
PCT/CN2022/135427 2022-02-28 2022-11-30 Procédé d'invite pour état de réponse d'instruction vocale et dispositif d'affichage WO2023160087A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN202210186106.XA CN114582343A (zh) 2022-02-28 2022-02-28 一种语音指令响应状态的提示方法及显示设备
CN202210186905.7A CN114495934A (zh) 2022-02-28 2022-02-28 一种语音指令响应状态的提示方法及显示设备
CN202210186905.7 2022-02-28
CN202210186106.X 2022-02-28

Publications (1)

Publication Number Publication Date
WO2023160087A1 true WO2023160087A1 (fr) 2023-08-31

Family

ID=87764596

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/135427 WO2023160087A1 (fr) 2022-02-28 2022-11-30 Procédé d'invite pour état de réponse d'instruction vocale et dispositif d'affichage

Country Status (1)

Country Link
WO (1) WO2023160087A1 (fr)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060097658A1 (en) * 2004-10-29 2006-05-11 Vicent Chiang Apparatus for adjusting brightness of indicator light on panel
CN106550517A (zh) * 2015-09-18 2017-03-29 中兴通讯股份有限公司 一种控制指示灯亮度的方法及终端
CN107277995A (zh) * 2017-07-05 2017-10-20 环球智达科技（北京）有限公司 指示灯亮度的调节方法
CN107665697A (zh) * 2017-10-20 2018-02-06 维沃移动通信有限公司 一种屏幕亮度的调节方法及移动终端
CN111833858A (zh) * 2019-04-17 2020-10-27 百度在线网络技术（北京）有限公司 基于音箱的语音交互状态显示方法和装置
CN113066490A (zh) * 2021-03-16 2021-07-02 海信视像科技股份有限公司 一种唤醒响应的提示方法和显示设备
CN113421532A (zh) * 2021-07-09 2021-09-21 海信视像科技股份有限公司 背光调整方法及显示设备
CN114495934A (zh) * 2022-02-28 2022-05-13 海信视像科技股份有限公司 一种语音指令响应状态的提示方法及显示设备
CN114582343A (zh) * 2022-02-28 2022-06-03 海信视像科技股份有限公司 一种语音指令响应状态的提示方法及显示设备

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060097658A1 (en) * 2004-10-29 2006-05-11 Vicent Chiang Apparatus for adjusting brightness of indicator light on panel
CN106550517A (zh) * 2015-09-18 2017-03-29 中兴通讯股份有限公司 一种控制指示灯亮度的方法及终端
CN107277995A (zh) * 2017-07-05 2017-10-20 环球智达科技（北京）有限公司 指示灯亮度的调节方法
CN107665697A (zh) * 2017-10-20 2018-02-06 维沃移动通信有限公司 一种屏幕亮度的调节方法及移动终端
CN111833858A (zh) * 2019-04-17 2020-10-27 百度在线网络技术（北京）有限公司 基于音箱的语音交互状态显示方法和装置
CN113066490A (zh) * 2021-03-16 2021-07-02 海信视像科技股份有限公司 一种唤醒响应的提示方法和显示设备
CN113421532A (zh) * 2021-07-09 2021-09-21 海信视像科技股份有限公司 背光调整方法及显示设备
CN114495934A (zh) * 2022-02-28 2022-05-13 海信视像科技股份有限公司 一种语音指令响应状态的提示方法及显示设备
CN114582343A (zh) * 2022-02-28 2022-06-03 海信视像科技股份有限公司 一种语音指令响应状态的提示方法及显示设备

Similar Documents

Publication Publication Date Title
US11812232B2 (en) Electronic device and music visualization method thereof
CN111033610A (zh) 电子装置及语音识别方法
US20180059774A1 (en) Display apparatus and control method thereof
CN114495934A (zh) 一种语音指令响应状态的提示方法及显示设备
JP2020513584A (ja) ディスプレイ装置及びその制御方法
US10061479B2 (en) Display system, information processing apparatus, computer readable recording medium, and power source control method
CN113066490B (zh) 一种唤醒响应的提示方法和显示设备
US11842707B2 (en) Systems and methods for adjusting light emitted from a display
CN113542851B (zh) 一种菜单刷新方法及显示设备
WO2023155529A1 (fr) Dispositif d'affichage, système domestique intelligent et procédé de commande multi-écran pour dispositif d'affichage
WO2022105417A1 (fr) Dispositif d'affichage et procédé de commande de dispositif
CN115641824A (zh) 画面调整设备、显示设备及画面调整方法
US11962865B2 (en) Display apparatus and process method for display apparatus
CN114582343A (zh) 一种语音指令响应状态的提示方法及显示设备
WO2023160087A1 (fr) Procédé d'invite pour état de réponse d'instruction vocale et dispositif d'affichage
CN112562666A (zh) 一种筛选设备的方法及服务设备
CN114915833A (zh) 一种显示器控制方法及显示设备、终端设备
WO2022193732A1 (fr) Procédé de commande de commutation pour canal de sortie audio et dispositif d'affichage
WO2022206392A1 (fr) Procédé d'activation d'application, dispositif d'affichage, terminal mobile et serveur
CN111479352B (zh) 显示设备及照明控制方法
CN114840114B (zh) 显示控制方法、装置及存储介质
CN114630163B (zh) 显示设备及快速开机方法
WO2022205793A1 (fr) Dispositif d'affichage et procédé de commande de dispositif
CN113965791B (zh) 悬浮窗口的返回方法和显示设备
WO2022222724A1 (fr) Procédé de commande de commutation pour canal de sortie audio, et dispositif d'affichage

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22928347

Country of ref document: EP

Kind code of ref document: A1