WO2020248788A1 - Procédé de commande vocale et dispositif d'affichage - Google Patents

Procédé de commande vocale et dispositif d'affichage Download PDF

Info

Publication number
WO2020248788A1
WO2020248788A1 PCT/CN2020/091476 CN2020091476W WO2020248788A1 WO 2020248788 A1 WO2020248788 A1 WO 2020248788A1 CN 2020091476 W CN2020091476 W CN 2020091476W WO 2020248788 A1 WO2020248788 A1 WO 2020248788A1
Authority
WO
WIPO (PCT)
Prior art keywords
application
voice
voice information
card
display device
Prior art date
Application number
PCT/CN2020/091476
Other languages
English (en)
Chinese (zh)
Inventor
王旭升
任晓楠
王峰
Original Assignee
海信视像科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 海信视像科技股份有限公司 filed Critical 海信视像科技股份有限公司
Publication of WO2020248788A1 publication Critical patent/WO2020248788A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/04—Real-time or near real-time messaging, e.g. instant messaging [IM]
    • H04L51/046—Interoperability with other network applications or services
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42203—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS] sound input device, e.g. microphone
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42204—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4781—Games
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/223—Execution procedure of a spoken command

Definitions

  • This application relates to the technical field of voice control, and in particular to a voice control method and display device.
  • Various applications such as video, chat, and game applications, are usually provided on display devices (such as smart TVs) to meet the diverse entertainment needs of users.
  • display devices such as smart TVs
  • voice control services which include controlling the running status of various applications on the display device according to the user's voice information.
  • the display device can control the two systems to run different applications at the same time, thereby providing a fusion scene.
  • the display device controls the first system to run a game application and controls the second system to run a chat application, thereby providing a "play and chat" scenario.
  • the integration scenario when users need to voice control these applications, because the voice control application is set in a specific system, but the two simultaneously running applications are in different systems, so the conventional voice control The method will not be able to realize the control of the dual-system integration scenario. Therefore, it is urgent to provide a voice control method suitable for dual-system fusion scenarios.
  • This embodiment provides a voice control method and a display device to perform voice control on a fusion scene of a display device with dual systems.
  • this embodiment provides a voice control method applied to a display device, and the method includes:
  • the application being a chess and card application, acquiring first voice information from the near-field voice module, and controlling the chess and card application to perform a card playing action according to the first voice information;
  • the method is applied to a second operating system on a second chip of the display device, the display device further includes a first operating system provided on a first chip, the first chip and The second chip is in communication connection; the application program is an application program installed in the first operating system or the second operating system.
  • the method further includes: in response to the application program including the chess and card application and the instant messaging application, obtaining first voice information from the near-field voice module, and according to the The first voice information controls the chess and card applications to perform card playing actions, and obtains second voice information from the far-field voice module, and sends the second voice information to the communication peer of the instant messaging application.
  • acquiring the first voice information from the near-field voice module includes: determining the first state information of the chess and card application; if the first state information indicates that the chess and card application is in a game state, Then, first voice information is obtained from the near-field voice module.
  • acquiring the second voice information from the far-field voice module includes: determining the second state information of the instant messaging application; if the second state information indicates that the instant messaging application is in a chat state, then The far-field voice module obtains second voice information.
  • controlling the chess and card application to perform card playing actions according to the first voice information includes:
  • the first voice information is sent to the chess and card application, the chess and card application is configured to receive the first voice information and send the first voice information to a cloud server, and receive a return from the cloud server.
  • the recognition result of the voice information, and the control of the chess and card application to execute the card playing action according to the recognition result is sent to the chess and card application, the chess and card application is configured to receive the first voice information and send the first voice information to a cloud server, and receive a return from the cloud server.
  • this embodiment provides a voice control method applied to a display device, and the method includes:
  • the first voice information is obtained from the near-field voice module, and the chess and card application are controlled to perform card playing actions according to the first voice information, and from the far-field voice
  • the module obtains the second voice information, and sends the second voice information to the communication peer of the instant messaging application.
  • this embodiment provides a display device, including a display device body and a remote controller,
  • the remote control is configured to receive a user's operation to control the focus in the interactive interface on the display screen of the display device body, and to receive the first voice input by the user;
  • the display device body includes:
  • the display screen is configured to display application windows of chess and card applications and/or instant messaging applications
  • a microphone configured to receive the second voice input by the user
  • the near-field voice module is configured to process the first voice received by the remote control, and generate first voice information according to the first voice;
  • a far-field voice module configured to process the second voice received by the microphone, and generate second voice information according to the second voice
  • the controller is configured to,
  • the application being a chess and card application, acquiring first voice information from the near-field voice module, and controlling the chess and card application to perform a card playing action according to the first voice information;
  • the display device body further includes a first operating system provided on a first chip, and a second operating system provided on a second chip, the first chip and the second chip Communication connection; the controller is installed on the second chip, and the application is an application installed in the first operating system or the second operating system.
  • the controller is further configured to, in response to the application program including the chess and card application and the instant messaging application, obtain first voice information from the near-field voice module, and According to the first voice information, the chess and card applications are controlled to perform the card playing action, and the second voice information is obtained from the far-field voice module, and the second voice information is sent to the communication pair of the instant messaging application end.
  • acquiring the first voice information from the near-field voice module includes: determining the first state information of the chess and card application; if the first state information indicates that the chess and card application is in a game state, from The near-field voice module obtains first voice information.
  • acquiring the second voice information from the far-field voice module includes: determining the second state information of the instant messaging application; if the second state information indicates that the instant messaging application is in a chat state, then The far-field voice module obtains second voice information.
  • controlling the chess and card application to perform card playing actions according to the first voice information includes:
  • the first voice information is sent to the chess and card application, the chess and card application is configured to receive the first voice information and send the first voice information to a cloud server, and receive a return from the cloud server.
  • the recognition result of the voice information, and the control of the chess and card application to execute the card playing action according to the recognition result is sent to the chess and card application, the chess and card application is configured to receive the first voice information and send the first voice information to a cloud server, and receive a return from the cloud server.
  • this embodiment provides a display device, including a display device body and a remote controller,
  • the remote control is configured to receive a user's operation to control the focus in the interactive interface on the display screen of the display device body, and to receive the first voice input by the user;
  • the display device body includes:
  • the display screen is configured to display application windows of chess and card applications and/or instant messaging applications
  • a microphone configured to receive the second voice input by the user
  • the near-field voice module is configured to process the first voice received by the remote control, and generate first voice information according to the first voice;
  • a far-field voice module configured to process the second voice received by the microphone, and generate second voice information according to the second voice
  • the controller is configured to,
  • the first voice information is obtained from the near-field voice module, and the chess and card application are controlled to perform card playing actions according to the first voice information, and from the far-field voice
  • the module obtains the second voice information, and sends the second voice information to the communication peer of the instant messaging application.
  • Fig. 1 exemplarily shows a schematic diagram of an operation scenario between a display device and a control device according to an embodiment
  • FIG. 2 exemplarily shows a block diagram of the hardware configuration of the control device 100 according to the embodiment
  • FIG. 3 exemplarily shows a block diagram of the hardware configuration of the display device 200 according to the embodiment
  • FIG. 4 exemplarily shows a block diagram of the hardware architecture of the display device 200 according to FIG. 3;
  • Fig. 5 exemplarily shows the first flowchart of the voice control method according to the embodiment
  • FIG. 6 exemplarily shows a schematic diagram of the functional configuration of the display device 200 according to the embodiment
  • FIG. 7a exemplarily shows a schematic diagram of software configuration in the display device 200 according to the embodiment
  • FIG. 7b exemplarily shows a configuration diagram of an application program in the display device 200 according to the embodiment.
  • FIG. 8 exemplarily shows a schematic diagram of a user interface in the display device 200 according to the embodiment.
  • Fig. 9 exemplarily shows the second flowchart of the voice control method according to the embodiment.
  • Fig. 10 exemplarily shows the third flowchart of the voice control method according to the embodiment
  • FIG. 11 exemplarily shows a schematic diagram of the display device according to the embodiment.
  • various external device interfaces are usually provided on the display device to facilitate the connection of different peripheral devices or cables to realize corresponding functions.
  • a high-definition camera is connected to the interface of the display device, if the hardware system of the display device does not have the hardware interface of the high-pixel camera that receives the source code, then the data received by the camera cannot be presented to the display of the display device. On the screen.
  • the hardware system of traditional display devices only supports one hard decoding resource, and usually only supports 4K resolution video decoding. Therefore, when you want to realize the video chat while watching Internet TV, in order not to reduce
  • the definition of the network video screen requires the use of hard decoding resources (usually the GPU in the hardware system) to decode the network video.
  • the general-purpose processor such as CPU
  • the video chat screen is processed by soft decoding.
  • this application discloses a dual hardware system architecture to realize multiple channels of video chat data (at least one local video).
  • module used in the various embodiments of this application can refer to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or a combination of hardware or/and software code that can execute related to the component Function.
  • remote control used in the various embodiments of this application refers to a component of an electronic device (such as the display device disclosed in this application), which can generally control the electronic device wirelessly within a short distance.
  • This component can generally use infrared and/or radio frequency (RF) signals and/or Bluetooth to connect to electronic devices, and can also include functional modules such as WiFi, wireless USB, Bluetooth, and motion sensors.
  • RF radio frequency
  • a handheld touch remote control uses a user interface in a touch screen to replace most of the physical built-in hard keys in general remote control devices.
  • gesture used in the embodiments of the present application refers to a user's behavior through a change of hand shape or hand movement to express expected ideas, actions, goals, and/or results.
  • the term "hardware system” used in the various embodiments of this application may refer to an integrated circuit (IC), a printed circuit board (Printed circuit board, PCB) and other mechanical, optical, electrical, and magnetic devices with computing , Control, storage, input and output functions.
  • IC integrated circuit
  • PCB printed circuit board
  • the hardware system is also usually referred to as a motherboard or a chip.
  • Fig. 1 exemplarily shows a schematic diagram of an operation scenario between a display device and a control device according to an embodiment. As shown in FIG. 1, the user can operate the display device 200 by controlling the device 100.
  • the control device 100 may be a remote controller 100A, which can communicate with the display device 200 through infrared protocol communication, Bluetooth protocol communication, ZigBee protocol communication or other short-distance communication methods for wireless or other communication
  • the display device 200 is controlled in a wired manner.
  • the user can control the display device 200 by inputting user instructions through keys on the remote control, voice input, control panel input, etc.
  • the user can control the display device 200 by inputting corresponding control commands through the volume plus and minus keys, channel control keys, up/down/left/right movement keys, voice input keys, menu keys, and switch buttons on the remote control. Function.
  • the remote control 100A can record the first voice of the user and transfer the first voice The voice is sent to the near field voice module of the display device 200.
  • the instant messaging application can communicate with the communication peer based on the user's instant operation.
  • instant messaging applications include, but are not limited to, WeChat, QQ, Momo and other applications.
  • the control device 100 can also be a smart device, such as a mobile terminal 100B, a tablet computer, a computer, a notebook computer, etc., which can be connected through a local area network (LAN, Wide Area Network), a wide area network (WAN, Wide Area Network), and a wireless local area network ((WLAN) , Wireless Local Area Network) or other networks communicate with the display device 200, and realize the control of the display device 200 through an application program corresponding to the display device 200.
  • LAN Local area network
  • WAN Wide Area Network
  • WLAN wireless local area network
  • the application can provide users with various controls through an intuitive user interface (UI, User Interface) on the screen associated with the smart device.
  • UI User Interface
  • both the mobile terminal 100B and the display device 200 can be installed with software applications, so that the connection and communication between the two can be realized through a network communication protocol, thereby realizing one-to-one control operation and data communication.
  • the mobile terminal 100B can establish a control command protocol with the display device 200, synchronize the remote control keyboard to the mobile terminal 100B, and control the display device 200 by controlling the user interface of the mobile terminal 100B; or the mobile terminal 100B
  • the audio and video content displayed on the screen is transmitted to the display device 200 to realize the synchronous display function.
  • the display device 200 can also communicate with the server 300 through multiple communication methods.
  • the display device 200 may be allowed to communicate with the server 300 via a local area network, a wireless local area network, or other networks.
  • the server 300 may provide various contents and interactions to the display device 200.
  • the display device 200 transmits and receives information, interacts with an Electronic Program Guide (EPG, Electronic Program Guide), receives software program updates, or accesses a remotely stored digital media library.
  • EPG Electronic Program Guide
  • the server 300 may be a group or multiple groups, and may be one or more types of servers.
  • the server 300 provides other network service content such as video on demand and advertising services.
  • the display device 200 may be a liquid crystal display, an OLED (Organic Light Emitting Diode) display, a projection display device, or a smart TV.
  • OLED Organic Light Emitting Diode
  • the specific display device type, size, resolution, etc. are not limited, and those skilled in the art can understand that the display device 200 can make some changes in performance and configuration as required.
  • the display device 200 may additionally provide a smart network TV function that provides a computer support function. Examples include Internet TV, Smart TV, Internet Protocol TV (IPTV) and so on.
  • IPTV Internet Protocol TV
  • the display device may be connected or provided with a camera, which is used to present the picture captured by the camera on the display interface of the display device or other display devices to realize interactive chats between users.
  • the image captured by the camera may be displayed on the display device in full screen, half screen, or any selectable area.
  • the camera is connected to the monitor rear shell through a connecting plate, and is fixedly installed on the upper middle of the monitor rear shell.
  • a connecting plate As an installable method, it can be fixedly installed at any position of the monitor rear shell to ensure its It is sufficient that the image capture area is not blocked by the rear shell, for example, the image capture area and the display device have the same orientation.
  • the camera can be connected to the display rear shell through a connecting plate or other conceivable connectors.
  • the connector is equipped with a lifting motor.
  • the user wants to use the camera or has an application to use the camera
  • it can be raised above the display.
  • the camera is not needed, it can be embedded behind the back shell to protect the camera from damage.
  • the camera used in this application may have 16 million pixels to achieve the purpose of ultra-high-definition display. In actual use, a camera with higher or lower than 16 million pixels can also be used.
  • the content displayed in different application scenarios of the display device can be merged in many different ways, so as to achieve functions that cannot be achieved by traditional display devices.
  • the user can video chat with at least one other user while watching a video program.
  • the presentation of the video program can be used as the background picture, and the video chat window is displayed on the background picture. Visually, you can call this function "watch and chat".
  • At least one video chat is performed across terminals.
  • the user can video chat with at least one other user while entering the education application for learning.
  • students can realize remote interaction with teachers while learning content in educational applications. Visually, you can call this function "learning and chatting”.
  • a video chat is conducted with players entering the game.
  • players entering the game.
  • a player enters a game application to participate in a game, it can realize remote interaction with other players. Visually, you can call this function "watch and play".
  • the game scene and the video image are merged, and the portrait in the video image is cut out and displayed on the game image, thereby improving user experience.
  • somatosensory games such as ball games, boxing games, running games, dancing games, etc.
  • human body postures and movements are acquired through a camera, limb detection and tracking, and key point data detection of human bones.
  • Animations are integrated in the game to realize games such as sports, dance and other scenes.
  • the user can interact with at least one other user in video and voice in the K song application.
  • multiple users can jointly complete the recording of a song.
  • the user can turn on the camera locally to obtain pictures and videos, which is vivid, and this function can be called "look in the mirror".
  • Fig. 2 exemplarily shows a configuration block diagram of the control device 100 according to an exemplary embodiment.
  • the control device 100 includes a controller 110, a communicator 130, a user input/output interface 140, a memory 190, and a power supply 180.
  • the control device 100 is configured to control the display device 200, and can receive user input operation instructions, and convert the operation instructions into instructions that can be recognized and responded to by the display device 200, and serve as an interactive intermediary between the user and the display device 200 effect.
  • the user operates the channel addition and subtraction keys on the control device 100, and the display device 200 responds to the channel addition and subtraction operations.
  • control device 100 may be a smart device.
  • control device 100 can install various applications for controlling the display device 200 according to user requirements.
  • the mobile terminal 100B or other smart electronic devices can perform similar functions to the control device 100 after installing an application for controlling the display device 200.
  • the user can install various function keys or virtual buttons of the graphical user interface that can be provided on the mobile terminal 100B or other smart electronic devices by installing applications to realize the function of the physical keys of the control device 100.
  • the controller 110 includes a processor 112, RAM 113 and ROM 114, a communication interface, and a communication bus.
  • the controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between internal components, and external and internal data processing functions.
  • the communicator 130 realizes communication of control signals and data signals with the display device 200 under the control of the controller 110. For example, the received user input signal is sent to the display device 200.
  • the communicator 130 may include at least one of communication modules such as a WIFI module 131, a Bluetooth module 132, and an NFC module 133.
  • the user input/output interface 140 wherein the input interface includes at least one of input interfaces such as a microphone 141, a touch panel 142, a sensor 143, and a button 144.
  • input interfaces such as a microphone 141, a touch panel 142, a sensor 143, and a button 144.
  • the user can implement the user instruction input function through voice, touch, gesture, pressing and other actions.
  • the input interface converts the received analog signal into a digital signal and the digital signal into a corresponding instruction signal, which is sent to the display device 200.
  • the microphone 141 is configured to receive a second voice input by the user and send the second voice to the display device 200 far-field voice module.
  • chess and card applications include, but are not limited to, go applications, chess applications, chess applications, backgammon applications, military chess applications, checkers applications, flying chess applications, mahjong applications, and poker applications.
  • the output interface includes an interface for sending the received user instruction to the display device 200.
  • it may be an infrared interface or a radio frequency interface.
  • the user input instruction needs to be converted into an infrared control signal according to the infrared control protocol, and sent to the display device 200 via the infrared sending module.
  • a radio frequency signal interface a user input instruction needs to be converted into a digital signal, which is then modulated according to the radio frequency control signal modulation protocol, and then sent to the display device 200 by the radio frequency transmitting terminal.
  • control device 100 includes at least one of a communicator 130 and an output interface.
  • the control device 100 is configured with a communicator 130, such as: WIFI, Bluetooth, NFC and other modules, which can encode user input instructions through the WIFI protocol, or Bluetooth protocol, or NFC protocol, and send to the display device 200.
  • a communicator 130 such as: WIFI, Bluetooth, NFC and other modules, which can encode user input instructions through the WIFI protocol, or Bluetooth protocol, or NFC protocol, and send to the display device 200.
  • the memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller 110.
  • the memory 190 can store various control signal instructions input by the user.
  • the power supply 180 is used to provide operating power support for each element of the control device 100 under the control of the controller 110. Can battery and related control circuit.
  • FIG. 3 exemplarily shows a hardware configuration block diagram of a hardware system in the display device 200 according to an exemplary embodiment.
  • the mechanism relationship of the hardware system can be shown in Figure 3.
  • one hardware system in the dual hardware system architecture is referred to as the first hardware system or A system, A chip, and the other hardware system is referred to as the second hardware system or N system, N chip.
  • the A chip contains the controller and various interfaces of the A chip
  • the N chip contains the controller and various interfaces of the N chip.
  • An independent operating system may be installed in the A chip and the N chip, so that there are two independent but interrelated subsystems in the display device 200.
  • the A chip and the N chip can realize connection, communication and power supply through multiple different types of interfaces.
  • the interface type of the interface between the A chip and the N chip may include general-purpose input/output (GPIO), USB interface, HDMI interface, UART interface, etc.
  • GPIO general-purpose input/output
  • USB interface USB interface
  • HDMI interface HDMI interface
  • UART interface UART interface
  • One or more of these interfaces can be used between the A chip and the N chip for communication or power transmission.
  • the N chip can be powered by an external power source, and the A chip can be powered by the N chip instead of the external power source.
  • the A chip may also include interfaces for connecting other devices or components, such as the MIPI interface for connecting to a camera (Camera) shown in FIG. 3, a Bluetooth interface, etc.
  • the N chip can also include a VBY interface for connecting to the display screen TCON (Timer Control Register), which is used to connect a power amplifier (Amplifier, AMP) and a speaker (Speaker). ) I2S interface; and IR/Key interface, USB interface, Wifi interface, Bluetooth interface, HDMI interface, Tuner interface, etc.
  • TCON Timer Control Register
  • AMP power amplifier
  • Speaker speaker
  • I2S interface I2S interface
  • IR/Key interface IR/Key interface
  • USB interface USB interface
  • Wifi interface Wireless Fidelity
  • Bluetooth interface HDMI interface
  • Tuner interface etc.
  • FIG. 4 is only an exemplary description of the dual hardware system architecture of the present application, and does not represent a limitation to the present application. In practical applications, both hardware systems can contain more or less hardware or interfaces as required.
  • FIG. 4 exemplarily shows a hardware architecture block diagram of the display device 200 according to FIG. 3.
  • the hardware system of the display device 200 may include an A chip and an N chip, and modules connected to the A chip or the N chip through various interfaces.
  • the N chip may include a tuner and demodulator 220, a communicator 230, an external device interface 250, a controller 210, a memory 290, a user input interface, a video processor 260-1, an audio processor 260-2, a display 280, and an audio output interface 272. Power supply. In other embodiments, the N chip may also include more or fewer modules.
  • the tuner and demodulator 220 is used to perform modulation and demodulation processing such as amplifying, mixing, and resonating broadcast television signals received through wired or wireless methods, thereby demodulating the user’s information from multiple wireless or cable broadcast television signals. Select the audio and video signals carried in the frequency of the TV channel, and additional information (such as EPG data signals).
  • the signal path of the tuner and demodulator 220 can be many kinds, such as: terrestrial broadcasting, cable broadcasting, satellite broadcasting or Internet broadcasting, etc.; and according to different modulation types, the signal adjustment method can be digitally modulated The method may also be an analog modulation method; and according to different types of received television signals, the tuner demodulator 220 may demodulate analog signals and/or digital signals.
  • the tuner and demodulator 220 is also used to respond to the TV channel frequency selected by the user and the TV signal carried by the frequency according to the user's selection and control by the controller 210.
  • the tuner demodulator 220 may also be in an external device, such as an external set-top box.
  • the set-top box outputs TV audio and video signals through modulation and demodulation, and inputs them to the display device 200 through the external device interface 250.
  • the communicator 230 is a component for communicating with external devices or external servers according to various communication protocol types.
  • the communicator 230 may include a WIFI module 231, a Bluetooth communication protocol module 232, a wired Ethernet communication protocol module 233, and an infrared communication protocol module and other network communication protocol modules or near field communication protocol modules.
  • the display device 200 may establish a control signal and a data signal connection with an external control device or content providing device through the communicator 230.
  • the communicator may receive the control signal of the remote controller 100 according to the control of the controller.
  • the external device interface 250 is a component that provides data transmission between the N chip controller 210 and the A chip and other external devices.
  • the external device interface can be connected to external devices such as set-top boxes, game devices, notebook computers, etc. in a wired/wireless manner, and can receive external devices such as video signals (such as moving images), audio signals (such as music), and additional information (such as EPG). ) And other data.
  • the external device interface 250 may include: a high-definition multimedia interface (HDMI) terminal 251, a composite video blanking synchronization (CVBS) terminal 252, an analog or digital component terminal 253, a universal serial bus (USB) terminal 254, red, green, and blue ( RGB) terminal (not shown in the figure) and any one or more.
  • HDMI high-definition multimedia interface
  • CVBS composite video blanking synchronization
  • USB universal serial bus
  • RGB red, green, and blue
  • the controller 210 controls the work of the display device 200 and responds to user operations by running various software control programs (such as an operating system and/or various application programs) stored on the memory 290.
  • various software control programs such as an operating system and/or various application programs
  • the controller 210 includes a read-only memory RAM 213, a random access memory ROM 214, a graphics processor 216, a CPU processor 212, a communication interface 218, and a communication bus.
  • RAM213 and ROM214, graphics processor 216, CPU processor 212, and communication interface 218 are connected by a bus.
  • the graphics processor 216 is used to generate various graphics objects, such as icons, operation menus, and user input instructions to display graphics. Including an arithmetic unit, which performs operations by receiving various interactive commands input by the user, and displays various objects according to display attributes. As well as including a renderer, various objects obtained based on the arithmetic unit are generated, and the rendering result is displayed on the display 280.
  • the CPU processor 212 is configured to execute operating system and application program instructions stored in the memory 290. And according to receiving various interactive instructions input from the outside, to execute various applications, data and content, so as to finally display and play various audio and video content.
  • the CPU processor 212 may include multiple processors.
  • the multiple processors may include one main processor and multiple or one sub-processors.
  • the main processor is used to perform some operations of the display device 200 in the pre-power-on mode, and/or to display images in the normal mode.
  • the communication interface may include the first interface 218-1 to the nth interface 218-n. These interfaces may be network interfaces connected to external devices via a network.
  • the controller 210 may control the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 280, the controller 210 may perform an operation related to the object selected by the user command.
  • the object may be any one of the selectable objects, such as a hyperlink or an icon.
  • Operations related to the selected object for example: display operations connected to hyperlink pages, documents, images, etc., or perform operations corresponding to the icon.
  • the user command for selecting the UI object may be a command input through various input devices (for example, a mouse, a keyboard, a touch pad, etc.) connected to the display device 200 or a voice command corresponding to the voice spoken by the user.
  • Chess and card applications are set to be installed in the first system of the A chip (ie, the first chip), and instant messaging applications are set to be installed in the second system of the N chip (ie, the second chip).
  • the N chip of the display device 200 in this embodiment is also configured to execute a voice control method.
  • the voice control method provided in this embodiment includes the following steps S501-S503.
  • Step S501 Determine an application in the display device 200 that provides display content for the display window in the current interactive interface.
  • the content provided by the application program is displayed in the display window of the current interactive interface of the display device 200, and different application programs correspond to different display windows.
  • the display window of only one application can be displayed at the same time, or the display windows of two running applications can be displayed at the same time.
  • the current interactive interface of the display device 200 only displays display windows of chess and card applications, or only displays display windows of instant messaging applications.
  • the display device 200 simultaneously displays the display window of the chess and card application and the display window of the instant messaging application in the current interactive interface at the same time.
  • the controller 210 After the controller 210 detects that a display window is displayed in the current interactive interface of the display device 200, it will determine the type of application that provides display content to the display window. Specifically, the controller 210 needs to determine whether the application is a chess or card application or an instant messaging application.
  • Step S502 in response to the application being a chess and card application, the controller 210 obtains first voice information from the near-field voice module, and controls the chess and card application to perform a card playing action according to the first voice information.
  • the controller 210 is also configured to receive installations in the first system or the second system.
  • the current state information sent by the application program after switching the application state realizes the management and control of the current state information of all the applications currently running on the display device 200.
  • Application switching application status includes application startup, application scene switching, etc. Moreover, whether it is an application in the first system or the second system, after switching the application state, it needs to send its current state information to the controller 210.
  • Example 1 after the playing card application in the first system is started, the playing card application sends its current state information to the controller 210 to indicate that it is currently in the initial starting state.
  • the playing card application after the playing card application in the first system performs a card playing action, the playing card application sends its current status information to the controller 210 to indicate its current card game status.
  • the WeChat sends its current status information to the controller 210 to indicate that it is currently in the initial startup status.
  • Example 4 after WeChat in the second system is closed, WeChat sends its current status information to the controller 210 to indicate that it is currently in an application closed state.
  • the controller 210 may first need to determine the first state information of the chess application, where the first state information Used to indicate the current status of chess and card applications, such as game status, game mode selection status, etc. If the first state information indicates that the chess and card application is in a game state, the first voice information is acquired from the near-field voice module to avoid redundant voice acquisition operations.
  • the controller 210 After the controller 210 obtains the first voice information from the near-field voice module, it can control the chess and card applications to execute the card playing action according to the first voice information.
  • the controller 210 when the controller 210 controls the chess and card application to execute the card playing action according to the first voice information, it first sends the first voice information to the cloud server, so that the first voice information can be accessed by the cloud server. Speech recognition. Second, the controller 210 receives the recognition result of the first voice information returned by the cloud server. Finally, the controller 210 sends the recognition result to the chess and card applications, and the recognition result is used to control the chess and card applications to perform the card playing action.
  • the controller 210 when the recognition result "playing two Aces" sent by the cloud server received by the controller 210, the controller 210 sends it to the poker application for poker The card application performs the operation of playing two Aces based on the result.
  • the controller 210 sends the first voice information to the chess and card applications when controlling the chess and card applications to perform the card playing action according to the first voice information, so that the chess and card applications can identify the first Voice information, and perform the card action based on the recognition result.
  • the chess and card application is configured to receive the first voice information and send the first voice information to the cloud server, receive the recognition result of the voice information returned by the cloud server, and execute the card playing action according to the recognition result.
  • the controller 210 when the controller 210 receives the first voice information, it sends the first voice information to the poker application. After receiving the first voice information, the poker application first sends the first voice information to the cloud server to recognize the first voice information. Subsequently, the poker application receives the recognition result "play two Aces" sent by the cloud server. Finally, the playing card application performs the operation of playing two Aces based on the recognition result.
  • Step S503 In response to the application being an instant messaging application, obtain second voice information from the far-field voice module, and send the second voice information to the communication peer of the instant messaging application.
  • the controller 210 may first need to determine the second state information of the instant messaging application.
  • the second status information is used to indicate the current status of chess and card applications, such as chat status, friend selection status, application setting status, and so on. If the second state information indicates that the instant messaging application is in a chat state, the second voice information is acquired from the far-field voice module to avoid redundant voice acquisition operations.
  • the controller 210 After the controller 210 obtains the second voice information from the far-field voice module, it can send the second voice information to the instant messaging application, so that it can send the second voice information to the communication peer, so as to realize the voice information between two ends or multiple ends. Transmission.
  • the controller 210 when the display window of the chess and card application is displayed on the current interactive interface on the display device 200, the controller 210 only executes steps S501-S502, and does not execute step S503.
  • the far-field voice communication module does not work, or the far-field voice communication module works but the second voice information generated by it is not responded by the controller 210.
  • the controller 210 When only the display window of the instant messaging application is displayed in the current interactive interface of the display device 200, the controller 210 only executes steps S501 and S503, and does not execute step S502. In addition, in this scenario, the near-field voice communication module does not work, or the near-field voice communication module works but the first voice information it generates is not responded by the controller 210.
  • the controller is configured to perform step S502 and step S503 according to a preset sequence after performing step S501.
  • the controller 210 may first perform step S502, or may first perform step S503.
  • the voice control method and display device can obtain the first voice information for controlling chess and card applications from the near-field voice module, and obtain the voice information from the far-field voice module even in the communication process of the communication application, thereby realizing the integration scenario Control or service different applications running at the same time.
  • the memory 290 includes storing various software modules for driving and controlling the display device 200.
  • various software modules stored in the memory 290 include: a basic module, a detection module, a communication module, a display control module, a browser module, and various service modules.
  • the basic module is the underlying software module used for signal communication between various hardware in the display device 200 and sending processing and control signals to the upper module.
  • the detection module is a management module used to collect various information from various sensors or user input interfaces, and perform digital-to-analog conversion and analysis management.
  • the voice recognition module includes a voice analysis module and a voice command database module.
  • the display control module is a module for controlling the display 280 to display image content, and can be used to play information such as multimedia image content and UI interfaces.
  • the communication module is a module used for control and data communication with external devices.
  • the browser module is a module used to perform data communication between browsing servers.
  • the service module is a module used to provide various services and various applications.
  • the memory 290 is also used to store and receive external data and user data, images of various items in various user interfaces, and visual effect diagrams of focus objects.
  • the user input interface is used to send a user's input signal to the controller 210, or to transmit a signal output from the controller to the user.
  • the control device (such as a mobile terminal or remote control) can send input signals input by the user, such as a power switch signal, a channel selection signal, and a volume adjustment signal, to the user input interface, and then the user input interface forwards the input signal to the control.
  • the control device may receive output signals such as audio, video, or data output from the user input interface processed by the controller, and display the received output signal or output the received output signal as audio or vibration.
  • the user may input a user command through a graphical user interface (GUI) displayed on the display 280, and the user input interface receives the user input command through the graphical user interface (GUI).
  • GUI graphical user interface
  • the user can input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through the sensor to receive the user input command.
  • the video processor 260-1 is used to receive video signals, and perform video data processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to the standard codec protocol of the input signal.
  • the video signal displayed or played directly on the display 280.
  • the video processor 260-1 includes a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, a display formatting module, and the like.
  • the demultiplexing module is used to demultiplex the input audio and video data stream. For example, if MPEG-2 is input, the demultiplexing module will demultiplex into video signals and audio signals.
  • the video decoding module is used to process the demultiplexed video signal, including decoding and scaling.
  • An image synthesis module such as an image synthesizer, is used to superimpose and mix the GUI signal generated by the graphics generator with the zoomed video image according to user input or itself to generate an image signal for display.
  • Frame rate conversion module used to convert the frame rate of the input video, such as converting the frame rate of the input 24Hz, 25Hz, 30Hz, 60Hz video to the frame rate of 60Hz, 120Hz or 240Hz, where the input frame rate can be compared with the source
  • the video stream is related, and the output frame rate can be related to the update rate of the display.
  • the input has a usual format, such as frame insertion.
  • the display formatting module is used to change the signal output by the frame rate conversion module into a signal that conforms to a display format such as a display, such as format conversion of the signal output by the frame rate conversion module to output RGB data signals.
  • the display 280 is used to receive the image signal input from the video processor 260-1, display video content and images, and a menu control interface.
  • the display 280 includes a display component for presenting a picture and a driving component for driving image display.
  • the displayed video content can be from the video in the broadcast signal received by the tuner and demodulator 220, or from the video content input by the communicator or the interface of an external device.
  • the display 220 simultaneously displays a user manipulation interface UI generated in the display device 200 and used to control the display device 200.
  • the display 280 it also includes a driving component for driving the display.
  • the display 280 is a projection display, it may also include a projection device and a projection screen.
  • the display 280 is used to display a display window in the current interaction interface between the display device 200 and the user, and the display content of the display window is provided by a corresponding application.
  • the display 280 may be used to display a display window of a chess and card application, or a display window of an instant messaging application.
  • the display 280 may display only one display window in the current interactive interface at the same time, for example, only display the display window of chess and card applications or the display window of instant messaging applications, or it may display only one display window in the current interactive interface at the same time.
  • Two display windows are displayed at the same time, such as the display window of chess and card applications and the display window of instant messaging applications.
  • the audio processor 260-2 is used to receive audio signals, and perform decompression and decoding according to the standard codec protocol of the input signal, as well as audio data processing such as noise reduction, digital-to-analog conversion, and amplification processing, and the result can be in the speaker 272 The audio signal to be played.
  • the audio processor 260-2 includes a near-field voice module and a far-field voice module.
  • the near field is configured to process the first voice received by the microphone on the remote control, and generate first voice information according to the first voice.
  • the far-field voice module is configured to process the second voice received by the microphone on the body of the display device, and to generate second voice information according to the second voice.
  • the audio output interface 270 is used to receive the audio signal output by the audio processor 260-2 under the control of the controller 210.
  • the audio output interface may include a speaker 272 or output to an external audio output terminal 274 of a generator of an external device, such as : External audio terminal or headphone output terminal, etc.
  • the video processor 260-1 may include one or more chips.
  • the audio processor 260-2 may also include one or more chips.
  • the video processor 260-1 and the audio processor 260-2 may be separate chips, or they may be integrated with the controller 210 in one or more chips.
  • the power supply is used to provide power supply support for the display device 200 with power input from an external power supply under the control of the controller 210.
  • the power supply may include a built-in power supply circuit installed inside the display device 200, or may be a power supply installed outside the display device 200, such as a power interface that provides an external power supply in the display device 200.
  • the A chip may include a controller 310, a communicator 330, a detector 340, and a memory 390. In some embodiments, it may also include a user input interface, a video processor, an audio processor, a display, and an audio output interface. In some embodiments, there may also be a power supply that independently powers the A chip.
  • the communicator 330 is a component for communicating with external devices or external servers according to various communication protocol types.
  • the communicator 330 may include a WIFI module 331, a Bluetooth communication protocol module 332, a wired Ethernet communication protocol module 333, and an infrared communication protocol module and other network communication protocol modules or near field communication protocol modules.
  • the communicator 330 of the A chip and the communicator 230 of the N chip also interact with each other.
  • the WiFi module 231 of the N chip is used to connect to an external network and generate network communication with an external server and the like.
  • the WiFi module 331 of the A chip is used to connect to the WiFi module 231 of the N chip, and does not directly connect to an external network or the like. Therefore, for the user, a display device as in the above embodiment can externally display a WiFi account.
  • the detector 340 is a component used by the chip of the display device A to collect signals from the external environment or interact with the outside.
  • the detector 340 may include a light receiver 342, a sensor used to collect the intensity of ambient light, which can adaptively display parameter changes by collecting ambient light, etc.; it may also include an image collector 341, such as a camera, a camera, etc., which can be used to collect external Environmental scenes, as well as gestures used to collect user attributes or interact with users, can adaptively change display parameters, and can also recognize user gestures to achieve the function of interaction with users.
  • the external device interface 350 provides components for data transmission between the controller 310 and the N chip or other external devices.
  • the external device interface can be connected to external devices such as set-top boxes, game devices, notebook computers, etc., in a wired/wireless manner.
  • the controller 310 controls the work of the display device 200 and responds to user operations by running various software control programs (such as installed third-party applications, etc.) stored on the memory 390 and interacting with the N chip.
  • various software control programs such as installed third-party applications, etc.
  • the controller 310 includes a read-only memory ROM 313, a random access memory RAM 314, a graphics processor 316, a CPU processor 312, a communication interface 318, and a communication bus.
  • the ROM 313 and the RAM 314, the graphics processor 316, the CPU processor 312, and the communication interface 318 are connected by a bus.
  • the CPU processor 312 runs the system startup instruction in the ROM, and copies the operating system stored in the memory 390 to the RAM 314 to start the startup operating system. After the operating system is started, the CPU processor 312 copies various application programs in the memory 390 to the RAM 314, and then starts to run and start various application programs.
  • the CPU processor 312 is used to execute the operating system and application instructions stored in the memory 390, communicate with the N chip, transmit and interact with signals, data, instructions, etc., and execute various interactive instructions received from external inputs Various applications, data and content, in order to finally display and play various audio and video content.
  • the communication interface may include the first interface 318-1 to the nth interface 318-n. These interfaces may be network interfaces connected to external devices via a network, or network interfaces connected to the N chip via a network.
  • the controller 310 may control the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 280, the controller 210 may perform an operation related to the object selected by the user command.
  • the graphics processor 316 is used to generate various graphics objects, such as icons, operation menus, and user input instructions to display graphics. Including an arithmetic unit, which performs operations by receiving various interactive commands input by the user, and displays various objects according to display attributes. As well as including a renderer, various objects obtained based on the arithmetic unit are generated, and the rendering result is displayed on the display 280.
  • Both the graphics processor 316 of the A chip and the graphics processor 216 of the N chip can generate various graphics objects. Differentily, if application 1 is installed on the A chip and application 2 is installed on the N chip, when the user is in the interface of the application 1 and the user inputs instructions in the application 1, the A chip graphics processor 316 generates a graphic object. When the user is on the interface of Application 2 and performs the user-input instructions in Application 2, the graphics processor 216 of the N chip generates the graphics object.
  • Fig. 6 exemplarily shows a schematic diagram of a functional configuration of a display device according to an exemplary embodiment.
  • the memory 390 of the A chip and the memory 290 of the N chip are respectively used to store the operating system, application programs, content and user data, etc., under the control of the controller 310 of the A chip and the controller 210 of the N chip. Perform system operations that drive the display device 200 and respond to various operations of the user.
  • the memory 390 of the A chip and the memory 290 of the N chip may include volatile and/or nonvolatile memory.
  • the memory 290 is specifically used to store the operating program that drives the controller 210 in the display device 200, and store various application programs built in the display device 200, and various application programs downloaded by the user from an external device, and application programs.
  • the memory 290 is used to store system software such as an operating system (OS) kernel, middleware, and applications, and to store input video data and audio data, and other user data.
  • OS operating system
  • the memory 290 is specifically used to store driver programs and related data such as the video processor 260-1 and the audio processor 260-2, the display 280, the communication interface 230, the tuner and demodulator 220, and the input/output interface.
  • the memory 290 may store software and/or programs.
  • the software programs used to represent an operating system (OS) include, for example, a kernel, middleware, application programming interface (API), and/or application programs.
  • the kernel may control or manage system resources, or functions implemented by other programs (such as the middleware, API, or application program), and the kernel may provide interfaces to allow middleware and APIs, or application access Controller to control or manage system resources.
  • the memory 290 includes a broadcast receiving module 2901, a channel control module 2902, a volume control module 2903, an image control module 2904, a display control module 2905, an audio control module 2906, an external command recognition module 2907, a communication control module 2908, and an optical receiver Module 2909, power control module 2910, operating system 2911, and other application programs 2912, browser module, etc.
  • the controller 210 executes various software programs in the memory 290, such as: broadcast and television signal reception and demodulation function, TV channel selection control function, volume selection control function, image control function, display control function, audio control function, external command Various functions such as identification function, communication control function, optical signal receiving function, power control function, software control platform supporting various functions, and browser function.
  • the memory 390 includes storing various software modules for driving and controlling the display device 200.
  • various software modules stored in the memory 390 include: a basic module, a detection module, a communication module, a display control module, a browser module, and various service modules. Since the functions of the memory 390 and the memory 290 are relatively similar, please refer to the memory 290 for related parts, and will not be repeated here.
  • the memory 390 includes an image control module 3904, an audio control module 2906, an external command recognition module 3907, a communication control module 3908, an optical receiving module 3909, an operating system 3911, and other application programs 3912, a browser module, and so on.
  • the controller 210 executes various software programs in the memory 290, such as: image control function, display control function, audio control function, external command recognition function, communication control function, light signal receiving function, power control function, support for various Functional software control platform, and various functions such as browser functions.
  • the external command recognition module 2907 of the N chip and the external command recognition module 3907 of the A chip can recognize different commands.
  • the external command recognition module 3907 of the A chip may include a graphic recognition module 2907-1.
  • the graphic recognition module 3907-1 stores a graphic database, and the camera receives When the graphics instructions are sent to the outside world, they correspond to the instructions in the graphics database to control the display device.
  • the voice receiving device and the remote controller are connected to the N chip, the external command recognition module 2907 of the N chip may include a voice recognition module 2907-2.
  • the graphics recognition module 2907-2 stores a voice database, and the voice receiving device, etc.
  • the external voice commands or time correspond to the commands in the voice database to control the display device.
  • the control device 100 such as a remote controller is connected to the N chip, and the key command recognition module interacts with the control device 100.
  • Fig. 7a exemplarily shows a configuration block diagram of a software system in the display device 200 according to an exemplary embodiment.
  • the operating system 2911 includes operating software for processing various basic system services and for implementing hardware-related tasks, acting as a medium for data processing between application programs and hardware components.
  • part of the operating system kernel may include a series of software to manage the hardware resources of the display device and provide services for other programs or software codes.
  • part of the operating system kernel may include one or more device drivers, and the device drivers may be a set of software codes in the operating system to help operate or control devices or hardware associated with the display device.
  • the drive may contain code to manipulate video, audio, and/or other multimedia components. Examples include displays, cameras, Flash, WiFi, and audio drivers.
  • the accessibility module 2911-1 is used to modify or access the application program, so as to realize the accessibility of the application program and the operability of its display content.
  • the communication module 2911-2 is used to connect to other peripherals via related communication interfaces and communication networks.
  • the user interface module 2911-3 is used to provide objects that display the user interface for access by various applications, and can realize user operability.
  • the control application 2911-4 is used to control process management, including runtime applications.
  • the event transmission system 2914 can be implemented in the operating system 2911 or in the application 2912. In some embodiments, it is implemented in the operating system 2911 on the one hand, and implemented in the application program 2912 at the same time, for monitoring various user input events, and responding to the recognition results of various events or sub-events according to various events. And implement one or more sets of pre-defined operation procedures.
  • the event monitoring module 2914-1 is used to monitor input events or sub-events of the user input interface.
  • the event recognition module 2914-1 is used to input the definitions of various events to various user input interfaces, recognize various events or sub-events, and transmit them to the processing to execute the corresponding one or more groups of processing programs .
  • the event or sub-event refers to the input detected by one or more sensors in the display device 200 and the input of an external control device (such as the control device 100).
  • an external control device such as the control device 100.
  • various sub-events of voice input, gesture input sub-events of gesture recognition, and sub-events of remote control button command input of control devices include multiple forms, including but not limited to one or a combination of pressing up/down/left/right/, confirming keys, and pressing keys.
  • non-physical buttons such as moving, pressing, and releasing.
  • the interface layout management module 2913 which directly or indirectly receives various user input events or sub-events monitored by the event transmission system 2914, is used to update the layout of the user interface, including but not limited to the position of each control or sub-control in the interface, and the container
  • the size, position, level, etc. of the interface are related to various execution operations.
  • the application layer of the display device includes various application programs that can be executed on the display device 200.
  • the application layer 2912 of the N chip may include, but is not limited to, one or more applications, such as video-on-demand applications, application centers, and game applications.
  • the application layer 3912 of the A chip may include, but is not limited to, one or more applications, such as a live TV application, a media center application, and so on. It should be noted that the application programs contained on the A chip and the N chip are determined according to the operating system and other designs, and the present invention does not need to specifically limit and divide the application programs contained on the A chip and the N chip.
  • Live TV applications can provide live TV through different sources.
  • a live TV application may use input from cable TV, wireless broadcasting, satellite services, or other types of live TV services to provide TV signals.
  • the live TV application can display the video of the live TV signal on the display device 200.
  • Video-on-demand applications can provide videos from different storage sources. Unlike live TV applications, VOD provides video display from certain storage sources. For example, the video on demand can come from the server side of cloud storage, and from the local hard disk storage that contains the stored video programs.
  • Media center applications can provide various multimedia content playback applications.
  • the media center can provide services that are different from live TV or video on demand, and users can access various images or audio through the media center application.
  • Application center can provide storage of various applications.
  • the application program may be a game, an application program, or some other application program that is related to a computer system or other device but can be run on a display device.
  • the application center can obtain these applications from different sources, store them in the local storage, and then run on the display device 200.
  • FIG. 8 exemplarily shows a schematic diagram of a user interface in the display device 200 according to an exemplary embodiment.
  • the user interface includes multiple view display areas, for example, a first view display area 201 and a play screen 202, where the play screen includes one or more different items laid out.
  • the user interface also includes a selector indicating that the item is selected, and the position of the selector can be moved through user input to change the selection of different items.
  • multiple view display areas can present display screens of different levels.
  • the first view display area can present video chat item content
  • the second view display area can present application layer item content (eg, webpage video, VOD display, application program screen, etc.).
  • the presentation of different view display areas has different priorities, and the display priorities of the view display areas are different between view display areas with different priorities.
  • the priority of the system layer is higher than that of the application layer.
  • the same level of display screen can also be presented.
  • the selector can switch between the display area of the first view and the display area of the second view, and when the size and position of the display area of the first view change, the second view The size and position of the display area can be changed at any time.
  • both the A chip and the N chip can be independently installed with Android and various APPs, so that each chip can realize a certain function, and the A chip and the N chip can realize a certain function in cooperation.
  • the method is applied to the state synchronization module in the display device, and the state synchronization module is set in the second system. Please refer to FIG. 9.
  • the method specifically includes the following steps S901-S903.
  • Step S901 The state synchronization module receives the current state information sent by the application after the application state is switched, and the application is an application installed in the first system or the second system.
  • application state switching includes application startup, application scene switching, and so on. Moreover, whether it is an application in the first system or the second system, after switching the application state, it needs to send its current state information to the state synchronization module, so that the state synchronization module can check all the current running on the display device. The current status information of the application is controlled.
  • Example 1 After the playing card application in the first system is started, the playing card application sends its current state information to the state synchronization module to indicate that it is currently in the initial startup state.
  • Example 2 After the playing card application in the first system performs a card play action, the playing card application sends its current status information to the status synchronization module to indicate its current card game status.
  • Example 3 After the chat application in the second system is started, the chat application sends its current state information to the state synchronization module to indicate that it is currently in the initial startup state.
  • Example 4 After the chat application in the second system is closed, the chat application sends its current state information to the state synchronization module to indicate that it is currently in the application closed state.
  • Step S902 The state synchronization module obtains the voice information of the voice module according to the current state information.
  • the state synchronization module After receiving a piece of current state information, the state synchronization module needs to analyze the current state information to determine its corresponding application and the state type of the application.
  • the state synchronization module when the state synchronization module receives the current state information shown in the second example, it analyzes the current state information and determines that it is a specific card game state of the poker application, and then the voice in the second system The module obtains the current voice information in real time.
  • the state synchronization module when the state synchronization module receives the current state information shown in the third example, it parses the current state information and determines that it is the initial startup state of the recording application, and then the voice module in the second system Get current voice information in real time.
  • the state synchronization module will receive the current state information sent by the poker application and the chat application respectively. After the state synchronization module obtains the voice information in real time from the voice module, it separates the voice information into two channels to provide services for the playing card application and the chat application respectively.
  • the voice module may be a far-field voice module, such as a microphone of a display device, or a near-field voice module, such as a microphone of a remote controller of the display device, which is not limited in this embodiment.
  • Step S903 The state synchronization module controls the application program according to the current state information and voice information.
  • game applications such as playing cards will switch the application state every time the scene is switched, for example, after each card is played, and request a voice message from the voice module; and the chat scene is starting Between and closing, usually only need to send a recording request to the voice module once, after that, you can continuously receive voice messages. Therefore, in this fusion scenario, a scenario in which a game application and a chat application simultaneously request voice information of the voice module is bound to be involved.
  • the state synchronization module After the state synchronization module receives the voice information sent by the voice module, it will separate the voice information to obtain two channels of voice information.
  • the status synchronization module sends one of them to the chat application, so that the chat application sends to the peer device currently chatting to maintain the current chat service.
  • the state synchronization module sends another channel of voice information to the cloud server to be recognized by the cloud server, and returns the recognition result. After receiving the recognition result, the state synchronization module sends a control instruction to the game application according to the recognition result and the current state information of the game application.
  • the status synchronization module when the recognition result obtained by the status synchronization module is "playing two aces", the status synchronization module will send a control instruction to instruct the poker application to perform two aces. A's operation.
  • step S901 the playing card application will jump to step S901 and send the current state information to the state synchronization module again.
  • the voice control method based on the dual-system integration scenario provided in this implementation further includes the following steps S1001-S1002 before step S901.
  • Step S1001 The state synchronization module receives an application start request sent by an application of the first system or the second system.
  • the state synchronization module is used to manage the states of all applications in the first system and the second system, and to manage the state switching of the managed applications. Therefore, when each application in the first system and the second system is started, it needs to send an application start request to the state synchronization module.
  • the playing card application when the user activates the playing card application in the first system through the remote control, the playing card application sends an application activation request to the state synchronization module.
  • the chat application when the user starts the chat application of the second system through the voice control module of the display device, the chat application sends an application start request to the state synchronization module.
  • Step S1002 The state synchronization module sends an application start instruction to the application program.
  • the application program starts the application program after receiving the application start instruction.
  • the playing card application in the first system starts the playing card application after receiving the application start instruction sent by the state synchronization module in the second system.
  • the chat application in the second system starts the chat application after receiving the application start instruction sent by the state synchronization module in the second system.
  • the display device simultaneously manages the current state information of the two systems through a state synchronization module, and divides the voice information of the voice module into two channels Perform processing to simultaneously manage and control the two applications in the dual system based on the same voice information.
  • the display device includes a first system and a second system.
  • the two systems can communicate through the RPC communication mechanism.
  • the second system is configured as the main system of the display device, which can obtain the user's voice information and run applications such as chat
  • the first system is configured as the auxiliary system of the display device and can run applications such as games.
  • a first state management module 1101 and a first communication module 1102 are provided in the first system.
  • a first communication module 1103, a state synchronization module 1104, a voice module 1105, and a second communication module 1106 are provided in the second system.
  • the first state management module 1101 is configured to send the current state information of the application program to the state synchronization module 1104 after the application program of the first system switches the application state.
  • the first communication module 1103 is configured to send the current state information of the application to the state synchronization module 1104 after the application of the second system switches the application state.
  • the state synchronization module 1104 is used to receive the current state information of the first system or the second system application; obtain the voice information of the voice module 1105 according to the current state information; and control the application based on the current state information and voice information.
  • the voice module 1105 is used to obtain the user's voice information.
  • the state synchronization module 1104 is further configured to receive an application startup request sent by the application program; and send an application startup instruction to the application program.
  • the state synchronization module 1104 is also used to send voice information to the cloud server; receive the recognition result returned by the cloud server; control the application according to the recognition result and current status information .
  • the state synchronization module 1104 is further configured to send voice information to the application according to the current state information.
  • the display device provided in this embodiment can simultaneously control the current state information of the two systems through a state synchronization module in a fusion scene such as "play while chatting", and divide the voice information of the voice module into two channels for processing.
  • a state synchronization module in a fusion scene such as "play while chatting”
  • divide the voice information of the voice module into two channels for processing.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

L'invention concerne un procédé de commande vocale et un dispositif d'affichage (200). Le procédé et le dispositif d'affichage (200) comprennent les étapes suivantes : détermination d'un programme d'application dans le dispositif d'affichage (200) qui fournit un contenu d'affichage pour une fenêtre d'affichage dans une interface interactive courante (S501) ; en réponse au fait que le programme d'application est une application de jeux d'échecs et de cartes, acquisition de premières informations vocales auprès d'un module vocal en champ proche et commande de l'application de jeux d'échecs et de cartes pour exécuter une action de jeu de cartes en fonction des premières informations vocales (S502) ; et en réponse au fait que le programme d'application est une application de messagerie instantanée, acquisition de secondes informations vocales auprès d'un module vocal en champ lointain et envoi des secondes informations vocales à un homologue de communication de l'application de messagerie instantanée (S503). Le procédé de commande vocale et le dispositif d'affichage (200) décrits peuvent réaliser la gestion et la commande ou le service de différents programmes d'application s'exécutant en même temps dans un scénario fusionné.
PCT/CN2020/091476 2019-06-10 2020-05-21 Procédé de commande vocale et dispositif d'affichage WO2020248788A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN201910498059 2019-06-10
CN201910498059.0 2019-06-10
CN201911306539.9 2019-12-18
CN201911306539.9A CN112071312B (zh) 2019-06-10 2019-12-18 一种语音控制方法及显示设备

Publications (1)

Publication Number Publication Date
WO2020248788A1 true WO2020248788A1 (fr) 2020-12-17

Family

ID=73658661

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/091476 WO2020248788A1 (fr) 2019-06-10 2020-05-21 Procédé de commande vocale et dispositif d'affichage

Country Status (2)

Country Link
CN (1) CN112071312B (fr)
WO (1) WO2020248788A1 (fr)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103611294A (zh) * 2013-11-27 2014-03-05 南京航空航天大学 一种棋牌类游戏语音控制装置及其控制方法
WO2015011217A1 (fr) * 2013-07-24 2015-01-29 Telefonica Digital España, S.L.U. Interface utilisateur utilisant des étiquettes rfid ou la voix comme entrée
CN104409076A (zh) * 2014-12-02 2015-03-11 上海语知义信息技术有限公司 用于棋牌类游戏的语音操控系统及语音操控方法
CN105390135A (zh) * 2015-10-12 2016-03-09 四川长虹电器股份有限公司 游戏语音控制系统
CN105978783A (zh) * 2016-03-21 2016-09-28 上海季诺金融信息服务有限公司 一种基于即时通讯软件的语音控制方法
CN109036411A (zh) * 2018-09-05 2018-12-18 深圳市友杰智新科技有限公司 一种智能终端语音交互控制方法及装置
CN109243444A (zh) * 2018-09-30 2019-01-18 百度在线网络技术（北京）有限公司 语音交互方法、设备及计算机可读存储介质

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104424953B (zh) * 2013-09-11 2019-11-01 华为技术有限公司 语音信号处理方法与装置
KR102202110B1 (ko) * 2014-04-30 2021-01-13 삼성전자주식회사 서비스 제공 방법, 전자 장치 및 저장 매체
US9412394B1 (en) * 2015-03-09 2016-08-09 Jigen Labs, LLC Interactive audio communication system
CN205430726U (zh) * 2015-12-14 2016-08-03 深圳市轻生活科技有限公司 可组网的语音识别灯及其语音识别灯控系统
CN105843466B (zh) * 2016-03-14 2020-03-31 广州趣丸网络科技有限公司 一种实时语音的方法及装置
CN106331956A (zh) * 2016-11-04 2017-01-11 北京声智科技有限公司 集成远场语音识别和声场录制的系统和方法
CN107331402B (zh) * 2017-06-19 2020-06-23 依偎科技（南昌）有限公司 一种基于双麦克风的录音方法及录音设备
CN107566874A (zh) * 2017-09-22 2018-01-09 百度在线网络技术（北京）有限公司 基于电视设备的远场语音控制系统

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015011217A1 (fr) * 2013-07-24 2015-01-29 Telefonica Digital España, S.L.U. Interface utilisateur utilisant des étiquettes rfid ou la voix comme entrée
CN103611294A (zh) * 2013-11-27 2014-03-05 南京航空航天大学 一种棋牌类游戏语音控制装置及其控制方法
CN104409076A (zh) * 2014-12-02 2015-03-11 上海语知义信息技术有限公司 用于棋牌类游戏的语音操控系统及语音操控方法
CN105390135A (zh) * 2015-10-12 2016-03-09 四川长虹电器股份有限公司 游戏语音控制系统
CN105978783A (zh) * 2016-03-21 2016-09-28 上海季诺金融信息服务有限公司 一种基于即时通讯软件的语音控制方法
CN109036411A (zh) * 2018-09-05 2018-12-18 深圳市友杰智新科技有限公司 一种智能终端语音交互控制方法及装置
CN109243444A (zh) * 2018-09-30 2019-01-18 百度在线网络技术（北京）有限公司 语音交互方法、设备及计算机可读存储介质

Also Published As

Publication number Publication date
CN112071312A (zh) 2020-12-11
CN112071312B (zh) 2024-03-29

Similar Documents

Publication Publication Date Title
WO2020248668A1 (fr) Dispositif d'affichage et procédé de traitement d'image
CN112073797B (zh) 一种音量调节方法及显示设备
WO2021031623A1 (fr) Appareil d'affichage, procédé de partage de fichiers et serveur
WO2021031629A1 (fr) Appareil d'affichage et procédé d'application d'un bouton multifonction pour dispositif de commande
WO2021189358A1 (fr) Dispositif d'affichage et procédé de réglage de volume
CN110708581B (zh) 显示设备及呈现多媒体屏保信息的方法
WO2020248680A1 (fr) Procédé et appareil de traitement de données vidéo et dispositif d'affichage
WO2021031598A1 (fr) Procédé d'ajustement auto-adaptatif pour la position d'une fenêtre de dialogue en ligne vidéo, et dispositif d'affichage
WO2020248697A1 (fr) Dispositif d'affichage et procédé de traitement des données de communication vidéo
WO2021031589A1 (fr) Dispositif d'affichage et procédé de réglage d'espace de gamme dynamique de couleurs
WO2020248699A1 (fr) Procédé de traitement du son et appareil d'affichage
CN111385631B (zh) 一种显示设备、通信方法及存储介质
WO2020248681A1 (fr) Dispositif d'affichage et procédé d'affichage des états de commutation bluetooth
CN112463267B (zh) 在显示设备屏幕上呈现屏保信息的方法及显示设备
WO2021031620A1 (fr) Dispositif d'affichage et procédé de réglage de luminosité de rétroéclairage
WO2020248790A1 (fr) Procédé de commande vocale et dispositif d'affichage
WO2021169125A1 (fr) Dispositif d'affichage et procédé de commande
WO2020248654A1 (fr) Appareil d'affichage et procéder pour afficher des applications de façon conjointe
CN112073666B (zh) 一种显示设备的电源控制方法及显示设备
CN112073777B (zh) 一种语音交互方法及显示设备
CN112073812B (zh) 一种智能电视上的应用管理方法及显示设备
CN112073759B (zh) 双系统之间通信方式的选取及调度方法、装置及显示设备
WO2020248788A1 (fr) Procédé de commande vocale et dispositif d'affichage
CN112073773A (zh) 一种屏幕互动方法、装置及显示设备
CN112073808A (zh) 一种色彩空间切换方法及显示装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20822561

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20822561

Country of ref document: EP

Kind code of ref document: A1