WO2022095820A1

WO2022095820A1 - Text input method, electronic device, and system

Info

Publication number: WO2022095820A1
Application number: PCT/CN2021/127888
Authority: WO
Inventors: 胡凯; 卞苏成; 周星辰
Original assignee: 华为技术有限公司
Priority date: 2020-11-09
Filing date: 2021-11-01
Publication date: 2022-05-12
Also published as: CN114489876A

Abstract

Provided are a text input method, electronic device, and system, said method comprising: a first electronic device displaying a text input interface by means of a display screen, the text input interface comprising a text input box; in response to the display of the text input interface, the first electronic device sending a first message, the first message being used for indicating that text input is required for the first electronic device; a second electronic device detecting a preset operation of a user and monitoring for said first message; in response to detecting a preset operation of the user and receiving the first message, the second electronic device detecting content inputted by the user; in response to an operation of the user inputting the first content, the second electronic device sending the first content to the first electronic device; the first electronic device displaying in the text input box text content corresponding to the first content. The invention helps to increase user convenience when inputting text on a device, and also reduces disruption to the user.

Description

A method, electronic device and system for text input

This application claims the priority of the Chinese patent application with the application number of 202011240756.5 and the application title of "A Text Input Method, Electronic Device and System" filed with the China Patent Office on November 9, 2020, the entire contents of which are incorporated by reference in this application.

technical field

The present application relates to the field of terminals, and more particularly, to a method, electronic device and system for text input.

Background technique

At present, the input operation of smart large-screen devices (for example, smart TVs) is mainly controlled by the handheld remote control equipped with the smart large screen. However, due to the limitation of the size of the handheld remote control, the remote control only has channel switching, volume adjustment, As well as a few physical buttons such as up, down, left, and right direction controls, it is extremely inconvenient to input text content such as text, account numbers, and passwords.

With the development of smart devices, there are more and more devices that require input, such as setting the shelf life of food or even setting an automatic purchase plan when using a smart refrigerator to store fresh food. Using its own dedicated remote for input to each smart device is cumbersome and tedious.

SUMMARY OF THE INVENTION

The present application provides a method, electronic device and system for text input, which help to improve the convenience of a user when entering text on the device, and can also reduce interference to the user.

In a first aspect, a system is provided, the system includes a first electronic device and a second electronic device, the first electronic device is configured to display a text input interface through a display screen, the text input interface includes a text input box; the first electronic device an electronic device, which is further configured to send a first message in response to displaying the text input interface, where the first message is used to indicate that the first electronic device needs to perform text input; the second electronic device is configured to detect the user's preset set to operate and monitor the first message; the second electronic device is also used to detect the content input by the user in response to detecting the user's preset operation and receiving the first message; the second electronic device is also used to In response to detecting an operation of the user inputting the first content, the first content is sent to the first electronic device; the first electronic device is further configured to display text content corresponding to the first content in the text input box.

In the embodiment of the present application, when the first electronic device needs to input text, the user can pick up any device around him (for example, a mobile phone, a Pad) for input, which helps to improve the convenience of the user when inputting text. to enhance the user experience. At the same time, before the user performs a preset operation on the second electronic device and receives the first message, the second electronic device will not display any prompt information that may interfere with the user, thereby avoiding interference to the user, thereby helping to improve user experience.

In some possible implementations, the first electronic device is specifically configured to: in response to displaying the text input interface, send a plurality of first messages within a preset time period.

In some possible implementations, the second electronic device is specifically configured to: start monitoring the first message in response to detecting the user's preset operation; and detect the content input by the user in response to receiving the first message.

In this embodiment of the present application, the second electronic device starts to monitor the first message after detecting the user's preset operation, so as to avoid interference to the user caused by the device not detecting the preset operation prompting the user for input.

In some possible implementations, the second electronic device is specifically configured to: in response to receiving the first message, detect a user's preset operation; in response to detecting the user's preset operation, detect content input by the user.

In this embodiment of the present application, the second electronic device may be in a state of monitoring the first message all the time, and when the second electronic device receives the first message, it starts to detect the user's preset operation, which can prevent other devices from receiving the first message without The electronic device that detects the preset operation prompts the user for input and causes interference to the user.

In some possible implementations, the second electronic device is specifically configured to: in response to receiving the first message, detect a user's preset operation; in response to detecting the user's preset operation and receiving the first message and When it is detected that the time interval between the preset operations of the user is less than the preset time interval, the content input by the user is detected.

In this embodiment of the present application, other electronic devices may not detect the user's operation for a period of time after receiving the first message, and the user may not use other electronic devices for input. Then, when the other electronic devices detect the preset operation of the user, the other electronic devices may ignore the first message, or the other electronic devices may not prompt the user for input.

In some possible implementations, the second electronic device is specifically configured to: in response to detecting a preset operation of the user and receiving the first message, and the first electronic device is at a preset angle of the second electronic device Within the range (for example, the device facing the second electronic device is the first electronic device), the content input by the user is detected.

With reference to the first aspect, in some implementations of the first aspect, the second electronic device is specifically configured to: in response to detecting the user's preset operation and receiving the first message, display an input method; The text content entered by the input method.

In this embodiment of the present application, when the second electronic device detects the user's preset operation and receives the first message, it can display the input method, which helps to improve the convenience of the user when entering text, thereby improving the user's experience. At the same time, before the user performs a preset operation on the second electronic device and receives the first message, the second electronic device will not display any prompt information that may interfere with the user, thereby avoiding interference to the user, thereby helping to improve user experience.

With reference to the first aspect, in some implementations of the first aspect, the second electronic device is specifically configured to: in response to detecting the user's preset operation and receiving the first message, detect the voice content input by the user; After detecting the operation of the user inputting the first voice content, send the first voice content to the first electronic device; wherein, the first electronic device is specifically used to: determine the text content corresponding to the first voice content; The text content is displayed in the text input box.

In the embodiment of the present application, when the second electronic device detects the user's preset operation and receives the first message, it can monitor the voice content input by the user, which helps to improve the convenience of the user when entering text. Thereby improving the user experience. At the same time, before the user performs a preset operation on the second electronic device and receives the first message, the second electronic device will not display any prompt information that may interfere with the user, thereby avoiding interference to the user, thereby helping to improve user experience.

In some possible implementations, the second electronic device is specifically configured to: in response to detecting the user's preset operation and receiving the first message, prompt the user to select text input or voice input; when it is detected that the user selects text When an input operation is performed, the input method is displayed; when an operation of the user selecting voice input is detected, the voice content input by the user is detected.

With reference to the first aspect, in some implementations of the first aspect, the second electronic device is further configured to: before detecting the content input by the user, display first prompt information, where the first prompt information is used to prompt the first prompt information The second electronic device is a device capable of inputting to the first electronic device.

In this embodiment of the present application, when the second electronic device detects the preset operation of the user and receives the first message, the second electronic device can prompt the user to input text through a reminder box, which helps the user to clarify that the second electronic device can be used as an input device

With reference to the first aspect, in some implementations of the first aspect, the first electronic device is further configured to: display second prompt information through the display screen before displaying the text content in the text input box, the second The prompt information is used to prompt the user to input to the first electronic device through the second electronic device.

In this embodiment of the present application, before receiving the user's input from the second electronic device, the first electronic device may prompt the user to input through the second electronic device through the display screen, which helps the user to clarify that the second electronic device can be used as the input device .

With reference to the first aspect, in some implementations of the first aspect, the second electronic device is specifically configured to: start monitoring the first message when an operation of starting the first application program by the user is detected.

With reference to the first aspect, in some implementations of the first aspect, the first application is a remote control application.

With reference to the first aspect, in some implementations of the first aspect, the first electronic device is a smart TV.

A second aspect provides a text input method, which is applied to an electronic device, wherein the method includes: the electronic device detects a user's preset operation and monitors a first message, where the first message is used to indicate Another electronic device needs to perform text input; in response to detecting the preset operation of the user and receiving the first message, the electronic device detects the content input by the user; in response to detecting the operation of the user inputting the first content, the electronic device The device sends the first content to the other electronic device.

In some possible implementations, in response to detecting the preset operation of the user and receiving the first message, the electronic device detects the content input by the user, including: in response to detecting the preset operation of the user, starting to monitor the first message; in response to receiving the first message, detecting the content input by the user.

In some possible implementations, in response to detecting the preset operation of the user and receiving the first message, the electronic device detecting the content input by the user includes: in response to receiving the first message, detecting the user's Preset operation; in response to detecting the user's preset operation, the content input by the user is detected.

In some possible implementations, in response to detecting the preset operation of the user and receiving the first message, the electronic device detecting the content input by the user includes: in response to receiving the first message, detecting the user's Preset operation; in response to detecting the user's preset operation and the time interval between receiving the first message and detecting the user's preset operation is less than the preset time interval, detecting the content input by the user.

In some possible implementations, in response to detecting the preset operation of the user and receiving the first message, the electronic device detects the content input by the user, including: in response to detecting the preset operation of the user and receiving the first message The first message, and the first electronic device is within a preset angle range of the second electronic device (for example, the device facing the second electronic device is the first electronic device) to detect the content input by the user.

With reference to the second aspect, in some implementations of the second aspect, in response to detecting the user's and preset operation and receiving the first message, the electronic device detects the user's input, including: in response to detecting the user's input the preset operation of the device and receiving the first message, the electronic device displays the input method; the electronic device detects the text content input by the user through the input method.

With reference to the second aspect, in some implementations of the second aspect, in response to detecting the user's and preset operation and receiving the first message, the electronic device detects the user's input, including: in response to detecting the user's input and the preset operation and receiving the first message, the electronic device detects the voice content input by the user.

In some possible implementations, in response to detecting the user's preset operation and receiving the first message, the electronic device detects the user's input, including: in response to detecting the user's preset operation and receiving the user's input The first message prompts the user to select text input or voice input; when an operation of selecting text input by the user is detected, the input method is displayed; when an operation of selecting voice input by the user is detected, the voice content input by the user is detected.

With reference to the second aspect, in some implementations of the second aspect, before the electronic device detects the content input by the user, the method further includes: the electronic device displays prompt information, where the prompt information is used to prompt the user that the electronic device is capable of A device for input to the other electronic device.

With reference to the second aspect, in some implementations of the second aspect, the electronic device detecting a preset operation of the user includes: the electronic device detecting an operation of the user starting the first application program.

With reference to the second aspect, in some implementations of the second aspect, the first application is a remote control application.

In conjunction with the second aspect, in some implementations of the second aspect, the other electronic device is a smart TV.

In a third aspect, a text input device is provided, the device comprising: a first detection unit for detecting a user's preset operation; a receiving unit for monitoring a first message, where the first message is used to indicate another The electronic device needs to perform text input; the second detection unit is used to detect the content input by the user in response to the first detection unit detecting the preset operation of the user and the receiving unit receiving the first message; the sending unit is used to respond After the second detection unit detects an operation of the user inputting the first content, the first content is sent to the other electronic device.

In a fourth aspect, an electronic device is provided, comprising: one or more processors; a memory; and one or more computer programs. Wherein, one or more computer programs are stored in the memory, the one or more computer programs comprising instructions. When the instruction is executed by the electronic device, the electronic device is caused to perform the method of text input in any possible implementation of the above-mentioned second aspect.

In a fifth aspect, there is provided a computer program product comprising instructions, when the computer program product is run on a first electronic device, the electronic device is made to perform the text input method of the second aspect above.

In a sixth aspect, a computer-readable storage medium is provided, comprising instructions that, when the instructions are executed on an electronic device, cause the electronic device to perform the text input method described in the second aspect above.

In a seventh aspect, a chip is provided for executing an instruction, and when the chip runs, the chip executes the text input method described in the second aspect above.

Description of drawings

FIG. 1 is a schematic diagram of a hardware structure of an electronic device provided by an embodiment of the present application.

FIG. 2 is a block diagram of a software structure provided by an embodiment of the present application.

FIG. 3 is a set of graphical user interfaces provided by an embodiment of the present application.

FIG. 4 is another set of graphical user interfaces provided by an embodiment of the present application.

FIG. 5 is another set of graphical user interfaces provided by the embodiments of the present application.

FIG. 6 is another set of graphical user interfaces provided by an embodiment of the present application.

FIG. 7 is another set of graphical user interfaces provided by the embodiments of the present application.

FIG. 8 is a system architecture diagram provided in an embodiment of the present application.

FIG. 9 is a schematic flowchart of a method for text input provided by an embodiment of the present application.

FIG. 10 is a schematic block diagram of an apparatus for text input provided by an embodiment of the present application.

FIG. 11 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. Wherein, in the description of the embodiments of the present application, unless otherwise stated, “/” means or means, for example, A/B can mean A or B; “and/or” in this document is only a description of the associated object The association relationship of , indicates that there can be three kinds of relationships, for example, A and/or B, can indicate that A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "plurality" or "plurality" refers to two or more than two.

Hereinafter, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may expressly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

The methods provided in the embodiments of the present application can be applied to mobile phones, tablet computers, wearable devices, in-vehicle devices, augmented reality (AR)/virtual reality (VR) devices, notebook computers, ultra-mobile personal computers (ultra mobile personal computers) -mobile personal computer, UMPC), netbook, personal digital assistant (personal digital assistant, PDA) and other electronic devices, the embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

Exemplarily, FIG. 1 shows a schematic structural diagram of an electronic device 100 . The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2 , mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, buttons 190, motor 191, indicator 192, camera 193, display screen 194, and Subscriber identification module (subscriber identification module, SIM) card interface 195 and so on. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light. Sensor 180L, bone conduction sensor 180M, etc.

It can be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100 . In other embodiments of the present application, the electronic device 100 may include more or less components than shown, or combine some components, or separate some components, or arrange different components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU) Wait. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.

The controller may be the nerve center and command center of the electronic device 100 . The controller can generate operation control signals according to the instruction opcode and timing signal, and complete the control of fetching and executing instructions.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is cache memory. This memory may hold instructions or data that have just been used or recycled by the processor 110 . If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby increasing the efficiency of the system.

In some embodiments, the processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transceiver (universal asynchronous transmitter) receiver/transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (SIM) interface, and / or universal serial bus (universal serial bus, USB) interface, etc.

The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may contain multiple sets of I2C buses. The processor 110 can be respectively coupled to the touch sensor 180K, the charger, the flash, the camera 193 and the like through different I2C bus interfaces. For example, the processor 110 may couple the touch sensor 180K through the I2C interface, so that the processor 110 and the touch sensor 180K communicate with each other through the I2C bus interface, so as to realize the touch function of the electronic device 100 .

The I2S interface can be used for audio communication. In some embodiments, the processor 110 may contain multiple sets of I2S buses. The processor 110 may be coupled with the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170 . In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface, so as to realize the function of answering calls through a Bluetooth headset.

The PCM interface can also be used for audio communications, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to realize the function of answering calls through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

The UART interface is a universal serial data bus used for asynchronous communication. The bus may be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, a UART interface is typically used to connect the processor 110 with the wireless communication module 160 . For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface, so as to realize the function of playing music through the Bluetooth headset.

The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 . MIPI interfaces include camera serial interface (CSI), display serial interface (DSI), etc. In some embodiments, the processor 110 communicates with the camera 193 through a CSI interface, so as to realize the photographing function of the electronic device 100 . The processor 110 communicates with the display screen 194 through the DSI interface to implement the display function of the electronic device 100 .

The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface may be used to connect the processor 110 with the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, and the like. The GPIO interface can also be configured as I2C interface, I2S interface, UART interface, MIPI interface, etc.

The USB interface 130 is an interface that conforms to the USB standard specification, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices.

It can be understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is only a schematic illustration, and does not constitute a structural limitation of the electronic device 100 . In other embodiments of the present application, the electronic device 100 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.

The charging management module 140 is used to receive charging input from the charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from the wired charger through the USB interface 130 . In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100 . While the charging management module 140 charges the battery 142 , it can also supply power to the electronic device through the power management module 141 .

The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 . The power management module 141 receives input from the battery 142 and/or the charging management module 140 and supplies power to the processor 110 , the internal memory 121 , the external memory, the display screen 194 , the camera 193 , and the wireless communication module 160 . The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, battery health status (leakage, impedance). In some other embodiments, the power management module 141 may also be provided in the processor 110 . In other embodiments, the power management module 141 and the charging management module 140 may also be provided in the same device.

The wireless communication function of the electronic device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, the baseband processor, and the like.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, the antenna 1 can be multiplexed as a diversity antenna of the wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 may provide wireless communication solutions including 2G/3G/4G/5G etc. applied on the electronic device 100 . The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA) and the like. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and then turn it into an electromagnetic wave for radiation through the antenna 1 . In some embodiments, at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 may be provided in the same device as at least part of the modules of the processor 110 .

The modem processor may include a modulator and a demodulator. Wherein, the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and passed to the application processor. The application processor outputs sound signals through audio devices (not limited to the speaker 170A, the receiver 170B, etc.), or displays images or videos through the display screen 194 . In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be independent of the processor 110, and may be provided in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellites Wireless communication solutions such as global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared technology (IR). The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110 , perform frequency modulation on it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2 .

In some embodiments, the antenna 1 of the electronic device 100 is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC , FM, and/or IR technology, etc. The GNSS may include a global positioning system (global positioning system, GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (quasi -zenith satellite system, QZSS) and/or satellite based augmentation systems (SBAS).

The electronic device 100 implements a display function through a GPU, a display screen 194, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.

Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light). emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diode (quantum dot light emitting diodes, QLED) and so on. In some embodiments, the electronic device 100 may include one or N display screens 194 , where N is a positive integer greater than one.

The electronic device 100 may implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.

The ISP is used to process the data fed back by the camera 193 . For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin tone. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be provided in the camera 193 .

Camera 193 is used to capture still images or video. The object is projected through the lens to generate an optical image onto the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other formats of image signals. In some embodiments, the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.

A digital signal processor is used to process digital signals, in addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy and so on.

Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos of various encoding formats, such as: Moving Picture Experts Group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4 and so on.

The NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process the input information, and can continuously learn by itself. Applications such as intelligent cognition of the electronic device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.

The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100 . The external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example to save files like music, video etc in external memory card.

Internal memory 121 may be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by executing the instructions stored in the internal memory 121 . The internal memory 121 may include a storage program area and a storage data area. The storage program area may store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), and the like. The storage data area may store data (such as audio data, phone book, etc.) created during the use of the electronic device 100 and the like. In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (UFS), and the like.

The electronic device 100 may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, an application processor, and the like. Such as music playback, recording, etc.

The audio module 170 is used for converting digital audio information into analog audio signal output, and also for converting analog audio input into digital audio signal. Audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be provided in the processor 110 , or some functional modules of the audio module 170 may be provided in the processor 110 .

Speaker 170A, also referred to as a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music through the speaker 170A, or listen to a hands-free call.

The receiver 170B, also referred to as "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a call or a voice message, the voice can be answered by placing the receiver 170B close to the human ear.

The microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can make a sound by approaching the microphone 170C through a human mouth, and input the sound signal into the microphone 170C. The electronic device 100 may be provided with at least one microphone 170C. In other embodiments, the electronic device 100 may be provided with two microphones 170C, which can implement a noise reduction function in addition to collecting sound signals. In other embodiments, the electronic device 100 may further be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and implement directional recording functions.

The earphone jack 170D is used to connect wired earphones. The earphone interface 170D may be the USB interface 130, or may be a 3.5mm open mobile terminal platform (OMTP) standard interface, a cellular telecommunications industry association of the USA (CTIA) standard interface.

The pressure sensor 180A is used to sense pressure signals, and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A may be provided on the display screen 194 . There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, and the like. The capacitive pressure sensor may be comprised of at least two parallel plates of conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example, when a touch operation whose intensity is less than the first pressure threshold acts on the short message application icon, the instruction for viewing the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, the instruction to create a new short message is executed.

The gyro sensor 180B may be used to determine the motion attitude of the electronic device 100 . In some embodiments, the angular velocity of electronic device 100 about three axes (ie, x, y, and z axes) may be determined by gyro sensor 180B. The gyro sensor 180B can be used for image stabilization. Exemplarily, when the shutter is pressed, the gyro sensor 180B detects the angle at which the electronic device 100 shakes, calculates the distance that the lens module needs to compensate for according to the angle, and allows the lens to counteract the shake of the electronic device 100 through reverse motion to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenarios.

The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.

The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can detect the opening and closing of the flip holster using the magnetic sensor 180D. In some embodiments, when the electronic device 100 is a flip machine, the electronic device 100 can detect the opening and closing of the flip according to the magnetic sensor 180D. Further, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, characteristics such as automatic unlocking of the flip cover are set.

The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes). The magnitude and direction of gravity can be detected when the electronic device 100 is stationary. It can also be used to identify the posture of electronic devices, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc.

Distance sensor 180F for measuring distance. The electronic device 100 can measure the distance through infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

Proximity light sensor 180G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes. The light emitting diodes may be infrared light emitting diodes. The electronic device 100 emits infrared light to the outside through the light emitting diode. Electronic device 100 uses photodiodes to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100 . When insufficient reflected light is detected, the electronic device 100 may determine that there is no object near the electronic device 100 . The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in holster mode, pocket mode automatically unlocks and locks the screen.

The ambient light sensor 180L is used to sense ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touch.

The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to realize fingerprint unlocking, accessing application locks, taking pictures with fingerprints, answering incoming calls with fingerprints, and the like.

The temperature sensor 180J is used to detect the temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold value, the electronic device 100 reduces the performance of the processor located near the temperature sensor 180J in order to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 caused by the low temperature. In some other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

Touch sensor 180K, also called "touch panel". The touch sensor 180K may be disposed on the display screen 194 , and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”. The touch sensor 180K is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to touch operations may be provided through display screen 194 . In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100 , which is different from the location where the display screen 194 is located.

The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire the vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M can also contact the pulse of the human body and receive the blood pressure beating signal. In some embodiments, the bone conduction sensor 180M can also be disposed in the earphone, combined with the bone conduction earphone. The audio module 170 can analyze the voice signal based on the vibration signal of the vocal vibration bone block obtained by the bone conduction sensor 180M, so as to realize the voice function. The application processor can analyze the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M, and realize the function of heart rate detection.

The keys 190 include a power-on key, a volume key, and the like. Keys 190 may be mechanical keys. It can also be a touch key. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100 .

Motor 191 can generate vibrating cues. The motor 191 can be used for vibrating alerts for incoming calls, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, playing audio, etc.) can correspond to different vibration feedback effects. The motor 191 can also correspond to different vibration feedback effects for touch operations on different areas of the display screen 194 . Different application scenarios (for example: time reminder, receiving information, alarm clock, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

The indicator 192 can be an indicator light, which can be used to indicate the charging state, the change of the power, and can also be used to indicate a message, a missed call, a notification, and the like.

The SIM card interface 195 is used to connect a SIM card. The SIM card can be contacted and separated from the electronic device 100 by inserting into the SIM card interface 195 or pulling out from the SIM card interface 195 . The electronic device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card and so on. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the plurality of cards may be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to realize functions such as call and data communication. In some embodiments, the electronic device 100 employs an eSIM, ie: an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiments of the present application take an Android system with a layered architecture as an example to exemplarily describe the software structure of the electronic device 100 .

FIG. 2 is a block diagram of the software structure of the electronic device 100 according to the embodiment of the present application. The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, which are, from top to bottom, an application layer, an application framework layer, an Android runtime (Android runtime) and a system library, and a kernel layer. The application layer can include a series of application packages.

As shown in Figure 2, the application package can include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and so on.

The application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

As shown in Figure 2, the application framework layer may include window managers, content providers, view systems, telephony managers, resource managers, notification managers, and the like.

A window manager is used to manage window programs. The window manager can get the size of the display screen, determine whether there is a status bar, lock the screen, take screenshots, etc.

Content providers are used to store and retrieve data and make these data accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone book, etc.

The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on. View systems can be used to build applications. A display interface can consist of one or more views. For example, the display interface including the short message notification icon may include a view for displaying text and a view for displaying pictures.

The phone manager is used to provide the communication function of the electronic device 100 . For example, the management of call status (including connecting, hanging up, etc.).

The resource manager provides various resources for the application, such as localization strings, icons, pictures, layout files, video files and so on.

The notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages, and can disappear automatically after a brief pause without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also display notifications in the status bar at the top of the system in the form of graphs or scroll bar text, such as notifications of applications running in the background, and notifications on the screen in the form of dialog windows. For example, text information is prompted in the status bar, a prompt sound is issued, the electronic device vibrates, and the indicator light flashes.

Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.

The core library consists of two parts: one is the function functions that the java language needs to call, and the other is the core library of Android.

The application layer and the application framework layer run in virtual machines. The virtual machine executes the java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, safety and exception management, and garbage collection.

A system library can include multiple functional modules. For example: surface manager (surface manager), media library (media library), 3D graphics processing library (eg: OpenGL ES), 2D graphics engine (eg: SGL), etc.

The Surface Manager is used to manage the display subsystem and provides a fusion of 2D and 3D layers for multiple applications.

The media library supports playback and recording of a variety of commonly used audio and video formats, as well as still image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is the layer between hardware and software. The kernel layer contains at least display drivers, camera drivers, audio drivers, and sensor drivers.

It should be understood that the embodiments of the present application may be applicable to systems such as Android, IOS, or Hongmeng.

FIG. 3 is a set of graphical user interfaces (graphical user interface, GUI) provided by the embodiments of the present application.

Referring to (a) in FIG. 3 , the user is using the TV remote control to search for movies, the movie search display interface is displayed on the smart TV, and the cursor of the smart TV is located in the text input box 301 at this time. When the smart TV detects that the user moves the cursor to the text input box 301, a broadcast message can be sent to surrounding devices, and the broadcast message can be used to indicate that the smart TV needs to perform text input.

In one embodiment, the smart TV can also send the broadcast message to surrounding devices when it detects that the user moves the cursor to a certain key (eg, "ABC" key) in the input method displayed by the smart TV.

It should be understood that the movie search display interface displayed by the smart TV as shown in (a) of FIG. 3 can also be referred to as a text input interface. The text input interface in this embodiment of the present application may include a text input box, or the text input interface may include a text input box and an input method.

In one embodiment, the broadcast message may further carry a communication address of the smart TV (for example, an internet protocol (Internet protocol, IP) address, a port number, or a Bluetooth address, etc.).

Referring to (b) in FIG. 3 , a reminder box 302 appears on the mobile phone, and the user may choose not to use the TV remote control for text input, but to select the mobile phone for text input. The prompt box 302 may prompt the user "it is detected that the smart TV needs to perform text input, whether to open the remote control application for input". After the mobile phone detects that the user clicks on the control 303, the GUI shown in (c) of FIG. 3 can be displayed.

In one embodiment, when the mobile phone detects that the user clicks on the control 303, the mobile phone can establish a connection with the smart TV through the communication address carried in the broadcast message.

In one embodiment, after the mobile phone and the smart TV are connected, the mobile phone can also send device information of the mobile phone to the smart TV (for example, the device name "P40" of the mobile phone and the user name "Tom" of the mobile phone). After receiving the device information sent by the mobile phone, the smart TV can display prompt information on the display screen of the smart TV, for example, the prompt information is "Please enter text on Tom's P40".

Referring to the GUI shown in (c) of FIG. 3 , the GUI is the display interface of the remote control application on the mobile phone. The display interface includes a number of functional controls, such as on/off, mute, input, menu, extended keys, change channel, volume down or up, and back. After the mobile phone detects that the user clicks on the input control 304, the mobile phone can display the GUI as shown in (d) of FIG. 3 .

In one embodiment, if the mobile phone is in the lock screen interface when receiving the broadcast message, when the mobile phone detects that the user clicks on the control 303, the mobile phone can turn on the camera to collect the user's face information. If the face information collected by the camera matches the face information preset in the mobile phone, the mobile phone can be unlocked first to enter the non-lock screen interface, and the remote control application will be automatically opened in the non-lock screen interface. Alternatively, when the mobile phone detects that the user clicks on the control 303, the mobile phone can collect the user's fingerprint information. If the collected fingerprint information matches the preset fingerprint information in the mobile phone, the mobile phone can be unlocked first, thereby entering the non-lock screen interface, and automatically opening the remote control application in the non-lock screen interface.

Referring to the GUI shown in (d) of FIG. 3 , the GUI is another display interface of the remote control application on the mobile phone. When the mobile phone detects that the user clicks on the control 304, the mobile phone can call up the input method of the mobile phone. The user can input the text content through the input method called up by the mobile phone. When the mobile phone detects that the user has input the text content "movie 1" in the text input box 306 and clicks the control 305, the mobile phone can send the text content to the smart TV.

In one embodiment, when the mobile phone detects that the user clicks on the control 303, the mobile phone can also directly display the GUI shown in (d) of FIG. 3 . That is to say, after the mobile phone detects the user's operation of clicking on the control 303, the mobile phone can first perform an unlocking operation through the collected face information or fingerprint information, thereby entering the non-lock screen interface. In the non-lock screen interface, the mobile phone automatically opens the remote control application and calls up the input method of the mobile phone, thereby displaying the GUI shown in (d) of FIG. 3 .

In one embodiment, if the mobile phone is in a non-lock screen interface when receiving the broadcast message, the mobile phone can directly call up the input method of the mobile phone on the non-lock screen interface without opening the remote control application. The user can input text through the input method called up by the mobile phone. When the mobile phone detects that the user has input the text content "movie 1" in the text input box 306 through the input method and clicks the control 305, the mobile phone can send the text content to the smart TV.

In one embodiment, the mobile phone can also send the text content input by the user to the smart TV in real time. Exemplarily, when the mobile phone detects that the user inputs the text content "electricity" in the text input box 306, the text information can be sent to the smart TV, so that the text content "electricity" is displayed in the text input box 301 of the smart TV. . When the mobile phone detects that the user then enters the text content "Shadow" in the text input box 306, the mobile phone can continue to send the text content "Shadow" to the smart TV, so that the text content "movie" is displayed in the text input box 301 of the smart TV . When the mobile phone detects that the user then enters the text content "1" in the text input box 306, the mobile phone can continue to send the text content "1" to the smart TV, so that the text content "movie 1" is displayed in the text input box 301 of the smart TV ".

It should be understood that if the mobile phone detects that the user has deleted the text content in the text input box 306, the mobile phone can indicate the text content deleted by the user to the smart TV in real time, so that the text content in the text input box 301 of the smart TV is the same as The text content in the text input box 306 of the mobile phone is synchronized.

Referring to (e) in FIG. 3 , after receiving the text content sent by the mobile phone, the smart TV can display the text content (eg, "movie 1") in the text input box 301 of the smart TV. In addition, the smart TV can display information corresponding to the movie 1 (for example, information such as genre, director, starring, etc.).

In the embodiment of the present application, after receiving the broadcast message sent by the smart TV, the mobile phone can provide text input for the smart TV, which helps to improve the convenience of the user when entering text. At the same time, the mobile phone and the smart TV do not need to be devices under the same account. As long as the mobile phone is near the smart TV, it can provide text content input for the smart TV, which helps to improve the user experience.

FIG. 4 is another set of GUIs provided by the embodiments of the present application.

Referring to the GUI shown in (a) of FIG. 4 , when the mobile phone receives the broadcast message sent by the smart TV, the mobile phone can display a text input icon 401 on the lock screen interface. When the mobile phone detects the user's operation of clicking on the icon 401, the GUI as shown in (b) of FIG. 4 can be displayed.

In one embodiment, when the mobile phone detects that the user clicks on the icon 401, the mobile phone can establish a connection with the smart TV according to the communication address of the smart TV carried in the broadcast message.

Referring to the GUI shown in (b) of FIG. 4 , the GUI is a display interface after the mobile phone detects that the user has clicked the icon 401 . The mobile phone can display the input method on the lock screen interface, and the user can input text content in the text input box 403 . After the mobile phone detects that the user has entered text content (eg, "movie 1") in the text input box 403 and clicked the operation of the control 402, the mobile phone can send the text content to the smart TV. The smart TV can display a GUI as shown in (e) of FIG. 3 .

In one embodiment, the mobile phone can send the content in the text input box 403 to the smart TV in real time, so that the text content in the text input box 301 of the smart TV is synchronized with the content in the text input box 403 of the mobile phone.

In the embodiment of the present application, after the mobile phone receives the broadcast message sent by the smart TV, an icon can be used to prompt the user that the mobile phone can assist the smart TV to input text content, so that the input of the text content can be completed in the locked screen state. There is no need for the phone to enter the unlocked state, and there is no need for the phone to open the remote control application. The mobile phone can provide text input for the smart TV under the lock screen interface, which helps to improve the convenience of the user when entering text on the large-screen device, thereby improving the user experience. At the same time, the mobile phone and the smart TV do not need to be devices under the same account. As long as the mobile phone is near the smart TV, it can provide text content input for the smart TV, which helps to improve the user experience.

Referring to the GUI shown in (c) of FIG. 4 , the GUI is the display interface after the mobile phone detects that the user has clicked the icon 401 . When the mobile phone detects that the user clicks the icon 401, the mobile phone can turn on the camera to collect the user's face information. If the face information collected by the camera matches the preset face information in the mobile phone, the mobile phone can be unlocked first, so as to enter the non-lock screen interface, and automatically call up the input method of the mobile phone in the non-lock screen interface. Alternatively, when the mobile phone detects that the user clicks on the icon 401, the mobile phone can collect the fingerprint information of the user. If the collected fingerprint information matches the preset fingerprint information in the mobile phone, the mobile phone can be unlocked first, thereby entering the non-lock screen interface, and the input method of the mobile phone is automatically called up in the non-lock screen interface.

In the embodiment of the present application, after the mobile phone receives the broadcast message sent by the smart TV, an icon can be used to prompt the user that the mobile phone can assist the smart TV to input text content, and the mobile phone can automatically call up the input method after entering the non-lock screen interface. , you don't need your phone to open the remote control app. The mobile phone can provide text input for the smart TV without starting the application on the non-lock screen interface, which helps to improve the convenience of the user when entering text on the large-screen device, thereby improving the user experience. FIG. 5 is another set of GUIs provided by the embodiments of the present application.

Referring to (a) in FIG. 5 , the user is using the TV remote control to search for movies, a movie search display interface is displayed on the smart TV, and the cursor of the smart TV is located in the text input box 501 at this time. When the smart TV detects that the user moves the cursor to the input box 501, a broadcast message may be sent to surrounding devices, and the broadcast message may be used to indicate that the smart TV needs to perform text input.

In one embodiment, the broadcast message may also carry the communication address (eg, IP address, port number, or Bluetooth address, etc.) of the smart TV.

Referring to (b) in FIG. 5 , the user can pick up the mobile phone, at this time the mobile phone can be in an unlocked state and the mobile phone displays the mobile phone desktop.

Referring to the GUI shown in (c) of FIG. 5 , the GUI is the desktop of the mobile phone. When the mobile phone detects that the user clicks the icon 502 of the remote control application on the desktop, the mobile phone can open the remote control application. When the mobile phone detects that the user has opened the remote control application, it can start listening to the broadcast messages sent by the surrounding devices. After the mobile phone receives the broadcast message sent by the smart TV, the mobile phone can display the GUI shown in (d) of FIG. 5 .

Referring to the GUI shown in (d) of FIG. 5 , the GUI is the display interface of the remote control application of the mobile phone. When the mobile phone receives the broadcast message sent by the smart TV, the mobile phone can display a reminder box 503, wherein the reminder box 503 includes the prompt information "it is detected that the smart TV needs to perform text input, whether to use the input function for input". After the mobile phone detects that the user clicks on the control 504, the mobile phone can display the GUI shown in (e) of FIG. 5 .

In one embodiment, when the mobile phone detects that the user clicks on the control 504, the mobile phone can establish a connection with the smart TV according to the communication address of the smart TV carried in the broadcast message.

Referring to the GUI shown in (e) of FIG. 5 , the GUI is another display interface of the remote control application of the mobile phone. After the mobile phone detects that the user clicks on the control 504, the mobile phone can call up the input method of the mobile phone. When the mobile phone detects that the user has input the text content "movie 1" in the text input box 506 and clicks the control 505, the mobile phone can send the text content to the smart TV.

Referring to the GUI shown in (f) of FIG. 5 , the GUI is another display interface of the remote control application of the mobile phone. After the mobile phone detects that the user clicks on the control 504, the mobile phone can detect the voice content input by the user. As shown in (f) of FIG. 5, the user utters the voice content "movie 1". After the mobile phone detects the voice content input by the user, it can determine the text content "movie 1" corresponding to the voice content, so that the mobile phone sends the text content to the smart TV.

In the embodiment of the present application, the mobile phone may include a speech recognition (automatic speech recognition, ASR) module, and the main function of the ASR module is to recognize the user's speech content as text content.

In one embodiment, the mobile phone can also send the voice content to the smart TV after detecting the voice content input by the user. The smart TV can convert the voice content into text content, so as to be displayed in the text input box 501 .

In one embodiment, after the mobile phone detects that the user clicks on the control 504, the mobile phone may further prompt the user to select text input or voice input. If the user selects text input, the mobile phone can call up an input method, so that the user can input text content through the input method; or, if the user selects voice input, the mobile phone can start to detect the voice content input by the user.

In one embodiment, when the mobile phone detects the operation of the user opening the remote control application, it can start to monitor the broadcast message. When the mobile phone receives the broadcast message sent by the smart TV, the mobile phone may not display the reminder box 503, and the mobile phone may directly call up the input method to detect the text content input by the user, or the mobile phone may start to monitor the voice content input by the user.

In one embodiment, the mobile phone can send the content in the text input box 506 to the smart TV in real time, so that the text content in the text input box 501 of the smart TV is synchronized with the content in the text input box 506 of the mobile phone.

Referring to (g) in FIG. 5 , after receiving the text content sent by the mobile phone, the smart TV can display the text content (for example, "movie 1") in the input box 501 of the smart TV. In addition, the smart TV can display information corresponding to the movie 1 (for example, information such as genre, director, starring, etc.).

In the embodiment of the present application, the mobile phone and the smart TV do not need to be connected or bound in advance, but a connection between the input and the input is temporarily established through dynamic matching when the smart TV needs input. When the smart TV needs to input, the user can pick up any device (eg, mobile phone, Pad) around him to input, which helps to improve the convenience of the user when inputting text, thereby improving the user experience. At the same time, the mobile phone starts monitoring after opening the remote control application, and prompts the user to input text through a reminder box after listening to the broadcast message, which helps the user to clarify that the mobile phone can be used as an input device. Before the user initiates an input operation on the mobile phone, the mobile phone will not display any prompt information that may interfere with the user, thereby avoiding interference to the user, thereby helping to improve the user experience.

FIG. 6 is another set of GUIs provided by the embodiments of the present application.

Referring to the GUI shown in (a) of FIG. 6 , the GUI is the desktop of the mobile phone. When the mobile phone detects that the user clicks the icon of the remote control application on the desktop, the mobile phone can start monitoring broadcast messages sent by surrounding devices and display the GUI shown in (b) of FIG. 6 .

Referring to the GUI shown in (b) of FIG. 6 , the GUI is the display interface of the remote control application of the mobile phone. When the mobile phone does not receive the broadcast message sent by the smart TV, the color of the buttons on the display interface is gray (gray means the controls are unavailable).

Referring to the GUI shown in (c) of FIG. 6 , the GUI is another display interface of the remote control application of the mobile phone. When the mobile phone receives the broadcast message sent by the smart TV, the color of the buttons on the display interface changes from gray to black (black indicates that the controls can be used). At this time, the user can use the input function on the remote control to input the text content. When the mobile phone detects that the user clicks on the input control 601, the mobile phone can display a GUI as shown in (d) of FIG. 6 .

In one embodiment, when the mobile phone receives the broadcast message sent by the smart TV, the mobile phone can establish a connection with the smart TV according to the communication address of the smart TV carried in the broadcast message.

Referring to the GUI shown in (d) of FIG. 6 , the GUI is another display interface of the remote control application of the mobile phone. After the mobile phone detects that the user clicks on the input control 601, the mobile phone can call up the input method of the mobile phone. When the mobile phone detects that the user has input the text content "movie 1" in the text input box 602 and clicks the control 603, the mobile phone can send the text content to the smart TV.

It should be understood that, in this embodiment of the present application, when the mobile phone receives the broadcast message sent by the smart TV, the mobile phone can directly display the GUI shown in (d) of FIG. 6 . That is to say, when the mobile phone receives the broadcast message sent by the smart TV, the color of the buttons on the display interface changes from gray to black, and the mobile phone can automatically call up the input method of the mobile phone.

Referring to the GUI shown in (e) of FIG. 6 , the GUI is another display interface of the remote control application of the mobile phone. After the mobile phone receives the broadcast message, the mobile phone can detect the voice content input by the user. As shown in (e) of FIG. 6, the user utters the voice content "movie 1". After the mobile phone detects the user's voice content, it can determine the text content "movie 1" corresponding to the voice content, so that the mobile phone sends the text content to the smart TV.

In one embodiment, when the mobile phone receives the broadcast message sent by the smart TV, the color of the buttons on the display interface of the remote control application changes from gray to black, and the mobile phone can also prompt the user to select text input or voice input. If the user selects text input, the mobile phone can call up an input method, so that the user can input text content through the input method; or, if the user selects voice input, the mobile phone can start to detect the voice content input by the user.

In one embodiment, the mobile phone can send the content in the text input box 602 to the smart TV in real time, so that the text content in the text input box of the smart TV is synchronized with the content in the text input box 602 of the mobile phone.

After receiving the text content sent by the mobile phone, the smart TV can display the text content (for example, "movie 1") in the input box of the smart TV. In addition, the smart TV can display information corresponding to the movie 1 (for example, information such as genre, director, starring, etc.).

In the embodiment of the present application, the mobile phone and the smart TV do not need to be connected or bound in advance, but a connection between the input and the input is temporarily established through dynamic matching when the smart TV needs input. When the smart TV needs to input, the user can pick up any device (eg, mobile phone, Pad) around him to input, which helps to improve the convenience of the user when inputting text, thereby improving the user experience. At the same time, the mobile phone starts monitoring after opening the remote control application, and reminds the mobile phone that the mobile phone can be used as an input device through the change of the color of the control after monitoring the broadcast message. Before the user initiates an input operation on the mobile phone, the mobile phone will not display any prompt information that may interfere with the user, thereby avoiding interference to the user, thereby helping to improve the user experience.

FIG. 7 is another set of GUIs provided by the embodiments of the present application.

Referring to the GUI shown in (a) of FIG. 7 , the GUI is the lock screen interface of the mobile phone. When the mobile phone detects the user's preset operation on the lock screen interface (for example, the mobile phone detects that the user draws "S" on the lock screen interface), the mobile phone can start monitoring broadcast messages sent by surrounding devices. When the mobile phone detects the broadcast message sent by the smart TV, the mobile phone can display the GUI as shown in (b) in Figure 7 .

In one embodiment, the triggering condition for the mobile phone to start monitoring broadcast messages sent by surrounding devices may be that the mobile phone detects that the user draws a pattern of a preset shape on the currently displayed interface; Empty gesture; alternatively, the mobile phone may detect that the user presses the physical buttons of the mobile phone (for example, the volume key and the power button); alternatively, the mobile phone may detect the user's preset gesture on the current interface and press the physical button operation.

Referring to the GUI shown in (b) of FIG. 7 , the GUI is another lock screen interface of the mobile phone. In response to receiving the broadcast message sent by the smart TV, the mobile phone may display a text input icon 701 on the lock screen interface. When the mobile phone detects the user's operation of clicking on the icon 701, the GUI as shown in (c) in FIG. 7 may be displayed, or the GUI as shown in (d) in FIG. 7 may be displayed.

Referring to the GUI shown in (c) of FIG. 7 , the GUI is the display interface after the mobile phone detects that the user has clicked the icon 701 . The mobile phone can display the input method on the lock screen interface, and the user can input text content in the text input box 703 . After the mobile phone detects that the user has input text content (eg, "movie 1") in the text input box 703 and clicks the operation of the control 702, the mobile phone can send the text content to the smart TV. The smart TV can display a GUI as shown in (e) of FIG. 3 .

Referring to the GUI shown in (d) of FIG. 7 , the GUI is another display interface after the mobile phone detects that the user has clicked the icon 701 . In response to detecting that the user clicks on the icon 701, the mobile phone can turn on the camera to collect the user's face information. If the face information collected by the camera matches the preset face information in the mobile phone, the mobile phone can be unlocked first, so as to enter the non-lock screen interface, and automatically call up the input method of the mobile phone in the non-lock screen interface. Alternatively, when the mobile phone detects that the user clicks the icon 701, the mobile phone can collect the fingerprint information of the user. If the collected fingerprint information matches the preset fingerprint information in the mobile phone, the mobile phone can be unlocked first, thereby entering the non-lock screen interface, and the input method of the mobile phone is automatically called up in the non-lock screen interface. After the mobile phone detects that the user has input text content (eg, "movie 1") in the text input box 703 and clicks the operation of the control 702, the mobile phone can send the text content to the smart TV. The smart TV can display a GUI as shown in (e) of FIG. 3 . In one embodiment, after detecting the user's preset operation, if the mobile phone receives a broadcast message sent by the smart TV, it can directly display the GUI shown in (c) or (d) of FIG. 7 . That is, in response to the mobile phone receiving the broadcast message sent by the smart TV, the mobile phone can also directly call up the input method on the lock screen interface, or the mobile phone can call up the input method after entering the non-lock screen interface.

In one embodiment, the mobile phone can also send the text content input by the user to the smart TV in real time. Exemplarily, when the mobile phone detects that the user inputs the text content "electricity" in the text input box 703, the text information can be sent to the smart TV, so that the text content "electricity" is displayed in the text input box of the smart TV. When the mobile phone detects that the user then enters the text content "Shadow" in the text input box 703, the mobile phone can continue to send the text content "Shadow" to the smart TV, thereby displaying the text content "Movie" in the text input box of the smart TV. When the mobile phone detects that the user then enters the text content "1" in the text input box 703, the mobile phone can continue to send the text content "1" to the smart TV, so that the text content "movie 1" is displayed in the text input box of the smart TV .

FIG. 8 shows a system architecture diagram provided in an embodiment of the present application. The system includes device A and device B, where device A is an input device (eg, the mobile phone in FIG. 3 to FIG. 7 ); device B is an input device (eg, the smart TV in FIG. 3 and FIG. 5 ).

Device B detects that the text input box of device B has the focus, and device B enters the input state at this time.

After the input management module 810 of the device B receives the information that the device B has entered the input state, it notifies the input state sending module 820 of the device B to send a broadcast message to the surrounding devices, and the broadcast message is used to indicate that the device B needs to perform text input; at the same time; , the input content receiving module 830 of the notification device B enters the input content receiving state.

After receiving the above instruction, the input state sending module 820 of device B sends the broadcast message to surrounding devices. After receiving the input content receiving state instruction, the input content receiving module 830 of the device B starts to monitor the messages containing the input content sent by the surrounding devices.

Exemplarily, the broadcast message may be a Bluetooth low energy (Bluetooth low energy, BLE) data packet, and the BLE data packet may carry indication information, where the indication information is used to indicate that the device B needs to perform text input. The BLE data packet includes a protocol data unit (PDU), and the indication information may be carried in the service data (service data) field in the PDU, or may also be carried in the manufacturer specific data (manufacturer specific data) in the PDU ) field. For example, the payload of the service data field may include multiple bits, wherein the multiple bits include scalable bits. Device A and Device B can agree on the content of a certain extensible bit. When a certain extensible bit is 1, device A can know that device B needs to perform text input.

In one embodiment, the broadcast message may also carry the media access control (media access control, MAC) address of device B. Exemplarily, if the broadcast message is a BLE data packet, the MAC address of device B may be carried in an access address (access address) field in the BLE data packet.

Exemplarily, the broadcast message may be a user datagram protocol (user datagram protocol, UDP) data packet, and the UDP data packet may carry the indication information, and the indication information is used to indicate that the device B needs to perform text input. The UPD packet includes the data portion of the IP datagram. The data portion of an IP datagram may include extensible bits. Device A and Device B can agree on the content of a certain extensible bit. When a certain extensible bit is 1, device A can know that device B needs to perform text input.

In one embodiment, the UDP data packet can carry the IP address and port number of device B (including source port number and destination port number, wherein the source port number refers to the port number used by device B when sending data, and the destination port number is Refers to the port used by device B to receive data). The IP address and port number of device B can be carried in the UDP header of the data part of the IP datagram. Alternatively, the UDP data packet may also carry the IP address instead of the port number.

The input state receiving module 850 of the device A may always be in a broadcast message monitoring state. After receiving the broadcast message sent by the input state sending module 820 of the device B, the input state receiving module 850 notifies the input management module 840 of the device A of the event that the device B needs text input. The input management module 840 of device A notifies the display screen to display a reminder box (as shown in (b) in FIG. 3 ), or prompts the user with a text input icon (as shown in (a) in FIG. 4 ).

After detecting that the user clicks on the control 303, the device A opens the remote control application. After device A detects that the user has clicked on the input control 304, the input method is displayed. Alternatively, after detecting the operation of the user clicking the icon 401 on the lock screen interface, the device A displays the input method on the lock screen interface.

After acquiring the text content input by the user in the text input box, the input management module 840 of the device A can send the text content to the input content sending module 860 of the device A.

Exemplarily, if the BLE data packet sent by device B to device A carries the MAC address of device B, device A can establish a Bluetooth connection with device B after acquiring the MAC address of device B. Device A can send the text content to device B through BLE data packets. This textual content may be carried in a service data field or a vendor specific data field in the PDU. For example, the payload of the service data field may include multiple bits, wherein the multiple bits include scalable bits. Device A can use encoding methods such as GBK, ISO8859-1 or Unicode (for example, UTF-8, UTF-16) to encode the text content input by the user detected by device A, and carry the encoded information in a or multiple scalable bits. After receiving the BLE data packet sent by device A, device B can decode the information on the corresponding bits to obtain the text content input by the user on device A.

Exemplarily, if the UDP data packet carries the IP address and destination port number of device B. Device A can establish a transmission control protocol (TCP) connection with device B through the IP address and destination port number. Device A can send the text content detected by device A and input by the user to the destination port number through a TCP connection.

Exemplarily, if the UDP data packet carries the IP address of the device B but does not carry the destination port number, then the device A may not establish a TCP connection with the device B after obtaining the IP address of the device B. Device A can send a UDP data packet to device B, where the UDP data packet can carry the text content input by the user detected on device A. Exemplarily, the text content may be carried in a data portion of an IP datagram in a UDP data packet. The data part includes an extensible field, and device A and device B can agree on a certain extensible bit to carry the text content. Device A can use encoding methods such as GBK, ISO8859-1, or Unicode to encode the text content input by the user detected by device A, and carry the encoded information on one or more expandable bits. After receiving the UDP data packet sent by the device A, the device B can decode the information on the corresponding bits, so as to obtain the text content input by the user on the device A.

After receiving the text content sent by the input content sending module 860, the input content receiving module 830 in the input content receiving state sends the text content to the input management module 810 of the device B. The input management module 810 of device B displays the received text content in the text input box.

The internal implementation process in the GUI shown in FIG. 3 and FIG. 4 is given above, wherein the input state receiving module 850 of the device A can always be in the broadcast message listening state. The following describes the internal implementation process in the GUI shown in FIG. 5 and FIG. 6 . In the internal implementation process of the GUI shown in FIG. 5 and FIG. 6 , the difference from FIG. 3 and FIG. 4 is that the input state receiving module 850 of device A starts to monitor the broadcast when device A detects the preset operation of the user. information.

After device A detects that the user opens the remote control application, the input management module 840 notifies the input state receiving module 850 to enter the broadcast message monitoring state, thereby starting to monitor the broadcast message; or, when the device A detects that the user clicks on the input control, the input method is displayed. , the input management module 840 monitors that the mobile phone displays the input method. The input management module 840 notifies the input state receiving module 850 to enter the broadcast message monitoring state, thereby starting to monitor broadcast messages.

It should be understood that the embodiments of the present application are described by taking the device A entering the broadcast message monitoring state after detecting that the user has opened the remote control application as an example, and the embodiments of the present application are not limited thereto. Device A may or may enter the broadcast message listening state after detecting that other applications (for example, App1) are opened. Exemplarily, after device A detects that the user has clicked on the icon of App1, App1 of the application layer will send the label corresponding to App1 (eg, process identifier (PID)) to the system service of the application framework layer. The process name corresponding to the App1, the system service can determine that App1 is started according to the label and the process name. After determining that App1 is started, the system service can trigger the input state receiving module 850 of device A (eg, the wireless communication module in FIG. 1 ) to enter the broadcast message monitoring state.

In one embodiment, the device A may also enter the broadcast message monitoring state after detecting the preset operation of the user. Exemplarily, the preset operation may be an operation such as double-clicking, long-pressing, folding or unfolding the screen, and the like. When the device A detects the user's preset operation, it can trigger the input state receiving module 850 to enter the broadcast message monitoring state. When device A receives broadcast messages sent by surrounding devices, it can automatically display the input method.

The input state receiving module 850 of device A receives the broadcast message sent by the input state sending module 820 of device B, and learns that device B needs text input. It should be understood that for the manner in which the device B sends the broadcast message, reference may be made to the descriptions in the foregoing embodiments, which are not repeated here for brevity.

The input state receiving module 850 of device A notifies the input management module 840 of device A of the event that device B needs text input; the input management module 840 of device A obtains the text content input by the user through the input method service; the input management module of device A 840 invokes the input content sending module 860 of device A to send the text content input by the user to device B. It should be understood that, for the manner in which device A sends user input content to device B, reference may be made to the description in the foregoing embodiment, which is not repeated here for brevity.

After receiving the text content sent by the input content sending module 860 of the device A, the input content receiving module 830 of the device B in the input content receiving state sends the text content to the input management module 810 of the device B. The input management module 810 of device B displays the received text content in the text input box.

Through the above process, the cross-device text input function from device A to device B is completed.

In the above process, the communication between device A and device B may select Bluetooth communication or local area network communication as required. When device B sends a broadcast message to the outside world, it can choose either one of Bluetooth or a local area network, or both methods can be sent at the same time.

When device A sends the text content input by the user to device B, it can choose the fastest way to send according to whether it has been paired with device B by Bluetooth or whether it is in the same local area network.

Exemplarily, if device A receives the BLE data packet sent by device B and includes the MAC address of device B, device A can determine whether Bluetooth pairing with device B has been performed before according to the MAC address of device B. If Bluetooth pairing is performed between device A and device B, device A can perform Bluetooth pairing with device B and establish a Bluetooth connection. After the Bluetooth connection is established, device A can send a BLE data packet to device B, where the BLE data packet carries the text content input by the user.

Exemplarily, if the broadcast message received by device A includes the IP address of device B, device A can determine whether device A and device B are in the same local area network according to the IP address of device B. If device A is determined to be in the same local area network as device B, then if the UDP data packet also carries the destination port number of device B, then device A can establish a TCP connection with device B. After the TCP connection is established, device A can send the text information to device B through the TCP connection. Alternatively, if device B only carries the IP address of device B without the destination port number in the UDP data packet, device A can send a UDP data packet to device B, where the UDP data packet carries the text content input by the user.

In one embodiment, in order to ensure that the text content input by the user sent by device A to device B is not leaked, device A can use the encryption key of device B to encrypt and send the text content. For example, the public key and the private key are stored in the device B, and the device B can carry its own public key in the broadcast message. Exemplarily, the public key of device B may be carried in a service data field or a vendor-specific field in the BLE data packet. When device A sends the text content to device B, it can use the public key of device B to encrypt the text content. Exemplarily, if device A and device B establish a TCP connection, device A can send the text content encrypted by the public key to device B through the TCP connection; or, if device A does not establish a TCP connection with device B, then device B A UDP data packet can be sent to device A, and the data part of the IP datagram in the UPD data packet can carry the text content encrypted by the public key; or, device A can send a BLE data packet to device B, the service in the BLE data packet The data field or the vendor-specific field can carry the text content encrypted by the public key. After receiving the text content encrypted by the public key, the device B can decrypt it by using the private key to obtain the text content sent by the device A. Other devices can also monitor the text content encrypted by the public key, but since they do not have the private key of device B, they cannot decrypt the text content encrypted by the public key. Thus, it is ensured that the text content sent by device A to device B is not leaked.

In the embodiment of the present application, device A and device B do not need to actively enter into pairing in advance or establish a connection through the network. When device B needs to input text, device A can dynamically obtain relevant information, and can assist device B to complete the input of text content .

FIG. 9 shows a schematic flowchart of a method 900 for text input provided by an embodiment of the present application. The method 900 can be performed by the first electronic device and the second electronic device, and the method 900 includes:

S901, the first electronic device displays a text input interface through a display screen, where the text input interface includes a text input box.

Exemplarily, as shown in FIG. 3( a ), the display interface of the smart TV is a text input interface, and the text input interface includes a text input box 301 .

S902: In response to displaying the text input interface, the first electronic device sends a first message, where the first message is used to indicate that the first electronic device needs to perform text input.

In one embodiment, the first electronic device responds to displaying the text input interface, including that the first electronic device responds to the current focus of the first electronic device in the text input box of the text input interface; or, the first electronic device responds In response to displaying the input interface, the device includes that the first electronic device responds that the current focus of the first electronic device is on a certain key of the input method displayed on the text input interface.

Exemplarily, as shown in FIG. 3 (a), the display interface of the smart TV is a text input interface, and the current focus of the smart TV displayed on the text input interface is the text input box 301 . In response to the current focus of the smart TV being the text input box 301, the smart TV sends the first message.

In one embodiment, sending the first message by the first electronic device in response to displaying the text input interface includes: the first electronic device sending the first message to one or more devices in response to displaying the text input interface.

Exemplarily, the one or more devices and the first electronic device are devices under the same account (for example, a Huawei account); or, the one or more devices and the first electronic device are under the same family group. account.

Exemplarily, the one or more devices may be devices that have completed Bluetooth pairing with the first electronic device; or, the one or more devices may be devices under the same Wi-Fi as the first electronic device.

In one embodiment, sending the first message by the first electronic device in response to displaying the text input interface includes: the first electronic device, in response to displaying the text input interface, sending a broadcast message to surrounding devices, the broadcast message using to indicate that the first electronic device needs to perform text input.

It should be understood that, for the manner in which the first electronic device sends the broadcast message, reference may be made to the descriptions in the foregoing embodiments, which are not repeated here for brevity.

S903, the second electronic device detects the preset operation of the user and monitors the first message.

It should be understood that, in this embodiment of the present application, the order in which the second electronic device detects the user's operation and monitors the first message is not specifically limited. The second electronic device may first detect the user's operation, and then receive the first message. a message; or, the second electronic device may also receive the first message first, and then detect the user's preset operation.

In one embodiment, the preset operation may be an operation by the user to open an application; or, the preset operation may be a preset gesture of the user (for example, the user draws a preset pattern on the display screen of the second electronic device. , or the user's space gesture); or, the preset operation may also be an operation of the user pressing a certain physical button; or, the preset operation may also be a combination of a preset gesture and pressing a physical button.

In one embodiment, the preset operation may also be an operation of the user picking up the second electronic device.

Exemplarily, the second electronic device includes a gyro sensor (eg, the gyro sensor 180B in FIG. 1 ), and the second electronic device can detect whether the user picks up the second electronic device through the gyro sensor.

In one embodiment, the preset operation may also be an operation for the user to unlock the second electronic device.

In one embodiment, the second electronic device may detect whether the first electronic device is within a preset angle range of the second electronic device while detecting the user's preset operation and monitoring the first message.

Exemplarily, the second electronic device may be a device capable of calculating an angle of arrival (AOA). For example, the second electronic device may include a compass, a Bluetooth/Wi-Fi antenna array, and the like. The second electronic device can calculate the orientation of the first electronic device, the Bluetooth/WiFi antenna array of the second electronic device can receive the wireless signal of the first electronic device, and calculate the first electronic device according to formulas (1) and (2). Orientation:

where d is the distance between the Bluetooth/Wi-Fi antenna array of the second electronic device and the Bluetooth/Wi-Fi antenna of the first electronic device,

is the phase difference between the Bluetooth/Wi-Fi antenna array of the second electronic device and the Bluetooth/Wi-Fi antenna of the first electronic device, λ is the phase difference of the Bluetooth signal (for example, the first message) sent by the first electronic device wavelength, and θ is the angle of arrival.

In one embodiment, the second electronic device detects the user's preset operation and monitors the first message, including: the second electronic device detects the user's preset operation; in response to detecting the user's preset operation, the second electronic device detects the user's preset operation. The device starts listening for the first message.

Exemplarily, as shown in (c) of FIG. 5 , after the mobile phone detects that the user has opened the remote control application, the mobile phone starts to monitor the first message.

Exemplarily, as shown in (a) of FIG. 7 , after the mobile phone detects the operation of the user drawing "S" on the display interface, the mobile phone starts to monitor the first message.

It should be understood that, considering that there may be a certain time interval between when the second electronic device displays the text input interface and when the mobile phone detects the user's preset gesture, the first electronic device can display the text input interface within the first preset time period ( For example, 1 minute) to send multiple first messages. In this way, it can be ensured that the second electronic device can receive the first message after detecting the user's preset operation.

In one embodiment, the second electronic device detects the preset operation of the user and monitors the first message, including: the second electronic device monitors the first message; in response to receiving the first message, the second electronic device detects the user's Preset action.

Exemplarily, as shown in (c) of FIG. 5 , before the mobile phone detects that the user has opened the remote control application, the mobile phone receives the first message. However, since the mobile phone has not detected the user's preset operation, the mobile phone can first save the first message without giving the user any text input prompt. As shown in (d) in FIG. 5 , when the mobile phone detects the operation of the user opening the remote control application, the mobile phone can directly display the reminder box 503, thereby prompting the user to use the mobile phone to input text to the smart TV; or, when the mobile phone detects When the user opens the remote control application, the mobile phone can directly call up the input method to display the GUI shown in (e) of FIG. 5 .

Exemplarily, as shown in (a) of FIG. 7 , before the mobile phone detects that the user draws “S” on the lock screen interface, the mobile phone receives the first message. However, since the mobile phone has not detected the preset operation of the user, the mobile phone can first save the first message without giving the user any text input prompt. When the mobile phone detects that the user draws "S", the mobile phone can directly display the GUI shown in (b) in FIG. 7 ; or, the mobile phone can directly display the GUI shown in (c) in FIG. 7 ; or , the mobile phone displays the GUI shown in (d) in FIG. 7 after entering the lock screen interface.

It should be understood that, considering that there may be a certain time interval between when the first electronic device displays the text input interface and when the second electronic device detects the user's preset gesture, the second electronic device may also monitor the first message all the time. If the mobile phone detects the user's preset gesture after receiving the first message. In this case, when detecting that the text input interface is displayed, the first electronic device may send a plurality of first messages within a second preset time period (for example, 5 seconds). After receiving the first message, the second electronic device can start to detect the user's preset operation.

S904, in response to detecting the user's preset operation and receiving the first message, the second electronic device detects the content input by the user.

It should be understood that the second electronic device may start to monitor the first message in response to detecting the user's preset operation; or, the second electronic device may start to detect the user's preset operation in response to monitoring the first message . In this embodiment of the present application, there may be no relationship between the second electronic device detecting the user's preset operation and monitoring the first message, as long as the second electronic device detects the user's preset operation and receives the first message, The second electronic device can then detect the user's input.

In one embodiment, in response to detecting the user's preset operation and receiving the first message, the second electronic device detects the content input by the user, including: in response to detecting the user's preset operation and receiving the first message , and the time interval between the second electronic device detecting the user's preset operation and the second electronic device receiving the first message is less than the preset time interval, and the second electronic device detecting the content input by the user .

In one embodiment, in response to detecting the user's preset operation and receiving the first message, the second electronic device detects the content input by the user, including: in response to detecting the user's preset operation, receiving the first message and determining that the first electronic device is within a preset angle range of the second electronic device, and the second electronic device detects the content input by the user.

In one embodiment, in response to detecting the user's preset operation and receiving the first message, the second electronic device detects the content input by the user, including: if the second electronic device receives the first message in the first The user's preset operation is detected within three preset time periods, and the second electronic device detects the user's input.

In this embodiment of the present application, the second electronic device may not detect the user's preset operation for a long time after receiving the first message, so the user may not use the second electronic device to input text to the first electronic device. The second electronic device may ignore the first message after exceeding the third preset time period. That is, if the second electronic device detects the user's preset operation after the third preset time period, the second electronic device will not give the user any text input prompt; or, the second electronic device will not call up the input method. In this way, interference to the user caused by performing a preset operation on the second electronic device after the user has not used the second electronic device for a long time can be avoided.

Exemplarily, as shown in (e) of FIG. 5 , when the mobile phone detects that the user has opened the remote control application and receives the first message, the mobile phone can call up the input method, so that the mobile phone starts to detect the content input by the user.

Exemplarily, as shown in (c) of FIG. 7 , when the mobile phone detects that the user draws an “S” on the lock screen interface and receives the first message, the mobile phone can call up the input method, so that the mobile phone can call up the input method. Start detecting user input.

In one embodiment, the first message sent by the first electronic device may be received by multiple electronic devices (for example, the multiple electronic devices include a second electronic device, a third electronic device, and a fourth electronic device). If within a fourth preset time period after the first electronic device sends the first message, the second electronic device and the third electronic device request to establish a connection with the first electronic device, the first electronic device can communicate with the second electronic device, the third electronic device The three electronic devices establish a connection, so that the second electronic device and the third electronic device can call up the input method to detect the content input by the user.

After the fourth preset time period, if the fourth electronic device also receives the first message, the fourth electronic device requests to establish a connection with the first electronic device. At this time, the first electronic device may reject the request of the fourth electronic device, so that no text input prompt will be displayed on the fourth electronic device or the fourth electronic device will not call up the input method. This also helps to avoid interference to the user caused by the prompt information or input method displayed on the electronic device that receives the first message after a period of time.

In one embodiment, the first electronic device may also only establish a connection with the device (eg, the second electronic device) that first requests to establish a connection. For requests from other electronic devices, the first electronic device may ignore its requests. This helps avoid interference when inputting from multiple devices.

S905, the second electronic device sends the first content to the first electronic device in response to detecting an operation of the user inputting the first content.

Exemplarily, as shown in (e) of FIG. 5 , after the mobile phone detects the operation of the user inputting the text content movie 1", the mobile phone can send the text content to the intelligence.

Exemplarily, as shown in (f) of FIG. 5 , after the mobile phone detects the operation of the user inputting the voice content movie 1", the mobile phone can send the text content to the intelligence.

Exemplarily, as shown in (e) of FIG. 6 , after the mobile phone detects the operation of the user inputting the voice content movie 1", the mobile phone can send the text content to the intelligence.

Exemplarily, as shown in (c) of FIG. 7 , after the mobile phone detects the operation of the user inputting the text content movie 1", the mobile phone can send the text content to the intelligence.

It should be understood that, in the embodiment of the present application, the mobile phone can send the detected content to the smart TV in real time, and the specific process can refer to the description in the above embodiment, which is not repeated here for brevity.

S906, the first electronic device displays the text content corresponding to the first content in the text input box.

Exemplarily, as shown in (e) of FIG. 3 , after receiving the text content "movie 1" sent by the mobile phone, the smart TV may display the text content in the text input box.

Exemplarily, if the smart TV receives the voice content sent by the mobile phone, the smart TV can first convert the voice content into text content, so as to display the text content in the text input box 301 .

In one embodiment, if the second electronic device detects that the content input by the user is voice content, the second electronic device can also convert the voice content into text content and send it to the first electronic device, so that the first electronic device can Corresponding text content is displayed in the text input box 301 .

In the embodiment of the present application, when the first electronic device needs to input text, the user can pick up any device (eg, mobile phone, Pad) around him to input, which helps to improve the convenience of the user when inputting text. to enhance the user experience. At the same time, the second electronic device can prompt the user to input text through a reminder box when detecting the user's preset operation and receiving the first message, which helps the user to identify that the second electronic device can be used as an input device. Before the user performs a preset operation on the second electronic device and receives the first message, the second electronic device will not display any prompt information that may interfere with the user, thereby avoiding interference to the user, thereby helping to improve user experience .

FIG. 10 shows a schematic block diagram of an apparatus 1000 for text input provided by an embodiment of the present application. The apparatus 1000 can be set in the second electronic device in the above-mentioned FIG. 9 , and the apparatus 1000 includes: a first detection unit 1010 for detecting a preset operation of a user; a receiving unit 1020 for monitoring the first message, the The first message is used to indicate that another electronic device needs to perform text input; the second detection unit 1030 is also used to detect the user when the first detection unit 1010 detects the user's preset operation and the receiving unit 1020 receives the first message The input content; the sending unit 1040 is configured to send the first content to the other electronic device in response to the second detection unit 1030 detecting an operation of the user inputting the first content.

FIG. 11 shows a schematic structural diagram of an electronic device 1100 provided by an embodiment of the present application. As shown in FIG. 11, the electronic device includes: one or more processors 1110, one or more memories 1120, and the one or more memory stores 1120 store one or more computer programs, the one or more computer programs Include instructions. When the instruction is executed by the one or more processors 1110, the second electronic device (or, the mobile phone in the above embodiment) executes the technical solution in the above embodiment.

An embodiment of the present application provides a system for text input, including a first electronic device and a second electronic device, and the system is used to implement the technical solution for text input in the foregoing embodiment. The implementation principle and technical effect thereof are similar to the related embodiments of the above method, and are not repeated here.

The embodiments of the present application provide a computer program product, which enables the second electronic device (or the mobile phone in the above embodiment) to execute the technical solutions in the above embodiments when the computer program product runs on the second electronic device. The implementation principle and technical effect thereof are similar to the related embodiments of the above method, and are not repeated here.

An embodiment of the present application provides a readable storage medium, where the readable storage medium contains instructions, when the instructions are executed on a second electronic device (or, the mobile phone in the above-mentioned embodiment), the second electronic device causes the second electronic device to be executed. The technical solutions of the above embodiments are implemented. The implementation principle and technical effect thereof are similar, and are not repeated here.

An embodiment of the present application provides a chip, which is used for executing an instruction, and when the chip is running, executes the technical solutions in the foregoing embodiments. The implementation principle and technical effect thereof are similar, and are not repeated here.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described systems, devices and units may refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution, and the computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited to this. should be covered within the scope of protection of this application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

A system comprising a first electronic device and a second electronic device, characterized in that:

the first electronic device, configured to display a text input interface through a display screen, the text input interface comprising a text input box;

The first electronic device is further configured to, in response to displaying the text input interface, send a first message, where the first message is used to indicate that the first electronic device needs to perform text input;

the second electronic device, configured to detect the user's preset operation and monitor the first message;

The second electronic device is further configured to detect the content input by the user in response to detecting the preset operation and receiving the first message;

The second electronic device is further configured to send the first content to the first electronic device in response to detecting an operation of the user inputting the first content;

The first electronic device is further configured to display text content corresponding to the first content in the text input box.
The system according to claim 1, wherein the second electronic device is specifically used for:

in response to detecting the preset operation and receiving the first message, displaying an input method;

The text content input by the user through the input method is detected.
The system according to claim 1, wherein the second electronic device is specifically used for:

In response to detecting the preset operation and receiving the first message, detecting the voice content input by the user;

In response to detecting an operation of the user inputting the first voice content, sending the first voice content to the first electronic device;

Wherein, the first electronic device is specifically used for:

determining the text content corresponding to the first voice content;

The text content is displayed in the text input box.
The system according to any one of claims 1 to 3, wherein the second electronic device is further configured to:

Before the content input by the user is detected, first prompt information is displayed, where the first prompt information is used to prompt the second electronic device to be a device capable of inputting to the first electronic device.
The system according to any one of claims 1 to 4, wherein the first electronic device is further configured to:

Before the text content is displayed in the text input box, second prompt information is displayed on the display screen, where the second prompt information is used to prompt the user to input an input to the first electronic device through the second electronic device .
The system according to any one of claims 1 to 5, wherein the second electronic device is specifically used for:

When it is detected that the user starts the first application and the first message is received, the content input by the user is detected.
The system according to claim 6, wherein the first application is a remote control application.
The system according to any one of claims 1 to 7, wherein the first electronic device is a smart TV.
A method for text input, the method being applied to an electronic device, characterized in that the method comprises:

The electronic device detects the user's preset operation and monitors a first message, where the first message is used to indicate that another electronic device needs to perform text input;

In response to detecting the preset operation of the user and receiving the first message, the electronic device detects the content input by the user;

The electronic device transmits the first content to the other electronic device in response to detecting an operation of the user inputting the first content.
The method according to claim 9, wherein, in response to detecting the user's preset operation and receiving the first message, the electronic device detects the user's input, comprising:

In response to detecting the preset operation of the user and receiving the first message, the electronic device displays an input method;

The electronic device detects the text content input by the user through the input method.
The method according to claim 9, wherein, in response to detecting the user's preset operation and receiving the first message, the electronic device detects the user's input, comprising:

In response to detecting the preset operation of the user and receiving the first message, the electronic device detects the voice content input by the user.
The method according to any one of claims 9 to 11, wherein before the electronic device detects the content input by the user, the method further comprises:

The electronic device displays prompt information, and the prompt information is used to prompt the user that the electronic device is a device capable of inputting to the other electronic device.
The method according to any one of claims 9 to 12, wherein the detecting the preset operation of the user comprises:

The electronic device detects that the user starts the operation of the first application program.
The method according to claim 13, wherein the first application is a remote control application.
The method according to any one of claims 9 to 14, wherein the other electronic device is a smart TV.
An electronic device, characterized in that, one or more processors; one or more memories; the one or more memories store one or more computer programs, and the one or more computer programs include instructions, when The instructions, when executed by the one or more processors, cause the electronic device to perform a method of text input as claimed in any one of claims 9 to 15.
A computer-readable storage medium, characterized in that it includes computer instructions that, when the computer instructions are executed on an electronic device, cause the electronic device to perform the text input process according to any one of claims 9 to 15. method.