WO2022135254A1 - Procédé d'édition de texte, dispositif électronique et système - Google Patents

Procédé d'édition de texte, dispositif électronique et système Download PDF

Info

Publication number
WO2022135254A1
WO2022135254A1 PCT/CN2021/138655 CN2021138655W WO2022135254A1 WO 2022135254 A1 WO2022135254 A1 WO 2022135254A1 CN 2021138655 W CN2021138655 W CN 2021138655W WO 2022135254 A1 WO2022135254 A1 WO 2022135254A1
Authority
WO
WIPO (PCT)
Prior art keywords
electronic device
text content
content
text
information
Prior art date
Application number
PCT/CN2021/138655
Other languages
English (en)
Chinese (zh)
Inventor
张丽
唐吴全
余平
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2022135254A1 publication Critical patent/WO2022135254A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/26—Speech to text systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/10—Text processing
    • G06F40/103—Formatting, i.e. changing of presentation of documents
    • G06F40/109—Font handling; Temporal or kinetic typography
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/10—Text processing
    • G06F40/166—Editing, e.g. inserting or deleting
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue

Definitions

  • the present application relates to the field of terminals, and more particularly, to a method, electronic device and system for editing text.
  • ASR automatic speech recognition
  • the present application provides a method, electronic device and system for editing text, which help to improve the editing efficiency of text editing by users.
  • a system in a first aspect, includes a first electronic device and a second electronic device, wherein the first electronic device is used to acquire audio content; the first electronic device is further configured to report to the second electronic device Send first information, where the first information is the audio content, or the first information is the first text content corresponding to the audio content; the second electronic device is configured to display the first text content according to the first information ; the second electronic device is further configured to display the second text content in response to the user's editing operation on the first text content, where the second text content is the edited text content of the first text content.
  • the first electronic device can send the acquired text content corresponding to the audio content to the second electronic device, so that the text content can be displayed on the second electronic device, so that it is convenient for the user to display the text content on the second electronic device Edit the text content to help improve the efficiency of the user when editing the text content.
  • the first electronic device may store information of one or more electronic devices, and when the first electronic device acquires audio, the first electronic device may select a suitable one from the one or more electronic devices A second electronic device that performs text editing, thereby sending the first information to the second electronic device.
  • the second electronic device in response to receiving the first information, may open a first application, and display the first text content in the first application; the second electronic device may The input method of the second electronic device edits the first text content.
  • the second electronic device is further configured to send the second text content to the first electronic device.
  • the second electronic device is further configured to send the second text content to the first electronic device when the first operation of the user is detected.
  • the first operation is an operation in which the user clicks to save.
  • the second electronic device may acquire the text content edited by the user and send the edited text content to the first electronic device, so that the first electronic device may save the edited text content.
  • the edited text content can also be obtained by the first electronic device without any additional operation by the user.
  • the editing operation includes a format modification operation for the first text content, wherein the second electronic device is further configured to send the first electronic device the Format information of the second text content.
  • the second electronic device may also send the format information of the edited text content to the first electronic device, so that the first electronic device can restore the second electronic device according to the format information.
  • User-edited text content on an electronic device when the user modifies the format of the text content, the second electronic device may also send the format information of the edited text content to the first electronic device, so that the first electronic device can restore the second electronic device according to the format information.
  • the format information of the second text content includes the font color, font size, font background color, font inclination, font underline and the second text content of the second text content.
  • One or more types of carriage return operations in the text content include the font color, font size, font background color, font inclination, font underline and the second text content of the second text content.
  • the first electronic device is further configured to: display the first electronic device according to the audio content before receiving the second text content sent by the second electronic device a text content; after receiving the second text content sent by the second electronic device, replace the first text content with the second text content.
  • the first electronic device may display the corresponding first text content when acquiring the audio content, and after the first electronic device receives the user-edited second text content sent by the second electronic device, the first electronic device The electronic device can replace the previous first text content with the second text content, so that it is convenient for the user to watch the edited text content on the first electronic device and the second electronic device at the same time, which helps to improve the user's experience.
  • the first electronic device is further configured to send a query request, and the query request is used for the device that received the query request to determine whether it has a text editing function; in response to receiving the query request To the response sent by the second electronic device, the first information is sent to the second electronic device, where the response is used to indicate that the second electronic device has a text editing function.
  • the first electronic device before sending the first information to the second electronic device, may inquire about a device with a text editing function. After determining that the second electronic device has a text editing function, the first electronic device can send the first information to the second electronic device, so as to facilitate the user to edit the text content on the second electronic device and help improve the user's ability to Efficiency when editing text.
  • the second electronic device is specifically configured to: in response to receiving the first information, prompt the user whether to perform text editing on the second electronic device; The user determines to perform a text editing operation on the second electronic device to display the first text content.
  • the second electronic device when the second electronic device receives the first information, it may first prompt the user whether to agree to perform text editing on the second electronic device. If the second electronic device detects the operation agreed by the user, it can display the text content, which helps to avoid interference to the user, and the user can choose a device he thinks is suitable for text editing, which helps to improve the user's experience.
  • the first electronic device may also send request information to the second electronic device, where the request information is used to request the second electronic device to edit the first text content; the second electronic device may respond to The request information prompts the user whether to agree to edit the text content on the second electronic device; if the second electronic device detects that the user agrees to edit the text content on the second electronic device, the second electronic device can display the First text content.
  • the first information is the first text content
  • the first electronic device is specifically configured to: in response to acquiring the audio content, convert the audio content into the first text content; sending the first text content to the second electronic device.
  • the first electronic device after acquiring the audio content, can convert the audio content into text content, so as to send the text content to the second electronic device, and the second electronic device can display the corresponding text content, thereby facilitating the The user edits the text content.
  • the first information is the audio content
  • the second electronic device is specifically configured to: in response to receiving the audio content, convert the audio content into the audio content a text content; display the first text content.
  • the first electronic device may send the acquired audio content to the second electronic device, and the second electronic device may convert the audio content into text content and display the text content, thereby facilitating the user to edit the text content .
  • the account of the first electronic device is associated with the account of the second electronic device.
  • a text editing method is provided, the method is applied to a first electronic device, the method includes: the first electronic device acquires audio content; the first electronic device sends first information to the second electronic device, The first information is the audio content, or the first information is the first text content corresponding to the audio content, so that the second electronic device displays the first text content according to the first information and detects the user's An editing operation for text content.
  • the method further includes: receiving, by the first electronic device, second text content sent by the second electronic device, where the second text content Text content after editing the first text content on the electronic device.
  • the method further includes: receiving, by the first electronic device, format information of the second text content sent by the second electronic device.
  • the format information of the second text content includes the font color, font size, font background color, font inclination, font underline and the second text content of the second text content.
  • One or more types of carriage return operations in the text content include the font color, font size, font background color, font inclination, font underline and the second text content of the second text content.
  • the method further includes: before receiving the second text content sent by the second electronic device, the first electronic device displays the first electronic device according to the audio content a text content; after receiving the second text content sent by the second electronic device, the first electronic device replaces the first text content with the second text content.
  • the method includes: before sending the first information to the second electronic device, the first electronic device sends a query request, where the query request is used to receive The device of the query request determines whether it has a text editing function; in response to receiving a response sent by the second electronic device, the first electronic device sends the first information to the second electronic device, and the response is used to indicate the second electronic device.
  • Electronic devices have text editing capabilities.
  • the first information is the first text content
  • the method further includes: in response to acquiring the audio content, the first electronic device converts the audio content is the first text content; the first electronic device sends the first text content to the second electronic device.
  • the account of the first electronic device is associated with the account of the second electronic device.
  • a text editing method is provided, the method is applied to a second electronic device, and the method includes: the second electronic device receives first information sent by the first electronic device, where the first information is the first information The audio content obtained by the electronic device, or the first information is the first text content corresponding to the audio content; the second electronic device displays the first text content according to the first information; In the content editing operation, the second electronic device displays the second text content, and the second text content is the edited text content of the first text content.
  • the method further includes: the second electronic device sending the second text content to the first electronic device.
  • the editing operation includes a format modification operation for the first text content
  • the method further includes: the second electronic device sends the first electronic device the first electronic device. 2.
  • the format information of the text content is included in some implementations of the third aspect.
  • the format information of the second text content includes the font color, font size, font background color, font inclination, font underline and the second text content of the second text content.
  • One or more types of carriage return operations in the text content include the font color, font size, font background color, font inclination, font underline and the second text content of the second text content.
  • the method further includes: before receiving the first information sent by the first electronic device, receiving, by the second electronic device, a query sent by the first electronic device request, the query requests the second electronic device to determine whether it has a text editing function; the second electronic device sends a response to the first electronic device, where the response is used to indicate that the second electronic device has a text editing function.
  • the second electronic device displays the first text content according to the first information, including: in response to receiving the first information, the second electronic device Prompting the user whether to perform text editing on the second electronic device; in response to the user determining that the text editing is performed on the second electronic device, the second electronic device displays the first text content.
  • the first information is the audio content
  • the method further includes: in response to receiving the audio content , and convert the audio content into the first text content.
  • the account of the first electronic device is associated with the account of the second electronic device.
  • an apparatus comprising: an acquiring unit for acquiring audio content; a sending unit for transmitting first information to a second electronic device, where the first information is the audio content, or the The first information is the first text content corresponding to the audio content, so that the second electronic device displays the first text content according to the first information and detects a user's editing operation on the first text content.
  • an apparatus comprising: a receiving unit configured to receive first information sent by a first electronic device, where the first information is audio content acquired by the first electronic device, or the first information The information is the first text content corresponding to the audio content; the display unit is used to display the first text content according to the first information; the detection unit is used to detect the user's editing operation on the first text content; the display unit, It is also used to display the second text content, where the second text content is the edited text content of the first text content.
  • an electronic device comprising: one or more processors; a memory; and one or more computer programs. Wherein, one or more computer programs are stored in the memory, the one or more computer programs comprising instructions. When the instructions are executed by the electronic device, the electronic device is caused to execute the method in any one of the possible implementations of the second aspect above.
  • an electronic device comprising: one or more processors; a memory; and one or more computer programs. Wherein, one or more computer programs are stored in the memory, the one or more computer programs comprising instructions. When the instructions are executed by the electronic device, the electronic device is caused to execute the method in any one of the possible implementations of the third aspect above.
  • a computer program product containing instructions, when the computer program product is run on a first electronic device, the electronic device causes the electronic device to perform the method described in the second aspect; or, when the computer program product is executed on a first electronic device
  • the computer program product when running on the second electronic device, causes the electronic device to perform the method of the third aspect above.
  • a computer-readable storage medium comprising instructions, when the instructions are executed on a first electronic device, the electronic device causes the electronic device to perform the method described in the second aspect; or, when the instructions are executed When the instruction is executed on the second electronic device, the electronic device is caused to execute the method described in the third aspect.
  • a chip for executing an instruction, and when the chip is running, the chip executes the method described in the second aspect; or, the chip executes the method described in the third aspect.
  • FIG. 1 is a schematic diagram of a hardware structure of an electronic device provided by an embodiment of the present application.
  • FIG. 2 is a block diagram of a software structure provided by an embodiment of the present application.
  • FIG. 3 is a set of graphical user interfaces provided by an embodiment of the present application.
  • FIG. 4 is another set of graphical user interfaces provided by an embodiment of the present application.
  • FIG. 5 is another set of graphical user interfaces provided by the embodiments of the present application.
  • FIG. 6 is another set of graphical user interfaces provided by an embodiment of the present application.
  • FIG. 7 is another set of graphical user interfaces provided by the embodiments of the present application.
  • FIG. 8 is another set of graphical user interfaces provided by the embodiments of the present application.
  • FIG. 9 is another set of graphical user interfaces provided by the embodiments of the present application.
  • FIG. 10 is another set of graphical user interfaces provided by the embodiments of the present application.
  • FIG. 11 is a schematic diagram of a system architecture provided by an embodiment of the present application.
  • FIG. 12 is a schematic flow chart of a method for editing text in accordance with an embodiment of the present application.
  • FIG. 13 is a schematic diagram of another system architecture provided by an embodiment of the present application.
  • FIG. 14 is a schematic structural diagram of an apparatus provided by an embodiment of the present application.
  • FIG. 15 is another schematic structural diagram of an apparatus provided by an embodiment of the present application.
  • FIG. 16 is another schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • first and second are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implicitly indicating the number of indicated technical features.
  • a feature defined as “first” or “second” may expressly or implicitly include one or more of that feature.
  • plural means two or more.
  • the methods provided in the embodiments of the present application can be applied to mobile phones, tablet computers, wearable devices, in-vehicle devices, augmented reality (AR)/virtual reality (VR) devices, notebook computers, ultra-mobile personal computers (ultra mobile personal computers) -mobile personal computer, UMPC), netbook, personal digital assistant (personal digital assistant, PDA) and other electronic devices, the embodiments of the present application do not impose any restrictions on the specific types of electronic devices.
  • AR augmented reality
  • VR virtual reality
  • notebook computers notebook computers
  • netbook personal digital assistant
  • PDA personal digital assistant
  • FIG. 1 shows a schematic structural diagram of an electronic device 100 .
  • the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2 , mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, buttons 190, motor 191, indicator 192, camera 193, display screen 194, and User identification (subscriber identification module, SIM) card interface 195 and so on.
  • SIM Subscriber identification module
  • the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light. Sensor 180L, bone conduction sensor 180M, etc.
  • the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100 .
  • the electronic device 100 may include more or less components than shown, or combine some components, or separate some components, or arrange different components.
  • the illustrated components may be implemented in hardware, software, or a combination of software and hardware.
  • the processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU) Wait. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
  • application processor application processor, AP
  • modem processor graphics processor
  • graphics processor graphics processor
  • ISP image signal processor
  • controller memory
  • video codec digital signal processor
  • DSP digital signal processor
  • NPU neural-network processing unit
  • the controller may be the nerve center and command center of the electronic device 100 .
  • the controller can generate an operation control signal according to the instruction operation code and timing signal, and complete the control of fetching and executing instructions.
  • a memory may also be provided in the processor 110 for storing instructions and data.
  • the memory in processor 110 is cache memory. This memory may hold instructions or data that have just been used or recycled by the processor 110 . If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby increasing the efficiency of the system.
  • the processor 110 may include one or more interfaces.
  • the interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transceiver (universal asynchronous transmitter) receiver/transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (SIM) interface, and / or universal serial bus (universal serial bus, USB) interface, etc.
  • I2C integrated circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • PCM pulse code modulation
  • UART universal asynchronous transceiver
  • MIPI mobile industry processor interface
  • GPIO general-purpose input/output
  • SIM subscriber identity module
  • USB universal serial bus
  • the I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL).
  • the processor 110 may contain multiple sets of I2C buses.
  • the processor 110 can be respectively coupled to the touch sensor 180K, the charger, the flash, the camera 193 and the like through different I2C bus interfaces.
  • the processor 110 may couple the touch sensor 180K through the I2C interface, so that the processor 110 and the touch sensor 180K communicate with each other through the I2C bus interface, so as to realize the touch function of the electronic device 100 .
  • the I2S interface can be used for audio communication.
  • the processor 110 may contain multiple sets of I2S buses.
  • the processor 110 may be coupled with the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170 .
  • the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface, so as to realize the function of answering calls through a Bluetooth headset.
  • the PCM interface can also be used for audio communications, sampling, quantizing and encoding analog signals.
  • the audio module 170 and the wireless communication module 160 may be coupled through a PCM bus interface.
  • the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to realize the function of answering calls through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.
  • the UART interface is a universal serial data bus used for asynchronous communication.
  • the bus may be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication.
  • a UART interface is typically used to connect the processor 110 with the wireless communication module 160 .
  • the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function.
  • the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface, so as to realize the function of playing music through the Bluetooth headset.
  • the MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 .
  • MIPI interfaces include camera serial interface (CSI), display serial interface (DSI), etc.
  • the processor 110 communicates with the camera 193 through a CSI interface, so as to realize the photographing function of the electronic device 100 .
  • the processor 110 communicates with the display screen 194 through the DSI interface to implement the display function of the electronic device 100 .
  • the GPIO interface can be configured by software.
  • the GPIO interface can be configured as a control signal or as a data signal.
  • the GPIO interface may be used to connect the processor 110 with the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, and the like.
  • the GPIO interface can also be configured as I2C interface, I2S interface, UART interface, MIPI interface, etc.
  • the USB interface 130 is an interface that conforms to the USB standard specification, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like.
  • the USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio through the headphones.
  • the interface can also be used to connect other electronic devices, such as AR devices.
  • the interface connection relationship between the modules illustrated in the embodiments of the present application is only a schematic illustration, and does not constitute a structural limitation of the electronic device 100 .
  • the electronic device 100 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.
  • the charging management module 140 is used to receive charging input from the charger.
  • the charger may be a wireless charger or a wired charger.
  • the charging management module 140 may receive charging input from the wired charger through the USB interface 130 .
  • the charging management module 140 may receive wireless charging input through the wireless charging coil of the electronic device 100. While the charging management module 140 charges the battery 142 , it can also supply power to the electronic device through the power management module 141 .
  • the power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 .
  • the power management module 141 receives input from the battery 142 and/or the charging management module 140 and supplies power to the processor 110 , the internal memory 121 , the external memory, the display screen 194 , the camera 193 , and the wireless communication module 160 .
  • the power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, battery health status (leakage, impedance).
  • the power management module 141 may also be provided in the processor 110 .
  • the power management module 141 and the charging management module 140 may also be provided in the same device.
  • the wireless communication function of the electronic device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, the baseband processor, and the like.
  • Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals.
  • Each antenna in electronic device 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization.
  • the antenna 1 can be multiplexed as a diversity antenna of the wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.
  • the mobile communication module 150 may provide wireless communication solutions including 2G/3G/4G/5G etc. applied on the electronic device 100 .
  • the mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA) and the like.
  • the mobile communication module 150 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation.
  • the mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and then turn it into an electromagnetic wave for radiation through the antenna 1 .
  • at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110 .
  • at least part of the functional modules of the mobile communication module 150 may be provided in the same device as at least part of the modules of the processor 110 .
  • the modem processor may include a modulator and a demodulator.
  • the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal.
  • the demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing.
  • the low frequency baseband signal is processed by the baseband processor and passed to the application processor.
  • the application processor outputs sound signals through audio devices (not limited to the speaker 170A, the receiver 170B, etc.), or displays images or videos through the display screen 194 .
  • the modem processor may be a stand-alone device.
  • the modem processor may be independent of the processor 110, and may be provided in the same device as the mobile communication module 150 or other functional modules.
  • the wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellites System (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions.
  • WLAN wireless local area networks
  • BT Bluetooth
  • GNSS global navigation satellites System
  • frequency modulation frequency modulation, FM
  • NFC near field communication technology
  • infrared technology infrared, IR
  • the wireless communication module 160 may be one or more devices integrating at least one communication processing module.
  • the wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 .
  • the wireless communication module 160 can also receive the signal to be sent from the processor 110 , perform frequency modulation on it, amplify it, and convert it into electromagnetic waves
  • the antenna 1 of the electronic device 100 is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology.
  • the wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC , FM, and/or IR technology, etc.
  • the GNSS may include a global positioning system (global positioning system, GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (quasi -zenith satellite system, QZSS) and/or satellite based augmentation systems (SBAS).
  • GPS global positioning system
  • GLONASS global navigation satellite system
  • BDS Beidou navigation satellite system
  • QZSS quasi-zenith satellite system
  • SBAS satellite based augmentation systems
  • the electronic device 100 implements a display function through a GPU, a display screen 194, an application processor, and the like.
  • the GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor.
  • the GPU is used to perform mathematical and geometric calculations for graphics rendering.
  • Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.
  • Display screen 194 is used to display images, videos, and the like.
  • Display screen 194 includes a display panel.
  • the display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light).
  • LED diode AMOLED
  • flexible light-emitting diode flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diode (quantum dot light emitting diodes, QLED) and so on.
  • the electronic device 100 may include one or N display screens 194 , where N is a positive integer greater than one.
  • the electronic device 100 may implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, and the like.
  • the ISP is used to process the data fed back by the camera 193 .
  • the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye.
  • ISP can also perform algorithm optimization on image noise, brightness, and skin tone.
  • ISP can also optimize the exposure, color temperature and other parameters of the shooting scene.
  • the ISP may be provided in the camera 193 .
  • the camera 193 is used to capture still images or video.
  • the object is projected through the lens to generate an optical image onto the photosensitive element.
  • the photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor.
  • CMOS complementary metal-oxide-semiconductor
  • the photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal.
  • the ISP outputs the digital image signal to the DSP for processing.
  • DSP converts digital image signals into standard RGB, YUV and other formats of image signals.
  • the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.
  • a digital signal processor is used to process digital signals, in addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy and so on.
  • Video codecs are used to compress or decompress digital video.
  • the electronic device 100 may support one or more video codecs.
  • the electronic device 100 can play or record videos of various encoding formats, such as: Moving Picture Experts Group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4 and so on.
  • MPEG Moving Picture Experts Group
  • MPEG2 moving picture experts group
  • MPEG3 MPEG4
  • MPEG4 Moving Picture Experts Group
  • the NPU is a neural-network (NN) computing processor.
  • NN neural-network
  • Applications such as intelligent cognition of the electronic device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.
  • the external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100 .
  • the external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example to save files like music, video etc in external memory card.
  • Internal memory 121 may be used to store computer executable program code, which includes instructions.
  • the processor 110 executes various functional applications and data processing of the electronic device 100 by executing the instructions stored in the internal memory 121 .
  • the internal memory 121 may include a storage program area and a storage data area.
  • the storage program area can store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), and the like.
  • the storage data area may store data (such as audio data, phone book, etc.) created during the use of the electronic device 100 and the like.
  • the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (UFS), and the like.
  • the electronic device 100 may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, an application processor, and the like. Such as music playback, recording, etc.
  • the audio module 170 is used for converting digital audio information into analog audio signal output, and also for converting analog audio input into digital audio signal. Audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be provided in the processor 110 , or some functional modules of the audio module 170 may be provided in the processor 110 .
  • Speaker 170A also referred to as a "speaker" is used to convert audio electrical signals into sound signals.
  • the electronic device 100 can listen to music through the speaker 170A, or listen to a hands-free call.
  • the receiver 170B also referred to as "earpiece" is used to convert audio electrical signals into sound signals.
  • the voice can be answered by placing the receiver 170B close to the human ear.
  • the microphone 170C also called “microphone” or “microphone” is used to convert sound signals into electrical signals.
  • the user can make a sound by approaching the microphone 170C through a human mouth, and input the sound signal into the microphone 170C.
  • the electronic device 100 may be provided with at least one microphone 170C. In other embodiments, the electronic device 100 may be provided with two microphones 170C, which can implement a noise reduction function in addition to collecting sound signals. In other embodiments, the electronic device 100 may further be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and implement directional recording functions.
  • the earphone jack 170D is used to connect wired earphones.
  • the earphone interface 170D can be the USB interface 130, or can be a 3.5mm open mobile terminal platform (OMTP) standard interface, a cellular telecommunications industry association of the USA (CTIA) standard interface.
  • OMTP open mobile terminal platform
  • CTIA cellular telecommunications industry association of the USA
  • the pressure sensor 180A is used to sense pressure signals, and can convert the pressure signals into electrical signals.
  • the pressure sensor 180A may be provided on the display screen 194 .
  • the capacitive pressure sensor may be comprised of at least two parallel plates of conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes.
  • the electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A.
  • the electronic device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A.
  • touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example, when a touch operation whose intensity is less than the first pressure threshold acts on the short message application icon, the instruction for viewing the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, the instruction to create a new short message is executed.
  • the gyro sensor 180B may be used to determine the motion attitude of the electronic device 100 .
  • the angular velocity of electronic device 100 about three axes ie, x, y, and z axes
  • the gyro sensor 180B can be used for image stabilization.
  • the gyro sensor 180B detects the angle at which the electronic device 100 shakes, calculates the distance that the lens module needs to compensate for according to the angle, and allows the lens to counteract the shake of the electronic device 100 through reverse motion to achieve anti-shake.
  • the gyro sensor 180B can also be used for navigation and somatosensory game scenarios.
  • the air pressure sensor 180C is used to measure air pressure.
  • the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.
  • the magnetic sensor 180D includes a Hall sensor.
  • the electronic device 100 can detect the opening and closing of the flip holster using the magnetic sensor 180D.
  • the electronic device 100 can detect the opening and closing of the flip according to the magnetic sensor 180D. Further, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, characteristics such as automatic unlocking of the flip cover are set.
  • the acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes).
  • the magnitude and direction of gravity can be detected when the electronic device 100 is stationary. It can also be used to identify the posture of electronic devices, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc.
  • the electronic device 100 can measure the distance through infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.
  • Proximity light sensor 180G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes.
  • the light emitting diodes may be infrared light emitting diodes.
  • the electronic device 100 emits infrared light to the outside through the light emitting diode.
  • Electronic device 100 uses photodiodes to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100 . When insufficient reflected light is detected, the electronic device 100 may determine that there is no object near the electronic device 100 .
  • the electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power.
  • Proximity light sensor 180G can also be used in holster mode, pocket mode automatically unlocks and locks the screen.
  • the ambient light sensor 180L is used to sense ambient light brightness.
  • the electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness.
  • the ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures.
  • the ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket, so as to prevent accidental touch.
  • the fingerprint sensor 180H is used to collect fingerprints.
  • the electronic device 100 can use the collected fingerprint characteristics to realize fingerprint unlocking, accessing application locks, taking pictures with fingerprints, answering incoming calls with fingerprints, and the like.
  • the temperature sensor 180J is used to detect the temperature.
  • the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold value, the electronic device 100 reduces the performance of the processor located near the temperature sensor 180J in order to reduce power consumption and implement thermal protection.
  • the electronic device 100 when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 caused by the low temperature.
  • the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.
  • Touch sensor 180K also called “touch panel”.
  • the touch sensor 180K may be disposed on the display screen 194 , and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”.
  • the touch sensor 180K is used to detect a touch operation on or near it.
  • the touch sensor can pass the detected touch operation to the application processor to determine the type of touch event.
  • Visual output related to touch operations may be provided through display screen 194 .
  • the touch sensor 180K may also be disposed on the surface of the electronic device 100 , which is different from the location where the display screen 194 is located.
  • the bone conduction sensor 180M can acquire vibration signals.
  • the bone conduction sensor 180M can acquire the vibration signal of the vibrating bone mass of the human voice.
  • the bone conduction sensor 180M can also contact the pulse of the human body and receive the blood pressure beating signal.
  • the bone conduction sensor 180M can also be disposed in the earphone, combined with the bone conduction earphone.
  • the audio module 170 can analyze the voice signal based on the vibration signal of the vocal vibration bone block obtained by the bone conduction sensor 180M, so as to realize the voice function.
  • the application processor can analyze the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M, so as to realize the function of heart rate detection.
  • the keys 190 include a power-on key, a volume key, and the like. Keys 190 may be mechanical keys. It can also be a touch key.
  • the electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100 .
  • Motor 191 can generate vibrating cues.
  • the motor 191 can be used for vibrating alerts for incoming calls, and can also be used for touch vibration feedback.
  • touch operations acting on different applications can correspond to different vibration feedback effects.
  • the motor 191 can also correspond to different vibration feedback effects for touch operations on different areas of the display screen 194 .
  • Different application scenarios for example: time reminder, receiving information, alarm clock, games, etc.
  • the touch vibration feedback effect can also support customization.
  • the indicator 192 can be an indicator light, which can be used to indicate the charging state, the change of the power, and can also be used to indicate a message, a missed call, a notification, and the like.
  • the SIM card interface 195 is used to connect a SIM card.
  • the SIM card can be contacted and separated from the electronic device 100 by inserting into the SIM card interface 195 or pulling out from the SIM card interface 195 .
  • the electronic device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1.
  • the SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card and so on. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the plurality of cards may be the same or different.
  • the SIM card interface 195 can also be compatible with different types of SIM cards.
  • the SIM card interface 195 is also compatible with external memory cards.
  • the electronic device 100 interacts with the network through the SIM card to implement functions such as call and data communication.
  • the electronic device 100 adopts an embedded SIM (embedded-SIM, eSIM) card, that is, an embedded SIM card.
  • the eSIM card can be embedded in the electronic device 100 and cannot be separated from the
  • the telephone cards in the embodiments of the present application include but are not limited to SIM cards, eSIM cards, universal subscriber identity modules (USIM), universal integrated telephone cards (universal integrated circuit cards, UICC) and the like.
  • the software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture.
  • the embodiments of the present application take an Android system with a layered architecture as an example to exemplarily describe the software structure of the electronic device 100 .
  • FIG. 2 is a block diagram of the software structure of the electronic device 100 according to the embodiment of the present application.
  • the layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate with each other through software interfaces.
  • the Android system is divided into four layers, which are, from top to bottom, an application layer, an application framework layer, an Android runtime (Android runtime) and a system library, and a kernel layer.
  • the application layer can include a series of application packages.
  • the application package can include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and so on.
  • the application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer.
  • the application framework layer includes some predefined functions.
  • the application framework layer may include window managers, content providers, view systems, telephony managers, resource managers, notification managers, and the like.
  • a window manager is used to manage window programs.
  • the window manager can get the size of the display screen, determine whether there is a status bar, lock the screen, take screenshots, etc.
  • Content providers are used to store and retrieve data and make these data accessible to applications.
  • the data may include video, images, audio, calls made and received, browsing history and bookmarks, phone book, etc.
  • the view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on. View systems can be used to build applications.
  • a display interface can consist of one or more views.
  • the display interface including the short message notification icon may include a view for displaying text and a view for displaying pictures.
  • the phone manager is used to provide the communication function of the electronic device 100 .
  • the management of call status including connecting, hanging up, etc.).
  • the resource manager provides various resources for the application, such as localization strings, icons, pictures, layout files, video files and so on.
  • the notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages, and can disappear automatically after a brief pause without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc.
  • the notification manager can also display notifications in the status bar at the top of the system in the form of graphs or scroll bar text, such as notifications of applications running in the background, and notifications on the screen in the form of dialog windows. For example, text information is prompted in the status bar, a prompt sound is issued, the electronic device vibrates, and the indicator light flashes.
  • Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.
  • the core library consists of two parts: one is the function functions that the java language needs to call, and the other is the core library of Android.
  • the application layer and the application framework layer run in virtual machines.
  • the virtual machine executes the java files of the application layer and the application framework layer as binary files.
  • the virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, safety and exception management, and garbage collection.
  • a system library can include multiple functional modules. For example: surface manager (surface manager), media library (media library), 3D graphics processing library (eg: OpenGL ES), 2D graphics engine (eg: SGL), etc.
  • surface manager surface manager
  • media library media library
  • 3D graphics processing library eg: OpenGL ES
  • 2D graphics engine eg: SGL
  • the Surface Manager is used to manage the display subsystem and provides a fusion of 2D and 3D layers for multiple applications.
  • the media library supports playback and recording of a variety of commonly used audio and video formats, as well as still image files.
  • the media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.
  • the 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.
  • 2D graphics engine is a drawing engine for 2D drawing.
  • the kernel layer is the layer between hardware and software.
  • the kernel layer contains at least display drivers, camera drivers, audio drivers, and sensor drivers.
  • FIG. 3 is a set of graphical user interfaces (graphical user interface, GUI) provided by the embodiments of the present application.
  • the GUI is the desktop of the mobile phone.
  • the desktop of the mobile phone includes icons of multiple applications, the multiple applications include clock, calendar, gallery, memo, file management, e-mail, music, calculator and so on.
  • the GUI as shown in (b) of FIG. 3 may be displayed.
  • the GUI is the display interface of the memo application.
  • a note recorded on September 1 is displayed on the display interface, and the content of the note is "Mom's birthday (September 28)".
  • the display interface also includes a control 301 for creating a new note.
  • the mobile phone detects that the user clicks on the control 301, the mobile phone can display a GUI as shown in (c) of FIG. 3 .
  • GUI is another display interface of the memo application. Users can edit the title and body content of the note on this interface.
  • the display interface also includes controls for list, text editing, inserting pictures, audio-to-text, and handwriting controls.
  • the mobile phone when the mobile phone detects that the user clicks the operation of the recording-to-text control 302, the mobile phone can send request information to the notebook computer, and the request information is used to request the notebook computer to correspond to the voice content acquired by the mobile phone. Edit for text content.
  • the laptop computer In response to receiving the request information, the laptop computer can display a reminder box 303, wherein the reminder box 303 includes the prompt information "it is detected that the mobile phone is recording to text, whether to perform text editing on the laptop computer".
  • the notebook computer can start App1 in response to the user's operation of clicking on the control 304 .
  • App1 is an application that can do text editing.
  • the mobile phone can detect the voice content in the environment, and convert the detected voice content into text content. After the mobile phone converts the voice content into text content, the mobile phone can also send the converted text content to the laptop. Thereby, the display interface of the memo application of the mobile phone and the display interface of App1 of the notebook computer are simultaneously displayed with the text content "the popularity changes again... social mentality".
  • the mobile phone may include a microphone and a speech recognition (automatic speech recognition, ASR) module, wherein the microphone is used to collect speech content in the environment, and the role of the ASR module is to convert the received speech content into text content.
  • ASR automatic speech recognition
  • the text content on the display interface of App1 of the notebook computer and the display interface of the memo application of the mobile phone may be synchronized in real time.
  • the mobile phone can convert the voice content into text content at a certain time interval (for example, 5 seconds). In 0-5 seconds, the mobile phone converts the collected voice content into text content, "reflecting the demands of people's death while the popularity changes and remains unchanged", so that the mobile phone can display the text content on the display interface of the memo, and the mobile phone can display the text content at the same time.
  • the content is sent to the laptop, so that the laptop can display the text content on the display interface of App1; in 5-10 seconds, the mobile phone converts the collected voice content into text content.
  • the laptop computer can display the cursor 306 after opening App1, wherein after the laptop computer receives the text content sent by the mobile phone, the laptop computer can display the cursor 306 at the back of the text content.
  • the mobile phone can continue to detect the voice content and convert the voice content into text content.
  • the mobile phone can synchronize the text content to the App1 of the laptop in real time.
  • the notebook computer can edit the text content in App1 in response to the user's editing operation.
  • users can add content to the text content sent by the mobile phone (add “1,” before “Hotness", and “2,” before “Guangzhi Think Tank”).
  • the user can modify the inaccurate content of the text (for example, "name” is modified to "public opinion", and "breakthrough” is modified to "top10").
  • the user can also modify the format of the text content (for example, the laptop detects the user's carriage return operation after the "reputation appeal").
  • the notebook computer in response to the notebook computer detecting that the user clicks the save control 307, the notebook computer can send the edited text content to the mobile phone, so that the mobile phone displays the notebook computer on the display interface of the memo. Edited text content.
  • the mobile phone converts the voice content into text content after detecting the voice content in the environment, and sends the text content to the laptop computer, while the mobile phone may not display the text content. After the mobile phone receives the edited text content from the laptop computer, the mobile phone can display the edited text content on the display interface of the memo application.
  • the notebook computer can edit the text content near the position. For example, when the current cursor 306 is behind "Social Mindset”, when the laptop detects that the user has added a symbol "?” after "Social Mindset”, the cursor 306 can move to the back of "Social Mindset?". At the same time, after the notebook computer receives other text content sent by the mobile phone, it can continue to display the received text content.
  • the notebook computer when the notebook computer continues to detect the user's editing operation on the text content, it can edit the text content received from the mobile phone.
  • the notebook computer can detect the user's operation of changing "positive success” to "Zheng Chenggong”; for another example, the notebook computer can detect the user's operation of adding a symbol ".” after "annual feature”; another example, The laptop can detect the user inserting the symbol "," in “Medical Education Housing Food Safety". As another example, a laptop can detect the user's press enter after "Social Mindset?".
  • the notebook computer when the notebook computer detects that the user clicks the save control 307, the notebook computer can send the edited text content to the mobile phone, so that the mobile phone displays the notebook computer on the display interface of the memo Edited text content.
  • the GUI is another display interface of the memo application.
  • the mobile phone detects that the user clicks the control 308 to stop recording, the mobile phone can display a GUI as shown in (i) in FIG. 3 .
  • the GUI is another display interface of the memo application.
  • the user can continue to edit the text content edited by the laptop on the mobile phone (for example, the user can insert a picture into the text content).
  • the mobile phone detects that the user clicks on the control 309, the mobile phone can save the text content displayed on the current interface and display the GUI shown in (j) in FIG. 3 .
  • the GUI is another display interface of the memo application.
  • the mobile phone can update the title of the note to "1, hotspot".
  • the mobile phone can display the GUI as shown in (k) in Figure 3.
  • the GUI is another display interface of the memo application.
  • the mobile phone can display 2 notes included in the current memo application, and the newly added note content can be displayed at the top.
  • the mobile phone can start to convert audio to text, and the laptop computer can continue to display the prompt box 301 .
  • the notebook computer can send a response to the mobile phone, where the response is used to indicate that the notebook computer can perform text editing.
  • the mobile phone can send all the text content 1 that is voice-to-text before receiving the response to the laptop, and the laptop can open App1 and display the text content 1.
  • the mobile phone can continue to send the text content 2 corresponding to the voice content input by the user to the notebook computer.
  • the notebook computer can append the text content 2 sent by the mobile phone to the back of the text content 1.
  • the mobile phone when the mobile phone performs the recording-to-text operation, the mobile phone can notify the notebook computer that the recording-to-text function is being performed, so that the notebook computer can prompt the user whether to edit the text content on the notebook computer.
  • the laptop computer can display the text content sent by the mobile phone in real time, thereby facilitating the user to edit the text content and helping to improve the user experience.
  • FIG. 4 is another set of GUIs provided by the embodiments of the present application.
  • the mobile phone when the mobile phone detects that the user clicks the recording-to-text control, the mobile phone may display the interface shown in (b) in FIG. 4 .
  • the mobile phone may send request information to the notebook computer, where the request information is used to request the notebook computer to edit the text content corresponding to the voice content obtained by the mobile phone.
  • the notebook computer in response to receiving the request information sent by the mobile phone, the notebook computer can start App1.
  • the mobile phone may display a reminder box 401, wherein the reminder box 401 includes prompt information "detecting the surrounding laptop computer, whether to send the text content to the laptop computer for editing".
  • the mobile phone detects that the user clicks the control 402, the mobile phone starts to detect the voice content in the environment.
  • the mobile phone after detecting the voice content in the environment, the mobile phone can start to convert the recording into text.
  • the mobile phone can collect the voice content and convert the voice content into text content to "change the popularity...social mentality", and the mobile phone can also send the converted text content to the laptop, so that the laptop can display the text content through App1.
  • the notebook computer when the notebook computer detects that the user edits the text content on the notebook computer and detects that the user clicks the save operation, the notebook computer can send the edited text content to the mobile phone, so that the mobile phone can The edited text content is displayed on the display interface of the memo.
  • processes (f) to (g) in FIG. 3 which are not repeated here for brevity.
  • the mobile phone when the mobile phone performs the recording-to-text operation, if it is determined that there is a device (eg, a laptop) that is convenient for the user to edit text, the mobile phone can prompt the user whether to perform text editing on the laptop.
  • the laptop computer can display the text content sent by the mobile phone in real time, thereby facilitating the user to edit the text content and helping to improve the user experience.
  • FIG. 5 is another set of GUIs provided by the embodiments of the present application.
  • the GUI is the desktop of the mobile phone.
  • the desktop of the mobile phone includes icons of multiple applications, and the multiple applications include a voice recorder application and the like.
  • the GUI as shown in (b) of FIG. 5 can be displayed.
  • the GUI is the display interface of the recorder application.
  • a recording file recorded on October 3 is displayed on the display interface, and the recording file is named "20201003_100832".
  • the display interface also includes a control 501 for creating a new recording file.
  • the mobile phone when the mobile phone detects that the user clicks on the control 501, the mobile phone can send instruction information to the notebook computer, where the instruction information is used to indicate that the mobile phone is recording.
  • the laptop computer In response to receiving the instruction information, the laptop computer can display a reminder box 502, wherein the reminder box 502 includes a prompt message "It is detected that the mobile phone is recording, do you want to convert the recording to text on the laptop computer?".
  • the notebook computer in response to the notebook computer detecting that the user clicks on the control 503, the notebook computer can launch App1.
  • the mobile phone In response to the mobile phone detecting that the user clicks on the control 501, the mobile phone can display a recording interface, wherein the recording interface shows that 25 seconds have been recorded and the mobile phone is expected to record more than 10 hours.
  • the mobile phone When the mobile phone collects the voice content, it can also send the voice content to the laptop computer. After the laptop computer receives the voice content, it can convert the voice content into text content. The text content is displayed in App1.
  • an ASR module may be included in the notebook computer, wherein the function of the ASR module is to convert the received voice content into text content.
  • the notebook computer can edit the text content in App1 in response to the user's editing operation.
  • users can add content to the text content sent by the mobile phone (add “1,” before “Hotness”, and “2," before “Guangzhi Think Tank”).
  • the user can modify the inaccurate content of the text (for example, "name” is modified to "public opinion", and “breakthrough” is modified to "top10”).
  • the user can also modify the format of the text content (for example, the laptop detects the user's carriage return operation after the "reputation appeal").
  • the notebook computer when it detects that the user clicks the save control, it can send the edited text content to the mobile phone, so that the mobile phone can save the edited text content in a certain application layer sequence (eg, memo).
  • a certain application layer sequence eg, memo
  • the mobile phone when the mobile phone detects that the user starts recording, the mobile phone can send instruction information to the surrounding notebook computers, so that the notebook computer prompts the user whether the recording needs to be converted into text on the notebook computer.
  • the notebook computer prompts the user whether the recording needs to be converted into text on the notebook computer.
  • FIG. 6 is another set of GUIs provided by the embodiments of the present application.
  • the mobile phone displays a caller ID interface, and the caller ID interface shows that the mobile phone number "XXX-XXXXXXX" from Shenzhen, Guangdong is making a call.
  • the mobile phone sends indication information to the surrounding notebook computers, where the indication information is used to instruct the mobile phone to detect the user's incoming call.
  • the laptop computer can display a reminder box 601, wherein the reminder box 601 includes the prompt information "The mobile phone has received an incoming call, do you want to convert the content of the call to text on the laptop computer?".
  • the notebook computer in response to the notebook computer detecting that the user clicks on the control 602, the notebook computer can open App1 and send a response to the mobile phone, which is used to indicate that the notebook computer can be used as a text content editing device.
  • the mobile phone In response to the mobile phone detecting the operation of the user accepting the incoming call, the mobile phone can establish a call connection to the terminal device.
  • the mobile phone can receive the voice content of another user sent by the peer device.
  • the cell phone In response to receiving another user's voice content, the cell phone can send the voice content to the laptop.
  • the mobile phone may also send indication information to the notebook computer when detecting the operation of the user accepting the incoming call, the indication information is used to indicate that the mobile phone is on a call, and request the notebook computer to edit the text content corresponding to the call content.
  • the laptop can prompt the user "The mobile phone is on a call, do you want to convert the content of the call to text on the laptop?".
  • the notebook computer detects that the user determines to use the notebook computer to convert the call content to text, the notebook computer can open App1 and receive the voice content of another user from the mobile phone. Thus, the notebook computer can convert the voice content into text content.
  • the mobile phone after receiving the response, can convert the acquired voice content of another user into text content, so that the mobile phone can send the text content to the laptop.
  • the notebook computer in response to receiving the voice content from the mobile phone, can convert the voice content into text content "Hi, there is a meeting tomorrow morning...please attend on time", thereby displaying through App1 the text content.
  • the notebook computer when the notebook computer detects that the user edits the text content on the notebook computer and detects that the user clicks the save operation, the notebook computer can send the edited text content to the mobile phone, so that the mobile phone can The edited text content is displayed on the display interface of the memo.
  • processes (f) to (g) in FIG. 3 which are not repeated here for brevity.
  • the mobile phone when the mobile phone detects an incoming call, if it is determined that there is a device (for example, a laptop computer) that is convenient for the user to edit text nearby, the mobile phone can indicate to the laptop computer that the mobile phone has detected an incoming call.
  • the laptop can convert the voice content obtained from the mobile phone into text content in real time and display it to the user, so as to facilitate the user to edit the text content and help improve the user experience.
  • FIG. 7 is another set of GUIs provided by the embodiments of the present application.
  • the mobile phone displays a video call interface of a social application, and the video call interface shows that another user is inviting the user to make a video call.
  • the mobile phone sends indication information to the surrounding notebook computers, where the indication information is used to instruct the mobile phone to detect another user's video call request.
  • the laptop computer may display a prompt box 701, wherein the prompt box 701 includes prompt information "The mobile phone has received a video call request, whether to convert the call content to text on the laptop computer?".
  • the notebook computer in response to the notebook computer detecting that the user clicks on the control 702, the notebook computer can open App1 and send a response to the mobile phone, which is used to indicate that the notebook computer can be used as a text content editing device.
  • the mobile phone In response to the mobile phone detecting the user's operation of accepting the video call request, the mobile phone can establish a video call connection to the terminal device.
  • the mobile phone can receive the voice content of another user sent by the peer device.
  • the cell phone In response to receiving another user's voice content, the cell phone can send the voice content to the laptop.
  • the mobile phone after receiving the response, can convert the acquired voice content of another user into text content, so that the mobile phone can send the text content to the laptop.
  • the notebook computer in response to receiving the voice content from the mobile phone, can convert the voice content into text content "Hi, there is a meeting tomorrow morning...please attend on time", thereby displaying through App1 the text content.
  • the notebook computer when the notebook computer detects that the user edits the text content on the notebook computer and detects that the user clicks the save operation, the notebook computer can send the edited text content to the mobile phone, so that the mobile phone can The edited text content is displayed on the display interface of the memo.
  • processes (f) to (g) in FIG. 3 which are not repeated here for brevity.
  • the mobile phone when the mobile phone detects a video call, if it is determined that there is a device (for example, a laptop computer) that is convenient for the user to edit text nearby, the mobile phone can indicate to the laptop computer that the mobile phone detects an incoming call.
  • the laptop can convert the voice content obtained from the mobile phone into text content in real time and display it to the user, so as to facilitate the user to edit the text content and help improve the user experience.
  • the mobile phone when the mobile phone receives an incoming call and a video call request, the mobile phone indicates the surrounding laptops, so that the laptop prompts the user whether to convert the call content to text on the laptop.
  • the mobile phone may also instruct the laptop computer after receiving a voice call request from another user, so that the laptop computer prompts the user whether to convert the call content to text on the laptop computer.
  • FIG. 8 is another set of GUIs provided by the embodiments of the present application.
  • the mobile phone displays a caller ID interface, and the caller ID interface displays that the mobile phone number "XXX-XXXXXXX" from Shenzhen, Guangdong is making a call.
  • the mobile phone sends indication information to the surrounding notebook computers, where the indication information is used to instruct the mobile phone to detect the user's incoming call.
  • the laptop computer can display a reminder box 801, wherein the reminder box 801 includes the prompt information "The mobile phone has received an incoming call, do you want to convert the content of the call to text on the laptop computer?".
  • the reminder box 801 can be hidden.
  • the mobile phone can establish a call connection to the terminal device.
  • the user detects that the user clicks the operation of the recording control 803, and can send instruction information to the notebook computer, and the instruction information is used to indicate that the mobile phone is making a call, Request the laptop to edit the text content corresponding to the call content.
  • the notebook computer may display the reminder box 801 again.
  • the laptop computer detects that the user clicks on the control 802
  • the laptop computer can send a response to the mobile phone, where the response is used to instruct the laptop computer to edit the text content corresponding to the call content.
  • the cell phone can send the call content to the laptop.
  • the ASR module of the notebook computer in response to receiving the call content sent by the mobile phone, can convert the call content into text content “Hi, tomorrow morning... join” and display it in App1.
  • the laptop detects the user's editing operation, can edit the text content, and displays the edited text content in App1 ("Leader: Hi, tomorrow morning... see”).
  • the mobile phone when the mobile phone detects that the user clicks the recording operation during the process of answering the call, the mobile phone sends instruction information to the notebook computer, thereby instructing the notebook computer to edit the text content corresponding to the call content.
  • the laptop can convert the voice content obtained from the mobile phone into text content in real time and display it to the user, so as to facilitate the user to edit the text content and help improve the user experience.
  • FIG. 9 is another set of GUIs provided by the embodiments of the present application.
  • the mobile phone can display the recording-to-text interface in the memo, and the mobile phone detects the voice content input by the user "reflecting the people's livelihood demands between the change of popularity and the constant change", and the mobile phone can convert the voice content to the voice content. Convert to text content and send that text content to the laptop.
  • the notebook computer can display the text content to the user by means of subtitles.
  • the notebook computer can display the text content "reflecting the demands of people's livelihood between the change of popularity and the constant change" in the subtitle display area 901 .
  • the mobile phone when the mobile phone detects the voice content "We have just obtained statistics from the Guangzhi Think Tank” input by the user, the mobile phone can convert the voice content into text content and send the text content to the laptop.
  • the notebook computer After receiving the text content, the notebook computer can display the text content to the user by means of subtitles.
  • the notebook computer can display the text content "We have just obtained statistics on Guangzhi Think Tank” in the subtitle display area 901 .
  • FIG. 10 is another set of GUIs provided by the embodiments of the present application.
  • the mobile phone can display the recording-to-text interface in the memo, and the mobile phone detects the voice content input by the user "reflecting the people's death demands between the hotness and the constant change", and the mobile phone can convert the voice content to the voice content. Convert to text content and send that text content to the laptop.
  • the notebook computer can display the text content to the user by means of subtitles, and at the same time, the notebook computer can also open App1 and display the text content in App1.
  • the laptop computer can display the text content "Reflecting the demands of people's livelihood between the change of popularity and the constant change" in the subtitle display area 1001, and simultaneously display the text content in App1.
  • the user can edit the text content in App1.
  • the mobile phone when the mobile phone detects the voice content "We have just obtained statistics from the Guangzhi Think Tank" input by the user, the mobile phone can convert the voice content into text content and send the text content to the laptop.
  • the notebook computer After the notebook computer receives the text content, it can display the text content to the user through subtitles. At the same time, the notebook computer can also display the text in App1 after "reflecting the demands of people's death between the hotness and the constant change". content.
  • the notebook computer can display the text content "We have just obtained the statistics of Guangzhi Think Tank” in the subtitle display area 1001, and display in App1 "The demand for people's death is reflected in the change of popularity and the constant change, we have just calculated the Guangzhi Think Tank. get”.
  • the system architecture includes device A and device B, where device A may be a device with a pickup function (eg, a mobile phone), and device B may be a device with an efficient editing function (eg, a laptop).
  • Device A includes a pickup module 1110 , an ASR module 1120 , a transceiver control module 1130 and a replacement module 1140 ;
  • device B includes a transceiver control module 1150 , a display unit 1160 and an editing control module 1170 .
  • the sound pickup module 1110 is used to acquire the voice content to be processed.
  • the manner in which the device A acquires the voice content is not specifically limited, and the acquisition manner may be various.
  • it can be real-time voice recorded from the surrounding environment of device A; or, it can also be the audio of the audio and video played by the user on the mobile phone; or, it can also be the audio (including remote and Near-end); it can also be the audio and video files in the mobile phone.
  • the pickup module is responsible for converting the relevant audio into a specific audio format, such as a pulse code modulation (PCM) audio stream of a certain sampling rate, which is used as an input to the ASR module.
  • PCM pulse code modulation
  • the ASR module 1120 may convert the speech content to textual content.
  • the deep neural network acoustic model is trained to obtain the phoneme sequence with the highest probability, and then through the corresponding language model, the text series with the highest probability is obtained, completing the process from speech content to text. content conversion.
  • the transceiver control module 1130 is used to control information synchronization between device A and other devices.
  • the transceiver control module 1130 may be responsible for transparent transmission of information, and the transceiver control module 1130 is not involved in the logical processing of the transmission content.
  • the transmission of information may be through a local area network, such as Bluetooth, Wi-Fi, or other transmission protocols (eg, the internet).
  • Device A establishes a connection network link with other devices (eg, device B) through a network communication protocol, and then initiates a start transmission.
  • transceiver control module 1150 For the functional description of the transceiver control module 1150, reference may be made to the aforementioned transceiver control module 1130, which is not repeated here for brevity.
  • the display unit 1160 is used to prompt the user whether to start text editing on the device B.
  • the transceiver control module 1150 receives the instruction sent by the transceiver control module 1130 for device A to acquire audio content (eg, voice content or audio files), or, the transceiver control module 1150 receives the instruction sent by the transceiver control module 1130 to perform text editing on device B.
  • a prompt can be popped up in the form of a notification through the display unit 1160 .
  • the device B If the device B detects that the user determines to perform text editing through the device B, it can send a response to the transceiver control module 1130 through the transceiver control module 1150, and the response can be used to indicate that the device B can be used as a text editing device. After the transceiver control module 1130 of device A receives the response, device A starts to transmit the ASR result to device B in real time.
  • the transceiver control module 1130 may send a query request, where the query request is used to query surrounding devices with text editing functions.
  • device A may store information such as device type, device name, and MAC address of device B.
  • device A can send BLE data packets to device B according to the MAC address of device B.
  • the BLE data packet may include a PDU, and the query request may be carried in a service data field in the PDU, or may also be carried in a vendor-specific data field in the PDU.
  • the payload of the service data field may include multiple bits, wherein the multiple bits include scalable bits.
  • Device A and Device B can agree on the content of a certain extensible bit. When a certain extensible bit is 1, device B can obtain device A and query whether device B has the text editing function.
  • the mobile phone when the mobile phone detects that the user clicks on the control 501 (the control 501 is the control for starting recording), the mobile phone can send a BLE data packet, and the BLE data packet can carry the query ask.
  • device A when device A detects that the user performs an operation of converting speech to text, device A can send a BLE data packet to device B according to the MAC address of device B.
  • the mobile phone when the mobile phone detects that the user clicks on the control 302 (the control 302 is a control for the mobile phone to perform voice-to-text conversion), the mobile phone can send a BLE data packet, and the BLE data packet can carry the query request.
  • the control 302 is a control for the mobile phone to perform voice-to-text conversion
  • the transceiver control module of device B can call the interface for querying the text editing function (for example, the content provider interface) to send a request to one or more applications in the application layer, and the request is used to request the application to determine whether it has the text editing function; If an application program has a text editing function, the application program can send a response to the data synchronization module, and the response is used to indicate that app2 has logged into the account. Therefore, device B can determine that device B has installed and logged in app2.
  • the text editing function for example, the content provider interface
  • device B can send a response to device A, and the response can be carried in a BLE data packet.
  • the BLE data packet may include a PDU, and the query request may be carried in a service data field in the PDU, or may also be carried in a vendor-specific data field in the PDU.
  • the payload of the service data field may include multiple bits, wherein the multiple bits include scalable bits.
  • Device A and Device B can agree on the content of a certain extensible bit. When a certain extensible bit is 1, device A can obtain the text clipping function of device B.
  • device A After device A determines that device B has the text editing function, device A can send the acquired text content and instruction information corresponding to the audio content to device B, where the instruction information is used to instruct device B to edit the text content.
  • device A may send a BLE data packet to device B according to the MAC address of device B.
  • the BLE data packet may include a PDU, and the text content and indication information may be carried in a service data field in the PDU, or may also be carried in a vendor-specific data field in the PDU.
  • the payload of the service data field may include multiple bits, wherein the multiple bits include scalable bits.
  • Device A and Device B can agree on the content of a certain extensible bit.
  • Device A can use encoding methods such as GBK, ISO8859-1 or Unicode (for example, UTF-8, UTF-16) to encode the text content output by the ASR module of device A, and carry the encoded information in an or on multiple scalable bits.
  • Device A may also set an extensible bit to 1.
  • the device B After receiving the BLE data packet, the device B can obtain the text content and the indication information by decoding, so that the device B can display the text content according to the indication information.
  • device A may send text content corresponding to the acquired audio content to device B.
  • device A may send a BLE data packet to device B according to the MAC address of device B.
  • the BLE data packet may include a PDU, and the text content may be carried in a service data field in the PDU, or may also be carried in a vendor-specific data field in the PDU.
  • the payload of the service data field may include multiple bits, wherein the multiple bits include scalable bits.
  • Device A and Device B can agree on the content of a certain extensible bit.
  • Device A can use encoding methods such as GBK, ISO8859-1 or Unicode (for example, UTF-8, UTF-16) to encode the text content output by the ASR module of device A, and carry the encoded information in an or on multiple scalable bits.
  • device B After receiving the BLE data packet, device B can obtain the text content by decoding, so that device B can display the text content.
  • device A and device B are devices under the same account, then device A can store information such as device type, device name, MAC address, and whether it has a text editing function of device B. Then, when device A acquires the audio content, device A can send the text content and indication information corresponding to the audio content to device B, where the indication information is used to instruct device B to edit the text content. In response to receiving the text content and the indication information, the device B may prompt the user whether to perform text editing on the device B through the display unit 1160 . In response to detecting that the user performs text editing on the device B, the device B can open an application that can be used for text editing, so as to display the text content obtained from the device A.
  • device A may send the text content corresponding to the audio content to device B.
  • the device B can edit the text content on the device B, so that the device B can prompt the user whether to perform text editing on the device B through the display unit 1160 .
  • the device B can open an application that can be used for text editing, so as to display the text content obtained from the device A.
  • the display unit 1160 can also be used to display the text content output by the ASR module (including the intermediate result and the final result); the display unit 1160 can also be used to display the text content edited by the user on the device B.
  • the transceiver control module 1150 of the device B can append the text content to the back of the previously displayed text content.
  • the transceiver control module 1150 of device B can call the QT interface and perform the following steps: (1) select all the text content displayed by App1; ( 2) Move the cursor to the end of the text; (3) Insert the new text content sent by device A; (4) Save all the text content in the current App1.
  • the editing control module 1170 is used for saving, editing and displaying control of the received information.
  • the results sent to device B through the ASR module 1120 may include two types:
  • the first type is an intermediate result. For example, before a sentence is finished, the text content determined by the ASR module 1120 is not finalized and can be used as an intermediate result. In order to reflect the real-time performance, the intermediate results are also displayed, but are not saved as the final results.
  • the ASR module of device A detects that every time the user speaks a word, it sends a corresponding word to device B, so that text content can be displayed on device A and device B synchronously. For example, the user says “I am XX", when device A detects that the user has said "I”, the ASR module of device A can send the text content ("I” to device B after determining the corresponding text "I” ), so that device B can display the text content "I”. When device A detects that the user has said "yes”, the ASR module of device A can send the text content ("yes") to device B after determining the corresponding text "yes”, so that device B can Append the text "Yes” after "I” until "I am XX” is displayed.
  • the ASR module of device A detects that every time the user speaks a phrase, it sends a corresponding phrase to device B, so that text content can be displayed on device A and device B synchronously. For example, the user says “no hardship can stop us from moving forward", when device A detects that the user has said "any", the ASR module of device A can send the corresponding word "any” to device B after determining the corresponding word "any” The text content ("any"), so that device B can display the text content "any”.
  • the ASR module of device A can send the text content ("difficult") to device B after determining the corresponding word "difficult", so that device B can Append the text "difficulty” after any” until it shows "no hardship can stop us from moving forward".
  • the content displayed by device A can be synchronized with the content displayed on device B, and device A may correct the text content previously converted by the ASR module when judging that the user has finished speaking.
  • the ASR module of device A detects that every time the user speaks a word or a phrase, it sends the corresponding text content to device B.
  • device A detects that the user says " When "thinking”, it is possible that the ASR module determines that the corresponding text is "xiang”, then device A will send the corresponding text content (“xiang”) to device B, and device B can display the text content "xiang”.
  • device A After judging that the user has finished saying this sentence, device A can determine that there is an error in the conversion of "Xiang” in the previously converted text content "Today I'm eating noodles", then device A can automatically correct the "Xiang” to "Xiang”. I want to”, thereby updating the text content spoken by the user (from "Today I want to eat noodles” to "Today I want to eat noodles”). Device A can send the updated text content to device B, so that device B can also update "Xiang” in the previously displayed text content "Today I want to eat noodles” to "Want", thereby displaying the updated text content "Today I want to eat noodles”.
  • the second is that after a sentence is spoken, the recognition result of the sentence determined by the ASR module 1120 will not be changed back in the future, and such a result will replace the previous intermediate result and display it as the saved result.
  • the ASR module of device A can convert each word or phrase spoken by the user into text content, but device A may not send text content to device B before judging that the user has not finished a sentence. Instead, when device A determines that the user has finished speaking a sentence and corrects the previously converted text content, it sends the text content corresponding to the sentence to device B.
  • the ASR module of device A can convert each word or phrase spoken by the user into text content and send it to device B.
  • Device B can display the text content sent by device A in the form of subtitles, but the device B may not display the text content in app1.
  • device A judges that the user has finished speaking a sentence (and corrects the previously converted text content)
  • device A can send the text content corresponding to the sentence and indication information to device B, and the indication information instructs device B to display in app1
  • the text content corresponding to the sentence After receiving the text content and the instruction information, device B can display the text content corresponding to the sentence in app1.
  • the editing control module 1170 will also re-save the edited result and display it, and the subsequent ASR The result is appended on top of that.
  • the editing control module 1170 is further configured to send the text content edited by the user to the sending and receiving control module 1150 , so that the sending and receiving control module 1150 sends the edited text content to the sending and receiving control module 1130 .
  • the replacement module 1140 is configured to replace the original displayed text content with the edited text content after receiving the edited text content sent by the transceiver control module 1130 .
  • the device A when the recording ends or the audio acquisition ends, the device A may indicate the end of the recording or the audio acquisition to the device B. After the user finishes editing on device B, the editing result can be synchronized to device A at one time, and the entire synchronization process is completed.
  • FIG. 12 shows a schematic flowchart of a text editing method 1200 according to an embodiment of the present application.
  • the method 1200 can be implemented by the above-mentioned device A and device B, and the method 1200 includes:
  • S1201 device A acquires audio content.
  • the mobile phone when the mobile phone detects the user's operation of accepting the incoming call, the mobile phone can acquire the voice content of another user.
  • the mobile phone when the mobile phone detects that the user accepts the operation of the video call request, the mobile phone can acquire the voice content of another user.
  • device A sends first information to device B according to the audio content.
  • the first information is text content corresponding to the audio content.
  • device A may first convert the voice content into text content through the ASR module, so as to send the text content to device B.
  • ASR module For the process of converting the voice content into text content by the device A, reference may be made to the descriptions in the foregoing embodiments, which are not repeated here for brevity.
  • the method before device A sends the voice content to device B, the method further includes:
  • Device A sends a query request to request the device that received the query request to determine whether it has a text editing function
  • device B In response to receiving the request information sent by device A, device B sends a response to device A, where the response is used to indicate that device B has a text editing function;
  • device A In response to receiving the response, device A sends device B the first information.
  • device A broadcasts the query request to surrounding devices.
  • device A may store device information of device B (eg, device A and device B are devices under the same account, or device A and device B are devices under different accounts in the same family group).
  • device B in response to receiving the query request, may prompt the user whether to perform text editing on device B; in response to the user determining to perform text editing on device B, device B sends the response to device A.
  • the laptop computer when the laptop computer detects the query request sent by the mobile phone, the laptop computer can display a prompt box 303, which includes the prompt message “It is detected that the mobile phone is recording and transferring. text, whether text editing is done on a laptop". In response to the user clicking on control 304, the laptop can send the response to the phone.
  • device A may send request information to device A, where the request information is used to request device B to edit the text content output by device A; Request device B to edit the text content corresponding to the audio content.
  • the mobile phone when the mobile phone detects that the user clicks on the control 302, the mobile phone can send a query request to the surrounding devices, and the query request is used to query whether the surrounding devices have a text editing function; In response to receiving the response sent by the laptop, the mobile phone can determine that the laptop has text editing capabilities. The mobile phone can continue to send request information to the notebook computer, and the request information is used to edit the text content output by the mobile phone. In response to receiving the request information from the cell phone, the laptop computer may display a prompt box 303 . In response to the user clicking on the control 304, the notebook computer can send a response to the mobile phone, the response being used to indicate that the notebook computer can edit the text content. In response to receiving the response, the cell phone can send the textual content to the laptop.
  • the mobile phone when the mobile phone detects the operation of the user clicking on the control 501, it sends a query request to the surrounding devices; in response to receiving the response sent by the notebook computer, the mobile phone can determine that the notebook computer has text Edit function.
  • the mobile phone can send request information to the laptop computer, where the request information is used to request the laptop computer to edit the text content corresponding to the audio content acquired by the mobile phone.
  • the laptop computer may display a prompt box 502, wherein the prompt box 502 includes prompt information "it is detected that the mobile phone is recording, whether to convert the recording to text on the laptop computer".
  • the notebook computer can send a response to the mobile phone, where the response is used to instruct the notebook computer to edit the text content corresponding to the recording.
  • the cell phone can send the textual content to the laptop.
  • device B can send the response information to device A through a BLE data packet, and the specific sending process can refer to the above-mentioned process of device A sending a BLE data packet to device B, which is not repeated here for brevity.
  • the request information may be carried in a user datagram protocol (user datagram protocol, UDP) data packet.
  • UPD packet includes the data portion of the IP datagram.
  • the data portion of an IP datagram may include extensible bits.
  • Device A and Device B can agree on the content of a certain extensible bit. When a certain extensible bit is 1, device B can know that device A requests to edit the text content.
  • the UDP data packet can also carry the IP address and port number of device A (including source port number and destination port number, where the source port number refers to the port number used by device A to send data, and the destination port number refers to the port number that device A receives The port used by the data), the IP address and port number of device A can be carried in the UDP header of the data part of the IP datagram.
  • device B can establish a transmission control protocol (transmission control protocol, TCP) connection with device A.
  • TCP transmission control protocol
  • device B can send the response to device A through the TCP connection.
  • the UDP data packet carries the IP address and destination port number of device A.
  • Device B can establish a TCP connection with device A through the IP address and destination port number. After that, device A can send the text content input by the ASR module of device A to device B through the TCP connection.
  • the method 1200 further includes: when the device A converts the voice content into text content, the device A displays the text content.
  • the mobile phone after detecting the voice content in the environment, the mobile phone can convert the voice content into text content, so as to display the text content through the display interface of the memo application.
  • S1203 Device B displays the text content corresponding to the audio content according to the first information.
  • the notebook computer after receiving the text content sent by the mobile phone, the notebook computer can display the text content through App1.
  • the notebook computer may display the edited text content.
  • the method 1200 further includes: device B sending the edited text content to device A.
  • device B detects the user's first operation, and sends the edited text content to device A.
  • the notebook computer when the notebook computer detects that the user clicks on the control 307, the notebook computer can send the edited text content to the mobile phone.
  • the notebook computer when the notebook computer detects that the user clicks on the control 307, the notebook computer can send the edited text content to the mobile phone.
  • device A may use the edited text content to edit the previously displayed text content.
  • the notebook computer may send the edited text content to the mobile phone.
  • the phone can replace the previously displayed text content with the text content edited by the user.
  • device B when device B detects an editing operation by the user on the text content sent by device A, it can edit the text content accordingly.
  • the notebook computer when the notebook computer detects that the user has edited the text content, it can edit the text content received from the mobile phone.
  • the notebook computer when the notebook computer can detect the user's operation of changing "positive success” to "Zheng Chenggong”; for another example, the notebook computer can detect the user's operation of adding a symbol ".” after "annual feature”; another example, The laptop can detect the user inserting the symbol "," in "Medical Education Housing Food Safety". As another example, a laptop can detect the user's press enter after "Social Mindset?".
  • Device B can send the edited text content to device A through a BLE data packet or through a TCP connection.
  • a BLE data packet or through a TCP connection.
  • TCP connection For the sending method, reference may be made to the description in the above embodiment, which is not repeated here for brevity.
  • device B if device B detects that the user has edited the format of the text content, when device B sends the edited text content to device A, it can also indicate the format information of the text content to device A.
  • the format of the text content may include a line feed (or carriage return) operation between certain two words in the text content; or, there is a space between certain two words.
  • the notebook computer while sending the edited text content to the mobile phone, the notebook computer can also indicate to the mobile phone that there is a line break between "Q" and "2", and "?” There is a newline between "3".
  • device B may send the edited text content and format information of the text content to device A.
  • the implementation manner of the device B sending the edited text content to the device A may refer to the description in the foregoing embodiment, which is not repeated here for brevity.
  • the following describes an implementation manner in which device B sends the format information of the edited text content to device A.
  • the text content format includes font size, font color, font inclination, font underline, font background color, and a carriage return operation after a certain text in this content.
  • device A may send the format information of the edited text content to device B through a BLE data packet.
  • This textual content may be carried in a service data field or a vendor specific data field in the PDU.
  • the payload of the service data field may include multiple bits, wherein the multiple bits include scalable bits.
  • device A may agree with device B on the content of some extensible bits. Exemplarily, when a certain extensible bit is 000, the device A can know that the character is not oblique, there is no underscore, and there is no carriage return operation after the character.
  • a certain extensible bit when a certain extensible bit is 100, device A can learn that the character is oblique, has no underline, and there is no carriage return operation after the character.
  • a certain extensible bit when a certain extensible bit is 010, device A can know that the character is not oblique, has an underline, and there is no carriage return operation after the character.
  • a certain extensible bit when a certain extensible bit is 001, device A can know that the character is not slanted, has no underscore, and there is a carriage return operation after the character.
  • device A may agree with device B on the content of some extensible bits.
  • a certain expandable bit is 000
  • device A can learn that the font color of the character is black.
  • a certain extensible bit is 001
  • device A can learn that the font color of the character is gray.
  • a certain expandable bit is 010
  • device A can know that the color of the character is blue.
  • a certain expandable bit is 100
  • device A can learn that the font color of the character is blue.
  • device A may agree with device B on the content of some extensible bits.
  • a certain expandable bit is 000
  • device A can learn that the font size of the character is 10.
  • a certain extensible bit is 001
  • device A can learn that the font size of the character is 12.
  • a certain extensible bit is 010
  • device A can learn that the font size of the character is 14.
  • a certain expandable bit is 100
  • device A can learn that the font size of the character is 18.
  • the format of the text content is not specifically limited in the embodiment of the present application.
  • the device B After receiving the format information of the edited text content, the device B can to display the edited text content.
  • the text content displayed on device B corresponds to the text content displayed on device A.
  • the method 1200 further includes: device A replaces the original text content with the edited text content received from device B; in response to receiving the edited text content from device B, device A displays a second interface , the second interface includes the edited text content.
  • the notebook computer sends the edited text content to the mobile phone in response to the operation of the user clicking the save control 307 .
  • the mobile phone can replace the text content displayed on the original interface with the edited text content.
  • device B when device B sends the edited text content to device A, it may also send the identification information of the edited text content to device A.
  • S1204 is an optional step, and device B may also not send the edited text content to device A. Instead, save the edited text content locally on device B.
  • FIG. 13 shows another system architecture provided by an embodiment of the present application.
  • the system architecture includes device A and device B, where device A may be a device with a pickup function (eg, a mobile phone), and device B may be a device with an efficient editing function (eg, a laptop).
  • Device A includes a pickup module 1310 and a transceiver control module 1320 ;
  • device B includes a receive transceiver control module 1330 , an ASR module 1340 , a display module 1350 and an editing control module 1360 .
  • the ASR module can be included in device B.
  • Device A can be responsible for acquiring voice content or audio files, and device A sends the acquired voice content or audio files to the transceiver control module 1330 of device B through the transceiver control module 1320, so that the ASR module 1340 of device B sends the voice content or audio files to the device B. Converted to text content, so that device B can display the converted text content to the user. Device B can also display the edited text content through the display module 1340 after detecting an editing operation of the text content by the user through the editing control module 1360 .
  • FIG. 14 shows a schematic block diagram of an apparatus 1400 provided by an embodiment of the present application.
  • the apparatus 1400 can be set in the above-mentioned device A in FIG. 12 , and the apparatus 1400 includes: an acquiring unit 1410 for acquiring audio content; a sending unit 1420 for transmitting first information to the second electronic device, the first The information is the audio content, or the first information is the first text content corresponding to the audio content, so that the second electronic device displays the first text content according to the first information and detects the user's first text content. edit operation.
  • FIG. 15 shows a schematic block diagram of an apparatus 1200 provided by an embodiment of the present application.
  • the apparatus 1500 can be set in the above-mentioned device B in FIG. 12 , and the apparatus 1500 includes: a receiving unit 1510 for receiving first information sent by a first electronic device, where the first information is acquired by the first electronic device audio content, or the first information is the first text content corresponding to the audio content; the display unit 1520 is used to display the first text content according to the first information; the detection unit 1530 is used to detect the user's An editing operation of text content; the display unit 1520 is further configured to display the second text content, where the second text content is the edited text content of the first text content.
  • FIG. 16 shows a schematic structural diagram of an electronic device 1600 provided by an embodiment of the present application.
  • the electronic device includes: one or more processors 1610, one or more memories 1620, and the one or more memories 1620 stores one or more computer programs, and the one or more computer programs include instruction.
  • the instruction is executed by the one or more processors 1610, the first electronic device or the second electronic device is caused to execute the technical solutions in the foregoing embodiments.
  • An embodiment of the present application provides a system including a device A and a device B, and the system is used to implement the technical solutions in the foregoing embodiments.
  • the implementation principle and technical effect thereof are similar to the related embodiments of the above method, and are not repeated here.
  • the embodiments of the present application provide a computer program product, which enables the device A to execute the technical solutions in the foregoing embodiments when the computer program product runs on the device A (or the mobile phone in the foregoing embodiment).
  • the implementation principle and technical effect thereof are similar to the related embodiments of the above method, and are not repeated here.
  • An embodiment of the present application provides a computer program product, which enables the device B to execute the technical solutions in the foregoing embodiments when the computer program product runs on the device B (or the notebook computer in the foregoing embodiment).
  • the implementation principle and technical effect thereof are similar to the related embodiments of the above method, and are not repeated here.
  • An embodiment of the present application provides a readable storage medium, where the readable storage medium contains instructions, when the instructions are executed on device A (or, the mobile phone in the above embodiment), the device A is caused to execute the above embodiment technical solution.
  • the implementation principle and technical effect thereof are similar, and are not repeated here.
  • An embodiment of the present application provides a readable storage medium, where the readable storage medium contains instructions, when the instructions are executed on a device B (or a notebook computer in the above embodiment), the device B is caused to perform the above implementation.
  • a device B or a notebook computer in the above embodiment
  • the device B is caused to perform the above implementation.
  • example technical solution The implementation principle and technical effect thereof are similar, and are not repeated here.
  • An embodiment of the present application provides a chip, which is used for executing an instruction, and when the chip is running, executes the technical solutions in the foregoing embodiments.
  • the implementation principle and technical effect thereof are similar, and are not repeated here.
  • the disclosed system, apparatus and method may be implemented in other manners.
  • the apparatus embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented.
  • the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the functions, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium.
  • the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution.
  • the computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program codes .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • User Interface Of Digital Computer (AREA)
  • Telephone Function (AREA)

Abstract

La présente invention concerne un procédé d'édition de texte, un dispositif électronique (100, 1600) et un système. Le procédé comprend les étapes suivantes : un dispositif A acquiert un contenu audio (S1201); le dispositif A envoie des premières informations à un dispositif B (S1202), les premières informations étant le contenu audio ou les premières informations étant un premier contenu de texte correspondant au contenu audio; le dispositif B affiche le premier contenu de texte selon les premières informations (S1203); et le dispositif B affiche un second contenu de texte à la suite d'une opération d'édition d'un utilisateur sur le premier contenu de texte (S1204), le second contenu de texte étant un contenu de texte obtenu après que le premier contenu de texte a été édité (S1205). La présente invention permet d'améliorer l'efficacité d'édition d'un utilisateur pour un contenu de texte.
PCT/CN2021/138655 2020-12-22 2021-12-16 Procédé d'édition de texte, dispositif électronique et système WO2022135254A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011529621.0A CN114664306A (zh) 2020-12-22 2020-12-22 一种编辑文本的方法、电子设备和系统
CN202011529621.0 2020-12-22

Publications (1)

Publication Number Publication Date
WO2022135254A1 true WO2022135254A1 (fr) 2022-06-30

Family

ID=82025546

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/138655 WO2022135254A1 (fr) 2020-12-22 2021-12-16 Procédé d'édition de texte, dispositif électronique et système

Country Status (2)

Country Link
CN (1) CN114664306A (fr)
WO (1) WO2022135254A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116737049B (zh) * 2022-11-22 2024-04-19 荣耀终端有限公司 音频播放方法及终端设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030055655A1 (en) * 1999-07-17 2003-03-20 Suominen Edwin A. Text processing system
CN103959282A (zh) * 2011-09-28 2014-07-30 谷歌公司 用于文本识别系统的选择性反馈
CN105159870A (zh) * 2015-06-26 2015-12-16 徐信 一种精准完成连续自然语音文本化的处理系统及方法
US20160125881A1 (en) * 2014-06-30 2016-05-05 Nuance Communications, Inc. Mobile Device for Speech Input and Text Delivery
CN109599115A (zh) * 2018-12-24 2019-04-09 苏州思必驰信息科技有限公司 用于音频采集设备和用户终端的会议记录方法和装置
CN109782997A (zh) * 2019-01-25 2019-05-21 腾讯科技（深圳）有限公司 一种数据处理方法、装置及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030055655A1 (en) * 1999-07-17 2003-03-20 Suominen Edwin A. Text processing system
CN103959282A (zh) * 2011-09-28 2014-07-30 谷歌公司 用于文本识别系统的选择性反馈
US20160125881A1 (en) * 2014-06-30 2016-05-05 Nuance Communications, Inc. Mobile Device for Speech Input and Text Delivery
CN105159870A (zh) * 2015-06-26 2015-12-16 徐信 一种精准完成连续自然语音文本化的处理系统及方法
CN109599115A (zh) * 2018-12-24 2019-04-09 苏州思必驰信息科技有限公司 用于音频采集设备和用户终端的会议记录方法和装置
CN109782997A (zh) * 2019-01-25 2019-05-21 腾讯科技（深圳）有限公司 一种数据处理方法、装置及存储介质

Also Published As

Publication number Publication date
CN114664306A (zh) 2022-06-24

Similar Documents

Publication Publication Date Title
WO2020253719A1 (fr) Procédé de d'enregistrement d'écran et dispositif électronique
WO2021000807A1 (fr) Procédé et appareil de traitement pour un scénario d'attente dans une application
WO2020078337A1 (fr) Procédé de traduction et dispositif électronique
CN109286725B (zh) 翻译方法及终端
KR102558615B1 (ko) 착신 호출이 있는 경우에 전자 디바이스 상에 비디오를 제시하기 위한 방법 및 전자 디바이스
WO2022037407A1 (fr) Procédé et système de réponse à un message, et dispositif électronique
CN112119641B (zh) 通过转发模式连接的多tws耳机实现自动翻译的方法及装置
EP3826280B1 (fr) Procédé de génération d'instruction de commande vocale et terminal
WO2020224447A1 (fr) Procédé et système pour ajouter un dispositif de maison intelligente à des contacts
WO2022042770A1 (fr) Procédé de commande d'état de service de communication, dispositif terminal et support de stockage lisible
EP4293997A1 (fr) Procédé d'affichage, dispositif électronique et système
WO2022135157A1 (fr) Procédé et appareil d'affichage de page, ainsi que dispositif électronique et support de stockage lisible
CN114115770A (zh) 显示控制的方法及相关装置
CN114827581A (zh) 同步时延测量方法、内容同步方法、终端设备及存储介质
WO2022206763A1 (fr) Procédé d'affichage, dispositif électronique et système
WO2022161006A1 (fr) Procédé et appareil de synthèse de photographie, et dispositif électronique et support de stockage lisible
WO2022143258A1 (fr) Procédé de traitement d'interaction vocale et appareil associé
WO2022135254A1 (fr) Procédé d'édition de texte, dispositif électronique et système
WO2022032979A1 (fr) Procédé d'appel de capacités d'autres dispositifs, dispositif électronique et système
WO2021052388A1 (fr) Procédé de communication vidéo et appareil de communication vidéo
WO2022135273A1 (fr) Procédé permettant d'invoquer des capacités d'autres dispositifs, dispositif électronique et système
WO2022007757A1 (fr) Procédé d'enregistrement d'empreinte vocale inter-appareils, dispositif électronique et support de stockage
WO2022062902A1 (fr) Procédé de transfert de fichier et dispositif électronique
WO2022052767A1 (fr) Procédé de commande de dispositif, dispositif électronique et système
WO2022206769A1 (fr) Procédé de combinaison de contenu, dispositif électronique et système

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21909250

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21909250

Country of ref document: EP

Kind code of ref document: A1