WO2020078330A1

WO2020078330A1 - Voice call-based translation method and electronic device

Info

Publication number: WO2020078330A1
Application number: PCT/CN2019/111120
Authority: WO
Inventors: 张鑫
Original assignee: 华为技术有限公司
Priority date: 2018-10-15
Filing date: 2019-10-15
Publication date: 2020-04-23
Also published as: CN109582976A

Abstract

A voice call-based translation method and an electronic device, which relate to the technical field of terminals, the method comprising: a first electronic device establishing a voice call connection with a second electronic device; when a voice call translation function of the first electronic device is started, the first electronic device acquiring a voice of a first user, reporting the voice of the first user to a translation server, receiving a translation result for the voice of the first user that is sent by the translation server, and sending the translation result for the voice of the first user to the second electronic device; receiving a voice of a second user that is sent by the second electronic device, reporting the voice of the second user to the translation server, receiving a translation result for the voice of the second user that is sent by the translation server, and broadcasting the translation result for the voice of the second user. Therefore, such a technical solution helps to improve communication efficiency between users speaking different languages.

Description

Translation method based on voice call and electronic equipment

This application requires the priority of the Chinese patent application submitted to the State Intellectual Property Office of China on October 15, 2018 with the application number 201811198715.7 and the invention titled "A Translation Method and Electronic Equipment Based on Voice Conversation." The reference is incorporated in this application.

Technical field

The present application relates to the technical field of terminals, in particular to a translation method and electronic equipment based on voice calls.

Background technique

The popularity of mobile phones and other electronic devices has made communication between people no longer restricted by time and space, which has brought great convenience to people's lives and work. However, due to geographical influence, the communication between the two parties may cause communication barriers due to different languages.

Therefore, it is of great practical value to study how to enable communication parties using different languages to achieve barrier-free communication through mobile phones.

Summary of the invention

The embodiments of the present application provide a voice call-based translation method and an electronic device, which help to improve the communication efficiency of users who use different languages when using electronic devices for communication.

In a first aspect, an embodiment of the present application provides a voice call-based translation method. The method includes:

The first electronic device establishes a voice call connection with the second electronic device; when the voice call translation function of the first electronic device is turned on, the first electronic device collects the voice of the first user and reports the first A user's voice, and receiving the translation result of the first user's voice sent by the translation server, and sending the translation result of the first user's voice to the second electronic device; and receiving the first The second user's voice sent by the two electronic devices reports the second user's voice to the translation server, and receives the translation result of the second user's voice sent by the translation server, and broadcasts the second The translation result of the user's voice.

In the embodiment of the present application, since the first electronic device can realize the translation of the double-ended voice during the user's call, it helps to improve the communication efficiency of users using different languages.

In a possible design, the first electronic device communicates with the translation server through a network, and the first electronic device collects the voice of the first user when the network signal strength is less than or equal to a preset threshold Then, translate the voice of the first user to obtain a translation result of the voice of the first user; after receiving the voice of the second user, translate the voice of the second user to obtain the The translation result of the second user's voice. Through the above technical solutions, it helps to reduce the possibility of speech translation failure due to the deterioration of the network signal.

In a possible design, when the network signal strength is less than or equal to the preset threshold, the first electronic device prompts the first user to switch the network, or automatically switches to the first electronic device to perform Voice translation, or automatically switch to a network with a network signal strength greater than the preset threshold. This helps reduce the possibility of speech translation failure due to network signal deterioration.

In a possible design, the first electronic device determines the translation server according to the local language and the peer language, and then reports the voice of the first user or the second user to the translation server Voice. Wherein, the local language is the language used by the first user, and the peer language is the language used by the second user. Therefore, it helps to enable the first electronic device to select a more appropriate translation server to translate the speech, and improve the reliability of speech translation.

In a possible design, when the first electronic device establishes a voice call connection with the second electronic device or after the first electronic device establishes a voice call connection with the second electronic device, when the When the end language is different from the peer language, the voice call translation function is automatically turned on. Helps simplify user operations.

In a possible design, the first electronic device activates the voice call translation function in response to the user's first operation. It helps to improve the interaction between the first electronic device and the user.

In a possible design, the first electronic device sends a prompt message to the second electronic device that the voice call translation function of the first electronic device has been turned on, and the language used for the prompt message is the language Peer language. It helps to improve the interaction between the first electronic device and the user.

In a possible design, the first electronic device acquires the local language and the opposite language; wherein, the local language is the system language, or the first user and the first language The language used by the first user in the last call record of the two users, or the native language of the place where the phone number of the first user belongs, or the place where the communication network accessed by the first electronic device is located Native language, or language set in response to user operation; the peer language is the native language of the place where the phone number of the second user belongs, or the last call record of the first user and the second user The language used by the second user, or the native language of the home area where the communication network accessed by the second electronic device is located, or the language set in response to user operation. Help to simplify the way of acquiring the local language and the opposite language.

In a possible design, when the first user's voice is not collected and the second user's voice is not collected, the first electronic device broadcasts the translation result of the second user's voice. Helps reduce the possibility of overlapping or incomplete voice announcements.

In a possible design, the first electronic device obtains when the second user's voice is received and the translation result of the second user's voice sent by the translation server is not received within the first duration The network signal strength; when the network signal strength is greater than a preset threshold, the second user is prompted to reduce the speech rate. It helps reduce the possibility that the translation server cannot recognize the voice for translation because the user's speech speed is too fast.

In a possible design, the first electronic device displays first prompt information on a display screen, and the first prompt information is used to prompt the first user to notify the second reduced speech rate; or, The first electronic device sends a first voice to a second electronic device, the first voice is used to prompt the second user to reduce the speech rate; or, the first electronic device plays a second voice to the first user Voice, and the second voice is used to prompt the first user to notify the second user to reduce the speech rate. Helps simplify the way to notify users to slow down.

In a possible design, when the translation result of the second user's voice buffered by the first electronic device exceeds a preset number, the translation result of the second user's voice is broadcast. Helps reduce the possibility of packet loss in the translation of speech.

In a possible design, the first electronic device broadcasts the translation result of the voice of the first user.

In a possible design, the first electronic device displays the text of the voice recording of the conversation between the first user and the second user and the text of the translation result of the voice on the display screen. It helps users understand what the other party is saying.

In a possible design, after the first electronic device establishes a voice call connection with the second electronic device, the local language is used to play or display a prompt message that the voice call translation function is turned on. It helps to remind users that the voice call translation function is turned on.

In a second aspect, an electronic device provided by an embodiment of the present application includes: one or more processors, memory, multiple application programs, and one or more computer programs; where one or more computer programs are stored in In the memory, when the computer program is executed by the electronic device, the foregoing first aspect and any possible design method related to the first aspect of the embodiments of the present application are implemented.

In a third aspect, a chip provided by an embodiment of the present application is coupled to a memory in an electronic device, so that the chip calls a computer program stored in the memory when the chip is running, to implement the first aspect of the embodiment of the present application And any possible design method involved in the first aspect.

According to a fourth aspect, a computer storage medium according to an embodiment of the present application stores a computer program, and when the computer program runs on an electronic device, the electronic device causes the electronic device to execute the first aspect and the first aspect of the embodiment of the present application Any possible design method involved in the aspect.

In a fifth aspect, a computer program product according to an embodiment of the present application, when the computer program product runs on an electronic device, causes the electronic device to perform the above-mentioned first aspect and any tasks related to the first aspect of the embodiment of the present application A possible design method.

In addition, for the technical effects brought by any of the possible design methods in the second aspect to the fifth aspect, please refer to the technical effects brought by different design methods in the method section, which will not be repeated here.

BRIEF DESCRIPTION

FIG. 1 is a schematic structural diagram of a communication system to which an embodiment of this application is applicable;

2 is a schematic diagram of a hardware structure of an electronic device provided by an embodiment of the present application;

3 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

4 is a schematic diagram of an application scenario of an embodiment of this application;

5 is a schematic diagram of a user interface for making a call according to an embodiment of the application;

6 is a schematic diagram of a user interface for setting a peer language according to an embodiment of the present application;

7 is a schematic diagram of system language setting according to an embodiment of the present application;

8 is a schematic diagram of an application scenario of an embodiment of this application;

9 is a schematic flowchart of a voice call-based translation method according to an embodiment of the present application;

10 is a schematic diagram of a user interface prompting the user to slow down the speech rate according to an embodiment of the present application;

11 is a schematic flowchart of a voice broadcast method according to an embodiment of the present application;

12 is a schematic diagram of a queue according to an embodiment of this application;

13 is a schematic diagram of a user interface for text display in an embodiment of the present application;

14 is a schematic diagram of another user interface according to an embodiment of the present application;

15 is a schematic diagram of another user interface according to an embodiment of the present application;

16 is a schematic diagram of a system setting interface of an embodiment of this application;

17 is a schematic diagram of a user interface for caller ID display according to an embodiment of the present application;

18 is a schematic structural diagram of another electronic device according to an embodiment of the present application.

detailed description

It should be understood that in the embodiments of the present application, "at least one" refers to one or more, and "multiple" refers to two or more than two. "And / or" describes the relationship of the related objects, indicating that there can be three relationships. For example, A and / or B can represent the following three relationships: A exists alone, A and B exist simultaneously, and B exists alone. A and B can be singular or plural. The character "/" generally indicates that the related object is a "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of a single item (s) or a plurality of items (s). For example, at least one (a) of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a, b and c, where a, b, c It can be single or multiple.

In order to enable communication parties using different languages (for example, communication parties with different mother tongues) to achieve barrier-free communication through electronic devices, the embodiments of the present application provide a translation method based on voice calls.

As shown in FIG. 1, it is a schematic structural diagram of a communication system to which embodiments of the present application are applicable. Specifically, the communication system architecture shown in FIG. 1 includes an electronic device 100, an electronic device 200, and a mobile communication network. The electronic device 100 and the electronic device 200 can communicate through a mobile communication network. Among them, the mobile communication network may be a second generation mobile communication (2nd-generation, 2G) network, or a third generation mobile communication (3rd-generation, 3G) network, or a fourth generation mobile communication (4th-generation, 4G) ) Network, you can also the fifth-generation mobile communication (5th-generation, 5G) network, etc.

As shown in FIG. 1, user A uses electronic device 100 and user B uses electronic device 200. In the case where user A and user B cannot communicate, for example, user A uses language 1 and user B uses language 2. For the communication of user B, in the embodiment of the present application, electronic device A can translate the collected voice of user A using language 1 into language 2 during the conversation between user A and user B, and then translate into language 2 through the mobile communication network The voice of is sent to the electronic device 200, so that the electronic device 200 can directly play the voice translated into the language 2 to the user B. The electronic device 200 can send the collected voice of the user B using language 2 to the electronic device 100 through the mobile communication network. After receiving the voice sent by the electronic device 200, the electronic device 100 translates the received voice into language 1. Then play it to user A, which helps user A and user B to communicate without language.

Take language 1 as Chinese and language 2 as English as an example. For example, if the voice of user A collected by the electronic device 100 is "I am Lucy, do you need any help?", The translation result of the voice of user A obtained is "This is Lucy speaking, may I help you?" ", And the voice of" This is Lucy speaking, may I help you? "Is sent to the electronic device 200, so that the electronic device 200 plays the voice of" This is Lucy speaking, may I help you? "To the user B, so that User B can understand the meaning expressed by user A without understanding Chinese. After collecting the voice "I want to buy ticket" of user B, the electronic device 200 sends the voice "I want to buy ticket" of user B to the electronic device 100 through the mobile communication network. After receiving the voice of "I want to buy ticket" sent by the electronic device 200, the electronic device 100 obtains the translation result of the voice of the user B "I want to buy a ticket" and replaces "I want to buy a ticket" Is played to user A, so that user A can understand the meaning of user B without understanding English.

In some other embodiments, the electronic device 100 may also send the collected voice uttered by user A using language 1 to the electronic device 200 through the mobile communication network, so that the electronic device 200 can play the voice uttered by user A using language 1 to User B. In addition, in some other embodiments, after receiving the voice using the language 2 sent by the electronic device 200 through the mobile communication network, the electronic device 100 in the embodiment of the present application may also play the voice using the language 2 to the user A.

In order to reduce the requirements on the processing performance of the electronic device 100, in some embodiments, the communication system architecture shown in FIG. 1 further includes a translation server. The electronic device 100 may send the collected voice of the user A or the received voice of the user B collected from the electronic device 200 to the translation server for translation. It should be noted that the translation server for translating the voice of user A collected by the electronic device 100 and the translation server for translating the voice of user B received by the electronic device 100 may be the same translation server or different translations. Server, the embodiment of the present application does not limit the number of translation servers in the communication system. It should be understood that the electronic device 100 may communicate with the translation server through wireless fidelity (Wi-Fi), Bluetooth, or a mobile communication network.

The electronic device 100 and the electronic device 200 in the embodiment of the present application may only have one party, for example, the electronic device 100 supports the voice call translation function during the conversation between the user A and the user B. In the embodiment of the present application, the electronic device 200 may support the voice call The translation function may not support the voice call translation function, which is not limited.

It should be understood that the electronic device 100 in the embodiment of the present application may be a portable electronic device, such as a mobile phone, a tablet computer, a wearable device (such as a smart watch) with a wireless communication function, an in-vehicle device, and the like. Exemplary embodiments of portable electronic devices include, but are not limited to

Or portable electronic devices of other operating systems. The above-mentioned portable electronic device may also be, for example, a laptop computer having a touch-sensitive surface (for example, a touch panel). It should also be understood that, in some other embodiments of the present application, the electronic device 100 may also be a desktop computer with a touch-sensitive surface (such as a touch panel).

Exemplarily, FIG. 2 is a schematic diagram of a hardware structure of a possible electronic device 100. It should be understood that the illustrated hardware structure of the electronic device 100 is only an example. Also, the electronic device 100 may have more or fewer components than shown in the figure, may combine two or more components, or may have different component configurations. The various components shown in the figures can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

As shown in FIG. 2, the electronic device 100 includes a processor 110, an internal memory 121, an external memory interface 122, an antenna 1, a mobile communication module 131, an antenna 2, a wireless communication module 132, an audio module 140, a speaker 140A, a receiver 140B, a microphone 140C, headset interface 140D, display screen 151, subscriber identification module (SIM) card interface 152, camera 153, button 154, sensor module 160, universal serial bus (USB) interface 170, charging management Module 180, power management module 181 and battery 182. In other embodiments, the electronic device 100 may further include a motor, an indicator, and the like.

The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (ISP), a controller, and a video Codec, digital signal processor (DSP), baseband processor, and / or neural-network processor (NPU), etc. Among them, different processing units may be independent devices, or may be integrated in one or more processors.

In some embodiments, a memory may also be provided in the processor 110 for storing instructions and data. Exemplarily, the memory in the processor 110 may be a cache memory. The memory may be used to store instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. The repeated access is avoided, and the waiting time of the processor 110 is reduced, thereby improving the efficiency of the system.

The internal memory 121 may be used to store computer executable program code, where the executable program code includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by executing instructions stored in the internal memory 121. The internal memory 121 may include a storage program area and a storage data area. Among them, the storage program area may store an operating system, at least one function required application programs (such as sound playback function, image playback function, etc.). The storage data area may store data (such as audio data, phone book, etc.) created during use of the electronic device 100 and the like. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash memory (universal flash storage, UFS), and so on.

The external memory interface 122 may be used to connect an external memory card (for example, a Micro SD card) to achieve expansion of the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 122 to realize the data storage function. For example, save music, video and other files in an external memory card.

Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, the antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 131 can provide a wireless communication solution including 2G / 3G / 4G / 5G and the like applied to the electronic device 100. The mobile communication module 131 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), and so on. The mobile communication module 131 can receive the electromagnetic wave signal from the antenna 1 and filter, amplify, etc. the received electromagnetic wave signal, and transmit it to the modem processor for demodulation. The mobile communication module 131 can also amplify the signal modulated by the modulation and demodulation processor and convert it into an electromagnetic wave signal through the antenna 1 to radiate it out. In some embodiments, at least part of the functional modules of the mobile communication module 131 may be provided in the processor 110. In some embodiments, at least part of the functional modules of the mobile communication module 131 and at least part of the modules of the processor 110 may be provided in the same device. For example, the mobile communication module 131 may send voice to the electronic device 200, or may receive voice sent by the electronic device 200.

The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a high-frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal is processed by the baseband processor and then passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 140A, a receiver 140B, etc.), or displays an image or video through a display screen 151. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110, and may be set in the same device as the mobile communication module 131 or other functional modules.

The wireless communication module 132 can provide wireless local area networks (wireless local area networks, WLAN) (such as Wi-Fi networks), Bluetooth (bluetooth, BT), and global navigation satellite systems (GNSS) that are applied to the electronic device 100. ), Frequency Modulation (FM), Near Field Communication (NFC), Infrared (IR) and other wireless communication solutions. The wireless communication module 132 may be one or more devices integrating at least one communication processing module. The wireless communication module 132 receives the electromagnetic wave signal via the antenna 2, frequency-modulates and filters the electromagnetic wave signal, and sends the processed signal to the processor 110. The wireless communication module 132 may also receive the signal to be transmitted from the processor 110, frequency-modulate it, amplify it, and convert it to electromagnetic waves through the antenna 2 to radiate it out. For example, the wireless communication module 132 can send the voice of the user A using the language 1 collected by the electronic device 100 to the translation server, or can send the voice of the user B using the language 2 collected by the electronic device 200 that the mobile communication module 131 receives The server can also receive the translation result of the voice sent by the translation server. It should be understood that in the embodiment of the present application, the speech translation result is a piece of audio.

In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 131, and the antenna 2 is coupled to the wireless communication module 132 so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include a global mobile communication system (global system for mobile communications, GSM), general packet radio service (general packet radio service, GPRS), code division multiple access (code division multiple access, CDMA), broadband Wideband code division multiple access (WCDMA), time division code division multiple access (time-division code division multiple access (TD-SCDMA), long-term evolution (LTE), BT, GNSS, WLAN, NFC , FM, and / or IR technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a beidou navigation system (BDS), and a quasi-zenith satellite system (quasi -zenith satellite system (QZSS) and / or satellite-based augmentation system (SBAS).

The electronic device 100 may implement audio functions through an audio module 140, a speaker 140A, a receiver 140B, a microphone 140C, a headphone interface 140D, an application processor, and the like. For example, music playback, recording, etc.

The audio module 140 can be used to convert digital audio information into an analog audio signal output and also to convert analog audio input into a digital audio signal. The audio module 140 may also be used to encode and decode audio signals. In some embodiments, the audio module 140 may be disposed in the processor 110, or some functional modules of the audio module 140 may be disposed in the processor 110.

The speaker 140A, also called "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music through the speaker 140A or answer hands-free calls.

The receiver 140B, also known as "handset", is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a call or a voice message, it can answer the voice by holding the receiver 140B close to the ear.

The microphone 140C, also called "microphone", "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can approach the microphone 140C through the person's mouth to make a sound. The microphone 140C can be used to collect the voice of the user A, and then convert the voice of the user A into an electrical signal. The electronic device 100 may be provided with at least one microphone 140C. In other embodiments, the electronic device 100 may be provided with two microphones 140C. In addition to collecting sound signals, it may also implement a noise reduction function. In other embodiments, the electronic device 100 may also be provided with three, four, or more microphones 140C to achieve sound signal collection, noise reduction, identify sound sources, and implement directional recording functions.

The headset interface 140D is used to connect wired headsets. The earphone interface 140D may be a USB interface 130, or a 3.5mm open mobile electronic device platform (OMTP) standard interface, the American Telecommunications Industry Association (cellular telecommunications industry association of the USA, CTIA) standard interface, etc. .

The electronic device 100 can realize a display function through a GPU, a display screen 151, an application processor, and the like. The GPU is a microprocessor for image processing, and connects the display screen 151 and the application processor. The GPU is used to perform mathematical and geometric calculations, and is used for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 151 can be used to display images, videos, and the like. The display screen 151 may include a display panel. The display panel may use a liquid crystal display (LCD), organic light-emitting diode (OLED), active matrix organic light-emitting diode or active matrix organic light-emitting diode (active-matrix organic light) emitting diodes (AMOLED), flexible light-emitting diodes (FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diodes (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 151, where N is a positive integer greater than 1.

The electronic device 100 can realize a shooting function through a camera 153, an ISP, a video codec, a GPU, a display screen 151, an application processor, and the like.

The ISP can be used to process the data fed back by the camera 153. For example, when taking a picture, the shutter is opened, and the optical signal is collected by the camera 153, and then the camera 153 converts the collected optical signal into an electrical signal, and transmits the electrical signal to an ISP for processing, which is converted into an image visible to the naked eye. ISP can also optimize the algorithm of image noise, brightness and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be set in the camera 193.

The camera 153 may be used to capture still images or video. Generally, the camera 153 includes a lens and an image sensor. Among them, the object generates an optical image through the lens and projects it to the image sensor. The image sensor may be a charge coupled device (charge coupled device, CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The image sensor converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other image signals. In some embodiments, the electronic device 100 may include 1 or N cameras 153, where N is a positive integer greater than 1.

The key 154 may include a power-on key, a volume key, and the like. The key 154 may be a mechanical key. It can also be a touch button. The electronic device 100 can receive key input and generate key signal input related to user settings and function control of the electronic device 100.

The sensor module 160 may include one or more sensors. For example, the touch sensor 160A, the fingerprint sensor 160B, the gyro sensor 160C, the pressure sensor 160D, the acceleration sensor 160E, and the like. In some embodiments, the sensor module 160 may further include an environment sensor, a distance sensor, a proximity light sensor, a bone conduction sensor, and the like.

The touch sensor 160A may also be referred to as a “touch panel”. The touch sensor 160A may be provided on the display screen 151, and the touch sensor 160A and the display screen 151 constitute a touch screen, also called a "touch screen". The touch sensor 160A is used to detect a touch operation acting on or near it. The touch sensor 160A may pass the detected touch operation to the application processor to determine the type of touch event. The visual output related to the touch operation can be provided through the display screen 151. In other embodiments, the touch sensor 160A may also be disposed on the surface of the electronic device 100, which is different from the location where the display screen 151 is located.

The fingerprint sensor 160 may be used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to realize fingerprint unlocking, access to application locks, fingerprint taking pictures, fingerprint answering calls, and the like.

The gyro sensor 160C may be used to determine the movement posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (ie, x, y, and z axes) may be determined by the gyro sensor 160C. The gyro sensor 160C can be used for shooting anti-shake. Exemplarily, when the shutter is pressed, the gyro sensor 160C detects the shaking angle of the electronic device 100, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to cancel the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 160C can also be used for navigation and somatosensory game scenes.

The pressure sensor 160D is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 160D may be provided on the display screen 151. There are many types of pressure sensors 160D, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. The capacitive pressure sensor may be at least two parallel plates with conductive material. When force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation is applied to the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 100 may also calculate the touched position based on the detection signal of the pressure sensor 180A. In some embodiments, touch operations that act on the same touch position but have different touch operation intensities may correspond to different operation instructions. For example, when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

The acceleration sensor 160E can detect the magnitude of acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used for recognizing the posture of electronic devices, and used in applications such as horizontal and vertical screen switching and pedometers.

In other embodiments, the processor 110 may also include one or more interfaces. For example, the interface may be a SIM card interface 152. For another example, the interface may also be a USB interface 170. As another example, the interface may also be an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit integrated audio (inter-integrated circuit, sound, I2S) interface, a pulse code modulation (pulse code modulation (PCM) interface, a universal asynchronous transceiver transmission (Universal asynchronous receiver / transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input / output (GPIO) interface, etc. It can be understood that, in the embodiment of the present application, the processor 110 may be connected to different modules of the electronic device 100 through an interface, so that the electronic device 100 can implement different functions. For example, taking pictures and processing. It should be noted that the embodiment of the present application does not limit the connection method of the interface in the electronic device 100.

Among them, the SIM card interface 152 may be used to connect the SIM card. The SIM card can be inserted into or removed from the SIM card interface 152 to achieve contact and separation with the electronic device 100. The electronic device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 152 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple SIM cards can be inserted into the same SIM card interface 152 at the same time. The types of the multiple cards may be the same or different. The SIM card interface 152 can also be compatible with different types of SIM cards. The SIM card interface 152 can also be compatible with external memory cards. The electronic device 100 interacts with the network through a SIM card to realize functions such as call and data communication. In some embodiments, the electronic device 100 uses eSIM, that is, an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

The USB interface 170 is an interface conforming to the USB standard specification. For example, the USB interface 170 may include a Mini USB interface, a Micro USB interface, a USB Type C interface, and so on. The USB interface 170 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones and play audio through the headphones. The USB interface 170 can also be used to connect with other electronic devices, such as augmented reality (AR) devices.

The charging management module 180 is used to receive charging input from the charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 180 may receive the charging input of the wired charger through the USB interface 170. In some wireless charging embodiments, the charging management module 180 may receive wireless charging input through the wireless charging coil of the electronic device 100. While the charging management module 140 charges the battery 182, it can also supply power to the electronic device through the power management module 181.

The power management module 181 is used to connect the battery 182, the charging management module 180 and the processor 110. The power management module 181 receives input from the battery 182 and / or the charging management module 180, and supplies power to the processor 110, internal memory 121, external memory, display screen 151, camera 153, mobile communication module 131, wireless communication module 132, and the like. The power management module 181 can also be used to monitor battery capacity, battery cycle times, battery health status (leakage, impedance) and other parameters. In some other embodiments, the power management module 181 may also be disposed in the processor 110. In other embodiments, the power management module 181 and the charging management module 180 may also be set in the same device.

The following uses the electronic device 100 as an example to describe in detail the translation method based on the voice call in the embodiment of the present application.

As shown in FIG. 3, it is a schematic structural diagram of an electronic device 100 according to an embodiment of the present application. Specifically, the electronic device 100 includes an interaction module, a voice collection module, a broadcast module, a communication module, and a translation module.

Among them, the interaction module is used to realize the interaction between the user and the electronic device 100. For example, the interaction module can be used to display a user interface. In some embodiments, the interaction module may be implemented by the display screen 151 in the hardware architecture of the electronic device 100 shown in FIG. 2, or may be implemented by the display screen 151 and the touch sensor in the hardware architecture of the electronic device 100 shown in FIG. 160A to achieve. The voice collection module is used to collect voice. In some embodiments, the voice collection module may be implemented by the microphone 140C in the hardware architecture of the electronic device 100 shown in FIG. 2. The broadcast module is used to broadcast voice, such as the translation result of voice during a call, or untranslated voice, etc. It should be noted that the broadcast module may be implemented by a receiver 140B, a speaker 140A, or a headset connected to the headset interface 140D in the hardware architecture of the electronic device 100 shown in FIG. 2. The communication module is used to communicate between the electronic device and other electronic devices or translation servers. For example, the communication module can be used to receive voice sent by other electronic devices, can also be used to send the voice collected by the voice collection module to the translation server, or can be used to send the voice received by the communication module to the translation server, and can also be used to receive the translation server Translated results of sent voice etc. Exemplarily, the communication protocols that the communication module may support include WiFi, Bluetooth, LTE, WCDMA, GSM, next-generation radio (NR), and so on. In the embodiment of the present application, the communication module may be implemented by the mobile communication module 131 and / or the wireless communication module 132 in the hardware architecture of the electronic device 100 shown in FIG. 2.

The translation module is used to obtain voice translation results. In some embodiments, as shown in FIG. 3, the translation module may include an access unit, a local translation unit, a peer translation unit, a coordination unit, a main control unit, a broadcast control unit, and a voice detection unit. For example, the access unit may be used to implement the selection of translation services. For example, when the network is disconnected or the signal is poor, the access unit adopts offline translation and performs translation through a translation engine pre-configured in the electronic device 100. It should be noted that the translation engine pre-configured in the electronic device 100 may be a software program or command for implementing voice translation, and may be stored in the internal memory 121 or the external memory 110. When the electronic device When translating the voice, the translation engine is called to obtain the translation result of the voice. For another example, when the network signal is good, the access unit adopts online translation, and further selects a corresponding translation server based on a pre-configured strategy. The local translation unit may be used to obtain the voice collected by the voice collection module and cache the translation result of the voice collected by the voice collection module. The peer translation unit may be used to obtain the speech received by the communication module and buffer the translation result of the speech received by the communication module. The broadcast control unit may be used to control the broadcast module to broadcast the translation result. In some embodiments, the local translation unit or the peer translation unit may also send the cached translation result of the voice to the broadcast module to automatically delete the translation result of the voice, and may also store the translation result of the voice to the internal memory 121 Or in an external memory connected to the external memory interface 122. The voice detection unit can be used to detect whether there is currently a voice input. The coordination unit can be used to coordinate the translation process and control the rhythm of voice playback. The main control unit can be used to receive user's operation instructions and coordinate various unit modules to realize the voice call translation function. It should be noted that the translation module in the embodiment of the present application may be implemented by the processor 110 in the hardware architecture of the electronic device 100 shown in FIG. 2. For example, the translation module may be implemented by an application processor in the processor 110. For another example, the translation module may also be a dedicated processor, integrated in the processor 110, or not integrated in the processor 110.

The application scenarios of the voice call-based translation method provided in the embodiments of the present application will be described below accordingly. The translation method based on the voice call in the embodiment of the present application can be applied to an application scenario where the electronic device 100 is the calling terminal and the electronic device 200 is the called terminal, and can also be applied to the electronic device 100 as the called terminal and the electronic device 200 as the calling terminal Application scenarios. Taking the electronic device 100 as the calling terminal and the electronic device 200 as the called terminal to realize the communication between the user A and the user B as an example, the translation method based on the voice call in the embodiment of the present application will be described in detail, in which the user A uses the electronic device 100. User B uses the electronic device 200.

The electronic device 100 in the embodiment of the present application may acquire the translation result of the voice of the user A after collecting the voice of the user A, and send the translation result of the voice of the user A to the electronic device 200, and the electronic device 100 may receive After reaching the voice of the user B sent by the electronic device 200, the translation result of the voice of the user B is obtained, and the translation result of the voice of the user B is played.

In order to facilitate the electronic device 100 to obtain the translation result of the voice of the user A and the translation result of the voice of the user B, in some embodiments, the electronic device 100 determines a device for voice translation and then obtains it from the device for voice translation The translation result of user A's voice and / or the translation result of user B's voice. It should be noted that the device used for voice translation may be the electronic device 100 itself or a translation server. When the device for voice translation is a translation server, the device for voice translation may be one translation server or multiple translation servers. In addition, the device for translating the voice of user A and the device for translating the voice of user B may be the same device, or may be different devices. Exemplarily, the electronic device 100 may determine a device for voice translation before initiating a call to user B. For example, the electronic device 100 may determine the device for user voice translation before determining to initiate a call to user B, and before initiating a call to user B. For another example, the electronic device 100 may also determine a device for voice translation after initiating a call to user B.

In the embodiment of the present application, the electronic device 100 may initiate a call to user B in response to the dialing operation of the phone number of user B.

Exemplarily, as shown in FIG. 4, the display screen 151 of the electronic device 100 displays the user interface 400. Among them, the user interface 400 includes a phone icon 401. In other embodiments, the user interface 400 may also include other application icons, such as a settings icon, WeChat icon, gallery icon, mailbox icon, camera icon, and so on. When the electronic device 100 detects an operation on the phone icon 401, it may display a user interface 410 on the display screen 151 in response to the operation on the phone icon 401. The user interface 410 includes a contact list 411 and a function bar 412. The contact list 411 is stored by user A according to his own needs. The function bar 412 includes function buttons for creating new contacts, scanning business cards, and more. For example, the electronic device 100 displays the user interface 420 on the display screen 151 in response to the user's operation on the user B in the contact list 411. The user interface 420 may include user B's name, phone number, mailbox, and other information, and may also include function buttons such as send information, add to favorites, and block incoming call numbers. The electronic device 100 may initiate a call to the user B in response to the operation of the phone number 12345659876, and display the user interface 430 on the display screen 151. For example, the operation on the phone number 12345659876 may be a touch operation on the phone number 12345659876. Other operations are also possible.

In addition, user A can also initiate a call to user B by directly inputting user B's phone number on user interface 500 of electronic device 100. For example, as shown in FIG. 5, the display screen 151 of the electronic device 100 displays a user interface 500 including a dial keypad, an input phone number “12345659876”, and a dial button 501. When the input telephone number is the telephone number of the user B, the electronic device 100 initiates a call to the user B in response to the operation of the dial button 501, and displays the user interface 430 shown in FIG. 4 on the display screen 151. It should be noted that, in the embodiment of the present application, user A may also use the electronic device 100 to initiate a call to user B through other methods, for example, to initiate a call to user B through voice.

In some embodiments, the electronic device 100 may determine the device for voice translation according to the network signal strength. Wherein, when the network signal strength is greater than a preset threshold, the electronic device 100 determines a device for voice translation from at least one preset translation server. However, the translation servers of different manufacturers have different accuracy for language translation. For example, the translation server provided by the manufacturer S is more accurate for the translation of languages in Southeast Asian languages. For another example, the translation server provided by the manufacturer M is more accurate for the language translation of the European language family. Therefore, in order to improve the accuracy of voice translation, when using the translation server, the electronic device 100 can determine the translation server for voice translation according to the preset local language and peer language, for example, when Chinese and Vietnamese translate each other, Select the translation server of vendor S. It should be noted that the local language is the language used by user A, and the opposite language is the language used by user B. When the network signal strength is less than or equal to a preset threshold, the electronic device 100 determines to perform voice translation through a translation engine configured in the electronic device 100 in advance. Through the above technical solution, it helps to improve the real-time performance of obtaining voice translation results under the condition of poor network signal.

It should be noted that, in the embodiment of the present application, the preset threshold may be pre-configured in the electronic device 100, or may be determined by the electronic device 100 according to a pre-configured algorithm. In addition, in the embodiment of the present application, the electronic device 100 may establish communication with the translation server through a mobile communication network (eg, 4G network, 5G network, etc.), and may also establish communication with the translation server through a WiFi network. In the embodiment of the present application, when the electronic device 100 communicates with the translation server through the mobile communication network, the preset threshold may be set as the first threshold. When the electronic device 100 and the translation server communicate with the translation server through WiFi, the preset threshold may be set as the second threshold. It should be noted that the first threshold and the second threshold are thresholds set according to different networks, and the sizes of the first threshold and the second threshold are not limited herein. For example, when the electronic device 100 turns on the mobile communication network and does not turn on the WiFi network, the electronic device 100 may send the voice to the determined translation server through the mobile communication network when the signal strength of the mobile communication network is greater than the first threshold. For another example, when the electronic device 100 turns on both the mobile communication network and the WiFi network, and when the network signal strength of the WiFi network is greater than the second threshold, in some embodiments, the electronic device 100 uses the WiFi network to communicate with the translation server. Thereby helping to save costs. For another example, when the network signal strength of the mobile communication network is greater than the first threshold and the network signal strength of the WiFi network is less than or equal to the second threshold, in some embodiments, the electronic device 100 uses the mobile communication network to communicate with the translation server. This helps to reduce the communication delay between the electronic device 100 and the translation server.

The following specifically describes the electronic device 100 determining a device for voice translation according to the network signal strength in conjunction with the architecture of the electronic device 100 shown in FIG. 3.

Exemplarily, the main control unit of the electronic device 100 receives the operation reported by the interaction module to initiate a call to user B or receives the operation reported by the interaction module to input the phone number of user B, and then sends the translation service access to the access unit Notice. It should be noted that the main control unit may directly send the notification of the translation service access to the access unit, or may send the notification of the translation service access to the access unit through the coordination unit, etc., which is not limited. After receiving the notification of the translation service access, the access unit obtains the network signal strength from the communication module, and determines the device for voice translation according to the network signal strength.

For example, when the electronic device 100 turns on the WiFi network and the mobile communication network, and the electronic device 100 sends a voice to the translation server through the WiFi network or the mobile communication network, when the network signal strength of the WiFi network is less than or equal to the second threshold and moves When the network signal strength of the communication network is less than or equal to the first threshold, it is determined to use a translation engine pre-configured in the electronic device 100 to translate the speech. For another example, the electronic device 100 may determine that the device used for voice translation is the translation server when the network signal strength of the WiFi network is greater than the second threshold. In a specific implementation, for example, when the WiFi network signal strength is greater than the second threshold, the access unit of the electronic device 100 acquires the preset local language and peer language, and then determines the usage according to the local language and peer language A server for voice translation. In some embodiments, the electronic device 100 may store the preset local language and the peer language in the public storage space, and the access unit may directly obtain the preset local language and the peer language from the public storage space. The local language and the peer language set in advance can also be obtained from the public storage space through the main control unit.

The local language and the opposite language may be manually configured by the user in advance, or may be automatically configured by the electronic device 100.

For example, as shown in FIG. 4, user A may set the peer language through the language setting 422 in the user interface 420. For example, if the native language of the user B is English, the electronic device 100 may respond to the user A's operation on the language setting 422, as shown in FIG. 6, a language list 600 is displayed on the display screen 151. The electronic device 100 displays English in the language setting 422 and hides the language list 600 in response to the operation of selecting “English”. It should be understood that user A may display different languages by sliding up and down in the area of language list 600. It should also be noted that the language included in the language list 600 is a language supported by a translation server capable of communicating with the electronic device 100 and / or a translation engine pre-configured in the electronic device 100, which may be the electronic device 100 before delivery What is set in the electronic device 100 may also be obtained from the translation server after detecting the operation of detecting the stored contact for the first time. In addition, in the embodiment of the present application, the electronic device 100 may also automatically set the language of the language setting 422. For example, the electronic device 100 may automatically set the language of the language setting 422 based on the home of the stored phone number of the user B. For example, if the electronic device 100 detects that the attribution of the user B's phone number is London, it automatically sets the language setting 422 to English. For another example, the electronic device 100 may also set the language setting 422 according to the language used by the user B during the conversation between the user A and the user B. For another example, the electronic device 100 may also set the language setting 422 to a pre-configured default language, for example, the default language is English. The default language may be configured by the electronic device 100 before leaving the factory, or may be set by the user as needed.

The local language involved in the embodiments of the present application may be the system language of the electronic device 100. Exemplarily, as shown in FIG. 7, the display screen 151 of the electronic device 100 displays the user interface 400. The user interface 400 includes a setting icon 700. The electronic device 100 displays the system setting user interface 710 on the display screen 151 in response to the operation of the setting icon 700. The system setting interface 710 includes language and region 711. In addition, the system setting interface 711 may also include setting buttons for other functions, such as enabling cloud backup and screen lock. The electronic device 100 displays the user interface 720 on the display screen 151 in response to the operation on the language and region 711. The user interface 720 includes a system language setting button 721. In addition, in some embodiments, the user interface 720 may further include a region setting button. The electronic device 100 may display a user interface 730 on the display screen 151 in response to the operation of the system language setting button 721, and the user interface 730 includes a language list 731. The language list 731 includes the languages supported by the electronic device 100. In some embodiments, in order to facilitate the user to find the language required by the user from the languages included in the language list 731, the user interface 730 may further include a search box 732. For example, the electronic device 100 may find “Simplified Chinese” from the languages included in the language list 731 in response to the “Simplified Chinese” input by the user A in the search box 732. The electronic device 100 may set the system language to simplified Chinese in response to the operation of selecting “Simplified Chinese” from the language list 732. In some other embodiments, the electronic device 100 may also be automatically set according to the attribution of the local number stored by the user. For example, when the electronic device 100 detects that the locality of the local number is Beijing, it automatically sets the system language to Chinese. For another example, when the electronic device 100 detects that the locality of the local number is Taiwan, it automatically sets the system language to traditional Chinese. It should be noted that in the embodiment of the present application, the user can set the local number on the user interface 410 shown in FIG. 4.

In addition, in some embodiments, the user interface 430 may further include a voice translation button 431. This helps to further simplify user operations. For example, the electronic device 100 may also display the user interface 800 on the display screen 151 in response to the operation of the voice translation button 431. The user interface 800 includes a local language setting 803 and a peer language setting 802. The electronic device 100 may set the local language in response to the operation of the local language setting 803 by the user A, and set the local language in response to the operation of the remote language setting 802 by the user A. The specific implementation of the electronic device 100 setting the local language in response to the operation of the local language setting 803 by the user A, and setting the local language in response to the operation of the remote language setting 802 by the user A, see the electronic device 100 responding to the user A For the operation of the language setting 422 on the user interface 420, the manner of setting the language is implemented, which will not be repeated here. In addition, in the embodiment of the present application, the electronic device 100 may also automatically set the language setting of the local language setting 803 and the remote language setting 802. For example, the electronic device 100 may automatically set the local language setting 803 to the system language and automatically set the peer language setting 802 to the language set by the language setting 422 in the user interface 420 in response to the operation of making a call to user B. For another example, the electronic device 100 may also set the language of the local language setting 803 based on the attribution of the local number. For specific implementation, please refer to the automatic setting of the system language mentioned above. For another example, the electronic device 100 may also set the language setting of the local language setting 803 according to the home location where the mobile communication network accessed by the electronic device 100 is located. For example, if the mobile communication network accessed by the electronic device 100 is China Mobile, the local language is automatically set to Chinese. For another example, the electronic device 100 may also set the language setting of the peer language setting 802 based on the attribution of the user B's phone number. For another example, the electronic device 100 may also obtain the most recent call record between user A and user B according to the local number and the phone number used when initiating a call to user B, and set the local language setting 803 to user A in the call record For the language used, set the peer language 802 to the language used by user B in the call log. For another example, the electronic device 100 may also set the language setting of the peer language setting 803 according to the home location where the communication network accessed by the electronic device 200 is located. For example, if the electronic device 100 acquires that the home of the communication network accessed by the electronic device 200 is the United States, the peer language is automatically set to English.

It should be noted that, in order to ensure the reliability of the automatically determined local language and the peer language, in some embodiments, the main control unit of the electronic device 100 may respond to the operation of selecting user B or inputting the phone number of user B, Send a language setting notification to the access unit. After receiving the language setting notification, the access unit obtains the call record with user B. According to the call record with user B, the local language setting 803 and the remote language setting 802 are set accordingly. If the access unit does not obtain the call record with user B, it can obtain the home of the phone number of user B and the home of the phone number of user A through the communication module, and based on the home of the phone number of user B The language setting 802 is set, and the local language setting 803 is set based on the attribution of the user A's phone number. It should be understood that if the access unit does not obtain a call record with user B, it may also set the local language setting 803 based on the system language. If the access unit cannot obtain the home location of the user B's phone number, it can also set the peer language setting 802 as the default language. It should be noted that if the access unit cannot obtain the home location of the user B's phone number, it may also send a notification to the main control unit that the automatic setting of the peer language setting 803 has failed. After receiving the notification that the automatic setting of the peer language setting 803 has failed, the main control unit prompts the user to manually set the peer language setting 803. For example, the main control unit may prompt the user to manually set the end language setting 803 by triggering the display screen 151 to display prompt information. For another example, the main control unit may also prompt the user by triggering the announcement module to play the prompt information that the user manually sets the end language setting 803. The embodiment of the present application does not limit the method of prompting the user to manually set the end language setting 803.

It should also be noted that, in order to facilitate user search, the local language setting 803 and the peer language setting 802 in the embodiment of the present application may also be set on the user interface 430.

When the local language and the peer language are the same, the electronic device 100 may not need to translate the voice. In some embodiments, the electronic device 100 may directly send the electronic to the electronic device 200 when the local language and the peer language are the same. The voice of the user A collected by the device 100, the electronic device 100 receives the voice of the user B sent by the electronic device 200, and plays the voice of the user B. This helps to improve the processing method of the simplified electronic device 100. The electronic device 100 can automatically start the voice call translation function when the local language and the peer language are different, and perform the translation process of the voice of the user A and the voice of the user B.

E.g. After the access unit automatically sets the local language and the remote language, or obtains the local language and the remote language set by the user, if it detects that the local language and the remote language are different, it executes the determination for voice translation. The process of the server. Specifically, the process that the access unit executes to determine the server for voice translation can refer to the foregoing embodiment, and details are not described herein again. If the access unit detects that the local language and the peer language are the same, there is no need to perform the process of determining the server for voice translation.

In order to improve the user experience, in other embodiments, after receiving the voice of the user B sent by the electronic device 200, the electronic device 100 broadcasts the voice of the user B. For example, if the local language preset in the electronic device 100 is simplified Chinese and the opposite language is English, if the voice sent by the user B to the electronic device 100 using the electronic device 200 is "Hello, This Mike is speaking.", The electronic device 100 After receiving "Hello, This is Mike speaking.", Broadcast "Hello, This Is Mike Mike speaking." To user A, and obtained the translation result of "Hello, This Is Mike Mike speaking." "Hello, this is Mike. ", Broadcast" Hello, I'm Mike "to user A. In some embodiments, the electronic device 100 also broadcasts the translation result of the voice of the user A. For example, the voice of user A collected by the electronic device 100 is "Hello, I want to buy a flight ticket." Then the translation result of "Hello, I want to buy a flight ticket." Is obtained "Hello, I want to buy a ticket. ”, not only to send the translation result“ Hello, I want to buy ticket. ”to the electronic device 200, but also to broadcast the translation result“ Hello, I want to buy a ticket. ” In addition, in some embodiments, the electronic device 100 may also send the collected voice of the user A to the electronic device 200. After receiving the voice of the user A sent by the electronic device 100, the electronic device 200 broadcasts the voice of the user A. After receiving the translation result of the voice of the user A sent by the electronic device 100, the electronic device 200 broadcasts the translation result of the voice of the user A. In other embodiments, the electronic device 100 may also send the translation result of the voice of the user B to the electronic device 200. After receiving the translation result of the voice of the user B, the electronic device 200 may also broadcast the translation result of the voice of the user B.

When the electronic device 100 receives the voice of user B sent by the electronic device 200, it directly broadcasts the voice of user B. In this case, if the electronic device 100 receives the voice translation result of user A, the translation result of user A's voice is broadcast. , It will cause the announcement of the voice of user B and the announcement of the translation result of the voice of user A to overlap. When the electronic device 100 collects the voice of the user A, if the translation result of the voice of the user A is received, in this case, the broadcast module may announce the result of the voice translation of the user A, which may cause the broadcasted voice to mean Incomplete.

Therefore, in some embodiments, after receiving the voice translation result, the electronic device 100 broadcasts the received voice translation result if no voice input is detected. In other embodiments, if the electronic device 100 detects that there is currently a voice input, it waits to detect the translation result of the received voice when no voice input is detected. This helps to reduce the possibility of conflict with the voice of the broadcast user B and the possibility of incomplete translation of the voice of the broadcast voice of the user A. In addition, in other embodiments, when the received translation result is the translation result of the voice of the user A, the electronic device 100 sends the translation result of the voice of the user A to the electronic device 200 when no voice input is detected. Therefore, it helps to reduce the possibility of overlapping voice announcements of the electronic device 200.

It should be noted that the electronic device 100 detects the voice input includes two cases: one is that the voice collection module of the electronic device 100 collects the voice of the user A, and the other is that the communication module of the electronic device 100 receives the collection of the electronic device 200 Voice of user B. When the electronic device 100 detects that there is no voice input, the embodiment of the present application refers to that the voice collection module of the electronic device 100 does not collect the voice of the user A, and the communication module does not receive the voice of the user B collected by the electronic device 200.

The manner of broadcasting speech and speech translation results in the embodiment of the present application will be described in detail below with the architecture of the electronic device 100 shown in FIG. 3.

As shown in FIG. 9, the electronic device 100 first acquires voice. For example, the electronic device 100 may obtain the voice of the user A through the voice collection module, or may receive the voice of the user B through the communication module, and then send the obtained voice to the communication module through the access unit, and then the communication module reports the voice to the translation The server then translates the voice by the translation server, and returns the translation result to the communication module, and finally broadcasts the translation result of the voice through the broadcast module. In order to ensure the rhythm of voice acquisition and voice broadcast, in some embodiments, the electronic device 100 may coordinate voice acquisition and voice translation based on voice acquisition and voice translation to coordinate voice acquisition and translation results. Broadcast control.

The control method for broadcasting the translation result of the voice of the user A is specifically described below.

After detecting that the electronic device 200 answers the call initiated by the electronic device 100 to the user B, the main control unit notifies the voice detection unit to monitor whether the voice acquisition module collects the voice of the user A, and whether the communication module receives the user B's voice. In the embodiment of the present application, the voice detection unit may report the voice detection result to the coordination unit in real time, or may report the voice detection result to the coordination unit every preset time period, and may also detect that the user A ’s When the voice and communication module does not receive the voice of user B, the voice detection result is sent to the coordination unit.

The voice collection module sends the collected user A's voice to the local translation unit. The local translation unit sends the voice of user A to the access unit, and then the access unit sends the voice of user A to the translation server through the communication module. After translating the voice of user A, the translation server returns the translation result of the voice of user A to the electronic device 100. The electronic device 100 receives the translation result of the voice of the user A sent by the translation server by the communication module, and sends the translation result of the voice of the user A to the local end through the access unit after the communication module receives the translation result of the voice of the user A The translation unit stores the translation result of user A's voice by the local translation unit. For example, the local translation unit may save the translation result of the voice of the user A into the first queue set in advance.

After receiving the translation result of user A's voice, the local translation unit reports the local voice translation result notification to the coordination unit. After the coordination unit receives the local voice translation result notification, if it is determined according to the voice detection result reported by the voice detection unit If there is no voice input, a notification is sent to the broadcast control unit to allow the voice translation result to be broadcast. It should be understood that, in the embodiment of the present application, the coordination unit may directly send a notification that allows the voice translation result to be broadcast to the broadcast control unit, or may send the notification that the voice translation result is broadcast to the broadcast control unit through the main control unit.

After the broadcast control unit receives the notification that the voice translation result is allowed to be broadcast, it notifies the broadcast module to obtain the voice translation result of user A for broadcast. In addition, if the coordination unit determines that there is no voice input according to the voice detection result reported by the voice detection unit, it also notifies the communication module to send the translation result of the voice of the user A to the electronic device 200.

After receiving the local voice translation result notification, if the coordination unit determines that there is a voice input according to the voice detection result reported by the voice detection unit, it sends a notification to the broadcast control unit to suspend the translation result. Therefore, the broadcast control unit can realize the control of the broadcast result of the speech translation module of the broadcast module. In the embodiment of the present application, if the coordination unit determines that there is a voice input according to the voice detection result reported by the voice detection unit, it may not send a notification to the broadcast control unit. After the coordination unit determines that there is no voice input according to the voice detection result reported by the voice detection unit, Then send a notification to the broadcast control unit to allow the voice translation result to be broadcast. In addition, if the coordination unit determines that the voice input is monitored according to the voice detection result reported by the voice detection unit, the communication module does not send the translation result of the voice of the user A to the electronic device 200. For example, the coordination unit may realize that the communication module does not send the translation result of the voice of the user A to the electronic device 200 by sending a notification to the communication module that the sending of the local voice translation result to the electronic device 200 is suspended. For another example, the communication module may not send the translation result of the voice of the user A to the electronic device 200 when the notification that the local end voice translation result is allowed to be sent to the electronic device 200 is not received.

In addition, when the electronic device 100 realizes the voice translation of the user A through the translation server, the electronic device 100 may not be able to obtain the translation result of the voice of the user A due to a network signal or a speed of speech. For example, if the communication connection is established between the electronic device 100 and the translation server through the network, the poor network signal may cause the electronic device 100 to fail to upload the voice of the user A to the translation server, or may cause the translation server to fail The translation result of the voice of the user A is returned to the electronic device 100 in time.

In some other embodiments, when the electronic device 100 detects the voice input of the user A, if the translation result of the voice of the user A is not received within the first period of time, the current network signal strength of the electronic device 100 is obtained. The strength is the signal strength of the network used by the electronic device 100 to communicate with the translation server, which is used to translate the speech of the user A. It should be noted that the first duration may be pre-configured in the electronic device 100, or may be determined by the electronic device 100 according to a preset algorithm. Take the network used by the electronic device 100 to communicate with the translation server as a mobile communication network as an example. If the network signal strength is less than or equal to the first threshold, the electronic device 100 switches to a translation engine pre-configured in the electronic device 100 for voice translation, switches to a serving cell with a good network signal, or prompts the user to switch the network, as in the electronic When the device 100 is pre-configured with at least two SIM cards respectively using different mobile communication networks, it automatically switches to another mobile communication network and uses another mobile communication network to communicate with the translation server, or when the electronic device 100 turns on the WiFi network, automatically Switch to a WiFi network whose signal strength of the WiFi network is greater than the second threshold, etc. Therefore, the above technical solution helps to reduce the possibility that speech translation is blocked due to the deterioration of the network signal. In some embodiments, when the network signal strength of the mobile communication network is greater than the first threshold or the WiFi network signal strength is greater than the second threshold, the electronic device 100 prompts the user A to slow down. Exemplarily, as shown in FIG. 10, the electronic device 100 may prompt the user to reduce the speech rate by displaying prompt information 1000 on the display screen 151. For example, the prompt information 900 may be "please speak slowly". For another example, when the electronic device 100 detects that the electronic device 100 is blank, it may prompt the user A to slow down by playing a prompt tone. For example, the prompt sound may be a "didi" prompt sound, or a "please slow down" voice. The prompt sound may be pre-configured in the electronic device 100. Therefore, through the above technical solution, it helps to reduce the possibility of translation disorder caused by the user's speech speed being too fast.

In a specific implementation, in some examples, if the coordination unit of the electronic device 100 determines that the voice collection module has a voice input according to the received voice detection result reported by the voice detection unit, but if the coordination unit does not receive it within the preset first time period For notification of the local voice translation result, the coordination unit sends a notification of network signal strength detection to the access unit. After receiving the notification of network signal strength detection, the access unit obtains the network signal strength from the communication module. It should be understood that the network signal strength acquired by the access unit from the communication module is the signal strength of the network used by the electronic device 100 to communicate with the translation server. The translation server is used to translate the voice of user A. Taking the network used by the electronic device 100 to communicate with the translation server as a mobile communication network as an example, if the network signal strength is greater than the first threshold, a network normal notification is sent to the coordination unit. After receiving the network normal notification, the coordination unit sends The control unit sends a notification prompting user A to reduce the speech rate. After receiving the notification prompting the user A to reduce the speech rate, the main control unit prompts the user A to reduce the speech rate. When the network signal strength acquired from the communication module is not greater than the first threshold, the access unit may send a notification of network abnormality to the coordination unit, and then the coordination unit sends the notification of network abnormality to the main control unit. After receiving the notification of network abnormality, the main control unit prompts the user to switch the network, or automatically switches the network. For example, the main control unit may display the current network abnormality through the display screen 151 to prompt the user to switch the network. In addition, in some embodiments, when the network signal strength acquired from the communication module is not greater than the first threshold, the access unit may also automatically switch the voice translation service to the translation engine pre-configured in the electronic device 100. When the access unit detects that the network is normal, it can switch to the translation server again for voice translation.

When the electronic device 100 detects that there is no voice input for a long time, but receives the translation result of the voice of the user A, it may also be caused by the poor network signal strength or the user's speech speed being too fast. Therefore, in some embodiments, the electronic device 100 obtains the current network signal strength of the electronic device 100 when it detects that there is no voice input within the second duration but receives the translation result of the voice of the user A, the network signal strength is electronic The device 100 is used for the signal strength of the network communicating with the translation server, which is used to translate the speech of the user A. Specifically, for the manner in which the electronic device 100 achieves coordination according to the strength of the network signal, refer to the electronic device 100 when the voice input of the user A is detected and the translation result of the voice of the user A is not received within the first period of time. A way to achieve network signal strength coordination.

During specific implementation, in some embodiments, if the coordination unit of the electronic device 100 determines that the voice collection module has no voice input within the second duration according to the voice detection result reported by the voice detection unit, the coordination unit receives the local voice translation result , The coordination unit sends a notification of network signal strength detection to the access unit. For the subsequent execution steps after the access unit receives the notification of network signal strength detection, you can refer to the case where the coordination unit does not receive the local voice translation result notification within the preset first time period when the voice acquisition module has voice input The subsequent execution steps after the access unit receives the notification of network signal strength detection will not be repeated here.

It should be noted that when the number of translation results of the voice of the user A cached by the electronic device 100 exceeds the preset number, it is not necessary to consider whether the electronic device 100 currently collects the voice of the user A or receives the voice of the user B, and directly broadcasts The translation result of user A's voice. Therefore, it helps to avoid that the voice translation result of the user A cached by the electronic device 100 overflows, resulting in the loss of information. For example, the first queue is used to cache the translation results of the voice of user A, and the preset number is the maximum number of translation results that can be stored in the first queue. Exemplarily, in a specific implementation, when the local translation unit detects that the number of cached translation results of the voice of the user A exceeds a preset number, it sends a notification to preferentially broadcast the translation results to the coordination unit. After receiving the notification of preferentially broadcasting the local voice translation result, the coordination unit sends a notification of broadcasting the local voice translation result to the broadcasting control unit, so that the broadcasting module broadcasts the translation result of the voice of the user A.

The control method of the broadcast of the translation result of the voice of the user B is specifically described below.

The communication module is used to receive the voice of the user B collected by the electronic device 200 and send the received voice of the user B collected by the electronic device 200 to the opposite translation unit. The peer translation unit sends the voice of user B to the access unit. Then the access unit sends the voice of user B to the translation server through the communication module. After translating the voice of user B, the translation server returns the translation result to the electronic device 100. The electronic device 100 receives the user B's voice translation result sent by the translation server from the communication module, and after the communication module receives the user B's voice translation result, sends the user B's voice translation result to the peer translation through the access unit unit. The translation result of the voice of user B is saved by the peer translation unit. For example, the peer translation unit may save the translation result of user B's voice into a pre-set second queue.

After receiving the translation result of the voice of the user B, the peer translation unit reports the notification of the peer voice translation result to the coordinating unit. If there is no voice input, a notification is sent to the broadcast control unit to allow the voice translation result to be broadcast. It should be understood that, in the embodiment of the present application, the coordination unit may directly send a notification that allows the voice translation result to be broadcast to the broadcast control unit, or may send the notification that the voice translation result is broadcast to the broadcast control unit through the main control unit.

After the broadcast control unit receives the notification that the voice translation result is allowed to be broadcast, it notifies the broadcast module to obtain the voice translation result of user B for broadcast. In addition, if the coordination unit determines that there is no voice input according to the voice detection result reported by the voice detection unit, it also notifies the communication module to send the translation result of the voice of the user B to the electronic device 200.

After receiving the notification of the voice translation result from the peer end, if the coordination unit determines that there is a voice input according to the voice detection result reported by the voice detection unit, it sends a notification to the broadcast control unit to suspend the translation result. Therefore, the broadcast control unit can realize the control of the broadcast result of the speech translation module of the broadcast module. In the embodiment of the present application, if the coordination unit determines that there is a voice input according to the voice detection result reported by the voice detection unit, it may not send a notification to the broadcast control unit, and then the coordination unit determines that no voice input has been received according to the voice detection result reported by the voice detection unit , And then send a notification to the broadcast control unit to allow the announcement of speech translation results. In addition, in some embodiments, the coordination unit may also not transmit the translation result of the voice of the user A to the electronic device 200 when it is determined that there is a voice input according to the voice detection result reported by the voice detection unit. For example, the coordination unit may realize that the communication module does not send the translation result of the voice of the user B to the electronic device 200 by sending a notification to the communication module to suspend the sending of the peer voice translation result to the electronic device 200, or may not receive permission from the communication module Sending the notification of the translation result of the peer voice to the electronic device 200, so that the communication module does not send the translation result of the voice of the user B to the electronic device 200.

In addition, when the electronic device 100 realizes the voice translation of the user B through the translation server, the electronic device 100 may not be able to obtain the translation result of the voice of the user B due to a network signal or a speed of speech. For example, if the communication connection is established between the electronic device 100 and the translation server through the network, the poor network signal may cause the electronic device 100 to fail to upload the voice of the user A to the translation server, or may cause the translation server to fail The translation result of the voice of the user B is returned to the electronic device 100 in time.

Therefore, in some other embodiments, the electronic device 100 obtains the current network signal of the electronic device 100 when the user B's voice sent by the electronic device 200 is received but the translation result of the user B's voice is not received within the third duration The network signal strength is the signal strength of the network used by the electronic device 100 to communicate with the translation server, and the translation server is used to translate user B's speech. It should be noted that the third duration may be pre-configured in the electronic device 100, or may be determined by the electronic device 100 according to a preset algorithm. In the following, the network used by the electronic device 100 for communication with the translation server is a mobile communication network as an example. When the network signal strength is less than or equal to the first threshold, the electronic device 100 switches to a translation engine pre-configured in the electronic device 100 for voice translation, or prompts the user to switch the network, or automatically switches to a network signal strength greater than the first threshold Mobile communication network, or automatically switch to a wireless communication network with a network signal strength greater than a first threshold. This helps to reduce the possibility that the network signal becomes worse and the speech translation is blocked. In other embodiments, when the network signal strength is greater than the first threshold, the electronic device 100 prompts the user B to slow down. This helps to reduce the possibility of translation disorder due to the user's rapid speech rate. Exemplarily, the electronic device 100 may prompt the user A to notify the user B to reduce the speech rate by displaying prompt information on the display screen 151. The electronic device 100 sends the translation result of the speech speed-reduced speech that the user A prompts the user B to the electronic device 200, so as to achieve the purpose of prompting the user B to reduce the speech rate. For another example, the electronic device 100 may also prompt the user A to notify the user B to slow down the speech by playing a prompt tone when the electronic device 100 detects a black screen. For example, the prompt sound may be a "didi" prompt sound, or a "please slow down" voice. The prompt sound may be pre-configured in the electronic device 100. For another example, the electronic device 100 may also automatically send a prompt tone to the electronic device 200, so that the electronic device 200 can play the prompt tone to the user B to achieve the purpose of prompting the user B to reduce the speech rate.

In specific implementation, in some examples, the coordination unit of the electronic device 100 determines that the communication module receives the voice from the electronic device 200 according to the received voice detection result reported by the voice detection unit, but if the coordination unit presets the third duration If the notification of the result of the voice translation of the peer end is not received, the coordination unit sends a notification of network signal strength detection to the access unit. After receiving the notification of network signal strength detection, the access unit obtains the network signal strength from the communication module. If the network signal strength is greater than the first threshold, it sends a network normal notification to the coordination unit. After the coordination unit receives the network normal notification, Send a notification to the main control unit prompting user B to reduce the speech rate. After receiving the notification prompting user B to decrease the speech rate, the main control unit prompts user B to decrease the speech rate. For example, after receiving the notification prompting the user B to reduce the speech rate, the main control unit may automatically send a prompt tone to the electronic device 200 through the communication module to prompt the user B to reduce the speech rate. For another example, the main control unit may also prompt the user of the peer end to speak too fast by displaying prompt information on the display screen 151 or playing a prompt sound to the user A through the broadcast module. After the user A sees the prompt information or hears the prompt sound, he orally prompts the user B to reduce the speech rate, and the electronic device 100 sends the translation result of the speech speed-reduced speech that the user A prompts the user B to the electronic device 200 to achieve the prompt The purpose of user B to reduce the speaking rate. When the network signal strength acquired from the communication module is not greater than the first threshold, the access unit may send a notification of network abnormality to the coordination unit, and then the coordination unit sends the notification of network abnormality to the main control unit. After receiving the notification of network abnormality, the main control unit switches to the network to the prompt. For example, the main control unit may display the current network abnormality through the display screen 151 to prompt the user to switch the network. In addition, in some embodiments, when the network signal strength acquired from the communication module is not greater than the first threshold, the access unit may also automatically switch the voice translation service to the translation engine pre-configured in the electronic device 100. When the access unit detects that the network is normal, it can also switch the voice translation service to the translation server again.

When the electronic device 100 detects that there is no voice input for a long time, but receives the translation result of the voice of the user B, it may also be caused by the poor network signal strength or the user's speech speed being too fast. Therefore, in some embodiments, the electronic device 100 obtains the current network signal strength of the electronic device 100 when it detects that there is no voice input within the fourth duration but receives the translation result of the voice of the user B, the network signal strength is electronic The signal strength of the network used by the device 100 to communicate with the translation server. It should be noted that the fourth duration may be pre-configured in the electronic device 100, or may be determined by the electronic device 100 according to a preset algorithm. Specifically, for the manner in which the electronic device 100 achieves coordination according to the strength of the network signal, refer to the electronic device 100 when the voice input of the user A is detected and the translation result of the voice of the user A is not received within the first period of time. A way to achieve network signal strength coordination. This helps to improve the reliability of voice translation during calls.

In specific implementation, in some embodiments, if the coordination unit of the electronic device 100 determines that the communication module within the fourth time period does not receive the voice from the electronic device 200 according to the voice detection result reported by the voice detection unit, but the coordination unit receives For the notification of the voice translation result of the peer end, the coordination unit sends a notification of network signal strength detection to the access unit. For the subsequent execution steps after the access unit receives the notification of network signal strength detection, you can refer to the case where the coordination unit does not receive the local voice translation result notification within the preset first time period when the voice acquisition module has voice input The subsequent execution steps after the access unit receives the notification of network signal strength detection will not be repeated here.

It should be noted that when the number of translation results of the voice of the user B cached by the electronic device 100 exceeds the preset number, it is not necessary to consider whether the electronic device 100 currently collects the voice of the user A or receives the voice of the user B, and directly broadcasts the user The translation result of B's voice. Therefore, it helps to avoid that the voice translation result of the user B cached by the electronic device 100 overflows, resulting in the loss of information. For example, the second queue is used to cache the translation results of the voice of user B, and the preset number is the maximum number of translation results that can be stored in the second queue. Exemplarily, in a specific implementation, the peer translation unit sends a notification of preferential broadcast of the peer voice translation result to the coordination unit. After receiving the notification of preferential broadcast of the peer voice translation result, the coordination unit sends the broadcast of the peer voice to the broadcast control unit The notification of the translation result causes the broadcast module to broadcast the translation result of user B's voice.

Exemplarily, as shown in FIG. 11, it is a schematic flowchart of a method for broadcasting voice according to an embodiment of the present application. The electronic device 100 executes the following steps.

In step 1101, a new voice translation result is received. It should be understood that the translation result of the new voice may include the translation result of the voice of the user A or the translation result of the voice of the user B. The translation result of the voice of the user A is the translation result of the local end voice, and the translation result of the voice of the user B is the translation result of the opposite end voice.

In step 1102, it is judged whether the translation result of the buffered voice exceeds a preset threshold, if yes, step 1104 is executed, otherwise step 1103 is executed.

In step 1103, it is judged whether there is a voice input. If yes, step 1105 is executed; otherwise, step 1104 is executed. It should be understood that voice input refers to that the electronic device 100 collects the voice of the user A and / or the electronic device 100 receives the voice of the user B sent by the electronic device 200. No voice input means that the electronic device 100 does not collect the voice of the user A, nor receives the voice of the user B sent by the electronic device 200.

Step 1104, the translation result of the voice is broadcasted, and the process ends. For example, the translation result of the broadcasted speech is the translation result of the earliest cached speech.

For example, voice translation results can be cached in the queue. For example, a queue diagram for buffering translation results of speech is shown in FIG. 12, when the queue is empty, the electronic device 100 may store the translation result of the untranslated speech that has been translated first to position 1 in the queue, and then After receiving the translation result of the new speech, if the translation result of the speech stored in the queue at position 1 has not yet been broadcast, the new speech translation result is stored at position 2 in the queue, and so on. The electronic device broadcasts the speech translation result from position 1 in the queue shown in FIG. 12.

Step 1105, when the input voice is the peer voice, broadcast the peer voice and obtain the translation result of the peer voice, and re-execute step 1101, when the input voice is the local voice, obtain the translation of the local voice As a result, step 1101 is re-executed, and the flow ends.

In addition, in some embodiments, the electronic device 100 displays the text corresponding to the voice of the real-time call record of the user A and the user B and the translation result of the corresponding voice on the display screen 151 during the call between the user A and the user B Text. Therefore, it helps users A and B to understand the meanings expressed by each other when the language is not available. In some embodiments, the electronic device 100 may display the text corresponding to the voice of the real-time call of the user A and the user B and the text of the translation result of the corresponding voice on the user interface 800. For example, the user interface 800 may include a call log text display area 804, and the text of the real-time call log of the user A and the user B is displayed through the call log text display area 804. For example, the voice of the user A collected by the electronic device 100 is "Hello, I want to buy a ticket." The voice of the received electronic device B is "OK, which day do you prefer" and the voice of the user A obtained The translation result is "Hello, I want to buy ticket", and the translation result of the obtained user B's voice is "OK, which day do you want to order?", As shown in Figure 13, in the call log text display Area 804 displays the text "Hello, I want to buy a ticket.", The text "Hello, I want to buy ticket", the text "OK, which do you prefer", and "Okay, where do you want to book?" The text? "

It should be noted that the text corresponding to the voice of the real-time call record of user A and user B and the text of the translation result of the corresponding voice may be obtained by the electronic device 100 based on voice recognition, or the electronic device 100 The voice of the real-time call record of user B and the translation result of the corresponding voice are reported to the voice translation server, and the voice translation server converts the voice into text and sends it to the electronic device 100. The embodiment of the present application does not limit the manner in which the electronic device 100 acquires text.

In other embodiments, in order to facilitate the determination of both the user A and the user B that the electronic device 100 has enabled the voice translation service, a prompt tone for the voice call translation function that has been turned on is configured in the electronic device 100 in advance, for example, pre-configured in the electronic device The language used for the prompt tone of the voice call translation function in 100 can be the system language or the set local language. When the system language or the local language changes, the voice configured in the electronic device 100 is pre-configured The language used for the prompt tone of the call translation function changes accordingly. After detecting that user B answers the call of user A, the electronic device 100 broadcasts the prompt tone of the voice call translation function pre-configured in the electronic device 100, and The electronic device 200 sends the translation result of the prompt sound that the voice call translation function is turned on. Among them, the language used for the translation result of the prompt tone with the voice call translation function turned on is the set opposite language. After receiving the translation result of the prompt tone that the voice call translation function is turned on, the electronic device 200 plays the prompt tone that the voice call translation function is turned on, so that the user B knows that the voice call translation function of the electronic device 100 is turned on.

After detecting that the user B answers the call of the user A, the electronic device 100 may also display a text on the display screen 151 that the voice call translation function has been activated. Take the text of the prompt tone that the voice call translation function is turned on as "Hello, welcome to use the voice call translation service. Please start the call after the voice broadcast is completed." For example, the electronic device 100 detects that user B answers user A After the call, the user interface 800 displays "Hello, welcome to use the voice call translation service, please start the call after the voice broadcast is completed.", As shown in Figure 14.

In order to facilitate the next time the electronic device 100 sets the local language setting 803 and the remote language setting 802, the electronic device 100 saves the current language settings 803 and The settings made by the peer language setting 802.

In the embodiment of the present application, a virtual button for turning on or off a voice call translation function may also be set on the electronic device 100. This helps users to control whether to turn on or off the voice call translation function. The user can turn on or off the voice call translation function by operating the virtual button, or the electronic device 100 can automatically determine whether to turn on or off the voice call translation service to set the virtual button.

For example, as shown in FIG. 15, the user interface 800 includes a virtual button 1400, which is used to turn on or off the voice call translation function. Specifically, when the virtual button 1200 is turned off, the electronic device 100 turns off the voice call translation function; when the virtual button 1200 is turned on, the electronic device 100 turns on the voice call translation function. In addition, it should be noted that when the virtual button 1200 is turned off, the local language setting 803 and the peer language setting can be hidden or grayed out in the user interface 800. For another example, as shown in FIG. 16, a voice translation setting button 1601 may also be added to the system setting interface 1600, where the voice translation setting button 1601 is used to turn on or off the voice call translation function. Among them, the electronic device 100 may display the system setting interface 1600 in response to operating the setting icon.

It should be noted that when the electronic device 100 serves as the called terminal and the electronic device 200 serves as the calling terminal, if the electronic device 200 does not have the voice call translation function, if the electronic device 100 has the voice call translation function, the electronic device 100 After receiving the call from user B, the translation process based on the voice call can be performed. Specifically, the electronic device 100 performs a translation method based on a voice call. For details, refer to the electronic device 100 as a calling end and the electronic device 200 as a called end to perform a translation method based on a voice call, which will not be repeated here.

For example, as shown in FIG. 17, after receiving the call from user B, the electronic device 100 may display the user interface 1700 on the display screen 151. The user interface 1700 includes a voice translation button 1701. The electronic device 100 may display the user interface 800 shown in FIG. 8 on the display screen 151 in response to the operation of the voice translation button 1701. For another example, the electronic device 100 may also turn on or off the voice call translation function in response to the operation of the voice translation button 1701. This helps users turn on voice call translation.

In addition, the translation method based on voice calls in the embodiments of the present application can also be applied to other applications that can perform real-time voice communication, such as WeChat. In addition, the above-mentioned embodiments of the present application are described only for two-person conversations, and the embodiments of the present application can also be applied to application scenarios of multi-person conversations, such as telephone conferences.

The various embodiments involved in this application may be used alone or in combination with each other to achieve different technical effects.

In the above embodiments provided by the present application, the method provided by the embodiments of the present application is introduced from the perspective of an electronic device as an execution subject. In order to realize each function in the method provided by the embodiments of the present application, the electronic device may include a hardware structure and / or a software module, and the above functions are implemented in the form of a hardware structure, a software module, or a hardware structure plus a software module. Whether one of the above functions is executed in a hardware structure, a software module, or a hardware structure plus a software module depends on the specific application of the technical solution and design constraints.

Based on the same concept, FIG. 18 shows an electronic device 1800 provided by the present application. The electronic device 1800 includes at least one processor 1810, memory 1820, and transceiver 1830. The processor 1810 is coupled to the memory 1820 and the transceiver 1830. The coupling in the embodiments of the present application is an indirect coupling or communication connection between devices, units, or modules, which may be electrical, mechanical, or other forms used for devices , Information exchange between units or modules.

Specifically, the memory 1820 is used to store program instructions.

The transceiver 1830 is used to receive or transmit data.

The processor 1810 is configured to call the program instructions stored in the memory 1820, so that the electronic device 1800 executes the voice call-based translation method provided in the embodiment of the present application.

It should be understood that the electronic device 1800 may be used to implement a voice call-based translation method according to an embodiment of the present application. For related features, reference may be made to the foregoing, and details are not described herein again.

Those skilled in the art can clearly understand that the embodiments of the present application can be implemented by hardware, firmware, or a combination thereof. When implemented in software, the above functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media includes computer storage media and communication media, where communication media includes any medium that facilitates transfer of a computer program from one place to another. A storage medium may be any available medium that can be accessed by a computer. Take this as an example, but not limited to: computer-readable media may include RAM, ROM, electrically erasable programmable read-only memory (electrically erasable programmable read only memory, EEPROM), compact disc read-only (memory, CD- ROM) or other optical disk storage, magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store a desired program code in the form of instructions or data structures and that can be accessed by a computer. Also. Any connection can become a computer-readable medium as appropriate. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, Then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, wireless, and microwave are included in the fixing of the associated media. As used in embodiments of the present application, disks and discs include compact discs (CDs), laser discs, optical discs, digital video discs (DVDs), floppy disks, and Blu-ray discs, where Disks usually copy data magnetically, while disks use lasers to copy data optically. The above combination should also be included in the protection scope of the computer-readable medium.

In short, the above is only an embodiment of the present application, and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made according to the disclosure of this application shall be included in the protection scope of this application.

Claims

A translation method based on voice calls, characterized in that the method includes:

The first electronic device establishes a voice call connection with the second electronic device;

When the voice call translation function is turned on, the first electronic device collects the first user's voice, reports the first user's voice to the translation server, and receives the first user's voice sent by the translation server Translate the result, and send the translation result of the voice of the first user to the second electronic device, receive the voice of the second user sent by the second electronic device, and report the second user to the translation server , And receive the translation result of the second user's voice sent by the translation server, and broadcast the translation result of the second user's voice.
The method of claim 1, wherein the first electronic device and the translation server communicate via a network, the method further comprising:

When the network signal strength is less than or equal to a preset threshold, the first electronic device collects the voice of the first user and translates the voice of the first user to obtain a translation of the voice of the first user Results; after receiving the voice of the second user, translate the voice of the second user to obtain a translation result of the voice of the second user.
The method according to claim 1 or 2, wherein before the first electronic device reports the voice of the first user or the voice of the second user to the translation server, the method further comprises:

The first electronic device determines the translation server according to the local language and the peer language, the local language is the language used by the first user, and the peer language is used by the second user language.
The method of claim 3, wherein the method further comprises:

When the first electronic device establishes a voice call connection with the second electronic device or after the first electronic device establishes a voice call connection with the second electronic device, when the local language and the peer language When different, the voice call translation function is automatically turned on.
The method according to any one of claims 1 to 3, wherein the method further comprises:

The first electronic device activates the voice call translation function in response to the user's first operation.
The method according to claim 4 or 5, wherein the method further comprises:

The first electronic device sends to the second electronic device a prompt message that the voice call translation function of the first electronic device is turned on, and the language used in the prompt message is the peer language.
The method according to any one of claims 3 to 6, wherein the method further comprises:

The first electronic device acquires the local language and the peer language;

Wherein, the local language is the system language, or the language used by the first user in the last call record of the first user and the second user, or the phone of the first user The native language of the place where the number belongs, or the native language of the place where the communication network accessed by the first electronic device is located, or the language set in response to the user operation;

The peer language is the native language of the place where the phone number of the second user belongs, or the language or the language used by the second user in the most recent call records of the first user and the second user The native language of the home place where the communication network accessed by the second electronic device is located, or the language set in response to user operation.
The method according to any one of claims 1 to 7, wherein the first electronic device broadcasts the translation result of the voice of the second user, including:

When the voice of the first user is not collected and the voice of the second user is not received, the first electronic device broadcasts the translation result of the voice of the second user.
The method of claim 2, wherein the method further comprises:

The first electronic device acquires the network signal strength when it receives the voice of the second user and does not receive the translation result of the voice of the second user sent by the translation server within the first time duration;

When the network signal strength is greater than the preset threshold, the first electronic device prompts the second user to decrease the speech rate.
The method of claim 9, wherein the first electronic device prompting the second user to decrease the speech rate includes:

The first electronic device displays first prompt information on the display screen, and the first prompt information is used to prompt the first user to notify the second reduced speech rate; or,

The first electronic device sends a first voice to the second electronic device, and the first voice is used to prompt the second user to decrease the speech rate; or,

The first electronic device plays a second voice to the first user, and the second voice is used to prompt the first user to notify the second user to decrease the speech rate.
The method according to claims 1 to 10, wherein the first electronic device broadcasts the translation result of the second user's voice, including:

When the translation result of the second user's voice buffered by the first electronic device exceeds a preset number, the translation result of the second user's voice is broadcast.
An electronic device, characterized by comprising: one or more processors and memory;

Multiple applications;

And one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer program is executed by the electronic device, the electronic device is implemented as claimed in claims 1 to 11 Any of the described methods.
A chip, characterized in that, the chip is coupled with a memory in an electronic device, so that the chip calls a computer program stored in the memory during operation to implement the method according to any one of claims 1 to 11.
A computer storage medium, wherein the computer-readable storage medium includes a computer program, and when the computer program is run on an electronic device, the electronic device is caused to perform the method according to any one of claims 1 to 11.