CN109257498B - Sound processing method and mobile terminal - Google Patents

Sound processing method and mobile terminal Download PDF

Info

Publication number
CN109257498B
CN109257498B CN201811152090.0A CN201811152090A CN109257498B CN 109257498 B CN109257498 B CN 109257498B CN 201811152090 A CN201811152090 A CN 201811152090A CN 109257498 B CN109257498 B CN 109257498B
Authority
CN
China
Prior art keywords
user
information
target
mobile terminal
microphone
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811152090.0A
Other languages
Chinese (zh)
Other versions
CN109257498A (en
Inventor
韦巍
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Vivo Mobile Communication Co Ltd
Original Assignee
Vivo Mobile Communication Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Vivo Mobile Communication Co Ltd filed Critical Vivo Mobile Communication Co Ltd
Priority to CN201811152090.0A priority Critical patent/CN109257498B/en
Publication of CN109257498A publication Critical patent/CN109257498A/en
Application granted granted Critical
Publication of CN109257498B publication Critical patent/CN109257498B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M1/00Substation equipment, e.g. for use by subscribers
    • H04M1/72Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72448User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions
    • H04M1/72454User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions according to context-related or environment-related conditions
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/32Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
    • H04L9/3226Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using a predetermined code, e.g. password, passphrase or PIN
    • H04L9/3231Biological data, e.g. fingerprint, voice or retina
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M1/00Substation equipment, e.g. for use by subscribers
    • H04M1/72Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H04M1/72409User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M1/00Substation equipment, e.g. for use by subscribers
    • H04M1/72Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H04M1/7243User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages
    • H04M1/72433User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages for voice messaging, e.g. dictaphones
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M1/00Substation equipment, e.g. for use by subscribers
    • H04M1/72Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H04M1/7243User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages
    • H04M1/72436User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages for text messaging, e.g. SMS or e-mail
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M1/00Substation equipment, e.g. for use by subscribers
    • H04M1/72Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H04M1/72442User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality for playing music files
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M1/00Substation equipment, e.g. for use by subscribers
    • H04M1/72Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72448User interfaces specially adapted for cordless or mobile telephones with means for adapting the functionality of the device according to specific conditions
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M2250/00Details of telephonic subscriber devices
    • H04M2250/74Details of telephonic subscriber devices with voice recognition means

Abstract

The invention provides a sound processing method and a mobile terminal. The method comprises the following steps: if the mobile terminal is connected with the earphone and the earphone is in an audio playing state, acquiring biological characteristic information of at least one user in a scene where the mobile terminal is located; determining a target preset user of which the biological characteristic information is matched with the biological characteristic information of the preset user in at least one user; determining a target user in the target preset users according to the priority of the scene; collecting sound information of a target user; and inputting the sound information into the earphone for playing. The invention can enable the user to hear the words of friends around while the user listens to the media audio by using the earphone.

Description

Sound processing method and mobile terminal
Technical Field
The present invention relates to the field of communications technologies, and in particular, to a sound processing method and a mobile terminal.
Background
The current mobile terminal (for example, a mobile phone) basically supports an earphone function, the earphone can be connected to the mobile phone by a wireless or wired manner, a user can answer or hang up the incoming call by manually adjusting a key of the earphone, and can also control the playing progress of applications such as video or music of the mobile phone, such as fast forward, rewind, pause, and the like, but the user inevitably encounters such an embarrassment in the process of using the earphone:
when a user listens to music or watches videos by using an earphone, the user generally does not pay attention to the surrounding environment actively, and when other people want to talk with the user, if the sound of the other people is lower than the volume of the music or the videos, the user is difficult to listen to the words spoken by the other people, so that the user cannot respond to the other people in time. In this regard, the current solution adopted by users is to pause the audio playback in the headphones to let the speaker at hand to speak the missing words again.
An effective solution is not provided at present for the problem of how to enable a user to hear the words of friends around while the user listens to media audio by using an earphone.
Disclosure of Invention
The embodiment of the invention provides a sound processing method and a mobile terminal, which aim to solve the problem that a user can not listen to media audio by using an earphone and can also hear words of friends nearby clearly in the related technology.
In order to solve the technical problem, the invention is realized as follows:
in a first aspect, an embodiment of the present invention provides a sound processing method, which is applied to a mobile terminal, and the method includes:
if the mobile terminal is connected with an earphone and the earphone is in an audio playing state, collecting biological characteristic information of at least one user in a scene where the mobile terminal is located;
determining a target preset user of the at least one user, wherein the biological characteristic information of the target preset user is matched with the biological characteristic information of the preset user;
determining a target user in the target preset users according to the priority of the scene;
collecting sound information of the target user;
and inputting the sound information into the earphone for playing.
In a second aspect, an embodiment of the present invention further provides a mobile terminal, where the mobile terminal includes:
the first acquisition module is used for acquiring the biological characteristic information of at least one user in the scene where the mobile terminal is located if the mobile terminal is connected with an earphone and the earphone is in an audio playing state;
the first determining module is used for determining a target preset user of which the biological characteristic information is matched with the biological characteristic information of the preset user in the at least one user;
the second determining module is used for determining a target user in the target preset users according to the priority of the scene;
the second acquisition module is used for acquiring the voice information of the target user;
and the input module is used for inputting the sound information to the earphone for playing.
In a third aspect, an embodiment of the present invention further provides a mobile terminal, including: a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the steps of the sound processing method.
In a fourth aspect, the embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the sound processing method.
In the embodiment of the invention, the collected biological characteristic information of the users around the mobile terminal is matched with the biological characteristic information of the friends of the users which is stored in advance, and the operation of collecting the sound information of the users around can be executed under the matching condition, namely under the condition that the friends of the users (preset users) exist in the users around; before sound information of surrounding users is collected, target users in matched target preset users can be determined according to the priority of the scene where the mobile terminal is located, so that the target users can be reasonably selected according to different scene noise sizes, sound of the target users is collected and input to an earphone to be played, and the users can hear words of friends around while listening to media audio by using the earphone.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.
FIG. 1 is a flow diagram of a sound processing method of one embodiment of the present invention;
FIG. 2 is a block diagram of a mobile terminal of one embodiment of the present invention;
fig. 3 is a schematic diagram of a hardware structure of a mobile terminal according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, a flowchart of a sound processing method according to an embodiment of the present invention is shown, and is applied to a mobile terminal, where the method may specifically include the following steps:
step 101, if the mobile terminal is connected with an earphone and the earphone is in an audio playing state, collecting biological characteristic information of at least one user in a scene where the mobile terminal is located;
the connection mode of the mobile terminal and the headset may include a wired connection, a wireless connection (e.g., bluetooth, infrared, etc.). The earphone is in the audio playing state, which means that the mobile terminal has audio output to the earphone, so that the earphone plays the audio. The audio may be any multimedia audio, such as music, video sounds, audio recordings, and so on.
When the condition that the earphone is connected and the earphone is in the audio playing state is met, the method provided by the embodiment of the invention can be used for acquiring the biological characteristic information of at least one user in the surrounding environment of the mobile terminal.
Among them, the biometric information of the present invention may include at least one of: voiceprint characteristic information, iris characteristic information and face characteristic information.
The scene in which the mobile terminal is located may be any scene, for example, a meeting room, a home, a subway, and the like, which is not limited in this respect, and the method in the embodiment of the present invention is limited to collecting the biometric features of the user around the mobile terminal in the scene, and certainly, the number of the users around the mobile terminal may be one or more.
In one example, when acquiring the voiceprint feature information, sound data of the periphery of the mobile terminal (for example, within a preset radius range with the mobile terminal positioned as a circle) may be acquired through a microphone of the mobile terminal, and then the voiceprint feature information is acquired for each sound data. Of course, any known or future developed method may be used to collect the voiceprint feature information of the surrounding users. That is to say, the method for acquiring the voiceprint feature information according to the embodiment of the present invention is not limited thereto.
In one example, when acquiring iris feature information and face feature information, a camera of the mobile terminal (including a front camera, a rear camera, or a camera at another position developed in the future) may be used to acquire portrait data of a user around the mobile terminal (for example, the mobile terminal is located in a circle and within a preset radius range), and then the iris feature information and the face feature information are extracted from the acquired face image. Of course, the iris feature information and the face feature information of the surrounding users may be collected in any known or future developed manner. That is to say, the method for acquiring the iris feature information and the face feature information according to the embodiment of the present invention is not limited thereto.
It should be noted that, since the mobile terminal user (e.g. the mobile phone owner) is also in the periphery of the mobile terminal, the biometric information of the at least one user collected in this step may include the biometric information of the mobile phone owner. However, the biometric information of the owner of the mobile phone is filtered out in step 102. Of course, a step of screening the information collected in step 101 may be added, the biometric information of the mobile phone owner is removed, and then step 102 is executed.
Step 102, determining a target preset user of which the biological characteristic information is matched with the biological characteristic information of a preset user stored in advance in the at least one user;
before step 101 is executed, the method of the embodiment of the present invention stores the biometric information of the preset user in advance. The preset users are friends of the mobile terminal users, and the number of the preset users can be one or more.
When the biological characteristic information of the preset user is stored in advance, the biological characteristic information can be set by a system or a user.
When the system setting mode is adopted, the method of the embodiment of the invention can firstly extract the relevant information from the information stored by the application program related to social contact of the mobile terminal and process the information so as to determine the biological characteristic information of the preset user.
Specifically, a buddy list that is more intimate with the mobile terminal user may be first determined according to buddy group information, address book group information, and the like extracted from the instant messaging software (wherein the buddy list may be sorted from high affinity to low affinity), wherein the affinity is high or low in relation to the contact affinity and the buddy group. Then, extracting the biological characteristic information of each friend in the friend list, for example, extracting the face image of the friend from the photo labeled with the name of the friend in the photo album, the head portrait of each contact in the address list, the head portrait of the friend of the instant messaging software and the like, and extracting the face characteristic information and the iris characteristic information from the face image; extracting the voice of the friend from the chat record of the instant messaging software, extracting voiceprint information of the voice and/or extracting the voice of the friend from the telephone recording, and extracting the voiceprint information of the voice; the names of friends and nicknames can be marked in the photo album and the sound recording. Thus, the system automatically sets and stores the friend list of the mobile terminal and the biological characteristic information of each friend in the friend list.
When a user-defined setting mode is adopted, the method provided by the embodiment of the invention can receive the human face image and/or the voice data input or selected by the mobile terminal user, then, the human face feature and/or the iris feature are extracted from the human face image, and the voiceprint feature is extracted from the voice data and is stored; and receiving the ranking information of the mobile terminal user to the plurality of friends so as to generate the ranking among the biological feature information of the plurality of friends.
Whether system settings or user-defined settings, the pre-stored biometric information of the pre-set user may be as shown in table 1. It should be noted that the pre-stored biometric information of the preset user does not necessarily include all types of biometric features, for example, the user B and the user D only include voiceprint features.
TABLE 1
Therefore, the embodiment of the invention can preset the biological characteristic information of the preset friend (namely the preset user) which is relatively close to the mobile terminal, thereby being convenient for carrying out voice processing on the words spoken by the specific friend when the method of the embodiment of the invention is executed.
When step 102 is executed, the biometric information of at least one user collected in step 101 may be matched with the preset biometric information of the preset user, so as to determine whether a preset friend of the mobile terminal user exists in the scene where the mobile terminal is located, that is, around the mobile terminal.
Specifically, for example, in step 101, a voiceprint feature a and a face feature a of the user 1 are collected; voiceprint feature B of user 2; voiceprint feature C of user 3; the voiceprint feature F of the user 4 can match the respective biometric information of the user 1, the user 2, the user 3 and the user 4 with the biometric information of each friend in the table 1, and then it can be determined through matching that the voiceprint feature a and the face feature a of the user 1 are matched with the voiceprint feature a and the face feature a of the user a, so that it can be determined that a friend of the mobile terminal user, namely the user a, exists in 4 users around the mobile terminal;
similarly, it can also be determined that a friend of the mobile terminal user, user B, exists among 4 users around the mobile terminal;
when the voiceprint feature C of the user 3 is matched with the biometric features of the respective friends in the table 1, it may be determined that the voiceprint feature C of the user 3 is matched with the voiceprint feature C of the user C, but the acquired biometric information of the user 3 does not include the face feature C in the table 1, so that the voiceprint feature of the user 3 is matched with the voiceprint feature of the user C, but the face feature of the user 3 is not matched with the face feature of the user C.
The determination criteria for matching the biometric features may be different in different embodiments, and may be matching of a part of the biometric features (i.e. matching of at least one biometric feature) or matching of all the biometric features. Compared with partial biological feature matching, the judgment standard for matching all biological features has higher accuracy for judging whether the periphery of the mobile terminal has the preset friends of the owner, and the judgment standards for matching the biological features of two users can be flexibly selected in different scenes.
Therefore, in the case that the determination criterion of the biometric matching is partial biometric matching, it can be determined that the biometric of the user 3 matches the biometric of the preset friend, user C; when the determination criterion of the biometric feature matching is all the biometric matches (that is, all the biometric features configured in advance for a certain preset friend in table 1 are matched with the biometric features of all the types of the users around the mobile terminal), since only the voiceprint feature C is included in the biometric information of the user 3 acquired in step 101, but the face feature C is not included, when the biometric information of the friends in table 1 is matched, only the voiceprint feature C of the user 3 can be matched with the voiceprint feature C of the user C, but the user 3 does not have the face feature that can be matched with the face feature C of the user C, so the user 3 does not match the preset friend whose biometric feature is matched in table 1.
The voiceprint feature F of the user 4 is not matched with the voiceprint feature of any friend in the table 1, so that the user 4 is not a target preset user whose biometric information is matched with the biometric information of the preset user.
Wherein, for the matching of two sets of features of the same biometric feature, the matching degree of the two sets of features may be greater than a preset threshold (e.g. 80%).
Here, the determination criterion of the matching of the biological characteristics is partial biological matching, that is, the biological characteristics of the two users are determined to be matched, and then it can be determined through this step that the biological characteristic information of the user 1, the user 2, and the user 3 among the four users 1 to 4 are respectively matched with the biological characteristics of 3 preset users (here, the user a, the user B, and the user C, respectively), so that the target preset users are the user 1 (that is, the user a), the user 2 (that is, the user B), and the user 3 (that is, the user C), respectively.
103, determining a target user in the target preset users according to the priority of the scene;
wherein the priority of the scene is related to the ambient noise of the scene;
in step 101, only the biometric information of the user around the mobile terminal (i.e., in the scene) is collected, and it is not determined what scene the mobile terminal is located in, so in this step, the location position of the mobile terminal may be obtained, and then a target coordinate range including the location position is searched in a preset coordinate range corresponding to different scenes (or a target scene where the location position is located is directly searched by using a map system), so as to determine the target scene corresponding to the target coordinate range, where the target scene is the scene where the mobile terminal is located.
The preset scenes include but are not limited to meeting rooms, mobile terminal user homes, subways and the like.
The embodiment of the invention can set the priority for each scene in advance, wherein the priority of the scene is related to the environmental noise of the scene;
therefore, which friend of the target preset user (i.e. the surrounding friends) is the friend needing to collect the sound can be determined according to the magnitude of the environmental noise of the scene.
In a scene with high environmental noise, the owner often only pays attention to the sound of a small number of friends; in a scene with low environmental noise, an owner often pays attention to the sound of a large number of surrounding friends, so that the target users needing to collect the sound in the target preset users can be determined according to the priority of the scene, wherein the number of the target users can be one or more.
Step 104, collecting the voice information of the target user;
for example, if the target user is a user, for example, user a, the voiceprint feature a of user a in table 1 can be used to collect the sound information belonging to user a in the surrounding sounds;
if the target user is a plurality of users, the voice information sent by each target user can be collected by using the voiceprint characteristics of each target user in the table 1;
in addition, when this step is executed, not only the sound information of the target but also the sounds made by other users or nature except the target user may be collected, but these are not the sounds of main concern, and therefore, it is necessary to perform signal attenuation on the collected sounds, so as to avoid that the sound of the target user concerned by the user is not obvious.
And 105, inputting the sound information into the earphone for playing.
When the sound information collected in step 104 is input to the earphone for playing, the process is a real-time input process, and along with the real-time collection of the sound information in step 104, the collected sound information is input to the earphone in real time, so that the sound of the target user played by the earphone is played sequentially according to the time sequence of the sound emitted by the target user, and the speaking sound of the peripheral target user is restored in the earphone.
In one embodiment, when sound information is input to the earphone for playing, multimedia audio (e.g., music playing pause, video playing pause) being played by the earphone can be paused, so that the sound played by the earphone is only the speaking sound of the target user (friend) concerned by the surrounding user.
In another embodiment, when the sound information is input to the earphone for playing, the multimedia audio being played by the earphone (e.g. music playing pause, video playing pause) may not be paused, so that the earphone plays the multimedia audio and also plays the speaking sound of the target user, and thus, the user can hear the speaking sound of the surrounding user and simultaneously does not affect the playing of the multimedia audio being listened to by the user.
In the embodiment of the invention, the collected biological characteristic information of the users around the mobile terminal is matched with the biological characteristic information of the friends of the users which is stored in advance, and the operation of collecting the sound information of the users around can be executed under the matching condition, namely under the condition that the friends of the users (preset users) exist in the users around; before sound information of surrounding users is collected, target users in matched target preset users can be determined according to the priority of the scene where the mobile terminal is located, so that the target users can be reasonably selected according to different scene noise sizes, sound of the target users is collected and input to an earphone to be played, and the users can hear words of friends around while listening to media audio by using the earphone.
Optionally, in an embodiment, when step 103 is executed, it may be implemented by:
determining the priority of the scene where the mobile terminal is located according to the preset priorities of different scenes;
different priorities can be set for different scenes according to the environmental noise of the scenes;
for example, the set scene priority is proportional to the ambient noise, and the scenes with the priorities from low to high are ordered as follows: home (priority 1), conference room (priority 2), company (priority 3), subway (priority 4), near construction site (priority 5);
if the set scene priority is inversely proportional to the environmental noise, the scenes with the priorities from high to low are ordered as follows: home (priority 1), conference room (priority 2), company (priority 3), subway (priority 4), near construction site (priority 5);
if the priority of the scene belongs to the first priority, determining the target preset user as a target user;
wherein the first priority represents that the noise value of the environmental noise of the scene is smaller, and the second priority represents that the noise value of the environmental noise of the scene is larger.
The setting of the scene priority is described as an example in which the scene priority is inversely proportional to the noise value of the environmental noise. For example, priorities with a preset priority value less than or equal to 3 are all of the first priority, and priorities with a priority value greater than 3 are all of the second priority.
For example, if the scene where the mobile terminal is located is a home, the priority of the scene is 1, and the scene belongs to the first priority, which indicates that the environmental noise of the scene where the mobile terminal is located is small, so that a multi-user mode can be selected for sound collection, that is, the sounds of all friends in the scene need to be collected, and therefore the target preset user is a target user, such as a user a, a user B, and a user C.
Preferably, when the target preset user is determined as the target user, if the number of the target preset users is multiple, after the sound information of the first target preset user (i.e. the first speaking friend) is collected, the sound information is subjected to context analysis (e.g. the sound information is converted into text and semantic keywords are extracted, e.g. user a says "has eaten at noon today", the keywords include eating, noon, and meal), and the context of the sound information is determined, so if the context of the spoken words of other target preset users is the same as the context of the spoken words of the first target preset user (e.g. user B says "i have eaten at today", the sentence includes at least one of the above-mentioned keywords), user a and user B are both target users, and if user C says "true weather today |)! "this user C is not the target user here, although it is a friend of the mobile terminal user, because its context is different from that of the first target preset user.
Therefore, in the multi-user mode, the embodiment of the invention collects the spoken sound information of the friend who speaks in the same context as the target user by establishing the virtual context circle of the friend, so that the mobile terminal user can know the conversation content of the surrounding friend to any one context.
If the priority of the scene belongs to a second priority and the number of the target preset users is one, determining the target preset users as target users;
for example, if the scene where the mobile terminal is located is in a subway, the priority of the scene is 4, and the scene belongs to the second priority, which indicates that the environment noise of the scene where the mobile terminal is located is more and larger, so that a single-person mode can be selected for sound collection, that is, the sound of all friends in the scene does not need to be collected completely, and only the sound of one friend is collected, where the number of target preset users is 1, that is, only one friend of the user exists around, and then the friend can be determined as the target user needing to collect the sound.
If the priority of the scene belongs to a second priority and the number of the target preset users is multiple, determining the target preset user with the highest contact affinity with the mobile terminal user in the multiple target preset users as a target user;
for example, if the scene where the mobile terminal is located is in a subway, the priority of the scene is 4, and the scene belongs to the second priority, which indicates that the environment noise of the scene where the mobile terminal is located is more, so that a single-person mode can be selected for sound collection, that is, the sounds of all friends in the scene do not need to be collected completely, and only the sound of one friend is collected, where the number of target preset users is multiple, that is, multiple friends of the user exist around the scene, and therefore, the target preset user with the highest affinity with the mobile terminal user (see table 1, it can be known that the affinity of the user a is the highest) among the multiple target preset users (e.g., the users a to C) can be determined as the target user (the user a).
The contact density here can be determined according to parameters such as contact frequency of preset time, tags to friends, and the like, and will not be described here again.
The environmental noise corresponding to the first priority is lower than the environmental noise corresponding to the second priority, that is, the environmental noise of the scene corresponding to the first priority is lower than the environmental noise of the scene corresponding to the second priority.
It should be noted that, in a specific implementation, the embodiment of the present invention may determine whether to adopt the single-person mode or the multi-person mode to acquire the voice information of the target user according to the priority (that is, the environmental noise) of the scene. In the single-person mode, the number of the target users is one, and in the multi-person mode, the number of the target users is multiple.
The priority of the scene belongs to the first priority, which indicates that the environmental noise is small, so that the mobile terminal user generally pays attention to the speaking voice of all the preset friends in the scene, and a multi-user mode is adopted.
If the priority of the scene belongs to the second priority, the environmental noise is relatively high, so that the user can hear the words spoken by the concerned friends clearly, a single person mode can be adopted, namely, only one friend is selected from the surrounding friends as a target user to collect the sound.
In this way, the embodiment of the invention determines whether one or all the friends in the peripheral target preset user are used as the objects for collecting the sound according to the environmental noise condition, namely the priority, of the scene where the mobile terminal is located, so that the sound of only one friend can be collected in the scene with larger environmental noise, and the sound of all the friends is collected in the scene with smaller environmental noise, thereby meeting the requirement of the user on sound collection and ensuring the clarity of the collected sound heard by the user.
Optionally, in one embodiment, when performing step 104, it may be implemented by:
in step 104, the method of the present embodiment is adopted to collect the sound of each target user regardless of whether the number of target users is one or more, that is, whether a single-person mode or a multi-person mode is selected.
Acquiring the real-time positioning of the target user;
the method comprises the steps that images of a target user can be collected in real time through a camera of a mobile terminal, and therefore the real-time positioning of the target user is determined based on the positioning of the mobile terminal; alternatively, the real-time location of the target user can be determined by acquiring GPS location information of the mobile terminal of the target user. Of course, the method for obtaining the real-time location of the target user is not limited to the two methods listed here, and may also include known methods that are not listed and other methods that are developed in the future, which is not described in detail herein.
Determining a first distance between a location at which a first microphone of the mobile terminal is located and the real-time location;
determining a second distance between a location at which a second microphone of the mobile terminal is located and the real-time location;
if the first distance is smaller than the second distance, determining the first microphone as a main microphone and determining the second microphone as a secondary microphone;
if the first distance is greater than the second distance, determining the second microphone as a primary microphone and the first microphone as a secondary microphone;
controlling the main microphone to collect the voice information of the target user, controlling the main microphone to perform signal enhancement on the collected voice information, controlling the auxiliary microphone to collect voice information except the voice information of the target user, and controlling the auxiliary microphone to perform signal attenuation on the collected voice information;
and extracting the sound information processed by the main microphone and the auxiliary microphone.
Specifically, in order to make the sound of the target user input to the headphones clearer, the method of the embodiment of the present invention may perform noise reduction processing on the sound source of the target user.
First, it is necessary to determine the respective distances between the two microphones of the mobile terminal and the real-time location of the target user, i.e. the first distance and the second distance, and to use the microphone with the closer distance between the location and the real-time location of the target user as the primary microphone and the other microphone as the secondary microphone.
For example, the two microphones are a microphone a and a microphone B, respectively, and the sound source position is C, and if the distance from the microphone a to the sound source position C is greater than the distance from the microphone B to the sound source position C, the microphone B is the primary microphone and the microphone a is the secondary microphone. Conversely, if the distance from microphone a to sound source position C is less than the distance from microphone B to sound source position C, then microphone a is the primary microphone and microphone B is the negative microphone.
Then, the method of the embodiment of the present invention may process the sound collected by the primary microphone and the secondary microphone, specifically, for the primary microphone, the primary microphone may be controlled to collect the sound information of the target user, and specifically, the method of controlling the primary microphone to collect the sound information of the target user may be to compare and match the voiceprint feature of the collected sound information with the voiceprint feature of the target user, so as to collect only the sound information, which is the sound information of the target user and has the voiceprint feature capable of being matched with the voiceprint feature of the target user. In addition, the main microphone is also controlled to perform signal enhancement on the collected voice information of the target user, and the method for enhancing the voice signal can be any one of the known technologies, which is not limited by the present invention.
For the secondary microphone, the secondary microphone may be controlled to acquire sound information other than the sound information of the target user, and the specific control method may be to extract voiceprint features from the sound information acquired by the secondary microphone, and compare and match the extracted voiceprint features with the voiceprint features of the target user, so as to acquire only sound information whose voiceprint features cannot be matched with the voiceprint features of the target user, that is, sound information other than the target user. Therefore, the sound information collected by the secondary microphone may include sound information of other users, and may also include noise of the environment. In addition, the embodiment of the present invention further controls the secondary microphone to perform signal attenuation on the collected sound information except for the target user, and the method of sound signal attenuation may be any one of the known technologies, which is not limited in the present invention.
And finally, extracting the voice information of the target user after the main microphone is processed, and extracting the voice information except the target user after the auxiliary microphone is processed to obtain the voice of the target user with enhanced signals and the voice of other noises after the signals are attenuated. Finally, in step 105, the sound input to the earphone includes the sound processed by the main microphone and the sound processed by the sub-microphone.
In this way, in the embodiment of the present invention, the microphone closer to the sound source position of the target user is used as the primary microphone, the microphone farther from the sound source position is used as the secondary microphone, the primary microphone is controlled to collect the sound information of the target user and perform signal enhancement on the sound information, the secondary microphone is controlled to collect other sound information and perform signal attenuation on the sound information, so that the sound of the target user concerned by the user in the sound extracted and input to the earphone is clearer and more obvious, and the user can listen to the extracted sound information of the target user conveniently. In addition, the main microphone is closer to the sound source, so that the collected sound information of the target user is clearer, and the definition of the sound information of the target user output by the earphone is further improved.
It should be noted that, when the number of target users is multiple, the main microphones corresponding to different target users may be different due to different positioning of different target users, for example, when the voice information of the user a is collected, the main microphone is the microphone a, and when the voice information of the user C is collected, the main microphone is the microphone B. In addition, when the number of the target users is multiple, the method of the embodiment of the invention can be respectively adopted for each target user to collect the voice information of the target user, and because the multiple target users are generally known mutually, the speaking time between the multiple target users is not overlapped generally, therefore, the method of the embodiment of the invention can collect the voice information and carry out signal enhancement and environmental sound signal reduction on the different target users when the different target users speak, so that the voice of each target user which is input to the earphone in sequence according to the speaking sequence is clear.
Optionally, in an embodiment, after step 104, the method according to an embodiment of the present invention may further include:
sending reminding information which indicates whether a mobile terminal user is asked to check surrounding chat information or not according to a preset time period;
the method comprises the steps that timing can be started after sound information of a target user is collected, and when the timing duration reaches the preset time period T, reminding information can be sent in a notification window, wherein the reminding information indicates that whether a mobile terminal user checks surrounding chat information or not is inquired, namely that a user is informed that friends speak around the user, and the user can select whether to check or not. Such as pushing the reminder information once every thirty minutes.
Under the condition that response information which is used for responding to the reminding information and represents that the surrounding chat information is viewed is received, a plurality of selectable chat information display modes are provided;
for example, if the reminding message has two options of "yes" and "no", and the user selects "yes", response information indicating that the user views the surrounding chat information in response to the reminding message may be received, so that the method of the embodiment of the present invention may provide a plurality of chat information presentation modes that may be selected by the user. The plurality of chat message display modes include, but are not limited to, text and voice. When the plurality of chat information display modes are provided, a text button and a voice button can be displayed for the user to select the chat information display mode.
Determining a selected target chat information display mode in the plurality of chat information display modes;
wherein, if the user selects the 'text' button, the target chat information display mode can be determined to be a text display mode.
If the user selects the voice button, the target chat information display mode can be determined to be a voice playing mode.
If the chat information display mode is a text display mode, converting the voice information of the target user into text information, and sequentially displaying the text information according to the sequence of the acquisition time of the voice information from morning to evening;
if the user selects the text option, the sound information of the target user collected in step 104 may be converted into text information, and since a plurality of pieces of sound information of the target user (where the target user may be a plurality of users) have been collected within a preset time period, the text information of the corresponding sound information may be sequentially displayed in the order from morning to evening according to the collection time of the sound information, so that the user can conveniently know the whole session.
Correspondingly, when step 105 is executed, if the chat information display mode is a voice playing mode, the voice information is sequentially input to the earphones to be played according to the sequence of the acquisition time of the voice information from morning to evening.
If the user selects the voice option, because a plurality of pieces of sound information of the target user (wherein, the target user may be a plurality of pieces of sound information) have been collected within a preset time period, the sound information collected in step 104 may be sequentially input to the earphone to be played according to the sequence of the collection time of the sound information from morning to evening, so that the user can conveniently know the whole conversation.
Therefore, after the voice information of the target user is collected, the voice information can not be immediately input into the earphone, the voice information can be periodically pushed according to a preset time period, and when the user wants to check the voice information, the voice information of the target user can be displayed in a mode of text display or voice playing input into the earphone according to the checking mode selected by the user, so that the voice information of the target user can be played more flexibly, and the problem that the audio and video which is listened by the user is interfered when the target user inputs the speaking voice into the earphone every time is avoided.
Referring to fig. 2, a block diagram of a mobile terminal of one embodiment of the present invention is shown. The mobile terminal of the embodiment of the invention can realize the details of the sound processing method in the embodiment and achieve the same effect. The mobile terminal shown in fig. 2 includes:
the first acquisition module 21 is configured to acquire biometric information of at least one user in a scene where the mobile terminal is located if the mobile terminal is connected with an earphone and the earphone is in an audio playing state;
a first determining module 22, configured to determine a target preset user of the at least one user whose biometric information matches biometric information of a preset user;
a second determining module 23, configured to determine a target user of the target preset users according to the priority of the scene;
the second acquisition module 24 is configured to acquire sound information of the target user;
and the input module 25 is configured to input the sound information to the earphone for playing.
Optionally, the second determining module 23 includes:
the first determining submodule is used for determining the priority of the scene where the mobile terminal is located according to the preset priorities of different scenes;
the second determining submodule is used for determining the target preset user as a target user if the priority of the scene belongs to the first priority;
a third determining submodule, configured to determine the target preset user as the target user if the priority of the scene belongs to the second priority and the number of the target preset users is one;
a fourth determining submodule, configured to determine, if the priority of the scene belongs to the second priority and the number of the target preset users is multiple, a target preset user with a highest affinity with the mobile terminal user among the multiple target preset users as a target user;
wherein the environmental noise corresponding to the first priority is lower than the environmental noise corresponding to the second priority.
Optionally, the second acquisition module 24 includes:
the acquisition submodule is used for acquiring the real-time positioning of the target user;
a fifth determining submodule, configured to determine a first distance between the location where the first microphone of the mobile terminal is located and the real-time location;
a sixth determining submodule, configured to determine a second distance between the location where the second microphone of the mobile terminal is located and the real-time location;
a seventh determining submodule, configured to determine the first microphone as a primary microphone and the second microphone as a secondary microphone if the first distance is smaller than the second distance;
an eighth determining submodule, configured to determine the second microphone as a primary microphone and the first microphone as a secondary microphone if the first distance is greater than the second distance;
the control submodule is used for controlling the main microphone to collect the sound information of the target user, controlling the main microphone to carry out signal enhancement on the collected sound information, controlling the auxiliary microphone to collect the sound information except the sound information of the target user, and controlling the auxiliary microphone to carry out signal attenuation on the collected sound information;
and the extraction submodule is used for extracting the sound information processed by the main microphone and the auxiliary microphone.
Optionally, the mobile terminal further includes:
the sending module is used for sending reminding information which indicates whether the mobile terminal user is inquired about to check the surrounding chat information according to a preset time period;
the providing module is used for providing a plurality of selectable chat information display modes under the condition of receiving response information which is used for responding to the reminding information and representing the viewing of the surrounding chat information;
a third determining module, configured to determine a selected target chat information display mode from the multiple chat information display modes;
the display module is used for converting the voice information of the target user into text information and sequentially displaying the text information according to the sequence of the acquisition time of the voice information from morning to evening if the target chat information display mode is a text display mode;
the input module 25 is further configured to, if the target chat information display mode is a voice playing mode, sequentially input the voice information to the earphones for playing according to the sequence of the acquisition time of the voice information from morning to evening.
The mobile terminal provided by the embodiment of the present invention can implement each process implemented by the mobile terminal in the above method embodiments, and is not described herein again to avoid repetition.
In the embodiment of the invention, the collected biological characteristic information of the users around the mobile terminal is matched with the biological characteristic information of the friends of the users which is stored in advance, and the operation of collecting the sound information of the users around can be executed under the matching condition, namely under the condition that the friends of the users (preset users) exist in the users around; before sound information of surrounding users is collected, target users in matched target preset users can be determined according to the priority of the scene where the mobile terminal is located, so that the target users can be reasonably selected according to different scene noise sizes, sound of the target users is collected and input to an earphone to be played, and the users can hear words of friends around while listening to media audio by using the earphone.
Figure 3 is a schematic diagram of a hardware structure of a mobile terminal implementing various embodiments of the present invention,
the mobile terminal 300 has a screen fingerprint recognition function, and the mobile terminal 300 includes but is not limited to: radio frequency unit 301, network module 302, audio output unit 303, input unit 304, sensor 305, display unit 306, user input unit 307, interface unit 308, memory 309, processor 310, and power supply 311. Those skilled in the art will appreciate that the mobile terminal architecture shown in fig. 3 is not intended to be limiting of mobile terminals, and that a mobile terminal may include more or fewer components than shown, or some components may be combined, or a different arrangement of components. In the embodiment of the present invention, the mobile terminal includes, but is not limited to, a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted terminal, a wearable device, a pedometer, and the like.
The radio frequency unit 301 is configured to acquire biometric information of at least one user in a scene where the mobile terminal is located if the mobile terminal is connected with an earphone and the earphone is in an audio playing state; collecting sound information of the target user; and inputting the sound information into the earphone for playing.
A processor 310, configured to determine a target preset user of the at least one user whose biometric information matches biometric information of a preset user; and determining a target user in the target preset users according to the priority of the scene.
In the embodiment of the invention, the collected biological characteristic information of the users around the mobile terminal is matched with the biological characteristic information of the friends of the users which is stored in advance, and the operation of collecting the sound information of the users around can be executed under the matching condition, namely under the condition that the friends of the users (preset users) exist in the users around; before sound information of surrounding users is collected, target users in matched target preset users can be determined according to the priority of the scene where the mobile terminal is located, so that the target users can be reasonably selected according to different scene noise sizes, sound of the target users is collected and input to an earphone to be played, and the users can hear words of friends around while listening to media audio by using the earphone.
It should be understood that, in the embodiment of the present invention, the radio frequency unit 301 may be used for receiving and sending signals during a message sending and receiving process or a call process, and specifically, receives downlink data from a base station and then processes the received downlink data to the processor 310; in addition, the uplink data is transmitted to the base station. In general, radio frequency unit 301 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 301 can also communicate with a network and other devices through a wireless communication system.
The mobile terminal provides the user with wireless broadband internet access through the network module 302, such as helping the user send and receive e-mails, browse webpages, access streaming media, and the like.
The audio output unit 303 may convert audio data received by the radio frequency unit 301 or the network module 302 or stored in the memory 309 into an audio signal and output as sound. Also, the audio output unit 303 may also provide audio output related to a specific function performed by the mobile terminal 300 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 303 includes a speaker, a buzzer, a receiver, and the like.
The input unit 304 is used to receive audio or video signals. The input Unit 304 may include a Graphics Processing Unit (GPU) 3041 and a microphone 3042, and the Graphics processor 3041 processes image data of a still picture or video obtained by an image capturing apparatus (e.g., a camera) in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 306. The image frames processed by the graphic processor 3041 may be stored in the memory 309 (or other storage medium) or transmitted via the radio frequency unit 301 or the network module 302. The microphone 3042 may receive sounds and may be capable of processing such sounds into audio data. The processed audio data may be converted into a format output transmittable to a mobile communication base station via the radio frequency unit 301 in case of the phone call mode.
The mobile terminal 300 also includes at least one sensor 305, such as a light sensor, motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor that adjusts the brightness of the display panel 3061 according to the brightness of ambient light, and a proximity sensor that turns off the display panel 3061 and/or a backlight when the mobile terminal 300 is moved to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), detect the magnitude and direction of gravity when stationary, and can be used to identify the posture of the mobile terminal (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), and vibration identification related functions (such as pedometer, tapping); the sensors 105 may also include fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which are not described in detail herein.
The display unit 306 is used to display information input by the user or information provided to the user. The Display unit 306 may include a Display panel 3061, and the Display panel 3061 may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like.
The user input unit 307 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the mobile terminal. Specifically, the user input unit 307 includes a touch panel 3071 and other input devices 3072. The touch panel 3071, also referred to as a touch screen, may collect touch operations by a user on or near the touch panel 3071 (e.g., operations by a user on or near the touch panel 3071 using a finger, a stylus, or any suitable object or attachment). The touch panel 3071 may include two parts of a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 310, and receives and executes commands sent by the processor 310. In addition, the touch panel 3071 may be implemented using various types, such as resistive, capacitive, infrared, and surface acoustic wave. The user input unit 307 may include other input devices 3072 in addition to the touch panel 3071. Specifically, the other input devices 3072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which are not described herein.
Further, the touch panel 3071 may be overlaid on the display panel 3061, and when the touch panel 3071 detects a touch operation on or near the touch panel, the touch operation is transmitted to the processor 310 to determine the type of the touch event, and then the processor 310 provides a corresponding visual output on the display panel 3061 according to the type of the touch event. Although the touch panel 3071 and the display panel 3061 are shown as two separate components in fig. 3 to implement the input and output functions of the mobile terminal, in some embodiments, the touch panel 3071 and the display panel 3061 may be integrated to implement the input and output functions of the mobile terminal, which is not limited herein.
The interface unit 308 is an interface through which an external device is connected to the mobile terminal 300. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 308 may be used to receive input (e.g., data information, power, etc.) from external devices and transmit the received input to one or more elements within the mobile terminal 300 or may be used to transmit data between the mobile terminal 300 and external devices.
The memory 309 may be used to store software programs as well as various data. The memory 309 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. Further, the memory 309 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.
The processor 310 is a control center of the mobile terminal, connects various parts of the entire mobile terminal using various interfaces and lines, and performs various functions of the mobile terminal and processes data by operating or executing software programs and/or modules stored in the memory 309 and calling data stored in the memory 309, thereby performing overall monitoring of the mobile terminal. Processor 310 may include one or more processing units; preferably, the processor 310 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 310.
The mobile terminal 300 may further include a power supply 311 (such as a battery) for supplying power to various components, and preferably, the power supply 311 may be logically connected to the processor 310 through a power management system, so as to implement functions of managing charging, discharging, and power consumption through the power management system.
In addition, the mobile terminal 300 includes some functional modules that are not shown, and thus, the detailed description thereof is omitted.
Preferably, an embodiment of the present invention further provides a mobile terminal, which includes a processor 310, a memory 309, and a computer program stored in the memory 309 and capable of running on the processor 310, where the computer program, when executed by the processor 310, implements each process of the sound processing method embodiment, and can achieve the same technical effect, and in order to avoid repetition, details are not described here again.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements each process of the sound processing method embodiment, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here. The computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.

Claims (8)

1. A sound processing method is applied to a mobile terminal, and is characterized by comprising the following steps:
the method comprises the steps that under the condition that the mobile terminal is connected with an earphone and the earphone is in an audio playing state, biological characteristic information of at least one user in the scene where the mobile terminal is located is collected;
determining a target preset user of the at least one user, wherein the biological characteristic information of the target preset user is matched with the biological characteristic information of the preset user;
determining a target user in the target preset users according to the priority of the scene; wherein the priority of the scene is related to the ambient noise of the scene;
collecting sound information of the target user;
inputting the sound information to the earphone for playing;
the collecting the voice information of the target user comprises:
acquiring the real-time positioning of the target user;
determining a first distance between a location at which a first microphone of the mobile terminal is located and the real-time location;
determining a second distance between a location at which a second microphone of the mobile terminal is located and the real-time location;
determining the first microphone as a primary microphone and the second microphone as a secondary microphone if the first distance is less than the second distance;
determining the second microphone as a primary microphone and the first microphone as a secondary microphone if the first distance is greater than the second distance;
controlling the main microphone to collect the voice information of the target user, controlling the main microphone to perform signal enhancement on the collected voice information, controlling the auxiliary microphone to collect voice information except the voice information of the target user, and controlling the auxiliary microphone to perform signal attenuation on the collected voice information;
and extracting the sound information processed by the main microphone and the auxiliary microphone.
2. The method according to claim 1, wherein the determining a target user of the target preset users according to the priority of the scene comprises:
determining the priority of the scene where the mobile terminal is located according to the preset priorities of different scenes;
determining the target preset user as a target user under the condition that the priority of the scene belongs to the first priority;
determining the target preset user as a target user under the condition that the priority of the scene belongs to a second priority and the number of the target preset users is one;
determining a target preset user with the highest contact affinity with the mobile terminal user from the plurality of target preset users as a target user under the condition that the priority of the scene belongs to a second priority and the number of the target preset users is multiple;
wherein the environmental noise corresponding to the first priority is lower than the environmental noise corresponding to the second priority.
3. The method of claim 1, wherein after the step of collecting the voice information of the target user, the method further comprises:
sending reminding information which indicates whether a mobile terminal user is asked to check surrounding chat information or not according to a preset time period;
under the condition that response information which is used for responding to the reminding information and represents that the surrounding chat information is viewed is received, a plurality of selectable chat information display modes are provided;
determining a selected target chat information display mode in the plurality of chat information display modes;
under the condition that the target chat information display mode is a text display mode, converting the voice information of the target user into text information, and sequentially displaying the text information according to the sequence of the acquisition time of the voice information from morning to evening;
the inputting the sound information into the earphone for playing comprises:
and under the condition that the target chat information display mode is a voice playing mode, sequentially inputting the voice information to the earphone to be played according to the sequence of the acquisition time of the voice information from morning to evening.
4. A mobile terminal, characterized in that the mobile terminal comprises:
the first acquisition module is used for acquiring the biological characteristic information of at least one user in the scene where the mobile terminal is located if the mobile terminal is connected with an earphone and the earphone is in an audio playing state;
the first determining module is used for determining a target preset user of which the biological characteristic information is matched with the biological characteristic information of the preset user in the at least one user;
the second determining module is used for determining a target user in the target preset users according to the priority of the scene;
the second acquisition module is used for acquiring the voice information of the target user;
the input module is used for inputting the sound information to the earphone for playing;
the second acquisition module comprises:
the acquisition submodule is used for acquiring the real-time positioning of the target user;
a fifth determining submodule, configured to determine a first distance between the location where the first microphone of the mobile terminal is located and the real-time location;
a sixth determining submodule, configured to determine a second distance between the location where the second microphone of the mobile terminal is located and the real-time location;
a seventh determining submodule, configured to determine the first microphone as a primary microphone and the second microphone as a secondary microphone if the first distance is smaller than the second distance;
an eighth determining submodule, configured to determine the second microphone as a primary microphone and the first microphone as a secondary microphone if the first distance is greater than the second distance;
the control submodule is used for controlling the main microphone to collect the sound information of the target user, controlling the main microphone to carry out signal enhancement on the collected sound information, controlling the auxiliary microphone to collect the sound information except the sound information of the target user, and controlling the auxiliary microphone to carry out signal attenuation on the collected sound information;
and the extraction submodule is used for extracting the sound information processed by the main microphone and the auxiliary microphone.
5. The mobile terminal of claim 4, wherein the second determining module comprises:
the first determining submodule is used for determining the priority of the scene where the mobile terminal is located according to the preset priorities of different scenes;
the second determining submodule is used for determining the target preset user as a target user if the priority of the scene belongs to the first priority;
a third determining submodule, configured to determine the target preset user as the target user if the priority of the scene belongs to the second priority and the number of the target preset users is one;
a fourth determining submodule, configured to determine, if the priority of the scene belongs to the second priority and the number of the target preset users is multiple, a target preset user with a highest affinity with the mobile terminal user among the multiple target preset users as a target user;
wherein the environmental noise corresponding to the first priority is lower than the environmental noise corresponding to the second priority.
6. The mobile terminal of claim 4, wherein the mobile terminal further comprises:
the sending module is used for sending reminding information which indicates whether the mobile terminal user is inquired about to check the surrounding chat information according to a preset time period;
the providing module is used for providing a plurality of selectable chat information display modes under the condition of receiving response information which is used for responding to the reminding information and representing the viewing of the surrounding chat information;
a third determining module, configured to determine a selected target chat information display mode from the multiple chat information display modes;
the display module is used for converting the voice information of the target user into text information and sequentially displaying the text information according to the sequence of the acquisition time of the voice information from morning to evening if the target chat information display mode is a text display mode;
the input module is further configured to, if the target chat information display mode is a voice playing mode, sequentially input the voice information to the earphone for playing according to the sequence of the acquisition time of the voice information from morning to evening.
7. A mobile terminal, comprising: memory, processor and computer program stored on the memory and executable on the processor, which computer program, when executed by the processor, carries out the steps of the sound processing method as claimed in any one of claims 1 to 3.
8. A computer-readable storage medium, characterized in that a computer program is stored thereon, which computer program, when being executed by a processor, realizes the steps in the sound processing method according to any one of claims 1 to 3.
CN201811152090.0A 2018-09-29 2018-09-29 Sound processing method and mobile terminal Active CN109257498B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811152090.0A CN109257498B (en) 2018-09-29 2018-09-29 Sound processing method and mobile terminal

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811152090.0A CN109257498B (en) 2018-09-29 2018-09-29 Sound processing method and mobile terminal

Publications (2)

Publication Number Publication Date
CN109257498A CN109257498A (en) 2019-01-22
CN109257498B true CN109257498B (en) 2021-01-08

Family

ID=65044841

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811152090.0A Active CN109257498B (en) 2018-09-29 2018-09-29 Sound processing method and mobile terminal

Country Status (1)

Country Link
CN (1) CN109257498B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021159369A1 (en) * 2020-02-13 2021-08-19 深圳市汇顶科技股份有限公司 Hearing aid method and apparatus for noise reduction, chip, earphone and storage medium
CN111343541A (en) * 2020-04-15 2020-06-26 Oppo广东移动通信有限公司 Control method and device of wireless earphone, mobile terminal and storage medium
CN112907900B (en) * 2021-01-29 2022-06-24 中煤科工集团重庆研究院有限公司 Slope monitoring entity risk early warning assessment model

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103716724A (en) * 2012-09-28 2014-04-09 联想(北京)有限公司 Sound collection method and electronic device
CN104158970A (en) * 2010-02-20 2014-11-19 华为终端有限公司 Environmental sound prompting method and mobile terminal

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4507450B2 (en) * 2001-05-14 2010-07-21 ソニー株式会社 Communication device and method, recording medium, and program
CN204633973U (en) * 2015-04-09 2015-09-09 深圳市豪恩声学股份有限公司 Earphone and speech recognition equipment thereof
CN107404682B (en) * 2017-08-10 2019-11-05 京东方科技集团股份有限公司 A kind of intelligent earphone
CN108391206A (en) * 2018-03-30 2018-08-10 广东欧珀移动通信有限公司 Signal processing method, device, terminal, earphone and readable storage medium storing program for executing
CN108540661A (en) * 2018-03-30 2018-09-14 广东欧珀移动通信有限公司 Signal processing method, device, terminal, earphone and readable storage medium storing program for executing
CN108521621B (en) * 2018-03-30 2020-01-10 Oppo广东移动通信有限公司 Signal processing method, device, terminal, earphone and readable storage medium

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104158970A (en) * 2010-02-20 2014-11-19 华为终端有限公司 Environmental sound prompting method and mobile terminal
CN103716724A (en) * 2012-09-28 2014-04-09 联想(北京)有限公司 Sound collection method and electronic device

Also Published As

Publication number Publication date
CN109257498A (en) 2019-01-22

Similar Documents

Publication Publication Date Title
CN109257498B (en) Sound processing method and mobile terminal
CN106973330B (en) Screen live broadcasting method, device and system
CN108234289B (en) Message display method and device and mobile terminal
CN110768805B (en) Group message display method and electronic equipment
CN108074574A (en) Audio-frequency processing method, device and mobile terminal
CN111444441A (en) Information prompting method, electronic equipment and storage medium
CN108462794B (en) Information display method and mobile terminal
CN108093119B (en) Strange incoming call number marking method and mobile terminal
CN111447223A (en) Call processing method and electronic equipment
CN111491058A (en) Method for controlling operation mode, electronic device, and storage medium
CN109889756B (en) Video call method and terminal equipment
CN108347527B (en) Incoming call prompting method and mobile terminal
CN108040003B (en) Reminding method and device
CN109348035A (en) A kind of recognition methods of telephone number and terminal device
CN107786427B (en) Information interaction method, terminal and computer readable storage medium
CN111277675B (en) Group member sorting method and electronic equipment
CN108763475B (en) Recording method, recording device and terminal equipment
CN109922199B (en) Contact information processing method and terminal
CN109729210B (en) Information display method and terminal equipment
CN112532787A (en) Earphone audio data processing method, mobile terminal and computer readable storage medium
CN110750198A (en) Expression sending method and mobile terminal
CN111459299A (en) Information input method and electronic equipment
CN109981890B (en) Reminding task processing method, terminal and computer readable storage medium
CN111326175A (en) Prompting method for interlocutor and wearable device
CN108347628B (en) Method for prompting member activation, mobile terminal and server

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant