CN110602582A

CN110602582A - Earphone device with full-natural user interface and control method thereof

Info

Publication number: CN110602582A
Application number: CN201910831992.5A
Authority: CN
Inventors: 杨晓东; 陈冠璇
Original assignee: Wuxi Zhonggan Microelectronics Co Ltd
Current assignee: Wuxi Zhonggan Microelectronics Co Ltd
Priority date: 2019-08-09
Filing date: 2019-09-04
Publication date: 2019-12-20

Abstract

The invention provides an earphone device and a control method thereof.A microphone captures environmental sounds, and when preset user interesting sounds appear in the environmental sounds, the earphone device is automatically triggered to enter an environmental interaction mode; controlling the earphone device to output an environment interaction prompt in the environment interaction mode; the environment interactive reminder at least comprises two different reminding forms, selects the environment interactive reminder in the corresponding form according to the external environment selected or sensed by the user, and controls the output of the earphone device. The invention can select and trigger the information interaction between the user and the environment in a full natural interface mode according to the use requirement and preference of the user, thereby improving the convenience of interaction and facilitating the use of the user.

Description

Earphone device with full-natural user interface and control method thereof

[ technical field ] A method for producing a semiconductor device

The present invention relates to the field of earphones, and in particular, to an earphone device having a full-natural user interface and a control method thereof.

[ background of the invention ]

At present, the earphone is widely applied, and in most cases, people can talk and listen to music and the like through the earphone. Currently, some headsets suppress ambient sound. There are conventional wired and wireless earphones that can physically cancel ambient sound by partially blocking the ear canal. In addition, Active Noise Canceling (ANC) headphones may electronically filter out ambient noise. Thus, the user of the headset often cannot clearly hear the surrounding ambient sound when wearing the headset. As a result, these users wearing headphones are often unable to respond quickly to critical external sounds, such as a person's inquiry, an alarm sound, a horn sound, and the like.

Noise reducing headsets currently on the market attempt to solve the above problem in two ways. One is a "fast alert mode," which is activated when the user lifts his hand over an ear cup of the headset, at which time the volume of the music being played by the headset is turned down while the external ambient sound is captured and played. The other is a "smart listening mode" in which the activity level of the user (i.e., walking, traveling or waiting) is detected to automatically adjust the ambient sound heard by the user.

However, the "fast alert mode" requires the user to decide whether to enter this mode, and often the user cannot respond in time to a sudden, unnoticed interactive request. For example, you are listening to music using the noise reduction earphone, and your friend talks to you by calling your name, and since you do not actively enter the "quick reminder mode", you cannot hear your friend shout your name, and then do not actively enter the "quick reminder mode". It can be seen that the user actively activating the "fast alert mode" still places a burden on the headset user to initiate this function. The "smart listening mode" varies the level of ambient sound heard based on the activity level. However, based on psychology, humans do not change auditory attention on an active basis only. Instead, humans will naturally only notice certain key sounds or words (e.g., name, siren, greeting, etc.). For example, the smart listening mode may block ambient sounds when the headset user is traveling in a vehicle. However, taxi drivers may ask you questions when you are in a car, and using the "smart listening mode" of sony's noise reduction headphones, you will not be able to hear the driver's voice. It can be seen that this mode is neither intuitive nor effective.

Therefore, there is a need for an improved solution to the above problems.

[ summary of the invention ]

The invention aims to provide an earphone device and a control method thereof, which can select and trigger information interaction between a user and the environment in a full natural interface mode according to the use requirement and preference of the user.

In order to achieve the above object, as a first aspect of the present invention, there is provided an earphone device including a microphone, a speaker, and a command interface for receiving an external control instruction, further including a control unit having an ambient sound monitor function;

the control unit captures environmental sounds through a microphone, and automatically triggers the earphone device to enter an environmental interaction mode when preset user interesting sounds appear in the environmental sounds;

under the environment interaction mode, the control unit controls the earphone device to output an environment interaction prompt; the environment interactive reminder at least comprises two different reminding forms, and the control unit selects the environment interactive reminder in the corresponding form according to the external environment selected or sensed by the user and controls the earphone device to output.

Further, the environment interaction reminder can comprise one or a combination of several forms of an auditory reminder, a tactile reminder and a visual reminder;

the auditory reminding can at least comprise one or more of interesting sound of a user, environment sound and preset prompt sound, and the control unit controls the loudspeaker to output the auditory reminding;

the tactile alert may comprise a vibration alert, the earphone device comprises a vibration unit, and the control unit outputs the vibration alert through the vibration unit;

the visual reminder may include a message reminder and/or a light reminder, the earphone device includes a communication unit, and the control unit generates a reminder message and transmits the reminder message through the communication unit.

Further, the device can further comprise a storage unit, which is used for storing the environmental noise characteristic information;

and the control unit detects the environmental sound captured by the microphone based on the environmental noise characteristic information in an environmental interaction mode to judge the sensed current environmental condition, and selects and outputs an environmental interaction prompt in a corresponding form according to the preset corresponding relation between the environmental condition and the environmental interaction prompt form.

Further, the control unit may determine whether to enter an environment interaction mode according to a current environment sound level, and output a corresponding environment interaction prompt according to the current environment sound level after entering the environment interaction mode; the ambient sound level is at least used for setting the current state of monitoring the ambient sound and the current output form of the ambient interaction reminding;

the ambient sound levels may include at least two of the following levels:

an active attention level, which indicates that the current earphone device is monitoring the environmental sound, and the environment interaction prompt which can be output is that at least all the environmental sound can be played to the earphone user;

the voice monitoring level indicates that the current earphone device monitors environmental sounds, and the output environmental interactive prompt is to play voice to the earphone user only;

the micro-monitoring stage is used for indicating that the current earphone device monitors environmental sounds, and the environment interactive prompt which can be output is to play a preset prompt tone or a tactile prompt or a visual prompt to the earphone user;

and the listening-free stage indicates that the environment listening function of the current earphone device is closed.

Further, the determining, by the control unit, the current ambient sound level may include:

when the control unit detects the environmental sound level preset by a user, taking the preset environmental sound level as the current environmental sound level; alternatively, the first and second electrodes may be,

the control unit detects the environmental sound captured by the microphone based on the environmental noise characteristic information to judge the perceived current environmental condition, and determines the current environmental sound level according to the preset corresponding relation between the environmental sound level and the environmental condition.

Further, the control unit may be further configured to determine whether to exit the environment interaction mode in the environment interaction mode, control the earphone device to continuously output the environment interaction reminder when it is determined that the environment interaction mode is not exited, and exit the environment interaction mode otherwise, where:

the earphone device is characterized in that sound characteristic parameters of a user are pre-stored in the earphone device, the control unit controls the microphone to collect sound signals near the lips of the user in the environment interaction mode, detects whether the user makes sound according to the sound characteristic parameters of the user, and exits the environment interaction mode when the detection determines that the user does not make sound within a preset time period.

Further, the ambient sound in the audible alert may be part or all of the ambient sound collected by the microphone that includes the sound of interest to the user, or,

the ambient sound in the audible alert may be some or all of the ambient sound collected by the microphone.

As a second aspect of the present invention, there is provided a control method of an ear speaker device, comprising:

a step of capturing an environmental sound;

detecting the environmental sound, and automatically triggering the earphone device to enter an environmental interaction mode when preset sound which is interesting to the user appears;

controlling the earphone device to output an environment interaction prompt in the environment interaction mode; wherein the content of the first and second substances,

the environment interactive reminder at least comprises two different reminding forms, and the earphone device is controlled to select the environment interactive reminder in the corresponding form to output according to the external environment selected by the user or sensed by the earphone device.

The environment interaction reminder may include one or a combination of several forms of an audible reminder, a tactile reminder, and a visual reminder, wherein:

the auditory reminding at least comprises one or more forms of interesting sound of the user, environment sound and preset prompt sound;

the tactile alert comprises a vibratory alert;

the visual alert comprises a message alert and/or a light alert.

Further, the method can also comprise the step of presetting environmental noise characteristic information, and,

and detecting the captured environmental sound based on the environmental noise characteristic information in an environmental interaction mode to judge the perceived current environmental condition, and selectively outputting an environmental interaction prompt in a corresponding form according to the preset corresponding relation between the environmental condition and the environmental interaction prompt form.

Further, an environment interaction mode exiting step may be further included, the environment interaction mode is judged whether to exit, when it is determined that the environment interaction mode is not exited, the earphone device is controlled to continuously output the environment interaction reminder, otherwise, the environment interaction mode is exited, wherein,

the method comprises the steps that sound characteristic parameters of a user are pre-stored in the earphone device, the earphone device is controlled to collect sound signals near the lips of the user in the environment interaction mode, whether the user makes sound or not is detected according to the sound characteristic parameters of the user, and the environment interaction mode is quitted when the user does not make sound within a preset time length.

Further, the method can also comprise a step of determining whether to enter an environment interaction mode according to the current environment sound level, and a step of outputting a corresponding environment interaction prompt according to the current environment sound level after entering the environment interaction mode; the ambient sound level is at least used for setting the current state of monitoring the ambient sound and the current output form of the ambient interaction reminding;

the ambient sound levels comprise at least two of the following levels:

Further, the step of determining the current ambient sound level may comprise:

when detecting the environmental sound level preset by a user, taking the preset environmental sound level as the current environmental sound level; alternatively, the first and second electrodes may be,

and detecting the environmental sound captured by the microphone based on the environmental noise characteristic information so as to judge the perceived current environmental condition, and determining the current environmental sound level according to the preset corresponding relation between the environmental sound level and the environmental condition.

Further, the ambient sound in the audible alert may be some or all of the ambient sound containing the sound of interest to the user, or alternatively,

the ambient sound in the audible alert may be some or all of the ambient sound.

Compared with the prior art, the earphone device can select and trigger information interaction between the user and the environment in a full-natural interface mode according to the use requirements and preferences of the user, so that the interaction convenience is improved, and the use by the user is facilitated.

[ description of the drawings ]

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise. Wherein:

fig. 1 is a block diagram of an earphone device according to an embodiment of the present invention;

fig. 2 is a functional flowchart of a control method of the headphone apparatus according to an embodiment of the present invention.

[ detailed description ] embodiments

The following detailed description of the invention refers to the accompanying drawings.

In the present invention, the terms "in one embodiment" and "in an embodiment" do not specifically mean that they are in the same embodiment, and may be in different embodiments.

Based on an analysis of the prior art, we have found that the current technology still forces the user to adapt to the limitations of the technology, following the traditions and procedures of the technology setup for using the headset. Therefore, the main object of the present invention is to improve the earphone device by using the design idea of Natural User Interface (Zero User Interface), which is to make the earphone device work according to the needs and preferences of the User by designing a User Interface that matches the experience or thinking model of the target User group in the expected use situation, so that the User can concentrate his attention on the activities to be performed in the most comfortable and Natural way, and forget the existence of the Interface.

In view of the above, the present invention provides an earphone device with a natural user interface, which can monitor the environmental sound when people wear the earphone device to listen to music or talk, and automatically control the earphone to output an environmental interaction prompt when a preset sound of interest to the user appears in the environmental sound, wherein the user interface is a natural way, and the user is comfortable and natural to use.

The earphone device can be a known or future possible wireless earphone such as a Bluetooth earphone, and can also be a wired earphone.

Fig. 1 is a block diagram of an earphone device 100 according to an embodiment of the present invention. As shown in fig. 1, the ear speaker device 100 includes an attenuator 110, a microphone 120, a control unit 130, a speaker 150, a command interface 160, and a storage unit 180. The attenuator 110, the microphone 120, the speaker 150, the command interface 160, and the storage unit 180 are all electrically connected to the control unit 130.

The microphone 120 is used to collect signals that contain primarily ambient sound, but may also contain some user speech. The microphone 120 may be a microphone or a microphone array.

The speaker 150 is used to play audio signals.

The control unit 130 is used to control the various modules and may have one or more functional modes.

In one embodiment, the headset device 100 may further include a communication unit 140. The communication unit 140 may include a wireless communication unit, and the wireless communication unit may be configured to establish a wireless communication connection with an external electronic device (e.g., a smart phone, a tablet computer, a notebook computer, a desktop computer, a vehicle-mounted communication system, etc.), transmit an audio signal from the control unit 130 to the external electronic device, and receive an audio signal from the external electronic device. The wireless communication unit can be a Bluetooth communication unit, and can also be other wireless communication units. In another embodiment, the communication unit may include a wired communication unit, and the wired communication unit may be used to establish a wired communication connection with an external electronic device (such as a smart phone, a tablet computer, a notebook computer, a desktop computer, an in-vehicle communication system, and the like), transmit an audio signal from the control unit 130 to the external electronic device, and receive an audio signal from the external electronic device.

The storage unit 180 may store keywords related to the user's interested sound, a sound sample, or feature parameters of the sound sample, etc. in advance; one or more environment interaction reminding forms can be prestored, such as identifiers corresponding to the environment interaction reminding forms; a default ambient interaction alert form may also be pre-stored. The storage unit 180 may also store the environmental noise characteristic information in advance. Of course, the storage unit 180 may also be used to store other information related to the operation of the earphone device and the user information.

The command interface 160 is used to receive various instruction information input by a user, including a control instruction, a status setting instruction, and the like. The control command includes a command for notifying the control unit 130 to enter a corresponding functional mode, and the command interface 160 transmits the command to the control unit 130 after receiving the command, so that the control unit 130 enters the corresponding functional mode. The state setting instruction comprises various information input by a user when setting the state information of the earphone device, so that the earphone device can execute corresponding operation according to the state information input by the user when in work.

The command interface 160 may be a key, and a plurality of keys or one key may be triggered to issue a control instruction to the control unit 130. The command interface 160 may also be a voice recognition interface, and a user speaks a preset voice control command, and the voice recognition interface transmits the voice control command to the control unit 130.

Fig. 2 is a functional flowchart of a method 200 for controlling an earphone device according to an embodiment of the present invention.

As shown in fig. 2, the control method 200 begins by entering step 220, capturing ambient sound, and at step 220, capturing ambient sound using the microphone 120. The captured ambient sound may be temporarily buffered in the storage unit 180.

At step 230, it is determined whether a preset user interest sound appears in the captured ambient sound, and if so, step 240 is entered, and if not, step 220 is returned to continue capturing ambient sound with the microphone 120.

In a specific embodiment, the preset sound of interest to the user may be a predetermined keyword or a predetermined sound. The predetermined keyword may be a word focused by the user, such as a name related to the identity of the user, e.g., a name, a nickname, a position, a title, etc., or a commonly used start word in a conversation, such as "hello", "disturb down", "disalignment", etc., and the predetermined sound may be, e.g., an alarm sound, a horn sound, a door closing sound, etc.

The sound of interest to the user may be preset by the user, or may be partially preset when the earphone device leaves the factory, such as various alarms, car whistling sounds, station announcement sounds, and the like. In one embodiment, when the user presets himself, after the earphone device obtains a sound sample in which the user is interested, the earphone device extracts characteristic parameters of the sound sample and stores the characteristic parameters in a classified manner. And judging whether the sound which is interested by the user appears in the environmental sound, and detecting the environmental sound by adopting a pattern recognition algorithm according to the characteristic parameters of the sound sample.

In step 240, an environment interaction mode is entered, and an environment interaction reminder is output in the environment interaction mode. Then step 250 is entered to determine whether to exit the ambient interaction mode, if so, the ambient interaction mode is exited, the process returns to step 220, and the microphone 120 continues to capture ambient sound; if not, returning to step 240, controlling the earphone device to continuously collect the environmental sound, and outputting an environmental interaction prompt. And the environment interaction prompt is continuously output, so that the user can fully learn the prompt in an environment interaction mode and complete the whole interaction process.

The environment interactive reminder at least comprises two different reminding forms, the control unit selects the environment interactive reminder in the corresponding form according to the external environment selected or sensed by the user and controls the earphone device to output

In one embodiment, the ambient interaction alert may be an audible alert, which may be in various forms, such as an ambient sound, that is, an ambient sound captured by the microphone 120 in real time and played by the speaker 150 in real time, so as to help the earphone user to know the external environment timely and comprehensively, a predetermined alert sound, such as a predetermined beep sound, predetermined music, ring sound, or the like, or a user interest sound, such as a predetermined keyword itself, a predetermined sound itself, or an ambient sound including a part or all of the user interest sound, or a voice conversation extracted from the ambient sound and including the user interest sound.

In addition, the environment interactive reminding can also be a touch reminding, for example, the vibration unit 170 can be additionally arranged on the earphone device, and the control unit controls the vibration unit 170, so that the earphone generates vibration to realize the touch reminding, and the touch reminding can also be realized through vibration of electronic equipment such as a smart phone, a tablet personal computer, a smart watch and a bracelet which are jointly used with the earphone device. The vibration unit 170 may be implemented by using an existing eccentric wheel motor or a linear vibration motor, or may be implemented by using other future technologies that can vibrate the apparatus body.

Of course, the environment interaction prompt may also be a visual prompt, for example, the message may be sent to a terminal device such as a smart phone, a tablet, a watch, and a bracelet used in combination with the headset in a message notification manner to implement the visual prompt. If the user wears the headset to watch a movie on the tablet pc, the control unit 130 of the headset device may generate a reminding message according to the information of the sound of interest of the user detected in step 230, and send the reminding message to the tablet pc through the communication unit of the headset for displaying, thereby reminding the user in the form of a screen message. The visual reminder can also be a light reminder, such as setting a flashing screen of a display screen on the earphone device or setting flashing of a key lamp.

The environment interactive reminder can be one of the above-mentioned auditory reminder, tactile reminder, visual reminder and other possible forms, but in order to provide a more natural and comfortable environment interactive reminder for the user, the environment interactive reminder is preferably of a plurality of forms, the user can select one of the above-mentioned auditory reminder, tactile reminder and visual reminder under different environments according to different requirements and preferences, and the environment interactive reminder can also be a combination of a plurality of forms, such as a combination of an environment sound and a tactile reminder, or a combination of a preset environment interactive sound and a tactile reminder, or a combination of a preset keyword and a visual reminder in a sound of interest of the user, and the like, so that a richer environment interactive reminder can be output, and the use is more convenient. Preferably, the earphone device can also automatically sense the external environment, and select the most suitable environment interaction reminding form according to the environment condition.

Therefore, in one embodiment, some environment interaction reminder may be set in the earphone device in advance to be a default reminder form, such as an environment sound. And provides a plurality of environment interaction reminding forms during the use process of the earphone device, so as to allow a user to select and combine the environment interaction reminding forms according to own preference and requirements. Thus, in step 240, it is first detected whether there is a user setting, and if there is no user setting, the speaker 150 is controlled to output an environment interaction alert in a default alert form in the environment interaction mode. If user settings are detected, the speaker 150, vibration unit 170 or other features of the communication unit 140 may be controlled to output the ambient interaction alert, alone or in combination, based on the user's selection and combination.

In another embodiment, to further adapt the headphone apparatus to the needs of the user, providing the most comfortable and natural form of alert, the headphone apparatus may automatically select a corresponding form of ambient interaction alert output based on the external environment it senses. The earphone device may pre-store a plurality of environmental noise characteristic information, and in step 240, the control unit determines a current environmental condition based on the environmental sound captured by the environmental noise characteristic information detection microphone in the environmental interaction mode, and automatically selects which kind or kinds of environmental interaction reminders to output based on the current environmental condition according to a corresponding relationship between the preset environmental condition and the environmental interaction reminding form. If various automobile whistles and wheel noises generated when the automobile runs are detected in the current environment, the surrounding environment can be judged to be the road, at this time, at least the automobile whistling sounds and the human voices in the environment sounds can be output as the sounds which are interesting to the user, and all captured environment sounds can also be output, so that the user can pay attention to the complex traffic safety condition. For another example, when it is detected that the current environmental sound has a very low noise level for a long time or is similar to the pre-stored library noise characteristic information, the external environment is considered to be very quiet, and when it is detected that a sudden sound occurs in the environmental sound, it is judged that the sudden sound belongs to the sound of interest of the user, and a gentle warning sound such as "beep" is output. . . A beep. . . "sound, or output a soft vibratory alert.

In one embodiment, the environmental interaction mode may be automatically exited after a duration of time. Then, in step 240, once the environment interactive mode is entered, timing is started, and in step 250, it is detected whether the duration of the environment interactive mode reaches a predetermined time, and when the duration reaches the predetermined time, the environment interactive mode is exited.

In another embodiment, the exit of the ambient interaction mode is decided by the user, e.g. the user controls the end of the ambient interaction mode by issuing an exit command to the headphone arrangement. The command interface 160 is detected in step 250, and it is determined whether the command currently input by the user is an instruction to exit the environment interaction mode, and the environment interaction mode is exited when it is confirmed that the exit instruction is received.

In the third embodiment, the sound characteristic parameters of the user are pre-stored in the earphone device, in the environment interactive mode, the control unit 130 of the earphone device controls the microphone (which may be the microphone 120, and more preferably, the microphone is located closest to the lips of the user) to capture sound signals near the lips of the user, and detects whether the user is making a sound according to the sound characteristic parameters of the user, and when the detection determines that the user is not making a sound within a predetermined time period, the step of exiting the environment interactive mode in step 250 is performed. Compared with the former two embodiments, the mode of exiting from the environment interaction mode is more in line with the design concept of a full natural user interface, the user hears the environment interaction prompt in a natural mode, naturally performs environment interaction (talking with people, paying attention to vehicles, paying attention to bus stop announcement information and the like), and naturally returns to the audio-visual environment before the environment interaction when the environment interaction is not needed, so that the use is very comfortable and convenient.

In a preferred embodiment, the method 200 for controlling the earphone device in the present invention may further include a step of determining whether to enter the environment interaction mode according to the current environment sound level, and a step of outputting a corresponding environment interaction alert according to the current environment sound level after entering the environment interaction mode. The ambient sound level may be used to set the current state of the headphone device for listening to ambient sound and/or the current output form of the ambient interaction alert, or may be used to set other operating states or operating forms of the headphone device.

In a preferred embodiment, the ambient sound level is used for setting a current listening state of the earphone device to the ambient sound and a current outputtable ambient interaction reminding form, wherein the ambient sound level includes at least two of the following levels:

Of course, other levels of ambient sound levels may be set, depending on the particular implementation needs.

The step of determining the current ambient sound level may specifically be based on a user setting, or on a way of the headphone arrangement sensing the ambient environment, wherein:

specifically, when the environmental sound level preset by the user is detected, the preset environmental sound level is used as the current environmental sound level.

Specifically, the manner of sensing the surrounding environment based on the earphone device is to detect the ambient sound captured by the microphone based on the ambient noise feature information to determine the current sensed environmental condition, and determine the current environmental sound level according to a preset correspondence between the environmental sound level and the environmental condition.

In one embodiment, when the environment interaction alert is output in the environment interaction mode, the playing of the audio feed signal (such as the music currently being played in the speaker 150) currently required to be played in the speaker 150 may be paused, so that the environment interaction alert played in the speaker 150 can be clearly heard by the user, of course, the audio feed signal currently required to be played in the speaker 150 may be attenuated by the attenuator 110 and played synchronously with the environment interaction alert, and the audio feed signal currently required to be played may be played through one of the left channel speaker and the right channel speaker of the speaker 150, and the environment interaction alert may be played through the other of the left channel speaker and the right channel speaker of the speaker 150. The audio feed signal may come from an external audio reproduction device (e.g., a mobile phone, a portable player, other headset devices, a sound device, etc.) that is wired or wirelessly connected to the headset device, or from a microphone on the headset device 100 that is primarily used to capture user voice information.

In the invention, the microphone can represent the earphone user to selectively process the prominent external auditory stimulation, so that the user can concentrate on the existing things, simultaneously, any related auditory clues can not be missed, natural and comfortable environment interactive reminding can be realized without user action when preset user interested sound appears in the environment sound, the interactive convenience is improved, and the use of the user is greatly facilitated.

In an application example, a user "zhang san" wears the earphone device 100 to walk on the road, and connects the earphone device with a smartphone to listen to music, at this time, the user "zhang san" is immersed in music sound, and suddenly a friend of an acquaintance calls "zhang san" to "zhang san, good morning, because the earphone device 100 monitors external environment sound in real time at this time, and when the keyword" zhang san "is monitored, the user automatically enters an environment interaction mode and plays the environment sound captured by a microphone. Therefore, the user can normally realize the conversation with other people without any action by using Zhang III, and automatically quits the environment interaction mode after the conversation with other people is stopped by detecting the Zhang III, so that the use is very convenient without active intervention of Zhang III in the whole process. Meanwhile, the earphone replaces the mode that three Zhang actively selects the interesting environmental sound of three Zhang, but not all the environmental sounds are played, so that the listening environment of three Zhang is purified, the sound quality of the user enjoying music is ensured, and no key environmental information is omitted. This detection and selection mechanism is completely natural in that it mimics the intuitive behavior of humans.

In another application example, the user "zhang san" wears the earphone device 100 and drives a car, and suddenly drives the car outside and sends out an alarm sound of the car, because the earphone device 100 monitors the sound of the outside environment in real time at this time, when the predetermined sound of the alarm sound of the car is monitored, the environment interaction mode is automatically entered and the alarm sound of the car monitored by the microphone is played, and the alarm sound is continuously played until the predetermined playing time is reached or the "zhang san" is manually cancelled, so that the user "zhang san" has enough time to notice the alarm sound and take corresponding measures. Therefore, the user can normally deal with the external emergency under the condition of wearing the earphone device 100 without making any action, and the earphone device is very convenient.

Ambient sound level indicator

In a preferred embodiment, the earphone device further comprises an ambient sound level indicator 190, the ambient sound level indicator 190 being indicative of the ambient sound level currently perceived by the earphone device 100. The control unit also determines whether to enter an environment interaction mode according to the currently perceived environment sound level, and outputs a corresponding environment interaction prompt according to the currently perceived environment sound level after entering the environment interaction mode.

The control unit may specifically acquire the currently perceived ambient sound level based on a user setting manner or based on a manner in which the headphone apparatus perceives the ambient environment, where:

Specifically, the manner of sensing the surrounding environment based on the earphone device is to detect the ambient sound captured by the microphone based on the ambient noise feature information to determine the current sensed environmental condition, and determine the current environmental sound level according to a preset correspondence between the environmental sound level and the environmental condition. The ambient sound level may include one or more of an active attention level, a human voice listening level, a micro listening level, and a non-listening level according to the ambient interaction alert output in the ambient interaction mode.

Wherein:

the micro-monitoring stage is used for indicating that the current earphone device monitors environmental sounds, and the environment interactive prompt which can be output is to play a preset prompt tone, a touch prompt or a visual prompt to an earphone user;

the listening-free stage indicates that the ambiance listening function of the current earphone arrangement is switched off, which may prevent the user from being disturbed. When the environmental sound level of earphone device current perception is the no monitoring level, because the environmental monitoring function of current earphone device has closed, consequently can not get into the environment interaction mode, also can not export the environmental interaction and remind, the user can through with earphone device sets up to the no monitoring level and comes initiative shielding off the environmental sound to avoid oneself to be disturbed.

Of course, other ambient sound levels may be added or modified in accordance with the teachings of the present invention for specific implementations.

The user can select the ambient sound level of the earphone device according to the use habit and the scene requirement, and the earphone device can automatically select the current ambient sound level for the user according to the current sensed ambient condition. Under the design concept of a full natural user interface, the initial ambient sound level of the earphone device can be set to an active attention level in consideration of the natural and comfortable use requirements of most users.

In a preferred embodiment, the ambient sound level indicator 190 may be an indicator light 190. In the active attention level, the indicator light is displayed as a light effect, such as green, such as flashing and the like; in the absence of a listening level, the indicator light is displayed in another state, such as red, such as no blinking, and so forth. Thus, by observing the indicator light 190, it is possible to know the level of ambient sound that can be heard by the user wearing the headset, and whether a conversation is welcomed, which gives the headset a new social attribute. Of course, the ambient sound level indicator 190 may also be implemented by other specific devices, such as a display screen that displays different images according to the ambient sound level.

In an application example, the user a wears the earphone device 100 in an office, and the indicator light 190 is green, so that the colleague can know that the user a welcomes a conversation after seeing the color of the indicator light 190, and can actively make a call and communicate with the user a, and the user a can hear the speaking voice of a friend even when wearing the earphone, and can respond in time, and does not need to manually switch the state of the earphone, and the whole process is very natural. In another example, when the user a wears the earphone device 100 to read a book, and the indicator light 190 is red, other people may know that the user a does not want to be disturbed and may not hear the sound after seeing the color of the indicator light 190, and other people may avoid disturbing the user a, and the user a may also avoid being disturbed by other people.

The setting of the environmental sound level indicator 190 may even change laws and regulations and usage habits in certain regions. For example, the U.S. california legislation currently prohibits the use of headphones in both ears while driving because people are concerned that drivers cannot hear the warning sounds of others when wearing headphones in both ears. However, the contradiction is that wearing the earphone by only one ear obviously greatly affects the stereo quality of the earphone, and brings discomfort to the user. This conflicting conflict results in the majority of drivers not using headphones. This clearly reduces the consumer population of headsets for the merchant. The use of an ambient sound level indicator will change this status. With a uniform ambient sound level indication, the driver can receive the ambient sound at any time, and the police can also clearly see if the driver has used the appropriate settings (i.e. is able to hear the ambient noise). This may make it legal to use the headphones in both ears of the driver.

Near-field monitoring and far-field monitoring

Current headset devices are essentially near-field listening, i.e. gathering ambient sound in the vicinity of the headset. The reasons for not providing far-field monitoring are, on one hand, that technical implementation is difficult and costly, and far-field monitoring is generally used for issuing voice control commands to household appliances, and that the headset does not need to be remotely controlled, and more importantly, because technicians are always accustomed to "immersing" users in their own small places when using the headset, and they do not need to and cannot give full attention to remote environmental information. This neglects some practical needs of people, for example, when children enjoy music in their bedrooms with earphones, they also need to hear the call of mom in the kitchen or the call of friends outdoors, which still falls into the thinking set of the technical limitation.

In an embodiment of the present invention, when the control unit performs the ambient sound monitoring function, near field monitoring or far field monitoring may be performed according to a user's selection. Near field listening may be performed by default in the initial state. And during near-field monitoring, the microphone can be used for monitoring the near-field environmental sound, and during far-field monitoring, the microphone can be used for monitoring the far-field environmental sound. In far-field monitoring, if a person at a far position (for example, 5-10 meters) calls a user wearing the earphone, the earphone device can still monitor related calling sound, and in near-field monitoring, if a person at a far position calls a user wearing the earphone, the earphone device cannot monitor the related calling sound.

In a preferred embodiment, the microphones 120 are a microphone array 120 in far-field listening. The control unit 130 controls the microphone array 120 to collect the ambient sound, performs continuous sound source pickup and recognition through sound source localization when detecting that there is a sudden sound in the ambient sound at a certain azimuth, and detects whether there is a sound of interest to the user to determine whether to trigger the headset to enter the ambient interaction mode.

In another preferred embodiment, the microphones 120 are a microphone array 120 in far-field listening. The control unit 130 controls the microphone array 120 to collect the ambient sound, and obtains multiple ambient sound signals. The control unit 130 detects whether there is a sound of interest to the user in each of the at least two ambient sound signals, and automatically triggers the headset to enter the ambient interaction mode when there is a sound of interest to the user in at least two of the at least two ambient sound signals.

Local privacy mode and remote mode

In a preferred embodiment, the control unit performs local privacy protection or remote recognition according to a user's selection while performing the ambient sound listening function. Local privacy protection may be performed by default in the initial state. Under the local privacy protection, whether the user is interested in the environmental sound can be detected by the local electronic equipment or the control unit according to a preset sound identification algorithm, networking is not needed for carrying out relevant identification work by a remote server, and therefore the privacy disclosure problem can be reduced to the maximum extent. But like amazon's Alexa and google assistant, both connect to the internet and do the relevant identification work by the server, which risks privacy disclosure because the relevant information may be easily intercepted by a third party via the internet. However, in order to provide convenience to the user, the earphone device of the present invention is also provided with remote recognition in which the control unit transmits the ambient sound captured by the microphone to the remote device through the wired/wireless communication unit to detect whether the ambient sound is a sound of interest to the user, and receives the detection result from the remote device.

Partial play mode and full play mode

In a preferred embodiment, the control unit may play, through the speaker, part or all of the ambient sounds including sounds of interest to the user, which are monitored by the microphone, in the ambient interaction mode according to a selection of the user, and in the partial play mode, only part of sound information including sounds of interest to the user, such as predetermined keywords or predetermined sounds, in the ambient sounds monitored by the microphone, is played through the speaker.

The foregoing description has disclosed fully preferred embodiments of the present invention. It should be noted that those skilled in the art can make modifications to the embodiments of the present invention without departing from the scope of the appended claims. Accordingly, the scope of the appended claims is not to be limited to the specific embodiments described above.

Claims

1. An earphone device comprises a microphone, a loudspeaker and a command interface for receiving external control instructions, and is characterized by further comprising a control unit with an environmental sound monitoring function;

under the environment interaction mode, the control unit controls the earphone device to output an environment interaction prompt; wherein the content of the first and second substances,

the environment interactive reminder at least comprises two different reminding forms, and the control unit selects the environment interactive reminder in the corresponding form according to the external environment selected or sensed by the user and controls the earphone device to output.

2. The headset device of claim 1, wherein the ambient interaction alert comprises one or a combination of an audible alert, a tactile alert, and a visual alert;

the auditory reminder at least comprises one or more forms of sound interested by the user, environment sound and preset prompt sound, and the control unit controls the loudspeaker to output the auditory reminder;

the touch reminding comprises a vibration reminding, the earphone device comprises a vibration unit, and the control unit outputs the vibration reminding through the vibration unit;

the earphone device comprises a communication unit, and the control unit generates a reminding message and sends the reminding message through the communication unit.

3. The ear speaker device according to claim 1, further comprising a storage unit for storing ambient noise characteristic information;

4. The earphone device according to claim 1, 2 or 3, wherein the control unit determines whether to enter an environment interaction mode according to the current environment sound level, and outputs a corresponding environment interaction prompt according to the current environment sound level after entering the environment interaction mode; the ambient sound level is at least used for setting the current state of monitoring the ambient sound and the current output form of the ambient interaction reminding;

the ambient sound levels comprise at least two of the following levels:

5. The headphone apparatus as claimed in claim 4, wherein the control unit determines the current ambient sound level, including:

6. The headset device according to claim 1, wherein the control unit is further configured to determine whether to exit the environment interaction mode in the environment interaction mode, and control the headset device to continuously output the environment interaction alert when determining not to exit the environment interaction mode, otherwise exit the environment interaction mode, wherein:

7. The headset of claim 2, wherein the ambient sounds in the audible alert are some or all of the ambient sounds collected by the microphone that include sounds of interest to the user, or,

the ambient sound in the audible alert is part or all of the ambient sound collected by the microphone.

8. A control method of an earphone device, characterized by comprising:

a step of capturing an environmental sound;

9. The method of claim 8,

the environment interaction reminding comprises one or a combination of several forms of auditory reminding, tactile reminding and visual reminding, wherein:

the tactile alert comprises a vibratory alert;

the visual alert comprises a message alert and/or a light alert.

10. The method according to claim 8, further comprising the step of presetting ambient noise characteristic information, and,

11. The method of claim 8, further comprising an ambient interaction mode exit step of determining whether to exit the ambient interaction mode, controlling the earphone device to continuously output the ambient interaction alert when it is determined not to exit the ambient interaction mode, and otherwise exiting the ambient interaction mode, wherein,

12. The method of claim 8, 9 or 10, wherein:

the method also comprises a step of determining whether to enter an environment interaction mode according to the current environment sound level, and a step of outputting a corresponding environment interaction prompt according to the current environment sound level after entering the environment interaction mode;

the ambient sound level is at least used for setting the current state of monitoring the ambient sound and the current output form of the ambient interaction reminding;

the ambient sound levels comprise at least two of the following levels:

13. The method of claim 12, wherein the step of determining the current ambient sound level comprises:

14. The method of claim 9, wherein:

the ambient sound in the audible alert is some or all of the ambient sound that contains the sound of interest to the user, or alternatively,

the ambient sound in the audible alert is part or all of the ambient sound.