CN103049077A - Sound feedback device and working method thereof - Google Patents

Sound feedback device and working method thereof Download PDF

Info

Publication number
CN103049077A
CN103049077A CN2011103116491A CN201110311649A CN103049077A CN 103049077 A CN103049077 A CN 103049077A CN 2011103116491 A CN2011103116491 A CN 2011103116491A CN 201110311649 A CN201110311649 A CN 201110311649A CN 103049077 A CN103049077 A CN 103049077A
Authority
CN
China
Prior art keywords
sound
volume
unit
compass
virtual
Prior art date
Application number
CN2011103116491A
Other languages
Chinese (zh)
Inventor
李后贤
李章荣
罗治平
Original Assignee
鸿富锦精密工业(深圳)有限公司
鸿海精密工业股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 鸿富锦精密工业(深圳)有限公司, 鸿海精密工业股份有限公司 filed Critical 鸿富锦精密工业(深圳)有限公司
Priority to CN2011103116491A priority Critical patent/CN103049077A/en
Publication of CN103049077A publication Critical patent/CN103049077A/en

Links

Abstract

A sound feedback device comprises a plurality of microphones, a sound source orientation judging unit, a sound volume judging unit, a language to character unit and a situational simulation unit. The sound feedback device can display a virtual compass, sound source orientation, sound volume and character information according to outside sound signals so as to achieve the effect of assisting people having hearing disorder. The invention further provides a working method of the sound feedback device.

Description

声音反馈装置及其工作方法 Audible feedback apparatus and operating methods

技术领域 FIELD

[0001] 本发明涉及一种声音反馈装置及该装置的工作方法。 [0001] A sound feedback relates to apparatus and a working method of the apparatus of the present invention.

背景技术 Background technique

[0002] 听力障碍人士由于失去听觉的感受能力,因此在日常生活中,将无法如一般人直接藉由听觉来察觉周围环境的状况或者确认人员的交谈内容,而仅能依赖双眼来留意、察觉周围环境的事件。 [0002] hearing impaired ability to feel because of the loss of hearing, so in daily life, will not be directly perceived by hearing conversations to confirm status or personnel surrounding environment such as the average person, and can only rely on your eyes to watch, perceive around event environment. 当事件发生在听力障碍人士的视线范围之外时,其将有可能对听力障碍人士带来危险。 When an event occurs out of sight of the hearing impaired, it will be possible for the hearing impaired at risk.

发明内容 SUMMARY

[0003] 鉴于以上内容,有必要提供一种声音反馈装置及其工作方法。 [0003] In view of the above, it is necessary to provide an apparatus and operating method of voice feedback.

[0004] —种声音反馈装置,包括: [0004] - voices feedback device, comprising:

若干麦克风,用于感测外界的声音信号; Several microphone for sensing the sound signal outside;

一声源方位判断单元,用于对各麦克风所接收的声音信号进行计算,以判断声源的方 A sound source position determination unit for calculating the sound signals received by the microphones to the sound source direction determination

位; Bit;

一音量判断单元,用于判断麦克风所接收的声音的音量大小,并判断其是否超过一设定值; A volume determining unit for determining the sound received by the microphone volume, and determines whether it exceeds a predetermined value;

一语言转文字单元,用于在麦克风所接收的声音的音量大小超过设定值时将声音信号转换为文字信息; A language-to-text unit, configured to, when the volume of sound received by the microphone exceeds a set value of the voice signal into text information;

一情境模拟单元,包括一虚拟罗盘模块及一对白处理模块,该虚拟罗盘模块用于虚拟一罗盘,并根据由声源方位判断单元所得到的声源方位以及音量判断单元所得到的音量大小在虚拟罗盘的对应位置进行标识,以提示使用者此时的声源方位及音量大小,该对白处理模块用于根据语言转文字单元所得到的文字信息调用对应的对白框,并在对白框中显示对应的文字;以及 A situation simulating unit comprises a compass module and a virtual dialogue processing module, the module for the virtual virtual compass a compass, and in accordance with the sound source position determination unit because the sound source position determination unit volume and the volume in the resulting corresponding to the position of the virtual compass identified, prompting the user to the sound source position and the volume at that time, the dialogue processing module according to text literal language unit information obtained call the corresponding dialogue box displayed in the dialogue box, and corresponding text; and

一显示单元,用于显示虚拟罗盘、声源方位、音量大小以及文字信息。 A display unit for displaying compass virtual sound source position, text size, and volume information.

[0005] 一种声音反馈装置的工作方法,包括: Work Method [0005] A sound feedback device, comprising:

通过若干麦克风感测外界的声音信号; Measured sound signal outside world through a number of sense microphones;

通过一声源方位判断单元对各麦克风所接收的声音信号进行计算,以判断声源的方 Calculates respective sound signals received by the microphone sound source position determination unit for determining a sound source side

位; Bit;

通过一音量判断单元判断麦克风所接收的声音的音量大小,并判断其是否超过一设定 By volume determination unit determines a volume of sound received by the microphone, and determines whether or not it exceeds a set

值; value;

当麦克风所接收的声音的音量大小超过设定值时,通过一语言转文字单元将声音信号转换为文字信息; When the volume of sound received by the microphone exceeds a set value, a language to text by the unit character information is converted into sound signal;

通过一虚拟罗盘模块虚拟一罗盘,并根据由声源方位判断单元所得到的声源方位以及音量判断单元所得到的音量大小在虚拟罗盘的对应位置进行标识,以提示使用者此时的声源方位及音量大小;通过一对白处理模块根据语言转文字单元所得到的文字信息调用对应的对白框,并在对白框中显示对应的文字;以及 A compass by a virtual virtual compass module, and identified in the corresponding position of the virtual volume according to the compass by the sound source position determination unit because the sound source position determination unit, and a volume obtained at this time to prompt the user of the sound source orientation and volume; white invoked by a processing module unit to text literal language of the information obtained corresponding to the dialogue box, and displays the corresponding character in the dialogue box; and

通过一显示单元显示虚拟罗盘、声源方位、音量大小以及文字信息。 Virtual compass display, a sound source orientation, volume, and text information through a display unit.

[0006] 上述声音反馈装置及其工作方法通过麦克风感测声音信号,并通过声源方位判断单元判断声音的方向,当声音的音量超过设定值时即通过语言转文字单元将声音信号转换为文字信息,并通过显示单元显示给使用者,以起到辅助听力障碍人士的作用。 [0006] The apparatus and operating method of the audible feedback sensed by the microphone sound signal, and the direction determining unit determines the orientation of the sound source of sound, when the sound volume exceeds the set value i.e. the language to text by a voice signal is converted to the unit text messages, and displayed to the user through the display unit to play a supporting role of the hearing impaired.

附图说明 BRIEF DESCRIPTION

[0007] 图1是本发明声音反馈装置的较佳实施方式的方框图。 [0007] FIG. 1 is a block diagram of an exemplary embodiment of the present invention, the sound feedback device.

[0008] 图2是图1中声音反馈装置的第一示意图。 [0008] FIG. 2 is a schematic view of FIG. 1 a first feedback of the sound.

[0009] 图3是若干麦克风的分布不意图。 [0009] FIG. 3 is not intended to several distributed microphones.

[0010] 图4是图1中语言转文字单元的方框图。 [0010] FIG. 4 is a block diagram of a language to text units.

[0011] 图5是图1中情境模拟单元的方框图。 [0011] FIG. 5 is a block diagram of a situation simulating unit of FIG.

[0012] 图6是图1中声音反馈装置的第二示意图。 [0012] FIG. 6 is a second schematic diagram of a feedback sound device.

[0013] 图7是图1中声音反馈装置的第三示意图。 [0013] FIG. 7 is a third schematic diagram of a feedback sound apparatus.

[0014] 图8及图9是图1中声音反馈装置的工作示意图。 [0014] FIG 8 and FIG 9 is a schematic view of a working apparatus in FIG acoustic feedback.

[0015] 图10是本发明声音反馈装置的工作方法的较佳实施方式的流程图。 [0015] FIG. 10 is a flowchart of an embodiment of the working method of the present invention the voice feedback device.

[0016] 主要元件符号说明 [0016] Main reference numerals DESCRIPTION

麦克风 10, AG声源方位判断单元 20音量判断单元 30语言转文字单元 50显示单元 60情境模拟单元 80眼镜 100显示屏 61声音解碼模块 500文字字形库模块 510码字匹配模块 520虚拟罗盘模块 800对白处理模块 810投影单元 62摄像头 63使用者 620车辆 600路人甲 610虚拟罗盘 820 Microphone 10, AG sound source position determination unit 20 determines a volume-to-text unit 30 languages ​​unit 50 display unit 60 eyeglasses situation simulating unit 80 audio decoder 61 module 100 displays the text font memory 500 module 510 module 520 matches the virtual codeword compass module 800 White The processing module 810 projection unit 62 camera user 620 of the vehicle 600 63 610 passerby virtual compass 820

如下具体实施方式将结合上述附图进一步说明本发明。 The following detailed description in conjunction with the accompanying drawings, the present invention is described.

具体实施方式 Detailed ways

[0017] 下面结合附图及较佳实施方式对本发明作进一步详细描述: [0017] The present invention will be described in further detail in conjunction with the accompanying drawings and preferred embodiments:

请参考图1,本发明声音反馈装置的较佳实施方式包括若干麦克风10、一声源方位判断单元20、一音量判断单元30、一语言转文字单元50、一显示单元60及一,清境模拟单元80。 Referring to FIG 1, a preferred embodiment of the present invention, the sound feedback means comprises a plurality of microphone 10, sound source position determination unit 20, a volume determination unit 30, a language-to-text unit 50, a display unit 60, and a, analog Chingjing unit 80. [0018] 请继续参考图2,本发明声音反馈装置可以为一头戴式装置,如一眼镜100。 [0018] Please continue to refer to FIG. 2, the present invention is a sound feedback means may be a head-mounted device, such as eyeglasses 100. 此时该显示单元60为显示屏61。 At this time, the display unit 60 a display screen 61. 该显示屏61设置于眼镜100的镜框中,该若干麦克风10分布设置于眼镜100的镜架上,该声源方位判断单元20、音量判断单元30、语言转文字单元50及情境模拟单元80为设置于眼镜100的镜架的内部的一内存中的软件系统,其工作原理将在后面进行详细的说明。 The display 61 is provided on the glasses frame 100, the plurality of microphones 10 is provided on the distribution frame glasses 100, the sound source position determination unit 20, a volume determination unit 30, unit 50 and the language-text unit 80 is a situation simulating a memory disposed in the interior of the eyeglass frame 100 in the software system, it works will be described in detail later.

[0019] 如图3所示,该若干麦克风10组成一数组型麦克风,其用于感测来自外界的声音信号。 [0019] As shown in FIG. 3, the plurality of the microphone array 10 consisting of a microphone, for sensing the sound signal from the outside. 该声源方位判断单元20用于对各麦克风10所接收的声音信号进行计算,以判断声源的方位,即声音来自何方。 The sound source position determination unit 20 for each of the sound signals received by the microphone 10 is calculated to determine the orientation of the sound source, i.e. the sound came from. 本实施方式中以7个麦克风10组成数组型麦克风为例进行说明,其中,图3中包括AG共七个麦克风,所有麦克风设置于不同的位置以感测不同位置的声音信号。 In the present embodiment, seven microphones microphone array consisting of 10 as an example, in which, in FIG. 3 includes a total of seven AG microphones, all microphones arranged in different positions to sense a sound signal different locations. 每一麦克风旁边所显不的柱状图即表不该麦克风所接收的声音信号的强度。 Next to each of the microphone is not significant histogram i.e. sound signal strength received by the microphone should table. 该声源方位判断单元20根据柱状图即可判断出麦克风A所接收的声音信号的强度最大,因此即可判断声源方位为麦克风A处。 The sound source position determination unit 20 determines that the histogram to the intensity of the sound signal received by the microphone A maximum, and therefore can determine the orientation of the sound source at the microphone A. 上述例子只是为了简单说明声源方位判断单元20的工作原理,在实际工作中,该声源方位判断单元20会通过更为复杂的计算,比如将得到的各声音信号先通过傅立叶转换,之后再利用定位算法、空间平均法等计算出更为精确的声源方位,此技术已在声源方位估测及追踪的领域得到了较为广泛的应用,在此不再赘述。 Each audio signal of the examples are for the operational principle of the sound source position determination unit 20, in practice, the sound source position determination unit 20 will pass more complex calculation, such as obtained by the first Fourier transform, and then after to calculate more accurate sound source localization algorithm using the azimuth, spatial averaging method, this technology has been estimated in the sound source azimuth track and field has been more widely used, are not repeated here.

[0020] 该音量判断单元30用于判断通过麦克风10所接收的声音的音量大小,并判断其是否超过一设定值,即声音信号的分贝强度是否超过设定值。 [0020] The determination unit 30 for determining the volume of the volume received by the microphone 10 of the voice, and determining whether it exceeds a predetermined value, i.e., decibel sound signal strength exceeds a set value.

[0021] 当接收的声音的分贝强度超过设定值时,该语言转文字单元50则对接收的声音信号进行运算,以将声音信号转换为文字信息。 [0021] When the sound intensity in decibels received exceeds the set value, the language-text unit 50 pairs of an audio signal received operation to the audio signal is converted into a text message. 请参考图4,该语言转文字单元50包括声音译码模块500、文字字形库模块510以及码字匹配模块520。 Please refer to FIG. 4, the language to text decoding unit 50 includes a sound module 500, text font library module 510, and matching module 520 codewords. 该声音译码模块500用于将麦克风10接收到的声音信号进行译码识别,该码字匹配模块520用于将译码后的声音信号根据文字字形库模块510内存储的文字与声音信号进行匹配,以得到声音信号所对应的文字信息。 The speech decoding module 500 for the microphone 10 receives a sound signal to be decoded to identify the codeword matching module 520 for decoding the sound signal with the sound signal according to the text character font library module 510 stores the match, to obtain character information corresponding to the audio signal.

[0022] 请参考图5,该情境模拟单元80包括一虚拟罗盘模块800及一对白处理模块810。 [0022] Please refer to FIG. 5, the simulation unit 80 includes a context virtual compass module 800 and a processing module 810 white. 该虚拟罗盘模块800用于虚拟一罗盘,本实施方式中,该虚拟罗盘以使用者的正前方作为O度角基准。 The compass module 800 to virtual a virtual compass, the present embodiment, the virtual user of the compass in front of O as the reference angle. 该虚拟罗盘模块800还用于根据由声源方位判断单元20所得到的声源方位在虚拟罗盘的对应位置进行标识,以提示使用者此时的声源方位。 The virtual compass module 800 is further configured to identify the position of the sound source according to the corresponding position determination unit 20 resulting in a virtual sound source compass orientation, the orientation of the sound source to prompt the user at this time. 同时,该虚拟罗盘模块800还用于根据由音量判断单元30所判断得到的音量大小在虚拟罗盘的对应位置显示音量的大小,如通过强度来指示音量的大小。 Meanwhile, the virtual compass module 800 is further configured according to the size of the volume as indicated by the size of the volume by the volume obtained intensity determination unit 30 determines the volume displayed in the corresponding position of the virtual compass.

[0023] 该对白处理模块810用于根据语言转文字单元50所得到的文字信息调用对应的背景以及对白框(如调用漫画风格的背景及对白框等),并在对白框中显示对应的文字。 [0023] The processing module 810 for white background and call the corresponding dialogue language to text block from text unit 50, the information obtained (such as calling and comic style Background dialogue boxes, etc.), and displays the corresponding character in the dialogue box . 如果该语言转文字单元50无法将麦克风10所接收的声音信号转换为对应的文字信息(比如当声音信号为车辆的喇叭声或动物的叫声)时,该对白处理模块810则根据音量的大小调用对应的提示符号(如感叹号“ !”等)。 If the language-text unit 50 converts sound signals can not be received by the microphone 10 corresponding to character information (such as when the sound signal is a horn sounds of a vehicle or animal), the dialogue processing module 810 according to the size of the volume call the corresponding prompt (such as an exclamation point, "!", etc.).

[0024] 该显示单元60用于显示虚拟罗盘、音量的大小、背景、对白框以及对白。 [0024] The virtual display unit 60 for displaying the compass, the volume size, background, dialogue and the dialogue box. 本实施方式中,该显示单元60为眼镜100的镜片,该镜片采用双层材质制作,即使用者通过该镜片可观看到实际的场景,同时,该镜片还可作为显示器使用,用于显示虚拟罗盘、音量的大小等信息。 In the present embodiment, the display unit 60 as a spectacle lens 100, the lens material produced using double, i.e., the user can view a real scene by the lens, while the lens may be used as a display for displaying a virtual compass, volume size and other information. 此时,使用者通过眼镜100所观看到的图像即为实际场景与虚拟罗盘、音量的大小等信息的迭加影像。 In this case, the user through the glasses 100 to view an image superimposed image is the actual compass virtual scene, information such as the size of the volume. 请继续参考图6,基于该显示单元60的功能,其它实施方式中,该显示单元60亦可为一投影单元62,该投影单元62设置于镜框上,用于将虚拟罗盘、音量的大小等信息直接投影至使用者的眼球。 Please refer to FIG. 6, based on the display function unit 60, in other embodiments, the display unit 60 may also be a projection unit 62, the projection unit 62 is disposed on the frame, for virtual compass, the volume size, etc. projection information directly to the user's eye. 该眼镜100的镜片则为普通的镜片,此时使用者仍可观看到实际场景与虚拟罗盘、音量的大小等信息的迭加影像。 The spectacle lens 100, compared with an ordinary lens, then the user can still view the actual scene of the virtual image is superimposed compass, information such as the size of the volume. 进一步,请继续参考图7,该显示单元60包括显示屏61及一摄像头63,该摄像头63安装于该眼镜100的镜框上,用于拍摄使用者前方场景的影像。 Further, please continue to refer to FIG. 7, the display unit 60 includes a display screen 61 and a camera 63, the camera 63 is mounted on the frame of the glasses 100 for capturing images of the scene in front of the user. 该眼镜100的镜片则为显示屏61,用于显示来自摄像头63所拍摄的影像以及虚拟罗盘、音量的大小等信息,如此,使用者亦可观看到实际场景与虚拟罗盘、音量的大小等信息的迭加影像。 The spectacle lens 100, compared with display 61 for displaying images and virtual compass, size of the volume 63 taken from the camera and other information, so the user can view the virtual scene to the actual compass, information such as the size of the volume the superimposed image.

[0025] 下面将以一实例来说明上述声音反馈装置的具体工作原理: [0025] Next will be described a specific example of the above-described working principle of the acoustic feedback means:

请参考图8及图9,使用者620的身后左侧有一车辆600、身后右侧有一路人甲610,且车辆600鸣笛、路人甲610提醒使用者“小心车辆”。 Please refer to FIG. 8 and FIG. 9, left behind a user 620 of the vehicle 600 has, all the way behind the right person A 610, and the whistle vehicle 600, 610 to remind the user passerby "beware of traffic." 该若干麦克风10组成的数组型麦克风即对外界的声音进行感测,该声源方位判断单元20则对感测得到的声音信号进行判断,此时,由麦克风A及G所感测到的声音信号的强度明显高于其它麦克风BF所感测到的声音信号,因此,该声源方位判断单元20判断得知此时外界的声音来自麦克风A及G的位置。 Array microphone of the plurality of microphones 10 composition, i.e. on the outside of the sound sensing, the sound source position determination unit 20 pairs of the sound signal sensed obtained was determined, this time, the sound signal sensed by the microphone A and G sensed significantly higher than the intensity of the sound signal sensed by the other microphone to BF, therefore, the sound source position determination unit 20 determines that sound from outside this case a and G of the position of the microphone. 该音量判断单元30判断音量的大小,此时假设音量的大小大于设定的分贝强度。 The volume determination unit 30 determines the size of the volume, intensity in decibels is assumed at this time is larger than the size of the volume setting.

[0026] 该语言转文字单元50则对接收的声音信号进行运算。 [0026] The language-to-text unit 50 pairs of received sound signal operation. 本实施方式中,来自麦克风A方向的声音信号记为第一声音信号,来自麦克风G方向的声音信号记为第二声音信号,那么该语言转文字单元50则无法将第一声音信号转换为对应的文字信息,第二声音信号则被转换为“小心来车”的文字信息。 The present embodiment, the sound signal from the microphone referred to a first direction A and a sound signal, the sound signal from the microphone referred to as direction G second sound signal, then the language-text unit 50 can not first sound signal into a corresponding text message, the second voice signal were converted to "be careful to the car," the text message.

[0027] 该虚拟罗盘模块800生成一虚拟罗盘820,并根据得到的声源方位在虚拟罗盘的对应位置进行标识,以提示使用者此时的声源方位,本实施方式中,该虚拟罗盘820的225度角位置及135度角位置即为声源方位。 [0027] The virtualization module 800 generates a virtual compass compass 820, and identifier according to the obtained position of the corresponding virtual sound source compass orientation, the orientation of the sound source to prompt the user at this time, in the present embodiment, the virtual compass 820 225 degree angle and 135 degree angle is the orientation of the sound source. 同时,该虚拟罗盘模块820还根据判断得到的音量大小在虚拟罗盘的对应位置(即225度角位置及135度角位置)通过强度(即图9中斜线区域的大小)指示来显示音量的大小。 Meanwhile, the virtual compass module 820 also determines the volume obtained according to the corresponding position (i.e., 225-degree angle and 135 degree angle) to the compass display in the virtual volume indicated by the intensity (i.e., the hatched region 9 in FIG size) size.

[0028] 该对白处理模块810根据语言转文字单元50所得到的文字信息调用对应的背景以及对白框,并在对白框中显示对应的文字,即图9中包括感叹号“! ”的对白框以及包括“小心来车”的对白框,且该对白框相应出现在虚拟罗盘820的对应位置处,以提示使用者虚拟罗盘820的225度角位置出现不明声音、135度角位置出现“小心来车”的声音。 [0028] The processing module 810 in accordance with the dialogue language text-to-text unit 50 calls the information corresponding to the obtained background and dialogue boxes, and displays the corresponding character in the dialogue box, i.e. in FIG. 9 includes the exclamation point "!" Dialogue boxes and including "be careful to car" dialogue box, and the corresponding dialogue box appears at the corresponding position of the virtual compass 820 to prompt the user virtual compass 820 of 225-degree angle of unknown sounds, 135-degree angle appears "to drive carefully "the sound of.

[0029] 同时,根据上文的描述可知,此时使用者所观看到的影像为实际场景、虚拟罗盘、音量的大小、对白框以及对白等信息的迭加,如图9所示。 [0029] Meanwhile, according to the above description can be seen, when the user is viewing the actual scene image, virtual information superimposed compass, size of the volume, as well as dialogue boxes and the like dialogue, as shown in FIG.

[0030] 请参考图10,上述声音反馈装置的工作方法包括以下步骤: [0030] Please refer to FIG. 10, the working methods of the sound feedback means comprises the steps of:

步骤S1:该若干麦克风10感测外界的声音信号。 Step S1: the plurality of sound signal microphone 10 senses the outside.

[0031] 步骤S2 :该声源方位判断单元20判断声源的方位,即声音来自何方。 [0031] Step S2: the orientation of the sound source position determination unit 20 determines the sound source, i.e. the sound came from.

[0032] 步骤S3 :该音量判断单元30判断麦克风10所接收的声音的音量大小,并判断其是否超过一设定值,即声音信号的分贝强度是否超过设定值。 [0032] Step S3: the determination unit 30 determines the volume of the volume of sound received by the microphone 10, and determines whether it exceeds a predetermined value, i.e., decibel sound signal strength exceeds a set value. 若声音信号的分贝强度超过设定值,则执行步骤S4。 When the decibel sound signal strength exceeds a set value, step S4. 若声音信号的分贝强度未超过设定值,则返回至步骤SI。 When the decibel sound signal strength does not exceed the set value, the process returns to step SI.

[0033] 步骤S4 :该语言转文字单元50对接收的声音信号进行运算,以将声音信号转换为文字信息。 [0033] Step S4: The language-to-text unit 50 calculates the received sound signal to the sound signal into a text message.

[0034] 步骤S5 :该虚拟罗盘模块800虚拟一罗盘,并根据由声源方位判断单元20所得到的声源方位在虚拟罗盘的对应位置进行标识,以提示使用者此时的声源方位。 [0034] Step S5: The Virtual a virtual compass compass module 800, and the position identified by the sound source according to the corresponding position determination unit 20 resulting in a virtual sound source compass orientation, the orientation of the sound source to prompt the user at this time. 同时,该虚拟罗盘模块800还根据由音量判断单元30所判断得到的音量大小在虚拟罗盘的对应位置显示音量的大小,如通过强度来指示音量的大小。 Meanwhile, the virtual compass module 800 also displays the position corresponding to the size of the volume according to the volume obtained by the volume determining unit 30 determines the virtual compass, as indicated by the intensity of the size of the volume.

[0035] 步骤S6 :该对白处理模块810根据语言转文字单元50所得到的文字信息调用对应的背景以及对白框(如调用漫画风格的背景及对白框等),并在对白框中显示对应的文字。 [0035] Step S6: dialogue processing module 810 calls the corresponding background and dialogue language to text block from text unit 50, the information obtained (such as calling and comic style Background dialogue boxes, etc.), and the dialogue box displayed in the corresponding text. 如果该语言转文字单元50无法将麦克风10所接收的声音信号转换为对应的文字信息(比如当声音信号为车辆的喇叭声或动物的叫声)时,该对白处理模块810则根据音量的大小调用对应的提示符号(如感叹号“ !”等)。 If the language-text unit 50 converts sound signals can not be received by the microphone 10 corresponding to character information (such as when the sound signal is a horn sounds of a vehicle or animal), the dialogue processing module 810 according to the size of the volume call the corresponding prompt (such as an exclamation point, "!", etc.).

[0036] 步骤S7 :该显示单元60显示虚拟罗盘、音量的大小、背景、对白框以及对白。 [0036] Step S7: The display unit 60 displays the virtual compass, the volume size, background, dialogue and the dialogue box.

[0037] 上述声音反馈装置及其工作方法通过麦克风10感测声音信号,并通过声源方位判断单元20判断声音的方向,当声音的音量超过设定值时即通过语言转文字单元50将声音信号转换为文字信息,并通过显示单元60显示给使用者,以起到辅助听力障碍人士的作用。 [0037] The apparatus and operating method of a feedback sound by the sound source position determination unit 20 determines the direction of the sound, when the sound volume exceeds the set value i.e. the language to text via the microphone unit 10 by sensing a sound signal, the sound 50 signal is converted to text information, through the display unit 60 and displayed to the user to play a supporting role in the hearing impaired.

Claims (8)

1. 一种声音反馈装置,包括:若干麦克风,用于感测外界的声音信号;一声源方位判断单元,用于对各麦克风所接收的声音信号进行计算,以判断声源的方位;一音量判断单元,用于判断麦克风所接收的声音的音量大小,并判断其是否超过一设定值;一语言转文字单元,用于在麦克风所接收的声音的音量大小超过设定值时将声音信号转换为文字信息;一情境模拟单元,包括一虚拟罗盘模块及一对白处理模块,该虚拟罗盘模块用于虚拟一罗盘,并根据由声源方位判断单元所得到的声源方位以及音量判断单元所得到的音量大小在虚拟罗盘的对应位置进行标识,以提示使用者此时的声源方位及音量大小,该对白处理模块用于根据语言转文字单元所得到的文字信息调用对应的对白框,并在对白框中显示对应的文字;以及一显示单元,用于显示虚拟罗盘、声 1. A sound feedback device, comprising: a plurality of microphone for sensing the sound signal outside; sound source position determination unit, for each of the sound signals received by the microphone is calculated to determine the azimuth of the sound source; a volume determination means for determining the volume of sound received by the microphone, and determining whether it exceeds a predetermined value; unit when a language to text, received by the microphone for the sound volume of the audio signal exceeds a set value converted to text information; a situation simulating unit comprises a compass module and a virtual dialogue processing module, the module for the virtual virtual compass a compass, and a sound source from the sound source position determination unit, and a volume of the obtained azimuth determination unit volume obtained in the corresponding position of the virtual identifier of the compass, to prompt the user to the sound source position and the volume at this time, the white frame processing module for the dialogue language to text units based on the character information obtained corresponding to the call, and displays a corresponding text dialogue box; and a display unit for displaying virtual compass, sound 方位、音量大小以及文字信息。 Orientation, volume and text information.
2.如权利要求1所述的声音反馈装置,其特征在于:当该语言转文字单元无法将声音信号转换为文字信息时,该对白处理模块则在对白框中显示提示符号。 2. The sound feedback device according to claim 1, wherein: when the language-text unit can not convert the sound signal to character information, displays the dialog prompt processing module in the dialogue box.
3.如权利要求1所述的声音反馈装置,其特征在于:该显示单元为一投影单元,该投影单元用于接收虚拟罗盘、声源方位、音量大小以及文字信息并将其投影至使用者的眼球上。 3. The sound feedback device according to claim 1, wherein: the display unit is a projection unit, for receiving the projection unit compass virtual sound source position, volume and character information to the user and the projection on the eye.
4.如权利要求1所述的声音反馈装置,其特征在于:该显示单元包括一显示屏及一摄像头,该摄像头用于拍摄使用者前方场景的图像,该显示屏用于显示摄像头所拍摄得到的图像以及虚拟罗盘、声源方位、音量大小、文字信息。 Obtaining the captured display unit comprises a display and a camera, the camera for capturing an image of a scene in front of the user, the display screen for displaying the camera: acoustic feedback apparatus as claimed in claim 1, characterized in that the images and virtual compass, the sound source orientation, volume, text messages.
5. 一种声音反馈装置的工作方法,包括:通过若干麦克风感测外界的声音信号;通过一声源方位判断单元对各麦克风所接收的声音信号进行计算,以判断声源的方位;通过一音量判断单元判断麦克风所接收的声音的音量大小,并判断其是否超过一设定值;当麦克风所接收的声音的音量大小超过设定值时,通过一语言转文字单元将声音信号转换为文字信息;通过一虚拟罗盘模块虚拟一罗盘,并根据由声源方位判断单元所得到的声源方位以及音量判断单元所得到的音量大小在虚拟罗盘的对应位置进行标识,以提示使用者此时的声源方位及音量大小;通过一对白处理模块根据语言转文字单元所得到的文字信息调用对应的对白框,并在对白框中显示对应的文字;以及通过一显示单元显示虚拟罗盘、声源方位、音量大小以及文字信息。 A working method of acoustic feedback means, comprising: a plurality of sound signals by the microphone sensing outside; sound source position determination by means of sound signals received by the microphones is calculated, to determine the azimuth of the sound source; a through volume determination means determines the volume of sound received by the microphone, and determining whether it exceeds a predetermined value; when the volume of sound received by the microphone exceeds a set value, by a language to text unit to the sound signal into a text message ; virtual a virtual compass by a compass module, and identified in the corresponding position of the virtual volume according to the compass by the sound source position determination unit because the sound source position determination unit, and a volume obtained at this time to prompt the user to sound source orientation and volume; white invoked by a processing module unit to text literal language of the information obtained corresponding to the dialogue box, and displays the corresponding character in the dialogue box; compass and displaying the virtual sound source position by a display unit, volume and text information.
6.如权利要求5所述的工作方法,其特征在于:当该语言转文字单元无法将声音信号转换为文字信息时,该对白处理模块则在对白框中显示提示符号。 Working method according to claim 6, wherein: when the language-text unit can not convert the sound signal to character information, displays the dialog prompt processing module in the dialogue box.
7.如权利要求5所述的工作方法,其特征在于:该显示单元为一投影单元,该投影单元用于接收虚拟罗盘、声源方位、音量大小以及文字信息并将其投影至使用者的眼球上。 7. The method of working of claim 5, wherein: the display unit is a projection unit, for receiving the projection unit compass virtual sound source position, text size, and volume information to a user and projects eye on.
8.如权利要求5所述的工作方法,其特征在于:该显示单元包括一显示屏及一摄像头,该摄像头用于拍摄使用者前方场景的图像,该显示屏用于显示摄像头所拍摄得到的图像以及虚拟罗盘、声源方位、音量大小、文字信息。 8. The method of working of claim 5, wherein: the display unit includes a display and a camera, the camera for capturing an image of a scene in front of the user, the display screen for displaying the camera is photographed images and virtual compass, the sound source orientation, volume, text messages.
CN2011103116491A 2011-10-14 2011-10-14 Sound feedback device and working method thereof CN103049077A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN2011103116491A CN103049077A (en) 2011-10-14 2011-10-14 Sound feedback device and working method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN2011103116491A CN103049077A (en) 2011-10-14 2011-10-14 Sound feedback device and working method thereof

Publications (1)

Publication Number Publication Date
CN103049077A true CN103049077A (en) 2013-04-17

Family

ID=48061747

Family Applications (1)

Application Number Title Priority Date Filing Date
CN2011103116491A CN103049077A (en) 2011-10-14 2011-10-14 Sound feedback device and working method thereof

Country Status (1)

Country Link
CN (1) CN103049077A (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105554662A (en) * 2015-06-30 2016-05-04 宇龙计算机通信科技(深圳)有限公司 Hearing-aid glasses and hearing-aid method
CN106462383A (en) * 2014-06-03 2017-02-22 哈曼国际工业有限公司 Hands free device with directional interface
CN107209375A (en) * 2015-10-23 2017-09-26 深圳市柔宇科技有限公司 Head-mounted display device, and prompt information display system and method
CN108877407A (en) * 2018-06-11 2018-11-23 北京佳珥医学科技有限公司 Method, device and system for assistance of communication and augmented reality glasses
CN108899029A (en) * 2018-06-11 2018-11-27 北京佳珥医学科技有限公司 Method, device and system for focusing on voice of single sound source
CN109032545A (en) * 2018-06-11 2018-12-18 北京佳珥医学科技有限公司 Method and device for providing sound source information and augmented reality glasses
US10318016B2 (en) 2014-06-03 2019-06-11 Harman International Industries, Incorporated Hands free device with directional interface

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6629076B1 (en) * 2000-11-27 2003-09-30 Carl Herman Haken Method and device for aiding speech
CN201365285Y (en) * 2009-03-10 2009-12-16 胡礼斌 Hearing aid mobile phone applicable to the deaf
US20110221669A1 (en) * 2010-02-28 2011-09-15 Osterhout Group, Inc. Gesture control in an augmented reality eyepiece

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6629076B1 (en) * 2000-11-27 2003-09-30 Carl Herman Haken Method and device for aiding speech
CN201365285Y (en) * 2009-03-10 2009-12-16 胡礼斌 Hearing aid mobile phone applicable to the deaf
US20110221669A1 (en) * 2010-02-28 2011-09-15 Osterhout Group, Inc. Gesture control in an augmented reality eyepiece

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10318016B2 (en) 2014-06-03 2019-06-11 Harman International Industries, Incorporated Hands free device with directional interface
CN106462383A (en) * 2014-06-03 2017-02-22 哈曼国际工业有限公司 Hands free device with directional interface
CN105554662A (en) * 2015-06-30 2016-05-04 宇龙计算机通信科技(深圳)有限公司 Hearing-aid glasses and hearing-aid method
CN107209375A (en) * 2015-10-23 2017-09-26 深圳市柔宇科技有限公司 Head-mounted display device, and prompt information display system and method
CN108877407A (en) * 2018-06-11 2018-11-23 北京佳珥医学科技有限公司 Method, device and system for assistance of communication and augmented reality glasses
CN108899029A (en) * 2018-06-11 2018-11-27 北京佳珥医学科技有限公司 Method, device and system for focusing on voice of single sound source
CN109032545A (en) * 2018-06-11 2018-12-18 北京佳珥医学科技有限公司 Method and device for providing sound source information and augmented reality glasses

Similar Documents

Publication Publication Date Title
JP4439740B2 (en) Speech conversion system and method
KR101011543B1 (en) Method and apparatus for creating a multi-dimensional communication space for use in a binaural audio system
JP4995266B2 (en) Apparatus and method for detecting sound
RU2595943C2 (en) Audio system and method for operation thereof
US6961439B2 (en) Method and apparatus for producing spatialized audio signals
Shilling et al. Virtual auditory displays
US6925438B2 (en) Method and apparatus for providing an animated display with translated speech
RU2678361C1 (en) Audio navigation assistance
ES2592217T3 (en) An apparatus and method for converting a first signal parametric spatial audio signal in a second spatial audio Parametric
JP5067595B2 (en) The image display apparatus and method, and program
CN103988113B (en) Indirect wearable computing device having a bone conduction speaker
US8493204B2 (en) Displaying sound indications on a wearable computing system
EP0938832B1 (en) Method and device for projecting sound sources onto loudspeakers
US20130279724A1 (en) Auto detection of headphone orientation
US9888335B2 (en) Method and apparatus for processing audio signals
WO2015125626A1 (en) Display control device, display control method, and computer program
US20070195012A1 (en) Image display apparatus and method for displaying image
US20010046659A1 (en) System for improving reading & speaking
JPH11331827A (en) Television camera
CN101350931A (en) Method and device for generating and playing audio signal as well as processing system thereof
JP2001054200A (en) Sound delivery adjustment system and method to loudspeaker
JP2014180044A (en) Technique for localized perceptual audio
US10111013B2 (en) Devices and methods for the visualization and localization of sound
JP2012503935A (en) Automatically operable directional hearing aid and its method of operation
EP1507419A4 (en) Information processing apparatus, information processing system, and dialogist displaying method

Legal Events

Date Code Title Description
C06 Publication
C10 Entry into substantive examination
C02 Deemed withdrawal of patent application after publication (patent law 2001)