KR20140118060A - Portable teriminal and sound output apparatus and method for providing locations of sound sources in the portable teriminal - Google Patents

Abstract

Various embodiments of the present invention to provide locations of sound sources have been stated, and according to an embodiment of the present invention, the method to provide locations of sound sources in a portable terminal having a touch screen includes receiving sound signals of a plurality of sound sources; securing distances and directions among the terminal and the sound sources based on each sound signal of the sound sources; determining positions of the sound sources on the touch screen by using the secured distances and the direction of each sound source; and displaying the locations of the sound sources on the touch screen according to the determined positions. Other various applications are also possible.

Description

TECHNICAL FIELD [0001] The present invention relates to a portable terminal and a method for providing a location of a sound source in a portable terminal and a hearing aid. [0002]

Various embodiments of the present invention are directed to a portable terminal and a method for providing a hearing aid and sound source location.

Recently, various services and additional functions provided by terminals such as smart phones, mobile phones, and tablet PCs are gradually expanding. Such a terminal includes a camera, a microphone, a speaker, a display, a wireless communication module, and various sensors, and provides various functions that can be executed in the terminal to satisfy various needs of users.

Such a terminal can provide various visual UIs (User Interfaces) using a display. By visualizing a sound source using the display of the terminal and providing control over the sound source, It will be convenient for people with disabilities.

For example, if the terminal detects a surrounding sound source and provides a function of displaying the location of the sound source, not only a general user but also a hearing-impaired user can more conveniently use the terminal.

Accordingly, it is an object of various embodiments of the present invention to provide a portable terminal, a hearing aid, and a method for providing a location of a sound source by detecting the surrounding sound sources and displaying the location of the sound sources.

Another object of the various embodiments of the present invention is to provide a portable terminal, a hearing aid, and a method for providing a location of a sound source for allowing a user to confirm the location of nearby sound sources and selectively listen to a desired sound source .

It is another object of various embodiments of the present invention to provide a portable terminal capable of selecting a direction or a range or a distance or a position of a sound source to be heard by a user using various gestures and outputting a selected sound source using various gestures, And a method for providing a position of a hearing aid and a sound source.

According to various embodiments of the present invention, there is provided a method of providing a location of a sound source in a portable terminal having a touch screen, the method comprising: receiving a sound signal of a plurality of sound sources; And determining a position of the plurality of sound sources on the touch screen by using the distance and direction of each of the plurality of sound sources obtained; And displaying the positions of the plurality of sound sources on the touch screen according to the position.

In addition, various embodiments of the present invention provide an audio output method in a hearing aid, comprising: receiving a sound signal of a sound source selected by a user's gesture from a portable terminal through a touch screen and adjusted in volume; And outputting a sound signal.

In addition, according to various embodiments of the present invention, there is provided a portable terminal having a touch screen, comprising: a touch screen for displaying positions of a plurality of sound sources; a touch screen for displaying positions of the plurality of sound sources; Determining a position of the plurality of sound sources on the touch screen by using a distance and a direction of each of the obtained plurality of sound sources, and determining, based on the determined position, And a control unit for controlling the display unit.

In addition, various embodiments of the present invention are directed to a hearing aid having a wireless unit that receives a sound signal of a sound source selected by a user's gesture from a portable terminal through a touch screen and whose volume is adjusted, And an output unit for outputting.

The various embodiments of the present invention allow a user to utilize the display of the terminal to know the location of sound sources located in all directions with respect to the user.

In addition, according to the embodiments of the present invention, a user can select a sound source desired to be heard among the sound sources to only listen to the selected sound source, and can adjust the volume of the sound source while listening to the sound source .

In addition, according to embodiments of the present invention, in the case of a hearing-impaired person using a hearing aid, the position of the sound sources can be known through the terminal by interlocking the hearing aid and the terminal, and the desired sound source can be heard through the hearing aid.

In addition, according to embodiments of the present invention, it is possible to provide a more convenient user interface (UI) to a user by allowing a user to listen to a desired sound source among sound sources using various user gestures.

1 is a view for explaining a concept of operation of a terminal and an audio output device according to an embodiment of the present invention;
2 is a block diagram of a terminal according to an embodiment of the present invention.
3 is a flowchart illustrating an operation of displaying at least one sound source in the vicinity of a terminal according to an embodiment of the present invention.
4 is a diagram illustrating a radar scope screen for displaying sound sources existing around a terminal according to an embodiment of the present invention;
5 is a view showing a sound source location display screen for displaying sound sources existing in the vicinity of a terminal according to an embodiment of the present invention;
6 is a flowchart illustrating a sound output operation of a sound source of a terminal according to an embodiment of the present invention.
FIG. 7 is a diagram illustrating the surroundings of sound sources displayed on a terminal screen according to an embodiment of the present invention; FIG.
8 is a flowchart illustrating a sound source selection operation according to user gesture recognition in a terminal according to an embodiment of the present invention.
9 is a view for explaining a user gesture input indicating a sound source selection direction according to an embodiment of the present invention;
FIG. 10 is a view for explaining a user gesture input indicating a sound source selection range according to an embodiment of the present invention; FIG.
11 is a view for explaining a user gesture input indicating a sound source selection distance according to an embodiment of the present invention;
12 is a view for explaining a user gesture input indicating a sound source selection position according to an embodiment of the present invention;
13 is a block diagram of a hearing aid according to an embodiment of the present invention.
FIG. 14 is a flowchart of the operation of the hearing aid according to the embodiment of the present invention.

The terms used in this specification will be briefly described and the present invention will be described in detail.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. Also, in certain cases, there may be a term selected arbitrarily by the applicant, in which case the meaning thereof will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term, not on the name of a simple term, but on the entire contents of the present invention.

When an element is referred to as "including" an element throughout the specification, it is to be understood that the element may include other elements, without departing from the spirit or scope of the present invention. Also, the terms "part," " module, "and the like described in the specification mean units for processing at least one function or operation, which may be implemented in hardware or software or a combination of hardware and software .

Throughout the specification, "user gesture input" means input by a gesture performed by the user on the touch screen to control the terminal. For example, the gesture input described herein may include touch, touch & hold, drag, flick, hold, and the like.

"Touch" indicates an operation in which the user touches the screen using a finger or a touch input unit (stylus or pen). That is, the time difference between the touch-in point, which is the point at which the finger or the touch input unit touches the screen, and the touch-out point, which is the point at which the finger or the touch tool falls off the screen, is a predetermined touch time.

"Touch & hold" represents an operation in which the user touches the screen using a finger or a touch input unit (stylus or pen) and then maintains the touch input over a critical time. That is, the time difference between the touch-in point and the touch-out point is equal to or greater than the threshold time. In order to allow the user to recognize whether the touch input is a touch or a touch & hold, a feedback signal may be provided visually or audibly when the touch input is maintained for a predetermined time or more.

"Drag" means an operation of moving a finger or a touch input unit to another position in the screen while the user holds the touch after touching the finger or the touch input unit with the screen.

"Flick" indicates an operation in which a user drags very quickly using a finger or a touch input unit. It is possible to distinguish the drag and the flick based on whether the moving speed of the finger or the touch input unit is equal to or greater than the threshold speed.

The "hold" represents an operation in which the user holds a touch input over a predetermined time at a certain point in a state where the touch is touched at a certain point on the screen using a finger or a touch input unit (stylus or pen). That is, the time difference between the touch-in point and the touch-out point at a certain point is equal to or greater than the threshold time.

Hereinafter, various embodiments according to the present invention will be described in detail with reference to the contents described in the accompanying drawings. However, the present invention is not limited or limited by various embodiments. Like reference numerals in the drawings denote members performing substantially the same function.

1 is a view for explaining operation concepts of a terminal and an audio output device according to an embodiment of the present invention.

1, the terminal 100 may be a portable terminal, detects positions of at least one sound source existing around the terminal 100 (or a user) 100) and the sound source images (11 to 17) representing at least one sound source at the position of the at least one sound source based on the terminal (100).

Here, a sound source can mean the source from which the sound comes or its source. For example, a sound source may include a person making a sound, a player performing a performance, a speaker outputting music, a phone ringing a ringtone, and the like.

The terminal 100 may select a sound source image to output sound according to a user gesture input among the displayed at least one sound source images 11 to 17 and output a sound of a sound source corresponding to the selected sound source image. Also, the terminal 100 may provide the sound of the sound source corresponding to the selected sound source image to the sound output device 300 (e.g., a hearing aid, an ear microphone, a headset, etc.).

In addition, the terminal 100 selects at least one volume level among a plurality of preset volume levels for an arbitrary sound source according to a user's gesture input, and outputs sound of the sound source according to the selected volume level upon outputting sound of the sound source can do. Also, the terminal 100 may provide the sound of the sound source according to the selected volume level to the sound output device 300.

The audio output device 300 may be a device capable of outputting sounds such as a hearing aid, an ear microphone, a headset, and the like. In the embodiment of the present invention, the case where the sound output device 300 is a hearing aid will be described as an example. The hearing aid 300 can receive a sound signal from at least one sound source using a microphone and amplify it, and output the amplified sound through a speaker so that the user can hear the sound.

The hearing aid 300 can communicate with the terminal 100 (wired communication or wireless communication), and can receive and output sound of a selected sound source from at least one sound source from the terminal 100. Also, the hearing aid 300 can receive and output the sound of the sound source whose volume of the sound source is adjusted according to the volume level selected from the terminal 100.

2 is a block diagram of a terminal 100 according to an embodiment of the present invention. 2, the terminal 100 may include a control unit 110, a reflected wave transmission / reception unit 113, a sound source distance calculation unit 115, and a sound source direction calculation unit 117. The terminal 100 includes a mobile communication module 120, a sub communication module 130, a multimedia module 140, a camera module 150, a GPS module 157, an input / output module 160, a sensor module 170 ), A storage unit 175, and a power supply unit 180. The terminal 100 may also include at least one touch screen 190 and at least one touch screen controller 195.

The control unit 110 stores a ROM or a ROM 112 storing a control program for controlling the terminal 100 and a signal or data input from the outside of the terminal 100, And a RAM (RAM) 113 used as a storage area for a work to be performed. The CPU 111 may include a single core, a dual core, a triple core, or a quad core. The CPU 111, the ROM 112, and the RAM 113 may be interconnected via an internal bus. The control unit 110 includes a reflected wave transmission / reception unit 103, a sound source distance calculation unit 105, a sound source direction calculation unit 107, a mobile communication module 120, a sub communication module 130, a multimedia module 140, A touch panel 190, a touch panel 190, and a touch screen controller 195. The touch panel 190 is connected to the controller 150, the GPS module 157, the input / output module 160, the sensor module 170, the storage 175, the power supply 180, Can be controlled.

The reflected wave transmitting / receiving unit 113 can transmit the electromagnetic wave signal in all directions under the control of the control unit 110, and can receive the reflected wave signal in which the electromagnetic wave signal collides against the object and is reflected. The received reflected wave signal is transmitted to the controller 110 and can be used to detect an object in the vicinity of the terminal 100.

The sound source distance calculation unit 115 calculates a sound source distance by using various sound sources input through the first to nth microphones 162-1 to 162-n under the control of the controller 110, And distinguishes at least one sound source, and calculates a distance from the terminal 100 to each sound source. At this time, the distances from the terminal 100 to the respective sound sources are obtained by combining a plurality of sound sources inputted through at least two microphones among the first to n-th microphones 162-1 to 162-n to generate a plurality of microphone output signals, The distance to each sound source is calculated using the rate of sound attenuation according to the distance of each of the plurality of microphone output signals.

The sound source direction calculation unit 117 generates a sound or various sounds by various sound sources input through the first to nth microphones 162-1 to 162-n according to the control of the controller 110. [ And distinguishes at least one sound source, and calculates the direction of each sound source. At this time, the direction of each sound source calculates the direction of each sound source using the time delay between the sound source signals input through the at least two microphones among the first to nth microphones 162-1 to 162-n. The calculated direction of each sound source is transmitted to the controller 110.

The control unit 110 can detect the positions of at least one or more objects existing in all directions with reference to the position of the terminal 100 (or the user) using the reflected wave received by the reflected wave transmitting / receiving unit 113. [

The control unit 110 receives voice or various sounds from various sound sources input through the first through n-th microphones 162-1 through 162-n to distinguish at least one sound source have. At this time, the controller 110 identifies at least one sound source, and then analyzes frequency components of at least one sound source and compares a frequency analysis result of at least one sound source with a predetermined frequency component to determine at least one type of sound sources . For example, the control unit 110 can determine whether at least one sound source is a human voice, a cry of an animal, a telephone ring tone, or a sound of a vehicle.

The control unit 110 can calculate the sound intensity of each sound source and receives the distances from the terminal 100 to the respective sound sources through the sound source distance calculation unit 113, Lt; / RTI >

According to another embodiment of the present invention, the control unit 110 may not be the first to the n-th microphones 162-1 to 162-n, but may also be a sound source of various sound sources input through the plurality of microphones from the hearing aid 300 voice, or sound to discriminate at least one sound source, and calculate the intensity and direction of each sound source and the distance from the terminal 100 to the sound source. In this case, the hearing aid 300 may include at least two microphones, and may transmit voice or various sounds by various sound sources input through the at least two microphones to the terminal 100 through communication.

The control unit 110 detects a terminal image representing the terminal 100 on the touch screen 190 using the detected positions of the objects, the intensity of the detected sound source, the direction of the sound source, and the distances from the terminal 100 to the respective sound sources, So that the sound source images representing the sound sources are displayed. For example, the control unit 110 displays a terminal image and sound source images corresponding to the detected sound source on the radar scope screen, and displays the distance between the terminal 100 and each sound source, the direction angle of each sound source, At least one of them can be displayed together.

At this time, when the type of the sound source is human voice, the controller 110 grasps the location information of the person corresponding to the voice, receives the phone number and name of the person corresponding to the voice from the external service provider such as a communication service company, Together, they can also display the phone number and name of the person corresponding to the sound source.

Here, the radar scope screen is a screen capable of indicating the distance and direction of a sound source around the terminal 100. The sound source image may be an image of a figure, a character, or the like representing the sound source, and the sound source image may have a shape in which the color or size is distinguished according to the intensity and distance of the sound source.

For example, if the sound intensity of a sound source is large, the sound source image may have a larger size or a darker color than when the sound source has a small intensity. Also, if the sound source is close to the sound source, the sound source may have a larger size or a darker color than the sound source is far away. In addition, the shape, size and color of the sound source image can be selectively changed according to the sound intensity of the sound source and the distance of the sound source.

In addition, the controller 110 may control the terminal image and sound source images to be displayed on the sound source position display screen having a predetermined background other than the radar scope screen.

On the other hand, the control unit 110 determines whether there is a user gesture by the touchable input unit 168 such as a finger or an electronic pen in a state where a terminal image and at least one sound source images are displayed on the touch screen 190 do. At this time, the controller 110 may detect a user gesture input generated when the user performs a gesture using the finger or the input unit 168 on the touch screen 190. According to an embodiment of the present invention, the user gesture input may include at least touch, touch & hold, drag, flick, hold. Other types of user gesture input are also possible.

The control unit 110 may control the speaker 163 to select a sound source image to output sound according to a user's gesture input among at least one sound source images and to output a sound of a corresponding sound source image.

In addition, when at least one volume level of a predetermined plurality of volume levels is selected for an arbitrary sound source image according to a user's gesture input, the controller 110 controls the speaker (not shown) to output the sound of the corresponding sound source according to the selected volume level 163 can be controlled.

According to another embodiment of the present invention, the control unit 110 selects a sound source image to output sound according to a user's gesture input among at least one sound source images, and transmits the sound of the corresponding sound source to the hearing aid 300 So that the hearing aid 300 can output the sound of the sound source corresponding to the selected sound source image.

When at least one volume level of a plurality of predetermined volume levels is selected for an arbitrary sound source image according to a user's gesture input, the controller 110 controls the volume of the sound source, (300) so that the hearing aid (300) outputs the sound of the sound source whose volume is adjusted.

The mobile communication module 120 may connect the terminal 100 to an external device through mobile communication using at least one or more antennas (not shown) under the control of the controller 110. The mobile communication module 120 is connected to the mobile communication module 120 through a voice communication, a video communication, and a voice communication with a mobile phone (not shown) having a phone number input to the terminal 100, a smart phone (not shown), a tablet PC, And transmits / receives a radio signal for a text message (SMS) or a multimedia message (MMS).

The sub communication module 130 may include at least one of a wireless LAN module 131 and a local area communication module 132. For example, it may include only the wireless LAN module 131, only the short range communication module 132, or both the wireless LAN module 131 and the short range communication module 132.

The wireless LAN module 131 may be connected to the Internet at a place where an access point (AP) (not shown) is installed under the control of the controller 110. [ The wireless LAN module 131 supports the IEEE 802.11x standard of the Institute of Electrical and Electronics Engineers (IEEE). The short-range communication module 132 can perform short-range wireless communication between the portable terminal 100 and the image forming apparatus (not shown) under the control of the controller 110. [ The short-distance communication method may include bluetooth, infrared data association (IrDA), WiFi-Direct communication, and Near Field Communication (NFC).

The terminal 100 can communicate with the hearing aid 300 through at least one of the mobile communication module 120 and the sub communication module 130. [ At this time, the sound of the sound source can be transmitted to the hearing aid 300 through at least one of the mobile communication module 120 and the sub communication module 130. For example, the sound of the sound source corresponding to the sound source selected by the user through at least one of the mobile communication module 120 or the sub communication module 130. [ Also, the user can transmit the sound of the sound source whose volume is adjusted according to the volume level selected by the user to at least one of the mobile communication module 120 or the sub communication module 130 to the hearing aid 300.

The terminal 100 may include at least one of a mobile communication module 120, a wireless LAN module 131, and a local communication module 132 according to performance. Also, the terminal 100 may include a combination of the mobile communication module 120, the wireless LAN module 131, and the local communication module 132 according to performance. In the present invention, at least one of the mobile communication module 120, the wireless LAN module 131, and the short-range communication module 132, or a combination thereof, is referred to as a transmission / reception section, and this does not narrow the scope of the present invention.

The multimedia module 140 may include a broadcasting communication module 141, an audio reproducing module 142, or a moving picture reproducing module 143. The broadcast communication module 141 receives a broadcast signal (e.g., a TV broadcast signal, a radio broadcast signal, or a data broadcast signal) transmitted from a broadcast station through a broadcast communication antenna (not shown) under the control of the controller 110, (E. G., An Electric Program Guide (EPS) or an Electric Service Guide (ESG)). The audio playback module 142 may play back a digital audio file (e.g., a file having a file extension of mp3, wma, ogg, or wav) stored or received under the control of the controller 110. [ The moving picture playback module 143 may play back digital moving picture files (e.g., files having file extensions mpeg, mpg, mp4, avi, mov, or mkv) stored or received under the control of the controller 110. [ The moving picture reproducing module 143 can reproduce the digital audio file.

The multimedia module 140 may include an audio reproduction module 142 and a moving picture reproduction module 143 except for the broadcasting communication module 141. [ The audio reproducing module 142 or the moving picture reproducing module 143 of the multimedia module 140 may be included in the controller 110.

The camera module 150 may include at least one of a first camera 151 and a second camera 152 for capturing still images or moving images under the control of the controller 110. [ The camera module 150 includes a barrel section 155 for zooming in / zooming out to photograph a subject, a motor section 154 for controlling the movement of the barrel section 155, an auxiliary light source And a flash 153 that provides a flash memory. The first camera 151 may be disposed on the front surface of the portable terminal 100 and the second camera 152 may be disposed on the rear surface of the portable terminal 100. The distance between the first camera 151 and the second camera 152 is larger than 1 cm and smaller than 8 cm) in a manner different from the first camera 151 and the second camera 152 Dimensional still image or a three-dimensional moving image. The camera module 150 transmits the photographed photograph or image to the controller 110. [

The first and second cameras 151 and 152 may each include a lens system, an image sensor, and the like. The first and second cameras 151 and 152 convert an optical signal inputted (or photographed) through a lens system into an electrical image signal and output the electrical signal to the control unit 110, The moving image or the still image can be photographed through the image sensors 151 and 152.

The GPS module 157 receives the radio waves from a plurality of GPS satellites (not shown) on the earth orbit, and uses the time of arrival from the GPS satellites (not shown) to the terminal 100 The position of the terminal 100 can be calculated.

The input / output module 160 includes a plurality of buttons 161, first to nth microphones 162-1 to 162-n, a speaker 163, a vibration motor 164, a connector 165, a keypad 166 , And an earphone connection jack 167. The earphone connection jack 167 may be an earphone jack. The input / output module 160 is not limited to this, and a cursor control such as a mouse, a trackball, a joystick, or cursor direction keys communicates with the control unit 110. The cursor movement control Lt; / RTI >

Button 161 may be formed on the front, side, or rear surface of the housing of the terminal 100 and may include a power / lock button (not shown), a volume button (not shown), a menu button, A back button, and a search button (not shown).

Each of the first to nth microphones 162-1 to 162-n receives voice or sounds of various sound sources under the control of the controller 110 to generate an electrical signal. For example, the first through n-th microphones 162-1 through 162-n may be a microphone, a sound played by a player, a first speech by a first person, a telephone sound by a telephone, The sound of various sound sources existing in the vicinity of the sound source 100 can be received.

The speaker 163 may transmit various signals (for example, a radio signal, a broadcast signal, a radio signal, a radio signal, a radio signal, a radio signal, and the like) of the mobile communication module 120, the sub communication module 130, the multimedia module 140, or the camera module 150 under the control of the controller 110. [ A digital audio file, a digital moving picture file, a picture photographing, or the like) to the outside of the terminal 100. In addition, according to another embodiment, the speaker 163 may amplify the sound corresponding to the sound source selected by the user among the sound sources around the terminal 100 under the control of the controller 110, and output the amplified sound to the outside. The sound corresponding to such a sound source can be controlled in volume according to the user gesture input. The speaker 163 may be formed at one or a plurality of appropriate positions or positions of the housing of the terminal 100.

The vibration motor 164 can convert an electrical signal into a mechanical vibration under the control of the control unit 110. [ For example, when the terminal 100 in the vibration mode receives a voice call from another device (not shown), the vibration motor 164 operates. Or may be formed in the housing of the terminal 100. The vibration motor 164 may operate in response to the user's touching operation on the touch screen 190 and the continuous movement of the touch on the touch screen 190.

The connector 165 may be used as an interface for connecting the terminal 100 to an external device (not shown) or a power source (not shown). The terminal 100 transmits data stored in the storage unit 175 of the terminal 100 to an external device (not shown) through a cable connected to the connector 165 under the control of the controller 110, May receive data from a device (not shown). In addition, the terminal 100 may receive power from a power source (not shown) through a wire cable connected to the connector 165, or may charge the battery (not shown) using the power source.

The keypad 166 may receive a key input from a user for control of the mobile terminal 100. [ The keypad 166 includes a physical keypad (not shown) formed on the portable terminal 100 or a virtual keypad (not shown) displayed on the touch screen 190. A physical keypad (not shown) formed in the portable terminal 100 may be excluded depending on the performance or structure of the portable terminal 100. [

An earphone connecting jack (earphone connecting jack) 167 may be connected to the terminal 100 by inserting an earphone (not shown).

The input unit 168 may be inserted and stored in the terminal 100, and may be taken out or removed from the terminal 100 during use. A detachable attachment recognition switch (not shown) is provided in one area of the terminal 100 in which the input unit 168 is inserted and is operated in response to attachment and detachment of the input unit 168, And can provide signals corresponding to the mounting and dismounting of the input unit 168. The attachment / detachment recognition switch is provided in a region where the input unit 168 is inserted, and is provided so as to be in direct or indirect contact with the input unit 168 when the input unit 168 is mounted. Thus, the detachable attachment recognition switch generates a signal corresponding to the attachment or detachment of the input unit 168, based on the direct or indirect contact with the input unit 168, and provides the signal to the control unit 110.

The sensor module 170 includes at least one sensor for detecting the state of the terminal 100. [ For example, the sensor module 170 may include a proximity sensor that detects whether the user accesses the terminal 100, an illuminance sensor (not shown) that detects the amount of light around the terminal 100, A motion sensor (not shown) for detecting the motion of the portable terminal 100 (e.g., rotation of the portable terminal 100, acceleration or vibration applied to the portable terminal 100), detection of a point of the compass using a geomagnetic field A gravity sensor that detects the direction of action of gravity, and an altimeter that measures the altitude by measuring the atmospheric pressure. At least one of the sensors may detect the state, generate a signal corresponding to the detection, and transmit the signal to the control unit 110. The sensor of the sensor module 170 may be added or deleted depending on the performance of the terminal 100.

The storage unit 175 is connected to the mobile communication module 120, the sub communication module 130, the multimedia module 140, the camera module 150, the GPS module 157, the input / And may store signals or data input / output corresponding to the operation of the touch panel 160, the sensor module 170, and the touch screen 190. The storage unit 175 may store a plurality of control programs and a plurality of applications for controlling the portable terminal 100 or the control unit 110. [

The storage unit may include a storage unit 175, a ROM 112 in the control unit 110, a RAM 113 or a memory card (not shown) (e.g., an SD card, a memory stick ). The storage unit may include a nonvolatile memory, a volatile memory, a hard disk drive (HDD), or a solid state drive (SSD).

The storage unit 175 may store various applications such as navigation, a video call, a game, a time-based alarm application to the user, and an image for providing a graphical user interface (GUI) (Such as a menu screen, a standby screen, a radar scope screen, a sound source location display screen, etc.) necessary for driving the terminal 100, or a database or data related to a method of processing a touch input, Operating programs, still images captured by the camera module 150, images, and the like. The storage unit 175 is a medium that can be read by a machine (e.g., a computer). The term " machine-readable medium " is defined as a medium for providing data to the machine . The machine-readable medium may be a storage medium. The storage unit 175 may include non-volatile media and volatile media. All such media must be of a type such that the commands conveyed by the medium can be detected by a physical mechanism that reads the commands into the machine.

The machine-readable medium includes, but is not limited to, a floppy disk, a flexible disk, a hard disk, a magnetic tape, a compact disc read-only memory (CD-ROM) At least one of a punch card, a paper tape, a programmable read-only memory (PROM), an erasable PROM (EPROM), and a flash-EPROM .

The power supply unit 180 may supply power to one or a plurality of batteries (not shown) disposed in the housing of the terminal 100 under the control of the controller 110. One or more batteries (not shown) supply power to the terminal 100. In addition, the power supply unit 180 may supply power input from an external power source (not shown) to the terminal 100 through a wired cable connected to the connector 165. In addition, the power supply unit 180 may supply power to the terminal 100, which is wirelessly input from an external power source through wireless charging technology.

The terminal 100 may include at least one touch screen that provides a user interface corresponding to various services (e.g., call, data transmission, broadcasting, photo shooting, sound source location display) to the user. Each of the touch screens may transmit an analog signal corresponding to at least one touch input to the user interface to the corresponding touch screen controller. As described above, the terminal 100 may include a plurality of touch screens, and each of the touch screens may include a touch screen controller for receiving an analog signal corresponding to a touch. The terminal 100 according to the present invention may include at least one touch screen. Hereinafter, one touch screen will be described for convenience of explanation.

The touch screen 190 may receive at least one user gesture via a user's body (e.g., a finger including an index finger) or a touchable input unit 168 (e.g., a stylus pen, an electronic pen). In addition, the touch screen 190 includes a pen recognition panel (not shown) that recognizes when input through a pen, such as a stylus pen or an electronic pen, recognizes the distance between the pen and the touch screen 190 It can be grasped through the magnetic field. In addition, the touch screen 190 may receive a continuous movement of at least one touch as a user gesture. The touch screen 190 may transmit an analog signal corresponding to the inputted user gesture to the touch screen controller 195.

The touch screen 190 may be implemented by, for example, a resistive method, a capacitive method, an infrared method, or an acoustic wave method.

In addition, the touch screen 190 may be configured to detect touch or proximity of the user's body and the touchable input unit 168, respectively, so that the touch screen 190 can receive input by the user's body and the touchable input unit 168 sequentially And at least two touchscreen panels that are capable of < RTI ID = 0.0 > The at least two touch screen panels provide different output values to the touch screen controller 195, and the touch screen controller 195 recognizes different values input from the at least two touch screen panels (not shown) , It is possible to distinguish whether the input from the touch screen 190 is input by the user's body or input by the touchable input unit.

More specifically, the touch screen 190 includes a panel that senses input through a finger or an input unit 168 through a change in induced electromotive force, and a touch panel that senses contact with the touch screen 190 via a finger or an input unit 168 The panels may be formed in a structure in which the panels are in close contact with each other or are partially spaced apart and stacked in order. The touch screen 190 includes a plurality of pixels, and displays an image through the pixels. The touch screen 190 may be a liquid crystal display (LCD), an organic light emitting diode (OLED), or an LED.

3 is a diagram illustrating an operation of displaying at least one sound source in the vicinity of the terminal 100 according to an embodiment of the present invention.

Referring to FIG. 3, the terminal 100 can detect the directions of the north, south, east, and west directions based on the position of the terminal 100 using the geomagnetic sensor 172 in operation 310. Specifically, the terminal 100 detects the position of the terminal 100 using the GPS module 157 and detects the orientation of the terminal 100 based on the position of the terminal 100 using the geomagnetic sensor 172 .

The terminal 100 can detect the objects in the four sides of the terminal using the reflected wave in the operation 320 and detect the positions of the sensed objects. For example, the terminal 100 can transmit an electromagnetic wave signal through the reflected wave transmission / reception section 113 in all directions, and can receive a reflected wave signal in which the electromagnetic wave signal is reflected by being hit by an object. The terminal 100 can detect at least one or more objects using the received reflected wave signal and detect the position of at least one or more objects existing in all directions on the basis of the position of the terminal 100.

The terminal 100 can distinguish at least one sound source by analyzing sound input through a plurality of microphones in operation 330. [ For example, the terminal 100 may analyze voice or various sounds by various sound sources input through the first to n-th microphones 162-1 to 162-n to distinguish at least one sound source . According to another embodiment, the terminal 100 is not a first to an n-th mics 162-1 to 162-n, but a voice or a voice of various kinds of sound sources input through a plurality of microphones from the hearing aid 300 It is also possible to distinguish at least one sound source by receiving a sound.

In operation 340, the terminal 100 calculates the sound intensity of each sound source, the direction of each sound source, and the distance from the terminal 100 to each sound source, for at least one sound source. For example, the terminal 100 analyzes voices or various sounds by various sound sources input through the first through n-th microphones 162-1 through 162-n, and calculates sound intensity of each sound source . The control unit 110 may calculate distances from the terminal 100 to the respective sound sources through the sound source distance calculation unit 113 and calculate directions of the sound sources through the sound source direction calculation unit 115.

The terminal 100 is operated in the vicinity of the terminal 100 and the terminal 100 by using the positions of the detected objects and the calculated intensity of the source, the direction of the source, and the distances from the terminal 100 to the respective sources, And displays at least one sound source present.

For example, the terminal 100 may select at least one sound source to be displayed on the touch screen 190 among at least one sound sources existing around the terminal 100, And determines the position to be displayed on the screen. The terminal 100 displays at least one sound source at a determined position on the touch screen.

At this time, the terminal 100 may select sound sources within a predetermined distance from the terminal 100 among at least one sound source existing around the terminal 100 as at least one sound source to be displayed on the touch screen 190, It is possible to select a sound source having sound intensity equal to or higher than a predetermined sound intensity as at least one sound source to be displayed on the touch screen 190. [

The determination of where each of the selected at least one sound sources is to be displayed on the touch screen may be determined by considering the size of the touch screen 190 and the distance between the at least one sound sources from the terminal 100 have.

For example, assuming that the center point on the touch screen 190 is the position of the actual terminal 100, and the distance between the at least one sound source from the actual terminal 100 in consideration of the size of the touch screen 190 The position of at least one sound source may be determined based on the center point on the touch screen.

At this time, the terminal 100 may display the terminal image representing the terminal 100 in the position of the terminal 100 on the touch screen 190, and may display at least one of the at least one sound source on the touch screen 190 The sound source images representing the above sound sources can be displayed.

At this time, the terminal 100 displays the terminal image and the sound source images corresponding to the detected sound source on the screen of the predetermined display method, and displays the distance between the terminal 100 and each sound source, the direction angle of each sound source, At least one of the types of sound sources can be displayed together.

The sound source image may be a figure, a character, or the like representing the sound source, and the sound source image may have a shape in which the color or size is distinguished according to the intensity and distance of the sound source. For example, if the sound intensity of a sound source is large, the sound source image may have a larger size or a darker color than when the sound source has a small intensity. Also, if the sound source is close to the sound source, the sound source may have a larger size or a darker color than the sound source is far away. In addition, the shape, size and color of the sound source image can be selectively changed according to the sound intensity of the sound source and the distance of the sound source. Also, the sound source image may have a form indicating the type of the sound source depending on the type of sound source such as whether the sound source is a human voice, a phone ring tone, or a musical instrument sound.

At this time, when the type of the sound source is human voice, the controller 110 grasps the location information of the person corresponding to the voice, receives the phone number and name of the person corresponding to the voice from the external service provider such as a communication service company, Together, they can also display the phone number and name of the person corresponding to the sound source.

Meanwhile, the predetermined display method for displaying the sound sources existing in the vicinity of the terminal 100 and the terminal 100 includes a sound source position display method that is not a radar scope display method or a radar scope display method. Other display methods are not possible.

First, a description will be made of a case where the sound sources existing in the vicinity of the terminal 100 and the terminal 100 are displayed according to the radar scope display method. FIG. 4 is a block diagram of a radar system for displaying sound sources existing around the terminal according to an embodiment of the present invention. Fig.

Referring to FIG. 4, the radar scope screen is a screen capable of indicating the distance and direction of a sound source around the terminal 100. The radar scope screen displays azimuths indicating north, south, east, west, north, south, east, west, west, south, east, . Also, the distance between the terminal 100 and each sound source can be displayed on the radar scope screen, and the sound intensity 47 and the direction angle 49 of each sound source can be displayed for each sound source.

5 is a view illustrating a sound source location display screen for displaying sound sources existing in the vicinity of a terminal according to an embodiment of the present invention.

Referring to FIG. 5, the sound source position display screen is a screen that can indicate the position of a sound source around the terminal 100. In the sound source position display screen, a terminal image 50 indicating the terminal 100 and at least one sound source images 51 to 55 indicating a sound source may be displayed. The sound source images 51 to 55 may be displayed in a darker color than when the sound source intensity is small depending on the intensity of the sound source. The sound source image may also be displayed such that the sound source is a human voice (Bla Bla) or a phone ring sound.

As described above, in a state in which the sound sources around the terminal 100 are displayed, the terminal 100 can select a sound source according to a user gesture input, output the sound of the selected sound source, and adjust the volume of the sound source .

FIG. 6 is a flowchart illustrating a sound output operation of a sound source of the terminal 100 according to an embodiment of the present invention. FIG. 7 is a view illustrating a screen of the sound sources around the terminal 100 according to an embodiment of the present invention.

6 and 7, in operation 610, the terminal 100 displays the sound source images 71 to 75 corresponding to sound sources around the terminal image 70 and the terminal 100, It is possible to judge whether or not the image is touched. The user can touch the sound source image on the screen through the input unit 168 such as a finger, a touch pen, or a stylus.

If the sound source image is touched, the terminal 100 can output the sound of the sound source corresponding to the touched source image in operation 620. [ For example, when the sound source image 71 corresponding to BlaBla3 is touched as shown in FIG. 7 (a), the terminal 100 can output the sound of the sound source for the touched sound source image 71. [ At this time, according to another embodiment, the terminal 100 may transmit the sound of the sound source corresponding to the touched sound source image to the hearing aid 300 and output the sound of the sound source by the hearing aid 300. [

 7 (a), when the sound source image 71 corresponding to BlaBla3 is touched, the sound source images 71 that have not been touched to identify the touched sound source image 71 as shown in FIG. 7 (b) 72 to 75 may be displayed in a manner different from the touched sound source image 71. In this case, when a plurality of sound source images are touched, sounds of sound sources corresponding to a plurality of sound sources may be output, and a plurality of touched source images may be displayed in a different manner so as to be distinguished from other untouched sound source images .

The terminal 100 determines whether the volume step is selected by the user at operation 630. [ For example, the volume level can be selected through the user gesture as either a concentration step, a normal step, or a blocking step. The concentration step may be the step with the greatest volume size, and the step may be the middle step of the volume, and the step of blocking may be the step of blocking the volume. The volume step may further include the step of making the volume of the selected sound source larger than the volume of the other sound sources, or making the volume of the selected sound source smaller than the other volume.

According to the embodiment of the present invention, the volume level may be determined to be other than the three levels, and the volume of the concentration level, the normal level, and the blocking level may be selectively set according to the user or the developer.

If the convergence step is selected, the terminal 100 can output the sound of the sound source to the volume corresponding to the convergence step in operation 632. The terminal 100 may output the sound of the sound source to the volume corresponding to the normal step in the operation 634 if the normal step is selected. The terminal 100 may block the sound of the sound source at operation 636 if the blocking step is selected. At this time, according to another embodiment, the terminal 100 may transmit the sound of the sound source whose volume is adjusted according to the selected volume level to the hearing aid 300. [ The hearing aid 300 can output the sound of the sound source whose volume has been adjusted.

The terminal 100 determines whether the volume step is not selected or the sound source image being sounded in the operation 640 is touched during sound output of the sound source. If the sound source image being sounded is touched, the terminal 100 stops sound output of the sound source in operation 650. [

At this time, according to another embodiment, when the terminal 100 transmits the sound of the sound source to the hearing aid 300, the sound transmission of the sound source may be stopped.

As described above, when the terminal 100 displays the surrounding sound sources, the user selects the sound source so that the terminal 100 or the hearing aid 300 can output the sound of the selected sound source, By allowing the volume to be adjusted, hearing impaired people can hear the desired sound source.

Meanwhile, in the above-described embodiment of the present invention, the terminal 100 selects a sound source to output a sound depending on whether the user touches the user gesture. However, the terminal 100 may recognize various user gesture inputs So that the user can select a sound source to be heard, that is, a sound source to be output.

8 is a flowchart illustrating a sound source selection operation according to user gesture recognition in a terminal according to an embodiment of the present invention.

Referring to FIG. 8, the terminal 100 recognizes a user gesture input in operation 810. The user gesture input may be a single user gesture input or may be a sequential input of a plurality of user gesture inputs.

For example, the user gesture input may be a drag input. The user gesture input may also be a flick input after the touch & hold input. The user gesture input may also have a flick input followed by a hold input after the touch & hold input. In addition, the user gesture input may have a hold input at a specific point on the circumference of the radius indicated by the flick input after the touch & hold input.

The terminal 100 determines whether the user gesture input by the user is an input indicating the sound source selection direction in the result 820 of the user gesture input recognition result. For example, a user gesture input indicating a sound source selection direction may be a drag input in a desired direction after the terminal image is selected.

9 is a diagram for explaining a user gesture input indicating a sound source selection direction according to an embodiment of the present invention.

9, when the terminal 100 displays the sound source images 91 to 95 for at least one sound source in the vicinity of the terminal 100, the terminal 100 selects a terminal image 90 for BlaBla 3 from the terminal image 90 If there is a drag input in the direction of the sound source image 92, the sound source selection direction is determined according to the drag input.

The terminal 100 selects a sound source in the corresponding direction in operation 822 as the sound source selection direction is determined, and outputs the sound of the selected sound source.

Also, the terminal 100 determines in operation 830 whether the user gesture input is an input indicating a sound source selection range. For example, a user gesture input indicating a sound source selection range may have a touch input and a hold input in a terminal image, and then a certain range of flick input.

10 is a diagram for explaining a user gesture input indicating a sound source selection range according to an embodiment of the present invention.

10A and 10B, the terminal 100 displays the sound source images 1001 to 1005 for at least one sound source around the terminal image 1000 and the terminal image 1000 When there is a touch input and a hold input in the terminal image 1000 by the user as shown in FIG. 10A, and there is a flick input corresponding to a certain range, for example, a range of 5 m as shown in FIG. 10B, The sound source selection range is determined. At this time, the terminal 100 can display a touch-and-hold point according to the touch-and-hold input, and can determine a certain range according to the length of the flick input.

Also, the terminal 100 determines in operation 840 whether the user gesture input is an input indicating a sound source selection distance. For example, a user gesture input that indicates a source selection distance may be a flick input after the touch & hold input followed by a hold input.

11 is a view for explaining a user gesture input indicating a sound source selection distance according to an embodiment of the present invention.

11 (a) to 11 (c), the terminal 100 displays the sound source images 1011 to 1015 for at least one sound source around the terminal image 1010 and the terminal image 1010 If the user inputs touch and hold to the terminal image 1010 as shown in FIG. 11A and there is a corresponding flick input at a predetermined distance, for example, 5m, as shown in FIG. 11B, And the radius indicating the sound source selection distance is displayed from the terminal image 1010 as shown in FIG. 11 (c). In addition, when there is a hold input continuously followed by the flick input and the touch is held for a predetermined time on the radius indicating the sound source selection distance as shown in (c) of FIG. 11, the terminal 100 determines the distance of the corresponding radius as the sound source selection distance .

The terminal 100 can select a sound source within a predetermined distance in the operation 832 and output the sound of the selected sound source as the sound source selection distance is determined.

Also, the terminal 100 determines whether the user gesture input is an input indicating the sound source selection position in operation 850. [ For example, a user gesture input indicating a sound source selection position may be a case where there is a hold input at a specific point on the circumference of the radius that is indicated by the flick input after the touch & hold input.

12 is a diagram illustrating a user gesture input indicating a sound source selection position according to an embodiment of the present invention.

12 (a) to 12 (d), the terminal 100 displays the sound source images 1111 to 1115 for at least one sound source around the terminal image 1110 and the terminal image 1110 If a user inputs a touch and hold on the terminal image 1110 as shown in FIG. 12A, and there is a corresponding flick input at a predetermined distance, for example, 5 meters, as shown in FIG. 12B, The radius of the distance from the terminal image 1110 as shown in (c) of FIG. 12 (d), if the terminal 100 has a hold input for holding a touch for a predetermined time or more at an arbitrary position on the circumference of the radius indicating the distance from the terminal, Can determine the position on the circumference of the radius as the sound source selection position.

The terminal 100 can select a sound source at the corresponding position in the 832 operation and output the sound of the selected sound source as the sound source selection position is determined.

As described above, the terminal 100 according to the embodiment of the present invention recognizes user gesture input from various users, selects a sound source, and outputs a sound of the corresponding sound source so that the user can conveniently listen to the sound of the desired sound source do.

According to the embodiment of the present invention as described above, the terminal 100 may separately display the sound sources around itself, output sound of the sound source selected by the user, or adjust the volume of the sound to be output.

When the user selects a sound source, the terminal 100 transmits the sound of the selected sound source to the hearing aid 300 and outputs the sound to the hearing aid 300. [ Can output the sound of the selected sound source.

When the volume level of the sound source is selected by the user, the terminal 100 provides the sound of the sound source having the volume level of volume to the hearing aid 300 so that the hearing aid 300 can adjust the volume It is possible to output the sound of the sound source.

Hereinafter, the configuration and operation of the hearing aid 300 according to the embodiment of the present invention will be described in detail.

13 is a block diagram of a hearing aid 300 according to an embodiment of the present invention.

13, the hearing aid 300 includes a hearing aid control unit 310, a microphone unit 320, a conversion unit 330, an amplification unit 340, an output unit 350, a radio unit 360, 370, and a power supply unit 380. The hearing aid controller 310 controls the overall operation of the hearing aid 300 and includes a microphone 320, a conversion unit 330, an amplification unit 340, an output unit 350, a radio unit 360, a storage unit 370), and the power supply unit 380.

The microphone unit 320 may include at least two microphones, and may receive ambient sounds by sensing ambient sounds. For example, at least two or more microphones are located outside the hearing aid 300 and receive an analog sound signal from various sound sources in the user's ear, and under the control of the hearing aid controller 310, ). ≪ / RTI >

The conversion unit 330 may convert the analog sound signal input from the microphone unit 320 into a digital sound signal and convert the amplified digital sound signal into an analog sound signal. For example, the converting unit 330 converts the analog sound signal transmitted from the microphone unit 320 into a digital sound signal and transmits the digital sound signal to the hearing aid controller 310.

The hearing aid controller 310 performs digital signal processing such as unnecessary noise elimination, feedback control, amplification gain, non-linear amplification, and the like on the digital sound signal received from the converting unit 330, To the converting unit 330. At this time, the amplification unit 340 receives the digital sound signal converted by the conversion unit 330, amplifies the digital sound signal according to the amplification gain by the hearing aid control unit 310, and provides the amplified digital sound signal to the control unit 310. The converting unit 330 receives the digital sound signal processed by the hearing aid controller 310 and converts the digital sound signal to an analog sound signal and provides the analog sound signal to the output unit 350. The output unit 350 receives the analog sound signal from the conversion unit 330 and outputs the analog sound signal.

The wireless unit 360 may include any one of a mobile communication module (not shown), a wireless LAN module (not shown), and a local communication module (not shown), and may perform wireless communication with the terminal 100 . The short-range communication module may include an infrared communication module and a Bluetooth communication module. The wireless unit 360 wirelessly communicates with the terminal 100 under the control of the hearing aid controller 310 and receives a sound signal of a selected sound source or a volume adjusted sound source from the terminal 100, And transmits sound signals for various sound sources collected through the microphone unit 320 to the sound source 100.

The storage unit 370 stores a sound signal of a selected sound source received from the terminal 100 or a sound signal of a volume-controlled sound source, or a sound signal of a sound source collected through the microphone unit 320.

The power supply unit 380 supplies power to the respective components of the hearing aid 300.

According to the embodiment of the present invention, the hearing aid 300 configured as described above receives sound signals of various sound sources through the microphone unit 320, transmits the sound signals to the terminal 100, The sound signal of the sound source selected by the user and the sound signal of the sound source whose volume is adjusted can be received and output.

FIG. 14 is a flowchart of the operation of the hearing aid 300 according to the embodiment of the present invention.

Referring to FIG. 14, the hearing aid 300 can receive a sound signal of nearby sound sources through the microphone unit 320 in operation 1410. At this time, the hearing aid 300 can receive analog sound signals from various surrounding sound sources through at least two microphones which are located outside the hearing aid 300 and may exist at the user's ear position.

The hearing aid 300 can transmit a sound signal by the various sound sources inputted to the terminal 100 in operation 1420. For example, the hearing aid 300 can wirelessly communicate with the terminal 100 through the wireless unit 360, and can transmit sound signals from various sound sources to the terminal 100 by wireless communication. At this time, the terminal 100 receives a sound signal of various sound sources from the hearing aid 300, distinguishes at least one sound source by using a sound signal by the received various sound sources, The sound intensity of each sound source, the direction of each sound source, and the distance from the terminal 100 to each sound source, so that at least one sound source existing around the screen of the terminal 100 can be displayed.

Also, the terminal 100 may transmit the sound signal of the selected sound source to the hearing aid 300 when the sound source is selected by the user among the displayed at least one sound sources. Also, when the volume step for sound output of the sound source is selected by the user, the terminal 100 may transmit the sound signal of the sound source whose volume has been adjusted to the hearing aid 300 according to the selected volume step.

The hearing aid 300 can receive a sound signal of a selected sound source or a sound signal of a sound source whose volume has been adjusted from the terminal 100 through the wireless unit 360 in operation 430. [

When the sound signal of the selected sound source or the sound source of the sound source whose volume is adjusted is received from the terminal 100, the hearing aid 300 outputs the sound signal of the selected sound source or the sound source of the sound source whose volume is adjusted in operation 440, (350).

In the above description, the case where the hearing aid 300 transmits sound signals of various sound sources input through the microphone 320 to the terminal 100 has been described. However, according to another embodiment, The first through n-th microphones 162-1 through 162-n of the terminal 100 do not transmit sound signals of various sound sources input through the microphone unit 320 to the terminal 100 through the radio unit 360, A sound signal of the selected sound source and a sound signal of the sound source whose volume are adjusted by using the sound source 162-n.

According to the embodiments of the present invention as described above, the user can utilize the display of the terminal 100 to know the locations of the sound sources existing in all directions with respect to the user.

In addition, according to the embodiments of the present invention, a user can select a sound source desired to be heard among the sound sources to only listen to the selected sound source, and can adjust the volume of the sound source while listening to the sound source .

According to the embodiments of the present invention, in the case of a hearing-impaired person using the hearing aid 300, the hearing aid 300 and the terminal 100 are interlocked so that the position of the sound sources can be recognized through the terminal 100, ) So that the sound of the desired sound source can be heard.

In addition, according to embodiments of the present invention, it is possible to provide a more convenient user interface (UI) to a user by allowing a user to listen to a desired sound source among sound sources using various user gestures.

Various embodiments of the present invention may be implemented in the form of program instructions that can be executed on various computer means and recorded on a computer readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. Program instructions to be recorded on the medium may be those specially designed and constructed for the present invention or may be known and available to those of ordinary skill in the computer software arts.

Claims (22)

A method for providing a position of a sound source in a portable terminal having a touch screen,
An operation of receiving a sound signal of a plurality of sound sources,
Obtaining a distance and a direction between the terminal and the plurality of sound sources based on a sound signal of each of the plurality of sound sources;
Determining a position of the plurality of sound sources on the touch screen by using the distance and direction of each of the plurality of sound sources obtained;
And displaying the locations of the plurality of sound sources on the touch screen according to the determined location. The method according to claim 1,
Wherein the determining of the location of the plurality of sound sources on the touch screen comprises:
Selecting at least one sound source to be displayed on the touch screen among the plurality of sound sources,
And determining a position of the selected at least one sound sources on the touch screen. The method according to claim 1,
Wherein the operation of receiving the sound signals of the plurality of sound sources comprises:
And a sound signal of the plurality of sound sources is received from a hearing aid outside the terminal. The method according to claim 1,
Recognizing a user gesture input for selecting at least one of a plurality of locations of sound sources displayed on the touch screen;
Further comprising: adjusting a volume of the sound source corresponding to the selected position corresponding to the user gesture input. 5. The method of claim 4, wherein the adjusting the volume of the sound source corresponding to the selected position comprises:
And adjusting a volume of a sound source corresponding to a sound source selection direction corresponding to a user gesture input indicating a sound source selection direction on the touch screen. 5. The method of claim 4, wherein the adjusting the volume of the sound source corresponding to the selected position comprises:
And adjusting a volume of a sound source included in the sound source selection range corresponding to a user gesture input indicating a sound source selection range on the touch screen. 5. The method of claim 4, wherein the adjusting the volume of the sound source corresponding to the selected position comprises:
And adjusting a volume of a sound source corresponding to the sound source selection distance corresponding to a user gesture input indicating a sound source selection distance on the touch screen. 5. The method of claim 4,
The operation of adjusting the volume of the sound source includes:
The volume of the sound source is cut off, or the volume of the sound source is made larger than the volume of the other sound source, and the volume of the sound source is made smaller than the other volume. The method according to claim 1,
Further comprising detecting the distance and direction of objects corresponding to each of the plurality of sound sources using a reflected wave signal. 12. The method of claim 11,
Calculating sound intensity for each of the plurality of sound sources,
Further comprising displaying the sound intensity for each of the plurality of sound sources on the touch screen. In a method for sound output in a hearing aid,
An operation of receiving a sound signal of a sound source whose volume is selected by a user's gesture from a portable terminal through a touch screen,
And outputting a sound signal of the received selected sound source. A portable terminal having a touch screen,
A touch screen for displaying positions of a plurality of sound sources,
Acquiring distance and direction between the terminal and the plurality of sound sources based on a sound signal of each of the plurality of sound sources, and using the distance and direction of each of the plurality of sound sources, And a controller for determining positions on the screen and controlling the positions of the plurality of sound sources on the touch screen according to the determined positions. The method according to claim 1,
Wherein the control unit selects at least one sound source to be displayed on the touch screen among the plurality of sound sources and determines a position of the selected at least one sound sources on the touch screen. The method according to claim 1,
Further comprising a communication unit for receiving sound signals of the plurality of sound sources,
Wherein the communication unit receives a sound signal of the plurality of sound sources from a hearing aid outside the terminal. The apparatus of claim 1,
Wherein the control unit recognizes a user gesture input for selecting at least one of the positions of the plurality of sound sources displayed on the touch screen and adjusts the volume of the sound source corresponding to the selected position corresponding to the user gesture input. 16. The apparatus of claim 15,
And controls the volume of the sound source corresponding to the sound source selection direction corresponding to the user gesture input indicating the sound source selection direction to the touch screen. 16. The apparatus of claim 15,
And controls the volume of a sound source included in the sound source selection range corresponding to a user gesture input indicating a sound source selection range on the touch screen. 16. The apparatus of claim 15,
Wherein the controller controls the volume of the sound source corresponding to the sound source selection distance corresponding to the user gesture input indicating the sound source selection distance on the touch screen. 16. The method of claim 15,
The operation of adjusting the volume of the sound source includes:
The volume of the sound source is cut off or the volume of the sound source is made larger than the volume of the other sound source, and the volume of the sound source is made smaller than the volume of the other sound source. 13. The method of claim 12,
Further comprising a reflected wave transmitting / receiving section for transmitting the electromagnetic wave signal and receiving the reflected wave signal,
Wherein the controller detects distances and directions of objects corresponding to the plurality of sound sources using the reflected wave signal. 13. The apparatus according to claim 12,
Calculates sound intensity for each of the plurality of sound sources, and controls the sound intensity for each of the plurality of sound sources to be displayed on the touch screen. In a hearing aid,
A wireless unit for receiving a sound signal of a sound source selected by a user's gesture from a portable terminal through a touch screen and having a volume adjusted;
And an output unit for outputting a sound signal of the received selected sound source.

Images