KR20150009049A - Mobile terminal and controlling method thereof - Google Patents

Abstract

The present invention relates to a mobile terminal for improving user convenience and, specifically, to a mobile terminal which sets a sound obtaining area based on a user voice control order, and obtain a sound source in and around the sound obtaining area. The mobile terminal comprise: a camera unit for taking an image: a microphone unit including multiple microphones; and a control unit for setting a sound obtaining area in the image taken by the camera unit, and controlling the microphone unit to obtain a sound source in and around the determined sound obtaining area. The control unit sets the sound obtaining area based on a voice control order extracted from a user voice obtained through the microphone unit.

Description

[0001] MOBILE TERMINAL AND CONTROLLING METHOD THEREOF [0002]

The present invention relates to a mobile terminal and a control method thereof, and more particularly, to a mobile terminal capable of setting a sound region based on a voice control command of a user and acquiring a sound source around the sound region, and a control method thereof .

A terminal can be divided into a mobile terminal (mobile / portable terminal) and a stationary terminal according to whether the terminal can be moved. The mobile terminal can be divided into a handheld terminal and a vehicle mount terminal according to whether the user can directly carry the mobile terminal.

Such a terminal has various functions, for example, in the form of a multimedia device having multiple functions such as photographing and photographing of a moving picture, reproduction of a music or video file, reception of a game and broadcasting, etc. .

In order to support and enhance the functionality of such terminals, it may be considered to improve the structural and / or software parts of the terminal.

A user can generate a video file using a microphone and a camera provided in the mobile terminal.

However, in the conventional case, there is a problem that not only a sound source in which a subject intending to take a picture intensively occurs but also a surrounding sound source and noise are also recorded.

SUMMARY OF THE INVENTION The present invention has been proposed in order to solve the above problems, and it is an object of the present invention to provide a mobile terminal with improved user convenience.

Specifically, it is an object of the present invention to provide a mobile terminal capable of setting a sound range based on a voice control command of a user and acquiring sound sources around the sound range.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, unless further departing from the spirit and scope of the invention as defined by the appended claims. It will be possible.

According to an aspect of the present invention, there is provided a mobile terminal including: a camera unit for capturing an image; A microphone unit including a plurality of microphones; And a control unit for setting a sound region in the image photographed through the camera unit and controlling the microphone unit to acquire sound sources around the sound region set in advance, wherein the control unit acquires sound sources around the sound region And controls the microphone unit.

According to another aspect of the present invention, there is provided a method of controlling a mobile terminal including a microphone including a plurality of microphones, the method comprising: capturing an image; Extracting a voice control command from the voice of the user acquired by the microphone unit; Setting a breathing area in the photographed image based on the extracted voice control command; And controlling the microphone unit to acquire a sound source around the sound taking area.

The present invention can provide a mobile terminal with improved user convenience.

Specifically, the present invention has an effect that it is possible to provide a mobile terminal capable of setting a loudspeaker area based on a voice control command of a user and acquiring a sound source around the loud area.

In addition, the present invention is capable of setting a loudspeaker area without a separate touch input by a user, and thus has the effect of not being shaken or noisy when shooting a moving picture.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, unless further departing from the spirit and scope of the invention as defined by the appended claims. It will be possible.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.
2 is a front perspective view of a mobile terminal according to an embodiment of the present invention.
3 is a rear perspective view of a mobile terminal according to an embodiment of the present invention.
4 is a flowchart illustrating an operation of a mobile terminal according to an exemplary embodiment of the present invention.
5A to 5C are diagrams for explaining setting of a breath taking area according to an embodiment of the present invention.
6A to 6C are exemplary diagrams for explaining a breath taking area setting interface according to an embodiment of the present invention.
7A to 7C are diagrams for explaining various control methods of a breath taking area according to an embodiment of the present invention.
FIG. 8 is a block diagram for explaining an operation of acquiring and processing a sound source around a sounding region according to an embodiment of the present invention.

Hereinafter, a mobile terminal related to the present invention will be described in detail with reference to the drawings.

The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

The mobile terminal described in this specification may include a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), and navigation.

However, it will be understood by those skilled in the art that the configuration according to the embodiments described herein may be applied to a fixed terminal such as a digital TV, a desktop computer, and the like, unless the configuration is applicable only to a mobile terminal.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention. 1, the mobile terminal 100 includes a wireless communication unit 110, an audio / video (A / V) input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, An interface unit 170, a control unit 180, a power supply unit 190, and the like. The components shown in FIG. 1 are not essential, and a mobile terminal having more or fewer components may be implemented.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more modules for enabling wireless communication between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and the network in which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives broadcast signals and / or broadcast-related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. Two or more broadcast receiving modules may be provided to the mobile terminal 100 for simultaneous broadcast reception or broadcast channel switching for at least two broadcast channels.

The broadcast management server may refer to a server for generating and transmitting broadcast signals and / or broadcast related information, or a server for receiving broadcast signals and / or broadcast related information generated by the broadcast management server and transmitting the generated broadcast signals and / or broadcast related information. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

The broadcast-related information may refer to a broadcast channel, a broadcast program, or information related to a broadcast service provider. The broadcast-related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast-related information may exist in various forms. For example, an EPG (Electronic Program Guide) of DMB (Digital Multimedia Broadcasting) or an ESG (Electronic Service Guide) of Digital Video Broadcast-Handheld (DVB-H).

For example, the broadcast receiving module 111 may be a Digital Multimedia Broadcasting-Terrestrial (DMB-T), a Digital Multimedia Broadcasting-Satellite (DMB-S), a Media Forward Link Only (Mobile Broadcasting Business Management System), ISDB-T (Integrated Services Digital Broadcasting (ISDB-T)), Digital Multimedia Broadcasting (MBMS) Digital Broadcast-Terrestrial) or the like. Of course, the broadcast receiving module 111 may be adapted to other broadcasting systems as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 may be coupled to a base station on a mobile communication network such as but not limited to GSM (Gobal System for Mobile communications), CDMA (Code Division Multiple Access), WCDMA (Wideband CDMA), LTE (Long Term Evolution) And transmits and receives a radio signal to at least one of an external terminal and a server. The wireless signal may include various types of data depending on a voice call signal, a video call signal or a text / multimedia message transmission / reception.

The wireless Internet module 113 is a module for wireless Internet access, and may be built in or externally attached to the mobile terminal 100. Wireless Internet technologies include WLAN (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), GSM, CDMA, WCDMA, LTE Evolution (but not limited to) may be used.

The wireless Internet module 113, which performs wireless Internet access through the mobile communication network, is connected to the mobile communication module 110 through the mobile communication network, for example, from the viewpoint that the wireless Internet access by Wibro, HSDPA, GSM, CDMA, WCDMA, LTE, (112). ≪ / RTI >

The short-range communication module 114 refers to a module for short-range communication. Bluetooth, Radio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), ZigBee, and the like can be used as a short range communication technology.

The position information module 115 is a module for obtaining the position of the mobile terminal, and a representative example thereof is a Global Position System (GPS) module. According to the current technology, the GPS module 115 calculates distance information and accurate time information from three or more satellites, and then applies trigonometry to the calculated information to obtain a three-dimensional string of latitude, longitude, The location information can be accurately calculated. At present, a method of calculating position and time information using three satellites and correcting an error of the calculated position and time information using another satellite is widely used. In addition, the GPS module 115 can calculate speed information by continuously calculating the current position in real time.

Referring to FIG. 1, an A / V (Audio / Video) input unit 120 is for inputting an audio signal or a video signal, and may include a camera unit 121 and a microphone unit 122. The camera section 121 processes an image frame such as a still image or a moving image obtained by the image sensor in the video communication mode or the photographing mode. The processed image frame can be displayed on the display unit 151. [

The image frame processed by the camera unit 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. [ At least two camera units 121 may be provided depending on the use environment.

The microphone unit 122 receives an external sound signal through a microphone in a communication mode, a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 when the voice data is in the call mode, and output. Various noise reduction algorithms may be implemented in the microphone unit 122 to remove noise generated in receiving an external sound signal. The microphone unit 122 may also be provided in two or more depending on the use environment.

2 to 3, it is assumed that the camera unit 121 is implemented as a first camera unit 121-1 and a second camera unit 121-2 for convenience of explanation. It is also assumed that the microphone unit 122 is implemented to include a plurality of microphones 122-1, 122-2, 122-3, and 122-4.

The user input unit 130 generates input data for a user to control the operation of the terminal. The user input unit 130 may include buttons located on the front, rear, or side of the mobile terminal 100, a touch sensor (static pressure / static electricity), and a keypad, a dome switch ), A jog wheel, a jog switch, and the like.

The sensing unit 140 senses the current state of the mobile terminal 100 such as the open / close state of the mobile terminal 100, the position of the mobile terminal 100, the presence or absence of user contact, the orientation of the mobile terminal, And generates a sensing signal for controlling the operation of the mobile terminal 100. Examples of the sensing unit 140 include a gyroscope sensor, an acceleration sensor, and a geomagnetic sensor.

For example, when the mobile terminal 100 is in the form of a slide phone, it is possible to sense whether the slide phone is opened or closed. It is also possible to sense whether the power supply unit 190 is powered on, whether the interface unit 170 is connected to an external device, and the like. Meanwhile, the sensing unit 140 may include a proximity sensor 141.

The output unit 150 is for generating output related to the visual, auditory or tactile sense and includes a display unit 151, an audio output module 152, an alarm unit 153, and a haptic module 154 .

The display unit 151 displays (outputs) information processed by the mobile terminal 100. For example, when the mobile terminal is in the call mode, a UI (User Interface) or a GUI (Graphic User Interface) associated with a call is displayed. When the mobile terminal 100 is in the video communication mode or the photographing mode, the photographed and / or received video or UI and GUI are displayed.

The display unit 151 may be a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display display, and a 3D display.

Some of these displays may be transparent or light transmissive so that they can be seen through. This can be referred to as a transparent display, and a typical example of the transparent display is TOLED (Transparent OLED) and the like. The rear structure of the display unit 151 may also be of a light transmission type. With this structure, the user can see an object located behind the terminal body through the area occupied by the display unit 151 of the terminal body.

There may be two or more display units 151 according to the embodiment of the mobile terminal 100. For example, in the mobile terminal 100, a plurality of display portions may be spaced apart from one another, or may be disposed integrally with one another, and may be disposed on different surfaces, respectively.

The display unit 151 may include a display unit 151 and a touch sensor 137. The display unit 151 and the touch sensor 137 may have a layer structure or an integrated structure, Can be used. The touch sensor may be stacked on the display unit 151 to form a layer structure when the touch sensor has a form of a touch film, a touch sheet, a touch pad, or the like. ≪ / RTI >

The touch sensor 137 may be configured to convert a change in a pressure applied to a specific part of the display part 151 or a capacitance generated in a specific part of the display part 151 into an electrical input signal. The touch sensor 137 can be configured to detect not only the position and area to be touched but also the pressure at the time of touch.

If there is a touch input to the touch sensor 137, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and transmits the corresponding data to the controller 180. Thus, the control unit 180 can know which area of the display unit 151 is touched or the like.

Referring to FIG. 2, the proximity sensor 141 may be disposed in an inner region of the mobile terminal or in the vicinity of the touch screen, which is surrounded by the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or a nearby object without mechanical contact using the force of an electromagnetic field or infrared rays. The proximity sensor has a longer life span than the contact sensor and its utilization is also high.

Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. And to detect the proximity of the pointer by the change of the electric field along the proximity of the pointer when the touch screen is electrostatic. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of recognizing that the pointer is positioned on the touch screen while the pointer is not in contact with the touch screen is referred to as "proximity touch & The act of actually touching the pointer on the screen is called "contact touch. &Quot; The position where the pointer is proximately touched on the touch screen means a position where the pointer is vertically corresponding to the touch screen when the pointer is touched.

The proximity sensor detects a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, a proximity touch movement state, and the like). Information corresponding to the detected proximity touch operation and the proximity touch pattern may be output on the touch screen.

The audio output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output module 152 also outputs sound signals related to functions (e.g., call signal reception sound, message reception sound, etc.) performed in the mobile terminal 100. [ The audio output module 152 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event of the mobile terminal 100. Examples of events that occur in the mobile terminal include call signal reception, message reception, key signal input, touch input, and the like. The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than the video signal or the audio signal, for example, vibration. The video signal or the audio signal may be output through the display unit 151 or the audio output module 152 so that the display unit 151 and the audio output module 152 may be classified as a part of the alarm unit 153 have.

The haptic module 154 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 154 is vibration. The intensity and pattern of the vibration generated by the hit module 154 can be controlled. For example, different vibrations may be synthesized and output or sequentially output.

In addition to the vibration, the haptic module 154 may include a pin arrangement vertically moving with respect to the contact skin surface, a spraying force or suction force of the air through the injection port or the suction port, a touch on the skin surface, contact with an electrode, And various tactile effects such as an effect of reproducing a cold sensation using an endothermic or exothermic element can be generated.

The haptic module 154 can be implemented not only to transmit the tactile effect through the direct contact but also to allow the user to feel the tactile effect through the muscular sensation of the finger or arm. The haptic module 154 may include two or more haptic modules 154 according to the configuration of the portable terminal 100.

The memory 160 may store a program for the operation of the controller 180 and temporarily store input / output data (e.g., a phone book, a message, a still image, a moving picture, etc.). The memory 160 may store data on vibration and sound of various patterns outputted when a touch is input on the touch screen.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), a RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM A disk, and / or an optical disk. The mobile terminal 100 may operate in association with a web storage that performs a storage function of the memory 160 on the Internet.

The interface unit 170 serves as a path for communication with all external devices connected to the mobile terminal 100. The interface unit 170 receives data from an external device or supplies power to each component in the mobile terminal 100 or transmits data to the external device. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, A video input / output (I / O) port, an earphone port, and the like may be included in the interface unit 170.

The identification module is a chip for storing various information for authenticating the use right of the mobile terminal 100 and includes a user identification module (UIM), a subscriber identity module (SIM), a general user authentication module A Universal Subscriber Identity Module (USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the port.

When the mobile terminal 100 is connected to an external cradle, the interface unit may be a path through which power from the cradle is supplied to the mobile terminal 100, or various command signals input by the user to the cradle may be transmitted It can be a passage to be transmitted to the terminal. The various command signals or the power source input from the cradle may be operated as a signal for recognizing that the mobile terminal is correctly mounted on the cradle.

The controller 180 typically controls the overall operation of the mobile terminal. For example, voice communication, data communication, video communication, and the like. The control unit 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180. [

The controller 180 may perform a pattern recognition process for recognizing handwriting input or drawing input performed on the touch screen as characters and images, respectively.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components.

The battery may be an internal battery configured to be chargeable, and may be detachably coupled to the terminal body for charging or the like. The connection port may be configured as an example of an interface 170 through which an external charger for supplying power for charging the battery is electrically connected.

The various embodiments described herein may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein may be implemented as application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays May be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and other electronic units for performing other functions. In some cases, May be implemented by the control unit 180.

According to a software implementation, embodiments such as procedures or functions may be implemented with separate software modules that perform at least one function or operation. The software code may be implemented by a software application written in a suitable programming language. The software codes are stored in the memory 160 and can be executed by the control unit 180. [

FIG. 2 is a perspective view of a mobile terminal or a portable terminal according to the present invention, as viewed from the front. 3 is a rear perspective view of the portable terminal shown in FIG.

The disclosed mobile terminal 100 has a bar-shaped terminal body. However, the present invention is not limited thereto, and can be applied to various structures such as a slide type, a folder type, a swing type, and a swivel type in which two or more bodies are relatively movably coupled.

The body of the mobile terminal 100 includes a casing 101, 102, In this embodiment, the case may be divided into a front case 101 and a rear case 102. [ A variety of electronic components are embedded in the space formed between the front case 101 and the rear case 102.

In some cases, electronic components may be mounted on the surface of the rear case 102 as well. Electronic components mounted on the surface of the rear case 102 include a battery, a USIM card, a memory card, and the like which are detachable by the user. At this time, the rear case 102 may further include a rear cover 103 for covering the surface thereof. When the rear cover 103 is removed, the surface of the rear case 102 is exposed.

The cases 101, 102 and 103 may be formed by injection molding of synthetic resin or may be formed of a metal material such as stainless steel (STS) or titanium (Ti).

The terminal case 101, 102, 103 of the mobile terminal according to an embodiment of the present invention includes a display unit 151, an audio output unit 152, camera units 121 / 121-1, 121-2, a user input unit 130 131, 132, and 133, microphone units 122 and 122-1, 122-2, 122-3 and 122-4, an interface 170, and the like.

As described above, it is assumed that the camera unit 121 is implemented as the first camera unit 121-1 and the second camera unit 121-2 in FIGS. Further, it is assumed that the microphone unit 122 is implemented to include four microphones 122-1, 122-2, 122-3, and 122-4. Here, the number and arrangement of the camera unit 121 and the microphone unit 122 are only examples of the present invention, but are not limited thereto.

The display unit 151 occupies most of the main surface of the front case 101. The sound output unit 152 and the second camera 121-2 are disposed in the area adjacent to one end of the both ends of the display unit 151 and the user input unit 131 is disposed in the area adjacent to the other end. The user input unit 132, the interface 170, the microphone units 122 / 122-1, 122-2, 122-3, and 122-4 may be disposed on the side surfaces of the front case 101 and the rear case 102. [

The user input unit 130 is operated to receive a command for controlling the operation of the portable terminal 100, and may include a plurality of operation units 131, 132, and 133. The operation units 131, 132, and 133 may be collectively referred to as a manipulating portion, and may be employed in any manner as long as the user operates in a tactile manner.

Contents input by the first or second operation units 131, 132, and 133 may be variously set. For example, the first operation unit 131 receives a command such as start, end, scroll, and the like, and the second operation unit 132 inputs a command such as adjustment of the sound output from the sound output unit 152 And the third operation unit 133 can receive a command such as switching to the touch recognition mode of the display unit 151 or the like.

As shown in FIG. 3, when the rear cover 103 is coupled, only a side surface of the rear case 102 may be exposed. When the size of the rear cover 103 is reduced, Some may also be exposed. And may include an opening for exposing the first camera unit 121-1 and the sound output unit 152 'to the outside when the rear cover 103 covers the entire rear surface of the rear case 102. [

Referring to FIG. 3, a first camera unit 121-1 is disposed on the rear surface of the terminal body, that is, on the rear case 102. As shown in FIG. The first camera unit 121-1 has a photographing direction that is substantially opposite to that of the second camera unit 121-2 (see FIG. 2), and the second camera unit 121-2 As shown in FIG.

For example, the second camera unit 121-2 (see FIG. 2) has low pixels so that it is easy to capture a face of a user in the case of a video call or the like and transmit the same to the other party, and the first camera unit 121-1 It is preferable to have a high pixel since a general subject is photographed and is not immediately transferred. The first and second camera units 121-1 and 121-2 may be installed in the terminal body such that they can be rotated or pop-up.

A flash 123 and a mirror 124 are further disposed adjacent to the first camera unit 121-1. The flash 123 illuminates the subject toward the subject when the subject is photographed by the first camera unit 121-1. The mirror 124 allows the user to illuminate the user's own face or the like when the user intends to take a picture of himself / herself (self-photographing) using the first camera unit 121-1.

An acoustic output 152 'may be additionally disposed on the rear surface of the terminal body. The sound output unit 152 'may implement the stereo function together with the sound output unit 152 (see FIG. 2) and may be used for the implementation of the speakerphone mode in a call.

On the other hand, the microphone units 122-1, 122-2, 122-3, and 122-4 may be disposed on the front, side, and rear surfaces of the terminal body. 2 and 3, each microphone is connected to a lower front side 122-1 (see FIG. 2), a side upper side 122-2 (see FIG. 2), a lower side 122-3 , And the rear upper side 122-4 (see Fig. 3).

In this case, it is possible to define the first microphone for acquiring the voice of the user, the microphone unit 122-1 disposed on the front surface of the terminal body. The user operates the moving picture shooting function while viewing the front of the terminal in which the display unit 151 is disposed. Therefore, since the user's voice can be acquired more in the first microphone disposed on the front side of the terminal than the microphone disposed in the other position, the mobile terminal can acquire the sound source acquired from the first microphone and the microphone The user's voice (or voice control command) can be extracted by comparing the sound sources.

On the other hand, the position of the first microphone for acquiring the voice of the user can be changed in consideration of the state of the mobile terminal when the moving picture photographing function operates. For example, the mobile terminal determines an optimal microphone position for acquiring a user's voice in consideration of the position and orientation of the mobile terminal obtained from the gyroscope sensor and the geomagnetic sensor of the sensing unit 140, The microphone can be set to the first microphone.

Also, the arrangement and the number of the microphones are not limited to the embodiment and can be changed for the implementation of the optimum directional microphone and the user voice acquisition.

In addition to the antenna for talking and the like, a broadcast signal reception antenna 116 may be additionally disposed on the side of the terminal body. The antenna 116, which forms part of the broadcast receiving module 111 (see FIG. 1), can be installed to be able to be drawn out from the terminal body.

A power supply unit 190 for supplying power to the mobile terminal 100 is mounted on the terminal body. The power supply unit 190 may be built in the terminal body or may be detachable from the outside of the terminal body.

The rear case 102 may further include a touch pad 135 for sensing a touch. The touch pad 135 may be of a light transmission type for the display unit 151. [ In this case, if the display unit 151 is configured to output the time information on both sides (i.e., in the directions of both the front and back sides of the mobile terminal), the time information can be recognized through the touch pad 135 do. The information output on both sides may be all controlled by the touch pad 135. [

Meanwhile, the display for exclusive use of the touch pad 135 may be separately installed, so that the touch screen may be disposed in the rear case 102 as well.

The touch pad 135 operates in correlation with the display portion 151 of the front case 101. The touch pad 135 may be disposed parallel to the rear of the display unit 151. The touch pad 135 may have a size equal to or smaller than that of the display unit 151.

Hereinafter, embodiments related to a control method that can be implemented in a mobile terminal configured as above will be described with reference to the accompanying drawings.

For convenience of explanation, it is assumed that the mobile terminal according to the present invention includes at least one of the components shown in FIG.

Specifically, the mobile terminal according to the present invention includes a camera unit 121 for capturing an image of the components shown in FIG. 1, and a microphone unit 122 including a plurality of microphones. And a control unit 180 for controlling the microphone unit 122 so as to set a sound effect area on the image photographed through the camera unit 121 and to acquire sound sources around the set sound effect area. In this case, it is assumed that the control unit 180 sets a sound-pickup area based on the voice control command extracted from the voice of the user acquired through the microphone unit 122. [ In some cases, the mobile terminal according to the present invention may be assumed to further include a display unit 151 for displaying the picked-up image so that the picked-up area is visually identified.

Since the mobile terminal according to the present invention can be more easily implemented when the display unit 151 is a touch screen, which is a sub-concept thereof, it is assumed that the display unit 151 is a touch screen hereinafter. However, the display unit 151 is not necessarily a touch screen. If the display unit 151 is not a touch screen, the mobile terminal according to the present invention may further include a separate function unit (for example, the user input unit 130 of FIG. 1) for receiving user input.

Meanwhile, the control unit 180 according to an exemplary embodiment of the present invention stores the sound source and the photographed image obtained by centering the sound acquisition region as moving images, and removes the user's voice when moving images are stored. Further, the control unit 180 can compare the sound source obtained from the above-described first microphone with the sound source obtained from the microphone except for the first microphone, and acquire the user's voice. This will be described later.

Hereinafter, a mobile terminal according to the present invention will be described in detail with reference to an operation flow chart of a mobile terminal according to the present invention on the basis of the above-described assumption.

4 is a flowchart illustrating an operation of a mobile terminal according to an exemplary embodiment of the present invention. Before describing the operation flow chart of FIG. 4, first, it is assumed that the control unit 180 is in a state in which a moving image photographing function is being executed and an audio zooming mode is selected. Here, the audio zooming mode refers to a mode in which the user records intensively the sound source of a subject to be taken by the user and performs moving picture shooting.

Referring to FIG. 4, the camera unit 121 can capture an image (S410). Specifically, the camera section 121 can take an image in the direction of the subject on the front side of the user. Here, the camera unit 121 may be a front camera that is the first camera unit 121-1.

Then, the voice control command is extracted from the voice of the user acquired by the microphone unit 122 (S420), and the voice-activated region can be set in accordance with the voice control command thus extracted (S430).

5A and 5B, the control unit 180 may control to set a dictation area in a default defined area (for example, in FIG. 5B, the center of the captured image is set as a default Area < / RTI > Alternatively, the control unit 180 may delay the setting of the intake range until obtaining the control command.

Here, the sound-absorbing area means a region around a subject, space, or object that the user wants to focus on in the captured image. For example, in the case of FIG. 5C, it can be confirmed that the ellipse of the dotted line is set as the sound-absorbing area.

On the other hand, the setting of the breath taking area can be activated / deactivated according to the setting of the user. For example, FIG. 6 is a diagram illustrating an example of a sound range setting interface according to an embodiment of the present invention.

6A is a view showing an interface showing a sub menu of the moving picture photographing mode. When a user operation for selecting the audio zooming mode of FIG. 6A is input, an interface for selecting a lower mode of the audio zooming mode can be displayed. In this case, the user operation can be inputted by the touch input, the button input, and other sub-configurations of the user input unit 130. [ The audio zooming mode may include a screen zooming mode, a sounding area designation mode, a focus centering mode, and a sub-mode of a voice command mode. Here, the screen zoom interlocking mode is a mode in which the zoomed-in area is automatically set as a sounding area when the lens of the camera unit is zoomed in. The sounding area designating mode is a mode for setting a sounding area based on the touch input of the user, Quot; means a mode in which the focused area is set to the hunting mode.

According to an embodiment of the present invention, when a user operation for selecting a voice command mode is input, the control operation of the present invention can be performed

When the voice command mode is touched in FIG. 6A, the control unit can divide the display unit into a plurality of virtual areas. At this time, the number of divided areas may be automatically designated by the control unit or manually set by the user. For example, in the division selection menu shown in Fig. 6B, 3, 5, and 7 can be specified as the number of regions that can be divided.

Here, although the user operation is shown as touch input, it may be a voice control command

After the display unit is divided into the plurality of areas, if a voice command corresponding to a specific area is input from among the plurality of divided areas, the area indicated by the voice command can be set as the sounding area. For example, when the user speaks "1", the mobile terminal can extract the voice control command "1" from the user voice. Then, a breathing area can be set so as to correspond to the extracted voice control command. It can be confirmed that the picked-up area is set in the divided No. 1 area in FIG. 6C.

Also, the display unit 151 can visually identify the breathing area. For example, a dotted line in FIG. 5C and a solid line boundary line are illustrated in FIG. 6C. However, it is also possible to display the breathing area with an object such as an indicator / icon indicating a specific area. Then, the identification items of the breath taking area can fade out after a predetermined time elapses.

The identification items can be displayed only when a sound is generated in the corresponding area.

Returning to the description of steps S420 and S430 of FIG. 4, the mobile terminal transmits the sound source acquired from the microphone (s) except for the sound source acquired from the first microphone and the first microphone disposed at a position where the user can easily acquire the voice The user's voice can be obtained by comparison. Then, the voice control command can be extracted by analyzing the user voice. Here, as a technique for extracting a voice control command from a user's voice, a speaker-dependent type or a speaker-independent voice recognition technology may be used.

Here, the speaker dependent speech recognition technology is a technique of recognizing a voice by recording a necessary voice several times in advance and matching the user's voice. Speaker independent speech technology is a technique of recognizing the text itself without recording the voice contents in advance.

On the other hand, the voice control command means a sound source previously stored as an input of a user controlling the mobile terminal. Examples of the voice command for moving the loud sound region include words (e.g., left, right, left, and the like) having directional meaning, words (e.g., Quot; first time ", etc.). An embodiment of the voice control command will be described with reference to Fig.

Before describing FIG. 7, it is assumed that the breathing area is divided into five areas as described in FIGS. 6B and 6C, and the breathing area is set as the first breathing area.

Fig. 7A is an example for setting any one of a plurality of areas as a stooping area. As shown in FIG. 7A, when the user inputs the voice control command "2 ", the mobile terminal can change the sound effect area to the second sound effect area and display it on the display part 151. [ The voice control command changing to the different sound effect area may be changeable by the manufacturer or the user.

FIG. 7B is an example for setting at least two or more regions out of a plurality of regions at the same time as a sounding region. As shown in FIG. 7B, when a voice control command of "2 times 3" is inputted by the user, the mobile terminal changes the picked-up region to a new picked-up region where the picked-up region 2 and the picked-up region 3 are combined, (151). On the other hand, although not shown in FIG. 7B, when a voice control command of "2" is input in the new sound effect area combined by the above-described voice control command, the sound effect area can be reduced again to the second sound effect area. The voice control command to enlarge or reduce the breathing area may also be changeable by the manufacturer or the user.

FIG. 7C is an example for changing the breath taking area on the basis of the current breath taking area. As shown in FIG. 7C, when the voice control command "Left" of the user is input, the mobile terminal can move to the left based on the current reference. On the contrary, when the user's voice control command "Right" is inputted, the mobile terminal can move rightward to the current reference based on the sounding area. The voice control command to change the sounding area to the left or right may also be changeable by the manufacturer or the user. As an example, a voice control command of "left" or "left" may be used instead of a voice control command of "Left".

Referring again to FIG. 4, the mobile terminal can control the microphone 122 to acquire a sound source centered on the set sounding area (S440). Here, the sound source may include various kinds of sound sources such as a voice of a person, a sound of a musical instrument, and a noise generated by noise.

The mobile terminal according to the embodiment of the present invention removes the user's voice from the acquired sound source (S450), compresses the removed sound source and the image together, and stores the compressed sound source as a moving image (S460). In this case, the compressed moving picture can be stored in the memory 160 of the mobile terminal. The steps S440 through S460 of FIG. 4 will be described in detail with reference to FIG.

The step S440 of acquiring the sound source centering on the sound acquisition area may be performed by applying the audio zooming technique. Specifically, referring to the side microphones of FIG. 8, an audio zooming technique may be applied to acquire a sound source centered on the sound acquisition area set by the control unit 180 among all sound sources acquired through a plurality of microphones. In one embodiment of the audio zooming technique, a microphone having a high rate of acquiring a sound source generated in the sound acquisition area is selected, and the sound source acquired from the selected microphone is enhanced, and the sound source acquired from the unselected microphone is subjected to noise cancellation (Noise Cancellation) processing can be performed to acquire the sound source around the sound taking region. However, the audio zooming technique used to implement general directional microphones not limited to this embodiment can also be applied to the present invention. In addition, although FIG. 8 illustrates that the audio zooming technique is applied using a microphone disposed on the side of the mobile terminal, it is needless to say that a microphone disposed on the front and rear sides of the mobile terminal may also be used.

The step S450 of removing the user's voice from the sound source acquired at the center of the sound acquisition area will be described with reference to the rear microphone and the front microphone in Fig.

Referring to FIG. 8, the rear microphone disposed on the rear side of the mobile terminal can acquire a front sound in the direction of photographing more than the front microphone. On the other hand, the front microphone acquires a rear sound, which is the voice of the user, . Therefore, the sound source acquired from the front microphone can be compared with the sound source acquired from the rear microphone, and the user's voice can be extracted. The voice of the user extracted here can be used not only for judging a voice control command but also for removing from a sound source acquired at the center of a sound region.

As described above, the mobile terminal according to the present invention has the effect that the voice of the subject to be photographed can be recorded more clearly by deleting the user's voice from the sound source acquired at the center of the sounding area.

Then, the sound source obtained around the sound-absorbing region may be compressed and stored as a moving image in combination with the image photographed by the camera unit (S460). In this case, video compression storage can be performed according to H.264 / AVC, H.264 / HEVC standard, etc. which are generally used.

In FIG. 4, it is exemplified that the acquired sound source and image can be compressed together and stored as moving images. However, in the example of FIG. 4, the step S460 of storing moving images is not essential, so it may be omitted in some cases.

The mobile terminal according to the embodiment of the present invention has the effect of improving user convenience.

Specifically, it is possible to provide a mobile terminal capable of recognizing a voice of a user and acquiring a sound source around the sound acquisition region.

In addition, since the user can set the sound-absorbing area without inputting a separate touch, there is an effect that no shaking or noise occurs when the moving picture is shot.

In the above-described embodiment, it is assumed that the display unit 151 of the mobile terminal is a touch screen. If the display portion 151 of the mobile terminal is not a touch screen, the touch input in the above-described various embodiments may be replaced by other user inputs that a person skilled in the art can devise.

In addition, the mobile terminal according to the embodiment of the present invention has been exemplified that the breathing area is set / changed as speech. However, if a mode other than the voice command mode is selected in Fig. 6A, an area set by zooming in the camera lens, an area manually set by the user, a focused subject, and the like may be set as a loud area.

According to one embodiment of the present invention, the above-described method (operation flow diagram) can be implemented as a code that can be read by a processor on a medium on which the program is recorded. Examples of the medium that can be read by the processor include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, etc., and may be implemented in the form of a carrier wave (e.g., transmission over the Internet) .

The above-described mobile terminal 100 is not limited in the configuration and method of the above-described embodiments, but the embodiments may be modified such that all or some of the embodiments are selectively combined .

121:
122: microphone section
151:
180:

Claims (5)

A camera unit for capturing an image;
A microphone unit including a plurality of microphones; And
And a control unit for setting the sound image area in the image photographed through the camera unit and controlling the microphone unit to acquire sound sources around the set sound region,
Wherein,
And sets the sounding area on the basis of the voice control command extracted from the voice of the user acquired through the microphone unit. The method according to claim 1,
Wherein,
Storing a sound source acquired with the sounding region as a center and a captured image as a moving image,
And when the moving picture is stored, removes the voice of the user. The method according to claim 1,
Wherein at least one of the plurality of microphones comprises:
A first microphone disposed in front of the mobile terminal,
Wherein,
And compares the sound source acquired from the first microphone with the sound source acquired from the microphone except for the first microphone to acquire the user's voice. The method according to claim 1,
And a display unit for displaying the image on the image so as to visually identify the dirty area. A control method of a mobile terminal having a microphone unit including a plurality of microphones,
Capturing an image;
Extracting a voice control command from the voice of the user acquired by the microphone unit;
Setting a breathing area in the photographed image based on the extracted voice control command; And
And controlling the microphone unit to acquire sound sources at the center of the sound taking area.

Images