KR20140034528A - Mobile terminal and control method for mobile terminal - Google Patents

Abstract

The present invention relates to a mobile terminal and a control method thereof. The mobile terminal according to the present invention obtains the movement of a subject and a camera upon entry into a camera photographing mode, inactivates a voice recognition function while the camera or the subject is moving, and activates the voice recognition function when the camera or the subject is stopped. [Reference numerals] (AA) Start; (BB,DD,FF,HH,JJ) No; (CC,EE,GG,II,KK) Yes; (LL) End; (S101) Entry into a camera photographing mode; (S102) Obtain the movement of a camera and a subject; (S103) Is the camera stopped?; (S104) Is the subject stopped?; (S105) Inactivate a voice recognition function; (S106) Active the voice recognition function; (S107) Is voice input?; (S108) Compare the received voice input with a voice command; (S109) Is similarity greater than a predetermined value?; (S110) Capture an external image; (S111) Guide a voice recognition failure; (S112) Does the camera photographing mode end?

Description

Mobile terminal and control method for mobile terminal

The present invention relates to a mobile terminal and a control method of the mobile terminal.

A terminal can be divided into a mobile terminal and a stationary terminal depending on whether the terminal is movable. Again, the mobile terminal may be divided into a handheld terminal and a vehicle mount terminal according to whether the user can carry the mobile phone directly.

Recently, many functions of recognizing a user's voice as a command have been installed in a mobile terminal. In particular, the function of operating the camera by recognizing the user's voice supports the user to take a picture even when the user is far from the terminal.

On the other hand, in the process of operating the camera by recognizing the user's voice, there is a case that the camera malfunctions because the recognition rate of the voice command falls. In particular, due to the characteristics of the prior art in which the voice recognition function is always activated while the camera is being driven, a noise that is not a voice command is accepted as a voice input and malfunctions frequently.

Also, since the speech rate, speech size, and intonation are different from person to person and even the same person may speak voice commands with different speech characteristics depending on the situation, even if the same voice command is spoken, voice recognition is successful depending on the person and the situation. It does not happen.

An object of the present invention is to provide a mobile terminal and a control method of the mobile terminal for improving the recognition rate of the voice command for driving the camera.

A mobile terminal according to an aspect of the present invention, a microphone; A camera for photographing a subject; And when the camera enters a shooting mode using the camera, controls activation of a voice recognition function using the microphone based on at least one of a movement of a mobile terminal and a movement of the subject. When the voice recognition function is activated, the voice recognition function is activated. And a control unit for outputting information informing that the device is activated.

In addition, a mobile terminal according to another aspect of the present invention, a microphone for receiving a voice input; A display module; A memory for storing a voice command corresponding to a camera photographing function; A camera for acquiring an external image; And a display result of comparing the voice input received in the camera photographing mode with a feature waveform of the voice command through the display module, wherein a similarity between the voice input and the voice command is preset based on the comparison result. If it is above, it includes a control unit for controlling the camera to capture the external image taken by the camera.

In addition, a control method of a mobile terminal according to an aspect of the present invention includes the steps of: entering a shooting mode using a camera; Obtaining movement of the mobile terminal and movement of the subject; Activating a voice recognition function using a microphone when the movement of the mobile terminal and the movement of the subject are less than or equal to a preset value; And outputting information indicating that the voice recognition function is activated.

In the mobile terminal and the method of controlling the mobile terminal according to the present invention, when the camera is moving or the subject is moving and is not ready to take a picture, the mobile terminal deactivates the voice recognition function to convert the input noise into the voice command. It can prevent malfunction caused by recognition. In addition, when the voice recognition is ready, it is notified to the user, thereby increasing the voice recognition success rate.

The mobile terminal previews the feature waveform that is the reference for the voice input to be spoken to the user in advance, and upon receiving the user's voice input, displays the result of comparing the feature waveform of the spoken voice input with the reference feature waveform. By doing so, it is possible for the user to intuitively grasp whether he or she has uttered correctly. In addition, when the user does not speak properly, the user has an effect of inducing the user to recognize the wrong speech and speak correctly based on the comparison result.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.
2 is a flowchart illustrating a control method of a mobile terminal according to a first embodiment of the present invention.
3 illustrates an example of displaying an icon representing a movement state of a camera.
4 illustrates an example of displaying an icon indicating a movement state of a subject.
5 illustrates an example of a screen for simultaneously displaying an icon representing a camera movement and an icon representing the movement of a subject.
6 illustrates an example of displaying an icon indicating that voice recognition is ready.
7 illustrates an example of displaying an icon indicating a state in which a voice input cannot be received.
8 illustrates an example of guiding completion of an autofocus operation and activation of a voice recognition function.
9 is a flowchart illustrating a control method of a mobile terminal according to a second embodiment of the present invention.
10 and 11 illustrate examples of displaying a feature waveform of a voice command preset in operation S203 of FIG. 9.
12 and 13 illustrate examples of displaying a result of comparing feature waveforms in step S206 of FIG. 8.

The above objects, features and advantages of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. It is to be understood, however, that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and similarities. Like reference numerals designate like elements throughout the specification. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. In addition, numerals (e.g., days, days, etc.) used in the description of the present invention are merely an identifier for distinguishing one component from another component

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

In addition, the suffix "module" and " part "for constituent elements used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

The mobile terminal described in this specification may include a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player) .

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a mobile terminal according to embodiments of the present invention.

The mobile terminal 100 includes a wireless communication unit 110, an audio / video input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, A controller 170, a controller 180, a power supply 190, and the like. The components shown in Fig. 1 are not essential, and a mobile terminal having more or fewer components may be implemented.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more modules for enabling wireless communication between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and the network in which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives a broadcast signal and / or broadcast related information from an external broadcast management server through a broadcast channel.

The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may refer to a server for generating and transmitting broadcast signals and / or broadcast related information, or a server for receiving broadcast signals and / or broadcast related information generated by the broadcast management server and transmitting the generated broadcast signals and / or broadcast related information. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

The broadcast-related information may refer to a broadcast channel, a broadcast program, or information related to a broadcast service provider. The broadcast-related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast-related information may exist in various forms. For example, an EPG (Electronic Program Guide) of a DMB (Digital Multimedia Broadcasting) or an ESG (Electronic Service Guide) of a DVBH (Digital Video BroadcastHandheld).

The broadcast receiving module 111 receives broadcast signals using various broadcasting systems. In particular, the broadcast receiving module 111 may be a digital multimedia broadcasting broadcasting (DMBT), a digital multimedia broadcasting satellite (DMBS), a media forward link only (MediaFLO), a digital video broadcasting ), And ISDBT (Integrated Services Digital Broadcast Terrestrial). Of course, the broadcast receiving module 111 may be adapted to other broadcasting systems that provide broadcast signals as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 transmits and receives radio signals to at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data depending on a voice call signal, a video call signal or a text / multimedia message transmission / reception.

The wireless Internet module 113 refers to a module for wireless Internet access, and the wireless Internet module 113 can be embedded in the mobile terminal 100 or externally. WLAN (WiFi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access) and the like can be used as wireless Internet technologies.

The short-range communication module 114 refers to a module for short-range communication. Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra wideband (UWB), ZigBee, and the like can be used as the short distance communication technology.

The location information module 115 is a module for confirming or obtaining the location of the mobile terminal. A typical example of the location information module is a GPS (Global Position System) module. According to the current technology, the GPS module 115 calculates information on a distance (distance) from three or more satellites to one point (object), information on the time when the distance information is measured, , It is possible to calculate three-dimensional position information according to latitude, longitude, and altitude of one point (individual) in one hour. Further, a method of calculating position and time information using three satellites and correcting the error of the calculated position and time information using another satellite is also used. The GPS module 115 continues to calculate the current position in real time and uses it to calculate speed information.

Referring to FIG. 1, an A / V (Audio / Video) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video communication mode or the photographing mode. The processed image frame can be displayed on the display module 151. [

The image frame processed by the camera 121 may be stored in the memory 160 or transmitted to the outside through the wireless communication unit 110. [ The camera 121 may be equipped with two or more cameras according to the configuration of the terminal.

The microphone 122 receives an external sound signal through a microphone in a communication mode, a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 when the voice data is in the call mode, and output. Various noise reduction algorithms may be implemented in the microphone 122 to remove noise generated in receiving an external sound signal.

The user input unit 130 generates input data for a user to control the operation of the terminal. The user input unit 130 may include a key pad dome switch, a touch pad (static pressure / capacitance), a jog wheel, a jog switch, and the like.

The sensing unit 140 senses the current state of the mobile terminal 100 such as the open / close state of the mobile terminal 100, the position of the mobile terminal 100, the presence or absence of user contact, the orientation of the mobile terminal, And generates a sensing signal for controlling the operation of the mobile terminal 100. For example, when the mobile terminal 100 is in the form of a slide phone, it may sense whether the slide phone is opened or closed. In addition, it may be responsible for a sensing function related to whether the power supply unit 190 is powered on, whether the interface unit 170 is connected to an external device, and the like. Meanwhile, the sensing unit 140 may include a proximity sensor.

The output unit 150 is for generating output related to visual, auditory or tactile sense and includes a display module 151, an acoustic output module 152, an alarm unit 153, and a haptic module 154 .

The display module 151 displays information processed by the mobile terminal 100. For example, when the mobile terminal 100 is in the call mode, a UI (User Interface) or a GUI (Graphic User Interface) associated with a call is displayed. When the mobile terminal 100 is in the video communication mode or the photographing mode, the photographed and / or received video or UI and GUI are displayed.

The display module 151 may be a liquid crystal display, a thin film transistor liquid crystal display, an organic light emitting diode, a flexible display, a 3D display And may include at least one.

Some of these displays may be transparent or light transmissive so that they can be seen through. This may be referred to as a transparent display. A typical example of the transparent display is a transparent LCD or the like. The rear structure of the display module 151 may also be of a light transmission type. With this structure, the user can see an object located behind the terminal body through the area occupied by the display module 151 of the terminal body.

There may be two or more display modules 151 according to the embodiment of the mobile terminal 100. For example, in the mobile terminal 100, a plurality of display modules may be spaced apart or integrally arranged on one surface, and may be disposed on different surfaces, respectively.

In a case where the display module 151 and the sensor for sensing the touch operation (hereinafter, referred to as 'touch sensor') have a mutual layer structure (hereinafter referred to as 'touch screen' It can also be used as an input device. The touch sensor may have the form of, for example, a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in a pressure applied to a specific portion of the display module 151 or a capacitance generated in a specific portion of the display module 151 into an electrical input signal. The touch sensor can be configured to detect not only the position and area to be touched but also the pressure at the time of touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and transmits the corresponding data to the controller 180. Thus, the control unit 180 can know which area of the display module 151 is touched or the like.

Referring to FIG. 1, a proximity sensor may be disposed in an inner region of the mobile terminal or in the vicinity of the touch screen, which is enclosed by the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or a nearby object without mechanical contact using the force of an electromagnetic field or infrared rays. The proximity sensor has a longer life span than the contact sensor and its utilization is also high.

Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor.

And to detect the proximity of the pointer by the change of the electric field along the proximity of the pointer when the touch screen is electrostatic. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of recognizing that the pointer is positioned on the touch screen while the pointer is not in contact with the touch screen is referred to as "proximity touch & The act of actually touching the pointer on the screen is called "contact touch. &Quot; The position where the pointer is proximately touched on the touch screen means a position where the pointer is vertically corresponding to the touch screen when the pointer is touched.

The proximity sensor detects a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, a proximity touch movement state, and the like). Information corresponding to the detected proximity touch operation and the proximity touch pattern may be output on the touch screen.

The audio output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory 160 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output module 152 outputs an acoustic signal related to a function (e.g., a call signal reception sound, a message reception sound, etc.) performed in the mobile terminal 100. The audio output module 152 may include a receiver, a speaker, a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event of the mobile terminal 100. Examples of events that occur in the mobile terminal include call signal reception, message reception, key signal input, touch input, and the like. The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than the video signal or the audio signal, for example, vibration. The video signal or audio signal may also be output through the display module 151 or the audio output module 152.

The haptic module 154 generates various tactile effects that the user can feel. Vibration is a representative example of the haptic effect generated by the haptic module 154. The intensity and pattern of the vibration generated by the hit module 154 can be controlled. For example, different vibrations may be synthesized and output or sequentially output.

In addition to the vibration, the haptic module 154 may be configured to perform various functions such as an effect of stimulation by a pin arrangement vertically moving with respect to a contact skin surface, an effect of stimulation by air spraying force or suction force through a jet opening or a suction opening, A variety of tactile effects such as an effect of stimulation through contact of an electrode, an effect of stimulation by an electrostatic force, and an effect of reproducing a cold sensation using a heat absorbing or exothermic element can be generated.

The haptic module 154 can be implemented not only to transmit the tactile effect through the direct contact but also to feel the tactile effect through the muscles of the user's finger or arm. The haptic module 154 may include two or more haptic modules 154 according to the configuration of the portable terminal 100.

The memory 160 may store a program for the operation of the controller 180 and temporarily store input / output data (e.g., a phone book, a message, a still image, a moving picture, etc.). The memory 160 may store data on vibration and sound of various patterns outputted when a touch is input on the touch screen.

The memory 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), a RAM At least one of a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read- Type storage medium. The mobile terminal 100 may operate in association with a web storage that performs a storage function of the memory 160 on the Internet.

The interface unit 170 serves as a path for communication with all external devices connected to the mobile terminal 100. The interface unit 170 receives data from an external device or receives power from the external device to transfer the data to each component in the mobile terminal 100 or to transmit data in the mobile terminal 100 to an external device. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, A video input / output (I / O) port, an earphone port, and the like may be included in the interface unit 170.

The identification module is a chip for storing various information for authenticating the use right of the mobile terminal 100 and includes a user identification module (UIM), a subscriber identity module (SIM) (Universal Subscriber Identity Module, USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the port.

When the mobile terminal 100 is connected to an external cradle, the interface unit may be a path through which power from the cradle is supplied to the mobile terminal 100, or various command signals input by the user to the cradle may be transmitted It can be a passage to be transmitted to the terminal. The various command signals or the power source input from the cradle may be operated as a signal for recognizing that the mobile terminal is correctly mounted on the cradle.

The control unit 180 typically controls the overall operation of the mobile terminal. For example, voice communication, data communication, video communication, and the like. The control unit 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180. [

The controller 180 may perform a pattern recognition process for recognizing handwriting input or drawing input performed on the touch screen as characters and images, respectively.

The power supply unit 190 receives an external power source and an internal power source under the control of the controller 180 to supply power for operation of each component.

The various embodiments described herein may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein may be implemented as application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays May be implemented using at least one of processors, controllers, microcontrollers, microprocessors, and electrical units for performing functions. In some cases such embodiments may be implemented using a controller 180 < / RTI >

According to a software implementation, embodiments such as procedures or functions may be implemented with separate software modules that perform at least one function or operation. The software code may be implemented by a software application written in a suitable programming language. In addition, the software codes may be stored in the memory 160 and executed by the control unit 180. [

Embodiments disclosed in this document may be implemented in the mobile terminal 100 described with reference to FIG. 1.

In this document, the display module 151 is assumed to be a touch screen 151. As described above, the touch screen 151 can perform both the information display function and the information input function. However, it should be clear that the present invention is not limited thereto.

In this document, the touch gesture refers to a gesture implemented by touching or touching the touch screen 151 in a touch manner, and the touch input means an input received by the touch gesture.

The touch gesture is divided into tapping, dragging, flicking, press, multi-touch, pinch in, and pinch-out depending on the operation. do.

Tapping refers to a touch gesture, such as a click of a mouse on a general computer, by gently pressing and releasing the touch screen 151 once.

In addition, dragging may be performed by moving to a specific position while touching the touch screen 151, and when the object is dragged, the object may be continuously displayed in accordance with the drag direction.

In addition, flicking refers to an operation of touching the touch screen 151 and then moving the contact in a specific direction (upward, downward, left, right or diagonal) and then releasing the contact point. The mobile terminal 100, The processing of the specific operation is performed based on the flicking direction, speed, and the like. For example, the page turning operation of the e-book can be performed based on the flicking direction.

In addition, the press refers to an operation of continuously holding the touch for a predetermined time or longer after touching the touch screen 151. [

In addition, the multi-touch means an operation of touching a plurality of points of the touch screen 151 at the same time.

In addition, the pinch-in refers to an operation of dragging the touch screen 151 in a direction in which a plurality of pointers that are multi-touching approach each other. That is, a drag that starts from at least one point among a plurality of points that are multi-touched on the touch screen 151 and occurs in a direction in which a plurality of points that are multi-touched approach each other.

In addition, pinch-out refers to an operation of dragging the touch screen 151 in a direction away from the plurality of pointers in multi-touch. That is, the drag starts from at least one point among a plurality of points that are multi-touched on the touch screen 151 and occurs in a direction in which the plurality of points that are multi-touch are separated from each other.

In this document, the sensing unit 140 may include a gyro sensor (Gyroscope), an accelerometer (Accelerometer), a geomagnetic sensor (Magnetic sensor).

The gyro sensor may acquire a rotational speed or angular speed according to the rotation of the mobile terminal 100 on the axis, that is, the rotation of the mobile terminal 100. In addition, the acceleration sensor may obtain gravity acceleration according to the movement of the mobile terminal 100. In addition, the geomagnetic sensor may acquire the orientation of the mobile terminal 100 like a compass.

The controller 180 is based on at least one of the angular velocity obtained through the gyro sensor of the sensing unit 130, the gravity acceleration obtained through the acceleration sensor, and the orientation of the mobile terminal 100 obtained through the geomagnetic sensor. ) Can be obtained. Here, the movement of the mobile terminal may include the movement, tilting, horizontal state, rotation, etc. of the body of the mobile terminal 100.

Hereinafter, a method of controlling a mobile terminal and an operation of the mobile terminal for implementing the same according to the first embodiment of the present invention will be described in detail with reference to the accompanying drawings.

2 is a flowchart illustrating a control method of a mobile terminal according to a first embodiment of the present invention. 3 to 8 are diagrams for describing the method of FIG. 2.

Referring to FIG. 2, the controller 180 enters a camera photographing mode based on a user control input input through the user input unit 180 (S101). Accordingly, the controller 180 turns on the camera 121 and displays a preview image of an external image input through the camera 121 on the touch screen 151. On the other hand, when entering the camera shooting mode, the initial state of the voice recognition function using the microphone 122 may be set to a deactivation state.

In the camera photographing mode, the controller 180 continuously acquires the movement of the camera 121 and the movement of the subject being photographed through the camera 121 (S102). In addition, the controller 180 determines whether the camera 121 and the subject are stopped based on the movement of the camera 121 and the subject acquired in step S102 (S103 and S104).

The controller 180 determines that the camera 121 is moving without stopping through the step S103, or if it is determined that the subject being photographed by the camera 121 is moving without stopping through the step S104, the microphone 122 is moved. The used speech recognition function is deactivated (S105). In addition, if the voice recognition function using the microphone 122 is currently in an inactive state, the state is maintained.

On the other hand, when both the camera 121 and the subject are stationary, the voice recognition function using the microphone 122 is activated (S106).

As the voice recognition function is activated, the controller 180 waits to receive a voice input of the user through the microphone 122 (S107).

When a voice input of a user is received through the microphone 122, the controller 180 extracts a feature pattern from the received voice input. The extracted feature pattern is compared with a feature pattern of a preset voice command (S108).

In addition, the controller 180 obtains a similarity degree between the received voice input and the preset voice command based on the feature pattern comparison result. Then, the similarity between the received voice input and the preset voice command is compared with the preset value (S109).

When the similarity between the received voice input and the preset voice command is greater than or equal to the preset value, the controller 180 controls the camera 121 to capture an external image input through the camera 121 (S110).

On the other hand, if the similarity between the received voice input and the preset voice command is less than the preset value, the controller 180 determines that the voice recognition has failed. Accordingly, the controller 180 outputs information guiding that voice recognition has failed (S111).

Steps S102 to S111 may be repeatedly performed until the camera photographing mode ends (S112).

In step S102, the controller 180 may acquire the movement of the camera 121 in various ways.

For example, the controller 180 may acquire the movement of the camera 121 through the sensing unit 110. That is, the controller 180 may acquire the movement of the camera 121 by acquiring the movement of the body of the mobile terminal 100 on which the camera 121 is mounted through the sensing unit 110. In this case, the sensing unit 110 may indicate the movement of the camera 121 by using the sensing value that detects and outputs the movement of the mobile terminal 100.

Also, for example, the controller 180 may acquire the movement of the camera 121 by comparing the image frames obtained through the camera 121 with each other using a scene change detection algorithm. In this case, the motion of the camera 121 may be represented by using a motion vector value obtained by comparing successive frames with each other or a difference of each pixel value.

In addition, in step S102, the controller 180 can obtain the movement of the subject by comparing the image frames obtained through the camera 121 using a scene change detection algorithm.

Meanwhile, according to the first embodiment of the present invention, the controller 180 may acquire the subject movement based on the scene change of the entire external image acquired through the camera 121, and only the movement of the specific object except the background Can also be obtained by the movement of. In the latter case, the controller 180 may control to activate the voice recognition function when the object to be photographed is stationary even if the background is moved. The controller 180 uses an object tracking algorithm, a face recognition algorithm, or the like to extract only a movement of a specific object from an external image. That is, only a specific object is extracted from each image frame obtained through the camera 121 using an object tracking algorithm or a face recognition algorithm, and only a change of the extracted object is detected regardless of the background to obtain only the movement of the specific object. Can be. In this case, a motion vector corresponding to a specific object may be used as the motion value of the subject.

Referring back to FIG. 2, in step S103, the controller 180 determines whether the camera 121 is in a stationary state by comparing the movement of the camera 121 acquired in step S102 with a preset first threshold value. . Here, the first threshold value is a value indicating how far the camera 121 is allowed to activate the voice recognition function and may be set by the user.

In operation S103, the controller 180 may display an icon indicating whether the camera 121 is moving on the screen.

For example, the controller 180 may display an icon on the screen while the camera 121 is moving, and remove the icon from the screen when the camera 121 is stopped, thereby indicating the movement state of the camera 121.

For example, the controller 180 may display different colors of the icons when the camera 121 is moving and when the camera 121 is moving, or display different icons on the screen when the camera 121 is moving and when the camera 121 is moving. By displaying, the movement state of the camera 121 can also be shown.

3 illustrates an example of displaying an icon representing a movement state of a camera.

Referring to FIG. 3, the controller 180 displays a preview image 3 of an external image input through the camera 121 on the touch screen 151.

In addition, when the camera 121 is moving without stopping, an icon 3a indicating that the camera 121 is moving is displayed on the preview image 3.

On the other hand, if the camera 121 stops without moving, the controller 180 makes the icon 3a disappear from the screen or changes the shape, color, and the like of the icon 3a, intuitively as when the camera 121 is moving. Make it distinct.

Referring back to FIG. 2, in step S104, the controller 180 determines whether the subject is in a stationary state by comparing the movement of the subject acquired in step S102 with a preset second threshold value. Here, the second threshold value is a value indicating how far the subject's movement can be allowed to activate the voice recognition function, and can be set by the user.

In operation S104, the controller 180 may display an icon indicating whether the subject is moving on the screen.

For example, the controller 180 may display an icon on the screen while the subject is moving, and remove the icon from the screen when the subject is stopped, thereby indicating the movement state of the subject.

Also, for example, the controller 180 may display a different color of the icon when the subject is moving and when the subject is moving, or display different icons on the screen when the subject is moving and when the subject is moving. It can also indicate status.

4 illustrates an example of displaying an icon indicating a movement state of a subject.

Referring to FIG. 4, the controller 180 displays a preview image 3 of an external image input through the camera 121 on the touch screen 151.

In addition, when the subject is moving without stopping, an icon 3b indicating that the subject is moving is displayed on the preview image 3.

On the other hand, when the subject stops without moving, the controller 180 makes the icon 3b disappear from the screen or changes the shape, color, etc. of the icon 3b so that it is intuitively distinguished from when the subject is moving.

In FIGS. 3 and 4, an example of displaying a motion state of the camera 121 and an icon representing a motion state of the subject, respectively, has been described as an example. In FIGS. 3 and 4, the movement state of the camera 121 and the subject Icons representing the movement state may be simultaneously displayed on the screen.

5 illustrates an example of a screen for simultaneously displaying an icon representing a camera movement and an icon representing the movement of a subject.

Referring to FIG. 5, when the camera 121 is moving without stopping, the controller 180 displays an icon 3a indicating that the camera 121 is moving on the preview image 3. In addition, when the subject is moving without stopping, an icon 3b indicating that the subject is moving is displayed on the preview image 3.

After that, when the camera 121 stops without moving, the controller 180 causes the icon 3a indicating the movement state of the camera 121 to disappear from the screen or changes the shape, color, etc. of the icon 3a. By changing, the camera 121 is intuitively distinguished from when the camera 121 is moving. In addition, the controller 180 may make the icon 3b representing the movement of the subject disappear from the screen or change the shape, color, etc. of the icon 3b when the subject stops without moving, so that the controller 180 may be intuitively distinguished from when the subject is moving. do.

Meanwhile, in the above-described FIGS. 3 to 5, when the camera 121 moves or the subject moves, the icon 3a indicating the movement state of the camera 121 or the movement of the subject is displayed. The voice recognition function is deactivated using only the icon 3b indicating. However, it is clear that the present invention is not limited thereto. According to the present invention, when the camera 121 is moved or the subject is in motion and the voice recognition function is deactivated, the controller 180 may display the icon 3a indicating the movement state of the camera 121 and the movement of the subject. At least one of the displayed icons 3b and an icon indicating that the voice recognition function is inactive may be displayed together. An icon indicating that the voice recognition function is inactive will be described in detail with reference to FIG. 7 to be described later.

Referring back to FIG. 2, in steps S105 and S106, the controller 180 may control activation / deactivation of the voice recognition function in various ways.

For example, the controller 180 may control activation / deactivation of the voice recognition function by controlling the on / off of the microphone 122. In this case, the controller 180 may activate the voice recognition function by turning on the microphone 122 and deactivate the voice recognition function by turning off the microphone 122.

In addition, for example, the controller 180 may control activation / deactivation of the voice recognition function by activating / deactivating a software module that performs the voice recognition function while the microphone 122 is turned on. In this case, the controller 180 may activate the voice recognition function by activating a software module that performs the voice recognition function, and deactivate the voice recognition function by deactivating the software module that performs the voice recognition function.

In addition, for example, the controller 180 may control activation / deactivation of the voice recognition function by simultaneously controlling on / off of the microphone 122 and activation / deactivation of a software module that performs the voice recognition function. In this case, the controller 180 activates the voice recognition function by turning on the microphone 122 and activating the software module that performs the voice recognition function, and deactivates the software module that turns off the microphone 121 and performs the voice recognition function. The voice recognition function can be deactivated by doing so.

Referring back to FIG. 2, in step S106, when the voice recognition function is activated, the controller 180 can output guide information for guiding that it is ready to receive a voice input.

For example, the controller 180 reads 'Ready?', 'Are you ready to take it?' For example, voice information indicating that the voice recognition is ready may be output through the sound output module 152.

For example, the controller 180 may display an icon or the like on the touch screen 151 indicating that the voice recognition is ready.

6 illustrates an example of displaying an icon indicating that voice recognition is ready.

Referring to FIG. 6, the controller 180 displays a preview image 3 of an external image input through the camera 121 on the touch screen 151.

In addition, when both the camera 121 and the subject are stationary, the voice recognition function is activated. In addition, as the voice recognition function is activated, an icon 3c indicating that the voice input is ready to be received is displayed on the preview image 3.

On the other hand, the controller 180 can also display on the screen an icon indicating that the voice recognition function is inactivated to receive a voice input.

7 illustrates an example of displaying an icon indicating a state in which a voice input cannot be received.

Referring to FIG. 7, the controller 180 displays a preview image 3 of an external image input through the camera 121 on the touch screen 151.

In addition, when at least one of the camera 121 and the subject is moving, the voice recognition function is deactivated. In addition, an icon 3d is displayed on the preview image 3 indicating that the speech recognition function is deactivated so that it is impossible to receive a speech input.

On the other hand, in FIG. 7 described above, the case in which only the icon indicating the non-receiving state of the voice input is displayed is described as an example when the voice recognition function is in an inactive state, but the present invention is not limited thereto. According to the present invention, the controller 180 may include at least one of an icon 3a representing the movement state of the camera 121 described above with reference to FIGS. 3 to 5 and an icon 3b representing the movement of the subject, and a voice. The icon 3d may also be displayed to indicate that the recognition function is inactive.

Referring back to FIG. 2, in step S106, when the auto focus function is being performed, the controller 180 activates the voice recognition function or the voice recognition function is activated when the auto focus operation is completed. Even if the autofocus operation is completed, guide information may be output to guide that the voice input is ready to be received.

8 illustrates an example of guiding completion of an autofocus operation and activation of a voice recognition function.

Referring to FIG. 8, when the auto focus function is completed while the voice recognition function is activated, the controller 180 displays an icon 3e indicating that the auto focus function is completed, and an icon 3c indicating that the voice recognition function is activated. Mark with In addition, the controller 180 can output guide information for guiding that it is ready to receive a voice input.

Referring back to FIG. 2, in step S111, when voice recognition fails, the controller 180 outputs a sound such as a voice, a buzzer, or the like through the sound output module 152, or touches a graphic object such as an icon. By displaying on the screen 151, guide information for guiding a failure in speech recognition can be output.

Meanwhile, although FIG. 2 illustrates a case in which the voice recognition function using the microphone 122 is activated only when both the camera 121 and the subject are stationary, the present invention is not limited thereto.

According to the present invention, the controller 180 may acquire only the movement of any one of the camera 121 and the subject, and control activation of the voice recognition function based on the movement. For example, the controller 180 may control whether the voice recognition function is activated by only the movement of the camera 180. Also, for example, the controller 180 may control activation of the voice recognition function only by the movement of the subject photographed by the camera 121.

According to the first embodiment of the present invention described above, when the camera 121 is moving or the subject is moving and is not ready to take a picture, the mobile terminal deactivates the voice recognition function to cancel input noise. It can prevent malfunction caused by recognition. In addition, by notifying the user when the voice recognition is ready, there is an effect of increasing the voice recognition success rate.

Hereinafter, a method of controlling a mobile terminal and an operation of a mobile terminal for implementing the same according to a second embodiment of the present invention will be described in detail with reference to the accompanying drawings.

9 is a flowchart illustrating a control method of a mobile terminal according to a second embodiment of the present invention. 10 to 13 are diagrams for describing the method of FIG. 2.

Referring to FIG. 8, the controller 180 enters a camera photographing mode based on a user control input input through the user input unit 180 (S201). Accordingly, the controller 180 turns on the camera 121 and displays a preview image of an external image input through the camera 121 on the touch screen 151.

When entering the camera photographing mode, the controller 180 checks whether the voice recognition function is activated (S202). Here, the method of controlling the activation / deactivation of the voice recognition function is performed in the same manner as described above using the steps S102 to S106 of FIG. 2, and thus, a detailed description thereof will be omitted below.

When the voice recognition function is activated, the controller 180 displays the feature waveform of the preset voice command on the preview image (S203). Here, the voice command is audio data that serves as a reference for determining whether a voice input received from a user corresponds to a specific control command. The voice command may be preset in the terminal production process or the application installation process or may be set by the user.

Meanwhile, when the voice input is activated, the controller 180 continuously monitors whether the voice input of the user is received through the microphone 122 (S204). When the user's voice input is received through the microphone 122, the feature waveform is extracted from the received voice input (S205).

In addition, the controller 180 obtains a similarity between the voice input and the preset voice command by comparing the feature waveform extracted from the voice input with the feature waveform of the preset voice command. In addition, the comparison result of the two feature waveforms is displayed on the preview image (S206).

Meanwhile, the controller 180 compares the similarity between the received voice input and the preset voice command with a preset value (S207). When the similarity between the received voice input and the preset voice command is greater than or equal to a preset value, the controller 180 controls the camera 121 to capture an external image input through the camera 121 (S208).

On the other hand, if the similarity between the received voice input and the preset voice command is less than the preset value, the controller 180 determines that the voice recognition has failed. Accordingly, the controller 180 outputs information guiding that the speech recognition has failed (S209).

Steps S202 to S209 may be repeatedly performed until the camera shooting mode ends (S210).

In step S203, the controller 180 may display the feature waveform of the voice command and each syllable of the voice command so that the user intuitively recognizes it.

10 and 11 illustrate examples of displaying a feature waveform of a voice command preset in step S203.

Referring to FIG. 10, the controller 180 displays a preview image 9 of an external image input through the camera 121 on the touch screen 151.

In addition, when the voice recognition function is activated, the text 9a guiding the voice command to which the picture capturing function is associated and the feature waveform 9b of the command are displayed on the preview image 9.

Referring to FIG. 11, the controller 180 can voice a feature waveform 9b of a voice command corresponding to a photographing function so that the controller 180 can intuitively grasp how the feature waveform appears when the user pronounces each syllable. Each syllable 9c of the command is displayed in a text format.

Referring back to FIG. 9, in step S206, when a voice input is received due to a user's utterance, the controller 180 compares a feature waveform extracted from the received voice input with a feature waveform of a preset voice command. In addition, the result of comparing the feature waveforms is displayed on the preview image. The comparison result of the feature waveforms may be displayed by superimposing the feature waveforms of the input voice input due to the user's speech on the feature waveforms of the preset voice command.

12 and 13 show examples of displaying a result of comparing feature waveforms in step S206.

12 and 13, the controller 180 compares a feature waveform between a preset voice waveform and a voice input input due to a user's speech. The comparison result 11a of the two feature waveforms is displayed in a form in which the two feature waveforms are superimposed so that the user can intuitively grasp the result 11a of comparing the two feature waveforms. FIG. 12 illustrates a case in which the similarity between two feature waveforms is determined to be greater than or equal to a preset value, and thus the voice recognition is successful. FIG. It is.

When the syllables of the voice command correspond to each syllable of the voice command in a text format, the controller 180 may highlight and display the syllable that fails to recognize the voice. For example, when speech recognition fails, the syllable that fails to recognize may be displayed in a different color from the remaining syllables, or the size of the syllable that fails to recognize may be displayed larger than the remaining syllables.

Referring back to FIG. 9, in step S209, when the voice recognition fails, the controller 180 outputs a sound such as a voice, a buzzer, or the like through the sound output module 152, or touches a graphic object such as an icon. By displaying on the screen 151, guide information for guiding a failure in speech recognition can be output.

In addition, in step S209, when the voice recognition fails, the controller 180 provides guidelines such as speech speed, speech size, and intonation, based on the result of the comparison of the feature waveforms to induce the user's speech correction. In addition, correction guide information may be output to indicate incorrect syllables. Here, an incorrect syllable indicates a syllable whose similarity is equal to or less than a preset value as a result of comparing the characteristic waveforms.

For example, when the speech size is small, the sound output "speak loudly" may be output through the sound output module 152 to induce the speech size to be increased.

In addition, for example, when the speech speed is fast, the sound output “speak slowly” may be output through the sound output module 152 in order to induce slow speech.

According to the second embodiment of the present invention described above, the mobile terminal previously shows a feature waveform that is a reference for a voice input to be uttered by the user, and when the user's voice input is received, the feature waveform of the uttered voice input is received. By displaying the result of comparing with the reference feature waveform as a reference, it is possible for the user to intuitively grasp whether or not he spoke correctly. In addition, when the user does not speak properly, the user has an effect of inducing the user to recognize the wrong speech and speak correctly based on the comparison result.

The above-described method of controlling a mobile terminal according to the present invention can be provided by being recorded in a computer-readable recording medium as a program for execution in a computer.

The control method of the mobile terminal according to the present invention can be executed through software. When executed in software, the constituent means of the present invention are code segments that perform the necessary tasks. The program or code segments may be stored in a processor readable medium or transmitted by a computer data signal coupled with a carrier in a transmission medium or communication network.

The computer-readable recording medium includes a mode type recording device in which data that can be read by a computer system is stored. Examples of computer-readable recording devices include ROM, RAM, CD-ROM, DVD ± ROM, DVD-RAM, magnetic tape, floppy disks, hard disks, optical data storage devices, and the like. The computer readable recording medium can also be distributed over network coupled computer devices so that the computer readable code is stored and executed in a distributed fashion.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. The present invention is not limited to the drawings, and all or some of the embodiments may be selectively combined so that various modifications may be made.

Claims (15)

MIC;
A camera for photographing a subject; And
When entering the shooting mode using the camera, the activation of the voice recognition function using the microphone is controlled based on at least one of the movement of the mobile terminal and the movement of the subject,
A controller for outputting information indicating that the voice recognition function is activated when the voice recognition function is activated
.
The method of claim 1,
Further comprising a sensing unit for acquiring the movement of the mobile terminal,
The controller activates the voice recognition function when the movement of the mobile terminal is equal to or less than a preset value.
The method of claim 1,
The controller acquires the movement of the subject through analysis of an image of the subject,
And the voice recognition function is activated when the movement of the subject is less than or equal to a preset value.
The method according to claim 2 or 3,
The controller provides a user interface for setting the preset value.
The method of claim 1,
The control unit controls the on / off of the microphone to control the activation of the voice recognition function.
The method of claim 1,
The control unit controls the activation of the voice recognition function by controlling the activation of the software module that receives the voice recognition function.
The method of claim 1,
Further comprising an audio output module,
The controller controls the sound output module to output voice information indicating that the voice recognition function is activated when the voice recognition function is activated.
The method of claim 1,
Further comprising a display module,
The controller controls the display module to display an icon indicating that the voice recognition function is activated when the voice recognition function is activated.
A microphone for receiving a voice input;
A display module;
A memory for storing a voice command corresponding to a camera photographing function;
A camera for acquiring an external image; And
A comparison result of comparing the voice input received in the camera shooting mode with a feature waveform of the voice command is displayed on the display module;
A controller for controlling the camera to capture the external image captured by the camera when the similarity between the voice input and the voice command is equal to or greater than a preset value based on the comparison result;
.
10. The method of claim 9,
When the controller enters the camera photographing mode, the controller displays a feature waveform of the voice command through the display module.
And displaying a feature waveform of the voice input received through the microphone overlapping with the feature waveform of the voice command.
11. The method of claim 10,
The control unit displays each syllable of the voice command in correspondence with a feature waveform of the voice command.
12. The method of claim 11,
The control unit displays a syllable having a similarity to a syllable of the voice input or less than a preset level among at least one syllable constituting the voice command based on the comparison result in a different color from the remaining syllables.
The method of claim 12,
If the similarity between the voice input and the voice command is less than a preset value, the controller may include at least one of syllables whose utterance rate, size, intonation, and similarity are less than or equal to a preset value to induce a user's utterance correction based on the comparison result. A mobile terminal, characterized in that for outputting calibration guide information including the information of.
Entering a shooting mode using a camera;
Obtaining movement of the mobile terminal and movement of the subject;
Activating a voice recognition function using a microphone when the movement of the mobile terminal and the movement of the subject are less than or equal to a preset value; And
Outputting information indicating that the speech recognition function is activated;
And transmitting the control information to the mobile terminal.
15. The method of claim 14,
Receiving a voice input when the voice recognition function is activated;
Comparing a feature waveform between a preset voice command and the voice input;
Displaying a comparison result of the feature waveforms; And
Capturing an external image input through the camera when the similarity between the voice command and the voice input is equal to or greater than a preset value based on the comparison result;
The control method of the mobile terminal further comprising.

Images