KR101574119B1 - Mobile terminal and method for displaying call contents thereof - Google Patents

Abstract

The present invention relates to a mobile terminal and a mobile terminal. According to the present invention, A sensing unit sensing a proximity between a mobile terminal and a human body; And a control unit for converting the voice signal received through the communication unit into text based on the sensing result of the sensing unit when it is determined that the proximity degree changes from the proximity state to the proximity cancel state during a voice call through the communication unit And a method of displaying a call content on the mobile terminal.

Mobile terminal, voice, text, conversion.

Description

Technical Field [0001] The present invention relates to a mobile terminal,

BACKGROUND OF THE INVENTION 1. Field of the Invention [0002] The present invention relates to a mobile terminal, and more particularly, to a mobile terminal capable of displaying a voice call content as text and a method of displaying the call content.

A terminal such as a personal computer, a notebook computer, or a mobile phone has various functions, for example, a multimedia device having a complex function such as photographing or photographing of a moving picture, reproduction of music or a moving picture file, (Multimedia player).

The terminal can move And may be divided into a mobile terminal and a stationary terminal depending on whether the mobile terminal is a mobile terminal or a mobile terminal. The mobile terminal can be divided into a handheld terminal and a vehicle mount terminal according to whether the user can directly carry the mobile terminal.

In order to support and enhance the functionality of the terminal, it is contemplated to improve the structural and / or software portion of the terminal.

2. Description of the Related Art [0002] In recent years, many terminals equipped with a function of recording / storing voice call contents by themselves have been widely used due to the necessity of searching and storing call contents. In addition, with the rapid development of speech recognition technology, terminals capable of executing specific functions (e.g., call connection, etc.) by combining such speech recognition technology are also being introduced.

The present invention provides a mobile terminal that is configured to facilitate checking, searching, and storing of conversation contents by converting a received voice signal into text and displaying the conversation contents, and a method of displaying a call content in the mobile terminal.

The present invention also provides a mobile terminal capable of memorizing or storing the contents of a call during a call, and a method of displaying a call content in the mobile terminal.

It is another object of the present invention to provide a mobile terminal which can easily check the contents of a call even in a case where it is difficult to grasp a clear voice as in a noisy environment, and a method of displaying a call content in the mobile terminal.

In addition, the present invention provides a mobile terminal in which a function of a message transmission, a receiver swap, a conference call, and the like can be easily performed on a party or a telephone number mentioned in a call, and a method of displaying a call content on the mobile terminal .

Other objects of the present invention will become readily apparent from the following description.

According to an aspect of the present invention for realizing the above-mentioned object, a mobile terminal according to the present invention includes: a communication unit; A sensing unit sensing a proximity between a mobile terminal and a human body of a user; And a control unit for converting the voice signal received through the communication unit into text based on the sensing result of the sensing unit when it is determined that the proximity degree changes from the proximity state to the proximity cancel state during a voice call through the communication unit .

In one embodiment, the control unit may display the text subjected to the voice-to-text conversion processing on a display.

In one embodiment, the control unit may display on the display a menu list relating to an executable function related to the selected text, when some or all of the text displayed on the display is selected.

In one embodiment, the sensing unit may include at least one of an infrared sensor, an ultrasonic sensor, a proximity sensor, a pressure sensor, and a touch sensor.

According to another aspect of the present invention, there is provided a method for displaying a call content in a mobile terminal, the method comprising: sensing proximity between a mobile terminal and a human body; Converting the voice signal received through the communication unit of the mobile terminal into text based on the sensing result when it is determined that the proximity degree changes from the proximity state to the proximity cancel state during the voice call; And displaying the text on a display.

The mobile terminal according to the present invention and the method for displaying a call content according to the present invention have the following effects.

The present invention can facilitate confirmation, search, and storage of conversation contents by converting the received speech signal into text and displaying it.

In addition, the present invention can memorize or store the contents of a call during a call.

In addition, the present invention can easily confirm the contents of a conversation even in a case where it is difficult to grasp a clear voice as in a noisy environment.

In addition, the present invention can easily perform operations such as message transmission, receiver swap, conference call, and the like to a party or a telephone number being referred to during a call.

The above objects, features and advantages of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. It is to be understood, however, that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and similarities. Like reference numerals designate like elements throughout the specification. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. Further, numerals (e.g., first, second, etc.) used in the description process of this specification are only an identifier for distinguishing one component from another component

Hereinafter, a mobile terminal related to the present invention will be described in detail with reference to the drawings. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

The mobile terminal described in this specification may include a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), navigation and the like. However, it will be understood by those skilled in the art that the configuration according to the embodiments described herein may be applied to a fixed terminal such as a digital TV, a desktop computer, and the like, unless the configuration is applicable only to a mobile terminal.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.

The mobile terminal 100 includes a wireless communication unit 110, an audio / video input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory unit 160, A controller 170, a controller 180, a power supply 190, and the like. The components shown in FIG. 1 are not essential, and a mobile terminal having more or fewer components may be implemented.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more modules for enabling wireless communication between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and the network in which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives broadcast signals and / or broadcast related information from an external broadcast management server through a broadcast channel.

The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may refer to a server for generating and transmitting broadcast signals and / or broadcast related information, or a server for receiving broadcast signals and / or broadcast related information generated by the broadcast management server and transmitting the generated broadcast signals and / or broadcast related information. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

The broadcast-related information may refer to a broadcast channel, a broadcast program, or information related to a broadcast service provider. The broadcast-related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast-related information may exist in various forms. For example, an EPG (Electronic Program Guide) of DMB (Digital Multimedia Broadcasting) or an ESG (Electronic Service Guide) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast receiving module 111 receives broadcasting signals using various broadcasting systems. In particular, the broadcasting receiving module 111 may be a Digital Multimedia Broadcasting-Terrestrial (DMB-T), a Digital Multimedia Broadcasting-Satellite (DMB-S) Only Digital Broadcast-Handheld (DVB-H), Integrated Services Digital Broadcast-Terrestrial (ISDB-T), and the like. Of course, the broadcast receiving module 111 may be adapted to other broadcasting systems that provide broadcast signals as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory unit 160.

The mobile communication module 112 transmits and receives radio signals to at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data depending on a voice call signal, a video call signal or a text / multimedia message transmission / reception.

The wireless Internet module 113 refers to a module for wireless Internet access, and the wireless Internet module 113 can be embedded in the mobile terminal 100 or externally. WLAN (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access) and the like can be used as wireless Internet technologies.

The short-range communication module 114 refers to a module for short-range communication. Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra wideband (UWB), ZigBee, etc. may be used as the short range communication technology.

The location information module 115 is a module for confirming or obtaining the location of the mobile terminal. A typical example of the location information module is a GPS (Global Position System) module. According to the current technology, the GPS module 115 calculates information on a distance (distance) from three or more satellites to one point (entity), information on the time when the distance information is measured, It is possible to calculate three-dimensional position information according to latitude, longitude, and altitude with respect to one point in one hour. Further, a method of calculating position and time information using three satellites and correcting the error of the calculated position and time information using another satellite is also used. The GPS module 115 continues to calculate the current position in real time and uses it to calculate speed information.

Referring to FIG. 1, an A / V (Audio / Video) input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video communication mode or the photographing mode. The processed image frame can be displayed on the display unit 151. [

The image frame processed by the camera 121 may be stored in the memory unit 160 or may be transmitted to the outside through the wireless communication unit 110. [ The camera 121 may be equipped with two or more cameras according to the configuration of the terminal.

The microphone 122 receives an external sound signal through a microphone in a communication mode, a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 when the voice data is in the call mode, and output. Various noise reduction algorithms may be implemented in the microphone 122 to remove noise generated in receiving an external sound signal.

The user input unit 130 generates input data for a user to control the operation of the terminal. The user input unit 130 may be implemented as a key pad, a dome switch, a touch pad (static / static), a jog wheel, a jog switch, or the like.

The sensing unit 140 senses the current state of the mobile terminal 100 such as the open / close state of the mobile terminal 100, the position of the mobile terminal 100, the presence or absence of user contact, the orientation of the mobile terminal, And generates a sensing signal for controlling the operation of the mobile terminal 100. For example, when the mobile terminal 100 is in the form of a slide phone, it is possible to sense whether the slide phone is opened or closed. In addition, it may be responsible for a sensing function related to whether the power supply unit 190 is powered on, whether the interface unit 170 is connected to an external device, and the like. Meanwhile, the sensing unit 140 may include a proximity sensor or the like to implement a method of displaying a call content according to the present invention to be described later.

The output unit 150 includes a display unit 151, an audio output module 152, an alarm unit 153, a haptic module 154, and the like, for generating output related to visual, auditory, .

The display unit 151 displays and outputs information processed by the mobile terminal 100. For example, when the mobile terminal is in the call mode, a UI (User Interface) or a GUI (Graphic User Interface) associated with a call is displayed. When the mobile terminal 100 is in the video communication mode or the photographing mode, the photographed and / or received video or UI and GUI are displayed.

The display unit 151 may be a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a three-dimensional display 3D display) or the like.

Some of these displays may be transparent or light transmissive so that they can be seen through. This may be referred to as a transparent display. A typical example of the transparent display is a transparent LCD or the like. The rear structure of the display unit 151 may also be of a light transmission type. With this structure, the user can see an object located behind the terminal body through the area occupied by the display unit 151 of the terminal body.

There may be two or more display units 151 according to the embodiment of the mobile terminal 100. [ For example, in the mobile terminal 100, a plurality of display portions may be spaced apart from one another or may be disposed integrally with each other, or may be disposed on different surfaces.

The display unit 151 is connected to a display unit 151 and a sensor for sensing a touch operation (hereinafter, referred to as 'touch sensor') having a mutual layer structure It can also be used as an input device. The touch sensor may have the form of, for example, a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in a pressure applied to a specific portion of the display unit 151 or a capacitance generated in a specific portion of the display unit 151 into an electrical input signal. The touch sensor can be configured to detect not only the position and area to be touched but also pressure and directionality at the time of touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and transmits the corresponding data to the controller 180. Thus, the control unit 180 can know which area of the display unit 151 is touched or the like.

At this time, the proximity sensor may be disposed in the inner region of the mobile terminal or in the vicinity of the touch screen, which is wrapped by the touch screen. The proximity sensor refers to a sensor that detects the presence of an object approaching a predetermined detection surface or an object existing in the vicinity of the detection surface without physical contact by using electromagnetic force or infrared rays. The proximity sensor has a longer life span than the contact sensor and its utilization is also high.

Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, and a magnetic proximity sensor.

And to detect the proximity of the pointer by the change of the electric field along the proximity of the pointer when the touch screen is electrostatic. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of explanation, the act of recognizing that the pointer is positioned on the touch screen while the pointer is not in contact with the touch screen is referred to as "proximity touch & The act of actually touching the pointer on the screen is called "contact touch. &Quot; The position where the pointer is proximately touched on the touch screen means a position where the pointer is vertically corresponding to the touch screen when the pointer is touched.

The proximity sensor detects a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, a proximity touch movement state, and the like). Information corresponding to the detected proximity touch operation and the proximity touch pattern may be output on the touch screen.

The audio output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory unit 160 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output module 152 may output an acoustic signal related to a function (e.g., a call signal reception sound, a message reception sound, etc.) performed in the mobile terminal 100. [ The audio output module 152 may include a receiver 152a (see FIG. 2a), a speaker 152b (see FIG. 2b), a buzzer, and the like.

The alarm unit 153 outputs a signal for notifying the occurrence of an event of the mobile terminal 100. Examples of events that occur in the mobile terminal include call signal reception, message reception, key signal input, touch input, and the like. The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than the video signal or the audio signal, for example, vibration. The video signal or the audio signal may also be output through the display unit 151 or the audio output module 152.

The haptic module 154 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 154 is vibration. The intensity and pattern of the vibration generated by the hit module 154 can be controlled. For example, different vibrations may be synthesized and output or sequentially output.

In addition to the vibration, the haptic module 154 may be configured to perform various functions such as an effect of stimulation by a pin arrangement vertically moving with respect to a contact skin surface, an effect of stimulation by air spraying force or suction force through a jet opening or a suction opening, A variety of tactile effects such as an effect of stimulation through contact of an electrode, an effect of stimulation by an electrostatic force, and an effect of reproducing a cold sensation using a heat absorbing or exothermic element can be generated.

The haptic module 154 can transmit the tactile effect through the direct contact and can also be implemented to feel the tactile effect through the muscular sense of the user's finger or arm. The haptic module 154 may include two or more haptic modules 154 according to the configuration of the portable terminal 100.

The memory unit 160 may store a program for the operation of the controller 180 and may store input / output data (e.g., a phone book, a message, a still image, a moving picture, etc.). The memory unit 160 may store data related to vibration and sound of various patterns outputted when a touch is input on the touch screen.

The memory unit 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory) (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM A disk, and / or an optical disk. The mobile terminal 100 may operate in association with a web storage that performs a storage function of the memory unit 160 on the Internet.

The interface unit 170 serves as a path for communication with all external devices connected to the mobile terminal 100. The interface unit 170 receives data from an external device or receives power from the external device to transfer the data to each component in the mobile terminal 100 or to transmit data in the mobile terminal 100 to an external device. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, A video input / output (I / O) port, an earphone port, and the like may be included in the interface unit 170.

The identification module is a chip for storing various information for authenticating the use right of the mobile terminal 100 and includes a user identification module (UIM), a subscriber identity module (SIM), a general user authentication module A Universal Subscriber Identity Module (USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the port.

The interface unit 170 may be a path through which power from the cradle is supplied to the mobile terminal 100 when the mobile terminal 100 is connected to an external cradle, May be a pathway to the mobile terminal. The various command signals or the power source input from the cradle may be operated as a signal for recognizing that the mobile terminal is correctly mounted on the cradle.

The control unit 180 typically controls the overall operation of the mobile terminal. For example, voice communication, data communication, video communication, and the like. The control unit 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180. [

The controller 180 may perform a pattern recognition process for recognizing handwriting input or drawing input performed on the touch screen as characters and images, respectively.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components.

The various embodiments described herein may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein may be implemented as application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays May be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing functions. In some cases, And may be implemented by the control unit 180.

According to a software implementation, embodiments such as procedures or functions may be implemented with separate software modules that perform at least one function or operation. The software code may be implemented by a software application written in a suitable programming language. Also, the software codes may be stored in the memory unit 160 and executed by the control unit 180. [

2A is a perspective view of an example of a mobile terminal or a mobile terminal according to the present invention.

The disclosed mobile terminal 100 has a bar-shaped terminal body. However, the present invention is not limited thereto, and can be applied to various structures such as a slide type, a folder type, a swing type, and a swivel type in which two or more bodies are relatively movably coupled.

The body includes a case (a casing, a housing, a cover, and the like) which forms an appearance. In this embodiment, the case may be divided into a front case 101 and a rear case 102. [ A variety of electronic components are embedded in the space formed between the front case 101 and the rear case 102. At least one intermediate case may be additionally disposed between the front case 101 and the rear case 102. [

The cases may be formed by injection molding a synthetic resin or may be formed to have a metal material such as stainless steel (STS) or titanium (Ti) or the like.

The display unit 151, the receiver 152a, the camera 121, the user input units 130/131 and 132, the microphone 122, the interface unit 170, and the like are disposed in the front body 101 .

The display unit 151 occupies most of the main surface of the front case 101. A receiver 152a and a camera 121 are disposed in an area adjacent to one end of both ends of the display unit 151 and a user input unit 131 and a microphone 122 are disposed in an area adjacent to the other end. The user input unit 132, the interface unit 170, and the like are disposed on the side surfaces of the front case 101 and the rear case 102.

The user input unit 130 is operated to receive a command for controlling the operation of the portable terminal 100 and may include a plurality of operation units 131 and 132. The operation units 131 and 132 may be collectively referred to as a manipulating portion and may be employed in any manner as long as the user operates in a tactile manner.

The contents inputted by the first or second operation units 131 and 132 may be variously set. For example, the first operation unit 131 receives commands such as start, end, and scroll, and the second operation unit 132 controls the size of the sound output from the receiver 152a, And a switch to the touch recognition mode.

FIG. 2B is a rear perspective view of the portable terminal shown in FIG. 2A.

Referring to FIG. 2B, a camera 121 'may be further mounted on the rear surface of the terminal body, that is, the rear case 102. The camera 121 'may have a photographing direction substantially opposite to the camera 121 (see FIG. 2A), and may be a camera having different pixels from the camera 121.

For example, the camera 121 may have a low pixel so that the face of the user can be photographed and transmitted to the other party in case of a video call or the like, and the camera 121 ' It is preferable to have a large number of pixels. The cameras 121 and 121 'may be installed on the terminal body such that they can be rotated or pop-up.

A flash 123 and a mirror 124 are additionally disposed adjacent to the camera 121 '. The flash 123 illuminates the subject when the subject is photographed by the camera 121 '. The mirror 124 allows the user to illuminate the user's own face or the like when the user intends to shoot himself / herself (self-photographing) using the camera 121 '.

A speaker 152b may be disposed on the rear surface of the terminal body. The speaker 152b may implement a stereo function with the receiver 152a (see Figure 2a) and may be used for the implementation of the speakerphone mode during a call.

In addition to the antenna for talking and the like, a broadcast signal reception antenna 116 may be additionally disposed on the side of the terminal body. The antenna 116, which forms part of the broadcast receiving module 111 (see FIG. 1), can be installed to be able to be drawn out from the terminal body.

A power supply unit 190 for supplying power to the portable terminal 100 is mounted on the terminal body. The power supply unit 190 may be built in the terminal body or may be detachable from the outside of the terminal body.

The rear case 102 may further include a touch pad 135 for sensing a touch. The touch pad 135 may also be of a light transmission type like the display unit 151. [ In this case, if the display unit 151 is configured to output time information on both sides, the time information can be recognized through the touch pad 135 as well. The information output on both sides may be all controlled by the touch pad 135. [ Alternatively, a display may be additionally mounted on the touch pad 135, and the touch screen may be disposed on the rear case 102 as well.

The touch pad 135 operates in correlation with the display portion 151 of the front case 101. The touch pad 135 may be disposed parallel to the rear of the display unit 151. The touch pad 135 may have a size equal to or smaller than that of the display unit 151.

3 is a front view of a portable terminal for explaining an operation state of the portable terminal according to the present invention.

Various types of time information can be displayed on the display unit 151. [ These pieces of information can be displayed in the form of letters, numbers, symbols, graphics, or icons.

At least one of the letters, numbers, symbols, graphics, or icons may be displayed in a predetermined arrangement for inputting such information, thereby being implemented as a keypad. Such a keypad may be called a so-called " soft key ".

FIG. 3 shows that a touch applied to a soft key is inputted through a front surface of a terminal body.

The display unit 151 may operate as an entire area or may be divided into a plurality of areas and operated. In the latter case, the plurality of areas can be configured to operate in association with each other.

For example, an output window 151a and an input window 151b are displayed on the upper and lower portions of the display unit 151, respectively. In the input window 151b, a soft key 151c in which a number for inputting a telephone number or the like is displayed is output. When the soft key 151c is touched, a number corresponding to the touched soft key is displayed on the output window 151a. When the first operation unit 131 is operated, call connection to the telephone number displayed on the output window 151a is attempted.

4 is a conceptual diagram for explaining the proximity depth of the proximity sensor. In FIG. 4, for example, a cross-section of a touch screen having proximity sensors capable of sensing three proximity depths is illustrated. However, it is a matter of course that a proximity sensor capable of sensing three or more proximate depths is also possible.

As shown in FIG. 4, when a pointer such as a finger of a user approaches the touch screen, the proximity sensor disposed in or near the touch screen senses the proximity sensor and outputs a proximity signal.

The proximity sensor may be configured to output different proximity signals according to a distance between the proximity-touched pointer and the touch screen (hereinafter referred to as "proximity depth").

The distance at which the proximity signal is output when the pointer approaches the touch screen is referred to as a detection distance. The proximity signal output from each proximity sensor is simply used by using a plurality of proximity sensors having different detection distances, The depth is known.

Specifically, when the pointer is completely contacted with the touch screen (d ₀ ), it is recognized as a contact touch. If the pointer is located on the touch screen at a distance less than the distance d ₁ , it is recognized as a proximity touch of the first proximity depth. If the pointer is located on the touch screen at a distance of d ₁ or more and less than distance d ₂ , it is recognized as a proximity touch of the second proximity depth. If the pointer is located on the touch screen at a distance of d ₂ or more and less than d ₃ distance, it is recognized as a proximity touch of the third proximity depth. If the pointer is located at a distance d ₃ or more on the touch screen, it is recognized that the proximity touch is released.

Accordingly, the controller 180 can recognize the proximity touch as various input signals according to the proximity distance and the proximity position of the pointer close to the touch screen, and can perform various operation control according to the various input signals. have.

Hereinafter, a mobile terminal capable of performing a call content display method and a method according to the present invention will be described in detail.

As described above with reference to FIGS. 1 to 4, the sensing unit 140 may perform various sensing operations as a premise for performing a specific function of the mobile terminal 100. In the following description of all drawings, only the case where the sensing unit 140 performs the sensing operation directly related to the present invention (i.e., the operation of sensing the proximity between the mobile terminal and the user's body) I will explain it mainly. Therefore, it should be noted that the same reference numerals and the same terminology are used, but it is first clarified that the sensing unit 140 indicated by the following figures corresponds to the sub-concept of the sensing unit 140 indicated in FIG.

Hereinafter, a general operation of the mobile terminal according to the present invention will be briefly described with reference to FIGS. 5 and 6. Hereinafter, a method of displaying a call content in a mobile terminal according to the present invention will be described with reference to FIG. Will be described in more detail with reference to FIG.

5 is a diagram illustrating a case where a proximity degree between a mobile terminal and a human body changes from a proximity state to a proximity release state during a voice call, and FIG. 6 is a diagram illustrating a case where, Conversion mode. Fig.

The present invention converts a voice signal received through the communication unit 110 (see FIG. 1) of the mobile terminal 100 into text and displays it through the display unit 151 of the mobile terminal 100, / Search < / RTI > as well as storing the contents of the call.

Several exemplary situations in which such a mobile terminal may be useful are described below.

5 (a) shows a general view when a user makes a voice call using a mobile phone.

When a user assumes a case where a user makes a voice call with a specific calling party using a mobile phone, the mobile phone is placed in a position very close to the user's body (in particular, the left or right face of the user's face). As shown in FIG. 5A, the mobile phone (particularly, the face on which the receiver 152a is located, that is, the front case 101 (see FIG. 1)) is brought into close contact with the ear It is common to make a voice call.

However, it is often difficult to clearly understand the voice of the other party in environments with a high ambient noise (for example, on a roadside, on a subway, near a construction site, etc.). As another case, although the reception of the voice signal is well performed, a voice call is sometimes interrupted by a howling effect or an echo effect caused by the problem of the mobile phone itself. Also, although it is a special case, it is also possible that a user with a hearing impairment receives a voice call connection. As described above, in a situation where a voice call is inadequate, the present invention, in which the contents of the conversation are converted into text and displayed on the display screen of the mobile phone, so that the user can immediately check and search the contents of the conversation It can be very useful.

The present invention can not necessarily be used effectively only in a situation where a voice call is inappropriate as described above. Even in a situation where a voice call is smoothly performed, the user may want to record and store the call information that is considered important during a call or that it is determined that recording is necessary during a call, immediately or later in a convenient manner. For example, the name, phone number, address, e-mail address, account number, company name, position, current status, name of specific place, telephone number, location, , A place, and the like may correspond to the information requiring the recording.

Even in the case of a terminal currently being released, a function of storing the contents of a call through a method of recording a received voice signal is supported. However, unlike the conventional voice recording method, There is a limitation that the recorded contents can be reproduced at a later time, but there is an inconvenience that the user can not visually check / search or store the contents of the conversation unless he / she records them separately on the notebook or the like.

In order to solve the above-mentioned problems, the mobile terminal according to the present invention converts voice call contents into text in accordance with a specific action taken by the user, .

Here, the specific operation taken by the user for the voice-to-text conversion may be as shown in Fig. 5, for example. That is, as shown in FIG. 5 (a), a user who is making a voice call by putting a mobile phone in close contact with the ear removes the mobile phone from the ear as shown in FIG. 5 (b) This may be the case of taking the same action as looking at the screen. 5 (a), the proximity between the mobile phone and the user's body is close to the state shown in FIG. 5 (a). By the intended operation of the user during voice communication, This is the case where the state changes to the state of the state. In this case, the mobile terminal of the present invention converts the contents of the voice call into text and displays it on the display screen.

In order to implement the above operation, the mobile terminal 100 according to the present invention includes a communication unit 110 (see FIG. 1) having a function of transmitting and receiving a call signal for voice communication, A sensing unit 140 for sensing the degree of proximity between the human body and a controller 180 (see FIG. 1) for performing voice-text conversion processing based on the speech recognition technology.

For example, the sensing unit 140 may be a non-contact type sensor capable of outputting a sensing signal corresponding to a distance between a mobile terminal and a human body, such as a proximity sensor, an infrared sensor, and an ultrasonic sensor. Also, the sensing unit 140 may be implemented as a contact sensor that senses whether or not the mobile terminal is in contact with the user's body. This can be more clearly understood from the description of FIG. 7 to be described later.

Here, the position of the sensing unit 140 is not particularly limited. For example, the sensing unit 140 may be located in an area where the display screen is located in the mobile terminal 100 (refer to 151 in FIG. 6) or a peripheral area thereof. As described above with reference to FIGS. 1 and 4, the proximity sensing is performed by placing the proximity sensor in the display screen area or its peripheral area.

Also, for example, the sensing unit 140 may be located in an area around the receiver 152a as shown in FIG. For the sake of convenience in the drawings, all of the drawings shown in Fig. 6 and below show that the sensing unit 140 is positioned around the receiver 152a.

In addition, the position of the sensing unit 140 may be variously modified within a range in which the degree of proximity between the mobile terminal and the human body can be sensed according to the intention of the present invention.

The control unit 180 determines whether the proximity degree changes from the proximity state to the proximity release state during the voice call based on the sensing result of the sensing unit 140. [ For example, the control unit 180 analyzes the sensing signal transmitted from the sensing unit 140, and if the distance between the mobile terminal and the user's body is greater than a predetermined reference distance (for example, 10 cm) It can be judged that it has changed to the released state.

If it is determined that the proximity is changed to the proximity cancel state, the control unit 180 converts the voice signal of the communication partner received through the communication unit 110 into text and displays it on the display unit 151 Reference). To this end, the controller 180 may be equipped with a voice recognition module for performing speech-to-text transformation processing. The speech-to-text conversion technology based on speech recognition is a known technology currently applied to various industrial fields, and a detailed description thereof will be omitted here.

Hereinafter, a general operation of the mobile terminal according to the present invention is described. Hereinafter, a method of displaying a call content in a mobile terminal according to the present invention will be described in detail with reference to the accompanying drawings, with reference to FIG. 7 . Hereinafter, for the sake of convenience, a non-contact type sensor such as a proximity sensor, an infrared sensor, an ultrasonic sensor, or the like is used as the sensing unit 140.

7 is a flowchart illustrating a method of displaying a call content in a mobile terminal according to an exemplary embodiment of the present invention.

When a voice call connection is established between the user and a specific call destination [S710], the control unit 180 activates the operation of the sensing unit 140 [S720].

That is, the sensing unit 140 may be turned on as the voice call connection is established. However, it is needless to say that the step of activating the operation of the sensing unit 140 may be omitted in some cases.

For example, if the proximity sensor is already operating before the voice call connection is made, for the implementation of the proximity touch as previously described in FIG. 4, it may not be necessary to activate the proximity sensor again.

In this case, the control unit 180 is not sensing for proximity touch with respect to the sensing result transmitted from the proximity sensor after the voice call connection is established, but sensing for implementing the method of displaying the conversation contents according to the present invention And the sensing result can be analyzed correspondingly.

The sensing unit 140 senses the proximity between the mobile terminal and the user's body [S730]. At this time, the proximity sensing may be continuously performed until a specific mode cancellation signal is received (S770). Although not explicitly shown in FIG. 7, it will be appreciated that sensing close to the sensing unit 140 will be terminated even when the voice call is terminated.

The sensing result by the sensing unit 140 may be immediately transmitted to the control unit 180. [

The control unit 180 determines whether the proximity between the mobile terminal and the human body has changed from the proximity state to the proximity release state based on the sensing result transmitted from the sensing unit 140 (S740).

For example, when the sensing unit 140 is implemented as a non-contact type sensor, the sensing signal transmitted from the sensing unit 140 to the control unit 180 contains information about a separation distance between the mobile terminal and the user's body. Therefore, the control unit 180 compares the proximity calculated based on the received sensing signal (i.e., the distance between the mobile terminal and the user's body) with preset reference distance information (for example, 10 cm) It is possible to distinguish whether the degree of proximity corresponds to the proximity state or the proximity release state.

Of course, the predetermined reference distance may vary depending on the type and the specification of the non-contact type sensor implementing the sensing unit 140, or may be changeable by the user.

As a result of the determination in step S740, if it is determined that the proximity is maintained in the proximity state, it corresponds to the case where the voice call is smoothly performed or there is no need to check the contents of the call with text, The control unit 180 maintains the current call mode without performing the voice-to-text conversion process. However, since the connection of the voice call is still maintained, the sensing operation of the proximity degree by the sensing unit 140 is continued.

If it is determined that the proximity degree is close to the proximity release state, the controller 180 switches the current call mode to the voice-text conversion mode. That is, the control unit 180 converts the voice signal of the calling party received through the communication unit 110 into text (S750), and displays the text through the display unit 151 (S760).

Here, the voice-to-text conversion process in step S750 may start from the time when the voice communication mode is switched from the normal mode to the voice-text conversion mode. However, this need not necessarily be the case, and based on the mode switching point, the voice-to-text conversion process is performed from the voice signal received through the communication unit 110 before a predetermined time or output through the receiver 152a a predetermined time ago Of course, it is possible to start.

For example, if the user intends to switch to the voice-to-text conversion mode because of difficulty in recognizing the voice of the calling party, even if the voice call is smooth, it is determined that there is important information that needs to be memorized during the conversation. If intended, there may be a temporal gap between the perception of the situation by the user and the point of action for mode switching. Accordingly, in order to prevent a case where important conversation contents that may exist between the time gaps are missed, it is possible to prevent the user from intentionally changing the mode switching time point by about 5 seconds or 10 seconds before It may be necessary to perform text conversion processing from the speech signal.

An example of a method of displaying text in step S760 is shown in FIG. 8A to 8D are diagrams illustrating, in chronological order, a case in which conversation contents are displayed as text on a display of the mobile terminal 100 in the voice-to-text conversion mode.

Referring to FIG. 8, all the conversation contents while the voice-to-text conversion mode is maintained are converted into texts in a time order (that is, a reception order of voice signals) and sequentially displayed on the display unit 151.

At this time, it can be confirmed that a time stamp 20 is added (inserted) in the middle of the entire text displayed on the display unit 151. This can provide convenience to a user who wishes to carry out a process of confirming / retrieving call contents and storing and editing the call contents later.

For example, the control unit 180 adds the time staff 20 every time the voice signal of the calling party is not received for a predetermined time interval or more (that is, when the other party stops the voice) The text for the received voice signal before and after the voice signal can be displayed on the display unit 151 separately. Alternatively, even when the other party does not stop the voice, it may be displayed in a suitable time unit or a sentence unit for convenience of confirmation / search of the contents of the call.

Also, although not clearly shown in FIG. 7, in the voice-text conversion mode, the call mode in the mobile terminal 100 may be switched to a speakerphone mode. This is because the user may need to maintain a smooth bidirectional communication with the calling party even in the proximity release state as shown in FIG. 5 (b).

For this, the control unit 180 may switch the call mode of the mobile terminal 100 to the speakerphone mode when the proximity degree is changed to the proximity cancel state according to the determination result of step S740.

However, since the user may use the voice-to-text conversion mode only for a short period of time (for example, only a specific important call content is checked with text) and then switch back to the original mode to make a voice call The above-described real-time two-way voice communication is not required. Therefore, it goes without saying that there may not be a step of switching to the soupcraft mode as described above or depending on the setting.

In addition, the operation of the speakerphone mode in the present invention may be different from the operation in the general speakerphone mode in the currently launched mobile terminal. In the case of the general speakerphone mode, the voice of the user is input to the microphone 122 (see FIG. 2B) and the voice of the calling party is output to the speaker 152b (see FIG. 2B). However, in the present invention, since the voice of the calling party is converted into text and displayed through the display unit 151, it is not always necessary to output the voice of the calling party through the speaker 152b have.

In addition, in the case where the user is forcibly switched to the general speakerphone mode in the proximity canceled state, there is a possibility that the contents of private conversation between the user of the mobile terminal 100 and the other party of the mobile terminal 100 are disclosed / released to the outside regardless of the intention of the user. Accordingly, in order to prevent the above-described cases, the speakerphone mode of the present invention may operate differently from the general speakerphone mode. For example, in the proximity release state, the voice of the called party may be automatically silenced. Or the voice of the called party according to the selection of the user (for example, pressing a specific key button, touching a soft key provided in a specific area of the touch screen, recognizing a predetermined operation such as shaking the mobile terminal) It may be silenced.

To this end, when the mobile terminal 100 is in the proximity canceling state as described above, the control unit 180 determines whether or not the voice of the called party moves according to a preset control flow or according to a command received from the user It is possible to perform a silent process so as not to be output to the outside via the audio output module 152 (that is, the receiver 152a and / or the speaker 152b) of the terminal 100. [

The speakerphone mode described above can be released together when the voice-to-text conversion mode is released. Alternatively, regardless of whether the voice-to-text conversion mode is canceled, the user may intend to continue the voice call in the speakerphone mode, so that whether to continue the speakerphone mode or to switch to the normal call mode, It is needless to say that they may be selected.

Also, the voice-text conversion mode in the present invention does not necessarily exclude voice output through the receiver 152a. That is, when the mobile terminal 100 enters the voice-to-text conversion mode, it is possible not only to output the voice but also to display the text related to the conversation contents. However, regardless of whether the text is displayed or not, have.

In the voice-to-text conversion mode, when the audio output through the receiver 152a is set to be maintained as it is, the sensing unit 140 described above may be implemented by a touch sensor. The reason for this is as follows.

The contact sensor can sense only the presence or absence of contact between the mobile terminal and the human body of the user. In this case, the controller 180 can recognize whether or not the contact between the mobile terminal 100 and the user's body has occurred. This means that the control unit 180 may erroneously perceive an unintended operation of the user (for example, a momentary detachment of the mobile terminal that has been closely attached to the ear) as an instruction to enter the voice-to-text conversion mode. However, even if such a situation occurs, when the voice output is set to be maintained even in the voice-to-text conversion mode, there is no inconvenience in voice call when viewed from the user's point of view.

The voice-to-text conversion mode as described above may be maintained until a mode cancellation signal is received (S770). The mode cancellation signal may be generated by a command input from the user for the purpose of releasing (terminating) the voice-text conversion mode. That is, the mode cancellation signal may be generated when a user inputs a command corresponding to a predetermined mode cancellation condition to the mobile terminal, in order to release the voice-to-text conversion mode. This will be described with reference to FIG. It is assumed that the display unit 151 is implemented as a touch screen in Fig. 9 (the same applies to Figs. 10 to 14 described later).

9 is a diagram for explaining a method of canceling a voice-text conversion mode by a user's touch operation according to an embodiment of the present invention. 9, when the user touches a position of the display unit 151 in a state in which the voice-text conversion mode is in progress (see FIG. 9A, which is the same as FIG. 8D) 9B), it can be confirmed that the voice-text conversion mode is released (see FIG. 9C).

9, when there is a touch operation on the touch screen in a state where the voice-to-text conversion mode is in progress, when the controller 180 receives a mode release command (a mode release signal is received) As shown in FIG. However, it is apparent that there are various methods for generating the mode release signal according to the release condition preset in the mobile terminal. For example, a user may select a key button provided specifically for releasing the mode or a soft key provided in one area of the touch screen.

When the mode-release signal is received, the voice-text conversion mode is released, and the entire text of the conversation contents before the release can be displayed on the display unit 151 in a state that can be confirmed and retrieved by the user. For example, when the voice-to-text conversion mode is released, a scroll bar 151d is generated on the display unit 151 as shown in FIG. 9 (c), thereby facilitating identification and retrieval of the entire text . At this time, the confirmation and retrieval of the text can be performed by the operation of the up / down key, the dragging or scrolling operation on the touch screen, and the like.

In the embodiment of the present invention shown in FIG. 7, not only the voice-text conversion mode but also the sensing operation by the sensing unit 140 may be terminated as the mode release signal is received through the step S770 However, it is needless to say that the method of displaying a call content according to the present invention is not limited thereto. For example, when the mode-release signal is received, the voice-text conversion mode currently being performed is canceled, but the sensing operation by the sensing unit 140 is maintained as it is so that the voice- .

Although it is not clearly shown in FIG. 7, it is obvious that the voice-text conversion mode can be automatically released according to the end of a voice call.

Unlike the above, if the mode-release signal is not received, the control unit 180 keeps the voice-text conversion mode. However, the controller 180 can continuously monitor whether the proximity release state is maintained even if the mode release signal is not received (S780). The monitoring of whether or not to maintain the proximity release state can be performed by analyzing the sensing signal transmitted through the sensing unit 140 as described above.

If it is determined that the proximity between the mobile terminal and the human body has changed to the close state as a result of the monitoring, the control unit 180 can release the current voice-to-text conversion mode (S790). For example, when the user who has checked the text displayed on the display unit 151 in the proximity cancel state as shown in FIG. 5B again makes a voice call in the normal mode in the proximity state as shown in FIG. 5 (a) This is the case if you want to. In this case, it may be returned to the step S740 of FIG.

Up to this point, the process of displaying the text related to the call content on the display screen in the voice-to-text conversion mode has been described with reference to the call content display method according to the present invention. Hereinafter, with reference to FIGS. 10 to 14, it is assumed that a part or all of the text related to the call content displayed on the display screen is selected, and the stored, deleted, or edited text is selected, .

FIG. 10 is a view for explaining a text selection method according to a touch operation of a user according to an embodiment of the present invention. FIG. 11 is a flowchart illustrating a text selection method according to another embodiment of the present invention, And FIG. 12 is a diagram for explaining a text selection method by a user's touch operation according to another embodiment of the present invention.

Here, Figs. 10 to 12 illustrate various cases in which a text is selected according to a user's touch operation in a state in which the voice-text conversion mode is released (see Fig. 9 (c)).

10, the user can select a continuous touch operation of dragging a portion of the text included in the text displayed on the display unit 151. In this case, 10 exemplifies a case where a telephone number ("01087654321" in this example) relating to a specific person included in the text is selected according to a continuous touch operation by the user (see FIGS. 10A and 10B) ).

If the text is selected as described above, the control unit 180 displays a menu list listing functions that can be executed in association with the selected text or a list of previously-specified functions, such as popping up a new window 151e, 151) (see Fig. 10 (c)). This allows the user to quickly and easily access the executable functions or applications with respect to the selected text.

Executable functions associated with the selected text may include, for example, text editing such as text storage and deletion, message transmission including SMS or MMS, email transmission, phonebook storage, and the like.

In addition, the user may wish to connect a call to a third party or telephone number mentioned during the call, even while making a voice call with a particular call party. In the case of many mobile terminals currently being released, a recipient swap function that allows a call with a third party to be put in a call waiting state while a call with the specific party is in a state of being in a call with the specific party, And a conference call function in which three parties are brought into the call to talk with each other. Accordingly, the executable functions related to the selected text may include a recipient swap function and a conference call function (see FIG. 10 (c)). Of course, it is obvious that various other functions may be included.

At this time, the pop-up window 151e related to the menu list can be displayed by the following method. For example, when a user who has completed selection of a text selects a specific key button (e.g., a "menu" button or the like as a soft key displayed at the bottom of the display unit 151), a pop-up window 151e have. Or when the user completes a touch for text selection and takes a finger off the touch screen, a pop-up window 151e related to the menu list may be displayed. It will be appreciated by those of ordinary skill in the art that there may be other ways of doing so without further explanation.

In the case of FIG. 10 described above, text selection is performed by continuous touch on the touch screen of the user. In contrast, according to the text selection method according to another embodiment shown in FIG. 11, text selection is performed by an intermittent touch on the user's touch screen.

For example, as shown in FIG. 11, when the user touches the one position 11 and the other position 12 on the touch screen one by one intermittently (see FIGS. 11 (a) and 11 (b) The text having the position 11 as the starting point and the other position 12 as the end point can be selected (see Fig. 11 (c)).

12, when the user temporarily touches the time stamp 20 displayed on the display unit 151 on the touch screen once (see Fig. 12 (a)), the time stamp 20 All of the segmented texts can be selected (see Fig. 12 (b)).

11 and 12, it is needless to say that the menu list related to the executable functions relating to the selected text can be displayed on the display unit 151 in the form of a pop-up window 151e or the like (FIG. 11 (d) And Fig. 12 (c)).

Although the case where a portion of the text is selected in the state where the voice-to-text conversion mode is released has been described above, the selection of the text may be performed for the entire text displayed on the display unit 151, Of course, can be performed even when the mode is in progress.

FIG. 13 is a diagram for explaining a method for storing and deleting all the contents of a call according to an embodiment of the present invention. FIG. 14 is a view for explaining a method for storing and deleting all contents of a call according to another embodiment of the present invention. to be. Here, FIG. 13 and FIG. 14 assume the case where the entire text is stored or deleted before the voice-text mode is released.

13, when the user touches the "save" button among the soft keys provided at the lower end of the display unit 151 in the state before the voice-text conversion mode is released (see , All the texts displayed on the display unit 151 up to that point are stored (see Fig. 13 (b)).

In FIG. 13, only the case where the entire text is stored by the terminal operation by the user in a state in which the voice-to-text conversion mode is not released is shown as an example, but it goes without saying that various other applications may exist. For example, when the user touches the "delete" button among the soft keys provided at the lower part of the display unit 151, the entire text may be deleted. In the case of touching the " (151e) or the like may be displayed. It is needless to say that the description can be applied to the method of selecting the text in the voice-to-text conversion mode release state described above with reference to FIGS.

In FIG. 13, only the case where the entire text is selected and stored or deleted while the speech-to-text conversion mode is currently in progress is shown, but it is obvious that a part of the text can also be selected.

For example, with the voice-to-text conversion mode remaining intact, some selection of text may also be performed in a manner similar to that shown in Figures 10-12. In this case, however, the touch operation of the user for selecting a part of the text needs to be set so as not to be recognized as the mode release signal described above. When set in this way, the user can perform certain functions associated with some selected texts (e.g., sending messages to a specific third party referred to during a call, swapping recipients, etc.) while still maintaining the voice- have.

Fig. 14 shows a case where the voice-to-text conversion mode is not released by the mode-release signal but is canceled by the end of the call. As can be seen from the figure, when the voice-text conversion mode is automatically released by the end of the call, a pop-up window for informing the user whether to store or delete the text related to the call content displayed up to that point is displayed on the display unit 151 (See Fig. 14 (b)).

Of course, unlike the above, in the case of automatic release of the voice-to-text conversion mode due to termination of the call, the same screen as in the case of release by the mode release signal may be displayed as in (d) It is self-evident.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the following claims And changes may be made without departing from the spirit and scope of the invention.

1 is a block diagram of a mobile terminal according to an embodiment of the present invention.

FIG. 2A is a front perspective view of a portable terminal according to an embodiment of the present invention; FIG.

FIG. 2B is a rear perspective view of a portable terminal according to an embodiment of the present invention; FIG.

3 is a front view of a portable terminal for explaining an operation state of the portable terminal according to the present invention.

4 is a conceptual diagram for explaining the proximity depth of the proximity sensor;

5 is a diagram illustrating a case where a proximity degree between a mobile terminal and a human body changes from a close state to a proximity cancel state during a voice call;

FIG. 6 illustrates that the mobile terminal operates in a voice-to-text conversion mode as it changes to the proximity release state; FIG.

FIG. 7 is a flowchart illustrating a method of displaying call content in a mobile terminal according to an embodiment of the present invention; FIG.

FIG. 8 is a diagram illustrating in chronological order a case where, in a voice-to-text conversion mode, a call content is displayed as text on a display of a mobile terminal;

9 is a diagram for explaining a method of releasing a voice-to-text conversion mode by a user's touch operation according to an embodiment of the present invention.

10 is a diagram for explaining a text selection method by a user's touch operation according to an embodiment of the present invention.

11 is a diagram for explaining a text selection method by a user's touch operation according to another embodiment of the present invention.

12 is a diagram for explaining a text selection method by a user's touch operation according to another embodiment of the present invention.

13 is a diagram for explaining a method for storing and deleting all contents of a call according to an embodiment of the present invention;

FIG. 14 is a diagram for explaining a method for storing and deleting a whole contents of a call according to another embodiment of the present invention; FIG.

Description of the Related Art

100: mobile terminal 110: wireless communication unit

140: sensing unit 151: display unit

152a: Receiver 180: Control section

Claims (16)

A communication unit; A sensing unit sensing a proximity between a mobile terminal and a human body; And When the proximity of the voice signal through the communication unit is determined to have changed from the proximity state to the proximity cancel state based on the sensing result of the sensing unit, the voice signal received through the communication unit is converted into text, Display on the display; And a control unit for executing an executable function related to the selected text when the text is selected while maintaining the voice call and converting the voice signal to text; Wherein the executable function in relation to the selected text comprises at least one of text editing, message or email transmission comprising at least one of text storage and text deletion, phonebook storage, recipient swap, conference call . delete The method according to claim 1, When the voice signal is received at intervals of a predetermined time or more, Wherein the control unit distinguishes between a text corresponding to a previously received voice signal and a text corresponding to a voice signal received later on the basis of the reception interval, and displays the separated text on the display. The method according to claim 1, Wherein part or all of the text displayed on the display is selectable. 5. The method of claim 4, Wherein the control unit displays a menu list related to an executable function related to the selected text on the display when a part or all of the text is selected. delete The method according to claim 1, Wherein the control unit silences the voice signal so that the voice signal is not output to the outside through the voice output module of the mobile terminal in the proximity release state. The method according to claim 1, Wherein the sensing unit is provided at a position of at least one of an area where the display screen is located in the mobile terminal, a peripheral area of the area where the display screen is located, and a receiver peripheral area. The method according to claim 1, Wherein the sensing unit includes at least one of an infrared sensor, an ultrasonic sensor, a proximity sensor, and a contact sensor. Sensing a proximity between a mobile terminal and a human body; Converting the voice signal received through the communication unit of the mobile terminal into text based on the sensing result when it is determined that the proximity degree changes from the proximity state to the proximity cancel state during the voice call; Displaying the text on a display; And Executing an executable function associated with the selected text when the text is selected while maintaining the voice call and converting the voice signal to text; Wherein the executable function in relation to the selected text comprises at least one of text editing, message or email transmission comprising at least one of text storage and text deletion, phonebook storage, recipient swap, conference call And displaying the contents of the call on the mobile terminal. 11. The method of claim 10, Wherein the displaying the text on the display comprises: And displaying the text of the previously received voice signal and the text of the subsequently received voice signal on the display on the basis of the reception interval when the voice signal is received at intervals of a predetermined time or longer A method of displaying call content on a mobile terminal. 11. The method of claim 10, Selecting some or all of the text displayed on the display; And Further comprising the step of displaying on the display a menu list of executable functions related to the selected text. delete 11. The method of claim 10, Further comprising switching the call mode of the mobile terminal to a speakerphone mode as the mobile terminal moves to the proximity release state. 11. The method of claim 10, Further comprising the step of silencing the voice signal so that the voice signal is not output to the outside via the voice output module of the mobile terminal when the voice signal is changed to the proximity cancel state. 11. The method of claim 10, Receiving an instruction corresponding to a release condition preset in the mobile terminal; And And canceling the voice-to-text conversion according to the received command.

Images