KR101449751B1 - Mobile Equipment Having Function Of Providing Touch Feedback During Video Call And Method For Providing Touch Feedback During Video Call - Google Patents

Abstract

A method for providing touch feedback during a video call and a mobile terminal having a touch feedback providing function during a video call allowing an operation corresponding to a touch input of a user during a video call to be executed in a mobile terminal of a video communication counterpart. The mobile terminal comprises: a camera section for photographing a subject during a video call and providing a photographed image; a display section for displaying an image provided from a mobile terminal of a video call partner during a video call in a first area, A control unit for performing control to transmit feedback information corresponding to a touch input to a mobile terminal of a video call partner corresponding to an event signal provided from a touch screen and a touch screen for providing an event signal corresponding to a touch input provided in a first area, . Accordingly, it is possible to provide feedback corresponding to a touch input to a mobile terminal of a video call recipient only by providing a touch input to a video of the other party during a video call, and by using the user's intention or emotion during a video call, It can be delivered to the calling party.

Mobile terminal, touch screen, feedback, video phone, video call

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a mobile terminal having a touch feedback providing function during a video call and a method of providing touch feedback during a video call.

The present invention relates to a mobile terminal having a video call function, and more particularly, to a mobile terminal having a function of providing touch feedback among video calls that can be applied to a mobile terminal having a touch screen, .

2. Description of the Related Art [0002] In recent years, mobile terminals have various functions such as digital broadcast reception, portable internet, and multimedia file playback in addition to the conventional mobile communication functions due to rapid development of processor technology, increase of memory capacity, and development of multimedia coding technology.

As the mobile terminal is provided with various functions as described above, the constituent devices of the mobile terminal are changed accordingly. That is, recently, in order to overcome the limitation of the size of the mobile terminal and effectively utilize various application programs, a touch screen having a combination of a keypad and a display device is employed in a mobile terminal.

The touch screen is interactive and intuitive to use the mobile terminal by touching a button or a graphic object displayed on the display device with a finger or a pen. Therefore, the input operation is simplified and the meaning of the input screen is easily recognized, It is possible to employ a display device having a larger visible screen in the mobile terminal because the installation space of the keypad is reduced.

In addition, in recent years, as the generation of the mobile communication system evolves, the video call service is commercialized, and it becomes possible to communicate while transmitting and receiving video and audio in real time. The video call service is based on 3G-324M of WCDMA, H.323 or SIP protocol of CDMA, and supplementary services such as alternative video transmission or chatting during video communication are provided.

However, the video communication service that is currently being commercialized is merely providing additional services such as a video call through video and audio or a chat during a video call, so it is necessary to develop an additional service capable of satisfying various demands of the user.

Accordingly, it is a first object of the present invention to provide a mobile terminal having a function of providing a touch feedback during a video call so that an operation corresponding to a touch input of a user during a video call can be executed in the mobile terminal of the video communication counterpart.

A second object of the present invention is to provide a method of providing touch feedback during a video call in which an operation corresponding to a touch input of a user during a video call can be performed in a mobile terminal of a video call counterpart.

According to another aspect of the present invention, there is provided a mobile terminal having a function of providing touch feedback during a video call, including a camera unit for photographing a subject during a video call and providing a photographed image, A touch screen for displaying an image provided from a mobile terminal of a video communication counterpart in a video communication in a first area, displaying the photographed image in a second area, and providing an event signal corresponding to a touch input provided in the first area, And a controller for transmitting feedback information corresponding to the touch input corresponding to the event signal provided from the touch screen to the mobile terminal of the video call counterpart. The control unit may control an operation of the mobile terminal in response to feedback information transmitted from a mobile terminal of the video call partner during a video call. The control unit may set a chat channel with the mobile terminal of the video call destination and control the feedback information corresponding to the touch input to be transmitted through the set chat channel. The mobile terminal having the touch feedback providing function during the video call may further include a storage unit for storing feedback information corresponding to the touch input. The control unit reads the feedback information corresponding to the event signal from the touch screen and reads the feedback information from the storage unit, converts the read feedback information into a chat message transmission standard, To be transmitted.

According to another aspect of the present invention, there is provided a method of providing touch feedback during a video call, the method comprising: displaying an image provided from a mobile terminal of a video call partner on a first area of a touch screen The method comprising the steps of: providing an event signal corresponding to a predetermined touch input in the first area; and transmitting feedback information corresponding to the touch input to the mobile terminal of the video call destination corresponding to the provided event signal . The method of providing touch feedback during a video call may further include receiving feedback information provided from a mobile terminal of the video call partner and controlling an operation of the mobile terminal according to the received feedback information. Wherein the step of transmitting the feedback information corresponding to the touch input to the mobile terminal of the video call counterpart corresponding to the provided event signal comprises the steps of reading the feedback information corresponding to the touch input from the storage unit, Converting the information into a chat message transmission standard, and transmitting the feedback information converted into the chat message transmission standard to the mobile terminal of the video call partner through a chat channel.

According to the mobile terminal having the function of providing the touch feedback among the video calls and the method of providing the touch feedback during the video call, when the user provides a predetermined touch input to the area where the video of the video caller is displayed, A feedback message corresponding to the input is transmitted to the mobile terminal of the video call destination and executed. Also, when a feedback message is provided from a mobile terminal of a video call partner during a video call, the mobile terminal performs an operation corresponding to the provided feedback message.

Accordingly, it is possible to provide feedback corresponding to the touch input to the mobile terminal of the other party of the video call only by providing the touch input to the video of the other party during the video call, and to use the user's intention or emotion during the video call in various ways The video call can be delivered to the other party.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail.

It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In the present application, the terms "comprises" or "having" and the like are used to specify that there is a feature, a number, a step, an operation, an element, a component or a combination thereof described in the specification, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the relevant art and are to be interpreted in an ideal or overly formal sense unless explicitly defined in the present application Do not.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In order to facilitate a thorough understanding of the present invention, the same reference numerals are used for the same constituent elements in the drawings and redundant explanations for the same constituent elements are omitted.

Hereinafter, in the embodiment of the present invention, when a user provides a predetermined touch input to a touch screen of a mobile terminal, the touch feedback refers to a predetermined function or operation corresponding to a provided touch input, And is executed in the mobile terminal of the other party of the call.

1 is a block diagram illustrating a configuration of a mobile terminal having a touch feedback providing function during a video call according to an exemplary embodiment of the present invention.

1, a mobile terminal according to an exemplary embodiment of the present invention includes a camera unit 110, a touch screen 120, a controller 130, a wireless transceiver 140, a microphone 150, a speaker 160, A storage unit 170, and a vibrator 180. [

Since the camera unit 110 corresponds to a known technology, although not shown in detail, when a video call function is activated, an optical signal of a subject incident through a lens is controlled by the controller 130 as an electrical signal through an image sensor Converts the digital image signal into an analog image signal through an analog-to-digital converter (ADC) provided therein, and provides the converted digital image signal to the controller 130.

In addition, the camera unit 110 may include an image processing processor (not shown) to convert image processing, compression, or image size for image quality into a preset size, and then provide the converted image to the controller 130.

The touch screen 120 may include a display device such as a liquid crystal display (LCD) or an organic light emitting diode (OLED) A user interface such as a menu, a key, and the like of the mobile terminal and an operation state of the mobile terminal, and provides a signal to the controller 130 as an event corresponding to the touch input input by the user.

The touch screen 120 may be a contact type capacitance type, an infrared light sensing type, a surface ultrasonic wave type, a piezoelectric type or a resistive type. The touch screen 120 may transmit a signal corresponding to a touch input inputted by a user to the controller 130 ).

For example, when a touch input is generated by the user when the resistive touch panel 120 is used, the touch screen 120 senses the potential difference of the touched position and then, based on the sensed potential difference, The X and Y coordinates of the position can be provided to the control unit 130 as an event signal.

The touch screen 120 divides the display area into a first area (refer to 310 in FIG. 3) and a second area (refer to 320 in FIG. 3) based on the control of the controller 130, 1, a video of the video communication partner transmitted from the mobile communication terminal of the video communication partner, or a video of the user photographed by the user in the second area, .

In addition, the touch screen 120 provides a signal corresponding to the touched position to the control unit 130 when a user provides a predetermined touch input to the first area during a video call. For example, if a user draws a circle through the touch of the first area during a video call, the touch screen 120 provides coordinate values on the first area corresponding to the circle traced by the user to the control unit 130 do.

When an event signal for instructing the execution of a video call is provided from the touch screen 120, the control unit 130 performs control for setting a video call call to a phone number of a video call destination specified by the user based on the event signal, do. In addition, the controller 130 controls the chat channel setting with the video call party based on the H.245 protocol in order to provide touch feedback during the video call, thereby setting a chat channel.

After that, the controller 130 displays the video call user interface screen divided into the first area and the second area on the touch screen 120, displays the image provided from the mobile terminal of the video call partner on the first area , And displays the user's image provided from the camera unit 110 in the second area.

In addition, when the user provides a predetermined touch input to the first area of the touch screen, that is, the area where the video of the video communication party is displayed, the controller 130 adjusts the corresponding feedback information to conform to the chat message standard If the feedback message is received from the mobile terminal of the video communication counterpart, the feedback information is extracted from the received feedback message and the mobile terminal is controlled accordingly.

The controller 130 may include a touch feedback processing module 131 to perform the touch feedback providing function.

Specifically, when the user provides a predetermined touch input to the first area of the touch screen during the video call, the touch feedback processing module 131 receives the coordinate value corresponding to the touch input of the user from the touch screen, The control unit 170 reads the feedback information corresponding to the determined touch content from the storage unit 170. The control unit 170 determines whether the touch information is the touch information.

For example, when the user draws a triangle through the touch on the first area, the touch feedback processing module 131 receives coordinate values corresponding to the triangle from the touch screen 120, and based on the provided coordinate values Determines that the content touched in the first area is a triangle, and reads out the feedback information (for example, effect sound generation) corresponding to the triangle to the storage unit 170.

Then, the touch feedback processing module 131 converts the feedback information read from the storage unit 170 into a chat message transmission standard, and transmits the converted feedback information to the mobile terminal of the video communication partner.

For example, the touch feedback processing module 131 may convert a feedback message into a UII (User Input Indication) message format defined in the H.245 protocol, and then transmit the feedback message to a mobile terminal of a video call partner through a chat channel.

In addition, when the feedback message is received from the mobile terminal of the video call recipient, the touch feedback processing module 131 extracts feedback information from the received message and controls the mobile terminal based on the extracted feedback information, And executes the touch feedback function transmitted from the terminal.

For example, when the feedback information extracted from the feedback message indicates an output for a predetermined sound source, the touch feedback processing module 131 reads the corresponding sound source among the sound sources stored in the storage unit 170, (160).

The type of feedback corresponding to the content touched by the user in the mobile terminal according to the embodiment of the present invention may be set by a user before the video call or may be set to a menu form at the time of manufacturing the mobile terminal.

For example, when the user draws a circle in the first area, the mobile terminal is vibrated. In the case of drawing a triangle, a predetermined effect sound is outputted. In the case of drawing a rectangle, the screen may be set to blink.

The video CODEC 133 encodes the digital video signal provided from the camera unit 110 into a preset video call format and provides the encoded video signal to the wireless transceiver 140 so that the video codec 133 is transmitted to the mobile terminal of the video call partner.

In addition, the image codec 133 decodes the image of the video communication partner provided from the mobile terminal of the video communication partner through a wireless transmitting / receiving unit 140 into a predetermined format, and then transmits the decoded video communication partner's image to the touch screen 120. [ So that the first area of the video call user interface screen displayed on the touch screen 120 is displayed.

Here, the image codec 133 can encode and decode an image using codecs such as H.261, H.263, H.264, and MPEG-4, for example.

The voice CODEC 135 receives the user's voice provided from the microphone 150, converts the received voice into a digital signal, encodes the converted digital signal to conform to a voice call or video call standard, and provides the converted digital signal to the wireless transmission / So that the voice or video is transmitted to the mobile terminal of the other party.

Also, the voice CODEC 135 decodes the voice of the other party from the mobile terminal of the voice or video communication partner through the wireless transceiver 140, converts it into an analog signal, and provides it to the speaker 160.

The voice codec 135 may be implemented using a codec such as QCELP (QualComm Code Excited Linear Predictive Coding), EVRC (Enhanced Variable Rate Codec), G.711, G.723, G.723.1, G.728, Lt; RTI ID = 0.0 > and / or < / RTI >

The control unit 130 may perform overall control necessary for the operation of the mobile terminal in addition to the control related to providing the touch feedback during the video call as described above. The controller 130 may be, for example, a mobile station modem (MSM) of a mobile terminal.

The wireless transceiver 140 converts a radio frequency (RF) signal received through the antenna ANT to a baseband signal and provides the baseband signal to the controller 130. The baseband signal Into a radio frequency signal and outputs it through the antenna ANT.

In particular, when the video call is performed, the wireless transceiver 140 transmits the video call signal of the user to the mobile terminal of the video call partner based on the control of the controller 130, Signal. In addition, the wireless transceiver 140 transmits a feedback message to the mobile terminal of the video communication partner through the chat channel based on the control of the controller 130 during the video communication, and transmits the feedback message transmitted from the mobile terminal of the video communication partner .

The microphone 150 receives a voice of a user during a video call, converts the voice into an electric signal, and provides the voice to the controller 130. The speaker 160 receives the audio signal of the video communication party decoded from the audio codec 135 and outputs it as a signal of an audio frequency band.

The storage unit 170 may be a nonvolatile memory such as a flash memory, an electrically erasable programmable read only memory (EEPROM), or the like. The storage unit 170 may store a system program (for example, an operating system) and / Or other applications may be stored.

In addition, the storage unit 170 stores feedback information corresponding to a touch input of a user during a video call.

The vibrator 180 vibrates based on the control of the control unit 130 during the video call to perform the touch feedback function. In addition, the vibrator 180 may vibrate based on the control of the controller 130 when a telephone or a message is received in the mobile terminal.

FIG. 2 is a flowchart illustrating a method of providing touch feedback during a video call according to an exemplary embodiment of the present invention. When a user provides a predetermined touch input to an image transmitted from a mobile terminal of a video communication counterpart during a video call, And the case where the touch feedback is executed in the mobile terminal of the video call destination.

Referring to FIG. 2, when power is applied to the mobile terminal, the controller 130 initializes the mobile terminal and determines whether an event signal for instructing a video call is generated (step 201).

Here, the controller 130 receives coordinate values corresponding to the user's touch input from the touch screen 120, checks the contents of the user interface indicated by the received coordinate values, and determines whether an event signal for instructing a video call is generated can do.

For example, when the user touches a number displayed on the touch screen 120 and inputs a phone number of a video call destination, and then touches a button indicating execution of a video call displayed on the touch screen 120, the touch screen 120 The controller 130 provides the coordinate value of the touched position to the controller 130. When the control unit 130 determines that the graphic object indicated by the coordinate value provided from the touch screen 120 is a video call execution button, .

If it is determined in step 201 that an event signal for instructing a video call is generated, the controller 130 performs control for connecting a video call to the mobile terminal of the video communication partner designated by the user, thereby setting up a video call.

In order to perform touch feedback control during a video call according to an exemplary embodiment of the present invention, the control unit 130 sets a video call call and simultaneously sets a chat channel with a mobile terminal of the video call partner based on the H.245 protocol .

The control unit 130 divides the video call user interface screen displayed on the touch screen 120 into a first area and a second area when a video call call is established with the mobile terminal of the video calling party, (I.e., a video call recipient) provided from the camera unit 110 is displayed in the second area (step 203).

Then, the control unit 130 determines whether a touch input is generated in the first area of the touch screen 120 in which the image provided from the mobile terminal of the video communication counterpart is displayed (step 205) If it is determined that the input is generated, the feedback information corresponding to the generated touch input is read from the storage unit 170 (step 207).

Here, the controller 130 receives coordinate values corresponding to the touch input generated in the first region from the touch screen 120, and stores feedback information corresponding to the user-touched contents based on the received coordinate values (170). The feedback information indicates a function that the user has to execute or operate in the mobile terminal of the video communication counterpart in accordance with the contents touched in the first area of the touch screen 120. [

For example, when the user touches the first area and draws a circle, the controller 130 receives coordinate values corresponding to the touched area from the touch screen 120, and based on the received coordinate values, And reads out the feedback information corresponding to the circle (for example, the vibration execution) from the storage unit 170. [

Then, the controller 130 converts the feedback message read from the storage unit 170 into a chat message transmission standard, and transmits the converted message to the mobile terminal of the video communication partner (step 209). Here, the control unit 130 may convert the feedback message into a UII (User Input Indication) message format defined in the H.245 protocol, and then transmit the feedback message to the mobile terminal of the video call partner through the established chat channel.

The controller 130 determines whether a feedback message has been received from the mobile terminal of the video communication partner (step 211). If it is determined that the feedback message has been received, the controller 130 extracts feedback information from the received feedback message (step 213) And controls the mobile terminal based on the extracted feedback information to execute the touch feedback function transmitted from the mobile terminal of the video call partner (step 215).

For example, when the feedback information extracted from the feedback message indicates an output for a predetermined sound source, the control unit 130 reads the corresponding sound source from the sound sources stored in the storage unit 170, and transmits the sound source to the speaker 160 Lt; / RTI >

If it is determined in step 205 that a predetermined touch input is not generated in the first area, the controller 130 proceeds directly to step 211 without performing steps 207 and 209.

Then, the control unit 130 determines whether an event for instructing termination of the video call has been generated. If it is determined that the event for instructing termination of the video call is not generated, the control unit 130 returns to step 203, executes steps 203 to 215, The process of providing the touch feedback during the video call according to the embodiment of the present invention is terminated.

FIG. 3 illustrates a user interface screen displayed on a touch screen in a process of providing touch feedback during a video call according to an exemplary embodiment of the present invention.

Referring to FIG. 3, when a video call is started, a video call user interface screen 300 is displayed on the touch screen 120 as shown in FIG. 3 (a) An image transmitted from the mobile terminal of the video communication partner is displayed in the area 310 and an image taken by the camera unit 110 provided in the mobile terminal of the user is displayed in the second area 320. [

As shown in FIG. 3 (a), during the video call while viewing the video call user interface screen 300, as shown in FIG. 3 (b) Providing a predetermined touch input (e.g., a circle) 311 through a touch to the area 310 provides feedback information (e.g., vibration execution) corresponding to the touch input 311 of the user as a message The mobile terminal of the video call recipient, who is provided to the mobile terminal of the other party of the call, performs an operation corresponding to the received feedback message (for example, vibration execution).

It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined in the appended claims. It will be possible.

1 is a block diagram illustrating a configuration of a mobile terminal having a touch feedback providing function during a video call according to an exemplary embodiment of the present invention.

2 is a flowchart illustrating a method of providing touch feedback in a video call according to an exemplary embodiment of the present invention.

FIG. 3 illustrates a user interface screen displayed on a touch screen in a process of providing touch feedback during a video call according to an exemplary embodiment of the present invention.

Description of the Related Art

110: camera unit 120: touch screen

130: control unit 131: touch feedback processing module

133: Video codec 135: Audio codec

140: wireless transmitting / receiving unit 180:

Claims (8)

A mobile terminal having a video call function, A camera unit for acquiring a second image photographed by a user of the terminal; A touch for displaying a first image provided from a mobile terminal of a video communication counterpart during a video call in a first area, displaying the acquired second image in a second area, and obtaining a coordinate value of a touch input received from the first area, screen; And If the coordinate value indicates a two-dimensional graphic, the touch input is determined to be a signal for instructing transmission of feedback information, and different feedback information is transmitted to the mobile terminal of the video call partner according to the shape of the two- And a control unit for performing a control for performing a control for providing a touch feedback during a video call. The apparatus of claim 1, wherein the control unit Wherein the controller controls the operation of the mobile terminal in response to feedback information transmitted from the mobile terminal of the video call partner during a video call. The apparatus of claim 1, wherein the control unit Wherein the control unit sets a chat channel with the mobile terminal of the video call destination and controls the feedback information corresponding to the touch signal to be transmitted through the set chat channel. The method of claim 1, wherein the mobile terminal having the touch feedback providing function Further comprising a storage unit for storing feedback information corresponding to the touch input. 5. The apparatus of claim 4, wherein the control unit The control unit reads the feedback information corresponding to the shape of the two-dimensional graphic form from the storage unit and converts the read feedback information to conform to the chatting message transmission standard and transmits the converted feedback information to the mobile terminal of the video communication counterpart. The mobile terminal having a function of providing touch feedback among video calls. Displaying an image provided from a mobile terminal of a video call partner on a first area of a touch screen; Obtaining a coordinate value for a predetermined touch input from the first region; When the coordinate value is formed as a two-dimensional graphic, the touch input is determined to be a signal for instructing transmission of feedback information, and different feedback information is transmitted to the mobile terminal of the video call partner according to the shape of the two- The method comprising the steps of: [7] The method of claim 6, Receiving feedback information provided from a mobile terminal of the video call partner; And And controlling the operation of the mobile terminal according to the received feedback information. The method of claim 6, wherein the step of transmitting the feedback information to a mobile terminal Reading the feedback information corresponding to the shape of the two-dimensional graphic form in a storage unit of the terminal; Converting the read feedback information into a chat message transmission standard; And And transmitting feedback information converted into the chat message transmission standard to a mobile terminal of the video call partner through a chat channel.

Images