KR20100050021A - Apparatus and method for surporting various declaration of intention in portable terminal - Google Patents

Abstract

PURPOSE: An apparatus and a method for offering various expressions of opinions in the video call event in a portable terminal are provided to talks over the telephone through the various expressions of opinions using a vision, a hearing and tactual sense by a user terminals. CONSTITUTION: An event sensor(101) senses event during the video call. An event handler(102) extracts the code corresponding to the event from the code mapping information of the event. A code converter(103) generates the video call data including the code. A controller(100) transmits the video call data to the corresponding portable terminal. The code is included in the data sector used by the application of the video call data frame.

Description

Apparatus and method for providing a variety of expressions during video calls on a mobile terminal {APPARATUS AND METHOD FOR SURPORTING VARIOUS DECLARATION OF INTENTION IN PORTABLE TERMINAL}

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a portable terminal, and more particularly, to an apparatus and a method for providing various expressions of speech using hearing, vision, and tactile sense during a video call.

In general, with the development of wireless communication technology, the spread of portable terminals is increasing exponentially, and as the spread of portable terminals is generalized, communication providers are providing various services to satisfy various needs of subscribers. For example, mobile service providers currently use wireless Internet services, short message services (SMS), multimedia messaging services (Multimedia Messaging), as well as space and time-independent calls. Service (hereinafter referred to as "MMS"), a multi-communication service capable of simultaneously transmitting and receiving voice calls and data in a voice channel environment (hereinafter referred to as "CMS"), and multimedia services such as image, video, and digital broadcasting. Various services are provided through portable terminals.

Such mobile terminals perform data access such as personal information management, information exchange with a computer, and the Internet, in addition to the basic functions of moving a mobile communication service area and making a call with a general subscriber or another mobile subscriber through a corresponding base station. It has additional functions such as In recent years, the portable terminal has been developed as a complex information device having various additional functions such as transmission and reception of images and videos, an MP3 player function, a camera function capable of taking a picture, and a digital broadcast reproduction function.

However, these portable terminals provide only audible effects due to voice calls among users, and visual effects due to text messages and video transmission and reception. That is, until now, the portable terminal does not have a function of transmitting various visual / audio / tactile effects to the counterpart portable terminal during video communication with the counterpart.

Accordingly, there is a need for a portable terminal capable of providing differentiated functions of the portable terminal by transmitting sensed data to the user visually, acoustically, and tactilely during a call using the portable terminal.

SUMMARY OF THE INVENTION An object of the present invention is to provide an apparatus and method for providing various expressions using visual, auditory, and tactile sensations during a video call in a portable terminal.

Another object of the present invention is to provide an apparatus and method for providing various pseudo expressions using a data area reserved for a video call message during a video call in a portable terminal.

According to a first aspect of the present invention for achieving the above object, in a transmission method for providing a variety of expressions during a video call in a portable terminal, the process of detecting an event during a video call, and for each predefined event And extracting a code corresponding to the event from code mapping information, generating video call data including the code, and transmitting the video call data to the corresponding portable terminal.

According to a second aspect of the present invention for achieving the above object, in a receiving method for providing a variety of expressions during a video call in a portable terminal, receiving the video call data and included in the video call data Comparing the code with the code in the predefined code mapping table for each event, and if it matches the code in the mapping table, a visual effect, auditory effect, and / or tactile effect corresponding to the code is generated. Characterized in that it comprises a process.

According to a third aspect of the present invention for achieving the above object, a transmission apparatus for providing a variety of expressions during a video call in a portable terminal, comprising: an event detector for detecting an event during a video call, and a predetermined angle And an event processor for extracting a code corresponding to the event from code mapping information for each event, a code converter for generating video call data including the code, and a controller for transmitting the video call data to a corresponding portable terminal. It is characterized by.

According to a fourth aspect of the present invention for achieving the above object, a reception apparatus for providing various expressions during a video call in a portable terminal, the receiving unit for receiving the video call data, and included in the video call data A code conversion unit for comparing the code with the code in the code mapping table for each event, and the visual effect, the auditory effect, and / or the tactile effect corresponding to the code when the code is matched with the code in the mapping table. Characterized in that it comprises a control unit for generating.

As described above, by transmitting and receiving auditory, visual, and tactile information between user terminals in a portable terminal, there is an advantage that a call can be made through various expressions. In addition, by using the reserved data area portion of the video call message, no additional fee is required.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description of the present invention, detailed descriptions of related well-known functions or configurations will be omitted if it is determined that the detailed description of the present invention may unnecessarily obscure the subject matter of the present invention. The following terms are defined in consideration of the functions of the present invention, and may be changed according to the intentions or customs of the user, the operator, and the like. Therefore, the definition should be made based on the contents throughout the specification.

Hereinafter, the present invention will be described with respect to a device and method for providing a variety of expressions using the visual, auditory, and tactile sense during a video call in a portable terminal.

The present invention converts a signal detected by various input devices (keypad, touchpad, sensor) into a single promise code (hereinafter referred to as "S-code") when the transmitting portable terminal makes a video call. A method of transmitting the call data together with the call data and expressing the converted code using various output devices (vibration, display, speaker) is provided. Therefore, during the video call, all the information that can be detected by the terminal can be shared among callers.

Hereinafter, a terminal for generating and transmitting the S-code is called the originating portable terminal, and a portable terminal receiving the S-code transmitted from the originating portable terminal and generating a vibration / sound / visual effect corresponding thereto. An embodiment of the present invention will be described as a receiving portable terminal.

The portable terminal according to an embodiment of the present invention is preferably a mobile communication terminal, a digital broadcast receiver, a personal digital assistant (PDA), a smart phone, an IMT-2000 (International Mobile Telecommunication 2000) terminal. Information communication, such as wideband code division multiple access (WCDMA) terminals, universal mobile telecommunication service (UMTS) terminals, mobile phones, portable multimedia players (PMPs), MP3 terminals, and notebook computers Of course, the present invention may be applied to devices and multimedia devices and applications thereof.

1 illustrates a portable terminal for providing various expressions using visual, auditory, and tactile sensations during a video call according to an exemplary embodiment of the present invention.

Referring to FIG. 1, the portable terminal includes a control unit 100, a memory unit 110, a vibration driver 120, a keypad unit 130, a display unit 140, an RF module 150, and a baseband processor 155. ), A codec 160, and a sensing unit 170.

The RF module 150 lowers the frequency of the radio frequency signal received through the antenna to the baseband processor 155 and raises the baseband signal from the baseband processor 155 through the antenna. Send. In addition, the RF module 150 transmits and receives an S-code corresponding to an event.

The baseband processor 155 processes a baseband signal transmitted and received between the RF module 150 and the controller 100. For example, in case of transmission, the channel coding and spreading of data to be transmitted are performed, and in case of reception, the function of despreading and channel decoding of the received signal is performed. .

The keypad 130 includes key numbers 0 to 9, a menu key, a menu key, a cancel key, a confirmation key, a call key (TALK), an end key (END), an Internet access key, A plurality of function keys such as navigation keys (up / down / left / right) are provided, and key input data corresponding to a key pressed by a user is provided to the controller 100. In addition to the present invention, the keypad unit 130 receives a number key corresponding to a predetermined S-code from a user during a video call and outputs it to the controller 100.

The display unit 140 displays status information, various moving images and still images generated during the operation of the portable terminal. The display unit 140 may use a color liquid crystal display (LCD).

The touch pad unit 141 is located in front of the display unit 140 and provides touch data by a user touch to the controller 100. For example, when the user touches the touch pad unit 141, touch data according to a touch area is provided to the controller 100, and the controller 100 coordinates at which a touch is generated based on the touch data. Can be calculated. In addition to the present invention, the touch pad unit 141 receives touch data corresponding to a predetermined S-code from a user touch and outputs the touch data to the controller 100 during a video call.

The sensing unit 170 detects acceleration, vibration and shock of a portable terminal as an acceleration sensor or / and an approach sensor and outputs it to the controller 100 or detects when another portable terminal approaches the controller 100. ) The sensing unit 170 may be implemented in various ways such as a temperature sensor, a photo sensor, etc. in addition to the acceleration sensor or / and the proximity sensor.

That is, in the embodiment of the present invention, for the control related to the activation / deactivation control of the visual / auditory / haptic function, it can be performed by the operation of the keypad 130 or the touch pad unit 141, or It may be performed by the sensed signal of the sensing unit 170. In addition, when the visual / audio / tactile function is activated, an input of an event for generating an S-code may be performed by any one or a combination of the function keys, the touch pad and keypad, and a sensing signal by the sensing unit 170. Can be.

The memory unit 110 may be generated by a user or received from the outside, including application programs (eg, application programs for performing visual, audio, and tactile function control) necessary for operating a function according to an embodiment of the present invention. Data (eg, picture and video data, text messages, etc.); The memory unit 110 stores setting information related to the tactile function. In addition, the memory unit 110 may include one or more buffers for temporarily storing user data generated during the execution of the application program.

In addition, the memory unit 110 is based on the visual / auditory / tactile function, the event for the generation of vibration, S-code, vibration pattern, the vibration level and the event for visual generation, S-code, video vibration of the other party, image flickering Information such as events, S-codes, beeps, and modulated signals for color inversion, and audio generation. Such information is preferably stored in the form of a mapping table. For example, the mapping table may include an event field indicating information input by a user (eg, a touch event, a key event, an event by a sensor, etc.) when the visual / audio / tactile function is activated, and an operation method of each event. It may be represented by an operation field to be indicated, an S-code field indicating a corresponding S-code according to the event, and an expression field indicating types of visual / audio / tactile senses represented by the event and the S-code.

The mapping table is one embodiment shown for convenience of explanation and understanding according to an embodiment of the present invention. The mapping table including the event information of the present invention may be combined in various forms.

In addition, it is preferable that the above event information can be arbitrarily added / modified by the user. In addition, it is preferable that the added / modified event information is shared between the transmitting portable terminal and the receiving portable terminal. Basically, the event information is basically set between the portable terminals, or the event information can be downloaded from any server for providing a visual / audio / tactile function service to the portable terminals. Therefore, it is preferable that the server notifies the user's portable terminal of the event information update.

In addition, the S-code generated by the portable terminal based on the mapping table is transmitted along with the video signal and the audio signal in the video call data frame. In the case of video conferencing, the portable terminal can transmit the converted S-code to one or more receiving portable terminals.

4 illustrates a video call data frame.

Referring to FIG. 4, the data frame structure is divided into a header area 400, a control area 402, a data area 404, an audio area 406, a video area 408, and a CRC area 410. Here, the data area 404 is a channel allocated for transmitting and receiving data when using a specific application in a video call. In the present invention, during the video call, information (eg, S-code, etc.) for the visual / audio / tactile function is transmitted and received through the data area 404.

The vibration driver 120 processes a function for vibration generation under the control of the controller 100. That is, the vibration driver 120 inputs a control signal transmitted from the controller 100 according to the vibration event and changes the magnitude of the current according to the control signal of the controller 100 to change the vibration pattern / vibration level. In this way, various types of vibrations can be generated. The period and the intensity are controlled and driven according to the current control, and the vibration is generated according to the vibration pattern / vibration level. The speaker 161 outputs a beep generated by the codec 160 under the control of the controller 100 or outputs a sound modified by the codec 160 under the control of the controller 100. Output The display unit 140 displays the counterpart image (the image vibrates or flickers, inverts the color of the image, etc.) which is modified under the control of the controller 100.

The controller 100 controls the overall operation of the portable terminal and the signal flow between the internal blocks. That is, the controller 100 includes a memory unit 110, the vibration driver 120, the keypad unit 130, the display unit 140, the RF module 150, the baseband processor 155, and the The signal flow between the components of the codec 160 and the sensing unit 170 is controlled. The controller 100 may include the baseband processor 155.

In particular, the control unit 100 controls whether or not the activation of the visual / auditory / haptic function and its operation according to an embodiment of the present invention, and determines the event input when the visual / auditory / tactile function is activated Control the function to generate S-code accordingly. The controller 100 controls vibration / sound / time generation of the portable terminal by the generated S-code, and controls a series of operations of transmitting the S-code to a receiving portable terminal desired by the user.

In addition, the control unit 100 analyzes the vibration pattern / vibration level corresponding to the S-code when receiving the video call data including the S-code from any portable terminal during the video call and based on the vibration of the portable terminal Control the occurrence. Or analyzes the visual and auditory effects corresponding to the S-code and controls sound and visual based thereon.

In addition, the control unit 100 receives a predetermined command code (S-code) for the vibration / sound / time generation in the data area 404 of the video call data frame when receiving the video call data through the RF module 150 Extract. In this case, when information or command code for vibration / sound / time generation is present in the data area 404, the controller 100 generates vibration / sound / time.

In addition, the control unit 100 compares the S-code in the video call data received from any mobile terminal with the S-code of the mapping table, and when a specific S-code matching the S-code of the mapping table is detected. Controls preset vibration / sound / visual generation.

To this end, the controller 100 further includes an event detector 101, an event processor 102, a code converter 103, and the like.

The event detector 101 detects a general event (eg, a call request / receive event, a text message send / receive event, an event by a sensor, etc.) according to a function of the portable terminal. In particular, the event detection unit 101 detects a visual / audio / tactile function activation request signal generated from the outside during the video call mode of the portable terminal according to an embodiment of the present invention. In addition, the event detecting unit 101 detects a vibration / sound / visual event (eg, a touch event, a key event, an event by a sensor, etc.) input by the user for generating vibration when the tactile function is activated.

In addition, the event detection unit 101 determines whether the visual, auditory, and haptic function is activated when receiving video call data including an S-code from any portable terminal. At this time, whether to activate / deactivate the visual / hearing / tactile function of the present invention can be always maintained, that is, on (on) state according to a user setting method. It is desirable to select deactivation.

The event processor 102 is in charge of processing events detected by the event detector 101. In particular, the event processing unit 102 activates the visual / hearing / tactile function in response to the event sensing unit 101 detecting the visual / hearing / tactile function activation request signal. In addition, the event processor 102 analyzes the vibration / sound / visual event when the event detector 101 detects any vibration / sound / visual event in the tactile function activation state and corresponds to the S-code. Extracts and transmits the vibration / sound / visual information according to the S-code to the code conversion unit 103.

In addition, the event processor 102 extracts the S-code corresponding to the vibration event and includes the data in the data area 404 of the video call data frame to generate video call data. The event processor 102 processes the video call data to be transmitted to a target receiving portable terminal under the control of the controller 100.

In addition, the event processor 102 processes vibration / sound / time when the event detector 101 detects information related to vibration / sound / time in the received video call data. Through this, the portable terminal may notify the reception of the video call data by vibration, sound, or time that can transmit the emotion that the user of the calling portable terminal wants to transmit.

The code converter 103 generates a control signal corresponding to the S-code and transmits the generated control signal to the vibration driver 120 to generate a corresponding vibration. Then, the vibration driver 120 performs an operation of generating vibration in the portable terminal based on the control signal. Alternatively, a control signal corresponding to an S-code is generated and transmitted to the speaker 161 to generate a corresponding sound, or a control signal corresponding to an S-code is generated and transmitted to the display unit 140 to generate a visual effect. do.

Meanwhile, FIG. 2 illustrates a schematic configuration example of a portable terminal required for describing an operation according to an exemplary embodiment of the present disclosure for convenience of description, but the present invention is not limited thereto. Therefore, the portable terminal of the present invention may further include a camera module, a Bluetooth module, a digital broadcast receiving module and the like according to the provision form thereof.

In the above, the schematic configuration and the operation thereof of the portable terminal according to the embodiment of the present invention have been described. Next, a transmission operation for generating and transmitting an S-code during a video call in the portable terminal and a reception operation method for receiving the S-code and generating vibration / sound / visual effects corresponding thereto will be described. do. However, the present invention is not limited to the contents described below, it should be noted that it can be applied to various embodiments based on the following examples.

2 is a diagram illustrating an operation of a transmitting portable terminal for providing various expressions using visual hearing and tactile sense during a video call according to an exemplary embodiment of the present invention.

Referring to FIG. 2, in step 200, the transmitting portable terminal is conducting a video call with the receiving portable terminal. Activate the mode to send to the portable terminal. When the mode is converted into a code corresponding to an event signal generated through various input devices (key pad, touch pad, sensor, etc.) and transmitted to the receiving portable terminal, the receiving portable terminal outputs the corresponding code. Can be output as auditory, visual, and tactile.

Thereafter, in step 204, the transmitting portable terminal selects how to output the sensed data from various input devices. For example, the transmitting portable terminal transmits a signal input through a touch screen, a keypad, or a voice recognition sensor or an acceleration sensor to the vibration (tactile), display (visual), or speaker (audio) in the receiving portable terminal. The transmitting portable terminal may select an output type in outputting a signal corresponding to the S-code from the receiving portable terminal. Depending on the implementation, the receiving portable terminal may choose. That is, one S-code is mapped to visual / audio / tactile, so that the portable terminal selects one, two, or three output types of visual, audio, and tactile functions according to one S-code.

Thereafter, when the event occurs in step 206 (that is, when an input signal is generated through various input devices (touch screen, keypad, or sensor)), the transmitting portable terminal proceeds to step 208 and uses the predefined mapping table. To generate an S-code corresponding to the event.

In step 208, the transmitting portable terminal generates an auditory or visual or tactile sense corresponding to the S-code. That is, before the processing corresponding to the S-code is outputted as sound / time / vibration in the receiving portable terminal, the receiving portable terminal is executed to confirm in advance.

In step 212, the transmitting portable terminal determines whether to transmit the S-code, and when transmitting the S-code, the transmitting portable terminal includes the S-code in the data area of the video call data format to configure the video call data. do.

In step 216, the transmitting portable terminal transmits the video call data.

If no event occurs in step 206 or if the S-code is not transmitted in step 212, the process proceeds to step 218 to perform a corresponding mode.

Thereafter, the transmitting portable terminal terminates the procedure of the present invention.

3 illustrates an operation of a receiving portable terminal for providing various expressions using visual, auditory, and tactile sensations during a video call according to an exemplary embodiment of the present invention.

Referring to FIG. 3, when the receiving portable terminal is conducting a video call with the transmitting portable terminal in step 300, the receiving portable terminal checks whether the sensing data transmission mode is activated in step 302, and when the sensing data transmission mode is activated. Proceed to the step to check if there is a code in the data area portion of the image data.

If there is an S-code in the image data in step 304, the receiving portable terminal proceeds to step 306 to extract the S-code to analyze an output pattern (sound / vibration / time) corresponding to the S-code. . That is, it analyzes which output of vibration, sound, and visual effect corresponding to the S-code is used.

In step 308, the receiving portable terminal outputs the output pattern as sound, vibration, or visual effect.

In step 310, the receiving portable terminal maintains the video call mode. That is, the video call is performed simultaneously with the transmitting portable terminal while performing the operation according to the S-code.

Thereafter, the receiving portable terminal terminates the procedure of the present invention.

5 illustrates an example of video, audio, and data processing during a video call according to an exemplary embodiment of the present invention.

Referring to FIG. 5, the transmitting portable terminal generates data through an input 500 (510) and inserts the generated data into a corresponding frame region to form a video call data frame (520). That is, the image through the camera 502 is converted into video data through the video codec 512 and inserted into the video area. The voice through the microphone 504 is converted into audio data through the audio codec 514 and inserted into the audio region. In addition to the present invention, the signal through the sensor 506 or button, the audio signal by the microphone 504, the visual signal by the camera 502, etc. are converted into the S-code via the code conversion unit 514, the data Is inserted into the area. That is, the signal to the visual, audio, and tactile is converted into an S-code, and the converted S-code is included in the data area of the video call data and transmitted to the receiving portable terminal.

In contrast, the portable receiver receiving the video call data 530 processes and outputs the video call data in step 550. That is, the video data in the video area is decoded and output to the display. The audio data in the audio area is decoded and output to the speaker, and the S-code in the data area is decoded and output to the corresponding output device (motor, speaker or display).

For example, when output to the display, the other party's image displayed on the terminal vibrates or blinks, or inverts the color of the image. If the speaker outputs "beep". Or transform the voice in the audio domain and output it (e.g. add noise, add low tones, etc.).

Meanwhile, in the detailed description of the present invention, specific embodiments have been described, but various modifications are possible without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the scope of the following claims, but also by the equivalents of the claims.

1 is a portable terminal for providing a variety of expressions using visual, auditory, and tactile during a video call according to an embodiment of the present invention,

2 is a flowchart illustrating an operation of a transmitting portable terminal for providing various expressions using visual hearing and tactile sense during a video call according to an embodiment of the present invention;

3 is a flowchart illustrating an operation of a receiving portable terminal for providing various expressions using visual hearing and tactile sense during a video call according to an embodiment of the present invention;

4 is a video call data frame structure,

5 is a view illustrating video, audio, and data processing during a video call according to an embodiment of the present invention.

Claims (22)

In the transmission method for providing a variety of expressions during the video call in the portable terminal, Detecting events during the video call, Extracting a code corresponding to the event from code mapping information for each event; Generating video call data including the code, a video signal, and an audio signal; And transmitting the video call data to the corresponding portable terminal. The method of claim 1, The code is included in a data area used by a particular application of the video call data frame. The method of claim 1, Determining whether to activate own visual effects, auditory effects, and vibration effects upon detecting the event; And when the at least one of the visual effect, the auditory effect, and the vibration effect is activated, generating the visual effect, the auditory effect, and the vibration effect in response to the event. The method of claim 1, And selecting which of the visual effects, the auditory effects, and the vibration effects for the event to use. The method of claim 1, The mapping information includes information on visual effects, auditory effects, and vibration effects, and may be added / deleted by a user. The method of claim 1, And after the visual, auditory, and / or vibrational effects of the event occur, maintaining the video call. The method of claim 1, The event occurs from a keypad, a touchpad, and / or a sensor. In the receiving method for providing a variety of expressions during the video call in the portable terminal, Receiving video call data; Comparing the code included in the video call data with a code in a predetermined code mapping table for each event; Generating a visual effect, an auditory effect, and / or a tactile effect corresponding to the code in the mapping table. The method of claim 8, And after the visual, auditory, and / or vibrational effects of the event occur, maintaining the video call. The method of claim 8, Determining whether to activate visual effects, auditory effects, and vibration effects when the codes of the mapping table match. The method of claim 8, The mapping information includes information on visual effects, auditory effects, and vibration effects, and may be added / deleted by a user. In the transmission device for providing a variety of expressions during the video call in the portable terminal, An event detector for detecting an event during a video call; An event processing unit for extracting a code corresponding to the event from predefined code mapping information for each event; A code converter for generating video call data including the code; And a control unit for transmitting the video call data to the corresponding portable terminal. The method of claim 12, The code is included in a data area used by a particular application of the video call data frame. The method of claim 12, The control unit Determining whether to activate the own visual effects, auditory effects and vibration effects when the event is detected, And when the at least one of the visual effect, the auditory effect and the vibration effect is activated, generating the visual effect, the auditory effect and the vibration effect in response to the event. The method of claim 12, The control unit Device for selecting the effect of the visual effect, auditory effect and vibration effect for the event. The method of claim 12, The mapping information includes information on visual effects, auditory effects, and vibration effects, and may be added / deleted by a user. The method of claim 12, The control unit And after the visual, auditory, and / or vibrational effects of the event occur, maintaining the video call. The method of claim 12, Wherein the event occurs from a keypad, a touchpad, and / or a sensor. In the receiving device for providing a variety of expressions during the video call in the portable terminal, A receiving unit for receiving video call data; A code conversion unit for comparing the code included in the video call data with a code in a predetermined code mapping table for each event; And a controller for generating a visual effect, an auditory effect, and / or a tactile effect corresponding to the code in the mapping table. The method of claim 19, The control unit And after the visual, auditory, and / or vibrational effects of the event occur, maintaining the video call. The method of claim 19, The control unit If it matches with the code of the mapping table, it characterized in that it determines whether to activate the visual effect, auditory effect and vibration effect. The method of claim 19, The mapping information includes information on visual effects, auditory effects, and vibration effects, and may be added / deleted by a user.

Images