KR100861284B1 - Method for processing of voice during image communication and portable terminal having the same - Google Patents

Abstract

A voice processing method during a video call and a portable terminal having a voice processing function during the video call are provided to prevent distortion or deletion of a voice caused by interference between a sender's voice and a receiver's voice during the video call, thereby offering a high-quality voice call. A controller(130) transmits a sender's temporarily stored voice to a receiver after voice outputting of the receiver stops and the sender's voice is temporarily stored, when the sender's voice is inputted while the receiver's voice is outputted, and outputs a temporarily stored voice of the receiver after the sender's voice inputting stops and the received voice of the receiver is temporarily stored, when the receiver's voice is received while the sender's voice is inputted. A storage unit(140) temporarily stores at least one of the sender's voice and the receiver's voice. An audio codec(125) receives a voice or sound signal from the controller, converts the signal into an analog signal after decoding the signal, and then provides the converted analog signal to a speaker(123).

Description

TECHNICAL FOR PROCESSING OF VOICE DURING IMAGE COMMUNICATION AND PORTABLE TERMINAL HAVING THE SAME}

The present invention relates to a portable terminal, and more particularly, to a portable terminal having a video processing voice processing method and a video processing voice processing function that can be applied to a portable terminal having a video call function.

Due to the evolution of mobile communication technology, full-scale video telephony has become commercially available, and it is now possible to make a call while looking at the other party using a portable terminal, regardless of the place, away from the existing mobile communication service mainly focused on transmitting and receiving voice calls and short messages.

In the video call, the camera of the mobile terminal is used to take a picture of the caller, process the recorded video to meet the transmission standard of the video call, and transmit the video call to the video call counterpart. It is a service that allows video call counterparts to talk while looking at each other by converting them to be displayed on the display device of the portable terminal and displaying them on the portable terminal of the caller.

Since video call requires user's video to be recorded, the user should call at least a certain distance away from the portable terminal.If not using a headset equipped with earphones and microphone, the user should be away from the portable terminal. The speaker and the microphone provided in the terminal exchange voice with the video call counterpart.

For the same reason, a portable terminal having a general video call function increases the output of a speaker during a video call than when using a voice call.

Since a typical portable terminal has a small size, the distance between the speaker and the microphone is close, and if the output of the speaker is increased during the video call as described above, an echo phenomenon may occur in which the voice output from the speaker is input into the microphone. In order to prevent such an echo phenomenon, an echo canceller is provided in a general portable terminal. However, when the caller and the receiver speak simultaneously during a video call due to the echo canceller, the voice is distorted or deleted. The voice is disconnected during the call, which causes a disadvantage that the voice is not normally delivered to the video call counterpart.

Accordingly, a first object of the present invention is to provide a voice processing method during a video call which can provide a good voice call between video call parties during a video call.

In addition, a second object of the present invention is to provide a portable terminal having a voice processing function during a video call that can provide a good voice call between video call parties during a video call.

According to an aspect of the present invention for achieving the first object of the present invention, the voice processing method of the video call, the voice of the caller is temporarily stored if the voice of the caller is input while the voice of the caller is being output. And transmitting the voice of the temporarily stored caller to the called party after the voice output of the called party is stopped. If the voice of the talker is input while the talker's voice is being output, temporarily storing the input talker's voice may determine whether the talker's voice is output based on the presence of the decoded voice data. If the voice of the talker is input while the talker's voice is being output, the step of temporarily storing the voice of the talker is performed by converting the talker's voice into digital data and then compressing the converted digital data using a predetermined codec. One data can be stored. The step of transmitting the temporarily stored voice of the called party to the called party after the output of the voice of the called party is stopped may determine that the output of the called party is stopped when the silent section of the called party is more than a predetermined time.

In addition, according to another aspect of the present invention for achieving the first object of the present invention, the voice processing method of the video call, if the voice of the caller is received while the voice of the caller is input, And temporarily storing the voice of the temporarily stored caller after the voice input of the caller is stopped. When the voice of the called party is received while the voice of the talker is being input, temporarily storing the received voice of the called party may include any one of a method of detecting a signal input of a predetermined level or more and a signal of a frequency band corresponding to the voice. The method may include determining whether the caller inputs a voice. If the voice of the called party is received while the voice of the talker is being input, temporarily storing the received voice of the called party may decode the voice of the called party and then temporarily store the decoded voice of the called party. The step of outputting the temporarily stored voice of the called party after the voice input of the caller is stopped may be determined that the voice input of the caller is stopped when the silent section of the caller's voice is longer than a predetermined time.

In addition, the portable terminal having a voice processing function during a video call according to an aspect of the present invention for achieving the second object of the present invention, if the voice of the caller is input while the voice of the caller is being output After temporarily storing the voice of the called party, the voice of the temporarily stored caller is transmitted to the called party, and when the voice of the called party is received while the voice of the called party is being inputted, the received voice of the called party is temporarily stored. And a controller configured to output a voice of the temporarily stored caller after the voice input of the caller is stopped and a buffer for temporarily storing at least one of the voice of the caller and the voice of the caller. The controller may determine whether the receiver outputs the voice based on the presence of the decoded voice data. The controller may determine whether the caller inputs a voice using any one method of detecting a signal input of a predetermined level or more and detecting a signal in a frequency band corresponding to a voice. The portable terminal having a voice processing function during the video call may further include an audio codec for converting the caller's voice into digital data and then compressing the converted digital data using a predetermined codec. The controller determines that the output of the talker's voice is stopped when the silent section of the talker's voice is greater than or equal to a predetermined time, and when the silent section of the talker's speech is more than the predetermined time, the caller's voice input is It can be judged as stopped.

According to the portable terminal having the voice processing method and the voice processing function during the video call as described above, when the caller and the caller speak at the same time after the video call is connected, the voice of the first speaker is processed and the person who speaks later. The voice of is temporarily stored in the buffer, and then the voice of the first speaker is processed and processed so that only the voice of one of the talker and the talker is processed at a predetermined time.

Therefore, it is possible to provide a good voice call by preventing distortion or deletion of the voice due to interference between the caller voice and the caller voice during the video call.

As the present invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description.

However, this is not intended to be limited to the specific embodiment of the present invention, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term and / or includes a combination of a plurality of related items or any item of a plurality of related items.

Hereinafter, with reference to the accompanying drawings, it will be described in detail a preferred embodiment of the present invention. Hereinafter, the same reference numerals are used for the same components in the drawings, and duplicate descriptions of the same components are omitted.

1 is a block diagram showing the configuration of a portable terminal having a voice processing function during a video call according to an embodiment of the present invention.

Referring to FIG. 1, a portable terminal according to an exemplary embodiment of the present invention includes a camera unit 111, a display unit 113, a graphic processor 115, a video codec 117, a microphone 121, and a speaker 123. , An audio codec 125, a controller 130, a storage 140, a key input unit 150, and a wireless transceiver 160.

Although the configuration of the camera unit 111 is well-known, it is not shown in detail, but includes a lens, an image sensor, and an analog-digital converter, and when an image capturing or video call function is executed, a subject incident through the lens The optical signal is converted into an electrical signal corresponding to the optical signal incident through the image sensor, and converted into a digital image corresponding to the converted electrical signal through an analog-to-digital converter and then provided to the graphic processor 115.

Here, the camera unit 111 may capture an image of 15 frames per second or 30 frames per second under the control of the controller 130.

The display unit 113 may be, for example, a display device such as a liquid crystal display (LCD) or an organic light emitting diode (OLED), and an image signal provided from the graphic processor 115. Based on the display, the user interface such as a menu, an operation state, an application program execution screen, and the like of the portable terminal are displayed.

In particular, during the video call, the display 113 receives the video call counterpart and / or the user's video from the graphic processor 115 and displays the received video call counterpart and / or the user's video in a predetermined display area. . In addition, when taking a picture, a preview image and a captured image are displayed.

The graphic processor 115 receives the digital image from the camera 111, reduces the size of the image by converting the color format of the digital image, and provides the digital image to the video codec 117. For example, the graphic processor 115 may receive raw data in the form of RGB (Red, Green, Blue) from the camera unit 111, encode the raw data, and convert the raw data into the YCbCr 420 format.

In addition, the graphic processor 115 receives an image (eg, an image of a video call counterpart) decoded from the video codec 117 and resolutions so that the received image can be displayed on a predetermined region of the display 113. After performing graphics processing such as a display position, a frame rate, and the like, the display unit 113 is provided to the display unit 113.

The video codec 117 receives a digital image from the graphic processor 115 or the controller 130, encodes the digital image in a predetermined video call format, and provides the digital image to the controller 130. In addition, the video codec 117 receives a predetermined image (for example, an image of a video call counterpart) provided from the controller 130, decodes the same, and provides the decoded image to the graphic processor 115.

The video codec 117 can encode and decode video using, for example, codecs such as H.261, H.263, H.264, and MPEG-4, and during video calls, H.263, MPEG-4 Simple profile level 0 can be used to encode and decode video.

The microphone 121 receives a user's voice in a video call and a voice call, converts the input user's voice into an electric signal corresponding thereto, and provides the same to an audio codec 125.

The speaker 123 receives the audio and / or sound signal decoded from the audio codec 125 and converted into an analog signal and amplified to a predetermined level, and outputs the audio and / or audio signal in the audio frequency band.

The audio codec 125 receives an electric signal in analog form corresponding to the user's voice from the microphone 121, converts it into a digital signal corresponding thereto, and encodes the converted digital signal to comply with a voice call or video call transmission standard. After that is provided to the control unit 130.

In addition, the audio codec 125 receives a predetermined voice and / or sound signal from the controller 130, decodes it, converts the signal into an analog signal, and amplifies the analog signal to the speaker 123.

The audio codec 125 is, for example, voiced using a codec such as QualComm Code Excited Linear Predictive Coding (QCELP), Enhanced Variable Rate Codec (EVRC), G.711, G.723, G.723.1, G.728, or the like. Encoding and decoding may be performed.

The controller 130 performs control and processing for voice call and video call, which are inherent functions of the portable terminal. For example, the controller 130 multiplexes the video and audio signals provided from the video codec 117 and the audio codec 125 and generates a baseband signal by channel coding the multiplexed signal. To provide. In addition, the controller 130 may decode and demultiplex the baseband signal provided from the wireless transceiver 160 and provide the demultiplexed video and audio signals to the video codec 117 and the audio codec 125, respectively. Can be.

In addition, when an event signal indicating a video call is provided from the key input unit 150, the controller 130 connects the video call call with the portable terminal of the video call counterpart, and then the user (ie, the caller) and the video call counterpart (ie , When the talker speaks at the same time, the voice of the first speaker among the user and the video call counterpart is processed, and the voice of the later speaker is temporarily stored and then processed after the voice of the first speaker is processed. And only the voice of any one of the called parties.

To this end, the controller 130 may include a voice call control module 131.

In detail, when the user's voice is input through the microphone 121 after the video call is connected, the voice call control module 131 checks whether the voice of the video call counterpart is output through the speaker 123, If it is determined that the voice is being output through the speaker 123, the user's voice input through the microphone 121 is stored in the buffer 141 of the storage 140 and then output through the speaker 123. After the voice of the video call counterpart is stopped, the voice of the user stored in the buffer 141 is processed and transmitted to the portable terminal of the video call counterpart.

Here, the user's voice is stored in the buffer 141 until the time when the voice of the video call counterpart stops.

In addition, when the voice call control module 131 receives the video call counterpart's voice after the video call is connected, the voice call control module 131 checks whether the user's voice is input through the microphone 121, and the user's voice controls the microphone 121. If it is confirmed that the input through the video call counterpart of the received in the buffer 141 of the storage unit 140, after the voice of the user input through the microphone 121 is stopped and stored in the buffer 141 The voice of the video call counterpart may be output through the speaker 123.

Here, the voice call control module 131 may determine that the voice of the user is input through the microphone 121 and that the voice of the video call counterpart is received based on the signal processed by the audio codec 125. That is, it is possible to determine whether the user's voice is input based on whether the user inputs a signal of a predetermined level or more in consideration of ambient noise input through the microphone 121 such as the user's voice, or a predetermined frequency band corresponding to the voice (for example, , 300Hz to 3.4 kHz) to detect the user's voice input. In addition, when data decoded by the audio codec 125 exists, it may be determined that the voice of the video call counterpart is received.

When the voice of the video call counterpart is output through the speaker 123, the voice call control module 131 may receive data processed by the audio codec 125 and store the received data in the buffer 141.

Here, the data processed by the audio codec 125 refers to data that is converted using a predetermined codec after converting a user voice having an analog form input through the microphone 121 into digital. In addition, when the voice of the user is input, the voice call control module 131 may store the voice of the video call counterpart decoded by the audio codec 125 in the buffer 141.

In addition, the voice call control module 131 generates a talk spurt and a silence section for the voice of the user who is input through the microphone 121 and the voice of the video call counterpart output through the speaker 123. When monitoring and the silent section is more than a predetermined time (for example, 500ms), it can be determined that the voice of the user or the voice of the video call counterpart is stopped.

The voice call control module 131 may be implemented as a software program that may be executed by a processor included in the portable terminal, or may be implemented as a separate independent hardware chip.

The storage unit 140 may be configured of a nonvolatile memory such as a flash memory, an electrically erasable and programmable read only memory (EEPROM), a system program (for example, an operating system) required for basic operation of the portable terminal, and / Or save other applications.

In addition, the storage 140 may store data generated by a user, and may store data generated during the execution of the system program and / or the application program.

In particular, the storage 140 is provided with a buffer 141 of a predetermined size, and the buffer 141 has a voice or video call counterpart of the user provided from the audio codec 125 under the control of the controller 130. You can temporarily store your voice.

The key input unit 150 includes a plurality of number, character input keys, and function keys for performing a special function, and when a key manipulation occurs by a user, the key input unit 150 provides a corresponding key input signal to the controller 130.

The wireless transceiver 160 converts a radio frequency (RF) signal received through the antenna 161 into a baseband signal and provides it to the controller 130, and the base provided from the controller 130. The band signal is converted into a radio frequency signal and output through the antenna 161.

In particular, the wireless transceiver 160 transmits a video call signal of a predetermined frame per second to the portable terminal of the video call counterpart by the video codec 117 and the audio codec 125 in a predetermined video call format. The video call signal transmitted from the other party's portable terminal is received and provided to the controller 130.

2 is a flowchart illustrating an audio processing process of a video call according to an exemplary embodiment of the present invention, and illustrates a processing process when a voice of a user is input while a voice of a video call counterpart is being output through the speaker 123. .

Referring to FIG. 2, first, the controller 130 connects a video call call to a portable terminal having a designated number when an event signal indicating a video call is provided (step 201).

Then, when the voice of the user (ie, the talker) is input through the microphone 121 (step 203), the controller 130 determines whether the voice of the video call counterpart (ie, the talker) is being output through the speaker 123. Check (step 205). Here, the controller 130 may determine that the voice of the video call counterpart is output through the speaker when there is audio data of the video call counterpart to be decoded.

If it is determined in step 205 that the voice of the current video call counterpart is being output, the controller 130 transmits the voice of the user input through the microphone 121 to the buffer 141 provided in the storage 140. Save (step 207). Here, the controller 130 stores the data provided by the audio codec 125, that is, the analog voice input through the microphone is converted into digital and then compressed by the predetermined codec.

Thereafter, the controller 130 determines whether the voice output of the video call counterpart is stopped (step 209), and if it is determined that the voice output of the video call counterpart is stopped, the controller 130 processes the voice of the user stored in the buffer 141 to process the video call counterpart. It transmits to the portable terminal of (step 211).

Here, the controller 130 monitors the talk spurt and the silence section of the voice of the video call counterpart, and when the silent section is more than a predetermined time (for example, 500 ms), the video call counterpart It can be determined that the audio output is stopped.

If it is determined in step 205 that the voice of the video call counterpart is not currently output, the controller 130 processes the voice of the user input through the microphone 121 in real time to the portable terminal of the video call counterpart. Transmit (step 213).

Thereafter, the controller 130 determines whether an event signal for instructing the end of the video call is input (step 215), and if it is determined that an event signal for instructing the end of the video call is input, the controller 130 ends the voice processing process during the video call. If it is determined that the event signal indicating the end of the video call is not input, the process returns to step 203 to sequentially perform the following steps.

3 is a flowchart illustrating a voice processing method during a video call according to another embodiment of the present invention, and illustrates a processing process when a voice of a video call counterpart is received while a user's voice is input through the speaker 123.

Referring to FIG. 3, first, when the event signal indicating the video call is provided, the controller 130 connects the video call with the portable terminal of the designated number (step 301).

Thereafter, when the voice of the video call counterpart is received (step 303), the controller 130 determines whether the voice of the user is input (step 305).

Here, the controller 130 may determine whether the user's voice is input based on whether the user inputs a signal of a predetermined level or more in consideration of ambient noise input through the microphone, such as the user's voice, or a predetermined frequency band corresponding to the voice. For example, it is possible to determine whether a user inputs a voice by detecting a signal of 300 Hz to 3.4 kHz.

If it is determined in step 305 that the voice of the user is currently input, the controller 130 stores the received video call counterpart voice in the buffer 141 provided in the storage 140 (step 307). ).

Thereafter, the controller 130 determines whether the user's voice input is stopped (step 309), and if it is determined that the user's voice input is stopped, the controller 130 processes the voice of the video call counterpart stored in the buffer 141 to operate the speaker 123. Output via step 311.

Herein, the controller 130 monitors a talk spurt and a silence section for the user's voice input, and if the voice section is longer than a predetermined time (for example, 500 ms), the user's voice input is You can judge it as stopped.

If it is determined in step 305 that the voice of the user is not currently input, the controller 130 processes the received voice of the video call counterpart and outputs it through the speaker 123 in real time (step 313).

Thereafter, the controller 130 determines whether an event signal for instructing the end of the video call is input (step 315), and if it is determined that an event signal for instructing the end of the video call is input, the controller 130 ends the voice processing process during the video call. If it is determined that the event signal indicating the end of the video call is not input, the process returns to step 303 and the subsequent steps are sequentially performed.

Although described with reference to the embodiments above, those skilled in the art will understand that the present invention can be variously modified and changed without departing from the spirit and scope of the invention as set forth in the claims below. Could be.

1 is a block diagram showing the configuration of a portable terminal having a voice processing function during a video call according to an embodiment of the present invention.

2 is a flowchart illustrating a voice processing method during a video call according to an embodiment of the present invention.

3 is a flowchart illustrating a voice processing method during a video call according to another exemplary embodiment of the present invention.

<Explanation of symbols for the main parts of the drawings>

111 camera portion 113 display unit

115: graphics processor 117: video codec

121: microphone 123: speaker

125: audio codec 130: control unit

131: voice call control module 140: storage unit

141: buffer 150: key input unit

160: wireless transmission and reception unit

Claims (13)

In the video call method, Temporarily storing the voice of the input caller when the voice of the caller is input while the voice of the caller is being output; And And transmitting the voice of the temporarily stored caller to the called party after the voice output of the called party is stopped. The method of claim 1, wherein when the voice of the talker is input while the voice of the talker is being output, temporarily storing the input caller's voice, And determining whether to output the voice of the called party based on the presence or absence of the decoded voice data. The method of claim 1, wherein if the voice of the talker is input while the voice of the talker is being output, the step of temporarily storing the input caller's voice, And converting the talker's voice into digital data, and then compressing the converted digital data using a predetermined codec. The method of claim 1, wherein after the voice output of the called party is stopped, transmitting the temporarily stored caller's voice to the called party, And determining that the voice output of the called party is stopped when the silent section of the called party is more than a predetermined time. In the video call method, Temporarily storing the voice of the called party when the voice of the called party is received while the voice of the talker is being input; And And outputting the temporarily stored voice of the temporarily stored caller after the voice input of the caller is stopped. The method of claim 5, wherein when the voice of the called party is received while the voice of the caller is being input, temporarily storing the received voice of the called party, And determining whether a voice is input by the caller using any one method of detecting a signal input of a predetermined level or more and detecting a signal in a frequency band corresponding to a voice. The method of claim 5, wherein when the voice of the called party is received while the voice of the caller is being input, temporarily storing the received voice of the called party, And after the voice of the called party is decoded, temporarily storing the decoded voice of the called party. The method of claim 5, wherein after the voice input of the caller is stopped, the voice of the temporarily stored caller is output. And determining that the voice input of the caller is stopped when the silent section of the caller's voice is more than a predetermined time. In a portable terminal having a video call function, If the voice of the talker is input while the voice of the talker is being output, the voice of the caller is temporarily transmitted to the talker after the voice of the talker is temporarily stored after the voice of the talker is stopped. A controller for temporarily storing a voice of the called party and receiving the temporarily stored voice of the called party after the voice of the called party is stopped when the voice of the called party is received; And And a storage unit having a buffer for temporarily storing at least one of the talker's voice and the talker's voice. The method of claim 9, wherein the control unit A portable terminal having a voice processing function during a video call, characterized in that it is determined whether or not the caller's voice is output based on the presence or absence of decoded voice data. The method of claim 9, wherein the control unit A portable terminal having a voice processing function of a video call, characterized in that it is determined whether or not the caller's voice is input by using any one of a method of detecting a signal input of a predetermined level or more and a signal of a frequency band corresponding to a voice. The portable terminal of claim 9, wherein the portable terminal has a voice processing function during the video call. And a audio codec for converting the voice of the talker into digital data and then compressing the converted digital data using a predetermined codec. The method of claim 9, wherein the control unit It is determined that the voice output of the talker is stopped when the silent section of the talker's voice is more than a predetermined time, and it is determined that the voice input of the talker is stopped when the silent section of the talker's voice is more than the predetermined time. A portable terminal having a voice processing function during a video call.

Images