KR20090090753A - Mobile terminal sharing decoded multimedia video and audio signal - Google Patents

Abstract

A mobile communication terminal sharing decoded multimedia video and audio signals is provided to share video and audio of stored multimedia contents with a counterpart terminal while reproducing the multimedia contents. A multimedia controller(15) generates digital multimedia audio data by reproducing stored multimedia data. An audio mixer(12) mixes an analog voice signal inputted from a microphone(11) with the digital multimedia audio data selectively to generate single digital audio data. An audio encoder(13) compresses the single digital audio data. A signal transceiver(19) modulates the compressed audio data and transmits it to a counterpart mobile communication terminal through a mobile communication network.

Description

Mobile terminal sharing decoded multimedia video and audio signal

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a mobile communication terminal technology. In particular, a multimedia call can be transmitted to a counterpart terminal as well as a voice of a caller while playing multimedia content through a terminal equipped with an audio file playing function such as MP3. In the case of a possible terminal, video files such as MPEG4, AVI, Divx, WMV, etc. are stored in memory and played, and the video images of the video call and multimedia contents are synthesized and transmitted to the user's camera. The present invention relates to a mobile communication terminal technology capable of transmitting an image together with a counterpart terminal.

Recently, various types of functions are added to mobile communication terminals as well as voice calls between users. In other words, music file playback function of various formats such as MP3, WMA, Ogg, camera shooting function, etc. are added almost basically, and in addition, DMB broadcasting reception function is added so that you can watch drama, news, etc. outdoors through mobile communication terminal. have.

In this case, in order to transmit MP3 music files to a counterpart terminal through a conventional mobile communication terminal, data should be transmitted by using the wireless Internet. However, there is a limitation that it takes much time to send and receive music files through the mobile communication terminal. In particular, there are many limitations in transmitting large files such as movie files through the wireless Internet.

Recently, a third generation mobile communication terminal capable of quickly transmitting multimedia data and making a video call by supporting a wide bandwidth and a fast processing function instead of the conventional second generation mobile communication has been released. However, the speed of transmitting large-capacity multimedia files to the other party's terminal is limited, even through the third generation mobile communication technology. This situation is not presented.

An object of the present invention is to provide a mobile communication terminal that can be shared with the other terminal while playing the video and audio of the multimedia content stored in the terminal.

A mobile communication terminal according to the present invention is a mobile communication terminal for processing a voice call by making a call with a counterpart terminal through a mobile communication network, comprising: a microphone for receiving an analog voice signal; A multimedia control unit for generating digital multimedia audio data by playing multimedia data stored in the terminal; An audio synthesizer configured to selectively synthesize analog voice signals and digital multimedia audio data to generate single digital audio data; An audio encoder for compressing single digital audio data through an audio codec to generate audio compressed data; And a signal transceiving unit configured to modulate the audio compressed data and transmit the audio compressed data to the counterpart mobile communication terminal through the mobile communication network.

According to the present invention, an audio file such as an MP3 file is stored and played through a mobile communication terminal, and the other party in the call can hear the audio currently being played.

In addition, a video call is supported, and a video screen played through a mobile terminal equipped with a memory and a hardware resource for storing video data such as a movie file and a hardware resource is displayed on the screen of the current terminal. There is an effect that can be shown in real time.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a mobile communication system between video call-only terminals.

The video call support mobile communication network 20 is a third generation mobile communication network providing a wide bandwidth, and includes WCDMA-based asynchronous and CDMA2000-based synchronous. In the case of WCDMA-based mobile communication networks, 3G.324M protocol is used to support video calls, and H.323 is used as a protocol to support video calls on CDMA2000.

The video call-only terminals 10 and 30 are equipped with a camera therein and have a function of capturing an image of a terminal user in real time and transmitting the image to a counterpart terminal. Encode the user's voice input through the camera of the terminal 10 and the user's voice input through the terminal microphone, respectively, and converts each encoded data into a transmittable signal to the other terminal 30 through the video call support mobile communication network (20) To send).

2 is a block diagram showing the overall configuration of a mobile communication terminal according to the present invention.

The mobile communication terminal according to the present invention can be considered differently by dividing into two cases, a general voice call only terminal and a video call terminal. In this case, when only a voice call is supported, the decoded multimedia audio signal may be shared with the counterpart terminal. When the video call is supported, both the decoded multimedia audio signal and the video signal may be shared.

■ Only voice calls are supported

When the mobile communication terminal 10 supports only a voice call, it refers to a terminal having a function of a general mobile communication terminal for handling a call by connecting to a counterpart terminal through a mobile communication network and the audio data reproducing function mentioned below. In FIG. 2, a function of sharing an audio signal with the other party through the remaining components is implemented except for the 'additional component added to a video callable terminal' indicated by a dotted line at the bottom.

The microphone unit 11 receives an analog voice signal of the terminal user.

The multimedia controller 15 reproduces the multimedia data stored in the terminal to generate digital multimedia audio data. The multimedia data refers to audio data such as MP3 that can be reproduced by the user pressing the play button of the terminal. The multimedia data is transmitted through an external line or downloaded through a wireless Internet and stored in the memory 14.

The audio synthesizing unit 12 generates single digital audio data by synthesizing the digital multimedia audio data generated by reproducing the analog voice signal and the multimedia data input through the microphone unit 11. At this time, the single digital audio data is the most basic PCM-type digital data without any special processing such as encoding, and the data stored in a general audio CD.

In addition, the audio synthesizing unit 12 may switch the audio synthesizing function by an external user operation. In other words, when the user turns on the audio synthesis function, the audio signal currently being played and the user's voice are synthesized. If the user turns off the audio synthesis function, the currently played audio signal is treated as data separate from the user's analog voice signal. It has no effect on the signal transmitted to the terminal.

A more specific embodiment of the audio synthesizer 12 will be described in more detail later with reference to FIG. 3.

The audio encoder 13 compresses single digital audio data generated by the audio synthesizer 12 through an audio codec to generate audio compressed data. In general, in a mobile communication system, audio compression is performed by using a vocoder suitable for processing a human voice of a relatively low frequency band, and in the present invention, in order to make good use of characteristics of an audio signal generated through multimedia playback, You can also apply audio codec.

The signal transceiver 19 modulates the audio compressed data generated by the audio encoder 13 and transmits the modulated audio compressed data to the counterpart mobile communication terminal through the mobile communication network 20.

Through the data movement and conversion between the above components, the audio signal reproduced in the terminal is synthesized with the user's voice and then transferred to the counterpart terminal as a single audio signal.

■ When the video call function is supported

In this case, the mobile communication terminal 10 is provided with a video call function, and the video call function for processing a video call by connecting to a video call dedicated terminal of the other party through a mobile communication network supporting video call and the multimedia mentioned below Means a terminal equipped with a data reproduction function.

The camera unit 16, the video synthesizer 17, and the video encoder 18 are further added to the components of the terminal that support only the above-mentioned voice call. The aforementioned signal transceiver 19 also supports a video call. By being configured to replace the video call signal transmission and reception unit 19 as much as possible, the function of sharing not only audio signals but also video signals with the other party is implemented.

Since the technical content until the audio compressed data is generated by the audio encoder 13 is the same as described above in the case where only a voice call is supported, the description below will be omitted, and the video codec is generated and the system codec is generated along with the audio compressed data. The process of compression and transmission is further described.

The aforementioned multimedia control unit 15 has a video playback function as well as an audio playback function, and reproduces multimedia data to generate digital multimedia video data. At this time, the multimedia data is a concept including all audio files such as MP3, WMA, Ogg, or video files such as MPEG, AVI, H.264, Divx, WMA, etc. The user moves by playing and watching movie files through the terminal. A communication terminal can be used as a portable media player (PMP).

The camera unit 16 captures an external image in real time to generate digital video call image data. In general, a cam of hundreds of thousands of pixels mounted separately for a video call is used separately from a camera of a millions of pixels embedded in a mobile communication terminal.

The video synthesizing unit 17 generates synthesized image data by using digital multimedia video data generated by reproducing digital video call image data and multimedia data generated by the camera unit 16.

At this time, the composite image data may be selectively set by the user. That is, the composite video may be composed in the form of a picture-in-picture (PIP) in which a camera image of the user is laid out in the background and the multimedia video image is arranged as a reduced image in a part of the entire screen, or vice versa. It can also be configured. In addition, instead of including the camera image of the user, the composite image may be provided so that the entire screen is filled with only the multimedia video image.

The synthesized image data generated by the video synthesizer 17 is digital data in the form of the most basic YUV RAW data without special processing such as encoding, like the single digital audio data previously generated by the audio synthesizer 12.

In addition, the terminal user may switch the video synthesis function similarly to the function of the audio synthesis unit 12. That is, when the user turns on the video compositing function, a synthesized image is generated. When the user turns off the video compositing function, the synthesized image is no longer generated.

A more specific implementation of the video synthesis unit 17 will be described in detail later with reference to FIG. 4.

The video encoder 18 generates video compressed data by compressing the synthesized image data generated by the video synthesizer 17 through a video codec. In general, video codecs such as MPEG4, H.263, and H.264 are used for video calls. The video codecs used by mobile communication networks, terminals, and servers are slightly different.

The video call signal transmitting and receiving unit 19 is configured to be able to process video data by adding more functions than the signal transmitting and receiving unit 19 in charge of audio data transmission. That is, the video compression data generated by the video encoder 18 is encoded and modulated with the video call system codec together with the audio compressed data generated by the audio encoder 13 and transmitted to a counterpart video call terminal through a multimedia mobile communication network. do.

The video and audio signals of the multimedia data reproduced in the terminal are synthesized with the user's voice and camera image through the data movement and conversion between the above components and transmitted to the counterpart terminal.

FIG. 3 is a block diagram illustrating a first embodiment in which the audio synthesis unit 12 of FIG. 2 is implemented to superimpose (A) analog signals and (B) a second embodiment in which digital data is superimposed and synthesized. It is also.

The audio synthesizing unit 12 according to the first embodiment of Fig. 3A includes a DAC module, an analog audio synthesizing module, and an ADC module.

The multimedia chip is a component of the same concept as the multimedia controller 15 described with reference to FIG. 2, and a chip for managing only a multimedia function may be added separately from a chip for controlling a mobile communication function in a terminal.

The multimedia chip is equipped with an audio codec and a video codec for playing multimedia data, which is a concept separate from the codec for encoding data transmitted to the outside through mobile communication. Codec. 3, only the case of processing audio data will be described once, and the case of processing video data will be described with reference to FIG. 4.

When multimedia data such as MP3 is played through the multimedia chip, digital multimedia audio data in the form of PCM is generated.

The DAC module converts digital multimedia audio data generated from the multimedia chip into an analog audio signal.

An analog audio signal containing the user's voice is input through the microphone, and the analog audio synthesis module generates a single analog audio signal by combining the analog audio signal output through the DAC module with the analog audio signal input through the microphone. do.

In this case, the sound expressed in analog is synthesized by simply superimposing two sounds, so that an analog audio synthesizing module does not need a complicated internal configuration and can be realized by simply connecting two signal lines. In FIG. 3 (A), a component denoted by 'ⓧ' between the microphone and the DAC module corresponds to an analog audio synthesis module.

A single analog audio signal synthesized through the analog audio synthesis module is converted into a single digital audio data of the PCM method by the ADC module, and converted into compressed audio data through an audio encoder.

In addition, before the analog audio signal output through the DAC module is output to the headphone or speaker of the terminal must be synthesized with the voice signal received from the other terminal. That is, an audio output unit for synthesizing and externally outputting the analog audio signal output through the DAC module and the analog voice signal received from the counterpart terminal may be further added.

The audio synthesis unit 12 according to the second embodiment of FIG. 3B includes an ADC module and a digital audio synthesis module.

First, an analog voice signal containing a user's voice is input through a microphone, and the analog voice signal is converted into PCM type digital voice data through an ADC module.

In addition, the multimedia chip reproduces multimedia data such as MP3 and generates digital multimedia audio data in the form of PCM.

With the two types of PCM digital data generated as described above, the digital audio synthesis module superimposes and synthesizes the digital voice data output from the ADC module and the digital multimedia audio data output through the multimedia chip. Create

In the case of PCM-type digital audio sound, analog audio sound is sampled at regular time intervals. Thus, two signals can be synthesized by simply overlapping the analog audio sound.

However, in the case of the digital audio synthesizing module, as in the case of analog, the two signal lines cannot be connected to each other. Instead, a hardwired block in which two input data signals are added by software or hardware in the chip is provided. Through this, two kinds of data signals are overlapped. In FIG. 3 (B), a component denoted by 'ⓧ' between the ADC module and the multimedia chip represents the digital audio synthesis module.

The single digital audio data is then converted into compressed audio data through the audio encoder.

In addition, in order to listen to the multimedia digital audio data generated through the multimedia chip through headphones or speakers, it must first be converted into an analog signal and synthesized with the voice signal received from the counterpart terminal before being output. To this end, an audio output unit for converting the digital multimedia audio data output through the multimedia control unit into an analog audio signal, superimposing the analog audio signal and the analog voice signal received from the counterpart terminal, and outputting the same may be further added. .

FIG. 4 is a block diagram illustrating a process of generating a synthesized image by the video synthesizer 17 of FIG. 2.

First, digital multimedia video data in the form of YUV RAW data is generated by decoding and playing multimedia data including video information on a multimedia chip. The multimedia chip is equipped with a video codec for playing multimedia data in terminals such as AVI, H.264, and MPEG4.

First, the digital multimedia video data generated in the multimedia chip constitutes screen 1 (Scene 1).

An external image is captured in real time through the camera to generate digital video call image data. The digital video call video data constitutes scene 2.

In this case, since screen 1 and screen 2 are digital data of YUV or RGB type, there is no need to go through analog-to-digital conversion process as in the case of audio.

The video synthesis unit receives screen 1 and screen 2 to generate a composite image. In this case, the synthesized image does not mean data in which the screen 1 and the screen 2 are overlapped, but means a reconstructed image so that the other party can easily recognize the information of the screen 1 and the information of the screen 2 according to the user's setting.

As a method of easily displaying two images, a composite image may be generated to configure a screen in a PIP form or split the screen so that one of the screen 1 and the screen 2 is arranged as a reduced image in a part of the other one. Can be.

Alternatively, the composite image may be generated to configure the screen 1 corresponding to the digital multimedia video data in full screen. In addition, it is also possible to generate a composite image to adjust the transparency of the screen 1 and the screen 2 so that they overlap each other.

When a synthesized image is generated through the video synthesizing unit, the synthesized image is converted into compressed video data through a video encoder.

In addition, before outputting the digital multimedia video data output through the multimedia chip on the LCD screen of the terminal, a video output unit for synthesizing and outputting the video call image received from the counterpart terminal may be further added. When the screen is configured by the PIP, the video output unit generates and outputs single PIP video data so that any one of the two types of images is arranged as a reduced image in the other part of the other region.

5 is a diagram illustrating a case where (A) a camera image and a multimedia image are displayed in a PIP form on the counterpart terminal screen, and (B) a multimedia image is displayed on a full screen.

FIG. 6 is a block diagram specifically illustrating a configuration implemented to transmit ① audio compressed data and ② video compressed data to a counterpart terminal by the video call signal transmitting and receiving unit 19 of FIG. 2.

As described above, ① audio compressed data is generated through the audio encoder of FIG. 3, and ② video compressed data is generated through the video encoder of FIG. 4.

In addition, the system codec binds ① audio compressed data and ② video compressed data together to encode the compressed data efficiently for data transmission, and modulates the encoded data to the analog signal for wireless transmission. It modulates with and transmits to outside space.

7 is a flowchart illustrating the overall operation of operating a mobile communication terminal according to the present invention.

Subsequently, the components and terms described with reference to FIGS. 7, 8, and 9 are technically the same as those described above with reference to FIGS. 1 to 6, so that a detailed technical configuration for implementing each process to be described later is identified. To do this, refer to the above description.

First, a digital video call image data is generated by capturing an external image in real time through a built-in camera of a video call terminal (ST10), and receives an analog audio signal through a microphone (ST20).

When the user selects the multimedia playback command during the video call (ST30), the multimedia data stored in the terminal memory is played back, so that digital multimedia video data and digital multimedia audio data are generated (ST40), and the multimedia playback command is selected. If not, a general video call process is performed.

At this time, if the multimedia data synthesizing command is selected by the terminal user (ST50), the synthesizing process of the data for the video call and the multimedia data currently being played is performed.

In other words, analog audio signals and digital multimedia audio data are synthesized to generate single digital audio data (ST60), and the generated single digital audio data is compressed through an audio codec to generate audio compressed data (ST70).

The digital video call video data and the digital multimedia video data are used to generate composite video data according to a user's selection (ST80), and the video data is generated by compressing the composite video data through a video codec (ST90).

In addition, by encoding and modulating the generated audio compressed data and the video compressed data through the video call system codec and transmitting them to the other party's dedicated video call terminal (ST100), the video and audio of the multimedia content played in the caller's terminal are received by the receiver terminal. You can see and hear. In addition, if the receiver terminal is provided with the same function as the caller terminal, it is also possible to watch video and audio of the multimedia content played in the receiver terminal on the caller terminal.

However, when the multimedia data synthesizing command is not selected, the analog voice signal is converted into digital audio data (ST110), and the converted digital audio data is compressed through an audio codec to generate audio compressed data (ST120). A general video call processing process of compressing digital video call video data through a video codec to generate video compressed data (ST130) is performed.

FIG. 8 is a flowchart illustrating a first embodiment of synthesizing a voice and multimedia audio input through a microphone of a terminal in FIG. 7 (ST60) in more detail.

That is, following step ST50 in which the multimedia data synthesizing command of FIG. 7 is selected, digital multimedia audio data is converted into analog audio data (ST61).

The analog audio data and the analog voice signal are synthesized to generate a single analog audio signal (ST62). To synthesize two analog signals, simply overlap two signal lines.

By converting a single analog audio signal generated by the above process into a single digital audio data of the PCM method, microphone voice and multimedia audio are synthesized and converted into digital data.

FIG. 9 is a flowchart illustrating a second embodiment of synthesizing voice and multimedia audio input through a microphone of a terminal in FIG. 7 (ST60) in more detail.

First, analog voice signals are converted into digital voice data of the PCM method (ST64).

Then, the digital voice data and the digital multimedia audio data are synthesized to generate single digital audio data (ST65). In this case, since both digital data are uncompressed sound data, the synthesis of the audio signal is achieved by adding the respective values.

Although the present invention has been described in more detail with reference to the examples, the present invention is not necessarily limited to these embodiments, and various modifications can be made without departing from the spirit of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

1 is a block diagram of a mobile communication system between video call terminals;

2 is a block diagram showing the overall configuration of a mobile communication terminal according to the present invention;

FIG. 3 is a block diagram illustrating a first embodiment in which the audio synthesis unit 12 of FIG. 2 is implemented to superimpose (A) analog signals and (B) a second embodiment in which digital data is superimposed and synthesized. Degree,

4 is a block diagram illustrating a process of generating a synthesized image by the video synthesizer 17 of FIG. 2.

5 is a diagram illustrating a case where (A) a camera image and a multimedia image are displayed in a PIP form on the counterpart terminal screen, and (B) a multimedia image is displayed on a full screen.

FIG. 6 is a block diagram specifically illustrating a configuration implemented to transmit ① audio compressed data and ② video compressed data to a counterpart terminal by the video call signal transmitting and receiving unit 19 of FIG. 2;

7 is a flowchart illustrating an overall operation process of operating a mobile communication terminal according to the present invention;

FIG. 8 is a flowchart illustrating a first embodiment of synthesizing voice and multimedia audio input through a microphone of a terminal in FIG. 7 (ST60);

FIG. 9 is a flowchart illustrating a second embodiment of synthesizing voice and multimedia audio input through a microphone of a terminal in FIG. 7 (ST60) in more detail.

Claims (7)

A mobile communication terminal for processing a voice call by connecting a call with a counterpart terminal through a mobile communication network, A microphone receiving an analog voice signal; A multimedia control unit for generating digital multimedia audio data by playing multimedia data stored in the terminal; An audio synthesizer configured to selectively synthesize the analog voice signal and the digital multimedia audio data to generate single digital audio data; An audio encoder for compressing the single digital audio data through an audio codec to generate audio compressed data; And A signal transceiving unit which modulates the audio compressed data and transmits the compressed audio data to a counterpart mobile communication terminal through a mobile communication network; A mobile communication terminal configured to include; The method according to claim 1, The audio synthesis unit, A DAC module for converting digital multimedia audio data generated by the multimedia controller into an analog audio signal; An analog audio synthesizing module for generating a single analog audio signal by superimposing and synthesizing the analog audio signal output through the DAC module and the analog audio signal input through the microphone; And An ADC module for converting the single analog audio signal into single digital audio data of a PCM method; Mobile communication terminal, characterized in that configured to include. The method according to claim 2, The mobile communication terminal further comprises an audio output unit for synthesizing and externally outputting the analog audio signal output through the DAC module and the analog voice signal received from the other terminal. The method according to claim 1, The audio synthesis unit, An ADC module for converting an analog voice signal input through the microphone into digital voice data of a PCM method; And A digital audio synthesizing module for generating a single digital audio data of a PCM type by superimposing and synthesizing the digital voice data output from the ADC module and the digital multimedia audio data output through the multimedia controller; Mobile communication terminal, characterized in that configured to include. The method according to claim 4, The mobile communication terminal further includes an audio output unit for converting the digital multimedia audio data output through the multimedia control unit into an analog audio signal, synthesizing and externally outputting the analog audio signal and the analog voice signal received from the counterpart terminal. Mobile communication terminal configured by. The method according to any one of claims 1 to 5, The mobile communication terminal is a terminal having a video call function for processing a video call by connecting with a video call dedicated terminal of the other party through a mobile communication network supporting a video call. The multimedia controller further includes a function of generating digital multimedia video data by playing multimedia data stored in a terminal. A camera unit for photographing external images in real time to generate digital video call image data; A video synthesizer configured to selectively generate composite image data using the digital video call image data and the digital multimedia video data; A video encoder for compressing the composite image data through a video codec to generate video compressed data; And A video call signal transmitting / receiving unit for encoding and modulating the video compressed data together with the audio compressed data generated by the audio encoder through a video call system codec and sending the video compressed data to a counterpart video call terminal through a multimedia mobile communication network; Mobile communication terminal, characterized in that further comprises. The method according to claim 6, The mobile communication terminal uses a digital multimedia video data output through the multimedia control unit and a digital video call image data received from a video call terminal dedicated to the other, and either one of the mobile communication terminals is arranged as a reduced image in the other part of the other region. And a video output unit configured to generate single PIP video data and output the external PIP video data.

Images