CN101102510A - Audio and video synchronization method for portable image terminal - Google Patents
Audio and video synchronization method for portable image terminal Download PDFInfo
- Publication number
- CN101102510A CN101102510A CNA200610086329XA CN200610086329A CN101102510A CN 101102510 A CN101102510 A CN 101102510A CN A200610086329X A CNA200610086329X A CN A200610086329XA CN 200610086329 A CN200610086329 A CN 200610086329A CN 101102510 A CN101102510 A CN 101102510A
- Authority
- CN
- China
- Prior art keywords
- video
- voice
- time parameter
- synchronous
- video camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Mobile Radio Communication Systems (AREA)
- Telephone Function (AREA)
- Telephonic Communication Services (AREA)
Abstract
The method comprises: using a portable video terminal to make a video call; during the video call, if the video camera shuts down abnormally and doesn't stop the de-compression operation of the video multimedia digital signal decoder, it calculates its time parameter value; if the video camera is started up, the last time parameter value is added into the image layer of the video bit-stream in order to sync the audio signal and video signal.
Description
Technical field
The present invention relates to a kind of image terminal, relate in particular to when mobile image models such as video call or video recording, the Voice ﹠ Video method for synchronous of the portable image terminal of the asynchronous problem of Voice ﹠ Video that the solution video camera produces.
Background technology
Fig. 1 is that the portable multimedia mobile communication system constitutes schematic diagram.The 1st multimedia mobile terminal machine (to call image terminal in the following text) 1 communicates with the 2nd image terminal 2 by carrying out wireless link 3 with mobile communications network 4.21,2 of terminating machines not only carry out multimedia transmission, and transmitting control data.
In order to carry out video call, be transferred to the other side after the image modulation that described image terminal 1,2 will capture by video camera.Meanwhile,, and after relying on inner codec to carry out demodulation, play by display unit 7 and loud speaker from the other side's receiving multimedia data, so just can realize and the other side between video call.
Concerning original video telephone, do not consider following situation: during using video telephone, video camera is closed, and perhaps in transmission of audio and video process, closes video suddenly.Therefore, if this thing happens,, audio/video synchronization is interrupted according to the audio/video synchronization setting means.
Original video telephone synchronization settings mode is to utilize time parameter (TR:Temporal Reference) numerical value of image layer (Picture layer) H.263 to contrast.Time parameter (TR) uses when setting frame frequency (frame rate), according to the frame frequency that sets, decides the interval of each interframe.This at interval and be time parameter.Time parameter is 8bit, has 0～255 numerical value, if this numerical value is bigger than 255, just calculate since 0 again.Video compression is formed by picture compression one by one, so when a frame is compressed, just generate image layer (Picture layer), just has time parameter (TR) numerical value of calculating this moment.Therefore, continue together to carry out before video compression finishes, having this numerical value just can be synchronous.
Fig. 2 is that the image layer of H.263 data flow constitutes schematic diagram.Compressed each frame all has image layer, and behind picture start sign indicating number (PSC:Picture Start Code) 22bit, time parameter (TR) numerical value is to 8bit.
H.263 bit data flow has form as shown in Figure 2.Narration (descriptor) to each several part is as follows.
Picture start sign indicating number (PSC; Picture start code) beginning of display image.Time parameter (TR; Temporal Reference) form by the following method: the numerical value of reference image data head before omitting later on according to former transmission data of time, or add that non-number with reference to image increases by 1 and forms.
Image type (PTYPE) comprises the detailed image modulation type, promptly comprises intranet or internet.Before PQUANT relied on any both sides' phone information to upgrade, what it was shown was both sides' phone modulation language that image is used.CPM is the symbol language that continues there is multiple spot and use multi-channel video transmission (Continuous Presence Multipoint) mode signal.If when only the sub-Bit data stream picture of PSBI image device-CPM being provided with, when having the bi-directional predicted frame of TRB frame, when having (as can be known) DBQUANT twocouese frame by the PB-frame, exist PEI about residue insertion information, for the existence of the data field (PSUPP and PEI) that shows the next option, it is set to " 1 ".PSUPP and PEI be extend information as a supplement together, as defining among the appendix L that is attached to H.263.
Here, the bit rate that described time parameter demonstration is set or the interFrameGap of frame per second, it is the numerical value between 0～255.There is following relation: TR (Current)=TR (Previous)+frame interval in described time parameter numerical value, (0≤TR＜256, TR is a fixed number).
Therefore, utilize the time parameter numerical value that calculates, just can know the quantity of the incoming frame that skip (SKIP).Therefore, just can make between Voice ﹠ Video synchronously.
For example, if use video camera according to per second 30 frames (30fps) input, H.263 7.5fps compresses according to per second, in ideal conditions, every interframe be spaced apart 4, and show 0,4,8,12 ..., 252,0 ... construction form.
Fig. 3 is when ideally the per second video camera of importing 30 frames (30fps) compresses according to 7.5fps, between time parameter and audio frequency, video synchronously.Can carry out video call in this way.
If carrying out video call, close video camera, when perhaps not importing, according to the setting of video camera, in existing video telephone, just video can be in the state of supspending.At this moment, the increase of time parameter numerical value H.263 just can stop.In this case, though video is ended, audio frequency also can continue compression and transmission, can hear sound to convenient.Here, if open the setting of video camera again, just the state of Fig. 4 can occur.
As shown in Figure 4, the video camera time parameter is that 8 o'clock (T1) stops, and video multimedia digital signal codec just can be out of service.At this moment, audio frequency continues output, and video camera (T2) is preceding opening, and continues to occur slowed-down video (Video delay) (T1-T2).Even the video camera operate as normal is because can't match with the time parameter numerical value that reality increases, so before the calculating unanimity of time parameter, can produce delay (delay) together (Td).At this constantly, the data of actual compression lump together with audio frequency, so just stationary problem can occur between Voice ﹠ Video.This is because time parameter numerical value causes as increasing the data existence all the time.
Summary of the invention
For addressing the above problem, the Voice ﹠ Video method for synchronous that the invention provides a kind of portable image terminal solves the stationary problem that the video camera On/Off is produced in the process of video call.The present invention relies on the variation of time parameter numerical value that slowed-down video is not taken place, thereby the influence that not closed by video camera, proceed compression, after increasing time parameter numerical value, in the moment that video camera is opened, join in the image layer as the final time parameter values, making can be synchronous between Voice ﹠ Video.
To achieve these goals, the Voice ﹠ Video method for synchronous of portable image terminal of the present invention is made of following three steps: in portable image terminal, carry out the step of video call; In carrying out process of video call, if video camera cuts out, make video multimedia digital signal codec not stop compression, but proceed compression, and calculate the step of time parameter numerical value; If described video camera is opened, the final time parameter values in this moment is joined in the image layer of video data stream, make the synchronous step of Voice ﹠ Video.
The Voice ﹠ Video method for synchronous of portable image terminal of the present invention comprises following two steps: under the mobile image model of terminating machine, no matter whether video camera improper mistake takes place, can make video multimedia digital signal codec proceed compression, and increase the step of time parameter numerical computations; If described video camera normally moves, will join in the video data stream at the time parameter numerical value that this moment calculates at last, make the synchronous step of Voice ﹠ Video.
Described video data stream is data flow H.263 preferably.
As mentioned above, the Voice ﹠ Video method for synchronous of portable image terminal of the present invention has following effect: when carrying out video call, can make video multimedia digital signal codec not be subjected to the influence of video camera On/Off to continue operation, reduce the audio/video asynchrony phenomenon takes place between video and audio frequency.
Simultaneously, the present invention also has following effect: when video camera temporarily is in run-stopping status, if audio frequency is at first opened, and video pictures occurs thereafter, can prevent the phenomenon that can not carry out simultaneously because of the asynchronous eyesight that causes of Voice ﹠ Video and the sense of hearing in advance.
Description of drawings
Fig. 1 is that the portable multimedia mobile communication system constitutes schematic diagram;
Fig. 2 is that the image layer of H.263 data flow constitutes schematic diagram;
Fig. 3 is that the video call Voice ﹠ Video of prior art is in synchronous regime figure;
Fig. 4 is the schematic diagram of prior art when timing error appears in Voice ﹠ Video when carrying out video call;
Fig. 5 is a portable image terminal pie graph of the present invention;
Fig. 6 be of the present invention when carrying out video call Voice ﹠ Video be in the synchronous regime schematic diagram;
Fig. 7 is the Voice ﹠ Video method for synchronous flow chart of portable image terminal of the present invention.
The symbol description of major part in the＜accompanying drawing 〉
11... radio-frequency unit 12... mobile base station modem chip
13... apparatus for processing audio 14... loud speaker
15... microprocessor 16... press key input device
17...DSP chip 17a... video multimedia digital signal codec
18... camera head 19...LCD device
20... storage device
Embodiment
Fig. 5 is the Voice ﹠ Video synchronizer pie graph of portable image terminal of the present invention.
As shown in Figure 5, radio-frequency unit (RF device) 11 has the radio communication function of portable telephone.Described radio-frequency unit 11 is made of following two parts: make the signal frequency of transmission that the RF transmitter that rising changes and amplifies take place; The signal low noise that receives is amplified, and the RF receiver that frequency is reduced.
Mobile base station modulator-demodulator (MSM) chip 12 is made of following two parts: the modulation transmitter that the signal that send is modulated; Carry out the demodulate reception machine of demodulation etc. to the received signal.That is, described mobile base station modem chip 12 is made of modulator-demodulator (MODEM) and Audio Codec (CODEC).Here, described codec comprises following two parts: the data codec of processing packing data etc.; The Audio Codec 12b of audio signals such as processing sound.Simultaneously, Audio Codec 12b is as the built-in sound codec (speech codec) of pocket telephone, can with and the other side between when conversing the voice signal of input encode.Simultaneously, MSM chip 12 carries out integral body control under the video call pattern, and the temporal information of the timer clock that 12a is generated is provided.
Apparatus for processing audio 13 is play the audio signal of the Audio Codec 12b output of described mobile base station modem chip 12 by loud speaker 14, or the input audio signal that microprocessor 15 generates is transferred to the Audio Codec 12b of described mobile base station modem chip 12.
Press key input device 16 by can input digit and the button of Word message and the function key of setting various functions constitute.Simultaneously, described press key input device 16 can also possess following button in embodiments of the present invention: the function key that is used to control mobile image video recording pattern; The shooting key that is used for actuated camera.Simultaneously, in an embodiment of the present invention, also can import synthetic text signal in synthetic mobile picture signal by described press key input device 16.
DSP (Digital Signal Processor:DSP) chip 17 has following function: the image signal encoding that camera head 18 is captured become can display image signals picture data.The picture signal of 17 pairs of camera heads of described dsp chip, 18 outputs is that unit handles with the frame, and described frame image data is exported according to the characteristic and the size of LCD device 19.Simultaneously, described dsp chip 17 also has video multimedia digital signal codec 17a, H.263 described video multimedia digital signal codec 17a can be, picture signal is encoded according to the mode of setting, perhaps will be decoded into original frame image data through the frame image data of coding.Described dsp chip 17 is under the control of described MSM chip 12, and the pattern according to setting generates or play mobile picture signal.
Camera head (camera module) 18 is by constituting with the lower part: photographic map as the time, be the camera sensor of the signal of telecommunication with the converting optical signals that perceives; The analog signal conversion that to take the photograph from described camera sensor is the signal processing apparatus of digital signal.Here, described camera sensor is assumed to be ccd sensor, and simultaneously, described camera sensor and signal processing apparatus can have integral structure, also can separate simultaneously.
In the present invention, in carrying out process of video call, even video camera cuts out, not closed by video camera influences, and video multimedia digital signal codec 17a proceeds compression.Thus,, also can proceed compression, so along with the time flows, time parameter numerical value also has certain increase because after video camera cuts out.After this, when opening video camera, be added in the image layer, produce delay, also can solve the stationary problem of Voice ﹠ Video even video camera cuts out video as the final time parameter values.
As shown in Figure 6, video camera close moment (T11) time parameter numerical value is 8 o'clock, and video multimedia digital signal codec is proceeded compression during carrying out video call.When carrying out this video compression, continue the transmission time parameter values.
After this, opening video camera (T12) constantly,,, also can solve the stationary problem between Voice ﹠ Video even video camera cuts out the generation slowed-down video if add the final time parameter values.For example video camera is opened the final time parameter values in the moment (T12) and join in the image layer, just can realize synchronously.
Fig. 7 is the Voice ﹠ Video method for synchronous flow chart of portable image terminal of the present invention.
As shown in Figure 7, carry out video call (S11).At this moment, judge whether video camera closes (S13).If video camera is closed, video multimedia digital signal codec is proceeded compression, and increases TR numerical value (S15).After this judge whether video camera is opened (S17) again.If video camera is unlocked, the last TR numerical value of described increase is joined (S19) in the image layer, realize between video and audio frequency synchronously.
In the present invention, not only when video call, and under the video recording pattern, even video camera abnormal running, video multimedia digital signal codec also can continue operation, and continue to increase time parameter numerical value, in the normal operation of equipment, can realize the synchronous of Voice ﹠ Video.
By above-mentioned explanation, those skilled in the art can carry out various change and modification fully in the scope that does not depart from the technology of the present invention thought.Therefore, technical scope of the present invention is not limited to the content of specification, must determine its technical scope according to the claim scope.
Claims (4)
1, a kind of Voice ﹠ Video method for synchronous of portable image terminal is characterized in that, may further comprise the steps:
In portable image terminal, carry out the step of video call;
In carrying out process of video call, if video camera cuts out, make video multimedia digital signal codec not stop compression, but proceed compression, and calculate the step of time parameter numerical value;
If described video camera is opened, the final time parameter values in this moment is joined in the image layer of video data stream, make the synchronous step of Voice ﹠ Video.
2, the Voice ﹠ Video method for synchronous of portable image terminal as claimed in claim 1 is characterized in that, closes from video camera that to begin to the delay that the video camera open period is caused be the delay of video.
3, a kind of Voice ﹠ Video method for synchronous of portable image terminal is characterized in that, may further comprise the steps:
In portable image terminal, under the mobile image model of terminating machine, no matter whether video camera improper mistake takes place, and can make video multimedia digital signal codec proceed compression, and increases the step of time parameter numerical computations;
If described video camera normally moves, will join in the video data stream at the time parameter numerical value that this moment calculates at last, make the synchronous step of Voice ﹠ Video.
As the Voice ﹠ Video method for synchronous of claim 1 or 3 described portable image terminals, it is characterized in that 4, described video data stream is a data flow H.263.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNA200610086329XA CN101102510A (en) | 2006-07-07 | 2006-07-07 | Audio and video synchronization method for portable image terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNA200610086329XA CN101102510A (en) | 2006-07-07 | 2006-07-07 | Audio and video synchronization method for portable image terminal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101102510A true CN101102510A (en) | 2008-01-09 |
Family
ID=39036560
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNA200610086329XA Pending CN101102510A (en) | 2006-07-07 | 2006-07-07 | Audio and video synchronization method for portable image terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101102510A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2010124499A1 (en) * | 2009-04-27 | 2010-11-04 | 中兴通讯股份有限公司 | Method and terminal for synchronously recording sounds and images of opposite ends based on circuit domain video telephone |
| CN101271720B (en) * | 2008-04-22 | 2011-06-22 | 中兴通讯股份有限公司 | Synchronization process for mobile phone stream media audio and video |
| CN116761030A (en) * | 2023-08-11 | 2023-09-15 | 南京汉卫教育科技有限公司 | Multi-machine-bit synchronous audio and video recording and playing system based on image recognition algorithm |
-
2006
- 2006-07-07 CN CNA200610086329XA patent/CN101102510A/en active Pending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101271720B (en) * | 2008-04-22 | 2011-06-22 | 中兴通讯股份有限公司 | Synchronization process for mobile phone stream media audio and video |
| WO2010124499A1 (en) * | 2009-04-27 | 2010-11-04 | 中兴通讯股份有限公司 | Method and terminal for synchronously recording sounds and images of opposite ends based on circuit domain video telephone |
| US8493429B2 (en) | 2009-04-27 | 2013-07-23 | Zte Corporation | Method and terminal for synchronously recording sounds and images of opposite ends based on circuit domain video telephone |
| CN116761030A (en) * | 2023-08-11 | 2023-09-15 | 南京汉卫教育科技有限公司 | Multi-machine-bit synchronous audio and video recording and playing system based on image recognition algorithm |
| CN116761030B (en) * | 2023-08-11 | 2023-10-27 | 南京汉卫教育科技有限公司 | Multi-machine-bit synchronous audio and video recording and playing system based on image recognition algorithm |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP2001251584A (en) | Coded data recorder | |
| CN101583009A (en) | Video terminal and method thereof for realizing interface content sharing | |
| KR20050121536A (en) | A method and a apparatus of synchronization video signal with audio signal for mobile phone | |
| US9917931B2 (en) | Radio communication system, radio apparatus, radio communication method, packet generation method, and method for reproducing data from packet | |
| CN101102510A (en) | Audio and video synchronization method for portable image terminal | |
| US8120636B2 (en) | Method for performing recording operation in video communication mode of wireless terminal and wireless terminal implementing the same | |
| CN100479517C (en) | Method for superposing voice in transmitting audio-video file | |
| CN1929647B (en) | Method for performing video communication service and mobile communication terminal employing the same | |
| US8634770B2 (en) | Portable terminal device | |
| CN100563334C (en) | In the video telephone mode of wireless terminal, send the method for view data | |
| CN101695120A (en) | Method and system for conducting video recording notification | |
| CN107205131A (en) | A kind of methods, devices and systems for realizing video calling | |
| US9838514B2 (en) | Radio communication system, radio apparatus, radio communication method, packet generation method, and method for reproducing data from packet | |
| CN113422997A (en) | Method and device for playing audio data and readable storage medium | |
| KR100669622B1 (en) | Synchronization method for audio and video of mobile picture phone | |
| KR100836609B1 (en) | Mobile communication terminal for frame rate controlling and its controlling method | |
| CN100499425C (en) | Multifunctional handset capable of recording and playing speech | |
| JP6646239B2 (en) | Wireless communication system, wireless device, wireless communication method, method of generating packet, and method of reproducing data from packet | |
| CN101005665A (en) | Method for transmitting message in a wireless terminal | |
| KR100711713B1 (en) | A mobile communication terminal having a function of transmitting the real time streaming data | |
| JP6781392B2 (en) | Wireless communication system, radio, wireless communication method, packet generation method and data reproduction method from packet | |
| KR100617564B1 (en) | A method of multimedia data transmission using video telephony in mobile station | |
| TW200515804A (en) | Method and device for transmitting image data | |
| KR100498345B1 (en) | Switching circuit for speech path of mobile phone | |
| KR100605995B1 (en) | Method for performing memo of voice in wireless terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |