JP2006310964A - Communication terminal and control method thereof, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide sender side and receiver side communication terminals connected with each other via a network whereby a system clock can automatically be synchronized with each other and the receiver side communication terminal can display video and audio transmitted from the sender communication terminal asynchronously. <P>SOLUTION: One of the communication terminals connected to the network 105 outputs a clock synchronization packet for synchronizing the system clock as a clock master and the other communication terminal receives the clock synchronization packet as a clock slave so as to synchronize the system clock and to synchronize the time of NTP time stamp between the communication terminals. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a communication terminal device, a control method therefor, and a program, and in particular, communication that synchronizes a system clock between terminals in a communication terminal device in which a plurality of terminals cooperate to generate a plurality of synchronized media streams. The present invention relates to a terminal device, a control method thereof, and a program.

  In recent years, images taken by a network camera are often viewed from a remote location via various communication media such as the Internet, and data such as video and audio signals are frequently transmitted and received using a videophone or the like. It has become like this.

  A communication terminal that generates and transmits a plurality of synchronized media streams such as video and audio, such as a network camera and a video phone, includes an audio encoding unit and a video decoding unit in one terminal, and thereby uses audio and video decoding units. The video signal is encoded, and the encoded data is transmitted to the receiving terminal via the network. At this time, a time stamp or the like is added to each media stream in the encoded data so that audio and video are displayed in synchronization with each other on the receiving terminal side.

  FIG. 6 is a block diagram showing an internal configuration of a system configured with conventional communication terminals.

  In FIG. 6, the communication terminal 500 includes a camera unit 501 for capturing video, a microphone unit 502 for inputting audio, and a video encoding unit 503 that encodes a video signal input from the camera unit 501. In addition, the audio encoding unit 504 that encodes the audio signal input from the microphone unit 502, the system clock is supplied to the video encoding unit 503 and the audio encoding unit 504, and the clock is counted to generate time stamp information. RTP that performs packetization processing for flowing the encoded data and time stamp information to the network based on information from the system clock unit 505, the video encoding unit 503, the audio encoding unit 504, and the system clock unit 505 (Real-time Transport Protocol) / RTCP (Real Time Streaming Protocol) transmission processing unit 506 and RTP / RTCP transmission The packetized data by the processing unit 506 and a network I / F unit 507 to be transmitted to the network.

  The communication terminal 600 includes a network I / F unit 508 for receiving data from the network, an RTP / RTCP reception processing unit 509 that separates and processes received data into audio / video / clock information, and RTP / RTCP reception. Based on clock information from the processing unit 509, a system clock recovery unit 510 that recovers a transmission-side system clock, a video decoding unit 511 that decodes video encoded data, and an audio decoding unit 512 that decodes audio encoded data. And a display unit 513 for displaying the decoded video signal and a speaker unit 514 for outputting the decoded audio signal.

  RTP / RTCP (RFC1889) is well known as a protocol for real-time transfer on an IP network. System clock synchronization between the transmission side and the reception side in RTP / RTCP is performed based on an RTP time stamp in an SR (Sender Report) included in RTCP sent from the transmission side to the reception side. In other words, the RTP time stamp in the SR is the RTP time stamp at the time when the SR was transmitted. Therefore, the RTP time stamp in the received SR is compared with the time counted by the system clock of the own terminal, and the clock is synchronized. Be taken.

Also, the RTP time stamp reference clock may be different for each media such as video and audio. In this case, synchronization between the media cannot be achieved only by the time stamp, but the RTP time stamp in the SR and the NTP ( The relationship between the NTP time stamp common to the media and the time stamp for each media is derived from the relationship between the time stamps of the Network Time Protocol, and synchronization between the media can be achieved.
JP 2003-153546 A

  The applicant has developed a communication terminal that can be used for a more flexible application such that a plurality of communication terminals are operated in cooperation by connecting the communication terminals via a network. If such a communication terminal can be constructed, it is conceivable that the degree of freedom of installation can be further expanded by performing monitoring by combining a plurality of network cameras and network audio input terminals (network microphones).

  In such a case, if the communication terminal that generates the media stream is different, an NTP time stamp serving as a time reference is generated for each terminal, and thus there is a problem that the NTP time stamps between the terminals cannot be synchronized.

  The present invention has been made in view of the above problems, and can automatically synchronize a system clock between a plurality of communication terminals connected to each other via a network, and an image transmitted from the communication terminal. It is an object of the present invention to provide a communication terminal device, a control method thereof, and a program that can be displayed on a communication terminal on the receiving side without any deviation from the voice.

  In order to achieve the above object, the communication terminal device according to claim 1 communicates with a plurality of communication terminal devices connected via a network, and the time generated from the reference clock in the real-time data to be transmitted In a communication terminal device to which information is added, a reference clock output means for outputting the reference clock, a reference clock synchronization signal transmitting means for transmitting a reference clock synchronization signal for synchronizing with the reference clock, and the reference clock synchronization signal A reference clock synchronization signal receiving means for receiving, and a communication that, when receiving the reference clock synchronization signal, synchronizes the reference clock between the communication terminal devices by the reference clock synchronization signal and outputs the time information as the reference clock. Synchronization means for synchronizing with the terminal device.

  In order to achieve the above object, a method for controlling a communication terminal apparatus according to claim 5 performs communication with a plurality of communication terminal apparatuses connected via a network and generates real-time data to be transmitted from a reference clock. In the control method of the communication terminal device to which the time information is added, a reference clock output step of outputting the reference clock, a reference clock synchronization signal transmission step of transmitting a reference clock synchronization signal for synchronizing with the reference clock, A reference clock synchronization signal receiving step for receiving the reference clock synchronization signal; and when receiving the reference clock synchronization signal, the reference clock synchronization signal is used to synchronize the reference clock between the communication terminal devices, and the time information is And a synchronization step for synchronizing with a communication terminal device that outputs a reference clock. .

  According to the present invention, communication is performed between a plurality of communication terminal devices connected via a network, and the communication terminal device that adds time information generated from the reference clock to real-time data to be transmitted is synchronized with the reference clock. When receiving a reference clock synchronization signal, the reference clock synchronization signal synchronizes the reference clock between the communication terminal devices, and the time information is synchronized with the communication terminal device that outputs the reference clock. The system clocks can be automatically synchronized between a plurality of communication terminals connected to each other, and can be displayed on the communication terminal on the receiving side without the video and audio transmitted from the communication terminal being shifted.

  Further, according to the present invention, information on the accuracy of the reference clock is exchanged between the communication terminal devices, and the communication terminal device having a higher accuracy of the reference clock is caused to transmit the reference clock synchronization signal. Thus, a more stable system can be constructed.

  According to the present invention, the time information is synchronized with the communication terminal device that outputs the reference clock after the synchronization of the reference clock between the communication device terminals is established. An efficient system that does not require time adjustment can be constructed.

  Further, according to the present invention, when the communication terminal device that outputs the reference clock synchronization signal is disconnected from the network, communication is performed between the communication terminal devices, and the communication terminal device that outputs the reference clock synchronization signal is determined. A more flexible system can be constructed using the communication terminal device.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram showing an overall configuration of a network system including communication terminals according to an embodiment of the present invention.

  In FIG. 1, network cameras 101 and 102 are communication terminals that encode captured video data and transmit it as real-time data. The network microphone 103 is a communication terminal that encodes a collected audio signal and transmits it as real-time data. The network camera 101 includes a clock synchronization unit 104a, and the network camera 102 includes a clock synchronization unit 104b. The network microphone 103 includes a clock synchronization unit 104c.

  In this network system, the network camera 101 serves as a clock master, and a clock synchronization packet for synchronizing the system clock in each communication terminal apparatus is broadcast on the network 105 at regular intervals. The network camera 102 and the network microphone 103 that receive the clock synchronization packet serve as clock slaves, and establish synchronization between their own system clock and the system clock of the network camera 101 by the clock synchronization packet.

  In each communication terminal operating as a clock slave, an NTP (Network Time Protocol) time stamp (time information) is generated using a system clock that is synchronized with the network camera 101 as a clock master as a reference clock. By synchronizing and synchronizing the time of the NTP timestamp generated from this system clock, it is possible to synchronize the media streams output from each communication terminal on the receiving terminal side of the clock synchronization packet. Note that a computer, a server, a device, and the like (not shown) may be installed on the network system. Needless to say, each communication terminal may be an information processing apparatus including a CPU, a RAM, a memory, and the like.

  Next, a method of synchronizing the system clock will be described with reference to FIG.

  FIG. 2 is a block diagram illustrating an internal configuration of the clock synchronization units 104a to 104c in FIG.

  In FIG. 2, the clock synchronization units 104a to 104c are composed of the same elements, and an oscillator 201 that outputs a system clock, a period counter 202 that counts the system clock and generates an NTP time stamp, and a received clock synchronization packet An oscillator control unit 203 that controls the oscillator 201 by comparing the value counted by the period counter 202 is provided.

  In the clock synchronization unit 104 a of the network camera 101 that is the clock master, the oscillator control unit 203 does not control the oscillator 201, and a system clock with a constant frequency is output from the oscillator 201. The period counter 202 counts a certain number of clocks to output a clock synchronization packet, and outputs a clock synchronization packet including an NTP time stamp when the count value reaches a set value.

  The operation of the clock synchronization units 104b and 104c in the network camera 102 and the network microphone 103 that are clock slaves will be described.

  The clock synchronization units 104b and 104c receive clock synchronization packets including the NTP time stamp at regular intervals and pass them to the oscillator control unit 203. The oscillator control unit 203 compares the interval of the NTP time stamp in the received clock synchronization packet with the interval of the value counted by the period counter 202 at the time of reception, and the receiving terminal side (here, the network camera 102 and the network microphone 103). If it is determined that the system clock period is longer than the system clock period on the transmitting terminal side (here, the network camera 101), control is performed to increase the oscillation frequency of the oscillator 201, and the system clock on the receiving terminal side is controlled. When it is determined that the interval is shorter than the cycle of the system clock on the transmission terminal side, control is performed to lower the oscillation frequency of the oscillator 201. Thereby, the system clock on the receiving terminal side can be synchronized with the system clock on the transmitting terminal side.

  Next, processing for determining a clock master from among a plurality of communication terminals will be described with reference to FIG.

  FIG. 3 is a flowchart showing processing for determining a clock master. This process is a process executed by each communication terminal on the network 105.

  In FIG. 3, the communication terminal connected to the network 105 first determines whether or not the clock synchronization packet output from the clock master is flowing on the network 105 (step S301), and the clock synchronization packet is transferred to the network 105. Is exchanged with the clock master that outputs the clock synchronization packet (step S304).

  On the other hand, when the clock synchronization packet is not flowing on the network 105, there is no clock master on the network 105, so whether or not there is a communication terminal (device) that can become the clock master on the network 105. In order to confirm the above, a device discovery packet is broadcasted (step S302).

  Next, when a communication terminal other than the own terminal that can become the clock master has been found (YES in step S303), information exchange with the communication terminal is performed (step S304). On the other hand, when the communication terminal cannot be found (NO in step S303), the communication terminal waits for a certain time (step S308) and broadcasts a device discovery packet again.

  In step S304, information exchange is performed when a clock master is found on the network 105 or a communication terminal other than the own terminal that can become the clock master is found. At this time, the communication terminals that can oscillate a clock with higher accuracy by exchanging capabilities such as the accuracy of the system clock of the own terminal and the priority to become the clock master are exchanged between the communication terminals. If these communication terminals have the same accuracy and ability, either one may be the clock master, and the clock master is determined so that it does not become the clock master at the same time.

  Next, it is determined whether or not it has become the clock master (step S305). If it has become the clock master, transmission of a clock synchronization packet is started (step S306). On the other hand, when it becomes a clock slave instead of a clock master, it starts receiving a clock synchronization packet and synchronizes its own system clock with the system clock on the transmitting terminal side (step S307).

  In this way, the system clock accuracy information is exchanged between the communication terminals, and the communication terminal with higher system clock accuracy becomes the clock master.

  Even when a new communication terminal is connected to the network 105, the new communication terminal and the clock master exchange information, and the communication terminal having a more accurate system clock becomes the clock master. When the communication terminal that has become the clock master does not exist on the network for some reason, there is no clock synchronization packet on the network, and the device discovery packet is transmitted in step S302.

  Next, an NTP timestamp adjustment process performed between communication terminals will be described with reference to FIG.

  FIG. 4 is a flowchart showing time adjustment processing of the NTP time stamp.

  In FIG. 4, the communication terminal that has become a clock slave by the processing of FIG. 3 receives the clock synchronization packet from the communication terminal that has become the clock master, and uses the system clock of its own terminal as the system clock of the clock master in the clock synchronization unit. Synchronize.

  When the clock synchronization packet is received (step S401), the number of system clocks representing the received time difference between the previously received clock synchronization packet and the currently received clock synchronization packet is synchronized with the system clocks of the clock master and the clock slave. The number of clocks is compared with each other, and the shift amount of the system clock between the clock master and the clock slave is measured. At this time, it is determined whether or not the measured deviation amount of the system clock between the clock master and the clock slave is within an allowable range (step S402). If it is determined that the deviation amount of the system clock exceeds the allowable range and is large (step S402) A control value for controlling the oscillator 201 is calculated based on the amount of deviation, and the oscillator 201 that generates the system clock is controlled (step S405).

  On the other hand, if the amount of clock deviation is within the allowable range (YES in step S402), it is determined that the system clock of the clock master and the system clock of the own terminal are synchronized, and the NTP included in the clock synchronization packet is determined. The time stamp is compared with the NTP time stamp generated by the internal system clock, and the amount of deviation of the time stamp (time) is measured. At this time, it is determined whether or not the measured deviation amount of the time stamp (time) is outside the allowable range (step S403). If the deviation amount of the time stamp exceeds the allowable range (YES in step S403), the clock slave The time of the NTP clock is adjusted with the clock master.

  FIG. 5 is a schematic diagram of the time synchronization of the NTP clock performed between the clock slave and the clock master.

  In FIG. 5, the clock slave transmits a time adjustment packet to the clock master as a time adjustment request packet (T11). On the clock master side, as a response packet to this request, information of the time (T22) when the request packet is received and the time (T23) when the response packet is transmitted is added to the clock slave.

  The clock slave calculates the time difference between the NTP time stamp generated by the clock master from each time of T11, T14, T22, and T23 and the NTP time stamp generated by the clock slave, and the NTP time stamp of its own terminal is the NTP time of the clock master. Set to match the time stamp.

  According to the above embodiment, between a plurality of communication terminals connected to the network 105, one of the communication terminals outputs a clock synchronization packet for synchronizing the system clock as a clock master, and the other one of the communication terminals As a clock slave, a clock synchronization packet is received, the system clock is synchronized between the communication terminals, and the time of the NTP time stamp is adjusted, so that a system is automatically established between a plurality of communication terminals connected to each other via a network. Clock synchronization can be achieved, and video and audio transmitted from the communication terminal can be displayed on the communication terminal on the receiving side without deviation.

  In addition, since a communication terminal that outputs a system clock with higher accuracy is a reference (clock master), a more stable system can be constructed.

  In addition, since the NTP time stamp is synchronized after the synchronization of the system clock is established, an efficient system that does not require time adjustment can be constructed.

  In addition, even when a communication terminal that outputs a system clock is disconnected from the network 105, a communication terminal that automatically synchronizes a new reference clock is determined, so that a more flexible system can be constructed.

  An object of the present invention is to supply a storage medium storing software program codes for realizing the functions of the above-described embodiments to a system or apparatus, and a computer (or CPU, MPU, or the like) of the system or apparatus as a storage medium. This can also be achieved by reading and executing the stored program code.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention.

  Examples of the storage medium for supplying the program code include a floppy (registered trademark) disk, a hard disk, a magneto-optical disk, a CD-ROM, a CD-R, a CD-RW, a DVD-ROM, a DVD-RAM, and a DVD. -RW, DVD + RW, magnetic tape, nonvolatile memory card, ROM, etc. can be used. Alternatively, the program code may be downloaded via a network.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) running on the computer based on the instruction of the program code. A case where part or all of the actual processing is performed and the functions of the above-described embodiments are realized by the processing is also included.

  Further, after the program code read from the storage medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. This includes the case where the CPU or the like provided in the board or the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

It is a figure which shows the whole network system structure comprised with the communication terminal which concerns on embodiment of this invention. It is a block diagram which shows the internal structure of the clock synchronizers 104a-104c of FIG. It is a flowchart which shows the process which determines a clock master. Time stamp time adjustment flowchart It is a schematic diagram of the time adjustment of the NTP clock performed between a clock slave and a clock master. It is a block diagram which shows the internal structure of the system comprised with the conventional communication terminal.

Explanation of symbols

101, 102 Network camera 103 Network microphone 104a, 104b, 104c Clock synchronization unit 201 Oscillator 202 Period counter 203 Oscillator control unit

Claims (9)

In a communication terminal device that performs communication with a plurality of communication terminal devices connected via a network and adds time information generated from a reference clock to real-time data to be transmitted,
Reference clock output means for outputting the reference clock;
A reference clock synchronization signal transmitting means for transmitting a reference clock synchronization signal for synchronizing with the reference clock;
Reference clock synchronization signal receiving means for receiving the reference clock synchronization signal;
A synchronization means for synchronizing a reference clock between communication terminal devices by the reference clock synchronization signal when receiving the reference clock synchronization signal, and synchronizing the time information to a communication terminal device that outputs the reference clock; A communication terminal device. Information exchanging means for exchanging information on the accuracy of the reference clock between the communication terminal devices;
The communication terminal apparatus according to claim 1, further comprising a control unit that causes the communication terminal apparatus having a higher accuracy of the reference clock to transmit the reference clock synchronization signal.   3. The synchronization means synchronizes the time information with a communication terminal device that outputs the reference clock after the synchronization of the reference clock between the communication device terminals is established. The communication terminal device described.   When a communication terminal apparatus that outputs the reference clock synchronization signal is disconnected from the network, the communication terminal apparatus further includes a determining unit that performs communication between the communication terminal apparatuses and determines a communication terminal apparatus that outputs the reference clock synchronization signal. The communication terminal device according to any one of claims 1 to 3, wherein In a control method for a communication terminal apparatus that performs communication with a plurality of communication terminal apparatuses connected via a network and adds time information generated from a reference clock to real-time data to be transmitted.
A reference clock output step of outputting the reference clock;
A reference clock synchronization signal transmission step of transmitting a reference clock synchronization signal for synchronizing with the reference clock;
A reference clock synchronization signal receiving step for receiving the reference clock synchronization signal;
A synchronization step of synchronizing the reference clock between the communication terminal devices by the reference clock synchronization signal when the reference clock synchronization signal is received, and synchronizing the time information to the communication terminal device that outputs the reference clock. A control method for a communication terminal device. An information exchange step for exchanging information on the accuracy of the reference clock between the communication terminal devices;
The communication terminal apparatus control method according to claim 5, further comprising a control step of causing the communication terminal apparatus having a higher accuracy of the reference clock to transmit the reference clock synchronization signal.   7. The synchronization step, wherein the time information is synchronized with the communication terminal device that outputs the reference clock after the synchronization of the reference clock between the communication device terminals is established. The communication terminal device control method described.   When the communication terminal device that outputs the reference clock synchronization signal is disconnected from the network, the communication terminal device further includes a determination step of performing communication between the communication terminal devices and determining a communication terminal device that outputs the reference clock synchronization signal. The communication terminal apparatus control method according to claim 5, wherein the communication terminal apparatus is controlled.   A computer-readable program for causing a communication terminal device to execute the control method according to any one of claims 5 to 8.

Images