JP2013201594A - Communication terminal apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a communication terminal apparatus that can share a browser, synchronize the same data with a communication terminal apparatus of a communication destination for displaying, and replay a display image including calls.SOLUTION: The communication terminal apparatus: includes a control section 110 and a storage section 160; receives a shared information concerning screen information to be shared from a first communication terminal apparatus that performed operation for starting sharing of the screen information; and displays the screen information on a display means and starts sharing of the screen information based on the received shared information, where the control section: stores the received shared information in the storage section; stores operation information in the storage section when receiving the operation information for the screen performed by the first communication terminal apparatus during sharing the screen information; and replays the screen according to the shared information and the operation information in replaying operation after sharing of the screen information is finished.

Description

  The present invention relates to a communication terminal device that is connected to an IP telephone network or the like and transmits and receives audio and video, and in particular, displays the same data with a communication terminal device of a communication partner and performs display synchronization. Both relate to a communication terminal device capable of replaying a display image including a call.

  2. Description of the Related Art In recent years, telephone devices that can receive various services such as data communication services by connecting to a wide-area communication network such as an IP telephone network or the Internet in addition to a general telephone line have become widespread. As a service for such a telephone device, a videophone service has been put into practical use. A user of a videophone service can make a voice call while transmitting and receiving a captured video by connecting a videophone device having a camera function and an image transmission function to a communication line.

  Also, various applications such as sharing memos created by each caller, sharing images obtained from other than camera shooting, sharing browsers for Internet resources, etc. A videophone device having a function of sharing data is also provided.

  For example, in the videophone device disclosed in the following Patent Document 1 (Japanese Patent Application Laid-Open No. 2004-228805), specific image data and the like related to conversation during a videophone call is transmitted simultaneously with the conversation through a network or the like by multitasking And the image data is transmitted to the other station during a call.

  Further, for example, the information processing apparatus disclosed in the following Patent Document 2 (Japanese Patent Laid-Open No. 2010-152772) has a function of a Web screen sharing / integration server, and controls event information on shared screen information shared by a shared terminal device A script, an event transmission script, and an event reception script are incorporated, and a shared screen (embedded shared screen information) in which these scripts are incorporated is transmitted to each shared terminal device.

JP 2004-228805 A JP 2010-152772 A

  In general, since a communication terminal device such as a videophone device is provided with a recording function, a call voice can be recorded and stored. Therefore, if the user records the call, the user can play back the saved call contents and check the contents again.

In addition, in a communication terminal device such as an IP videophone device, an image displayed on the screen can be stored. If a display image is recorded and stored for an arbitrary time, the user can store the stored recording data. It is possible to reproduce and reproduce the image display. However, in a conventional communication terminal device, it is common to store a display image as a moving image and to be able to reproduce (replay) it, and since it is stored as a moving image, the amount of data increases and a large storage capacity There was a problem that would be necessary.

  Further, as in the invention of the above-mentioned Patent Document 2, providing an integration server as an information processing apparatus that provides a shared screen to each terminal increases the cost because it is necessary to construct a large-scale system. This service is not always convenient for general users who use home electric appliances such as telephone devices.

  An object of the present invention is to solve the above-described problems. In a communication terminal device capable of sharing a browser, the same data is displayed with the communication terminal device of the communication partner, and is displayed. It aims at providing the communication terminal device which can replay a display image while performing a synchronization.

A communication terminal device that shares screen information with a communication partner,
The communication terminal device includes a control unit and a storage unit,
The shared information related to the screen information to be shared is received from the first communication terminal device that has performed the operation of starting the sharing of the screen information with the communication partner after displaying the screen information on the display means. Based on the information to be shared, the screen information is displayed on the display means to start sharing the screen information,
When the control unit stores the received shared information in the storage unit and receives operation information performed on the screen from the first communication terminal device while sharing the screen information , Storing the operation information in the storage unit,
When the replay operation is performed after the sharing of the screen information is completed, the control unit reproduces according to the shared information stored in the storage unit and the operation information.

  The present invention exhibits the excellent effects described below. That is, the communication terminal device according to the present invention, when starting to share screen information, displays the screen information on the display means and then performs an operation to start sharing the screen information with the communication partner. The shared information related to the screen information to be shared is received, based on the received shared information, the screen information is displayed on the display means, and the sharing of the screen information is started. When the operation information performed on the screen is received from the first communication terminal device while being stored in the storage unit and sharing the screen information, the operation information is stored in the storage unit It is. Thereby, the communication terminal device can reproduce according to the stored shared information and the operation information when there is a replay operation after the sharing of the screen information is completed. Further, since the screen information itself is not saved, it does not require a large storage capacity as in normal recording.

It is a block diagram which shows the structure of the telephone system containing the video telephone apparatus based on the Example of this invention. It is an internal block diagram which shows the structure of the video telephone apparatus based on the Example of this invention. It is the sequence diagram which showed the control in which the video telephone apparatuses based on the Example of this invention perform browser sharing. It is a figure which shows an example of the shared data preserve | saved at the video telephone apparatus of the receiving side. It is the sequence diagram which showed the control at the time of the replay of the recording data in the video telephone apparatus of the receiving side. It is a flowchart which shows operation | movement of the video telephone apparatus 100 of the transmission side in the browser sharing process based on the Example of this invention. It is a flowchart which shows operation | movement of the videophone apparatus 100 'of the receiving side in the browser sharing process based on the Example of this invention. It is a flowchart which shows the replay operation | movement of the video telephone apparatus 100 'of the receiving side in the browser sharing process based on the Example of this invention.

  Hereinafter, specific examples of the present invention will be described in detail with reference to examples and drawings. However, the embodiment described below exemplifies a videophone device for embodying the technical idea of the present invention, and is not intended to specify the present invention as this videophone device. The present invention can be equally applied to communication terminal devices of other embodiments included in the scope of claims.

  FIG. 1 is a block diagram showing a configuration of a telephone system including a videophone device according to an embodiment of the present invention. The system includes at least a videophone device 100, a wired LAN 101, an IP telephone router 102, a broadband router 103, a gateway 104, an IP telephone network 105, the Internet 106, a PSTN network 107 (Public Switched Telephone Network), and a subscription. A person telephone device 108 is included.

  The videophone device 100 of the present invention is connected to a wired LAN 101, which is connected to an IP communication network 105 via an IP telephone router 102, an Internet 106 via a broadband router 103, and a PSTN network 107 via a gateway 104. Communication is possible by connecting each. Details of the internal structure of the videophone device 100 will be described later.

  Examples of physical means constituting the wired LAN 101 include 10BASE-T (standardized as IEEE802.3i) and 100BASE-TX (standardized as IEEE802.3u) using a twisted pair cable.

The IP telephone router 102 and the broadband router 103 are network relay devices for interconnecting a plurality of IP networks. Specifically, OSI (Open Systems
Interconnection) Transfer is performed by analyzing a part of the protocol of the network layer (third layer) and the transport layer (fourth layer) in the reference model.

  In this embodiment, the IP telephone router 102 has a role of connecting two IP networks of the wired LAN 101 and the IP telephone network 105 to each other. The broadband router 103 has a role of connecting two IP networks of the wired LAN 101 and the Internet 106 to each other.

  The gateway 104 is a protocol converter for interconnecting networks having different protocol systems. For example, the gateway 104 connects the wired LAN 101 and the PSTN network 107 and performs signal conversion using a signaling protocol such as SIP, thereby enabling communication between both networks.

The IP telephone network 105 is a communication network using VoIP (Voice over Internet Protocol) technology for part or all of the telephone network, and FTTH (Fiber To The) is used as a communication line to be used.
A so-called broadband line such as Home) or ADSL (Asymmetric Digital Subscriber Line) is used. Note that a plurality of videophone devices 100 ′ are connected to the IP telephone network 105, but only one unit is shown in FIG. 1 for simplicity.

  Note that VoIP is a technology in which voice is compressed by various encoding methods, divided into packets, and transmitted in real time over an IP network. As a result, the IP telephone network 105 can provide a videophone service for transmitting and receiving images in addition to a voice call service.

  The Internet 106 is a wide area communication network constructed by interconnecting networks based on communication protocols. An international communication network is constructed by interconnecting large and small computer networks. As a communication protocol, TCP / IP is mainly adopted as a standard protocol.

  The PSTN network 107 is a general subscriber telephone line network. A subscriber telephone device 108 is connected to the terminal, and is used to make a voice call by connecting to a communication partner by a circuit switching method. The subscriber telephone device 108 uses the PSTN network 107 to allow the telephone subscriber to make a voice call with another subscriber telephone device connected to the PSTN network or with a videophone device connected to the wired LAN 101 or the IP telephone network 105. Telephone equipment.

  Next, the videophone device 100 of the present embodiment will be described with reference to FIG. FIG. 2 is an internal block diagram showing the configuration of the videophone device 100 according to the embodiment of the present invention. The videophone device 100 includes a control unit 110, a communication unit 120, an audio processing unit 130, a video and image processing unit 140, an operation unit 150, a recording unit 160, a clock unit 170, and a power supply unit 180.

  As will be described later, the videophone device 100 according to the present invention has a data sharing start operation between the videophone devices, and one videophone device 100 ′ (in the following description, the videophone device on the receiving side) ), When recording for replay is set by the user, in the one videophone device 100 ′, voice call data transmitted / received between the communication terminals during data sharing and data sharing related data sharing. The shared data is stored with a time stamp, and the data stored by the user can be reproduced later.

  The control unit 110 is a microprocessor that includes a CPU, a RAM, and a ROM. The CPU executes a control program stored in the RAM or ROM, thereby controlling and supervising the operation of each unit described below. .

  The communication unit 120 is for performing communication control when the videophone device 100 communicates with another videophone device (videophone device 100 ′ in FIG. 1). This includes control related to signaling, data transmission / reception, and data sharing processing.

  The signaling includes processing for creating, establishing, and disconnecting a communication session. Specifically, when the user operates the operation unit 150 to input a telephone number or the like, the video telephone device of the other party of the call is connected to a SIP server (not shown) connected to the IP telephone network 105 or the like. 100 'location information is acquired, the other party's videophone device 100' is called, a communication path is established when the other party answers, and a session disconnect message or the like is transmitted when the call ends. To end the session.

  For control related to data transmission / reception, IP packetization and IP packet transmission for dividing audio data, video data and image data into packets, packets received via the wired LAN 101 are rearranged in the correct order and actual data Receiving and assembling IP packets that extract and assemble parts.

When data is transmitted as a packet, a header indicating the transmission source, destination, order, transmission time, and usage (IP phone audio data, video data, image data) is added to each data divided into predetermined data lengths. . The received packet is assembled in the correct order for each of the audio, video, and image by referring to the header portion, and is output to the audio processing unit 130 and the video and image processing unit 140.

  The voice processing unit 130 performs A / D conversion on the voice input from the microphone 131, and then compresses the voice by the voice codec, and outputs the compressed data to the communication unit 120. Also, the voice data assembled in the communication unit 120 is decompressed by a voice codec, then D / A converted and output from the speaker 132.

  The video and image processing unit 140 includes a video codec, compresses video captured by the camera 141, and outputs the compressed data to the communication unit 120. Also, the video data assembled in the communication unit 120 is expanded. The video and image processing unit 140 includes an image codec, expands the image data assembled in the communication unit 120, and stores the RAM 161 of the control unit 110 and the memory 161 or the flash memory 162 constituting the recording unit 160. expand.

  The video and image processing unit 140 has a display control function. The video data output from the camera 141 and the image developed in the RAM 161 of the control unit 110 and the memory 161 constituting the recording unit 160 or the flash memory 162. Data is converted into a predetermined form that can be displayed on the display unit 142 (enlarged, reduced, images are arranged side by side, etc.), developed on a video memory (not shown) of the display unit 142, and displayed on the display unit 142. .

  The sound collected by the microphone 131 is converted into a sound signal and output to the sound processing unit 130. In addition, the audio signal output from the audio processing unit 130 is converted into audio by the speaker 132 and output. The microphone 131 and the speaker 132 may be fixedly provided on the videophone device 100, but may be a headset. In the case of a headset, it is not necessary to hold the microphone 131 by hand, so convenience is enhanced.

  The camera 141 is for photographing a subject in front of the videophone device 100 during a call. Normally, the user is positioned in front of the videophone device 100, so the camera 141 takes a picture of the user as a subject. The captured video is output to the video and image processing unit 140.

  The display unit 142 is a liquid crystal display unit, and displays video and images developed on the video memory of the display unit 142 by the display control function of the video and image processing unit 140.

  The operation unit 150 takes the form of a touch panel formed integrally with the display unit 142. The user performs an input operation by touching the part of the input icon displayed on the display unit 142. For example, when making a call, a numeric keypad for inputting a telephone number is displayed on the operation unit 150, and the user can input the number by pressing a corresponding part of the numeric keypad with a finger. Note that the operation unit 150 is not limited to the above, and may be a key or a button formed in a part of a casing constituting the videophone device 100.

  The recording unit 160 includes a memory 161 that is built in the videophone device 100 or a memory that can be connected / disconnected, such as a flash memory 162. The recording unit 160 stores data such as image data, text data, and browser information. When there is an instruction from the user, these data are transmitted as shared data to a videophone device 100 ′ as a communication partner or a server connected to the Internet 106. Further, the shared data transmitted from the videophone device 100 ′ as the communication partner and the shared data acquired from the server connected to the Internet 106 are stored.

The recording unit 160 also starts sharing data such as image data, text data, and browser information as shared data to be transmitted to the videophone device 100 ′ as the communication partner or a server connected to the Internet 106. The operation performed by the user via the operation unit 150 after the sharing is started by the operation until the sharing is ended by the sharing end operation, for example, WEB which is browser information displayed based on the URL information Operation information such as scrolling performed on the content page is stored in time series.

  Further, the recording unit 160 stores an address book in which the telephone numbers and names of a plurality of communication partners are associated with each other. The sharability information may be set in advance by the user via the operation unit 150, or may be sharable once permitted to be shared even if it is registered as a “non-sharable” communication partner. The setting may be changed as an appropriate communication partner.

  Here, data sharing refers to screen information such as image data, text data such as a memo pad, application information such as a browser, and operation information performed by a user on a screen displayed based on the screen information (hereinafter referred to as “data”) (Shared data) is transmitted to the videophone device 100 ′ of the other party of the call, the shared data is received by the videophone device 100 ′, the screen information is displayed based on the received shared data, and the shared data is displayed. An operation for the screen information is executed based on the included operation information, or the shared data transmitted from the videophone device 100 ′ of the other party is received by the videophone device 100, and the received shared data is received Based on the operation information included in the shared data and the screen information By executing the operation, the same image data, text data, and application are displayed on the display unit 142 in the same screen operation state in both the videophone device 100 used by the user and the videophone device 100 ′ that is the communication partner. This refers to the state that can be referenced.

  Furthermore, in the data sharing, in addition to the transmission / reception of the shared data in the one-to-one communication between the videophone device 100 and the videophone device 100 ′ of the communication partner, the shared data is transmitted via a server connected to the Internet 106. It is also included that transmits / receives or operates. For example, the shared data is transmitted by transmitting the shared data from the videophone device 100 to a server connected to the Internet 106, and the shared data is stored by accessing the above-mentioned server from the videophone device 100 ′ of the call partner. May be received or operated. Furthermore, the shared data transmitted from the videophone device 100 ′ as the call partner to the above-described server may be shared by accessing from the videophone device 100 so that it can be received or operated. .

  The sharing start operation refers to the shared data stored in the storage unit 160 of the videophone device 100 or the shared data temporarily stored in the RAM or the like of the control unit to the videophone device 100 ′ as the call partner. When transmitting, it means transmitting a reception request for shared data to the videophone device 100 'of the other party. Further, if the shared data is in the videophone device 100 ′ of the call partner, and the videophone device 100 ′ of the call partner wants to transmit this shared data to the videophone device 100 ′ (transmission request). To send to.

  Further, in the sharing start operation, when data is shared via a server (not shown) connected to the Internet 106, the shared data in the videophone device 100 is transmitted to the server, and the other party of the other party is called This includes transmitting a shared data reception request to the videophone device 100 ′.

  Further, in the sharing start operation, the shared data is in the videophone device 100 ′ of the call partner, and the videophone device 100 ′ of the call partner is transmitted so that the shared data is transmitted to a server connected to the Internet 106. Requesting to be included.

  The clock unit 170 is a circuit for measuring the current time, for example, using a crystal resonator that performs oscillation output at a predetermined frequency. The clock unit 170 can also manage not only time information but also calendar information related to the calendar such as the current month and day. The date / time information acquired by the clock unit 170 is used as a time stamp when storing information to be described later.

  The power supply unit 180 receives a predetermined DC voltage from the AC adapter 181 and supplies power to each component of the videophone device 100.

  When the recording setting is made by the user, the control unit 110 controls to record the voice call data transmitted / received between the videophone devices in the recording unit 160 in the meantime, and the user's recording / playback operation (replay operation) is performed. In the case, the control unit 110 reads out the corresponding voice call data from the recording unit 160 and reproduces it. In addition, when the recording setting of the display screen (for example, the display screen of the image taken by the camera of the other party's videophone device) is set by the user, the control unit 110 transmits from the other party videophone device during that time. Control is performed to record the image data in the recording unit 160, and when the user performs a recording / playback operation, the control unit 110 reads out the corresponding image data from the recording unit 160 and plays it back.

  In the control related to the data sharing process by the control unit 110, as described above, after the sharing is started by the sharing operation and before the sharing is ended by the sharing end operation, the user displays the videophone device 100 on the display. Screen information such as address information and URL information used to acquire data such as image data, text data, and browser information displayed on the unit 142 from a memory or a WEB server is stored in the recording unit 160 as shared data. Furthermore, image data, text data, and operation information such as a scroll operation for browser information (WEB content page) displayed on the display unit 142 of the videophone device 100 are stored in the recording unit 160 as shared data.

  Then, as a sharing start operation, a sharing request is notified to the video-phone device 100 ′ that is the call partner based on the user's instruction, that is, when the shared data is to be transmitted to the video-phone device 100 ′ that is the call partner. When a request for receiving shared data is transmitted to the telephone device 100 ′, and the shared data is requested to be transmitted from the videophone device 100 ′ of the other party, transmission of the shared data to the videophone device 100 ′ is performed. Send a request.

  When the sharing permission of the shared data is received from the videophone device 100 ′ of the other party in response to the above reception request, the URL information of the WEB content page stored in advance in the recording unit 160 as the shared data The screen information and operation information such as scrolling for the WEB content page are read from the recording unit 160 and transmitted to the videophone device 100 ′ of the other party.

  Further, when the shared data is received from the videophone device 100 ′ as the call partner in response to the above transmission request, or a call is made according to the sharing permission of the shared data transmitted from the videophone device 100. When the videophone device 100 receives the shared data transmitted from the other party's videophone device 100 ′, the display unit 142 is based on the URL information that is the screen information included in the received shared data. The WEB content page of the URL site is displayed on the browser activated above. Further, based on the operation information included in the received shared data, a scroll operation is performed on the WEB content page displayed on the display unit 142, and the other party's television set is also used in the videophone device 100 used by the user. Also on the telephone device 100 ′, a similar WEB content page is brought into a state in which it can be referred to on each display unit 142 in a state where the screen positions after the scroll operation coincide.

  As described above, in the videophone device 100 and the like according to the present invention, there is a data sharing start operation between the videophone devices, and one videophone device 100 ′ (in the following description, the videophone device on the receiving side) ), When recording for replay is set by the user, in the one videophone device 100 ′, voice call data transmitted / received between the communication terminals during data sharing and data sharing related data sharing. The shared data is stored with a time stamp, and the data stored by the user can be reproduced later.

  That is, when there is a sharing start operation between the communication terminal devices, and the videophone device 100 ′ sets recording for replay by the user, the control unit 110 in the one communication terminal device 100. Performs a control to add a time stamp to the voice call data transmitted and received between the communication terminals during that time based on the date and time information of the clock unit 170 and record it in the recording unit 160.

  Further, in this case, the control unit 110 starts sharing by the sharing operation and before sharing is ended by the sharing end operation, the shared data (from the videophone device 100 of the other party of sharing ( A time stamp is assigned to the operation information) based on the date and time information of the clock unit 170 and stored in the recording unit 160.

  In order to acquire data such as image data, text data, and browser information displayed on the display unit 142 of the videophone device 100 from the memory or the WEB server as shared data transmitted from the videophone device 100 of the other party of sharing. Address information, URL information, and operation information such as a scroll operation for browser information (WEB content page). Here, it is operation information such as a scroll operation for address information, URL information, and browser information (WEB content page) used to acquire at least data such as browser information from a memory or a WEB server. Each of the shared data is given a time stamp based on the date / time information of the clock unit 170 and stored in the storage unit 160.

  Thereby, in one videophone device 100 ′, when the user later starts replay of stored data at the time of sharing, after the sharing is started by the sharing operation, the sharing is ended by the sharing end operation. The image and voice call data displayed in between can be reproduced. In this case, since the time stamp is given to the voice call data and the operation information such as the scroll operation for the browser information (WEB content page), the voice data and the shared browser information are simultaneously reproduced based on the time stamp. Then, the call voice and the display image can be synchronized and replayed. Further, since the information on the shared browser screen is stored using the URL and its operation information instead of the displayed image information, the recorded data can be stored in a compact manner, and the storage unit 160 has a large storage capacity. Does not require capacity.

  In the videophone device 100 or the like, it is preferable that a recording mode for enabling the above-described storage operation can be set when the user desires to replay the call voice and display screen during sharing. Further, it is preferable that the user input the identification name of the stored data when setting the recording mode so that the replay target can be selected. When the user does not set the identification name, the telephone number or call date / time of the sharing partner may be automatically set. This setting is preferably performed in a communication session establishment step described later.

Hereinafter, with reference to FIGS. 3 to 5, the browser sharing process and the recording / replay process in the videophone devices 100 and 100 ′ of the present embodiment will be described in detail. 3 is a sequence diagram showing control for sharing browsers between videophone devices according to an embodiment of the present invention, and FIG. 4 is a shared object stored in the videophone device on the receiving side (recording side). FIG. 5 is a sequence diagram showing an example of data, and FIG. 5 is a sequence diagram showing control at the time of replaying recorded data in the videophone device on the receiving side (recording side). In the following description, a sharing start operation is performed in the videophone device 100 (FIG. 3, terminal A), and sharing is started with the videophone device 100 ′ (FIG. 3, terminal B). In the videophone device 100 ′, processing when the user performs recording setting will be described.

  In step S301 (see FIG. 3), the user operates the operation unit 150 of the videophone device (terminal A) 100 to connect the other party's videophone device (terminal B) 100 ′ connected to the IP telephone network 105. Enter the phone number. Alternatively, the user operates the operation unit 150 to call the address book stored in the recording unit 160 and select a call partner. Then, the communication unit 120 searches for the other party on the basis of the input telephone number or the information on the selected other party, calls the video telephone 100 'of the other party, and in step S302, the other party If the response is received, a communication session is established and a call is started. The call may be started from the videophone device 100 on the user side. Conversely, the videophone device 100 ′ on the other end of the call calls the videophone device 100 on the user side, and the user answers the call. A communication session may be established and a call may be started.

  In step S303, audio and video are transmitted and received between the videophone device 100 and the videophone device 100 'of the other party. Specifically, the voice input from the microphone 131 is A / D converted by the voice processing unit 130 and then compressed by the voice codec. The video captured by the camera 141 is displayed on the display unit 142 by the display control function provided in the video and image processing unit 140 and is compressed by the video codec provided in the video and image processing unit 140. Further, the compressed audio data and video data are each divided into IP packets in the communication unit 120 and transmitted to the wired LAN 101, and the transmitted audio data and video data are reproduced and displayed on the video telephone apparatus 100 ′ of the call partner. Is done.

  On the other hand, when voice data and video data transmitted as IP packets from the videophone device 100 ′ of the other party of call are received by the videophone device 100 on the user side, the communication unit 120 receives the IP packets as voice data, video data, etc. Divide by type, extract the actual data part from each divided packet, and assemble in order. Next, the audio data is decompressed by the audio codec of the audio processing unit 130, then D / A converted, output to the speaker 132, and output as audio. Further, the video data is expanded by a video codec included in the video and image processing unit 140, edited in a form that can be displayed on the display unit 142 by the display control function, and displayed on the display unit 142.

  The sequence up to this point is the same procedure as that for calling and displaying images between ordinary IP videophone devices.

  Next, the user activates a WEB browser on the display unit 142 of the videophone device 100, operates the operation unit 150 to input a URL, and accesses a WEB server (not shown) connected to the Internet 106. Make a page request. The videophone device 100 acquires data of a desired WEB page from the WEB server, and displays the WEB page on the browser on the display unit 142. The input URL information is stored in the storage unit 160 as shared data.

After such a normal browser operation is executed, when a sharing start operation is performed based on a user instruction in step S304, the videophone device 100 receives the shared data from the videophone device 100 ′. The request is transmitted, and the sharing permission of the shared data is received from the videophone device 100 ′ of the other party in response to the reception request.

  In step S305, the videophone device 100 reads the URL information of the WEB content page stored in the recording unit 160 from the recording unit 160, and transmits the URL information to the videophone device 100 'of the other party. Based on the received URL information, the videophone device 100 ′ of the call partner displays the WEB content page of the URL site on the browser activated on the display unit 142. Thereby, sharing of a browser is started.

  After the sharing of the browser is started, when the user performs a scroll operation by operating the operation unit 150 on the WEB page displayed on the browser on the display unit 142 in the videophone device 100, the call partner's TV is displayed. Operation information A to C (for example, scroll movement direction, scroll movement amount, key cursor, etc.) so that the same scroll operation can be performed on a similar WEB page displayed on the browser on the display unit 142 of the telephone device 100 ′. The coordinate position of the mouse pointer) is stored in the recording unit 160. In step S306, the stored operation information is read from the storage unit 160 by the control unit 110 and transmitted to the videophone device 100 'as shared data.

  As a result, the same WEB content page is displayed on each display unit 142 in a state in which various operations such as a scroll operation are reflected in both the videophone device 100 and the videophone device 100 ′ of the other party of the call. can do.

  On the other hand, in the videophone device 100 ′ of the other party (receiving side), the control unit 110 adds the shared data (URL information) transmitted from the videophone device 100 in step S305 based on the date / time information of the clock unit 170. A time stamp is given and stored in the recording unit 160 as shared data. Further, the control unit 110 assigns a time stamp to the shared data (operation information) transmitted from the videophone device 100 in step S306 based on the date / time information of the clock unit 170, and saves it as the shared data in the recording unit 160. To do.

  This processing is continued until a sharing end operation is performed in the videophone device 100 in step S307. When a sharing end operation is performed in the videophone device 100 in step S307, the line is disconnected in step S308 and the communication session is ended. To do.

  FIG. 4 shows an example of shared data stored in the recording unit 160 in the videophone device 100 ′ of the other party (receiving side). As shown in FIGS. 1 to 7, the recording unit 160 is given a time stamp based on the date and time information obtained by the clock unit 170, and URL information and operation information are stored in time series. The The operation information includes, for example, whether the mouse is moved up and down and its operation position coordinates (UP / DOWN and coordinates x, y), or whether the mouse is moving and its operation position (MOVE and coordinates x, y). Is recorded as follows. The voice data of the voice call during this time is similarly given a time stamp by the control unit 110 and stored in the storage unit 160 in time series.

  Next, processing when replaying (reproducing) audio data and screen information stored in the storage unit 160 in the videophone device 100 ′ of the other party (receiving side) will be described. FIG. 5 is a sequence diagram showing control at the time of replaying recorded data in the videophone device 100 'on the receiving side.

As shown in FIG. 5, the replay process is a single process in the videophone device 100 ′ of the other party (receiving side). When the replay is set by the user and the stored information recorded in the storage unit 160 is selected, the corresponding call voice data and shared data are read from the storage unit 160 and the time stamp given to the data Based on the time series. As a result, replay is performed.

  Next, details of browser sharing processing, recording, and replay processing in the videophone devices 100 and 100 ′ of this embodiment will be described with reference to FIGS. 6 to 8. 6 is a flowchart showing the operation of the transmitting-side videophone device 100 in the browser sharing process according to the embodiment of the present invention. FIG. 7 is a receiving-side videophone in the browser sharing process according to the embodiment of the present invention. FIG. 8 is a flowchart showing the replay operation of the videophone device 100 ′ on the receiving side in the browser sharing process according to the embodiment of the present invention.

  First, the operation of the transmitting-side videophone device 100 according to the embodiment of the present invention will be described with reference to FIG. In step S601, the control unit 110 activates a browser initialized to display the WEB content page based on the operation of the operation unit 150 by the user. In step S602, the user operates the operation unit 150 to operate the URL. To determine whether the operation has started.

  Next, when the control unit 110 determines in step S602 that the URL has been input and the operation has been started, the process proceeds to step S603, and the control unit 110 performs a normal browser operation. That is, a WEB page request is made to the WEB server connected to the Internet 106 based on the input URL information, the desired WEB page data is acquired from the WEB server, and the WEB page is displayed on the browser on the display unit 142. Is displayed. At this time, the control unit 110 stores the URL information input to display a desired WEB page in the recording unit 160.

  On the other hand, in step S604, the control unit 110 determines whether or not the browser operation is terminated without inputting a new URL. If the operation has ended, the process returns to step S602. If the operation has not ended, in step S605, the control unit 110 determines whether a sharing start operation has been performed based on a user instruction. If the control unit 110 determines in step S605 that a sharing start operation has been performed, the process proceeds to step S606. If it is determined that a sharing start operation has not been performed, the process returns to step S603.

  In step S606, control unit 110 transmits a shared data reception request to videophone apparatus 100 'to attempt a shared session connection. In step S607, the control unit 110 determines whether the shared session connection is successful. As this condition, it is determined that the shared session connection is successful when the sharing permission of the shared data is received from the videophone device 100 'of the other party of the call in response to the transmitted reception request. If it is determined that the shared session connection has failed, the process proceeds to step S614. In step S614, the control unit 110 notifies the failure of the shared session connection through the display unit 142, and the process returns to step S603.

  In step S607, when the control unit 110 determines that the shared session connection is successful, the process proceeds to step S608, where the URL data of the WEB content page stored in the recording unit 160 is read from the recording unit 160 and the other party is called. To the videophone device 100 ′.

Next, in step S609, the control unit 110 determines whether or not the user has started an operation such as a scroll operation by operating the operation unit 150 with respect to the WEB page displayed on the browser on the display unit 142. . When it is determined that the scroll operation has been performed, the control unit 110 performs an operation on the normal browser in step S610, proceeds to step S610, stores operation data corresponding to the scroll operation in the recording unit 160, and performs the process of step S611. , The operation data stored in the recording unit 160 is read and transmitted to the videophone device 100 ′ of the other party.

  In step S612, control unit 110 determines whether or not the browser operation has ended. If the operation has ended, the process returns to the process of step S609. If the operation has not ended, it is determined in the process of step S613 whether or not a sharing end operation has been performed in the videophone device 100. If it is determined in step S613 that the sharing end operation has been performed, the process returns to step S603. If it is determined that the sharing end operation has not been performed, the process returns to step S609. Note that the operation of this flowchart ends when the communication session ends.

  Next, the operation of the videophone device 100 'on the receiving side according to the embodiment of the present invention will be described with reference to FIG. First, in step S701, the control unit 110 activates a browser initialized to display a WEB content page based on the operation of the operation unit 150 by the user, and the user operates the operation unit 150 in step S702. Then, it is determined whether the operation is started by inputting the URL.

  Next, when the control unit 110 determines in step S702 that a URL has been input and an operation has started, the process proceeds to step S703, and the control unit 110 executes a normal browser operation. That is, a WEB page request is made to the WEB server connected to the Internet 106 based on the input URL information, the desired WEB page data is acquired from the WEB server, and the WEB page is displayed on the browser on the display unit 142. Is displayed. On the other hand, if the control unit 110 determines in step S702 that the URL has not been input and the operation has not started, the process proceeds to step S705.

  In step S703 in which a normal browser operation is being performed, the control unit 110 determines whether or not the browser operation has ended. If the operation is complete, the process returns to step S702. If the operation is not complete, the process returns to step S703.

  On the other hand, in step S705, control unit 110 determines whether or not a sharing start operation has been performed from videophone device 100 of the other party of the call. As this condition, it is determined that a sharing start operation has been performed when a shared data reception request is received from the videophone device 100 of the other party of the call. If it is determined that a sharing start operation has been performed, sharing permission of shared data is transmitted to the videophone device 100 of the other party of the call, and the process proceeds to step S706. When it is determined that the sharing start operation is not performed, the control unit 110 returns to the process of step S702.

  Next, in step S <b> 706, the control unit 110 starts recording the call voice (storing in the storage unit 160) with a time stamp based on the date / time information of the clock unit 170. In step S707, the initialized browser is activated, and in step S708, it is determined whether URL data has been received from the videophone device 100 of the other party. If it is determined that the URL data has been received, the control unit 110 causes the browser to read the WEB content page of the URL site based on the received URL data and display it on the display unit 142 in step S709. In the process of step S710, The control unit 110 adds a time stamp to the received URL information and stores it in the storage unit 160. Then, the process returns to step S708.

If it is determined in step S708 that URL data is not received, in step S711, the control unit 110 determines whether operation data has been received from the videophone device 100 of the other party of the call. When determining that the operation data (for example, scroll operation information) has been received, the control unit 110 performs a scroll operation on the WEB content page displayed on the display unit 142 based on the received operation data in step S712. The screen operation performed by the user on the WEB content page in the videophone device 100 is reflected. In step S713, the control unit 110 adds a time stamp to the received operation information and saves it in the storage unit 160 based on the date / time information of the clock unit 170. Then, the process returns to step S708.

  On the other hand, if it is determined in step S711 that the operation data has not been received, the process proceeds to step S714, where it is determined whether a sharing end operation has been performed in the videophone device 100 of the other party of the call. If it is determined in step S714 that the sharing end operation has been performed, in step S715, the control unit 110 ends the recording of the call voice (stored in the storage unit 160), and the process in step S703. Return to.

  If it is determined in step S714 that the sharing end operation has been performed, the process returns to step S708. Note that the operation of this flowchart ends when the communication session ends.

  Next, the replay operation in the videophone device 100 'on the receiving side according to the embodiment of the present invention will be described with reference to FIG. First, in step S <b> 801, the control unit 110 determines whether or not a replay operation is started by designating data to be replayed stored in the storage unit 160 based on an operation of the operation unit 150. If the replay operation is not started, the process returns to step S801.

  If the replay operation is started, the process proceeds to step S802, where the control unit 110 activates a browser initialized to display the WEB content page, and the corresponding call stored in the storage unit 160 in step S803. Start audio playback (replay).

  Next, in step S804, control unit 110 determines whether or not call recording reproduction has ended. If the reproduction is finished, the process returns to the process of step S801. If the reproduction is not finished, the control unit 110 refers to the date / time information acquired from the clock unit 170 in the process of step S805, and the elapsed time is stored in the storage unit 160. It is determined whether it is the time of the time stamp attached to the corresponding stored data. If it is not the time of the time stamp, the process returns to step S804.

  If it is the time of the time stamp, in the process of step S806, the control unit 110 determines whether or not the data that matches the time of the time stamp among the corresponding stored data stored in the storage unit 160 is URL data. If it is URL data, in the processing of step S807, the control unit 110 causes the browser to read the WEB content page of the URL site based on the URL data and display it on the display unit 142, and returns to the processing of step S804.

On the other hand, if the data matching the time of the time stamp is not URL data, the process proceeds to step S808, and the control unit 110 determines whether the data matching the time of the time stamp is operation data. If it is not operation data, the process returns to step S804. If it is operation data, the process proceeds to step S809, and the control unit 110 performs the operation, for example, the scroll operation, on the WEB content page displayed on the display unit 142 based on the operation data. The videophone device 100 plays back the screen operation performed on the WEB content page, and the process returns to step S804. The operation of this flowchart is ended by a replay end operation or the end of reproduction of stored data.

  As described above in detail, according to the present invention, when the screen information sharing is started, the screen information is displayed on the display means and then the operation for starting the sharing of the screen information with the communication partner is performed. Receiving shared information related to the screen information to be shared from one communication terminal device, displaying the screen information on a display unit based on the received shared information, and starting sharing the screen information; From the first communication terminal device to the screen while the screen information is shared. When the operation information performed on the operation information is received, the date and time information acquired from the clock unit is referred to, and a time stamp is given to the operation information and stored in the storage unit. Thereby, the communication terminal device can reproduce the stored shared information and the operation information according to the time stamp when the replay operation is performed after the sharing of the screen information is completed. Further, since the screen information itself is not saved, it does not require a large storage capacity as in normal recording.

  In addition, according to the communication terminal device of the present invention, when the screen information sharing is started, the control unit stores the call voice in the storage unit. Therefore, after the screen information sharing ends, the replay is performed. When there is an operation, the stored call voice can be reproduced, and the call voice can be easily replayed together with the screen replay.

  Further, according to the communication terminal device of the present invention, since the call voice refers to the date / time information acquired from the clock unit, the operation information is given a time stamp and stored in the storage unit. It is possible to synchronize with the replay of the call voice.

100 Videophone device 101 Wired LAN
102 IP telephone router 103 Broadband router 104 Gateway 105 IP telephone network 106 Internet 107 PSTN network 108 Subscriber telephone device 110 Control unit 120 Communication unit 130 Audio processing unit 131 Microphone 132 Speaker 140 Video and image processing unit 141 Camera 142 Display unit 150 Operation Unit 160 recording unit 161 memory 162 flash memory 170 clock unit 180 power supply unit 181 AC adapter

Claims (1)

A communication terminal device that shares screen information with a communication partner,
The communication terminal device includes a control unit and a storage unit,
The shared information related to the screen information to be shared is received from the first communication terminal device that has performed the operation of starting the sharing of the screen information with the communication partner after displaying the screen information on the display means. Based on the information to be shared, the screen information is displayed on the display means to start sharing the screen information,
When the control unit stores the received shared information in the storage unit and receives operation information performed on the screen from the first communication terminal device while sharing the screen information , Storing the operation information in the storage unit,
The control unit plays back according to the shared information stored in the storage unit and the operation information when a replay operation is performed after the sharing of the screen information is completed.

Images