JP2004229270A - Telephone communication system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system in which a videophone call can be made with a mobile videophone by utilizing a camera or a display device installed on the street for the mobile telephone that is conventionally incapable of utilizing services in the videophone. <P>SOLUTION: In the videophone call with the mobile videophone, a media converter in a service center sorts data to send moving image data to a public communication terminal on the street and to send audio data to a mobile telephone that does not include a moving image photographing and reproducing function. A call agent in the service center issues an ID in response to a request from the mobile telephone that does not include the moving image photographing and reproducing function under a voice call. The ID is received in the service center via a detector such as a sensor installed on the street, and video telephone services are performed by using the public communication terminal on the street around the detector. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a telephone communication system, and more particularly, to a mobile phone having no moving image capturing / reproducing function, and a videophone using a camera and a display device installed in a public or street.
[0002]
[Prior art]
FIG. 13 shows a conventional technique. In the figure, reference numeral 1 denotes a mobile phone having no moving image capturing / reproducing function, 6 denotes a mobile videophone, 8 denotes a mobile phone network, and 80 denotes an audio format conversion server for the mobile phones 1 and 6.
[0003]
In the prior art, only a voice call is provided by the mobile phone network 8 in a call between the mobile phone 1 having no moving image capturing / reproducing function and the mobile videophone 6, and the PCM voice is converted by the voice format conversion server 80 as necessary. The format is converted between data and digitally compressed voice data to realize voice communication.
[0004]
Also, in a videophone system using packet communication of a mobile phone network disclosed in "Videophone system in a complex mobile device" of Japanese Patent Application Laid-Open No. 2001-32672 (Patent Document 1), a simple mobile phone is used in addition to a mobile phone. A video phone unit equipped with a PHS radio is prepared, and a mobile phone is provided with a separate radio for a simple mobile phone PHS to secure a communication path with the video phone unit. The videophone unit realizes a videophone by packet communication of a mobile phone network via a mobile phone via this communication path.
[0005]
[Patent Document 1]
JP 2001-32672 A
[0006]
[Problems to be solved by the invention]
In the prior art, a videophone function and a voice call are possible through the mobile videophone 6. In addition, the portable videophone 6 and the mobile phone 1 having no moving image shooting / playing function can perform only voice communication through the audio format conversion by the media converter of the service center, and the mobile phone 1 without the moving image shooting / playing function can be used. Since there is no video recording / playback function, it was not possible to physically make a videophone call.
[0007]
Further, in the technology of Patent Literature 1 “Video phone system in complex mobile device”, a user must own or rent a mobile phone equipped with a video phone unit and a simple mobile phone PHS radio. For this reason, in some cases, the user has to buy a special non-conventional mobile phone equipped with a simple mobile phone PHS radio.
[0008]
The user has an unconventional specialty equipped with a video phone unit and a simple mobile phone PHS radio.
It is necessary to carry a portable telephone at all times and always keep both connected for standby, and the battery life is short and lacks practicality.
[0009]
In addition, since a packet network operated in a best-effort manner in which the communication band is not compensated is used, not only the band required for the moving image data cannot be secured, but also the overhead of relaying in a portable telephone equipped with a simple portable telephone PHS radio. Therefore, the delay amount increased, and the frequency of missing images was high overall, which was not practical.
[0010]
As described above, the conventional mobile phone and the technology described in Patent Literature 1 can solve the incompatibility with the mobile video phone, and realize a band for providing a sufficient service and a good delay amount. Did not.
[0011]
An object of the present invention is to provide that a call can be made between a mobile phone and a mobile videophone.
It is another object of the present invention to provide a system capable of making a videophone call using a mobile phone having no moving image shooting / playback function.
[0012]
[Means for Solving the Problems]
The present invention relates to a first telephone terminal having a function of performing voice and data communication, moving image shooting, and compressing / expanding and playing back a shot moving image in a telephone communication system, and a function of shooting a moving image or compressing / expanding a shot moving image. Then, it is possible to make a call with a second telephone terminal that does not have at least one of the functions of reproducing and displaying. For this reason, a service center having a call agent connected to a communication network and performing call control on a telephone terminal as a communication partner, and a media converter for converting image data and voice data, a display device and a photographing device are provided. In addition, a public communication terminal connected to a service center is used. The service center performs call control between the first and second telephone terminals via a call agent, and includes video data and audio data via a media converter for the first telephone terminal. Communication is performed in a format, and moving image data is transmitted to and received from a public communication terminal via a media converter.
[0013]
In a preferred example, a camera or a display device is installed as a public communication terminal, for example, on the street or in an office, and a service center remotely controls them. The service center distributes the video data to the public communication terminal for the videophone call with the mobile videophone (first telephone terminal), and transfers the audio data to the mobile phone (second telephone terminal) having no video shooting and playback functions. Turn around. The call agent of the service center issues an identifier (ID) in response to a request from a mobile phone that does not have a moving image capturing / reproducing function during a voice call, and this ID is displayed on the mobile phone. When the user indicates the ID displayed on the mobile phone to a detector such as a sensor or a scanner installed on the street, the detection result of the ID is transferred to the service center. The service center activates a public communication terminal installed around the detector to provide a videophone service to the public communication terminal. In this way, the user makes a videophone call using a public communication terminal on the street and a mobile phone having no moving image reproduction function.
[0014]
BEST MODE FOR CARRYING OUT THE INVENTION
The first embodiment will be described with reference to FIGS.
In FIG. 1, reference numeral 1 denotes a mobile phone without a moving image capturing / reproducing function, and 2 denotes a street sensor.
Alternatively, a camera, 3 is a street camera, 4a is a service center, 6 is a mobile videophone, 7 is the Internet, 8 is a mobile phone network, and 9 is a street display device.
[0015]
In FIG. 1, a user starts a call via the mobile phone network 8 between the mobile phone 1 having no moving image capturing / reproducing function and the mobile videophone 6. The service center 4a is composed of a plurality of call agents and a plurality of media converters, and is distributed and arranged on the mobile communication network 8, but is shown as a service center 4a in FIG. For a call from a mobile phone, the call agent calls the other party's mobile phone. If the audio format used between the mobile phones is different, the media converter of the service center 4a performs the audio format conversion between, for example, the PCM or half-rate (PSI-CELP) system and the AMR audio compression / decompression according to the instruction of the call agent. Do. With this conversion, in the present embodiment, a voice call between the mobile phone 1 and the mobile videophone 6 having no moving image capturing / reproducing function is realized.
[0016]
During a call, the user calls the service center 4a from the mobile phone 1 having no moving image capturing / reproducing function, requests an ID for receiving the videophone service, and obtains the ID. The ID includes, for example, a user's telephone number and time, an identification number assigned to a session between mobile phones during a call, a charging code during charging, a sequence number of a charging code, and a sequence of numbers that can be easily handled by the service center 4a. String, binary code. In addition to sending the ID as it is to the mobile phone, the service center 4a may generate a barcode and send it as bitmap image data.
[0017]
In the first embodiment, a voluntary ID request by a user is shown. In addition, the service center 4a recognizes that the call is a call with the portable videophone 6, and the portable service center 4a does not have a moving image photographing / reproducing function. In some cases, an ID for receiving the videophone service is sent to the telephone 1 to urge the use of the videophone.
[0018]
In addition, a telephone number stored in a SIM card incorporated in the mobile phone 1 having no moving image capturing / reproducing function, a phone-specific number, or these numbers and time, and a base station of the mobile phone network 8 being connected. An ID may be automatically generated from a combination with a unique number or the like.
[0019]
The requested ID generates a barcode on the mobile phone 1 having no moving image photographing / reproducing function and displays it on the screen. The user removes the mobile phone 1 having no moving image capturing / reproducing function from his / her ear during a call, and indicates the mobile phone 1 to the sensor or camera 2 on the street. The sensor or camera 2 scans and demodulates the ID and notifies the center server 4a. During this time, the connection between the mobile phones for the call is maintained, and the user brings the mobile phone 1 without the moving image capturing and reproducing function close to the ear and resumes the call.
[0020]
The street sensor or camera is a general CCD or CMOS sensor or camera, and performs scanning in the visible light range as described above. Since a CCD, CMOS sensor, or camera can receive infrared light by blocking visible light, it is also possible to present an ID by infrared communication using the IrDA infrared data communication port of the mobile phone.
[0021]
The service center 4a that has confirmed the validity of the presented ID activates the camera 3 on the street and starts photographing. The photographed image is sent to a mobile phone 1 having no moving image capturing / reproducing function in a so-called still image format such as JPEG or GIF, and is displayed on a screen of the mobile phone 1 having no moving image capturing / reproducing function. The user confirms from the display contents that the camera position and the user themselves are captured, and notifies the service center 4a of the confirmation. In this confirmation, the service center 4a obtains images from the cameras 3 on a plurality of streets around the user, sequentially sends the images to the mobile phone 1 having no moving image shooting / reproducing function, and allows the user to select the streets used for communication. The camera 3 can also be determined.
[0022]
Upon receiving the notification, the service center 4a asks the portable videophone 6 to change from voice communication to videophone. When the mobile videophone 6 receives the change to the videophone, the media converter of the service center 4a receives the video from the camera 3 on the street and flows through the mobile phone network 8 and does not have the video shooting / playback function. The data is superimposed on the data and converted into the data format of the videophone and transmitted to the mobile videophone 6.
[0023]
At the same time, the mobile videophone 6 transmits moving images and audio to the mobile phone network 8 in the videophone data format. The media converter of the service center 4a separates the data into a moving image and a sound, converts the data as necessary, and transmits the data to the mobile phone 1 having no moving image photographing / reproducing function. The moving image data is transmitted to the display device 9 on the street, and is displayed as a so-called picture-in-picture simultaneously with the moving image data from the camera 3 on the street. The picture-in-picture display is realized by combining or displaying moving image data from six portable videophones and three camera shots on the street on the street display device 9, respectively, in separate areas. It can also be realized by displaying data on the display device 9.
[0024]
In the first embodiment, a street camera 3 and a street sensor or camera 2 are installed on the street. However, by capturing an image of a user with the street sensor or camera 2, the camera 3 Alternatively, the street camera 3 may be omitted.
[0025]
In FIG. 2, reference numeral 11 denotes a two-dimensional barcode, 12 denotes a still image of a user photographed by a street camera, and 106 and 108 denote buttons. When the user presents the ID, a two-dimensional barcode 11 is displayed on the screen. It may be a one-dimensional barcode or bitmap data substituted for it. When the user completes the presentation of the barcode, the user presses the key of the mobile phone 1 having no moving image capturing / reproducing function associated with the button 106 to terminate the display of the barcode.
[0026]
When the service center 4a confirms the validity of the presented ID, it activates the street camera 3 and sends the photographed image in a still image format to the mobile phone 1 without the moving image photographing / reproducing function, and the photograph is taken by the street camera. It is displayed as a still image 12 of the user. The user confirms from the display contents that the camera position and the user are in the image, and presses the key of the mobile phone 1 having no moving image shooting / reproducing function associated with the confirmation button 108 to notify the service center 4a. I do.
[0027]
A series of operation flows of the present system will be described with reference to FIG.
As shown in FIG. 1, after the start of the voice call 101, the user of the mobile phone 1 without the moving image capturing / reproducing function tries the ID acquisition 103 following the center call 102 and obtains the ID. The ID is presented by IrDA 105b by IrDA or is displayed on the screen of the mobile phone 1 having no moving image photographing / reproducing function by the ID display 105a via the barcode generation 104 and presented to the sensor or camera 2 on the street.
[0028]
When the service center 4a confirms the validity of the ID, an image photographed by the camera 3 on the street is obtained at a photographing start 107 after the camera activation 106. The photographed image is sent in a still image format to the mobile phone 1 having no moving image shooting / playing function, prompting a public TV phone operation check 108, and when confirmed, notifying the mobile video phone 6 of changing from a voice call to a video phone, When accepted, the service center 4a starts the transmission video / audio synchronization processing 109 and transmits data in a videophone format to the mobile videophone 6.
At the same time, the data in the videophone format from six mobile videophones is displayed on the display device 9 on the upper street of the incoming video separation 901 in the service center 4a. When a call is made as a videophone call with a call continuation 110 and a disconnection 111 is reached, the display device 9 on the street also ends display 902, and the videophone call is completed.
[0029]
In FIG. 4, reference numeral 41 denotes an image input, 42 denotes an audio input, 42a denotes a codec, and 43 denotes an automatic editing machine.
FIG. 4 shows data conversion to a videophone format in the media converter of the service center 4a. The media converter of the service center 4a has a moving image input 41 and an audio input 42, and when the audio format needs to be converted, the input audio is converted into an appropriate format by the codec 42a and input to the automatic editor 43, respectively.
[0030]
Since a time stamp is added to each of the moving image and the sound, the automatic editing machine 43 compares the time stamps to find a set of almost the same moving image data and sound data, and converts this set to a videophone format. In this system, moving image data tends to be missing or delayed with respect to audio data, and the media converter of the service center 4a has the shortest data transfer path from the camera 3 on the street before switching to a videophone call. A media converter is searched and a path is switched to suppress an increase in delay. Regardless of the presence or absence of the path control, the larger delay between the time stamp of the audio data and the time stamp of the moving image data with respect to the real time referenced by the automatic editing machine 43 is used as the delay amount at the input terminal of the automatic editing machine 43. . By doing so, the audio data of the time stamp older than the time (the transmission deadline) obtained by subtracting the delay amount of the automatic editing machine 43 from the actual time is held in the buffer of the automatic editing machine 43, and the almost simultaneous time stamp is set. When the moving image data is input, the moving image data is combined with the moving image data and converted into a videophone format. When the transmission deadline matches the time stamp, the data is output and transmitted to the mobile videophone 6. The automatic editor 43 converts the video data and the audio data in the video phone format into video video formats on the basis of the set as a reference, and outputs and transmits the video phone format when the time stamps match the transmission deadline.
[0031]
When moving image data is lost and the transmission deadline is reached, moving image data used for conversion to the videophone format is substituted with the previous moving image data. When the automatic editing machine 43 determines that the moving image data is missing, the substitute is repeated as long as the missing data continues. When a moving image that should have been lost at a certain point of time arrives, the substitute is repeated as a frame that substitutes the moving image data that has arrived as long as the moving image arrives. When the vehicle arrives collectively as in the dumpling operation, the moving image data other than the one with the latest time stamp is discarded, and a new delay amount is calculated from this time stamp. The automatic editing machine 43 determines that a chronic delay has occurred, and updates the transmission deadline based on this delay amount. The audio data to be converted to the videophone format at the time of updating the update period is corrected and supplemented by pitch control, or is supplemented by silent data.
[0032]
In FIG. 5, reference numeral 44 denotes a video / audio separation, 45 denotes a video output, 46 denotes an audio output, and 46a denotes a codec.
[0033]
FIG. 5 shows separation of a moving image and a sound from the videophone format in the media converter of the service center 4a. The media converter of the service center 4a has a video / audio separation 44, a video output 45, and a voice output 46. When conversion of a voice format is required, the separated voice data is converted into an appropriate format by the codec 46a and output. The output moving image data is transmitted to the street display device 9, and the audio data is transmitted to the mobile phone 1 having no moving image photographing / reproducing function via the cellular phone network 8. It is reproduced on the mobile phone 1 having no function.
[0034]
As described above, as shown in FIGS. 1 to 5, a service center that cooperates with a camera and a display device on the street is installed in a call between a mobile phone and a mobile video phone that does not have a moving image capturing / reproducing function, in which a video call cannot be made only by a conventional voice call. Then, the service center converts the moving image data and the audio data into a videophone format, transmits the videophone format to the videophone, separates and reproduces and displays the video on a display device on the street, thereby realizing the videophone.
[0035]
The second embodiment will be described with reference to FIGS.
In FIG. 6, reference numeral 10 denotes a street microphone and speaker, and reference numeral 4b denotes a service center.
[0036]
For simplicity, a difference from FIG. 1 according to the first embodiment is shown. In the second embodiment, the same procedure is performed from the start of a voice call to the reproduction and display of moving image data on the display device 9 on the street. The voice separated at the service center 4b is transmitted through the mobile phone network 8 to the mobile phone 1 having no moving image capturing / reproducing function and reproduced by the mobile phone 1 having no moving image capturing / reproducing function. Is sent to the speaker 10 and reproduced by the speaker. At this time, a button for prompting the end of the voice call in the mobile phone network 8 is displayed on the mobile phone 1 having no moving image shooting / reproducing function, and when the user presses a key corresponding to this button, the service center 4b starts. The connection with the mobile phone 1 having no moving image shooting / reproducing function is cut off via the mobile phone network 8, and the mobile phone 1 without the moving image shooting / reproducing function is disconnected. At the same time, the service center 4b activates the microphone on the street and the microphone input of the speaker 10 or enables the microphone input, and replaces the audio data obtained from the mobile phone 1 without the moving image shooting / playback function with the moving image of the street camera 3 The data is converted into a videophone format in combination with the data and transmitted to the mobile videophone 6. The user who has been talking on the mobile phone 1 having no moving image shooting / reproducing function thereafter continues the videophone call in front of the street display device 9 or the street microphone and the speaker 10.
[0037]
In FIG. 7, reference numeral 112 denotes a button.
[0038]
For simplicity, differences from FIG. 2 showing the first embodiment are shown. In FIG. 7, the process is the same until the user presents the ID, confirms the position of the camera and the user himself from the displayed contents, and notifies the service center 4b. After that, when a videophone call is started, the service center displays the button 112 on the mobile phone 1 having no moving image capturing / reproducing function to urge the end of the voice call in the mobile phone network 8. When the user presses the button 112, the press is notified to the service center 4b, and the service center 4b hijacks the call between the mobile phone 1 and the mobile videophone 6 that do not have the moving image shooting and playback function, and has the moving image shooting and playing function. The mobile phone 1 is disconnected from the mobile phone network 8, and the display is shifted to the street display device 9, the street camera 3, and the videophone using the street microphone and speaker 10.
[0039]
A series of operation flows of the present system will be described with reference to FIG. For simplicity, a difference from FIG. 3 showing the first embodiment is shown. As shown in FIG. 8, the procedure from the voice call start 101 to the public TV phone operation confirmation 108 is the same. At this time, the video phone format data transmitted from the portable video phone 6 is separated at the service center 4b, Is reproduced by the mobile phone 1 which does not have the moving image photographing / reproducing function. The sound is also reproduced from a microphone and a speaker 10 on the street.
[0040]
The service center 4b prompts the mobile phone 1 having no moving image shooting / reproducing function to end the voice call in the mobile phone network 8, and the service center 4b can communicate with the mobile phone 1 having no moving image shooting / reproducing function if the user agrees. The mobile phone 1 takes over the call of the mobile videophone 6 and disconnects the mobile phone 1 having no moving image capturing / reproducing function from the mobile phone network 8 by the mobile phone disconnection 112.
[0041]
At the same time, as a call continuation 903, the display shifts to a videophone using a street display device 9, a street camera 3, and a street microphone and speaker 10. When the user ends the call, the call 904 is received and the call is completed.
[0042]
Next, a third embodiment will be described with reference to FIGS.
In FIG. 9, 5 is a video mail server having a storage function, and 4c is a service center.
[0043]
In FIG. 9, the user calls the service center 4c from the mobile phone 1 having no moving image capturing / reproducing function and follows the voice guidance, or activates a menu screen, a web browser, or a Java (registered trademark) application on the mobile phone to change the menu. Apply for video mail by selecting. Get the ID required for loopback use. The ID is, for example, a user's telephone number and time, an identification number assigned to a session between mobile phones during a call, a charging code during charging, a sequence number of a charging code, and a number sequence that can be easily handled by the service center 4a. String, binary code. In addition to sending the ID as it is to the mobile phone, the service center 4c may generate a barcode and send it as bitmap image data.
[0044]
A telephone number stored in a SIM card built in the mobile phone 1 having no moving image shooting / reproducing function, a phone-specific number, or these numbers and time, a unique number of a base station of the connected mobile phone network 8, and the like. By automatically generating an ID from the combination of the above, the acquisition of the ID from the service center 4c may be omitted.
[0045]
The requested ID generates a barcode on the mobile phone 1 having no moving image photographing / reproducing function and displays it on the screen. The mobile phone 1 having no moving image capturing / reproducing function is shown to the street sensor or camera 2 and the street sensor or camera 2 scans, demodulates the ID, and notifies the center server 4c.
[0046]
In this embodiment, the street sensor or camera is a general CCD or CMOS sensor or camera, and scans in the visible light range as described above. Since a CCD, CMOS sensor, or camera can receive infrared light by blocking visible light, it is also possible to present an ID by infrared communication using the IrDA infrared data communication port of the mobile phone.
[0047]
The service center 4c that has confirmed the validity of the presented ID activates the camera 3 on the street and starts photographing. The photographed image is sent to a mobile phone 1 having no moving image capturing / reproducing function in a so-called still image format such as JPEG or GIF, and is displayed on a screen of the mobile phone 1 having no moving image capturing / reproducing function. The user confirms the position of the camera and the image of the user himself from the displayed contents and notifies the service center 4c of the confirmation, and at the same time, the mobile phone 1 without the moving image shooting / playing function calls the service center 4c, The service center 4c starts recording. In addition, the service center 4c may call the mobile phone 1 having no moving image photographing / reproducing function and start recording when the user takes it.
[0048]
The service center 4c receives the moving image from the camera 3 on the street, superimposes it on the audio data being recorded, converts it into a videophone data format, and transmits it to the video mail server 5. The recording of the voice ends when the user has completed the task and the user's voice call is cut off, and notifies the video mail server 5 of the end of transmission when the video phone format data including the last voice data is transmitted. At the same time, the camera 3 on the street is stopped. The video mail server 5 stores the received moving image data and audio data in a videophone format.
[0049]
The video mail server 5 notifies the portable videophone 6 of the incoming video mail, connects the videomail server 5 from the portable videophone 6 as appropriate, and performs reproduction after downloading or downloading.
[0050]
In FIG. 10, 114 and 116 are buttons.
[0051]
When the user presents the ID, a two-dimensional barcode 11 is displayed on the screen. It may be a one-dimensional barcode or bitmap data substituted for it. When the user completes the presentation of the barcode, the user presses the key of the mobile phone 1 having no moving image capturing / reproducing function associated with the button 106 to terminate the display of the barcode.
[0052]
When the service center 4c confirms the validity of the presented ID, it activates the street camera 3 and sends the photographed image in a still image format to the mobile phone 1 having no moving image photographing / reproducing function, and the photograph is taken by the street camera. It is displayed as a still image 12 of the user. The user confirms from the display contents that the camera position and the user are in the picture, and presses the key of the mobile phone 1 having no moving image shooting / reproducing function corresponding to the button 114 for recording to notify the service center 4c. I do.
[0053]
When the button 114 is pressed, the mobile phone 1 having no moving image shooting / playing function calls the telephone number of the service center 4c separately assigned to the button 114. When the user is connected, the user starts recording according to the voice guidance. When the requirements are satisfied, the user presses the key of the mobile phone 1 having no moving image shooting / reproducing function associated with the button 116 to hang up the telephone. When the call is disconnected, the recording ends and the transmission of the video mail is completed.
[0054]
A series of operation flows of the present system will be described with reference to FIG. As shown in FIG. 11, the user of the mobile phone 1 having no moving image shooting / playing function tries the ID acquisition 103 subsequent to the center call 102 and obtains the ID. The ID is presented by IrDA 105b by IrDA or is displayed on the screen of the mobile phone 1 having no moving image photographing / reproducing function by the ID display 105a via the barcode generation 104 and presented to the sensor or camera 2 on the street.
[0055]
When the service center 4c confirms the validity of the ID, an image photographed by the camera 3 on the street is obtained at a photographing start 107 after the camera activation 106. The captured image is sent in a still image format to the mobile phone 1 having no moving image capturing / reproducing function, and the camera position confirmation 113 is urged and confirmed. At the same time, the mobile phone 1 having no moving image shooting / playing function calls the service center 4c, and when connected, the user starts recording in accordance with the voice guidance at the recording start 114, and the service center 4c performs the moving image sound synchronization processing 115 on the street. The video data is received from the camera 3, is superimposed on the audio data being recorded, is converted into a videophone data format, and is transmitted to the video mail server 5. The video mail server 5 stores the received data in the Video Mail storage 116 as needed, waits for the end, and completes the storage by disconnecting 117.
[0056]
After the completion, the video mail server 5 notifies the portable videophone 6 of the incoming video mail with the Video Mail incoming notification 601, connects the video mail server 5 from the portable videophone 6 as appropriate, and plays or downloads the video mail with the Video Mail playback 602. After playback. When the reproduction or download is completed, the portable videophone 6 disconnects from the video mail server 5 in disconnection 603.
[0057]
A fourth embodiment will be described with reference to FIG.
In FIG. 12, 4d is a service center, and 6a and 6b are portable videophones.
[0058]
For simplification, a difference from FIG. 6 showing the second embodiment is shown. In the fourth embodiment, a videophone call is made between the mobile videophone 6a and the mobile videophone 6b from the start of the call. The call agent of the service center 4d during a call performs call control and does not perform media conversion except for grasping the communication status.
[0059]
When the user of the portable videophone 6a switches from the portable videophone 6a to a street communication terminal including a street camera 3 and a street display device 9, and a street microphone and a speaker 10, and wishes to make a call, a service is provided. The center 4d is called to request an ID for receiving a videophone service by using a communication terminal on the street and obtain the ID. From the presentation of the ID by the portable videophone 6b and the street sensor or scanner 2, the service center
Confirmation of the validity of the ID in 4d is the same as the procedure shown in the first to third embodiments.
It is.
[0060]
The service center 4d that has confirmed the validity of the presented ID activates the street camera 3 and starts photographing. The photographed image is sent to the portable videophone 6a in a still image format such as so-called JPEG or GIF and displayed on the screen. The user confirms from the display contents that the camera position and the user themselves are captured, and notifies the service center 4d of the confirmation. In this confirmation, the service center 4d obtains images from a plurality of street cameras 3 around the user, sequentially sends the images to the mobile videophone 6a, and allows the user to select the street cameras 3 to be used for communication. You can also.
[0061]
Since the service center 4d is ready to use the communication terminals on the street, the service center 4d instructs the portable videophone 6b to switch the videophone data path to the media center of the service center 4d. When the route switching is successful, the service center separates the videophone format data received from the mobile videophone 6b, reproduces the audio data with the street speaker and the microphone 10 speaker, and displays the moving image data on the street display device 9. I do. Further, the media converter of the service center 4d converts the audio data collected by the street speaker and the microphone of the microphone 10 and the moving image data shot by the street camera into a videophone format and transmits the videophone format to the mobile videophone 6b. A videophone call with the videophone 6b is performed.
At the point of switching, the mobile videophone 6a disconnects the call on the mobile phone network 8.
[0062]
If a continuous video call with the mobile videophone 6a is desired by the mobile videophone 6a during a videophone call on the street communication terminal, the mobile videophone 6a calls the service center 4d and requests switching. The call agent of the service center 4d controls the media converter during the call, and switches a videophone call between the media converter and the mobile videophone 6b to a call between the mobile videophone 6b and the mobile videophone 6a.
When the switching is completed, the service center 4d releases the media converter and the communication terminal on the street.
[0063]
Next, a fifth embodiment will be described with reference to FIGS.
In FIG. 14, 3e is a street camera, and 4e is a service center.
[0064]
In FIG. 15, 12a is no camera input, 12b is an image of the street camera 3, 12c is an image of the street camera 3e, 12d is an image of a peripheral camera, 120 is an icon for calling the previous image with an icon, and 121 is an icon for calling the previous image. The selection decision icon 122 is an icon for calling the next imaging.
[0065]
In FIG. 17, reference numerals 14a, 14b, 14c, and 14d denote icons indicating a camera allocation, 14a1 denotes an icon indicating a camera allocation for arranging the imaging of the camera 1 on the main screen, and 14a2 denotes a camera allocation for arranging the imaging of the camera 2 on the main screen. Is an icon that indicates
[0066]
For simplicity, a difference from FIG. 1 showing the first embodiment is shown.
In the fifth embodiment, the user starts a call via the mobile phone network 8 between the mobile phone 1 having no moving image capturing / reproducing function and the mobile videophone 6, and thereafter, the service center 4e is presented. The processing until the validity of the ID is confirmed is the same.
[0067]
The service center 4e activates the street camera 3, the street camera 3e, and the surrounding street cameras to start shooting. The photographed image is sent to a mobile phone 1 having no moving image capturing / reproducing function in a so-called still image format such as JPEG or GIF, and is displayed on a screen of the mobile phone 1 having no moving image capturing / reproducing function.
As shown in FIG. 15, an image 12b of the street camera 3 and an image 12c of the street camera 3e are displayed on the screen of the mobile phone 1 having no moving image shooting and playback function, and the user can select any two images. Selection is made by pressing a button or key (hereinafter abbreviated as a button or the like) associated with 121.
[0068]
The image selected in FIG. 15 is scrolled in the horizontal direction and switched by pressing a button or the like associated with the icon 120 or the icon 122. The mobile phone 1 having no moving image shooting / reproducing function can switch between vertical scrolling and annular scrolling as shown in FIG. 16 by changing the screen setting.
[0069]
When the selection is completed, the size relationship and arrangement of the two imagings are selected by selecting the icons 14a, 14b, 14c, and 14d indicating the camera allocation in FIG. When the icon 14a indicating the camera allocation is selected, an icon 14a1 indicating the camera allocation for arranging the imaging of the camera 1 on the main screen is displayed. When this camera allocation is selected, the imaging of the camera 3 on the street is displayed on the main screen, and the image of the street 1 is displayed. The image of the camera 3e is arranged on the wipe screen.
[0070]
When the button or the like associated with the icons 120 and 122 is pressed, the camera 2 is switched to the icon 14a2 indicating the camera allocation to be arranged on the main screen, and the image of the street camera 3e is imaged by the street camera 3 on the main screen. On the wipe screen.
[0071]
The mobile phone 1 holds the setting of the camera allocation, and after selecting the imaging in FIGS. 15 and 16, it is also possible to display an icon 14 a 1 indicating the camera allocation for arranging the imaging of the camera 1 on the main screen. , 122, it is possible to switch the imaging of the camera 2 to the icon 14a2 indicating the camera allocation to be arranged on the main screen.
[0072]
In addition, since the mobile phone 1 holds the setting of the camera split, the image of the street camera 3 and the image of the street camera 3e are displayed according to the camera split without displaying the icon 14a1, and the icons 120 and 122 are displayed. It is also possible to switch the magnitude relationship by pressing the attached button or the like.
[0073]
The information on the camera allocation determined in this way is transmitted to the service center 4e. At the center 4e, the images captured by the cameras 3 and 3e on the street are combined according to the camera assignment, and transmitted to the portable videophone 6 of the other party as MPEG-4 stream data.
[0074]
FIG. 18 shows the format of the transmitted MPEG-4 stream data and the attribute values of the display area attribute.
The service center 4e sends the attribute value of the display area attribute to the portable videophone 6 as MPEG-4 stream data starting from a normal MPEG-4 header.
[0075]
The MPEG-4 stream data following the attribute value of the display area attribute is obtained by synthesizing the image of the street camera 3 and the street camera 3e with a bitmap image in either a program stream format or a transport stream format, and then encoding. Stream data is generated and transmitted in one of the format of the program stream and a transport stream format which is individually encoded and folded.
[0076]
In the case of the transport stream format, the transmitted stream data includes main video overlay prohibition, main video priority reproduction, wipe screen priority reproduction, main video wipe screen switching prohibition, wipe screen display / non-display switching prohibition, and wipe screen boundary point. Coordinates and attribute values such as overlay prohibition for prohibiting display of the wipe screen of the receiving terminal side camera are given, and displayed on the receiving terminal according to the attribute value of the display area attribute.
[0077]
When the stream data to be transmitted is in a program stream format, attribute values such as prohibition of main video overlay, coordinates of a wipe screen boundary point, and prohibition of overlay for prohibiting display of the wipe screen of the camera on the receiving terminal side are added, and display on the receiving terminal is performed. Displayed according to the attribute value of the area attribute.
[0078]
Next, a sixth embodiment will be described with reference to FIGS.
In FIG. 19, 4f is a service center, and 6c and 6d are portable videophones.
[0079]
In FIG. 20, 130 is a moving image display of 6c and 6d in a call state, 131 is a moving image display of 6c and 6d in a camera selection state, 132 is a moving image display of 6c and 6d in a layout selection state, and 133 is 6c and 6d in a composite call state. 134 is a moving image display of 6c and 6d in a composite call (common screen) state.
[0080]
In FIG. 21, 1301 and 1302 are moving image displays of 6c in a call state, 123 is an icon for presenting a barcode application for composite call, 124 is an icon for ending the barcode display and returning to the call state, and 1241 and 1251 are icons. The status display 125 is a button corresponding to an icon, and the user who has checked the status display 1251 presses the button. (This is called the trigger button function.)
In FIG. 19, a videophone call (call state 130) is performed between the mobile videophones 6c and 6d.
When the user in FIG. 21 presses a button or the like corresponding to the icon 123 for presenting a barcode application for a composite call of the mobile videophone 6c, the mobile videophone 6c generates or generates the barcode 11 including the call ID of the call state 130. Download and display on screen.
[0081]
The user presents the call ID to the service center 4f by displaying the barcode 11 on the sensor or camera 2 on the street. When the presentation is finished, the barcode display is terminated, and the display of the barcode 11 is terminated by pressing the button or key corresponding to the icon 124 for returning to the call state, and the display of the portable videophone 6c returns to the videophone call. .
[0082]
While the portable videophone 6c presents the series of barcodes 11, the videophone call between the portable videophones 6c and 6d continues, and voice and video are transmitted and received between the portable videophones 6c and 6d. The moving image is discarded without displaying only the mobile videophone 6c. In addition, while presenting the barcode 11, the portable videophone 6c can repeatedly send an image when the button or key corresponding to the icon 123 is pressed, and perform still display on the portable videophone 6d.
Still images can be displayed on the videophone 6d by sending a still command or an image to which the still display attribute is added. (This is called the still attribute.)
In FIG. 19, the service center 4f indicates the call ID included in the barcode 11 on the mobile phone network, and requests a copy of a videophone call packet between the mobile videophones 6c and 6d. When the mobile phone network confirms the validity of the call ID, the mobile phone network transfers a copy of the videophone call packet between the mobile videophones 6c and 6d to the service center 4f, but transmits the copy between the mobile videophones 6c and 6d via the mobile phone network. The videophone call continues.
[0083]
As shown in FIG. 21, the service center 4f receiving the presentation searches the street sensor or the street camera 3 surrounding the camera 2 to which the user has presented the barcode including the call ID, and finds the available street. The presence or absence of the camera 3 is notified to the mobile videophone 6c, and a status display 1251 is displayed on the mobile videophone 6c. The user presses the button or key corresponding to the icon 125, and if there is no available street camera 3, the display returns to the normal videophone call (call state 130), and the available street camera 3 exists. Then, the state transits to the camera selection (camera selection state 131).
[0084]
22 and 23 show camera selection and camera allocation in the sixth embodiment.
When the state transits to the camera selection state 131, the service center 4f activates the camera 3 on the street and starts photographing. The photographed image is sent to the portable videophone 6c for display. The service center 4f can also convert the captured image into a still image format such as so-called JPEG or GIF and send it to the portable videophone 6c for display.
[0085]
On the screen of the mobile videophone 6c, as shown in FIG. 22, an image 12b of the camera of the mobile videophone 6c and an image 12c of the camera 3 on the street are displayed. Selection is made by pressing the attached button or key. The image selected in FIG. 22 is horizontally scrolled and switched by pressing a button or a key corresponding to the icon 120 or the icon 122.
[0086]
When the selection is completed, the icons 14a, 14b, 14c, and 14d indicating the camera allocation in FIG. 23 are selected to select the magnitude relationship and arrangement of the two imagings (layout selection / synthesis state 132). When the icon 14b indicating the camera allocation is selected, an icon 14b1 indicating the camera allocation for arranging the imaging of the camera 1 on the main screen is displayed. When this camera allocation is selected, the imaging of the camera of the portable videophone 6c is displayed on the main screen. The image taken by the camera 3 on the street is arranged on the wipe screen.
[0087]
By operating the buttons or the like associated with the icons 120 and 122, the image of the camera 2 is switched to the icon 14b2 indicating the camera allocation to be arranged on the main screen, and the image of the street camera 3 is displayed on the main screen as the camera of the portable videophone 6c. Is arranged on the wipe screen.
[0088]
The mobile videophone 6c holds the setting of the camera allocation, and after selecting the imaging in FIG. 22, it is possible to display an icon 14b1 indicating the camera allocation for arranging the imaging of the camera 1 on the main screen. By operating a button or the like associated with, it is possible to switch the imaging of the camera 2 to the icon 14b2 indicating the camera allocation to be arranged on the main screen.
[0089]
Next, the encoding, combining, and decoding processes in the portable videophones 6c and 6d and the service center 4f according to the sixth embodiment will be described with reference to FIGS.
[0090]
In FIG. 24, 410 is an encoder of the service center 4f, 431 is an automatic editing machine of the service center 4f, 470 is a thinning function of the portable videophone 6c, 471 is a video encoder of the portable videophone 6c, and 473 is audio of the portable videophone 6c. Encoders 474a and 474b are video decoders of the mobile videophone 6d, 475a and 475b are frame memories of the mobile videophone 6d, and 476 is an audio decoder of the mobile videophone 6d.
[0091]
The service center 4f in FIG. 19 combines the video data in the transport stream format with the mobile videophone 6d, or the service center 4f combines the video data of the mobile videophone 6c with the video data from the camera 3 on the street, and The video data in the program stream format is transmitted to 6d.
In FIG. 24, an encoder 410 encodes data captured by a street camera 3 on a full screen and inputs the data to an automatic editing machine 431.
[0092]
The mobile videophone 6c encodes data captured by a camera provided in the mobile videophone 6c with a video encoder 471 and encodes audio with an audio encoder 473, converts the data into data of a mobile videophone format, and transmits the data to a mobile phone network.
[0093]
The mobile telephone network relays a copy of the data in the mobile videophone format to the service center 4f and inputs the data to the automatic editing machine 431 of the service center 4f. When the user allocates a moving image captured by the mobile phone 6c to the wipe screen by using a camera, the data to be input to the video encoder 471 is adjusted in advance by the thinning function 470 to the size of the thinned wipe screen.
[0094]
The automatic editing machine 431 of the service center 4f inputs the moving image data from the encoder 410 when the wipe screen is included in the image captured by the mobile videophone 6c according to the camera assignment for the mobile videophone format data from the mobile videophone 6c. Replace with
The automatic editing machine of the service center 4f outputs data in a transport stream format in which data of the mobile videophone format and moving image data of the wipe screen are alternately interleaved, and transmits the data to the mobile videophone 6d via the mobile phone network.
[0095]
The portable videophone 6d that has received the data in the transport stream format decodes the data in the portable videophone format with the video decoder 474a and the audio decoder 476, and the image data is stored in the frame memory 574a. The moving image data of the wipe screen is decoded by the video decoder 474b and stored in the frame memory 574b, and the display device of the portable videophone 6d reads the data from the frame memories 574a and 574b and displays the data.
[0096]
The automatic editing machine 431 of the service center 4f operates a quantization coefficient or the like at the time of encoding in accordance with the camera allocation, and sets the bit rate to a bit rate allowed by the mobile videophone. (Bit rate limit: Indicate quantization coefficient)
When the camera split uses the data captured by the portable videophone 6c as the main screen, the quantization coefficient is set to a value slightly reduced by 1 or 2 from the default value, and the portable videophone 6c is instructed to encode with this value. This value may be written in advance, for example, at the time of shipment of the portable videophone 6c from the factory and set as a default value for composite encoding.
[0097]
The moving image editor 431 of the service center 4f determines the upper limit of the bit rate occupied by the main screen by the instruction of the quantization coefficient, and calculates the remaining calculated from the upper limit of the audio bit rate defined in the videophone format. Is instructed to the encoder 410. By doing so, the service center 4f keeps the bit rate of the data in the transport stream format to be transmitted to the portable videophone 6d within an allowable value.
[0098]
When the camera split uses data captured by the mobile videophone 6c as a wipe screen, the moving image editor 431 of the service center 4f transmits the bit rate occupied by the wipe screen and the upper limit of the audio bit rate defined in the videophone format. It instructs the encoder 410 to set the remaining bit rate or quantization coefficient calculated from the value to a value slightly reduced by 1 or 2 from the default value. By doing so, the service center 4f keeps the bit rate of the data in the transport stream format to be transmitted to the portable videophone 6d within an allowable value. When the size of the wipe screen is large and the proportion of the size of the main screen is large, the mobile videophone 6c is instructed to encode the quantization coefficient with a value slightly reduced by one or two from the default value.
(Motion vector setting to wipe screen area prohibited)
(Insert and synthesize in macroblock units)
In FIG. 25, 411 is a motion vector search area restriction / coordinate conversion function of the service center 4f, 432 is an automatic editing machine of the service center 4f, 472 is a motion vector search area restriction / coordinate conversion function of the portable videophone 6c, and 474 is a mobile phone. The video decoder 475 of the videophone 6d is a frame memory of the mobile videophone 6d.
[0099]
The encoder 410 inputs data captured by the street camera 3 to the automatic editing machine 432. The mobile videophone 6c encodes data captured by a camera included in the mobile videophone 6c with a video encoder 471 and encodes audio with an audio encoder 473, converts the data into data of a mobile videophone format, and transmits the data to a mobile phone network. The mobile telephone network relays a copy of the data in the mobile videophone format to the service center 4f and inputs the data to the automatic editing machine 432 of the service center 4f.
[0100]
When the user assigns a video shot by the mobile phone 6c to the wipe screen with the camera split, the video of the street camera 3 becomes the main screen, encodes the entire area excluding the area occupied by the wipe screen, and sends it to the automatic editing machine 431. input. In this case, according to the instruction of the motion vector search area restriction / coordinate conversion function 411, the motion vector search is not performed in the area where the wipe screen is fitted, and the encoding is performed, thereby outputting moving image data that does not refer to the restricted area.
[0101]
The moving image captured by the portable videophone 6c is adjusted in advance to the size of the thinned wipe screen by the thinning function 470, and then input to the video encoder 471 and encoded. At the same time, with respect to the motion vector, the coordinate system of the wipe screen is switched to the coordinate system of the main screen on which the wipe screen is fitted by the motion vector search area restriction / coordinate conversion function 472 and encoded. The encoded moving image data is processed into data encoded by the audio encoder 473 and data in a portable videophone format, and transmitted to the portable telephone network. The automatic editing machine 432 of the service center 4f receives a copy of portable videophone format data from the portable telephone network and cuts out moving image data. The moving image data is managed in units of macro blocks, and is appropriately inserted in units of macro blocks at positions where the data of the wipe screen of the moving image data of the main screen enters. When the insertion is completed, the moving image data corresponding to the screen size of the portable videophone format or the display size of the main screen of the portable videophone 6d is completed.
The service center 4f processes the completed moving image data into a portable videophone format, outputs it to the portable telephone network, and transmits it to the portable videophone 6d.
[0102]
When the user assigns a video taken by the mobile phone 6c to the main screen by camera assignment, the video of the camera 3 on the street becomes the main screen, and encodes and inputs the entire area excluding the area occupied by the wipe screen to the automatic editing machine 431. I do. In this case, the video encoder 410 thins out the data captured by the camera 3 on the street, adjusts the data to the size of the wipe screen, and then encodes. At the same time, the motion vector search area restriction / coordinate conversion function 411 switches the coordinate system of the wipe screen to the coordinate system of the main screen on which the wipe screen is fitted, and encodes the motion vector.
[0103]
The video encoder 471 encodes the moving image captured by the portable videophone 6c without searching for the motion vector in the area where the wipe screen is to be fitted, according to the instruction of the motion vector search area restriction / coordinate conversion function 472. By doing so, moving image data that does not refer to the restricted area is output.
[0104]
The encoded moving image data is processed into data encoded by the audio encoder 473 and data in a portable videophone format, and transmitted to the portable telephone network. The automatic editing machine 432 of the service center 4f receives a copy of portable videophone format data from the portable telephone network and cuts out moving image data. The automatic editing machine 432 of the service center 4f reads this moving image data in units of macroblocks, and inserts the data output by the encoder 410 as appropriate in units of macroblocks when the wipe screen position reaches the position where it should enter. When the insertion is completed, the moving image data corresponding to the screen size of the portable videophone format or the display size of the main screen of the portable videophone 6d is completed.
The service center 4f processes the completed data into a portable videophone format, outputs the processed data to the portable telephone network, and transmits it to the portable videophone 6d.
[0105]
In either case where the image captured by the mobile videophone 6c becomes the main screen or the wipe screen, the mobile videophone 6d divides the received mobile videophone format data into audio and video, and the audio data is output by the audio decoder 476. The video data is decoded and reproduced, and the video data is decoded by the video decoder 474 and then output to the frame memory 475 for display.
(Bit rate limit: Indicate quantization coefficient)
When the camera split is the main screen of data captured by the mobile videophone 6c, the automatic editing machine 432 of the service center 4f transmits the quantization coefficient of the moving image data of the mobile videophone format data sent from the mobile videophone 6c. , A quantization coefficient is instructed to the encoder 410.
[0106]
When the bit rate of the data of the mobile videophone format transmitted to the mobile videophone 6d exceeds the maximum value, the automatic editing machine 432 sets the quantization coefficient to a value slightly reduced by one or two, and sends this value to the mobile videophone 6c. , And the bit rate is set to an allowable value.
[0107]
When the camera split uses the data captured by the mobile videophone 6c as the wipe screen, the moving image editor 432 of the service center 4f outputs the bit rate occupied by the wipe screen and the bit rate of the audio defined by the videophone format. It instructs the encoder 410 to set the remaining bit rate or quantization coefficient calculated from the upper limit value to a value obtained by slightly reducing the default value by one or two. By doing so, the service center 4f keeps the bit rate of the portable videophone format data to be transmitted to the portable videophone 6d within an allowable value. When the size of the wipe screen is large and the ratio of the size of the main screen to the size of the main screen is large, the portable videophone 6c is instructed to encode the quantization coefficient by a value slightly reduced by one or two from the default value.
[0108]
Next, FIG. 26 shows a seventh embodiment.
In FIG. 26, reference numeral 140 denotes a normal portable videophone call, 141 denotes a composite call, 142 denotes a composite call using an external display device, and 143 denotes a mobile videophone call using an external camera and an external display device.
Calls between mobile videophones include voice-only calls and videophone calls, both of which are communicated via a mobile network (140).
[0109]
When the video phone call is shifted to the synthesized call, the voice and video packets of the mobile video phone are copied to the center, and the voice and video packets obtained by combining the video of the street camera at the center are sent to the other mobile video phone ( 141). Since the voice also passes through the center, even if the user's face, for example, the lips, is reflected in the moving image, the time lag between the display and the utterance is minimized.
[0110]
After the start of the composite call, the other party can select an external display device, and the moving image synthesized at the center is sent to an external display device around the mobile videophone via a path different from a mobile network such as the Internet and displayed. (142). By making the sound also pass through the center, even if the user's face, for example, the lips, is reflected in the moving image, the time lag between the display and the utterance is suppressed.
[0111]
The user can also use the peripheral display device during the composite call. In this case, the user himself can be photographed with a camera on the street and videophone calls can be continued on an external display device. Into a narrow gap to capture the situation of the gap.
[0112]
In addition, since it is possible to turn off the camera of the portable videophone and stop the synthesized call (143), it is possible for the user to place a videophone call at home and place a videophone call on the large screen of the television.
[0113]
In addition, in the composition at the service center, by using a moving image stream encoded by restricting the reference of the motion vector, another moving image stream can be easily inserted into the moving image stream and synthesized.
[0114]
In addition, by providing a conversion mechanism for converting the coordinate system of another moving image stream to be inlaid and synthesized into the coordinate system of the inlaid stream, the video can be reproduced by a conventional decoder.
[0115]
Also, for the stream to be embedded and combined, the bit rate of the combined stream is controlled by designating or controlling the quantization coefficient at the time of encoding based on the quantization coefficient of the other stream. Transmission can be realized, and reproduction can be performed with a conventional decoder.
[0116]
In addition, by turning off the camera of the portable videophone and stopping the synthetic call, the user can make a videophone call at home using the large screen of the television while holding the portable videophone in his ear.
[0117]
In the above embodiment, the camera and the display device are installed on the street as public communication terminals. However, the installation place is not limited to the street and may be in an office or at home, and may be arbitrary.
[0118]
According to the above-described embodiment, by using a mobile phone that has not been able to receive a service by a videophone conventionally in cooperation with a camera or a display device, a call can be made between the mobile videophone and the videophone. As a result, the life of the conventional mobile phone can be extended.
[0119]
In addition, the operation rate of the camera on the street, the display device on the street, and the like, and the utilization rate of the communication network are improved. By switching communication from a mobile phone to a communication terminal consisting of a camera, a display device, and also a speaker and a microphone, it is possible to release the frequency and base station resources that are the resources of the mobile phone network used for calls by the mobile phone. In addition, congestion of the mobile phone network can be reduced and congestion can be avoided.
[0120]
In addition, by switching at least one of the communications between the mobile videophones to a communication terminal on the street, the number of surrounding objects reflected in the captured moving image data can be increased, thereby enriching the amount of information to be transmitted.
[0121]
Further, by switching the call to the public communication terminal, the amount of battery consumption of the portable videophone having a short continuous talk time can be reduced, and both the continuous talk time and the standby time can be extended.
[0122]
In addition, besides the matters described in the claims, the following are more main aspects.
(1) In the communication method according to claim 5, the identifier is converted into image data by either a call agent or a second telephone terminal, and the image data is displayed on a display unit of the second telephone terminal. The device reads this indication and demodulates it into an identifier.
(2) In the communication method according to claim 5, the identifier detector is a sensor or a scanner arranged on a communication network, and has a reading accuracy of 7 to 12 pixels per millimeter or 12 pixels or more per millimeter when reading the identifier. There is.
(3) The communication method according to claim 5, wherein the identifier detector is a photographing device, the depth of field is from 1 cm to 30 cm to ∞, and a part or the whole range of the depth of field. The reading accuracy of the subject is 7 to 12 pixels per millimeter, or 12 pixels or more.
(4) In the method according to the tenth aspect, one of the buttons of the second telephone terminal is dynamically assigned to image data creation, and the user operates this button to convert the identifier into image data and display the image data. thing.
(5) In the system according to claim 16, one of the buttons of the first telephone terminal is dynamically assigned to the request for the identifier, and the user operates the button to request the identifier from the call agent.
[0123]
(6) In the system according to claim 16, one of the buttons of the first telephone terminal is dynamically assigned to image data creation, and the user operates the button to request and receive an identifier from the call agent. Converting the identifier, creating image data and displaying it on the screen of the first telephone terminal;
[0124]
(7) In the system according to claim 16, one of the buttons of the first telephone terminal is dynamically assigned to image data creation, and the user operates this button to convert an identifier into a call agent and obtain the converted data. Requesting image data to be received, and displaying the received image data on the screen of the first telephone terminal.
[0125]
(8) The method according to claim 6, wherein the identifier displayed on the public communication terminal is read and demodulated by the second telephone terminal, and the user is assigned one of the buttons of the second telephone terminal for reading. When the user operates this button, the first, second, or all of the procedures for reading, demodulation, and sending to the call agent are processed.
[0126]
(9) The gateway and the mobile phone network according to claim 19, wherein the gateway transfers the received packet to the other mobile videophone of the mobile phone network, and at the same time, the call agent or the mobile phone network. (4) discarding a packet transmitted from one portable videophone to another portable videophone when a copy of the packet is transferred to a service center.
[0127]
(10) As the identifier (ID) displayed on the mobile videophone, one or both of the mobile videophone telephone numbers, the terminal ID, the ID of one or both of the connected base stations, and the like are used. Any one of a channel number, a slot number being used, billing information, or a combination thereof is used.
(11) The ID is displayed by infrared light or blinking light.
(12) The ID is to display a bitmap image formed by, for example, a barcode created by one of the mobile videophones based on an operation of a button of the mobile videophone.
[0128]
(13) One of the buttons of the one portable videophone is assigned to a composite call end notification, and the user operates the button to transmit the composite call end notification, and the light receiving unit ( Detector), or by notifying the service center and the call agent of the termination of the synthesized call to the service center and the call agent via the mobile network, thereby stopping the packet copying and stopping the packet discarding.
[0129]
(14) In the method of synthesizing a stream composed of audio and moving images and a moving image stream, the number of pixels is large and small between the frames of the moving image of the stream composed of audio and moving images and the frames of the moving image stream. A synthesizing method, comprising: inserting and synthesizing a moving image of a frame; and outputting a stream including the synthesized audio and moving image.
(15) In the synthesizing method of the above (14), an area into which a small frame in a large frame is fitted is a macroblock unit.
(16) In the synthesizing method of the above (14), the encoder that encodes the small frame to be fitted includes, in the coordinate system of the area where the large frame is fitted, the macroblock number of the moving image stream of the small frame and the reference block number of the motion vector. Conversion means for converting the
(17) In the synthesizing method according to the above (14), the encoder that encodes a moving image of a small frame to be fitted instructs the encoder that encodes the moving image of a large frame with a quantization coefficient, whereby a stream composed of audio and moving image Improving one of the image quality of a small frame and the image quality of a large frame into which a stream composed of audio and video output from a stream synthesis service is to be inserted, better than the other.
(18) In the synthesizing method of (14), the moving image stream is a moving image stream cut out from a stream composed of another audio and moving image.
(19) In the synthesizing method of (14), the synthesizing is to add display attribute information to the stream and output the stream.
(20) In the synthesizing method according to the above (14), a decoder for reproducing the stream to which the display attribute information is added, and a display device for displaying according to the display attribute information are provided.
[0130]
(21) In the synthesizing method according to claim 26, when the decoder fits and reproduces a moving image of a small frame in a large frame, the decoder specifies a position to be fitted.
(22) In the synthesizing method according to (21), the decoder and the display device are inhibited from displaying a video independent of the transport stream in a region of a large frame and a small frame.
(23) In the synthesizing method of (21), a position where the decoder and the display device display a video independent of the transport stream in a region of a large frame and a small frame is designated.
(24) In the synthesizing method according to (21), the decoder and the display device designate one or both of a large frame and a small frame as a static display.
(25) In the synthesizing method according to (21), the decoder and the display device may have different update frequency, appearance frequency, and frame rate between the large frame and the small frame.
(26) In the synthesizing method of (21), the large frame and the small frame output transport streams having different update frequencies, appearance frequencies, and frame rates.
(27) In the telephone communication system according to the above (14), the call agent charges the terminal that originated the original voice call for all call times.
[0131]
【The invention's effect】
According to the present invention, a videophone call can be made between a mobile phone and a mobile videophone using a communication terminal such as a camera or a display device.
[Brief description of the drawings]
FIG. 1 is a diagram of a telephone communication system according to a first embodiment.
FIG. 2 is a diagram showing a screen display of a mobile phone having no moving image capturing / reproducing function according to the first embodiment.
FIG. 3 is a diagram illustrating a series of operation flows of the system realized by the first embodiment;
FIG. 4 is a diagram showing data conversion into a videophone format in a service center 4a.
FIG. 5 is a diagram showing separation of a moving image and a sound from the videophone format of the service center 4a.
FIG. 6 is a system diagram showing a second embodiment.
FIG. 7 is a diagram illustrating a screen display of a mobile phone having no moving image capturing / reproducing function according to the second embodiment.
FIG. 8 is a diagram showing a series of operation flows of a system realized by the second embodiment.
FIG. 9 is a system diagram showing a third embodiment.
FIG. 10 is a diagram illustrating a screen display of a mobile phone having no moving image capturing / reproducing function according to the third embodiment.
FIG. 11 is a diagram showing a series of operation flows of a system realized by the third embodiment.
FIG. 12 is a system diagram showing a fourth embodiment.
FIG. 13 is a system diagram showing a conventional technique.
FIG. 14 is a system diagram showing a fifth embodiment.
FIG. 15 is a diagram illustrating camera selection in a fifth embodiment.
FIG. 16 is a diagram illustrating camera selection according to the fifth embodiment.
FIG. 17 is a diagram illustrating camera allocation according to the fifth embodiment.
FIG. 18 is a diagram illustrating a format of MPEG-4 stream data and an attribute value of a display area attribute according to the fifth embodiment.
FIG. 19 is a system diagram showing a sixth embodiment.
FIG. 20 is a diagram illustrating switching to a composite call according to the sixth embodiment.
FIG. 21 is a diagram illustrating an application for a composite call according to the sixth embodiment.
FIG. 22 is a diagram illustrating camera selection according to the sixth embodiment.
FIG. 23 is a diagram illustrating a camera split according to the sixth embodiment.
FIG. 24 is a diagram illustrating encoding, combining, and decoding processes in a mobile videophone and a service center according to the sixth embodiment.
FIG. 25 is a diagram illustrating encoding, combining, and decoding processing in a mobile videophone and a service center according to the sixth embodiment.
FIG. 26 is a diagram showing a seventh embodiment.
[Explanation of symbols]
1 ... mobile phone without video recording / playback function 2 ... street sensor or camera
3 ... street camera, 4a, 4b, 4c ... service center,
5 Video mail server 6, 6a, 6b Mobile videophone
7 ... Internet, 8 ... Mobile phone network,
9: Street display device, 10: Street microphone and speaker,
11 ... two-dimensional barcode,
12: Still images of users taken by street cameras, 106, 108, 112,
114, 116 ... button, 41 ... image input, 42 ... voice input,
42a: codec, 43: automatic editing machine, 44: video / audio separation, 45: video output, 46: audio output, 46a: codec.

Claims (27)

In a telephone communication system for performing at least a voice call between a first telephone terminal and a second telephone terminal via a communication network,
The first telephone terminal has a function of performing voice and data communication, capturing a moving image, and compressing / decompressing and reproducing / displaying the captured moving image.
The second telephone terminal does not have at least one of a function of capturing a moving image or a function of compressing / expanding the captured moving image and displaying the moving image,
A call center connected to the communication network and performing call control on a telephone terminal as a communication partner, a service center having a media converter for converting image data and audio data,
A public communication terminal having at least a display function and connected to the service center;
Including
The service center performs call control between the first and second telephone terminals via a call agent, and transmits video data to the first telephone terminal via the media converter. A telephone communication system for performing communication in a format including audio data and transmitting moving image data to the public communication terminal via the media converter. The public communication terminal includes a display device and a photographing device,
The call agent has a function of detecting a use request of the public communication terminal sent from the second telephone terminal,
When a request from the second telephone terminal is detected by the detection function,
The call agent activates the display device and the photographing device, and the media converter instructs transmission and reception of moving image data,
And instructing the first telephone terminal to switch to a videophone, the media converter disconnects the connection with the second telephone terminal, and establishes a connection between the public communication terminal and the first telephone terminal. The telephone communication system according to claim 1, wherein the telephone communication is performed. A server having a storage connected to the communication network,
The call agent has a function of detecting a use request of the public communication terminal sent from the second telephone terminal,
When a request from the second telephone terminal is detected by the detection function,
The call agent activates the public communication terminal, instructs the media converter to transmit and receive video data,
The media converter combines the audio data of the second telephone terminal and the video data and sends the synthesized data to the server;
The call agent instructs the server on the destination of the video data and the storage thereof,
The server notifies the first telephone terminal of the arrival of the moving image data, and the first telephone terminal obtains and reproduces the moving image data stored in the server according to the notification. Item 2. The telephone communication system according to Item 1. A communication method for making at least a voice call between a first telephone terminal and a second telephone terminal via a communication network,
Providing a call center connected to the communication network for performing call control between telephone terminals as communication partners, and a service center having a media converter for converting image data and audio data;
Preparing a public communication terminal including at least a display device and a photographing device and connected to the service center;
For a first telephone terminal having a voice communication function, a moving image photographing function, and a function of compressing / expanding and reproducing / displaying the photographed moving image, the media converter has a format including moving image data and audio data. Performing a communication;
The public terminal, which has at least a communication function by voice, but is transmitted from a second telephone terminal that does not have at least one of a function of capturing a moving image, or a function of compressing and expanding a captured moving image and reproducing and displaying the moving image, Recognizing a request for use by the call agent;
When the request is recognized, activating the display device and / or the image capturing device of the public communication terminal, and transmitting and receiving moving image data by the media converter;
A communication method, comprising: Assigning an identifier to the second telephone terminal under the control of the call agent in response to a request from the second telephone terminal;
Displaying information associated with the identifier on the second telephone terminal;
Detecting a display of the identifier on the second telephone terminal by a detector provided in the public communication terminal, and transmitting the detected information to the service center;
According to the received information of the identifier, enabling the call agent to use the public communication terminal near the second telephone terminal to transmit and receive image data;
The communication method according to claim 4, comprising: Displaying an identifier on a display device of the public communication terminal;
The second telephone terminal reads and demodulates the identifier and sends the demodulated identifier to the call agent;
The call agent allowing the second telephone terminal to use the public communication terminal indicated by the identifier;
The communication method according to claim 4, comprising: When permitting use, request video data from a plurality of the public communication terminals,
Cut out a still image from the moving image data received for each of the public communication terminals,
Transmitting a still image to the second telephone terminal to select one still image, and selecting a public communication terminal to be used for videophone communication;
7. The communication method according to claim 6, wherein: 6. The display button of the second telephone terminal, wherein one of the display buttons is dynamically assigned to a request for an identifier, and the user operates the button to request the identifier from the call agent. The communication method described. In a telephone communication system for performing at least a voice call between a first telephone terminal and a second telephone terminal via a communication network,
The first telephone terminal has a function of performing voice and data communication, capturing a moving image, and compressing / decompressing and reproducing / displaying the captured moving image.
The second telephone terminal does not have at least one of a function of capturing a moving image or a function of compressing / expanding the captured moving image and displaying the moving image,
A call agent that is connected to the communication network, performs call control on a telephone terminal as a communication partner, and has a function of detecting a request for using the public communication terminal; A service center having a media converter to perform;
Including at least an image display device and a photographing device, having a public communication terminal connected to the service center,
The call agent activates the public communication terminal, enables communication of moving image data to the media converter, and instructs the video phone terminal to shift to a video phone,
The media converter synthesizes audio data from the second telephone terminal and the moving image data from the public communication terminal and sends the synthesized data to the first telephone terminal,
And separating the audio data and image data from the first telephone terminal, sending the separated audio data to the second telephone terminal, and sending the separated moving image data to the display device of the public communication terminal. A telephone communication system, characterized in that: 10. The media converter according to claim 9, wherein the transmission time information added to the audio data and the transmission time information added to the moving image data are compared, and the audio data and the moving image data at almost the same time are synthesized. A telephone communication system as described. The media converter compares the clock included in the media converter with the transmission time information added to the audio data and the transmission time information added to the moving image data to determine the amount of delay. The time to transmit the synthesized data to the communication network from the processing time of the device and the spare time previously set by the media converter in advance, and the arrival time at the first telephone terminal is made uniform. The telephone communication system according to claim 9, wherein The media converter synthesizes the separated moving image data and the moving image data arrived from the imaging device at the time when the separation is completed, and transmits the synthesized data to the display device,
10. The telephone communication system according to claim 9, wherein an image from the first telephone terminal and an image photographed by the photographing device are simultaneously displayed on the display device. Before the public communication terminal is used, the call agent communicates from the media converter, which mediates communication between the second telephone terminal and the first telephone terminal, to the public communication terminal. 10. The telephone communication system according to claim 9, wherein control is performed to search for a media converter having the shortest route and to switch the media converter. In a telephone communication system for performing at least a voice call between a first telephone terminal and a second telephone terminal via a communication network,
A call center connected to the communication network for performing call control between telephone terminals as communication partners, and a service center having a media converter for converting image data and audio data;
For a first telephone terminal having a voice communication function, a moving image shooting function, and a function of compressing / expanding and reproducing / displaying the shot moving image, communication is performed in a format including moving image data and sound data,
A display device based on a request from a second telephone terminal that has at least a voice communication function but does not have at least one of a moving image shooting function or at least one of a function of compressing / expanding and reproducing / displaying the shot moving image. And a photographing device, enabling use of a public communication terminal connected to the service center,
The media converter transmits and receives audio data to and from the second telephone terminal, and transmits and receives moving image data to and from the display device or the photographing device of the public communication terminal.
The telephone communication system, wherein the call agent charges the second telephone terminal based on a talk time using the public communication terminal. 15. The telephone communication system according to claim 14, wherein the call agent charges the second telephone terminal a difference between the call time and the voice call. The call agent and the plurality of media converters can communicate with a plurality of public communication terminals,
When a call is made between the first telephone terminals via a communication network,
The call agent switches videophone communication with another first telephone terminal from the one first telephone terminal to one of the public communication terminals,
The telephone communication system according to claim 1, wherein one public communication terminal makes a videophone call through one of the media converters. The second telephone terminal or the first telephone terminal possessed by the user during a videophone call at the public communication terminal is transmitted from the public communication terminal to the call agent via the second telephone terminal possessed by the user. Or requesting the first telephone terminal to switch the call,
In response to the request, the call agent controls the media converter and a first telephone terminal as a communication partner, and performs a voice call at the second telephone terminal or a television at the first telephone terminal. 17. The telephone communication system according to claim 16, wherein a transition is made to a telephone call or a voice call. In a videophone communication system capable of communicating between a plurality of mobile videophones via a mobile phone network, a service center connected to the mobile phone network and connected to a detector via a network different from the mobile phone network And
Detecting the identifier displayed on the mobile videophone during a call by the detector, receiving information of the identifier transmitted from the detector,
A service center that receives a copy of a videophone format packet transmitted by the mobile videophone that has presented the identifier to another mobile videophone from the mobile phone network. 19. The service center according to claim 18, wherein the copy of the packet received by the service center includes audio and video, and the service center adds or synthesizes another video to the video and transmits the video to a mobile phone network. . 19. The portable videophone used for telephone calls according to claim 18, wherein a button of the portable videophone is assigned to transmission of an identifier, and a user operates the button to transmit information of the identifier. In the method of synthesizing a stream composed of audio and video and a video stream, there is a size relationship in terms of the number of pixels between the video frame of the stream composed of audio and video and the frame of the video stream. And a combining method. An encoder generates a large moving image out of moving images having a magnitude relationship, and the moving image encoder performs intra-frame encoding and differential encoding between frames, and calculates a motion vector of a region into which a small frame in a large frame is inserted in differential encoding. 22. The method according to claim 21, wherein no search is performed. Either an encoder that encodes a video of a small frame to be inserted or an encoder that encodes a video of a large frame receives a quantization coefficient from the other, so that a service composed of a stream composed of audio and a video and a video stream can be provided. 22. The synthesizing method according to claim 21, wherein a bit rate of a stream composed of audio and video to be output is controlled. In the method of synthesizing a stream composed of audio and video and a video stream, one frame of audio and video of the stream composed of audio and video and one audio and a plurality of video streams obtained by sequentially folding one frame of the video stream are included. A combining method comprising outputting a transport stream. A transport stream composed of audio and a plurality of moving image streams, wherein the plurality of moving image streams have a size relationship between the moving image streams in terms of the number of pixels of the frames, and a moving image of a small frame is inserted into a large frame and reproduced. The method according to claim 24, wherein 26. The combining method according to claim 25, wherein the transport stream is reproduced according to the display attribute information by adding the display attribute information by the composition that outputs the transport stream. In a communication method of performing voice and video communication via a communication network using a mobile phone terminal under the management of a service center,
The service center and the first telephone terminal transmit and receive audio data and video data,
Transmission and reception of information including audio data other than the video data is performed between the service center and the second telephone terminal,
In response to a request from the second telephone terminal, the service center activates the display device and / or the photographing device in the second vicinity as an associated service for the second telephone terminal,
And transmitting and receiving moving image data as a related service to the second telephone terminal between a service center and the display device and / or the photographing device.

Images