JP2006140773A - Mobile communications system and mobile communication terminal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a mobile communications system by which voice data and video data for allowing to specify a caller can be transmitted to a transmission destination when calling. <P>SOLUTION: A header is imparted respectively to the voice data of uttered voice and image data of an imaged face in live information of a user on the calling side. Then, both data are packetized; and when calling, the packetized data are inserted to information bits of an access control channel used by the communications system at calling. The header is shown as S1-S4 in the case of the voice data and as H1-H4 in the case of the image date in Figure. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a mobile communication system, and in particular, includes a mobile communication terminal (for example, a mobile phone device) that introduces a mobile communication system and a base station, and has an incoming call notification function that can specify a caller in the mobile communication terminal. The present invention relates to a mobile communication system.

  In recent mobile phone devices, a caller ID notification function is widespread. This function notifies the transmission partner of the telephone number of the telephone used by the caller, and the receiver can confirm who the telephone is from before calling.

  In addition, the mobile phone device has a function for storing a unique ring tone or ring melody in advance and ringing the unique ring tone or ring melody at the time of an incoming call in combination with the caller ID notification function. A receiver has also been developed, and the recipient can immediately know who the call came from by listening to this unique ringtone or ringtone.

  Further, instead of the above-mentioned ringing tone or ringing melody, an image or a character is stored in advance in the mobile phone device on the transmission side, and when the call is transmitted from the mobile phone device, the image is received at the reception side telephone device at the time of the incoming call. Some have been developed with the ability to display characters and characters.

  As a mobile phone device of this type, for example, in Patent Document 1 (Japanese Patent Laid-Open No. 2002-78025), video data from an external device is input and stored in a caller's mobile phone device, and a call is made. A mobile phone device is disclosed in which the video data is added to the call signal of the receiver so that the receiver can confirm the sender by visually recognizing the video data.

In the conventional mobile communication system described in the above background art, for example, in the mobile phone device disclosed in Patent Document 1, the video data set in advance by the sender is transmitted to the mobile phone device of the sender. This video data is displayed at the time of an incoming call in the receiver's mobile phone device, so that the receiver can confirm who the person is talking by viewing this display. .
JP 2002-78025 A

  However, the video data added to the calling signal at the time of calling needs to be stored in advance in the sender's mobile phone device from an external device, and only the sender's mobile phone device is used for outgoing calls. Therefore, there is a problem that it is impossible to transmit a favorite video at any time.

  Also, there is no mention of a specific method for adding a video set by the sender in advance to the call signal, and of course, no progress has been made in the idea of the specific implementation method. Then, it seems that it is difficult to employ | adopt the technique of the mobile telephone apparatus disclosed by the above-mentioned patent document 1 to the mobile communication system containing the present mobile telephone apparatus, a PHS system, etc.

  The present invention has been made in view of the above-described conventional problems, and provides a mobile communication system capable of transmitting audio data and video data that enable identification of a sender to a transmission destination when a call is made. The purpose is to do.

  Another object of the present invention is to provide a mobile communication terminal capable of transmitting audio data and video data that enable identification of a sender to a transmission destination when a call is made.

  In order to achieve the above object, a mobile communication system and a mobile communication terminal according to the present invention have the following features.

  The mobile communication system and mobile communication terminal according to the present invention comprise a plurality of mobile communication terminals capable of transmitting and receiving voice data and image data, wherein each of the plurality of mobile communication terminals is a voice Sound input means for obtaining sound data by inputting a sound, image pickup means for picking up an image of a subject and obtaining image data, sound output means for outputting sound data input by the sound input means or received sound data, Display means for displaying image data captured by the imaging means or received image data, and image data captured by the recording means and / or voice data input by the voice input means in response to a call request, Call preparation means for converting into a predetermined data format and then inserting it into an information bit field of a specific communication channel; Using said specific communication channels that are prepared by Kihatsuko preparation means, characterized in that a calling means for calling.

  With this configuration, the calling user can transmit the caller's current audio data and video data (both live information) to the transmission destination when making a call.

  Here, in the mobile communication system, the specific communication channel used at the time of the call can be a predetermined access control channel.

  By configuring in this way, audio data to be transmitted to a transmission destination at the time of outgoing call or captured video data can be transmitted using an access control channel based on an existing communication method, thereby reducing the system burden. To do.

  Further, in the mobile communication system, the image data and / or the audio data is temporarily stored in a storage unit, and is extracted from the storage unit by the call preparation unit when a call is made, and the specific data It is inserted into the information bit field of the communication channel.

  With this configuration, the storage unit can function as a transmission buffer, and smoother communication is possible.

  In the mobile communication system, the image data is any one of moving image data and still image data.

  By configuring in this way, there are many choices of image data that can be transmitted by the calling user. Further, it is possible to cope with a future communication method capable of transmitting moving image data having a large amount of information through a control channel.

  In the mobile communication system, any one of the mobile communication terminals receives an incoming call when receiving an incoming call using the access control channel from any other one of the mobile communication terminals. And when the voice data is inserted in the access control channel, the voice output means outputs the voice data.

  By configuring in this way, the receiving user can already receive small-scale audio data and output the audio when receiving an incoming call that is not during full-fledged communication. Can be useful. In addition, in the case of a calling user and a receiving user who are intimately related, the closeness can be further increased and a quick response of the receiving user can be made.

  In the mobile communication system, when one of the mobile communication terminals receives a call using the access control channel from the other one of the mobile communication terminals, the mobile communication terminal notifies the incoming call. When image data is inserted into the access control channel, the image data is displayed by the display means.

  With this configuration, the receiving user can already receive small-scale image data and display an image when an incoming call is not made during full-scale communication. Can be useful. In addition, in the case of a calling user and a receiving user who are intimately related, the closeness can be further increased and a quick response of the receiving user can be made.

  Further, in the mobile communication system, when any one of the mobile communication terminals makes a call to any other one of the mobile communication terminals, the base station confirms the call and is calling A notification to the effect is transmitted to any one of the mobile communication terminals on the calling side.

  By configuring in this way, the calling user can obtain a reliable response to the call from the own station, and thus can quickly confirm the presence or absence of a basic operation error at the time of calling.

  In the mobile communication system, the voice data inserted in the information bit field of the specific communication channel is voice data formed by inputting a voice of a caller.

  By configuring in this way, the calling user can make the receiving user hear a voice having his / her own characteristics when receiving a call at the receiving terminal. Helps to identify.

  In the mobile communication system, the image data inserted in the information bit field of the specific communication channel is image data obtained by capturing an image of a caller's face.

  By configuring in this way, the calling user can show his / her own face to the receiving user when receiving a call at the receiving terminal. Helps to identify.

  The mobile communication system includes means for converting image data obtained by capturing the caller's face into an animation image, and inserting the animation image into an information bit field of the specific communication channel. It is characterized by.

  With this configuration, it is possible to reduce the amount of information of transmission data at the time of calling. As the calling user, for example, if your real live face image varies from day to day, and you feel uncomfortable by yourself or feel somewhat embarrassed, use an animated image with your own characteristics. can do.

  The mobile communication system further includes means for converting an animation image inserted into an information bit field of the communication channel into a three-dimensional representation in advance.

  By configuring in this way, it becomes possible to faithfully reproduce the current facial features of the calling user that the receiving user is in daily contact with when receiving the call, and the calling user (caller) can be specified. This can be done more quickly and reliably.

  The mobile communication system further comprises means for converting an image of the image data and the animation image inserted into an information bit field of the communication channel into a thumbnail image.

  With this configuration, the calling user can display a list of image data used in the past on the display unit as thumbnail images, and when using the image data for subsequent call information, This is convenient for selecting image data to be transmitted.

  In addition, the present invention has been made in view of the problems of the above-described conventional mobile communication terminals, and provides a mobile communication terminal configured as a mobile communication terminal of the mobile communication system.

Here, the mobile communication terminal may be a mobile phone device.
Also provided is a computer program for controlling a computer that executes processing of a mobile communication terminal in the mobile communication system.

  As described above, according to the mobile communication system of the present invention, in the call function using only the current control channel, live information including image data and audio data of the calling user who can specify the sender. Thus, there is an effect that the receiving-side user can surely specify who the call partner is when receiving a call.

  Further, not only the usability of the calling user and the receiving user is improved and the response speed of the receiving user is improved, but also the receiving user can visually recognize the facial expression of the calling user, and so on. Since intimate communication is possible and entertainment is enhanced, the satisfaction of the calling user and the receiving user can be further improved.

  Hereinafter, the best embodiment of the mobile communication system of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a configuration diagram illustrating a configuration of a terminal of a mobile communication system according to an embodiment of the present invention.

  In the figure, a terminal 10 (a mobile phone device with a camera) of the mobile communication system of the present embodiment includes an antenna unit 101 that transmits and receives a modulated wave of a signal, a receiving unit 102 for a modulated wave, and a transmitting unit 103 for a modulated wave. A demodulating unit 104 that demodulates the received modulated wave, a modulating unit 105 that modulates a signal, a data processing unit 106 that processes audio data and image data and converts them into an output signal, and an audio signal that is output from a microphone or a speaker Is converted into a digital signal, and an image data processing unit 108 that converts a camera image signal into internal data is provided. Also, an animation image generation unit 109 that generates an animation image from image data, a three-dimensional correspondence unit 110 that processes the animation image into a three-dimensional image, a thumbnail processing unit 111 that processes the image data into a thumbnail image, and an external device A sound recording unit 112 that records the sound data of the external device, an external I / F unit 113 that inputs a sound signal from the external device, and a data format conversion unit 114 that converts the sound signal from the external device into sound data. . In addition, a speaker unit 121 (speech output unit) that outputs a voice signal, a microphone unit 122 (speech input unit) that inputs a voice of a caller, a camera unit 123 (imaging unit) that images a subject, and a shutter unit 124.

  Furthermore, a CPU 131 for controlling the whole, a storage 132 for storing control programs and data, an operation 133 for inputting operator commands and data, and a display 134 (display) for displaying messages and data Means), a notification unit 135 that notifies an incoming call, a power source unit 136 that transmits driving power, and a power source unit 136 that supplies driving power.

  Further, as an optional external device, an audio playback device 20 such as a CD / MD player can be provided.

  Hereinafter, the mobile communication system (camera-equipped mobile phone device 10) according to the present embodiment will be described in detail for each embodiment.

  First, when transmitting audio data, the user's audio is input to the microphone unit 122 of the terminal 10 (FIG. 1), and further converted into a digital signal by the audio data processing unit 107. Thereafter, the data is input to the data processing unit 106. The data processing unit 106 performs various types of data processing (for example, error handling processing, encryption processing, rearrangement processing to each data frame, etc.) that matches the communication standard. Thereafter, the output data of the data processing unit 106 is sent to the modulation unit 105, subjected to modulation processing for transmission on radio waves, and amplified by the transmission unit 103. The modulated wave amplified by the transmission unit 103 is transmitted from the antenna unit 101 to the air.

  Next, when transmitting image data, the image data captured by the camera unit 123 (imaging means) is converted by the image data processing unit 108 from the image data into a dedicated data format that matches the communication standard. The data is sent to the data processing unit 106. The data processing unit 106 performs various types of data processing (for example, error handling processing, encryption processing, rearrangement processing to each data frame, etc.) that matches the communication standard. Thereafter, image data is placed on the data frame, amplified by the transmission unit 103, and then transmitted from the antenna unit 101 to the air.

  Next, when receiving audio data / image data, the radio wave fed to the antenna unit 101 is received by the receiving unit 102, the audio data and image data are extracted from the carrier wave by the demodulating unit 104, and digitally processed by the data processing unit 106. Processed as data. Thereafter, the audio data is converted from digital to analog signal by the audio data processing unit 107 and output from the speaker unit 121 as audio. On the other hand, the image data is converted into internal data having a predetermined image format by the image data processing unit 108 and displayed on the display unit 134 under the control of the CPU unit 131.

  In addition, a conventional mobile communication terminal (such as a mobile phone device) can send mail and image data in addition to voice calls in recent years. However, particularly in the case of image data, the amount of information of transmission data is large. . Therefore, in the mobile communication system of the present embodiment, the image data of the image taken by the camera unit 123 of the terminal 10 (FIG. 1) is received by the image data processing unit 108, and in the case of still image data, JPEG, GIF, Alternatively, it is converted into a data compressed data format such as PNG, and in the case of moving image data, it is converted into a format such as MPEG4 or CIF. These converted data are transmitted together with audio data after being formed into a data frame.

  In addition, as for the conventional mobile phone system, at present, time division multiple access (TDMA) or code division multiple access (CDMA) is mainly used, and digital data is intermittently received or transmitted by one user. Is a system that can be used by a plurality of users efficiently.

  2 shows an example of a format of transmission / reception data in the mobile communication system according to the embodiment of the present invention. FIG. 2 (a) shows a TDMA frame, and FIG. 2 (b) shows a data format of a control CH. Indicates.

  The format of transmission / reception data shown in the figure is the same as the format of transmission / reception data in a conventional mobile communication system.

  In the figure, a field (region) indicated by a symbol R is a guard time / 4 bits, a field indicated by a symbol P is a preamble / 2 bits, a field indicated by a symbol CAC is a control signal / 112 bits (or 116 bits), and a symbol SW The field indicated by symbol indicates synchronization word / 20 bits, the field indicated by symbol CC indicates color code / 8 bits, and the field indicated by symbol G indicates guard time / 18 bits.

  FIG. 2 (a) shows an example of a TDMA frame in a communication system using the TDMA system. As shown in FIG. 2, a plurality of frames are regularly arranged. By using such a format, each of a plurality of users can use a dedicated frame in a time-sharing manner. As a result, a plurality of users can use the communication system at the same time.

  For example, PDC, which is a transmission method of a Japanese digital cellular phone system, uses a repetitive format of 36 time-divided TDMA frames called superframes, and the full rate is 3 multiplexed (3 people use simultaneously), half The rate is 6 multiplexed. That is, in the frame (number: 0, 1, or 2) shown in FIG. 2 (a), one user uses the same numbered frame.

  In this case, each time frame has a data format composed of a fixed number of bits, and has a plurality of data formats called CH. This data format is configured for each purpose and has a different bit configuration.

  FIG. 2B shows a data format of a general control channel (hereinafter, “channel” is described as “CH”), and one frame is composed of 280 bits. The bit configuration differs depending on the data format, but information bits are inserted in the area (112 bits and 116 bits) indicated by the code CAC. The other bits are control bits for smoothly performing communication between the base station (not shown) and the terminal 10.

  The number of bits is a specific example of the current PDC (Personal Digital Cellular), but in the future, in the method of a high-speed communication system called 3G (third generation), the number of bits will increase dramatically. Yes. In this case, even a large amount of data such as a moving image can be sent in a short time.

  Further, the data is packetized and transmitted on the packet dedicated CH. By the way, the data rate of the current PDC is about 64 [Kbps] to 128 [Kbps]. However, in the near future high-speed communication system called 3G, the bandwidth is expanded, so that the maximum data rate is 2 [Mbps], for example. As a result, a large amount of data can be communicated in a short time, and the amount of information that can be transmitted on the control CH is greatly increased.

  In the current mobile phone system, all information is converted into digital signals and transmitted / received. Therefore, information sent from the base station to the terminal 10 and information sent from the terminal 10 to the base station are all digital data.

  FIG. 3 is an explanatory diagram showing a method of disassembling / assembling data transmitted / received in the terminal of the mobile communication system according to the embodiment of the present invention.

  The transmission / reception data disassembly / assembly method shown in the figure is the same as the transmission / reception data disassembly / assembly method in the conventional mobile communication system.

  As shown in the figure, the data transmitted to the terminal 10 is intermittently received, assembled into one received data, and then converted into analog data.

  The data created by the terminal 10 is converted into digital data, and then decomposed and transmitted for intermittent transmission.

  As shown in the figure, each TDMA frame is processed with a time interval of exactly two slots. For example, when receiving the first 0 frame from the base station, the user in slot 0 transmits data from the terminal 10 to the base station after the elapse of two slots from the start of reception. Start receiving frame data.

  That is, in the mobile communication system (digital mobile phone system) according to the present embodiment, when data communication is performed from the terminal 10 to the other terminal via the base station, the transmission data is converted into digital data and then a dedicated TDMA frame. The received data is assembled into continuous digital data from the intermittently received data. Thereafter, it is converted into analog data, and in the case of audio data, it is output as audio from the speaker unit 121 (FIG. 1), and in the case of image data, it is displayed on the display unit 134 (FIG. 1). Such a system is the same as the system of the conventional mobile communication system.

  FIG. 4 is a sequence chart of a call process when calling a counterpart terminal to be called from a terminal in the mobile communication system according to the embodiment of the present invention.

  The user of the terminal 10 inputs the telephone number of the partner terminal to be called by the key input of the operation unit 133 of the terminal 10, and then makes a call request to the base station by pressing the transmission key of the operation unit 133 (calling). means). The access control CH is normally used as the CH used at this time. The data format of the access control CH is also as shown in FIG. 2B, and the access request command and the other party telephone number are inserted as digital data in the field CAC of the access control CH data and transmitted to the base station.

  On the other hand, upon receiving this information (access request command and destination telephone number), the base station accepts the access request from the terminal 10 and returns a request confirmation command to the terminal 10.

  Next, the base station confirms that the terminal 10 having this telephone number exists in the communication area, and makes a call using the paging CH. Thereby, the counterpart terminal notifies the user that there is an incoming call by ringing a ringtone. At the same time, the base station notifies the terminal 10 that the counterpart terminal is being called, and the terminal 10 also rings the ringing tone and informs the user that the counterpart terminal is being called. Notice.

  Thereafter, when the counterpart terminal responds to the incoming call notification (that is, the counterpart terminal requests the base station to set up a call), the base station and the terminal 10 communicate with the counterpart 10 so as to shift to a call CH prepared in advance. After instructing the terminal and shifting to the call CH prepared in advance, the call is started. Here, the base station control CH is used as the CH used by the base station for the reply to the terminal 10. The data format of this base station control CH is also the same as the data format shown in FIG.

  The above description is a simplified description of a general call process. Actually, there are other necessary tasks such as a radio quality check of an used CH and an authentication check. The description is omitted because it is not related to the communication system.

  In the following, based on the description of the general call process, when calling the partner terminal from the terminal 10 (mobile phone device), an image captured by the camera of the terminal 10 is being called during the incoming call at the partner terminal. An example of operation when displaying on the display unit of the counterpart terminal will be described in detail.

  FIG. 5 is an explanatory diagram showing selection items on the setting screen in the terminal of the mobile communication system according to the embodiment of the present invention, FIG. 5 (a) is a function setting screen, and FIG. 5 (b) is a call related function setting. FIG. 5C shows a past call data setting screen, FIG. 5D shows a call voice setting screen, and FIG. 5E shows a detailed function corresponding to each selection item of the call related function. .

  In the terminal 10 on the transmission side, first, a setting is made to use image transmission for an incoming call on the other side. Here, Mr. A who is the user of the terminal 10 selects the “call-related function” that is the fifth item on the setting screen shown in FIG. 5A of the terminal 10, and the setting shown in FIG. 5B. On the screen, the first item “use image for call information” is selected (an operation example when another item is selected for call information also conforms to the following description).

  With this setting, at the time of a subsequent call, a call using the access control CH is performed in this setting mode.

  In other words, in the case of the above setting, when the terminal 10 detects that it is in the calling mode (that is, when it is calling the other party to be called), the camera unit 123 can be automatically operated. . Here, although there is a distinction between a moving image and a still image as a captured image, when Mr. A operates the shutter unit 124 to select a still image, the moving image data is automatically excluded and the still image is displayed. Only images are valid as call information.

  In addition, when Mr. A first captures a moving image and then captures a still image, the first moving image is used as call information, and then the other party answers the call when the still image is obtained. Until then, the still image is displayed on the display unit of the other party's terminal (mobile phone device).

  When calling Mr. B's terminal from Mr. A's terminal 10 (calling), Mr. A inputs the telephone number of Mr. B's terminal from the operation unit 133. This telephone number is transmitted to the base station by the access control CH (call request).

  Thereby, as explained in the sequence chart of FIG. 4, the base station makes a response to the call request confirmation to the terminal 10 of Mr. A, and the base station uses the paging CH corresponding to the access control CH. Mr. B's terminal is started to be called, and at the same time, the base station notifies Mr. A's terminal 10 that Mr. B is being called. Simultaneously with this notification, the camera unit 123 becomes operable, and Mr. A starts imaging his / her face with the camera unit 123. At the same time, his / her face is displayed on the display unit 134 and the fact that Mr. B is being called is also displayed.

  The face data captured by Mr. A is converted into digital data by the image data processing unit 108, further converted into continuous digital data having a data format as shown in FIG. 6 to be described later, and temporarily stored in the storage unit 132. Is done. Note that the image captured by the camera unit 123 may of course be a surrounding landscape other than the face described above.

  Subsequently, the base station starts calling Mr. B's terminal by paging CH. At the time of this call, the face data captured by Mr. A once stored in the storage unit 132 is transmitted to the terminal having the telephone number (that is, Mr. B's terminal) via the base station. Here, the face data captured by Mr. A can be, for example, image data having a data format shown in FIGS. 6 and 7 to be described later. However, when the amount of information is small, it is not always packetized. There is no need to do. Whether or not packetization is possible is determined by the communication method (standard) adopted by the present system.

  When Mr. B who views the face data captured by Mr. A responds to this call, as explained in the sequence chart of FIG. 4, a call is made from the control CH in response to a request for call setting from the terminal of Mr. B to the base station. Move to CH. At this time, CH change is also instructed to the terminal 10 of Mr. A from the base station. When detecting the transition to the call CH, the CPU unit 131 of the terminal 10 stops displaying the captured image on the display unit 134 and displays “busy” on the display unit 134. This notifies Mr. A that Mr. B is answering the phone.

  FIG. 6 shows an example of a data format when packetizing transmission data using an access control CH for transmission to the base station. FIG. 6A shows a user packet data space as a CAC field. (See FIG. 2B.) Access control CH included in FIG. 2, FIG. 6B is image data inserted into the CAC field, and FIG. 6C is packetized image data inserted into the CAC field (call preparation) 6 (d) shows the audio data inserted into the CAC field, and FIG. 6 (e) shows the data format when the audio data in the CAC field is packetized.

  In the figure, fields S1 to S4 indicate audio data, and fields indicated by codes V1 to V4 indicate image data. The fields indicated by the symbols R, P, and G are as described with reference to FIG. In the following, for the sake of convenience, the audio data of the field indicated by reference signs S1 to S4 are indicated by reference signs S1 to S4, and the image data of the field indicated by reference signs V1 to V4 are indicated by reference signs V1 to V4. I do not distinguish.

  The data format shown in FIG. 6A is the data format of the access control CH when the access control CH is configured using the data format of the user packet CH suitable for high-speed communication of data with a large amount of information.

  In the data format shown in FIG. 6B, the image data V1 to V4 are packetized by the data processing unit 106, and each packet is denoted by reference numerals H1 to H4 as in the data format shown in FIG. 6C. Managed with header bits.

  In this case, tags are attached to the image data V1 to V4. This tag is mpeg or cif if it is a moving image, and tags such as jpeg, gif, png if it is a still image. The image type can be distinguished by this tag.

  FIG. 7 is an explanatory diagram showing an example of assembling image / audio data in the same data format as that of the user packet CH. FIG. 7A shows a data format when the call data is only an image, and FIG. b) shows a data format when the call data is both image data and audio data.

  The data format shown in FIG. 7A is that when Mr. A calls Mr. B, the face of Mr. A is imaged, the imaged data is transmitted using the user packet CH, and the other party is called. This is an example of the data format of the user packet CH, and the packetized image data of Mr. A consists of image data V1 with a header H1 after the call command data. Call command data configured in the same data format is transmitted by the access control CH.

  Since this call command data includes a call request for calling Mr. B and Mr. B's telephone number, when calling Mr. B from the base station according to the above-described call process (sequence chart shown in FIG. 4). The same user packet CH is also assigned to the data format of the paging CH. That is, the paging CH having the data format shown in FIG.

  Mr. B's terminal (hereinafter referred to as a terminal having the same configuration as Mr. A's terminal 10) receives this paging CH by the receiving unit 102 (FIG. 1) and demodulates it into analog data by the demodulating unit 104. Thereafter, the data processing unit 106 converts the analog data into digital data, and when the CPU unit 131 detects a call command included in the digital data, the CPU unit 131 indicates that there is a call by image or voice. Check the header bits of this digital data to inform. When the information indicated by the header bits is the headers H1 to H4, the CPU 131 determines that the packet data is image data, and after the image data processing unit 108 converts the data into analog data, the display unit 134 is displayed.

  Further, when the CPU 131 detects from the call command data that the packet data is just call data, the notification section 135 uses the ringing tone, lighting, or output pattern of the vibrator or the like set in advance by B Is notified of incoming calls by calling information.

  As a result, Mr. B can confirm Mr. A's moving image on the display unit 134. Therefore, Mr. B can surely confirm who is calling and whether it is an emergency request from the facial expression in the moving image. It is possible to answer this call by judging whether it is a normal business or not.

  Furthermore, when Mr. A picks up his face and uses it for calling and also uses voice for calling, selecting item 3 related to calling (see FIG. 5B), calling Mr. B. It is informed from the base station to Mr. A, and at the same time, the camera unit 123 and the microphone unit 122 operate. Accordingly, when Mr. A inputs sound into the microphone unit 122, the sound data processing unit 107 converts the analog sound data into continuous digital sound data indicated by fields S1 to S4 in FIG. 6D. Thereafter, the data processing unit 106 packetizes as shown in FIG. Each packet is managed with a header from J1 to J4 as shown in FIG. 6 (e).

  Further, when it is desired to use both image data and audio data as call data, as shown in FIG. 7B, the image data and audio data are inserted into the user-packet data space with headers, respectively. Just as in the case of Of course, when Mr. A wants to use only the voice for the call information, as described above, only the microphone unit 122 can be activated at the same time as Mr. B's call and only the voice data is sent out. Same as above, but the CH at the call information is the control CH and the call CH at the time of the call. Therefore, Mr. B will listen to Mr. A's voice continuously by changing the CH. There may be a slight time lag such as handoff.

  It is also possible to select the sixth item “change voice to different data” on the call-related setting screen shown in FIG. However, when selecting this item, it is necessary to first register the voice of another person other than Mr. A input from the microphone unit 122 in advance. Hereinafter, the process at the time of this pre-registration will be described.

  The voice of another person other than Mr. A input from the microphone unit 122 is converted into digital data by the voice data processing unit 107 and then stored (recorded) in the voice recording unit 112 under the control of the data processing unit 106. Is done. A name is assigned at the time of recording the voice, and thereafter the name is managed. In addition, after the digital output of the audio playback device 20 such as a CD player or MD player is input via the external I / F unit 113 in advance and converted to the audio standard of this communication system by the data format conversion unit 114. It is also possible to store (record) the voice recording unit 112 for use.

  FIG. 8 shows an example of a memory map of the storage unit provided in the terminal of the mobile communication system according to the embodiment of the present invention.

  All of the above-described captured images are stored in the storage unit 132, but before this storage, this image is processed into a thumbnail image by the thumbnail processing unit 111, and this image is further processed into the processed thumbnail image. And stored in the storage unit 132.

  That is, a tag indicating the format is attached to the captured image, and is stored in the storage unit 132 together with the thumbnail image. Accordingly, Mr. A can display a list of image data used in the past on the display unit 132 as thumbnail images, and can be used as subsequent call information.

  When Mr. B's terminal 10 receives the packet data having the data format shown in FIG. 7B, the data processing unit 106 converts the packet data into only the image data shown in FIG. 7A. , Convert to digital data. Thereafter, when the call command data is first detected, the CPU unit 131 notifies the notification unit 135 of the incoming call, and then detects the header bit. Here, by detecting the header bit H1, the image data V1 is stored as image data in the storage unit 132, and by detecting the header bits J1 and J2, the audio data S1 and S2 are stored as audio data in the storage unit 132. The Next, the header bit H2 is detected, and the image data V2 is stored in the storage unit 132 as image data. Finally, the header bit H3 is detected, and the image data V3 is stored in the storage unit 132 as image data.

  After confirming that there are no more headers, the CPU unit 131 sends the image data stored in the storage unit 132 to the image data unit 108 in a batch and instructs conversion to an analog image. At the same time, the audio data stored in the storage unit 132 is also sent to the audio data processing unit 107 at a time to instruct conversion to analog audio.

  Accordingly, the image data processing unit 108 displays the moving image of Mr. A on the display unit 134, and the audio data processing unit 107 outputs the sound of Mr. A from the speaker unit 121. As a result, Mr. B can receive a call with Mr. A's face image and voice uttered by Mr. A.

  In addition, the user can use an animation image that is an example of a still image as call information. For example, an animated image can be used when the face image of his / her real live varies from day to day and he / she feels a little embarrassed by himself / herself. Conventionally, there are many such ideas for animation, and recently, an idea of using an animation image called an avatar as a user's transformation has been put into practical use.

  However, an animation image that is an example of a conventional still image is not created based on the self-portrait of the user on the transmission side, and is transmitted as an animation image that is not similar to the user on the transmission side. For this reason, the user of the transmission destination cannot estimate who the transmission is.

  In the terminal 10 of the mobile communication system according to the present embodiment, the transmission-side user's face is imaged, and the animation image generation unit 109 generates an optimal animation image from the captured image. It is converted into a three-dimensional animation image and used for call information.

  Therefore, it is a necessary condition that this animation image is a human face imaged as a still image. Also, in order to animate with the limited information processing capability of a mobile phone device, emphasis is placed on creating such three-dimensional face data by aligning feature points and pasting textures.

  The feature points when extracting the face of the user on the sending side are 7 points for eyes (both), nose, mouth, chin, cheek (both), and 3 hairstyles. Estimate the direction of the, and create a front-facing texture. By mapping this texture with a three-dimensional model, a three-dimensional animation image is generated.

  Of course, by registering in advance the information that can be emphasized by the user on the transmission side in the animation image generation unit 109, the facial expression of the transmission user on the 3D animation image of the user on the transmission side, for example, “crying” It can also be combined with facial expressions such as “I want” and “I'm angry”.

  FIG. 9 shows an example of a method for generating an animation image from a captured image of the user's face. FIG. 9A shows a display screen of the captured image of the user's face, and FIG. The processing screen of the animation image produced | generated from a captured image is shown.

  The captured image of the user's face is displayed on the display unit 137 at the time of registration. Further, an animation image generated from the captured image can be displayed by operating the operation unit 133. Further, items that can be selected for processing (representing and synthesizing) the animation image are displayed on the display screen of the animation image.

  In the animation image processing screen shown in FIG. 9B, for example, the user presses the numeric key 2 of the operation unit 133 corresponding to the number of the mouth, and the mouth of the animation image is packed. By depressing the numeric key 3, it is possible to perform expression composition such as tears flowing from the eyes of the animation image.

  For example, when the number key 2 is pressed and the expression is synthesized, the user on the transmission side, for example, voices, for example, “I am hungry, do not go to meals” when the call is made. Speaking toward the microphone unit 122), the other party's terminal displays a 3D animation in which the mouth of the user on the transmitting side is packed, and at the same time, as a ringing tone, “I am hungry, do not go to meals” , Voice to be uttered is output.

  In order to display the 3D animation, the user on the transmission side selects and determines “use 3D animation” as the fourth item in advance on the call-related setting screen shown in FIG. 5B. It is necessary to keep it.

  When a call is made, a call request is made to the other party to be called, and a self-portrait by the camera unit 123 is captured by operating the shutter unit 124, and at the same time the own 3D animation image is displayed on the display unit 134. The same 3D animation image is transmitted as incoming information, and voice data is also transmitted in the same manner as moving image data by inputting voice into the microphone unit 122.

  Incidentally, in the current communication system, there is no charge for the exchange (communication) between the terminal 10 and the base station using the control CH.

It is a block diagram which shows the structure of the terminal of the mobile communication system which concerns on embodiment of this invention. 1 shows an example of a format of transmission / reception data in a mobile communication system according to an embodiment of the present invention, where (a) shows a TDMA frame and (b) shows a data format of a control channel. It is explanatory drawing which shows the decomposition | disassembly / assembly method of the data transmitted / received in the terminal of the mobile communication system which concerns on embodiment of this invention. It is a sequence chart of a call process in the case of calling the other party terminal which wants to call from a terminal in the mobile communication system concerning the embodiment of the present invention. It is explanatory drawing which shows the selection item of the setting screen in the terminal of the mobile communication system which concerns on embodiment of this invention, (a) is a function setting screen, (b) is a setting screen of a call related function, (c) is the past. A call data setting screen, (d) shows a call voice setting screen, and (e) shows detailed functions corresponding to each selection item of the call-related functions. FIG. 6 shows an example of a data format when packetizing transmission data using an access control CH for transmission to a base station, (a) is an access control CH including a user packet data space in a CAC field; (B) is the CAC field image data, (c) is the CAC field image data packetized, (d) is the CAC field audio data, (e) is the CAC field audio data packetized The data format is shown respectively. It is explanatory drawing which shows an example in the case of assembling image / audio data by the data format similar to the user packet CH, (a) is a data format in case call data is only an image, (b) is call data is image data. The data format in the case of both audio data and audio data is shown. 1 shows an example of a memory map of a storage unit provided in a terminal of a mobile communication system according to an embodiment of the present invention. 1 shows an example of a method for generating an animation image from a captured image of a user's face, (a) shows a display screen of the captured image of the user's face, and (b) shows an animation image generated from the captured image. The processing screen is shown.

Explanation of symbols

10 terminal (terminal of mobile communication system of this embodiment)
20 audio playback device 101 antenna unit 102 receiving unit 103 transmitting unit 104 demodulating unit 105 modulating unit 106 data processing unit 107 audio data processing unit 108 image data processing unit 109 animation image generating unit 110 three-dimensional correspondence unit 111 thumbnail processing unit 112 audio recording Unit 113 external I / F unit 114 data format conversion unit 121 speaker unit 122 microphone unit 123 camera unit 124 shutter unit 131 CPU unit 132 storage unit 133 operation unit 134 display unit 135 notification unit 136 power supply unit 137 battery

Claims (15)

In a mobile communication system comprising a plurality of mobile communication terminals capable of transmitting and receiving audio data and image data,
Each of the plurality of mobile communication terminals is
Voice input means for inputting voice and obtaining voice data;
Imaging means for capturing an image of a subject and obtaining image data;
Voice output means for outputting voice data input by the voice input means or received voice data;
Display means for displaying image data captured by the imaging means or received image data;
In response to a call request, the image data picked up by the image pickup means and / or the sound data input by the sound input means are converted into a predetermined data format and then inserted into an information bit field of a specific communication channel. Call preparation means;
Calling means for calling using the specific communication channel prepared by the call preparation means;
A mobile communication system comprising:   The mobile communication system according to claim 1, wherein the specific communication channel used at the time of calling is a predetermined access control channel.   The image data and / or the audio data is temporarily stored in a storage unit, and when a call is made, the image data and / or the audio data is extracted from the storage unit by the call preparation unit and inserted into the information bit field of the specific communication channel The mobile communication system according to claim 1 or 2, characterized in that:   The mobile communication system according to any one of claims 1 to 3, wherein the image data is any one of moving image data and still image data.   When any one of the mobile communication terminals receives a call using the access control channel from any other one of the mobile communication terminals, the mobile communication terminal notifies the incoming call and the access control The mobile communication system according to any one of claims 1 to 4, wherein when voice data is inserted in a channel, the voice data is output by the voice output means.   When one of the mobile communication terminals receives an incoming call using the access control channel from the other one of the mobile communication terminals, the mobile communication terminal notifies the incoming call and transmits image data to the access control channel. The mobile communication system according to any one of claims 1 to 5, wherein the image data is displayed by the display means when the is inserted.   When any one of the mobile communication terminals makes a call to any other one of the mobile communication terminals, the base station confirms the call and notifies that the call is being made. The mobile communication system according to any one of claims 1 to 6, wherein the mobile communication system is transmitted to any one of call side mobile communication terminals.   The movement according to any one of claims 1 to 7, wherein the voice data inserted into the information bit field of the specific communication channel is voice data formed by inputting a voice of a caller. Communications system.   The movement according to any one of claims 1 to 8, wherein the image data inserted into the information bit field of the specific communication channel is image data obtained by capturing an image of a caller's face. Communications system.   The image data obtained by imaging the caller's face is converted to animation image data, and the animation image data is inserted into an information bit field of the specific communication channel. 10. The mobile communication system according to 9.   The mobile communication system according to claim 10, further comprising means for converting animation image data inserted into the information bit field of the communication channel into data of a three-dimensional image representation in advance.   The mobile communication according to any one of claims 1 to 11, further comprising means for converting the image of the image data and the animation image inserted into the information bit field of the communication channel into a thumbnail image. system.   A mobile communication terminal configured as a mobile communication terminal of the mobile communication system according to claim 1.   A cellular phone device configured as a mobile communication terminal of the mobile communication system according to claim 1.   The computer program for controlling the computer which performs the process of the mobile communication terminal in the mobile communication system of Claim 1 thru | or 12.

Images