JP3462267B2 - Information communication terminal equipment - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information communication terminal device, and more particularly to an information communication terminal device suitable for communication of so-called multimedia information such as characters, images and voices.

[0002]

When transmitting with a transmission path of the Prior Art limited bandwidth to images and audio information, it is impossible to compress the information. In that case, deterioration of image quality and sound quality is unavoidable, and how to reduce such deterioration or how much to allow such deterioration has been a problem. In the current situation, when the bandwidth of the transmission line is narrow, it is not possible to meet the demand for obtaining image / audio information with sufficiently small deterioration.

[0003]

[0004]

[0005]

[0006]

[0007]

[0008]

[0009]

[0010]

[0011]

The purpose of the 0008] The present invention is to provide an information communication terminal apparatus capable of transmitting information of high quality using a limited transmission path of the transmission band.

[0012]

[0013]

[0014]

[0015]

[0016]

[0017]

[0018]

In order to achieve the above-mentioned object, the information communication terminal apparatus according to the present invention has a first transmission mode for transmitting information to be transmitted in real time without passing through the first memory. A second transmission mode in which the information is transmitted through the first memory, and a part of the information is transmitted in real time without passing through the first memory, and another part of the information is transmitted. And a transmitter having a third transmission mode for transmitting the information via the first memory, and information transmitted from the transmitter in the first transmission mode in real time without passing through the second memory. A second receiving mode for receiving, a second receiving mode for receiving the information transmitted in the second transmitting mode via the second memory, and a third transmitting mode for transmitting the information. For information that has been transmitted in real time among the information, And a receiver having a third reception mode for receiving in real time the information transmitted through the first memory without passing through the second memory and receiving the information through the second memory. Is characterized by.

Further, the receiving device has a recording means, and for the information transmitted from the transmitting device in the first transmission mode, the information is recorded by the recording means,
Regarding the information transmitted from the transmitter in the third transmission mode, the information transmitted in real time among the information and the information transmitted via the first memory are integrated. It is characterized by recording by a recording means.

[0020]

[0021]

[0022]

[0023]

[0024]

[0025]

[0026]

[0027]

[0028]

[0029]

[0030]

According to the present invention, in addition to the conventional function of enabling real-time communication, the sender and the receiver have memories, and once the information to be transmitted is taken into the memory, the sender requests it. It has the function of encoding and transmitting with the desired image quality and sound quality. Therefore, when real-time communication is not required as in an answering machine, for example, the information can be taken into the memory once, and the information can be finely encoded and transmitted over time.

When real-time communication is required, in addition to the conventional mode that allows deterioration of image quality and sound quality to some extent,
Only the information that can be transmitted in a narrow band can be transmitted in real time by communicating in real time, and the remaining information necessary to create more precise information can be stored in the memory once, It can be transmitted to the receiver in non-real time. On the receiving side, the information transmitted in real time and the information once captured in the memory are integrated and recorded, so that the communication contents performed later can be reproduced with fine image quality and sound quality.

As described above, according to the present invention, while using a narrow band equivalent to that of the conventional one, the image quality of the same level as the conventional one can be obtained.
In addition to real-time communication with sound quality, it is possible to communicate fine information, so that information can be transmitted with the image quality and sound quality required by the user.

[0033]

[0034]

[0035]

[0036]

[0037]

[0038]

[0039]

[0040]

Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1) FIG. 1 is a block diagram of an information communication terminal apparatus according to the present embodiment.
(B) shows the receiving device.

In the transmitter shown in FIG. 1A, first, the image information is taken in through the camera 11 and the voice information is taken in through the microphone 12, and the image information and the voice information are encoded by the encoder 13. On the other hand, the image quality / sound quality (mode) required by the sender can be set from the input unit 14 such as a keyboard, and based on this, the mode information corresponding to the image quality / sound quality requested by the mode generation unit 15 is set. Is output.

The coding unit 13 determines at what compression rate the coding should be performed, that is, the coding rate, based on the mode information from the mode generation unit 15, and based on this, for example, the value of the quantization step size or 1 The number of frames to be encoded in a second is set. The encoded information from the encoding unit 13 is sent to the signal separation unit 16 at the next stage. The signal separation unit 16 separates and outputs the information to be transmitted in real time from the information to be transmitted in real time and the information to be transmitted anew, based on the mode information from the mode generation unit 15.
The multiplexing unit 18 multiplexes the coding information and the mode information from the signal separating unit 16. The output of the multiplexing unit 18 is transmitted to the receiving device shown in FIG. 1B via a transmission line (not shown).

In the present embodiment, the transmitter is the first to third transmitters.
Has a transmission mode of. Hereinafter, the operation of the transmission device
Each of the third transmission modes will be described.

<First Transmission Mode> Now, assuming that the user requests only real-time communication via the input unit 14, the mode generation unit 15 matches the bandwidth of the transmission line used at that time. The mode information indicating the coding rate is output to the coding unit 13, and the coding unit 13 adaptively changes, for example, the quantization step size so as not to exceed the coding rate, and performs compression coding. Further, in the signal separation unit 16, similarly, the mode generation unit 1
It is known from the mode information from 5 that all the coded information from the coding unit 13 may be sent to the transmission line, and therefore all the coded information from the coding unit 13 may be transmitted to the multiplexing unit 1.
It is sent out to the transmission line through 8.

<Second Transmission Mode> On the other hand, if only communication with fine information in non-real time is requested via the input section 14, the mode generating section 15 can obtain the requested fine information. The mode information indicating the coding rate of is output to the encoding unit 13, and the encoding unit 13 performs compression encoding according to the mode information. At this time, if the band of the transmission path is equal to or lower than the coding rate, it is impossible to send all the coding information output from the coding unit 13 in real time. Therefore, the signal separation unit 16 once stores all the coded information from the coding unit 13 in the memory 17 based on the information from the mode generation unit 15 that "all non-real-time transmission is sufficient". After that, the contents of the memory 17 are read out at a transmission rate suitable for the band of the transmission line and sent to the transmission line through the multiplexing unit 18.

<Third Transmission Mode> Further, it is assumed that a request is made through the input unit 14 for wanting to perform real-time communication, but for confirming the contents at that time again as a fine image later. At this time, since information on how much a fine image is required is also input from the input unit 14 at the same time, the mode generating unit 15 outputs mode information indicating a coding rate that can achieve the definition. It In the encoding unit 13, compression encoding is performed based on this information, and the signal is sent to the signal separation unit 16. Here, the amount of information capable of real-time communication, that is, the transmission rate below the band of the transmission path is used. Information is separated so that For example, the amount of information is calculated from the low frequency component, and when the predetermined amount of information (the amount of information that can be transmitted through the transmission path) is reached, the remaining information is stored in the memory 17. The information thus captured in the memory 17 is also sent out to the transmission line through the multiplexing unit 18. Here, the output information from the signal separating unit 16 and the output information from the memory 17 are multiplexed by the multiplexing unit 18. Are transmitted to the transmission line.

Therefore, the information amount for real-time communication directly output from the signal separation unit 16 and the detailed information output from the memory 17 so that the sum of the information amounts of these two systems falls within the band of the transmission path. Are controlled in the signal separator 16 and the memory 17 by the mode information from the mode generator 15. Further, both the real-time communication information and the fine information have a header. The header of the real-time communication information has the information number, and the header of the fine detailed information has the header for the fine information. Numbers are added to indicate which real-time communication information the information is for fine definition.

On the other hand, the receiver has first to third reception modes corresponding to the first to third transmission modes of the transmitter. That is, although the receiving device performs different processing depending on each transmission mode, the determination is made based on the mode information sent from the transmitting side. The operation of the receiving device will be described below for each of the first to third receiving modes.

<First Reception Mode> First, when only real-time communication is requested, the information sent from the transmission path is directly passed through the separation unit 21 and decoded by the decoding unit 23, and an image is displayed. The information is displayed on the monitor 26, and the voice information is recorded on the speaker 2.
7 are output respectively. The decrypted information is recorded in the recording unit 2 such as a VTR or a disc memory.
5 can be stored, and the contents can be referred to later.

<Second Reception Mode> On the other hand, when only fine information communication in non-real time is requested, all output signals from the separation unit 21 are once taken into the memory 24 by the mode information indicating that. After that, the data is decrypted by the decryption unit 23 and output from the monitor 26 and the speaker 27. Also,
These pieces of information can also be recorded in the recording unit 25, and the contents can be referred to later.

<Third Reception Mode> Further, when there is a request to confirm the contents by fine information later while performing the real-time communication, the separation unit 21 uses the mode information indicating that there is information. Is separated into two lines. One is real-time communication information, which is directly decoded by the decoding unit 23 and output from the monitor 26 and the speaker 27. The other is all information including real-time communication information and definition information, which is based on the number information written in the header part of each information, from the same information source to the signal separation unit 21.
Memory 2 so that the information separated by
Taken in 4. After that, when the real-time communication is completed, it is read from the memory 24, decoded by the decoding unit 23 and output to the monitor 26 or the speaker 27, or recorded by the recording unit 25. The information recorded in the recording unit 25 is fine information, and the content can be confirmed with a fine image after communication is completed.

In this case, if the decoding unit 23 also has a memory, the information fetched from the separating unit 21 to the memory 24 is only the fine information, and the real-time communication information is the decoding unit 23. It is also possible to combine the two pieces of information stored in the internal memory and decode the fine information later before performing the actual decoding.

(Embodiment 2) FIG. 2 is a block diagram of an information communication terminal apparatus according to the present embodiment.
(B) shows the receiving device. In the description of the present embodiment, the parts corresponding to those in FIG. 1 are designated by the same reference numerals, and in this embodiment, the positions of the encoding unit 13 and the signal separating unit 16 of the transmitting device and the positions of the separating unit 21 and decoding unit 23 of the receiving device are Each of the points is the same as the case of FIG.
The difference is that is added. Similar to the first embodiment, the operation will be described for each mode.

(First Transmission Mode) Now, assuming that only real-time communication is requested via the input unit 14, the mode generation unit 15 will perform encoding that is in the band of the transmission path used at that time. It outputs mode information indicating the rate, and accordingly, so as not to exceed the encoding rate, for example, in the case of an image, separation of information by sub-sampling of pixels,
The signal separating unit 16 separates the image / sound information by processing such that only a part of the frame is subjected to real-time communication. In addition, in the encoding unit 13, similarly, the mode generation unit 1
It is known from the mode information from 5 that all the information from the signal separation unit 16 is sent to the transmission line, so that the quantization step size and the like are adjusted to compress and encode the amount of code that can be transmitted on the transmission line. And send it to the transmission path. Although the information is reduced by the signal separating unit 16 here, only the encoding unit 13 may be compressed to a desired definition and output.

<Second Transmission Mode> On the other hand, if only communication with fine information in non-real time is requested via the input unit 14, the mode generating unit 15 can obtain the requested fine information. The mode information indicating the coding rate of is output, the information is reduced by the signal separation unit 16 in accordance with the mode information, and the information is stored in the memory 17. After that, the data is read again from the memory 17 at a rate at which it can be transmitted through the transmission line, compression-encoded by the encoding unit 13, and then sent out to the transmission line. Although the information is reduced by the signal separating unit 16 here as well, in this mode as well, the signal separating unit 16 does not perform compression, and only the encoding unit 13 compresses to a desired definition and outputs. May be.

<Third Transmission Mode> Further, it is assumed that a request is made through the input unit 14 to perform real-time communication, but to confirm the contents at that time again as a fine image again. At this time, how much fine image is required is also input from the input unit 14 at the same time.
The mode generation unit 15 outputs mode information that indicates a coding rate that can achieve the definition. Signal separation unit 1
6, the signal is separated by sub-sampling or the like based on this mode information, and a part of the signal is separated by the encoding unit 1.
3 and the remaining signals are taken into the memory 17.
The encoding unit 13 adjusts the quantization step size and the like so that the amount of information that enables real-time communication, that is, the transmission rate equal to or less than the band of the transmission line, is compressed and encoded, and then sent to the transmission line. To be done. On the other hand, the information in the memory 17 is read so that the sum of the information and the information output from the encoding unit 13 during the real-time communication is equal to or less than the above transmission rate, encoded, and transmitted to the transmission line. Alternatively, the encoding of the information in the memory 17 may be started at the stage when the real-time communication is finished.

Both the real-time communication information and the fine information have a header. The header of the real-time communication information has the information number, and the header of the fine information has the header. A number is added to indicate which real-time communication information the fine-detailed information corresponds to.

On the other hand, the receiver has first to third reception modes corresponding to the first to third transmission modes of the transmitter. That is, although the receiving device performs different processing depending on each transmission mode, the determination is made based on the mode information sent from the transmitting side. The operation of the receiving device will be described below for each of the first to third receiving modes.

<First Reception Mode> When only real-time communication is requested, the information sent from the transmission path is directly decoded by the decoding unit 23 and passed through the separation unit 21 and the switching unit 28. Then, the image information is output to the monitor 26 and the audio information is output from the speaker 27. The decrypted information can be recorded by the recording unit 25, and the contents can be referred to later.

<Second Reception Mode> On the other hand, when only fine information communication in non-real time is requested, all the decoded signals of the decoding section 23 are separated by the separation section 2 by the mode information indicating the request.
It is taken into the memory 24 by 1. Then memory 2
The information is read out from the monitor No. 4 and passed through the switching unit 28 to be output from the monitor 26 and the speaker 27 at the speed actually taken in by the camera 11 and the microphone 12 in FIG. Also,
This information can also be recorded by the recording unit 25,
You can refer to the contents later.

<Third Reception Mode> Further, when there is a request to confirm the contents by fine information later while performing real-time communication, the decoding unit 23 uses the mode information indicating that request. The decrypted information is separated into two systems by the separation unit 21. One is real-time communication information,
This immediately passes through the switching unit 28 and is output from the monitor 26 and the speaker 27. The other is information including real-time communication information and definition information, which is separated by the signal separation unit 21 from the same information source based on the number information written in the header of each information. The information is captured in memory 24 so that the pieces of information are combined. After that, when the real-time communication is completed, the information is read from the memory 24 and the switching unit 28 outputs the information from the memory 24 from the monitor 26 or the speaker 27, or the information is stored in the recording unit 25. Will be recorded. The information recorded in the recording unit 25 is fine information, and its content can be confirmed by a fine image after communication is completed.

In this case, if the separation unit 21 also has a memory, the information fetched from the separation unit 21 into the memory 24 may be only the fine information, and the real-time communication information in the separation unit 21. It is also possible to combine the two pieces of information stored in the memory in the switching unit 28 before the actual output when the fine information is decoded later.

(Third Embodiment) FIG. 3 is a block diagram of an information communication terminal apparatus according to the present embodiment.
(B) shows the receiving device. It should be noted that the present invention is particularly effective for a portable information communication terminal in which the memory capacity cannot be made too large, and the transmitting device and the receiving device are provided in each terminal.

The transmitting apparatus shown in FIG. 3A has an encoding unit 3
1, a memory 32, a storable amount display unit 33, a switching unit 34, and a wireless communication unit 35. Here, the encoding unit 31
Has a function of compressing and coding the input image information and sending the coded information to the memory 32 and the switching unit 34, which are information storage means. The memory 32 accumulates the image information encoded by the encoding unit 31, and sends the image information to the switching unit 34 according to an instruction from the switching unit 34. The storable amount display unit 33 has a function of indicating the storable amount of the memory 32 of its own device and a function of obtaining the storable amount of the memory of the communication partner and indicating the value. The switching unit 34 has a function of transferring information from the encoding unit 31 and information from the memory 32 to the wireless communication unit 35 according to an instruction from the outside. The wireless communication unit 35 has a function of performing communication processing with each mobile terminal.

On the other hand, the receiving apparatus shown in FIG. 3B has a radio communication section 41, a switching section 42, a decoding section 43, a memory 44 and a storable amount display section 45. The wireless communication unit 35 and the wireless communication unit 41 may use a common wireless communication unit. The wireless communication unit 41 has a function of performing communication processing with each mobile terminal. The switching unit 42 has a function of transferring the received image signal obtained from the wireless communication unit 41 to the decoding unit 43 and the memory 44 according to an instruction from the outside. The decoding unit 43 has a function of performing a process of decoding an image signal from the image coding information input from the switching unit 42 and the memory 44. The memory 44 has a function of accumulating the image information transferred from the switching unit 42 and transferring it to the decoding unit 42 according to an instruction.
The storable amount display unit 45 has a function of indicating the storable amount of the memory 44 of the own device, and a function of transferring information for notifying the transmitting side of the storable amount of the memory 44 of the own device to the switching unit 42. It has a function of obtaining the accumulable amount in the memory of the communication partner (in this case, the memory 32 of the transmitting device) and indicating the value.

Next, the operation of this embodiment will be described.

In the transmitting device shown in FIG. 3A, when transmitting image information, first, at an arbitrary time, the switching unit 34
Is a switching unit 42 of the receiving device shown in FIG.
Is inquired about the amount of information that can be stored in the memory 44 via the wireless communication unit 35. Upon receiving this inquiry via the wireless communication unit 41, the switching unit 42 on the receiving device side receives the information on the storable amount, that is, the information on the remaining capacity of the memory 44 from the storable amount display unit 45, and switches on the transmitting device side. It is transmitted to the unit 34.

The switching unit 34 on the side of the transmitting device, which has received the information on the storable amount, records the value in the storable amount display unit 33. Then, the storable amount display unit 33 regulates the encoding unit 31 or the memory 32 according to the value displayed on the storable amount display unit 33, thereby controlling the transmission rate of the image information. The control of the transmission rate may be performed by the switching unit 34 according to the value displayed on the storable amount display unit 33.

As described above, according to this embodiment, the information transmission flow control such as image information from the transmission side is performed by transmitting the availability of the memory 44 on the reception side to the transmission side in the form of the accumulable amount information. It can be performed.

(Embodiment 4) Next, an embodiment concerning display of a transmission / reception screen in the information communication terminal device according to the present invention will be described. FIG. 4 is a configuration diagram of a computer system included in the information communication terminal device according to the present embodiment, which includes a keyboard 51, a mouse 52, and an arithmetic control unit (CPU) 5.
3, a disk memory 54, a RAM 55, a monitor 56, a speaker 57, a network controller 58 and a mail server 59.

(Embodiment 4-1) FIG. 5 is a transmission / reception screen of an information terminal according to an embodiment of the present invention. In the transmission screen shown in FIG. 5, To: 121 is a column for entering a destination, Su
bject: 122 is a column for entering the title of the communication content, character window 123 is a column for writing the communication text, Tex
At t: 124, a column for selecting whether or not to transmit the character information portion is selected by the display 127 of ◯ ×. Image: 1
25 is a column for selecting whether to transmit an image, Sou
nd: 126 is a field for selecting whether or not to transmit audio, and a selection field 129 selects whether or not to compress images and audio.

For image compression encoding, for example, JPEG, MPEG1, MPEG2, etc., which are international standard systems, and for audio compression encoding, for example, G.I. 728, LD-CELP and various 8 kbp
A system such as s, 4 kbps, 2 kbps, or 1 kbps can be selected.

The moving image is displayed on the window 131 together with a button-shaped icon 130 for performing an operation such as reproduction. By clicking this with a mouse, reproduction display, stop, reverse reproduction, etc. of the moving image can be executed. The position from the start point is displayed as a bar graph display 132 as a percentage of the total amount of the moving image file. Voice indicates the file name column 1
A button display 133 and a bar graph 135 are displayed at 34. The character, image, and audio information displayed on the transmission side screen is designated by the above selection operation and is transmitted by the transmission button 128.

This transmission operation can be executed by the computer system shown in FIG. That is, the keyboard 5
The operation control unit 53 creates character data by the operation 1 and writes it in the large-capacity disk memory 54 such as a hard disk, cuts out the display portion, and transfers it to the RAM 55 which is a display memory. The information for one screen stored in the RAM 55 is sent to the monitor 56 and displayed. Video,
The voice data can be taken in if necessary. The data stored in advance in the disk memory 54 is taken out as needed, and the moving image data is displayed as a moving image in the window 131 of FIG. 5 by sequentially rewriting the disk memory 54. In addition, the voice is output from the speaker 57 according to an operation instruction from the mouse 52 or the like. The transmission data is sent to the electronic mail server 59 via the network controller 58 and sent to the receiving side via the line.

On the receiving side, the data is restored by the operation almost opposite to the above, and the display is performed as shown in FIG. 5 (b). That is, the received data is numbered to the receiving side and entered in the column 136 on the left side of FIG. Further, the sender name is in the next column 137, and the reception date or date is the column 138.
, The title is entered in the field 139. In the case of image data such as a moving image, the first frame, an appropriate frame other than the first frame, and the like are taken out, the compression-coded data is restored, and the screen size is reduced for heading to make a small window. It is displayed in the columns 140a, 140b, 140c and the like. If there is voice information, marks 141a and 141b indicating it are displayed. When there is no voice, the mark column 141c is blank, is in thin letters, or is indicated by the column itself being omitted. In these tabular lists, when the information section is clicked with a mouse or the like, character information is displayed in a window. If a user clicks on the image heading portion, a moving image window (similar to 131 in FIG. 5A) opens, and by operating buttons on the upper part, moving image display, stop, etc. can be executed.

FIG. 6 is a functional block diagram showing the structure of the transmission side for realizing the display of the transmission / reception screen of FIG. In FIG. 6, text data 101 and moving image data 10
2. The voice data 103 is input. Text data 1
01 and the moving image data 102 are combined and edited by the combining / editing unit 104, and the audio data 103 is edited by the editing unit 105. The text data 101 is sent to the display unit 106 and the mail composition processing unit 110, the composition / editing result of the composition / editing unit 104 is transmitted to the display unit 106, and the mail data composition processing unit is compressed and encoded by the compression unit 108. 11
Sent to 0. The editing result of the editing unit 105 is the reproduction unit 107.
Is sent to the mail composition processing unit 110 after being compressed and encoded by the compression unit 109. Various control signals such as the control size position 111, the fetch section 112, the synchronization control 113, and the definite transmission data 114 given to each unit are input by the keyboard 51 or the mouse 52 in FIG.

(Embodiment 4-2) FIG. 7 is a display example of a screen according to the present embodiment. (A) is a screen before setting a headline, and the operation buttons at the top of the screen are buttons for instructing low speed reproduction. There is (SLOW) 151. If this button 151 is clicked, the moving image is set to the low speed reproduction or frame advance state, and the "heading setting button" 152 is clicked in the scene where it is desired to set as a heading, it is registered as a heading.
Headings 153 and 15 reduced to a small screen as shown in (b)
4 and the like are generated one after another. In this case, in addition to simply reducing the screen, it is possible to positionally specify a specific portion and register that portion in the index screen. As the method of transmitting the header information, (1) the transmitting side adds a flag indicating a header to a specific frame of the moving image sequence, and the receiving side reproduces the header screen from the flag information. If necessary, position information is added.

(2) The index information created on the transmitting side is added to the moving image information and transmitted. In this case, not only an image created from the first moving image series as a heading, but also a still image different from this, a mark 155 as shown in FIG. 7B, an image of a face image 156 of oneself, an image of a signature, or the like. Can be attached and transmitted.

FIG. 8 is a functional block diagram showing the structure of the transmitting side for realizing the display screen of FIG. In FIG. 8, moving image information is read from the moving image file 201 by the variable speed reading / display control unit 202. In this case, in addition to the standard speed data 203, the read / display speed can also be specified as high speed or low speed by the speed instruction data 204 from the keyboard 51 or the mouse 52 in FIG. Further, the setting memory 205 that can be set by the keyboard 51 or the mouse 52 generates a flag indicating a headline in a specific frame of the moving image sequence. The output of the setting memory 205 is displayed on the display unit 206 together with the moving image information from the variable speed reading / display control unit 202, and is further transmitted as transmission data. Upon receiving this transmission data, the receiving side displays a display as shown in FIG.

(Embodiment 4-3) FIG. 9 is a reception screen in this embodiment. When the receiving screen is small, the image caption screen is unavoidably small, so that the image is reduced to one line of characters as shown in FIG.

FIG. 10 is a block diagram showing the structure of the transmitting side for realizing the display screen of FIG. In FIG. 10, text data 301, moving image data 302, and audio data 303 are input, the text data 301 and moving image data 302 are combined and edited by the combining / editing unit 304, and the audio data 303 is edited by the editing unit 305. To be done. The text data 301 is sent to a mail composition processing unit (not shown).

The composition / editing result of the composition / editing unit 304 is
After reduction processing is performed by the reduction processing unit 306 to enable reduced display as shown in FIG. 9, it is sent to the display unit 308 and a mail composition processing unit (not shown) via the memory 307. The editing result of the editing unit 305 is sent to the reproducing unit 309 and also to the mail composition processing unit (not shown) via the memory 307. The control signal given to each part is input by the keyboard 51 or the mouse 52 of FIG.

(Embodiment 4-4) FIG. 11 is a display screen on the receiving side in this embodiment. In FIG. 4, the disk memory 54 such as a hard disk becomes full immediately after a large amount of moving image data is accumulated, which makes it inconvenient to access not only moving images but also voice and character information. Therefore,
It is desirable to move information to media that can store large amounts of data. In this embodiment, the name of the medium used as an auxiliary storage device different from the hard disk, which is the disk memory 54, is listed in the tabular display field 161. The HDD is a hard disk other than the disk memory 54, and includes, for example, D, E, F, G, and H. SC
SI is another hard disk having a SCSI interface, and there are eight hard disks from 0 to 7. MO is a magneto-optical disk, and Tape is a tape medium such as a magnetic tape. Magneto-optical disks and tapes may be connected offline, and auxiliary operations such as exchanging magneto-optical disks, inserting and rewinding tapes, and cueing are performed as necessary. The transfer to these media may be performed manually, but the usage amount (remaining capacity) of the disk memory 54, which is a hard disk, is constantly monitored, and if a certain limit is exceeded, the data is regularly transferred to the auxiliary storage device. You may transfer. After the transfer, for example, the transfer destination medium name 162, 1 such as SCSI or MO
63 or the like is displayed.

(Embodiment 4-5) FIG. 12 is a functional block diagram showing the structure of the transmitting side according to this embodiment. In FIG. 12, text data 401 and moving image data 402 are input. Text data 401 and video data 402
After being combined and edited by the combining / editing unit 404,
The mail composition processing unit 40 is compressed and encoded by the compression unit 405.
Sent to 6. The text data 401 is further sent to the mail composition processing unit 406 via the text addition unit 403.

The decoding program file 408 stores various types of decoding programs or IDs on the receiving side for compressed and encoded moving image data. The compression unit 4 according to the destination list (destination receiver capability list) 409.
The compression encoding method in 05 is selected, and the decoding program or ID corresponding to the compression encoding method is read from the decoding program file 408 and input to the mail composition processing unit 406.

Control information such as size, position, arrangement, and moving image capturing section is input by the keyboard 51 or the mouse 52 shown in FIG.

(Embodiment 5) FIG. 13 is a block diagram of a multimedia information transmission system according to this embodiment. This multimedia information transmission system includes a multimedia database device 510 and a multimedia information terminal device 520.
Consists of.

First, the multimedia database device 5
10, the multimedia data storage unit 511 stores a large amount of multimedia data such as multimedia documents and electronic mails by using a storage medium such as a magnetic disk device or a semiconductor memory device. Further, the data transmission unit 512 transmits each data in response to a request from the multimedia information terminal device 520 connected by the network. The data transmission control unit 513 divides the data in the database into fixed data segments according to the type of information requested by the user, and sequentially transmits the divided data segments to the multimedia information terminal device 520.

This data segment is divided into an appropriate size for each type of data such as characters, figures, voices, and moving images. For example, in the case of character data, a unit such as a line or paragraph in the display terminal is used. In the case of moving image data, this corresponds to a frame unit or a unit such as a part of one frame. Settings related to the configuration and size of these data segments may be determined by the administrator of the database system or the management program, or may be instructed by the user of the multimedia information terminal device 520.

Further, each data segment is accompanied by information for controlling display in the multimedia information terminal device 520, for example, display attribute information such as display position coordinates, display size, display color, font type, line type and the like. further,
The transmission order of each of these data segments may be dynamically changed, such as the order being changed depending on the data type and size of each data segment, or unnecessary data segments not being transmitted.

On the other hand, the multimedia information terminal device 520
In the data reception unit 521, each data segment transmitted from the multimedia database device 510 is transmitted.
And receive them in the multimedia data temporary storage unit 5
Accumulate in 22. Each accumulated data segment is displayed on the multimedia data display unit 523 under the control of the multimedia data display control unit 524 in response to the user's request. Multimedia data temporary storage unit 522
Is, for example, first in first out (FIFO) or LRU
It can be realized as a buffer storage device managed by an algorithm such as (Least Recently Used).
As a result, the data segment once transmitted can be re-displayed many times without retransmission for a certain period of time. Further, by using an expandable storage medium in the multimedia data temporary storage unit 522 so that the storable size is changed according to the purpose or frequency of use, more efficient information display can be performed.

Next, the operation of this embodiment will be described with reference to FIGS.

14 and 15 show a method of transmitting a data segment by taking moving image data as an example.
FIG. 14 shows an example of a procedure for transmitting moving image data consisting of 6 frames on the database and transmitting this to the multimedia information terminal device 520. Here, each frame (frame) is treated as a data segment of a moving image. Frame 1 is transmitted first, then frames 2 and 3 are skipped, and frame 4 is transmitted. After that, by transmitting frames 2, 5, 3, and 6 in this order, all moving image data can be transmitted.

Here, the time T when the transmission of the frame 4 is completed
In the case of 1, the multimedia information terminal device 520 has already received two frames, so by displaying frames 1 and 4, the user can distinguish the approximate image of this moving image. If it is found that the moving image is unnecessary, it is not necessary to send the moving image thereafter. On the other hand, if necessary, the reproduction of the remaining frames is completed while the reproduction is repeated several times, so that complete moving image data can be reproduced at time T3. In addition to the above, the first frame, then the last frame, and then
Various orderings are conceivable, such as ordering in the binary search method such as intermediate frames.

On the other hand, in FIG. 15, there is a moving image consisting of three frames on the database, and a quarter of each frame
Is a data segment. As shown in the figure, for example, frame 1-part (c), 2-
(C), 3- (c), the lower left part of the image is transmitted first, and then the data segment is transmitted as part (d), part (b), part (a). By sending the part that changes more drastically first, the user can know the points of the moving image data in the multimedia information terminal device 520 earlier.

In FIG. 16, the transmission data can be reduced by transmitting by reducing the spatial resolution of the image on the database. As a result, the frame can be transmitted quickly until the end of the frame, and the user can quickly check the rough contents of the image. On the display side, the screen size is reduced and displayed. If the image is displayed with the image quality reduced, it can be displayed in the same size as the image on the database.

FIG. 17 is an example of application to an electronic mail system using the above-described method of transmitting moving image data, for example. Here, the character string segments T1 ... T8 and the moving image segments V1a ... V3d are mixed and transmitted. At the time t1 when the transmission of the segment V1d is completed, the user can roughly understand the moving image and the electronic mail and the intended purpose, and continue the reception if necessary, or stop the reception if not necessary. ,
You can move to the next email display. It is possible to completely reproduce the moving image data in the state where all the transmission is completed.

The transmission method of the moving image and the character string data segment has been described above with reference to FIGS. 14 to 17, but the data such as the figure and the voice are segmented in the same manner, and the transmission order or the transmission order is changed. By doing so, it is possible to realize the transmission and display of multimedia data that meets the user's request. In the case of voice, in the case of audio multiplex or stereo system in which a plurality of channels exist, it is possible to transmit from one channel first, and in the case of a figure, it may become a key figure. An implementation method is possible in which the graphic type or the graphic having a large display size is preferentially transmitted.

FIG. 18 shows a data transmission / reception system selection unit 5 provided in the multimedia information terminal device 520 of FIG.
25 shows an example of the setting screen when the user selects the type of information (character, figure, image, voice) or display content to be displayed using 25.

(Sixth Embodiment) FIG. 19 is a block diagram of an information communication terminal apparatus according to the present embodiment, (a) shows a transmitting apparatus, and (b) shows a receiving apparatus.

In the transmitting device shown in FIG. 19A, an image signal and an audio signal are input. The power analysis unit 601 obtains signal power in order to analyze the state of voice, determines that the signal power is voiced when the signal power is higher than a threshold value set in consideration of noise, and is equal to or lower than the threshold value. In the case of, it is determined that there is no sound. The information amount analysis unit 602 analyzes the information amount of a moving image. For example, the difference between the target frame (current frame) and the previous frame is obtained, and the residual signal is summed, or the code amount of each frame before being encoded and quantized by the image encoding unit 605. To calculate. It is determined that the amount of information is large when these values are large, and that the amount of information is small when these values are small.

The rate distribution unit 603 is the power analysis unit 601.
How to allocate the bandwidth of the communication path to the transmission of voice and image is determined according to the state determined in. When the voice is silent, most of the band is allocated so as to be used for image transmission, and when there is little image information and the voice is voiced, the band of the voice is widened. Speech coding unit 604
The image encoding unit 605 compresses and encodes the audio signal and the image signal based on the code amount determined by the rate distribution unit 603. On the other hand, the rate distribution unit 6 is set at regular intervals.
The flag generation unit 606 creates a flag indicating the rate distribution determined in 03. The multiplexing unit 607 multiplexes the coded signals from the audio coding unit 604 and the image coding unit 605 with a flag generated by the flag generation unit 606 and outputs the multiplexed signal.

FIG. 23 shows an example of rate distribution of images and sounds. From 1 frame to 20 frames, there is sound and there is a large amount of image information, so each image and sound is 8 Kbps.
, And 4 Kbps. Also, since 21 to 40 frames are sound and the amount of image information is small, images and sounds are divided into 6 Kbps and 6 Kbps, respectively, and since 41 to 70 frames are silent, only images are 12 Kbps. Is encoded with. At this time, the flag generator 606 multiplexes the transmission rate of the image signal as a flag.

On the other hand, the receiving apparatus shown in FIG. 19B receives the multiplexed signal transmitted from the transmitting apparatus of FIG. 19A, reads the rate distribution of voice and image from the flag, and separates them. The unit 608 separates the audio and image encoded signals. The encoded signals of voice and image are decoded and output by the voice decoding unit 609 and the image decoding unit 610, respectively.

As described above, by changing the rate distribution according to the state of voice or image, it is possible to assign a large rate to a signal having a large amount of information, and it is possible to effectively utilize the communication path.

(Embodiment 7) In the conventional motion-compensated prediction, since the information of only the image is used, whether to search uniformly in all directions or to finely select the direction with less prediction error in all directions by tree search. I was exploring. Image encoding unit 6 of the seventh embodiment
In 05, using the voiced / non-voiced determination information from the power analysis unit 601 of the voice, when the voice signal is in the voiced section, the motion vector search points are increased in the vertical direction near the mouth of the face in the center of the screen. You can also By setting the motion vector search point based on the state of the voice in this way,
It is possible to search for an appropriate motion vector suitable for voice information.

This embodiment will be described more specifically with reference to FIG. FIG. 20 shows the configuration of a moving picture coding apparatus based on the motion compensation prediction method that can be used as the picture coding unit 605 of FIG. The subtracter 701 takes the difference (prediction error) between the image signal input to the terminal 700 and the prediction signal from the motion compensation prediction circuit 709. This prediction error signal is converted into a DCT coefficient by a DCT (discrete cosine transform) circuit 702, further quantized by a quantization circuit 703, and then variable length coded by a variable length coding circuit 711.

On the other hand, the output of the quantization circuit 703 is locally decoded through the inverse quantization circuit 704 and the inverse DCT circuit 705, added with the prediction signal by the adder 706, and then motion-compensated and predicted through the frame memory 708. It is input to the circuit 709. The motion compensation prediction circuit 709 obtains a motion vector from the input moving image signal and the image signal stored in the frame memory 708, and performs motion compensation prediction using this motion vector to create a prediction signal. The motion vector information is also input to the variable length coding circuit 712. The outputs of the variable length coding circuits 711 and 712 are the multiplexer 71.
It is synthesized in 3 and output.

Here, the search point of the motion vector in the motion compensation prediction circuit 709 is the motion vector search point setting unit 7
It is set by 10. The motion vector search point setting unit 710 receives the voiced / non-voiced determination information from the power analysis unit 601 in FIG. 19, and based on the voiced / non-voiced determination information, the voice signal is in the voiced section as described above. In the case of 1, the motion vector search points are set so as to increase the motion vector search points in the vertical direction near the mouth of the face in the center of the screen.

(Embodiment 8) FIG. 21 is a block diagram of an information communication terminal apparatus according to this embodiment, (a) shows a transmitting apparatus, and (b) shows a receiving apparatus.

In the transmitter shown in FIG. 21 (a), the transmitter shown in FIG.
A signal having two or more pieces of media information such as accumulated image / sound / character data as shown in (a) is compressed and transmitted with a predetermined bandwidth. That is, the information generation schedules F1 and F of each medium detected from the accumulated signals
2, ..., Fn are input to the scheduler 611. Figure 24
If a moving image is to be sent at once as in (b), the moving image must be compressed according to the transmission bandwidth, and there is a vacancy in the transmission bandwidth in the period A until the beginning of sending the moving image.
When the moving image information is expanded in the past direction of time and the band is compressed by using this free band, the amount of information lost is reduced, and therefore the image is less deteriorated. Thus, in order to perform compression based on the future information generation schedule, the scheduler 611 determines the transmission schedule and transmission bit rate of each medium. For example, when transmitting the signal of FIG. 24A, 0 to 60 of the period A as shown in FIG.
Up to seconds 10Kbps, 2K for video, text, and audio respectively
The video and audio are transmitted at 12 kbps and 4 Kbps, respectively, during the video period from 60 seconds to 90 seconds. Let us roughly estimate the transmission rate when following such a schedule. Video transmission rate is 9
It becomes 60 Kbps, and if the moving image is sent at 10 Hz, the number of frames becomes 300, so that 3.
It will be compressed to 2 Kbits. When transmitted as it is as shown in FIG. 24B, the moving image becomes 1.2 Kbps per frame.

In this embodiment, since the information generation schedule is sent together with the multimedia signal, it becomes possible to send at a transmission rate higher than that in FIG. In accordance with the transmission rate determined by the scheduler 611, the encoding unit 612 determines each media signal S1,
S2, ..., Sn are encoded. The encoded signal is expanded in the buffer 613 in the time direction, and the multiplexing unit 614 is performed.
In, each media signal and information generation schedule are multiplexed and output.

On the other hand, in the receiver shown in FIG. 21 (b),
The separation unit 615 separates each media signal from the information generation schedule, sends the media signal to the buffer 616, and sends the information generation schedule to the timing generator 617. The buffer 616 and the timing generator 617 perform decoding in the decoding unit 618 according to the information generation schedule and output.

As described above, according to this embodiment, a transmission schedule is created from the future information generation schedule so that the scheduler 611 can effectively use the allowable transmission band.
By performing the encoding based on this, it is possible to improve the quality of the medium (image quality / sound quality).

(Ninth Embodiment) FIG. 22 is a block diagram of an information communication terminal device for remote monitoring according to this embodiment.
(A) shows a transmitter and (b) shows a receiver.

In FIG. 22A, the monitoring camera 621
The image information obtained in (1) is encoded and multiplexed in the image encoding / multiplexing unit 622. The multiplexed signal is an information distribution unit 626.
Are distributed to several communication channels and transmitted to the receiving side.

In the receiving apparatus shown in FIG. 22B, the signal transmitted through the transmission path is combined by the information combining section 631, further separated and decoded by the separation / image decoding section 629, and output to the monitor 630. .

In the present embodiment, normally, the minimum number of lines is used to transmit information for monitoring with little change in the image. When information exceeding the capacity of the connected communication path is generated, a change in the information amount of the multiplexed signal is monitored by the information amount monitor 623 in the transmitting device in order to connect a new line to increase the transmission capacity. Here, when information exceeding the current transmission capacity is detected, the communication path control unit 624 connects the information distribution unit 626 to the new line. The control signal multiplexing unit 625 multiplexes the line control signal into a signal of a communication channel that is normally connected and sends the signal to the receiving device side.

In the receiving apparatus, the control signal separating section 628 separates the above-mentioned line control signal, and a new line is connected to the receiving side according to this control signal. A signal indicating that the connection of the communication path is completed is sent back to the transmitting device side, and the information distribution unit 6
26 confirms the connection completion and starts distributing information to each line.
Until the new line is connected, the overflow signal is stored in the buffer 627 and is transmitted after the new line is connected.

As described above, according to the present embodiment, the minimum number of lines is used when there is little change in the image, and a new line is connected and transmitted when the amount of information exceeds the transmission capacity. The signal overflowing before connection is stored in the buffer 627 so that information is not lost, and the number of lines is controlled by the generated code amount of the monitor image, so that the line use cost can be reduced. it can.

[0122]

As described in the foregoing, main according to the present invention
A mode to send and receive in real time without memory, and a memory
Mode for sending and receiving via the memory and part of the information
Send and receive the remaining part through the memory in real time without going through
You can select the mode to
Although it is being used, it has the same image quality and sound quality as before.
In addition to time communication, it becomes possible to communicate detailed information,
It is possible to transmit information with the image quality and sound quality required by Za.

[0123]

[0124]

[0125]

[0126]

[0127]

[0128]

[0129]

[0130]

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an information communication terminal device according to a first embodiment.

FIG. 2 is a block diagram showing a configuration of an information communication terminal device according to a second embodiment.

FIG. 3 is a block diagram showing a configuration of an information communication terminal device according to a third embodiment.

FIG. 4 is a block diagram showing a configuration of an information communication terminal device according to a fourth embodiment.

FIG. 5 is a diagram showing an example of a display screen according to Example 4-1.

FIG. 6 is a functional block diagram showing a configuration of a transmission side that realizes the display screen of FIG.

FIG. 7 is a diagram showing an example of a display screen according to Example 4-2.

8 is a functional block diagram showing a configuration of a transmitting side that realizes the display of FIG.

FIG. 9 is a diagram showing an example of a display screen according to Example 4-3.

10 is a functional block diagram showing a configuration of a transmitting side that realizes the display of FIG.

FIG. 11 is a diagram showing an example of a display screen according to Example 4-4.

FIG. 12 is a functional block diagram showing a configuration of a transmission side according to the embodiment 4-5.

FIG. 13 is a block diagram showing a configuration of an information communication terminal device according to a fifth embodiment.

FIG. 14 is a diagram showing a first transmission example of moving image data according to the fifth embodiment.

FIG. 15 is a diagram showing a first transmission example of moving image data according to the fifth embodiment.

FIG. 16 is a diagram for explaining a display method according to the fifth embodiment.

FIG. 17 is a diagram showing an example of transmission of electronic mail data according to the fifth embodiment.

FIG. 18 is a diagram showing an example of a data transmission / reception method setting screen according to the fifth embodiment.

FIG. 19 is a block diagram showing a configuration of an information communication terminal device according to a sixth embodiment.

FIG. 20 is a diagram showing the configuration of an image coding unit in the seventh embodiment.

FIG. 21 is a block diagram showing the configuration of an information communication terminal device according to an eighth embodiment.

FIG. 22 is a block diagram showing the configuration of an information communication terminal device according to a ninth embodiment.

FIG. 23 is a diagram showing rate distribution of audio and image information according to the sixth embodiment.

FIG. 24 is a diagram showing a transmission schedule according to the ninth embodiment.

[Explanation of symbols]

11 ... TV camera 12 ... Mike 13 ... Encoding unit 14 ... Input section 15 ... Mode generator 16 ... Signal separation unit 17 ... First memory 18 ... Multiplexing unit 21 ... Separation part 22 ... Mode information extraction unit 23 ... Decoding unit 24 ... Second memory 25 ... Recording unit 26 ... Monitor 27 ... speaker

Continuation of front page (51) Int.Cl. ⁷ Identification code FI H04L 29/12 H04L 13/00 317 H04N 7/14 (72) Inventor Toshiaki Watanabe 1 Komukai Toshiba-cho, Kawasaki-shi, Kanagawa Prefecture Toshiba Corporation Research and Development Center (72) Inventor Koji Yamaguchi 1 Komukai Toshiba-cho, Sachi-ku, Kawasaki-shi, Kanagawa Toshiba Research and Development Center (72) Inventor Akinobe Yamashita Komu-shi Toshiba-cho, Kawasaki-shi, Kanagawa Toshiba Research & Development Center (72) Inventor Naoaki Kuratate 1-30 Oedonaka, Kita-ku, Osaka-shi, Osaka (56) In-house Toshiba Kansai Branch (56) Reference JP-A-2-17778 (JP, A) ) JP-A-4-301949 (JP, A) JP-A-5-167861 (JP, A) JP-A-4-40186 (JP, A) JP-A-55-134576 (JP, A) JP-A-1- 213085 (JP, A) (58) Fields surveyed (Int.Cl. ⁷ , DB name) H04L 29/06 H04L 12/54 H04L 12/58 H04L 12/64 H04L 13/08 H04L 29/12 H04N 7/14

Claims (2)

(57) [Claims] 1. A first transmission mode in which information to be transmitted is transmitted in real time without passing through a first memory, and the information is transmitted in the first mode.
And a second transmission mode for transmitting the information through the first memory, and transmitting a portion of the information in real time without passing through the first memory, and transmitting another portion of the information through the first memory. And a first reception mode for receiving information transmitted from the transmission device in the first transmission mode in real time without passing through a second memory. A second reception mode in which the information transmitted in the second transmission mode is received via the second memory, and information transmitted in the third transmission mode in real time. The third reception mode in which the received information is received in real time without passing through the second memory, and the information transmitted through the first memory is received through the second memory. And a receiving device having Information and communication terminal equipment. 2. The receiving device has a recording means, and for the information transmitted from the transmitting device in the first transmission mode, the information is recorded by the recording means, and the information is transmitted from the transmitting device to the third device. The information transmitted in the transmission mode is recorded by the recording means after the information transmitted in real time among the information and the information transmitted via the first memory are integrated and then decoded. The information communication terminal device according to claim 1, wherein