JP2002111780A - Image transmitter - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image transmitter applicable to plural communication lines of different data communication speeds and applicable in the communication line where the data communication speed largely fluctuates during communication. SOLUTION: This image transmitter is provided with an image encoding part (1) for encoding image data, a first storage part (2) for storing codes outputted by the image encoding part (1), an audio encoding part (3) for encoding audio data, a second storage part (4) for storing the codes outputted by the audio encoding part (3), a multiplex part (5) for assembling a transmission frame (37) from the codes stored in the first storage part (2) and the second storage part (4), a data transmission part (7) for transmitting the transmission frame (37) and a transmission speed setting part 6. Within a data amount transmittable at the transmission speed set by it, the multiplex part 5 assembles one or more transmission frames and instructs transmission.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image transmitting apparatus that encodes image data and sound data, and multiplexes and transmits generated codes, and more particularly to an image transmitting apparatus that can cope with a plurality of different data communication speeds.

[0002]

2. Description of the Related Art Japanese Patent Application No. Hei.
FIG. 7 shows a block diagram of an image transmission device of the type 1-16-1459. In this device, the image code output from the image transmission unit 91 is stored in a first storage unit, and the transmission unit 93 multiplexes a control code with the image code to assemble a transmission frame and transmits the frame. When the receiving unit 95 receives the transmission frame, the receiving unit 95 separates the transmission frame into an image code and a control code, the image code is stored in the second storage unit 96, and the image decoding unit stores the image code stored in accordance with the control code for each predetermined decoding unit. It was characterized by decoding. Here, the control code controls the activation of the image decoding unit 97 of the receiving device. Specifically, the control code indicates how many decoding units have been received at the time when it is received by the receiving device. In the image transmission device of the above, it is stored in a 1-byte header. The transmission and reception of multiplexed data is performed using a transmission frame as a basic unit, and the transmission frame includes a 1-byte header including a control code, an audio code, and an image code. The size of the transmission frame is an integral multiple of the frame size that can be efficiently processed on the communication path. This conventional image transmission apparatus has an effect that the delay time of an image is shortened and the increase in data accompanying transmission is small. Conventionally, the size of a transmission frame, the size of a multiplexed header and each code are determined in advance,
It was common knowledge of the sending side and the receiving side. Since an audio code is always included in a transmission frame, transmission of the transmission frame has been started by a sound encoding completion interrupt. When it was necessary to change the amount of audio code stored in a transmission frame, that information could be stored in a one-byte header. by this,
Even if data transmission is delayed due to transmission errors, etc.
The transmission speed of the audio code was temporarily increased to prevent interruption of the reproduced sound. However, such a mechanism can only correct the data transmission speed expected by the image transmission device when it is not achieved slightly.

[0003]

As described above, the conventional image transmission apparatus can be suitably implemented when the data communication path provides a substantially constant communication speed. As the data communication path, there are a telephone line using a modem and a data communication path such as ISDN, which provides a substantially constant communication speed. However, the data communication path in a PHS or a mobile phone includes two lines. However, the best-effort method is used in which the communication speed is reduced to half of the maximum speed by limiting the number of lines used to one according to the situation. Since the communication speed changes depending on the degree, there is no value guaranteed as a constant communication speed. In a conventional image transmission apparatus, since an audio code is always stored in a transmission frame which is a basic unit of processing, when applied to a significantly different line such as a data communication speed of twice or half, the audio code becomes There was a problem that the sound overflowed and the reproduced sound was interrupted. In order to prevent such a problem from occurring, the size and structure of the transmission frame must be newly redesigned. However, since these are common knowledge of the transmission / reception unit and are prerequisites for communication, these must be used. In the case of changing the parameters, there is a problem that it is necessary to negotiate and form an agreement between the transmitting device and the receiving device for parameters suitable for the line to be used. Also, if the maximum data communication speed cannot be used temporarily, the communication speed of the best-effort communication channel, which reduces the communication speed by half and maintains the line connection, will be maintained even after the negotiation has been concluded. When the speed changed, there was a problem that negotiations had to be made. In addition, when renegotiation is started, data transmission is temporarily stopped. Therefore, if renegotiation occurs frequently, there is a problem that transmission efficiency is reduced.

[0004]

Means for Solving the Problems In order to solve the above-mentioned problem, a first aspect of the present invention is to store an image encoding unit 1 for encoding image data and a code output by the image encoding unit 1. A first storage unit 2 that encodes audio data, an audio encoding unit 3 that encodes audio data, a second storage unit 4 that stores codes output by the audio encoding unit 3, and a first storage unit 2
And a multiplex unit 5 for assembling a transmission frame 37 from the codes stored in the second storage unit 4, and a data transmission unit 7 for transmitting the transmission frame 37, further comprising the multiplex unit 5, the audio encoding unit 3 notifies the multiplex unit 5 of the completion of audio encoding when the encoding of the sound data for a predetermined time unit is completed, The multiplexing unit 5 counts the number of times of the audio encoding completion notification. There is provided an image transmission apparatus characterized by assembling at least the transmission frames 37 and instructing the data transmission unit 7 to transmit.

According to a second aspect of the present invention, the transmission speed setting unit changes the set transmission speed during communication in accordance with a transmission speed at which the data transmission unit can transmit. An image transmission device according to the above is provided.

The image encoding unit 1 compresses and encodes input image data. As an image encoding method, JPEG of a known still image compression method and H.264 of a moving image compression method are used.
H.263 or the MPEG system can be used, and other systems can also be used. The image encoding unit 1 is an LSI
May be used as a hardware encoder or a software encoder.

[0007] The audio encoding unit 3 compresses and encodes input sound data. As an audio encoding method, TrueSpeech 8.5 or G.264 is used. Various encoding methods such as 729 and MP3 can be used, and can be executed by hardware or software.

[0008] The first storage unit 2 and the second storage unit 4 are Fi
rst In First Out memory. The transmission speed setting unit 6 stores the data transmission speed setting expected by the image transmission apparatus of the present invention, and instructs the multiplex unit 5.
This may be a selector for selecting one of a plurality of representative transmission speeds, or a memory for storing a number indicating the data transmission speed. The transmission speed setting unit 6 may hold a constant value set at the time of shipment from the factory, or may store a transmission speed selected by a user operation. Further, the transmission speed setting unit 6 according to claim 2 changes the set transmission speed during communication according to the transmission speed at which the data transmission unit 7 can transmit.

The multiplex unit 5 reads the codes stored in the first storage unit 2 and the second storage unit 4,
It is copied to a memory in the multiplex unit 5 to assemble the transmission frame 37, and may be executed by an arithmetic processing unit and software, or may be executed by dedicated hardware. The multiplexing unit 5 is provided with means for receiving the audio encoding completion notification and counting the number of times. This can use a counter circuit when the audio encoding completion notification is a hardware electrical signal. If the audio encoding completion notification is a software interrupt, event, or semaphore, the number of times is counted by software.

The transmission frame 37 is obtained by multiplexing an image code and an audio code to make a unit in transmission processing. The multiplex unit 5 further includes means for assembling one or more transmission frames 37 within the amount of data that can be transmitted at the transmission speed set by the transmission speed setting unit 6. Is limited in advance, the number of transmission frames 37 corresponding to the number is stored in advance, and the number of transmission frames 3 corresponding to the number is stored in advance.
7 is created. When the transmission speed set by the transmission speed setting unit 6 specifies an arbitrary number, a unit for comparing the number of transmission frames × size and the transmission speed × transmission period is provided, and the number of transmission frames is determined by the condition. Assemble to satisfy. Here, the transmission cycle is a predetermined number of times counted by the multiplex unit 5 times a time of a unit of encoding of a sound to be encoded by the audio encoding unit 3.

The data transmitting unit 7 transmits data with high reliability to the data receiving unit 11 located at the end of the communication path via the network 8, and uses software such as a data communication protocol and an error control protocol and modulation. Hardware such as devices and line interfaces. Claim 2
The data transmission unit 7 has means for notifying the transmission speed setting unit 6 that the transmission speed has changed during communication, but this may be a select signal for notifying the transmission speed to the outside, or the communication speed has been changed. At this time, an interrupt may be generated to notify the transmission speed setting unit 6 by software.

[0012]

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a preferred embodiment of the present invention. FIG. 2 is a block diagram of an image receiving apparatus suitable for use with the embodiment of FIG.

In FIG. 1, the transmission speed setting unit 6 is 14.4 kbp.
Select and specify one of three types: s, 28.8 kbps, and 57.6 kbps. The transmission speed is set by the user on a user interface (not shown), and the transmission speed setting unit 6
Is used as a setting at the time of negotiation with the data receiving unit 9 when the data transmitting unit 7 starts communication.

In FIG. 1, an image encoding unit 1 compresses and encodes input image data and outputs the code to a first storage unit 2. The image data is stored in the camera and the AD converter 9
Is generated as digital data. The camera and the AD converter 9 may be replaced by a digital video deck.

As a coding system of the image coding unit 1, a JPEG baseline system which is a known still image coding system is used. As shown in FIG. 3, the image is divided into seven block lines, and one block line is set as a decoding unit. A block line is an area in which a minimum coded unit (MCU), which is the minimum unit handled by the JPEG baseline method, is arranged in one horizontal row until the width of the screen is covered, where the MCU is a luminance component of 16 pixels vertically and 16 pixels horizontally. And data of two color difference components of 8 pixels vertically and 8 pixels horizontally. The decoding unit is a unit that the image decoding unit 12 performs decoding in one process on the receiving apparatus side in FIG.

In FIG. 1, the end address of the decoding unit code in the first storage unit 2 is notified from the image encoding unit 1 to the multiplex unit 5, and the multiplex unit 5 stores the end address in a table. .

The audio encoding unit 3 compresses and encodes the input sound data, and outputs the output code to the second storage unit 4. Sound data is stored in the microphone and AD converter 1
0 generated. Audio encoding unit 3 is Tr
Approximately 8.5 kbps audio data using a method called ueSpeech 8.5
To encode. This method has a unit of 30 milliseconds and 3
Output 2-byte fixed size code. The audio encoding unit 3 notifies the multiplex unit 5 when the encoding of one unit of 32 bytes is completed.

The multiplex unit 5 counts the audio encoding completion notification from the audio encoding unit 3 every 30 milliseconds, and performs the operation once every two times, that is, one every 60 milliseconds.
At this time, the transmission frame 37 having the structure shown in FIG. 5 is assembled.

FIG. 5 shows three types: one transmission frame 37, two consecutive transmission frames, and four consecutive transmission frames. The transmission frame 37 shown in FIG. 5 is a fixed-length data unit of 108 bytes, and includes a header 34, an audio code 35, and a transmission unit 36 of an image code. The header 34 is typically one byte, but can be extended to two bytes as needed.
The audio code 35 stores two unit codes of 32 bytes, that is, 64 bytes, and stores the image code in the remaining 43 bytes or a 42-byte area when there is a header extension. The stored image code is a transmission unit 36
The number including the last byte of the decoding unit of the image code is stored in the header 34 as a control code and transmitted.

First, the multiplexing unit 5 stores the second storage unit 4
The audio code of two units is read out from and copied to an internal memory. Next, the image code is read from the first storage unit 2 and copied to an internal memory. At this time, by referring to the last address of the decoding unit notified by the image encoding unit 1, the number of the last address of the decoding unit stored in the transmission unit 36 of the image code is set in a flag in the header 34. set.

FIG. 4 is shown in order to explain the decoding unit. FIG. 4 shows a code for one screen output from the image encoding device 1, which is divided at the boundary of the code 31 of the block line included therein, and a code for one screen similarly to the decoding unit 32 included therein. It is a figure which shows below what was divided by the boundary of, and shows both. That is, the decoding unit 32 is data that includes the code 31 of the block line, has a larger size, and has a byte boundary. The reason that the decoding unit 32 has a byte boundary is that data reception is notified in units of bytes, and if the image decoder 14 is activated after all the decoding units 32 have been received, one block line can be decoded. It is. As described above, the decoding unit is data that the image decoder 14 can execute one process without delay.

Since the code 31 of the block line does not often have a byte boundary, the last byte of the decoding unit 32 contains several bits of the code of the next block line. However, after the code of the last seventh block line, EO
Since the I (End Of Image) marker 33 follows, the EOI marker 33 is the last byte of the decoding unit. The number of the last byte of the decoding unit included in the transmission unit 36 of the image code shown in FIG. 5 is stored in the header 34 of the transmission frame 37 and transmitted. On the receiving device side, the image decoding units 14 need only be activated by the number of final bytes of the decoding unit. In the present embodiment, the decoding unit is determined by dividing one image into seven block lines. However, by reducing the decoding unit in this way, the time from when the image is captured to when it is displayed on the receiving device side is determined. The delay time is reduced. In addition, as shown in FIG.
Varies depending on the size of the audio code 35 and also depends on the presence or absence of extension of the header 34. Regardless of the size, the size of one transmission frame 37
A transmission unit 36 of an image code is stored for each of them, and the header 34 is counted by counting the number of final bytes of the decoding unit included therein.
May be stored.

The multiplex unit 5 assembles one transmission frame in this way, and determines the number of transmission frames 37 generated in one process according to the transmission speed specified by the transmission speed setting unit 6. . That is, the transmission speed setting unit 6
If 14.4 kbps is specified, one process, that is, 6
One transmission frame 37 is assembled every 0 milliseconds, which is shown in the upper part of FIG. Transmission speed setting part 6 is 28.8 k
If bps is specified, two transmission frames 37 are assembled for each processing, and the state of the transmission frames at this time is as shown in the middle part of FIG. Transmission speed setting part 6 is 5
When 7.6 kbps is specified, four transmission frames 37 are assembled for each process, and the state of the transmission frames at this time is as shown in the lower part of FIG.

When assembling a plurality of transmission frames 37, the audio code 35 is stored in the first transmission frame 37, but it is not necessary to store the audio code in the second and subsequent transmission frames. However, if transmission is delayed due to a transmission error or the like, untransmitted audio codes are accumulated in the second storage unit 4 and can be stored. In the header 34, the number of coding units of the audio code stored in the transmission frame 37 is stored and notified to the receiving device side. Usually two units of audio code 35
Can be stored in the first transmission frame 37 in the first transmission frame 37 in a case where untransmitted audio codes are accumulated in the second storage unit 4. 3 units of audio codes are stored. If there are still untransmitted audio codes, 2
The third and subsequent transmission frames 37 can store up to three units of audio codes. In this way, even if transmission is delayed, interruption of sound reproduced on the receiving side can be prevented.

The multiplex unit 5 passes the transmission frame 37 assembled in this way to the data transmission unit 7 and requests transmission. The data transmission unit 7 is a TA (terminal adapter) that can communicate with the data reception unit 11 at a maximum of 64 kbps. Alternatively, the data transmitting unit 7 may be a data communication card for packet communication of a cdnaOne mobile phone capable of communicating at a maximum of 64 kbps, and in that case, the data receiving unit 11 is a dedicated modem or a TA connected to the Internet. I just need.

As described above, even if the data communication path is 64 kbps at maximum, the communication speed setting unit 6 sets the data communication path to 28.8 kbps or 14.4 kbps.
If a low communication speed of kbps is selected, the image transmission device of the present invention transmits data at the set speed, and since the transmission speed of the audio code is secured, the audio code overflows on the transmission device side, The problem that the reproduction sound is interrupted on the receiving device side is less likely to occur. If a low communication speed is set, the transmission speed of the image code decreases,
The interval at which images are displayed on the receiving device side is increased.

Next, the receiving apparatus will be described with reference to FIG. This is a block diagram of an image receiving device suitable for use with the embodiment of FIG. The data receiving unit 11 acquires data coming from the network 8. The data receiving unit 11 is 1
When the transmission frame 37 of 08 bytes is received, it is passed to the demultiplex unit 12. The demultiplex unit 12 refers to the header 34 in the transmission frame 37 to know whether or not the audio code 35 is present, and if so, how many units are present. Remember. The fourth storage unit 16 stores the difference between the arrival timing of the received audio code and the decoding timing of the audio code in a FIFO (First In Fir).
st Out) Memory. The audio decoding unit 17 is periodically activated by a 30-millisecond timer, decodes and refers to the audio code stored in the fourth storage unit, and outputs sound data to the audio reproduction unit 18.

The demultiplex unit 12 transmits the transmission frame 3
Since the size of the audio code 35 and the presence / absence of the header extension are known with reference to the header 34 in 7, the size of the transmission unit 36 of the image code is calculated. Image code transmission unit 36
Is stored in the third storage unit 13. Further, the number of the last byte of the decoding unit stored in the header 34 is referred to, and the image decoding unit 14 is instructed to execute the decoding process by the number. The third storage unit 13 stores and accumulates image codes that are received in a fragmented manner to form a decoding unit. The image decoding unit 14 is a demultiplexing unit 12
When the image data for one screen is decoded by receiving the instruction from, the image display device 15 is instructed to display the image data.

The image receiving apparatus shown in FIG. 2 does not know the data communication speed of the network 8 or the transmitting apparatus of the present invention. Regardless of the data communication speed, it operates according to the same processing and procedure. As described above, by using the image transmitting apparatus of the present invention, it is not necessary to negotiate with the image receiving apparatus to form an agreement to change the size and structure of the transmission frame 37, and the transmission speed can be changed without permission. .

FIG. 6 is a block diagram showing a second preferred embodiment according to the second aspect of the present invention. This embodiment is characterized by a transmission speed setting unit 6 and a data transmission unit 7, and the other parts are the same as those of the first embodiment, so that the description of the same parts will be omitted.

In FIG. 2, the transmission speed setting unit 6 is 14.4 kbp.
Select and specify one of three types: s, 28.8 kbps, and 57.6 kbps. The transmission speed is set by the user on a user interface (not shown), and the transmission speed setting unit 6
Is used as a setting at the time of negotiation with the data receiving unit 9 when the data transmitting unit 7 starts communication.

The data transmission unit 7 is a best effort type data transmission unit that communicates with the data reception unit 9 of FIG. 2 via the network 8 and uses a PHS that changes the data communication speed during communication. For example, if there are no other users and radio waves are low, two lines are bundled to provide a data communication speed of 64 kbps, but depending on circumstances such as congestion, the number of used lines may be reduced to one. Provides a data transmission rate of 32 kbps.

If the user sets 64 kbps and connects at 64 kbps at the start of communication, if the data transmission unit 7 drops the data communication speed to 32 kbps due to deterioration of conditions during communication, data transmission The unit 7 notifies the transmission speed setting unit 6. The transmission speed setting unit 6 changes the set transmission speed according to the notification. Then, the multiplex unit 5 refers to the transmission speed set by the transmission speed setting unit 6 when assembling the transmission frame 37, so that the transmission frame 37 is assembled within the amount of data that can be transmitted at that speed.

For example, when the data transmission unit 7 uses two lines, the data communication speed is 64 kbps, and the user can use 58400 bps. Therefore, the transmission speed setting unit 6 sets the transmission speed to 57600 bps. At this time, since four transmission frames are assembled every 60 milliseconds, the data generation rate is 57600 bps.
When the data transmission unit 7 uses only one line, the data communication speed is 32 kbps, and the user can use 29200 bps. Therefore, the transmission speed setting section 6 sets the transmission speed to 28800 bps. At this time, since two transmission frames are assembled every 60 milliseconds, the data generation rate is 28800 bps.

As described above, in the second embodiment, the transmission speed can be changed depending on the convenience of the image transmitting device without negotiating with the receiving device. Even in such a case, the transmission speed of the audio code is ensured, and problems such as overflow of the audio code on the transmission device side and interruption of the reproduced sound on the reception device side hardly occur. When operating at a low communication speed, the transmission speed of the image code is reduced, so that the interval at which images are displayed on the receiving device side is increased.

[0036]

According to the image transmitting apparatus of the present invention, the following effects can be obtained. It is not necessary to redesign the size of the transmission frame and its structure even on communication paths with significantly different data communication speeds. These are common knowledge between the transmitting side and the receiving side, but since no change is required, negotiation before starting communication is not required.

Further, even in a communication channel in which the data communication speed changes greatly during communication, the transmitting device can respond flexibly by continuously changing the transmitting speed without negotiating agreement with the receiving device. On the side, reception and decoding can be performed by the same processing and procedure without knowing how the communication path and the transmission device set the data communication speed.

[Brief description of the drawings]

FIG. 1 is a block diagram of a first preferred embodiment of an image transmission device of the present invention.

FIG. 2 is a block diagram of an image receiving device suitable for use with the image transmitting device of FIG. 1;

FIG. 3 is a diagram illustrating an image to be encoded and block lines that are decoding units included in the image to be encoded.

FIG. 4 is a diagram illustrating a relationship between a code output from the image encoding apparatus 1, a code 31 of a block line included in the code, and a decoding unit 32 corresponding to the code.

FIG. 5 is a diagram showing a transmission frame 37 assembled and output by the multiplex unit 5 and a transmission unit 36 of a header 34, an audio code 35, and an image code included therein.

FIG. 6 is a block diagram of a second preferred embodiment of the image transmission device of the present invention.

FIG. 7 is a block diagram of a conventional image transmission device.

[Explanation of symbols]

 Reference Signs List 1 image encoding unit 2 first storage unit 3 audio encoding unit 4 second storage unit 5 multiplex unit 6 communication speed setting unit 7 data transmission unit 8 network 9 camera and AD conversion unit 10 microphone and AD conversion unit 11 Data receiving unit 12 Demultiplex unit 13 Third storage unit 14 Image decoding unit 15 Image display device 16 Fourth storage unit 17 Audio decoding unit 18 Audio reproduction unit 21 First block line of image 22 Seventh of image Block line 31 Code of first block line 32 First decoding unit 33 EOI marker 34 Header 35 Audio code 36 Transmission unit of image code 37 Transmission frame 91 Image encoding unit 92 First storage unit 93 Transmission unit 94 Network 95 Receiving unit 96 Second storage unit 97 Image decoding unit

Claims (2)

[Claims] 1. An image encoding unit that encodes image data.
(1), a first storage unit (2) for storing a code output from the image encoding unit (1), an audio encoding unit (3) for encoding sound data, and the audio encoding unit (3) a second storage unit (4) for storing a code output, and a code stored in the first storage unit (2) and the code stored in the second storage unit (4). In an image transmission apparatus including a multiplex unit (5) for assembling the transmission frame (37) and a data transmission unit (7) for transmitting the transmission frame (37), a transmission speed setting for instructing a transmission speed to the multiplex unit (5). Unit, the audio encoding unit (3) notifies the multiplex unit (5) of completion of audio encoding when the encoding of sound data for a predetermined time unit is completed, and the multiplex unit (5) counts the number of notifications of the audio encoding completion, and when the predetermined number of notifications are counted, the transmission is completed. An image characterized by assembling one or more transmission frames (37) within the amount of data that can be transmitted at the transmission speed indicated by the speed setting unit (6), and instructing the data transmission unit (7) to transmit. Transmission device. 2. The image according to claim 1, wherein the transmission speed setting unit changes the set transmission speed during communication according to a transmission speed at which the data transmission unit can transmit. Transmission device.