JP3827133B2 - Image transmission apparatus and image transmission method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image transmission apparatus and an image transmission method for performing real-time transmission of continuous stream data such as moving images via a network such as Ethernet or Fast Ethernet.
[0002]
[Prior art]
In recent years, video (continuous stream data) such as moving images taken by a camera on the transmitting device (server) side is transmitted in real time via a LAN such as Ethernet or Fast Ethernet, and monitored by a transmission receiving device (client). A monitoring system using a so-called network is considered, and there is a demand mainly for premises monitoring.
[0003]
At this time, when transmitting the video shot by the camera on the server side, the replayed image on the client side is disturbed due to packet collision or loss on the network, so there are several methods to avoid this It is considered.
[0004]
In such a network image transmission system, as shown in FIG. 1, a transmission device (server) 1 and a reception device (client) 3 are connected via a LAN 2 such as Ethernet or Fast Ethernet. Then, the server 1 compresses the video signal captured by the camera by the MPEG compression circuit 5 in accordance with, for example, the MPEG1 standard, packetizes the compressed data by the Ethernet circuit 7, and transmits the packetized data to the Ethernet 2. The client 3 receives this data from the Ethernet 2, takes out the video data with the Ethernet circuit 8, decompresses it with the MPEG decompression circuit 9, outputs it to the display monitor 4, and displays it to display the video captured by the server 1. This can be confirmed on the client 3 side.
[0005]
The image transmission apparatus 1 will be described in more detail. For example, as shown in FIG. 2, an MPEG compression circuit 5 that compresses an input video signal according to the MPEG1 standard, and compressed data (elementary stream) from the MPEG compression circuit 5 is a data bus. 6 and an Ethernet circuit 7 for sending to the Ethernet (network) 2. The MPEG compression circuit 5 mainly includes an A / D converter 11 that converts an analog input video signal into a digital signal, and an MPEG1 encoder LSI 12 that compresses the digital video signal output from the A / D converter 11 according to the MPEG1 standard. The FIFO memory 13 temporarily stores moving image data compressed by the MPEG1 encoder LSI 12, the CPU 14 that controls the entire MPEG compression circuit 5, and the DRAM 15 that is a memory used to operate the CPU 14. .
[0006]
In the image transmission apparatus 1 having such a configuration, the input video signal is A / D converted by the A / D converter 11, compressed by the MPEG1 encoder LSI 12, and output in the form of an elementary stream. This is temporarily held in the FIFO memory 13 and sequentially read out in a first-in first-out manner in accordance with a request from the CPU 14. The data read from the FIFO memory 13 is sent to the Ethernet circuit 7 via the data bus 6, where header information such as Ethernet, UDP, IP, and TCP is added and then sent to the Ethernet 2. The
[0007]
Here, the MPEG1 encoder LSI 12 writes the compressed data (elementary stream) to the FIFO memory 13 as needed, but all the write operations to the FIFO memory 13 are performed by the MPEG1 encoder LSI 12. On the other hand, a read operation from the FIFO memory 13 is performed by software operating on the CPU 14. Hereinafter, a conventional operation from reading an elementary stream held in the FIFO memory 13 to generating an Ethernet packet will be described with reference to a timing chart of FIG.
[0008]
The frame pulse shown in FIG. 13A is a signal synchronized with the frame period of the input video signal, and requests the hardware interrupt once per frame period from the CPU 14. When a hardware interrupt is generated from the CPU 14 ((B) in the figure), an interrupt processing routine (Frame ISR) is executed ((C) in the same figure), where an elementary stream read operation from the FIFO memory 13 is performed. In this case, when one hardware interrupt occurs, all data constituting one picture (I picture, P picture, or B picture) is read from the FIFO memory 13. Since the amount of data constituting one picture varies from frame to frame, the MPEG1 encoder LSI 12 writes the number of bytes of data for each frame to be written into the FIFO memory 13 into the internal register (FIFO read Register) 12a. .
[0009]
Here, the program operation of the interrupt processing routine (Frame ISR) will be further described with reference to the flowchart shown in FIG. As described above, the frame ISR phase A1 (see FIG. 13C) is started by a hardware interrupt that occurs in the frame period of the input video signal. As shown in FIG. 14A, the operation of the Frame ISR starts with reading the contents of the FIFO read Register 12a, which is an internal register of the MPEG1 encoder LSI 12, and data to be read within the current frame period (data for one frame). Is obtained (step 101). Next, data is read from the FIFO memory 13 by this number of bytes and written to an area (area name: DRAM MPEG area) secured on the DRAM 15 (step 102). Then, 0 is written in the FIFO read register 12a (step 103). In the FIFO read register 12a, the amount of data to be read in the next frame by the MPEG1 encoder LSI 12 is written before the next interrupt occurs. Finally, the network process is called to end the Frame ISR (step 104).
[0010]
By calling the Frame ISR, the phase B1 of the network process starts (see FIG. 13D). The operation of this network process is a process of generating a packet from an elementary stream written in the DRAM MPEG area on the DRAM 15 as shown in FIG. 14B. Refer to FIG. 3 for this procedure. While explaining.
[0011]
First, the elementary stream written in the DRAM MPEG area on the DRAM 15 is divided into 1460 bytes (the last read data may be 1460 bytes or less) and read (FIG. 3 (A)), 8 bytes UDP header and 20-byte IP header are added (FIGS. 3B and 3C: step 105). Then, this data is written into a data area of a controller LSI (not shown) of the Ethernet circuit 7 and a transmission procedure is performed (step 106). Thereafter, hardware processing by the Ethernet circuit 7 is performed. Further, a 14-byte Ethernet header is added to this data to form a 1502-byte packet (FIG. 3D), and this packet is transmitted to Ethernet 2 (see FIG. 13E). Then, it is checked whether there is still data written in the DRAM MPEG area (step 107). If data remains (step 107 → Y), the processing of steps 105 to 107 is repeated. In this way, a continuous elementary stream is divided into packets (= 1502 bytes) and sequentially transmitted to the Ethernet 2. If there is no more data to be read from the DRAM MPEG area (step 107 → N), this network process is terminated.
[0012]
The processing described with reference to FIGS. 14A and 14B is processing performed within one frame period. When the hardware interrupt of the next frame occurs, the Frame ASR phase A2 starts, and the processing from step 101 is similarly performed. By such software processing, the elementary stream output from the MPEG1 encoder LSI 12 is packetized and transmitted to the Ethernet 2.
[0013]
[Problems to be solved by the invention]
  When the moving image data output from the image transmission device 1 is transmitted to the image reception device 3 via the network 2 to which a plurality of PCs such as a LAN are connected, if the network 2 is congested and traffic increases, The transmission rate of the network 2 is the image data to be transmitted.Encoding rateIt may become lower than. For this reason, packet collision may occur in the network 2 and data may be lost. Since retransmission is repeated at the Ethernet level, data transmission takes time. As a result, the data reading from the FIFO memory 13 of the MPEG compression circuit 5 and the memory (not shown) of the Ethernet circuit 7 and the operation of adding a TCP (UDP) / IP / Ethernet header are delayed, and normal operation is performed. You will not be able to send Ethernet packets. For this reason, since there is a missing portion in the data received by the image receiving device 3, there is a problem that even if this is decoded, the decoding process cannot be performed normally, and the image displayed on the display monitor 4 is disturbed.
[0014]
This problem will be described in more detail. As described above with reference to FIGS. 3, 13, and 14, the outline of the program execution timing in the conventional method is that when a hardware interrupt occurs, Frame ISR is executed. In this Frame ISR, all data (data constituting one picture) to be read in one frame period is read from the FIFO memory 13 and the contents are copied to a memory area on the DRAM 15. Thereafter, a network process is called by the CPU 14, 1460 bytes of data is read from the memory area of the DRAM 15, and a UDP, IP, or Ethernet header is added thereto to form a 1052 byte packet. This is held in the memory of the Ethernet circuit 7 and is sequentially transmitted to the Ethernet 2. This series of processes is repeated until there is no data copied to the DRAM 15, and a plurality of packets are transmitted within one frame period.
[0015]
  In such an image transmission device (server) 1, an input imageEncoding rateIs high (= the amount of data is large), the read operation of the FIFO memory 13 of the MPEG compression circuit 5 is not completed within one frame period, and the subsequent network process cannot be executed. Alternatively, since the process is executed over the next frame period, the delay is gradually accumulated.
[0016]
Further, when the network 2 is congested, packet retransmission due to packet collisions frequently occurs, and transmission processing takes time. As a result, it becomes impossible to transmit all data to be processed within one frame period.
[0017]
As described above, when a normal packet cannot be transmitted, the image receiving apparatus (client) 3 cannot perform normal decoding processing, and thus a distorted image is displayed.
[0018]
In the conventional method described above, each of the three memories ((1) the FIFO memory 13 of the MPEG compression circuit 5, (2) the memory area secured on the DRAM 15 and (3) the memory of the Ethernet circuit 7) each stores data. Since it is held, it is very difficult to detect a delay in processing, and it is difficult to cope with a delay in processing.
[0019]
Therefore, in the present invention, among these three memories, (2) the memory area secured on the DRAM 15 and (3) the memory usage of the Ethernet circuit 7 are reduced, and the total memory usage is reduced as much as possible (1) MPEG compression. By concentrating on the FIFO memory 13 of the circuit 5, if the usage amount of the FIFO memory 13 is examined, it can be determined whether the read operation from the FIFO memory 13 or the data transmission to the network 2 is completed within one frame period. Like that. The object of the present invention is to prevent loss of data and to perform good image reproduction by the image receiving device 3 by adjusting the amount of data sent to the network according to the use state of the FIFO memory 13. .
[0020]
[Means for Solving the Problems]
As means for achieving the above object, according to the present invention, (1) the memory amount used in the entire image transmission apparatus (server) is concentrated in the FIFO memory of the MPEG compression circuit as much as possible. (2) The FIFO memory of the MPEG compression circuit Three methods are used: checking the usage status; and (3) adjusting the amount of data sent to the network according to the usage status. The present invention intends to provide an image transmission apparatus and an image transmission method described below as an invention showing a specific configuration and method using these three means.
[0021]
1. An image transmission device that transmits information data including image data via a network in order to reproduce an image by the image reception device,
  An A / D converter that A / D converts the incoming video signal into digital image data;
  Encoding means for compressing and encoding the digital image data in MPEG format;
  First storage means for sequentially and temporarily storing the compressed image data compressed by the encoding means;
  A capacity of one packet for temporarily storing the compressed image data of one packet read from the first storage means;A FIFO memory control circuit;
  Second storage means for sequentially and temporarily storing the compressed image data for one packet output from the FIFO memory control circuit;
  Said second storage meansSending means for packetizing the compressed image data read out for each packet from the packet and sending it to the network;
  Control means for controlling the first storage means;
Have
  The control means determines the usage status of the first storage means based on a plurality of patterns corresponding to the write address and the read address position in the first storage means, and the pattern and a continuation frame of the pattern Depending on the number, compressed picture data compressed by deleting the B picture of the temporarily stored compressed picture data, deleting the B picture and the P picture, or reducing the coding rate of the coding means Control whether to temporarily store by reducing the amountAn image transmission apparatus.
[0022]
2.The control means further controls to increase the compression rate of the image data compressed by the encoding means before the accumulated data amount of the first accumulation means is rewritten by sequentially incoming data. The image transmission apparatus according to claim 1, further comprising:
[0023]
3.An image transmission method for transmitting information data including image data via a network in order to reproduce an image in an image receiving device,
  An incoming video signal is A / D converted into digital image data, the digital image data is compression-encoded and sequentially stored in the first storage means, and is read out from the first storage means for each packet data. Temporarily accumulates in the second accumulator, one packet of compressed image data output from the second accumulator is temporarily accumulated in the third accumulator sequentially, and one packet from the third accumulator. When the read compressed image data is packetized for transmission and sent to the network,
  Control for thinning out and outputting B picture or B picture and P picture from the first storage means and compression of the image data before the stored data amount of the first storage means is rewritten by incoming data An image transmission method comprising controlling at least one of control for increasing the rate.
[0025]
DETAILED DESCRIPTION OF THE INVENTION
An image transmission apparatus and an image transmission method according to an embodiment of the present invention will be described with reference to the drawings. The image transmission apparatus of the present invention is used in a network image transmission system as shown in FIG. 1, and has a configuration as shown in FIG. 2, for example. Here, since the schematic configuration in FIGS. 1 and 2 has already been described in the prior art, the description of the same part is omitted, and the program configuration and circuit configuration unique to the present application are as follows. (1) Image transmission apparatus (server) Aggregate the total amount of memory used in the FIFO memory of the MPEG compression circuit as much as possible. (2) Check the usage status of the FIFO memory of the MPEG compression circuit. (3) Adjust the amount of data sent to the network according to the usage status. This will be described below with reference to other drawings, focusing on the three means.
[0026]
First, (1) this program operation is shown in FIG. 4 in that the memory amount used in the entire image transmission apparatus (server) 1 is concentrated in the FIFO memory (storage means, first storage means) 13 of the MPEG compression circuit 5 as much as possible. This will be described with reference to the timing chart shown in FIG. 5 and the flowchart shown in FIG.
[0027]
First, before a hardware interrupt occurs, steps 116 to 119 of the MPEG process shown in FIG. 5B are executed. Here, initial setting of the MPEG1 encoder LSI (encoding means) 12 is first performed (step 116). Next, the MPEG1 encoder LSI 12 is accessed, the compression rate is set to a predetermined value (eg, 1.15 Mbps), and the predetermined value is stored in an area secured on the DRAM 15 (the area name is bitrate) (step 117). ). Then, encoding (compression) by the MPEG1 encoder LSI 12 is started (step 118), and a call from the Frame ISR is waited (step 119).
[0028]
In step 118, when encoding is started, a frame pulse synchronized with the frame period of the input video signal is output from the MPEG1 encoder LSI 12 (see FIG. 4A). Then, a hardware interrupt of the CPU 14 is generated by this signal (see FIG. 4B), and Frame ISR is called to start phase A1 (see FIG. 4C).
[0029]
In this Frame ISR, processing as shown in FIG. 5A is performed. First, the value of the FIFO read register 12a in the MPEG1 encoder LSI 12 is read, and the number of bytes of data to be read at present (the amount of data constituting one picture) is obtained from the FIFO memory 13 (step 111). If the number of bytes is 0 (step 112 → Y), the data is not held in the FIFO memory 13, and the interrupt process is terminated. However, if the number of bytes is not 0 (step 112 → N), the number of bytes is copied to the area (area name is FIFO number) secured on the DRAM (second storage means) 15 (step 113). Then, the FIFO read register 12a is reset (0 is written) (step 114), the MPEG process is called, and the interrupt process is terminated. Thus, the phase A1 shown in FIG.
[0030]
Then, the MPEG process called from the Frame ISR resumes processing, and phase B1 shown in FIG. 4 (D) starts. In the MPEG process, when there is a call from Frame ISR (step 119 → Y), 1460 bytes (data to be transmitted in one packet) are read from the FIFO memory 13 of the MPEG compression circuit 5 and once stored in the FIFO memory control circuit 16. The data is secured and output to the data bus 6 in accordance with the operation timing of the CPU 14, and this data is copied to the area secured on the DRAM 15 (the area name is DRAM MPEG area) (step 120). Then, 1460 bytes are subtracted from the value of the number of bytes recorded in the FIFO number on the DRAM 15, and the value is written again in the FIFO number (step 121). Thereafter, a Data Reduce subroutine is executed to adjust the amount of data sent to the Ethernet (step 122). The data reduce subroutine will be described later in detail with reference to FIG.
[0031]
Thereafter, 1460-byte data stored in the DRAM MPEG area on the DRAM 15 is read and a UDP / IP header is added (step 123). Then, this data is written into a data area (not shown) of the controller LSI of the Ethernet circuit (sending means) 7 and a transmission procedure is performed (step 124). Thereafter, hardware processing by the Ethernet circuit 7 is performed. That is, a 14-byte Ethernet header is added to this data to form a 1502-byte packet (see FIG. 3D), and this packet is transmitted to Ethernet 2 (see FIG. 4E).
[0032]
Then, it is checked whether or not the FIFO number of the DRAM 15 is 0 (step 125). If it is not 0 (step 125 → N), since there is still data to be transmitted, the process returns to step 120 to start phase B2. This phase B2 performs the same processing as the phase B1, and repeats the processing of steps 120 to 125 (see FIG. 4D). In this way, the continuous elementary stream is divided into packets (= 1502 bytes) and sequentially sent out to the Ethernet 2 until the FIFO number becomes 0 in phase B3, phase B4. When the FIFO number of the DRAM 15 becomes 0 (step 125 → Y), the data reading from the FIFO memory 13 is completed, so this MPEG process is terminated. Then, when the next frame period starts (see FIG. 4A), when a hardware interrupt occurs (see FIG. 4B), Frame ISR phase A2 starts (see FIG. 4C). ), The process shown in the flow of FIG. 5 described above is executed.
[0033]
As described above, in this embodiment, only data for one packet is copied to the DRAM 15, the packet is transmitted from the Ethernet circuit 7, and then the data of the next packet is copied to the DRAM 15. Therefore, the amount of data stored in the DRAM 15 and the Ethernet circuit 7 is only one packet, and most of the compressed image data output from the MPEG1 encoder LSI 12 is only accumulated in the FIFO memory 13. Therefore, it is possible to grasp the processing status of packet transmission only by monitoring the FIFO memory 13.
[0034]
In other words, in the conventional method, all data for one frame is read from the FIFO memory 13 and stored in the DRAM 15, and then packetized every 1460 bytes (1502 bytes including the header) and transmitted to the Ethernet. For this reason, if the compression rate of the input image is high or the processing within one frame period is not in time due to network congestion, etc., the generation process of the Ethernet packet is wrinkled, and in the worst case, one packet can be generated. Sometimes it disappeared. However, in this embodiment, by reading data from the FIFO memory 13 by 1460 bytes at a time, priority is given to sending 1 packet (1502 bytes including header) in units of Ethernet 2. It is possible to avoid the generation failure.
[0035]
Further, since the amount of data accumulated in the FIFO memory 13 increases accordingly, “use status of the FIFO memory 13 = packet transmission status”, and the FIFO memory 13 is monitored to check whether processing within one frame period is in time. It becomes possible to discriminate.
[0036]
Next, a method for monitoring the use status of the FIFO memory will be described below. First, (2) checking the usage status of the FIFO memory 13 of the MPEG compression circuit 5 (checking the remaining amount of data in the FIFO memory 13) will be described.
[0037]
Only the peripheral main circuit configuration of the FIFO memory 13 in the MPEG compression circuit 5 is shown in FIG. In this embodiment, the FIFO memory 13 that can perform the write operation and the read operation asynchronously is used. In the figure, the flow of image data is that compressed image data (elementary stream) output from the MPEG1 encoder LSI 12 is input to the write side of the FIFO memory 13. The data held in the FIFO memory 13 is read according to a request from the CPU 14. The write side control is performed by the write clock (WCLK) and write enable (WE) output from the MPEG1 encoder LSI 12 and the write reset (WRST) output from the FIFO memory control circuit 16, and the read side control is performed from the CPU 14. An output read clock (RCLK), a read enable (RE) output from the FIFO memory control circuit 16, and a read reset (RRST) are used. The write operation is performed by the WCLK and WE signals output from the MPEG1 encoder LSI 12. For example, as shown in FIG. 7, when WE is at "L" level, 8 cycles per cycle are synchronized with the rise of the WCLK input. It is written in bits (1 byte).
[0038]
In this embodiment, in order to detect the remaining amount of data held in the FIFO memory 13, the current write address and read address of the FIFO memory 13 are checked, and these values are compared, thereby comparing the FIFO memory 13. The remaining amount of data held in
[0039]
First, the write address is obtained by counting the number of WCLKs when WE is at "L" level in the FIFO memory control circuit 16 shown in FIG.
[0040]
For example, when using a FIFO memory 13 with a capacity of 2M bits (256K words x 8 bits = 262144 bytes), the data width (1 byte = 8 bits) on the vertical axis and the number of words on the horizontal axis indicate the entire memory. The amount of data is as shown in FIG. Since the write operation is performed by writing 1 byte (= 8 bits) at the rising edge of WCLK, as shown in FIG. 8B, using a 19-bit counter (write address counter), 1 to 40000 By counting up to HEX (= 262144), the number of bytes (write address) of data written to the FIFO memory 13 can be counted. Similarly, the read address can be counted by the FIFO memory control circuit 16 by counting the number of RCLKs when the RE is at the “L” level, and counting the number of bytes of data read from the FIFO memory 13.
[0041]
If the CPU 14 periodically reads out and compares the value of the write address thus obtained and the value of the read address, the remaining amount of data held in the FIFO memory 13 can be obtained. For example, when the write address is 300 HEX (= 768) and the read address is 20 HEX (= 32), the amount of data remaining in the FIFO memory 13 can be known by subtracting the read address from the write address. In this case, 768−32 = 736 bytes of data remain in the FIFO memory 13 without being read out.
[0042]
As described above, in the present embodiment, not all the data within one frame period (data constituting one picture) is read from the FIFO memory 13, but only the number of bytes necessary for packetization is read. For this reason, the amount of data held in the FIFO memory 13 is larger than in the past. In other words, processing delay of the entire server 1 is collected in the FIFO memory 13. For example, when packets cannot be sent out smoothly due to the congestion of the network 2, the packet transmission is given the highest priority in this embodiment, so that the read operation from the FIFO memory 13 is also delayed. However, since the write operation is performed independently (asynchronously), the write address reaches 40000HEX, data is written again from the 0 address, and eventually the read address may be overtaken. This is because new data is overwritten on data that has not yet been read from the FIFO memory 13, so that the continuous data disappears in the middle, resulting in discontinuous data that is interrupted. A packet is generated and transmitted, and a distorted image is displayed because the client 3 cannot normally perform decoding processing.
[0043]
In the present embodiment, in order to cope with such a situation, the FIFO memory 13 is monitored, and the amount of data transmitted to the Ethernet 2 is adjusted so that the write address does not overtake the read address, thereby minimizing image disturbance. To the limit. A method for adjusting the amount of data sent to the Ethernet 2 (3) adjusts the amount of data sent to the network according to the usage status will be described below.
[0044]
When comparing the write address and the read address obtained above, the amount of data read from the FIFO memory 13 in one frame period (the amount of data that constitutes one picture) differs for each picture, compared to B and P pictures. It is necessary to consider that the amount of data is the largest in the case of an I picture. As a result of measurement in the present embodiment, the amount of data constituting an I picture in one frame period is approximately 10,000 bytes or less at a bit rate of 1.15 Mbps. This means that a maximum of 10000 bytes of the read address is updated for each hardware interrupt (Frame ISR). Therefore, in this embodiment, instead of comparing only the current values of the write address and the read address, a window having a width of 5000 bytes before and after the write address and the read address is provided as shown in FIG. Use to make comparisons.
[0045]
Here, the relationship between the state of the FIFO memory 13 and the write / read address when performing the comparison will be described with reference to FIG. When a hardware interrupt occurs (when Frame ISR is executed), if the FIFO read register is 0, the read operation from the FIFO memory 13 is not performed, so the read address cannot overtake the write address. Absent. Therefore, it is only necessary to determine the use status of the following five patterns excluding this case.
(1) Pattern 1
As shown in (Pattern 1) in FIG. 10, the write address is always preceded and the address position in a normal state where the processing is completed within one frame period is shown. In this case, the relationship of the addresses of the FIFO memory 13 is expressed by Equation 1. R represents a read address, and W represents a write address.
R−5000 <R + 5000 <W−5000 <W + 5000 (Formula 1)
[0046]
(2) Pattern 2
Pattern 2 shows a state in which the read address approaches the write address and a part of the comparison range overlaps as shown in (Pattern 2) in FIG. It means that it is done without any problem, and this is also normal. In this case, the relationship of the addresses of the FIFO memory 13 is expressed by Equation 2.
R−5000 <W−5000 <R + 5000 <W + 5000 (Formula 2)
[0047]
(3) Pattern 3
In pattern 3, as shown in (pattern 3) of FIG. 10, the write address reaches the final address (262143), and once returns to 0, the write operation continues, which is also a normal state. In this case, the relationship of the addresses of the FIFO memory 13 is expressed by Equation 3.
W−5000 <W + 5000 <R−5000 <R + 5000 (Formula 3)
[0048]
(4) Pattern 4
Pattern 4 shows a state in which the write address approaches the read address from the state of pattern 3 and a part of the comparison range overlaps as shown in (pattern 4) of FIG. At this time, since the new data is not written on the data that has not yet been read from the FIFO memory 13, the continuity of the data is not disturbed (the received image at the client 3 is not disturbed). However, it can be seen that the reading operation of the FIFO memory 13 is delayed, and in this state, the data is overwritten. Therefore, in order to recover the delay of the read operation, it is necessary to prioritize the read operation of the FIFO memory 13 (for example, to reduce the amount of packet transmission by deleting P and B pictures). Will be described later). In this case, the relationship of the addresses of the FIFO memory 13 is expressed by Equation 4.
W−5000 <R−5000 ≦ W + 5000 <R + 5000 (Formula 4)
[0049]
(5) Pattern 5
Pattern 5 shows a state in which the read operation of the FIFO memory 13 is further delayed from the state of pattern 4 and the write address has overtaken the read address, as shown in (pattern 5) of FIG. In this state, since new data is written on data that has not been read, the continuity of data has already been lost. However, in this case as well, it is necessary to reduce the packet transmission amount and quickly return to the normal state. In this case, the relationship of the addresses of the FIFO memory 13 is as shown in Equation 5.
R−5000 ≦ W−5000 <R + 5000 ≦ W + 5000 (Formula 5)
[0050]
Next, a procedure for adjusting the amount of data transmitted to the Ethernet 2 based on the use status of the FIFO memory 13 derived as described above will be described. The data amount is adjusted by two processes of FIFO address ISR and Data Reduce subroutine, and respective operation flowcharts are shown in FIG. 11 and FIG. The Data Reduce subroutine is a subroutine executed in step 122 of the MPEG process that has been described in detail with reference to FIG.
[0051]
First, the operation of the FIFO address ISR will be described with reference to FIG. The FIFO address ISR is a process executed by a software interrupt generated at regular intervals by the timer function of the CPU 14. In this process, the write address and read address values of the FIFO memory 13 generated by the FIFO memory control circuit 16 are read and stored in an area secured on the DRAM 15.
[0052]
First, the write address and read address generated by the FIFO memory control circuit 16 are read out, and the respective values are copied to areas on the DRAM 15 (area names are referred to as write address and read address) (step 131). Then, the values of (write address ± 5000) and (read address ± 5000) are compared to determine each of the above patterns 1 to 5 (step 132).
[0053]
Here, a flag indicating the delay of the read operation of the FIFO memory 13 (area name is read late) and an area for storing pattern information (area name is state) are secured on the DRAM 15 in advance. In the case of pattern 1, 0 is set for read late and 1 is set for state (step 133). Similarly, in the case of pattern 2, 0 is set to read late and 2 is set to state (step 134). In the case of pattern 3, 0 is set to read late and 3 is set to state (step 135). In the case of pattern 4 (when the read operation is delayed), 1 is set to read late and 4 is set to state (step 136). In the case of pattern 5 (when the read operation is further delayed), 1 is set to read late and 5 is set to state (step 137). Thereafter, the interrupt process is terminated.
[0054]
Next, the operation of the Data Reduce subroutine will be described with reference to the flowchart shown in FIG. In this Data Reduce subroutine, the pattern information determined and stored by the FIFO address ISR is read to reduce the amount of packets sent in the case of patterns 4 and 5, and the processing time free here is used as the read operation of the FIFO memory 13. In other words, an operation of recovering the processing delay within one frame period is performed.
[0055]
First, the header of the elementary stream stored in the DRAM MPEG area by the MPEG process is searched to find the picture coding type, the picture type (I, P, B) is determined (step 141), and secured on the DRAM 15 1 is written to I, 2 is written to P, and 3 is written to B (step 142). Then, the state indicating the pattern type is read (step 143), and when the states are 1, 2 and 3, the packet is transmitted as usual because the processing of one frame period is in time. When pattern 5 is entered by the previous frame interrupt, an area is reserved on the DRAM 15 with the name of state 5 count in order to store that, and when states are 1, 2 and 3 In this case, 0 is substituted for this (step 144). Thereafter, the subroutine processing is terminated, and the continuation of the MPEG process in FIG. 5B (from step 123 in FIG. 5B) is performed.
[0056]
If the state is 4 in step 143, there is a delay in the read operation of the FIFO memory 13, so the amount of data sent to the Ethernet 2 needs to be reduced. Therefore, first, the value of state5 count is read and checked (step 145).
[0057]
If the value of this state5 count is not 0 (step 145 → N), it means that the state of the FIFO memory 13 was pattern 5 in the previous frame interrupt, and it is pattern 4 in the current frame. Therefore, it can be seen that the state has improved from pattern 5 to pattern 4. However, since it is necessary to continue the process of pattern 5 to further improve the state, the process moves to the process of step 149 (the process contents thereafter will be described in the process of state 5).
[0058]
If the value of state5 count is 0 (step 145 → Y), this indicates that the previous frame was in a state other than pattern 5. In the process of state 4 in this case, the data amount is reduced by deleting the B picture data from the I, B, and P pictures. First, the value of the picture area on the DRAM 15 is read to determine the type of picture (step 146). If the picture value is 3 and a B picture is indicated, packet generation and transmission are not performed. Jump to step 125 of the MPEG process (processing after packet transmission). When the picture value is 1 or 2 and indicates an I or P picture, the subroutine is finished and the continuation of the MPEG process (step 123 in FIG. 5B) is executed in order to perform packet generation / transmission as usual.
[0059]
When the state is 5 in step 143, the value of state5 count is updated by 1 (step 147). By doing this, it is possible to know how many frames of pattern 5 have continued by reading the value of state5 count thereafter. Next, it is checked whether or not the state5 count is 10 (step 148). This is performed to change the subsequent processing depending on whether the pattern 5 has continued for 10 frames.
[0060]
If the value of state5 count is not 10 (step 148 → N), normal processing in state5 is performed. Here, the data amount is reduced by deleting the data of the B and P pictures out of the I, B, and P pictures. First, the value of the picture area on the DRAM 15 is read, and when the picture value is 2 or 3 and a B or P picture is indicated, packet generation and transmission are not performed. Therefore, step 125 (FIG. 5B) of the MPEG process shown in FIG. Jump to (Process after packet transmission). When the picture value is 1 and an I picture is indicated, packet generation / transmission is performed as usual, so the subroutine processing is terminated and the continuation of the MPEG process (step 123 in FIG. 5B) is executed.
[0061]
  If the value of state5 count is 10 (step 148 → Y), it means that pattern 5 has continued for 10 frames. In this case, it is considered that the data in the FIFO memory 13 has already been overwritten and the continuity of the data has been lost. It also means that the state has not improved even if the pattern 5 processing (deleting P and B pictures) is performed for 10 frames, so it is considered that no improvement can be expected even if the same operation is repeated as it is. . Therefore, if the value of state5 count is 10, the input signalEncoding rateTo lower the value.
[0062]
  First, in order to set the bit rate, the encoding process in the MPEG1 encoder LSI 12 is temporarily stopped (step 150). And currentEncoding rateIn order to know the bitrate, the bitrate value on the DRAM 15 is read (step 151), 800kbps for 1.15Mbps, 600kbps for 800kbps, 400kbps for 600kbps, and 200kbps for 400kbps (step 152). ). Thereafter, the MPEG1 encoder LSI 12 is accessed, and the set bitrate value is set to the bit rate of the data to be encoded and output (step 153). Then, after returning the state5 count to 0 (step 154), the process jumps to step 118 of the MPEG process in FIG. 5B in order to restart the encoding process.
[0063]
In this way, by controlling the packet transmission amount according to the remaining amount of data in the FIFO memory 13, the disturbance of the received image in the client 3 can be minimized.
[0064]
In the present embodiment described above, the read operation delay (usage status of the FIFO memory 13) is detected by comparing the write address and read address values of the FIFO memory 13. Then, by adjusting the amount of data sent to the network 2 according to this delay, packet collision that occurs on the network 2 is avoided, and the disturbance of the received image in the client 3 is alleviated.
[0065]
【The invention's effect】
The image transmission apparatus and the image transmission method of the present invention can adjust the data amount of an image transmitted from a server to a transmission rate adapted to a network environment used as a transmission path.
[0066]
In addition, there is an effect that it is possible to avoid packet collision / disappearance that occurs on the network and to reduce image disturbance in the receiving apparatus.
[Brief description of the drawings]
FIG. 1 is a configuration diagram illustrating an example of a network image transmission system in which an image transmission apparatus and an image transmission method of the present invention are used.
FIG. 2 is a configuration diagram showing an embodiment of an image transmission apparatus of the present invention.
FIG. 3 is a diagram for explaining an Ethernet packet generation procedure;
FIG. 4 is a timing chart for explaining the operation of the embodiment of the image transmission apparatus and the image transmission method of the present invention.
FIG. 5 is a flowchart for explaining an embodiment of an image transmission apparatus and an image transmission method of the present invention.
FIG. 6 is a configuration diagram showing an example of a peripheral circuit of a FIFO memory of the image transmission apparatus of the present invention.
FIG. 7 is a timing chart for explaining the write operation of the FIFO memory.
FIG. 8 is a diagram for explaining a count operation of a write address of a FIFO memory.
FIG. 9 is a diagram for explaining a method of comparing addresses in a FIFO memory;
FIG. 10 is a diagram for explaining a state pattern of a FIFO memory;
FIG. 11 is a flowchart for explaining the operation of a FIFO address ISR;
FIG. 12 is a flowchart for explaining the operation of the Data reduce subroutine.
FIG. 13 is a timing chart for explaining the operation of a conventional image transmission apparatus.
FIG. 14 is a flowchart for explaining a conventional image transmission apparatus.
[Explanation of symbols]
  1 Image transmission device
  2 Ethernet (network)
  3 Image receiver
  4 Display monitor
  5 MPEG compression circuit
  6 Data bus
  7 Ethernet circuit (transmission means)
  8 Ethernet circuit
  9 MPEG expansion circuit
  11 A / D converter
  12 MPEG1 encoder LSI (encoding means)
  12a Internal register (FIFOreadregister)
  13 FIFO memory (storage means, first storage means)
  14 CPU
  15 DRAM (second storage means)
  16 FIFO memory control circuit

Claims (1)

An image transmission device that transmits information data including image data via a network in order to reproduce an image by the image reception device,
An A / D converter for A / D converting the incoming video signal into digital image data;
Encoding means for compressing and encoding the digital image data in MPEG format;
First storage means for sequentially and temporarily storing the compressed image data compressed by the encoding means;
A FIFO memory control circuit having a capacity of one packet for temporarily storing the compressed image data of one packet read from the first storage means ;
Second storage means for sequentially and temporarily storing the compressed image data for one packet output from the FIFO memory control circuit;
Sending means for packetizing the compressed image data read from the second storage means for each packet and sending it to the network;
Control means for controlling the first storage means;
Have
The control means discriminates the usage status of the first storage means based on a plurality of patterns corresponding to the write address and the read address position in the first storage means, and the pattern and the continuation frame of the pattern Depending on the number, the compressed image data compressed by deleting the B picture of the temporarily stored compressed image data, deleting the B picture and the P picture, or reducing the encoding rate of the encoding means An image transmission apparatus characterized in that control is made as to whether or not temporary storage is performed by reducing the amount .

Images