KR20090113171A - Information processing device and method - Google Patents

Abstract

It is possible to provide an information processing device and method which can transfer encoded data with a small delay. A data control unit (137) temporarily accumulates in a memory unit (301), encoded data successively supplied in the ascending order of the bandwidth of components and counts the code amount of the accumulated encoded data. When the code amount has reached a predetermined amount, acquisition of encoded data is terminated. A part or whole of the encoded data which has been accumulated in the memory unit (301) is read out and supplied as encoded data for return, together with a packetization unit (302). The present invention can be applied to a digital triax system.

Description

Information processing apparatus and method {INFORMATION PROCESSING DEVICE AND METHOD}

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing apparatus and method, and more particularly, to an information processing apparatus and method capable of transmitting coded data at low latency.

Background Art Conventionally, as a transmission / reception apparatus of a video image, there is a thing called a triax system which is used for sports relaying in broadcasting stations and stadiums. Until now, the triax system has been mainly used for analog video, but with the digitization of image processing in recent years, it is thought that digital triax systems for digital video will become popular in the future.

In a general digital triax system, a video image is captured by a camera head and sent to a transmission path (main video image), the main video image is received by the camera control unit, and the image is output on the screen.

The camera control unit also transmits a return video image to the camera head side separately from the main line video image. The return video image may be obtained by converting the main line video image supplied from the camera head or may be a video image input from the outside from the camera control unit. The camera head outputs this return video image on a screen, for example.

In general, the transmission path between the camera head and the camera control unit has a limited band, and it is necessary to compress the video image in order to transmit the transmission path. For example, if the main video image transmitted from the camera head toward the camera control unit is an HDTV (High Definition Television) signal (current signal is about 1.5 Gbps), it is realistic to compress it to about 150 Mbps, which is about one tenth. .

There are various methods for compressing such video, for example, there are Moving Picture Experts Group (MPEG) and the like (see Patent Document 1, for example). FIG. 1 shows an example of a conventional digital triax system in the case of compressing an image as described above.

The camera head 11 has a camera 21, an encoder 22, and a decoder 23. The video data (video) obtained by the camera 21 is encoded by the encoder 22, and the encoded data. Is supplied to the camera control unit 12 via the main line D10 which is one system of the transmission cable. The camera control unit 12 has a decoder 41 and an encoder 42. When acquiring the encoded data supplied from the camera head 11, the camera control unit 12 decodes it by the decoder 41 and decodes the decoded video data. It is supplied to the main view 51 which is a display for main line video via the cable D11, and an image is displayed.

Also, in order to confirm to the user of the camera head 11 whether or not the camera control unit 12 has received an image transmitted from the camera head 11, the image data is the return video image as the camera control unit 12. Is resent from the camera head 11 to the camera head 11. In general, since the bandwidth of the return line D13 for transmitting this return video image is narrow compared with the main line D10, the camera control unit 12 encodes the image data decoded by the decoder 41 in the encoder 42. ) Is encoded again to generate coded data having a desired bit rate (normally, a lower bit rate than when the main line is transmitted), and the coded data is a return line (D13) that is one system of the transmission cable as a return video image. Through the camera head (11).

When the camera head 11 acquires the encoded data (return video image), the decoder 23 decodes the decoder 23, and the decoded image data is returned through the cable D14, which is a display for the return video image. 31) to display an image.

That is the basic configuration and operation of the digital triax system.

Patent Document 1: Japanese Patent Application Laid-Open No. 9-261633

<Start of invention>

Problems to be Solved by the Invention

However, in such a method, the delay time is long until the encoding is started by the encoder 22 (after the video image signal is obtained by the camera 21) and the output of the decoded image data from the decoder 23 is started. There was a risk of losing. Moreover, the encoder 42 was also needed for the camera control unit 12, and there exists a possibility that a circuit size and a cost may increase.

2 shows a relationship between the timings of the respective processes performed on the video data.

As shown in FIG. 2, in the process start timing of the encoder 22 of the camera head 11 and the output start timing of the decoder 41 of the camera control unit 12, the camera head 11 and Even if the time required for transmission between the camera control units 12 is zero, for example, a delay of P [msec] occurs due to encoding or decoding processing.

And even if the encoder 42 encodes the decoded video data immediately, the encoder 42 further processes P until the decoder 23 of the camera head 11 starts output by processing such as encoding or decoding. There is a delay of [msec].

That is, the delay of (P × 2) [msec], which is twice the delay occurring in the main video video, after the encoding is started at the encoder 22 and the output of the decoded video data is started from the decoder 23 is started. Occurs. In a system requiring low latency, such a method cannot sufficiently shorten the delay time.

The present invention has been proposed in view of such a conventional situation, and allows the encoded data to be transmitted with low latency.

Means for solving the problem

An aspect of the present invention is an information processing apparatus for encoding coded image data to generate coded data, wherein the coefficient data decomposed for each frequency band is required to generate coefficient data for one line of the subband of the lowest band component. Rearrangement means for rearranging in advance in order of performing a synthesizing process for synthesizing coefficient data of a plurality of subbands decomposed into frequency bands and generating image data for each line block including image data for a line number; Encoding means for encoding the coefficient data rearranged by the arrangement means for each line block to generate encoded data, storage means for storing encoded data generated by the encoding means, and the encoded data is stored by the storage means. Each time a plurality of line blocks are stored, the sum of the code amounts of the encoded data is calculated. If the sum total of the code amount calculated by the calculation means, the calculation means reached to the target quantity of codes, an information processing apparatus including output means for outputting the encoded data stored in the storage means.

The output means can change the bit rate of the encoded data.

The rearrangement means can rearrange the coefficient data in the order of the low pass component to the high pass component for each line block.

The rearrangement means and the encoding means may be further provided with control means for controlling the parallel operation for each line block.

The rearrangement means and the encoding means can perform each processing in parallel.

It is possible to further include a filter means for generating the plurality of sub bands of coefficient data decomposed for each frequency band by performing filter processing for each of the line blocks with respect to the image data.

Decoding means for decoding the encoded data may be further provided.

Modulating means for generating a modulated signal by modulating the encoded data in different frequency domains, amplifying means for frequency multiplexing and amplifying the modulated signal generated by the modulating means, and a modulated signal amplified by the modulating means; It may be further provided with a transmission means for transmitting.

On the basis of the attenuation rate in the frequency domain, it may be further provided with a modulation control means for setting the modulation method of the modulation means.

In the case where the attenuation rate in the frequency domain is equal to or greater than the threshold value, it may be further provided with control means for setting a large signal point distance for the high frequency component.

In the case where the attenuation rate in the frequency domain is equal to or greater than the threshold, it may be further provided with control means for setting a large amount of error correction bits for the high frequency component.

In the case where the attenuation ratio in the frequency domain is equal to or greater than the threshold value, it may be further provided with control means for setting a high compression ratio for the high frequency component.

The modulation means can be modulated by the OFDM method.

It is possible to further include a synchronization control unit for controlling synchronization timing between the encoding means and the decoding means for decoding the encoded data by using image data having a smaller data amount than a threshold.

The image data in which the data amount is smaller than the threshold value can cause all pixels to be images for one picture of black color.

One aspect of the present invention is an information processing method of an information processing apparatus that encodes image data to generate encoded data, wherein coefficient data decomposed for each frequency band is converted into coefficient data for one line of the subband of the lowest band component. For each line block containing the image data of the number of lines necessary to generate, rearranged in advance in order of synthesizing the coefficient data of the plurality of subbands decomposed into frequency bands to generate image data, The arranged coefficient data is encoded for each line block to generate encoded data, the generated encoded data is stored, and each time the encoded data is stored for a plurality of the line blocks, the sum of the code amounts of the encoded data is calculated. If the sum of the calculated code amounts reaches the target code amount, An information processing method including a step of outputting data.

In one aspect of the present invention, the coefficient data decomposed for each frequency band is divided into frequency bands for each line block including image data corresponding to the number of lines necessary for generating coefficient data for one line of the subband of the lowest band component. The data can be rearranged in advance in the order of performing a combining process of synthesizing the coefficient data of the decomposed plurality of subbands to generate image data. The rearranged coefficient data is encoded for each line block to generate encoded data. When the encoded data is stored and the encoded data is stored for a plurality of the line blocks, the total of the code amounts of the encoded data is calculated, and when the total of the calculated code amounts reaches the target code amount, the memory is stored. The coded data is output.

Effect of the Invention

According to the present invention, the bit rate of data to be transmitted can be easily controlled. In particular, the bit rate can be easily changed without decoding the encoded data.

1 is a block diagram showing a configuration example of a conventional digital triax system.

FIG. 2 is a diagram showing a relationship between timings of respective processes performed on video data in the digital triax system of FIG. 1; FIG.

3 is a block diagram showing an example of the configuration of a digital triax system to which the present invention is applied;

4 is a block diagram illustrating a detailed configuration example of a video signal encoder of FIG. 3.

5 is a schematic diagram for schematically explaining a wavelet transform;

6 is a schematic diagram for schematically explaining a wavelet transform;

Fig. 7 is a schematic diagram showing an example in which filtering by lifting a 5x3 filter is performed up to a decomposition level = 2.

8 is a schematic diagram schematically illustrating the flow of wavelet transform and wavelet inverse transform according to the present invention;

9 is a schematic diagram for explaining an example of the mode of transfer of encoded data.

10 is a diagram illustrating a configuration example of a packet.

FIG. 11 is a block diagram illustrating a detailed configuration example of the data converter of FIG. 3. FIG.

12 is a block diagram illustrating a configuration example of a video signal decoder of FIG. 3.

13 is a schematic diagram schematically showing an example of parallel operation.

14 is a diagram illustrating an example of the state of bit rate conversion.

FIG. 15 is a diagram showing a relationship between timings of respective processes performed on video data in the digital triax system of FIG. 3; FIG.

16 is a block diagram illustrating a detailed configuration example of a data control unit in FIG. 11.

FIG. 17 is a flowchart for explaining an example of the flow of main processing executed in the entire digital triax system of FIG. 3; FIG.

18 is a flowchart for explaining an example of a detailed flow of an encoding process.

19 is a flowchart for explaining an example of a detailed flow of a decoding process.

20 is a flowchart for explaining an example of a detailed flow of a bit rate conversion process.

21 is a block diagram illustrating another example of the video signal encoder of FIG. 3.

Fig. 22 is a schematic diagram for explaining the flow of processing when a video signal coding unit performs rearrangement of wavelet coefficients.

Fig. 23 is a schematic diagram for explaining the flow of processing when a video signal decoding unit performs rearrangement processing of wavelet coefficients.

24 is a view for explaining an example of the state of counting the amount of data.

25 is a diagram for explaining another example of how the count of the data amount counts.

26 is a block diagram illustrating another configuration example of a data control unit.

27 is a flowchart for explaining another example of bit rate conversion processing.

Fig. 28 is a block diagram showing another configuration example of a digital triax system to which the present invention is applied.

FIG. 29 is a block diagram showing a configuration example of a conventional digital triax system corresponding to the digital triax system of FIG.

30 is a block diagram illustrating another configuration example of a camera control unit.

Fig. 31 is a block diagram showing a configuration example of a communication system to which the present invention is applied.

32 is a schematic diagram illustrating an example of a display screen.

33 is a diagram illustrating an example of a frequency distribution of a modulated signal.

34 shows an example of attenuation characteristics of a triax cable.

35 is a block diagram illustrating another configuration example of a digital triax system.

36 is a flowchart for explaining an example flow in a rate control process;

Fig. 37 is a block diagram showing another configuration example of a digital triax system.

38 is a diagram illustrating an example of the state of data to be transmitted.

39 is a block diagram illustrating yet another example of the configuration of a digital triax system.

40 is a flowchart for explaining an example of the flow of a control process;

41 is a diagram illustrating a configuration example of an information processing system to which the present invention is applied.

<Code description>

100: Digital Triax System

120: video signal encoder

136: video signal decoder

137: data controller

138: data conversion unit

301: memory section

302: packetization unit

321: Depacketize part

353: line block determination unit

354: cumulative value counting unit

355: cumulative result determination unit

356: encoded data storage control unit

357: first coded data output unit

358: second coded data output unit

453: coded data storage control unit

454: Accumulation determination unit

456: group determination unit

457: cumulative value counting unit

458: cumulative result determination unit

459: first encoded data output unit

460: second coded data output unit

512 camera control unit

543: data control unit

544: memory

581 camera control unit

601: communication device

602: communication device

623: data controller

643: data control unit

1113: rate control unit

1401: modulation control unit

1402: encoding control

1403: C / N ratio measuring unit

1404: error rate measuring unit

1405: measurement result determination unit

1761: synchronization control unit

1771: synchronization control unit

<Best Mode for Carrying Out the Invention>

EMBODIMENT OF THE INVENTION Hereinafter, embodiment of this invention is described.

3 is a block diagram showing a configuration example of a digital triax system to which the present invention is applied.

In FIG. 3, the digital triax system 100 is a coaxial cable connecting a video camera, a camera control unit, and a switcher at the time of studio recording, relaying, etc. in a television broadcast station or a production studio. The system superimposes and transmits a plurality of signals such as a signal, a video signal of a return (return), a synchronization signal, and also supplies power.

In this digital triax system 100, the transmission unit 110 and the camera control unit 112 are connected via a triax cable (coaxial cable) 111. Transmission of a digital video signal and a digital audio signal (hereinafter referred to as a main line signal) which are actually broadcasted or used as a material from the transmission unit 110 to the camera control unit 112, and from the camera control unit 112 to the video camera unit ( The audio signal for the income and the digital video signal for the return to 113) are transmitted through the triax cable 111.

The transmission unit 110 is built in, for example, a video camera device not shown. The present invention is not limited to this, and the transmission unit 110 may be used as an external device to the video camera device by being connected to the video camera device in a predetermined method. In addition, the camera control part 112 is a device generally called a CCU (Camera Control Unit), for example.

In addition, since the digital audio signal has little relation with the well-known of this invention, the description for avoiding the complexity is omitted.

The video camera unit 113 is configured in, for example, a video camera apparatus (not shown) and receives light from a subject incident through the optical system 150 having a lens, a focus mechanism, a zoom mechanism, an iris adjustment mechanism, and the like. Light is received by an imaging device (not shown) made of (Charge Coupled Device) or the like. The image pickup device converts the received light into an electric signal by photoelectric conversion, and performs a predetermined signal processing to output a baseband digital video signal. This digital video signal is mapped and output in the format of HD-SDI (High Definition-Serial Data Interface), for example.

In addition, the video camera unit 113 is connected to a display unit 151 used as a monitor and an income 152 for exchanging voices with the outside.

The transmission unit 110 separates and synthesizes video from the video signal encoder 120 and the video signal decoder 121, the digital modulator 122 and the digital demodulator 123, the amplifier 124 and the amplifier 125. Has a portion 126.

In the transmission unit 110, a digital video signal of a base band mapped to, for example, a format of HD-SDI, is supplied from the video camera unit 113. The digital video signal is data of a main line video image, which is compressed and encoded by the video signal encoder 120 to be encoded data (encoded stream) and supplied to the digital modulator 122. The digital modulator 122 modulates the supplied encoded stream into a signal of a type suitable for transmission through the triax cable 111 and outputs the modulated signal. The signal output from the digital modulator 122 is supplied to the video splitter / combiner 126 via the amplifier 124. The video separating / composing unit 126 transmits the supplied signal to the triax cable 111. This signal is supplied to the camera control unit 112 via the triax cable 111.

Moreover, the signal output from the camera control part 112 is supplied to the transmission unit 110 via the triax cable 111, and is received. The received signal is supplied to the video separation / synthesizing unit 126, where a portion of the digital video signal and another portion of the signal are separated. A portion of the digital video signal of the received signal is supplied to the digital demodulator 123 through the amplifier 125, and modulated into a signal of a type suitable for transmission through the triax cable 111 at the camera controller 112 side. And demodulate the coded stream.

The encoded stream is supplied to the video signal decoder 121, and the compressed code is decoded to form a baseband digital video signal. The decoded digital video signal is mapped and output in the HD-SDI format and supplied to the video camera unit 113 as a return digital video signal (data of a return video image). This return digital video signal is supplied to a display unit 151 connected to the video camera unit 113, and used for a monitor of a return video image for a photographer.

The camera control unit 112 includes a video separation / synthesis unit 130, an amplifier 131 and an amplifier 132, a front end unit 133, a digital demodulator 134, and a digital modulator 135 and a video signal decoder. 136 and a data control unit 137.

The signal output from the transmitting unit 110 is supplied to and received by the camera control unit 112 through the triax cable 111. The received signal is supplied to the video separation / combining unit 130. The video separation / combining unit 130 supplies the supplied signal to the digital demodulation unit 134 through the amplifier 131 and the front end unit 133. The front end section 133 has a gain control section for adjusting the gain of the input signal, a filter section for performing predetermined filter processing on the input signal, and the like.

The digital demodulator 134 demodulates the signal modulated into a signal of a format suitable for transmission via the triax cable 111 on the transmission unit 110 side, and restores the encoded stream. This encoded stream is supplied to a video signal decoding unit 136, and the compressed code is decoded to form a baseband digital video signal. This decoded digital video signal is mapped and output in the format of HD-SDI, and output to the outside as a digital video signal for main line.

The digital audio signal is supplied to the camera controller 112 from the outside. The digital audio signal is, for example, supplied to the photographer's income 152 and used to convey an instruction by voice to the photographer from the outside. The video signal decoder 136 decodes the coded stream supplied from the digital demodulator 134 and supplies the coded stream before decoding to the data control unit 137. The data control unit 137 converts the bit rate into an appropriate value in order to process the encoded stream as an encoded stream of the return digital video signal.

In addition, hereinafter, for convenience of explanation, the video signal decoding unit 136 and the data control unit 137 are collectively referred to as a data conversion unit 138. That is, the data converter 138 is a processor that includes a video signal decoder 136 and a data controller 137 to perform processing related to data conversion such as decoding and bit rate conversion. Of course, the data conversion unit 138 may also perform other conversion processing.

In general, the return digital video signal may have a lower image quality than the main digital video signal. Therefore, the data control unit 137 lowers the bit rate of the supplied encoded stream to a predetermined value. The details of the data control unit 137 will be described later. By the data control unit 137, the encoded stream whose bit rate is changed is supplied to the digital modulator 135. The digital modulator 135 modulates the supplied encoded stream into a signal of a type suitable for transmission through the triax cable 111 and outputs the modulated signal. The signal output from the digital modulator 135 is supplied to the video splitter / combiner 130 through the front end 133 and the amplifier 132. The video separation / combining unit 130 multiplexes this signal with other signals and sends the signal to the triax cable 111. This signal is supplied to the transmission unit 110 via the triax cable 111 as a digital video signal for return.

The video separation / combining unit 126 supplies the supplied signal to the digital demodulation unit 123 through the amplifier 125. The digital demodulator 123 demodulates the supplied signal, restores the encoded stream of the return digital video signal, and supplies it to the video signal decoder 121. The video signal decoding unit 121 decodes the encoded stream of the supplied return digital video signal, and supplies it to the video camera unit 113 when the return digital video signal is obtained. As described above, the video camera unit 113 supplies the return digital video signal to the display unit 151 to display the return video image.

Although details will be described later, the data control unit 137 changes the bit rate without decoding the encoded stream of the digital video signal of the main line signal, so that the encoded stream whose bit rate is changed is encoded for the digital video signal. It can be used as a stream and can be transmitted to the video camera unit 113. As a result, the digital triax system 100 may shorten the delay time until the return video image is displayed on the display unit 151. In addition, since the camera control unit 112 does not need to provide an encoder for the return digital video signal, the circuit scale and the cost of the camera control unit 112 can be reduced.

4 is a block diagram illustrating a detailed configuration example of the video signal encoder 120 of FIG. 3.

In FIG. 4, the video signal encoder 120 includes a wavelet transform unit 210, an intermediate calculation buffer unit 211, a coefficient rearrangement buffer unit 212, a coefficient rearrangement unit 213, and a quantization unit 214. ), An entropy encoder 215, a rate controller 216, and a packetizer 217.

The input image data is temporarily stored in the calculation buffer unit 211 on the way. The wavelet transform unit 210 performs wavelet transform on the image data accumulated in the calculation buffer unit 211 on the way. That is, the wavelet converter 210 reads image data from the calculation buffer unit 211 on the way and performs a filter process by an analysis filter to generate data of coefficients of low and high frequency components, and generates generated coefficient data. Is stored in the calculation buffer unit 211 on the way. The wavelet converter 210 has a horizontal analysis filter and a vertical analysis filter, and performs an analysis filter process on both of the screen horizontal direction and the screen vertical direction for the image data group. The wavelet transforming unit 210 reads the coefficient data of the low pass component stored in the calculation buffer unit 211 on the way, performs a filter process on the read coefficient data by an analysis filter, and performs the processing of the high pass component and the low pass component. Generate the data of the coefficient again. The generated coefficient data is stored in the calculation buffer unit 211 on the way.

If the decomposition level reaches a predetermined level by repeating this process, the wavelet transform unit 210 reads the coefficient data from the calculation buffer unit 211 on the way, and converts the read coefficient data into the coefficient rearrangement buffer unit. (212).

The coefficient rearrangement unit 213 reads coefficient data written in the coefficient rearrangement buffer unit 212 in a predetermined order and supplies the coefficient data to the quantization unit 214. The quantization unit 214 quantizes the coefficient data to be supplied and supplies it to the entropy encoding unit 215. The entropy coding unit 215 encodes the supplied coefficient data by a predetermined entropy coding method such as, for example, Halfman coding or arithmetic coding.

The entropy encoding unit 215 operates in conjunction with the rate control unit 216 and is controlled so that the bit rate of the output compressed coded data becomes approximately constant. That is, the rate control unit 216 reaches the point or time at which the bit rate of the data compression-coded by the entropy encoder 215 reaches the target value based on the encoded data information from the entropy encoder 215. The control signal for controlling to end the encoding process by the entropy encoding unit 215 immediately before is supplied to the entropy encoding unit 215. The entropy encoding unit 215 supplies the encoded data to the packetizing unit 217 at the time when the encoding process ends in accordance with the control signal supplied from the rate control unit 216. The packetizer 217 sequentially packetizes the supplied encoded data and outputs the encoded data to the digital modulator 122 of FIG.

Next, the processing performed by the wavelet transform unit 210 will be described in more detail. First, the wavelet transform will be described schematically. In the wavelet transformation for image data, as shown schematically in FIG. 5, a process of dividing image data into a band having a high spatial frequency and a low band is performed for data of a band having a low spatial frequency at which the result of the division is obtained. Repeat it recursively. In this way, efficient compression coding is made possible by driving data of a band having a low spatial frequency into a smaller area.

5 is an example of the case where the division process of the area | region L of the lowpass component and the area | region H of the highpass component with respect to the lowest-pass component area | region of image data is repeated 3 times, and division level = 3. In Fig. 5, "L" and "H" represent a low pass component and a high pass component, respectively, and the order of "L" and "H" shows the band of the result which the front side divided | segmented in the horizontal direction, and a rear side was a vertical direction. The band of the result of dividing is shown. In addition, the number before "L" and "H" shows the division level of the area | region.

In addition, as can be seen from the example of FIG. 5, the process is performed step by step from the lower right area to the upper left area of the screen, and low-pass components are driven. That is, in the example of FIG. 5, the area at the lower right of the screen is the area 3HH having the lowest low frequency component (which contains the most high frequency component), and the area at the upper left where the screen is divided into four is further divided into four, The upper left area of the four divided areas is further divided into four parts. The region of the upper left corner is the region 0LL containing the most low pass component.

The conversion and division of the low pass component repeatedly is because the energy of the image is concentrated in the low pass component. This point is shown in FIG. 6 as the division level is advanced from the state of division level = 1 in which an example is shown in A of FIG. 6 as in the state of division level = 3 in which an example is shown in B of FIG. It is also understood from the formation of the sub band as shown in B. FIG. For example, the division level of the wavelet transform in FIG. 5 is 3, and as a result, 10 subbands are formed.

The wavelet transform unit 210 performs the above-described processing using a filter bank composed of a low pass filter and a high pass filter. In addition, since a digital filter usually has a multi-tap length impulse response, that is, filter coefficients, it is necessary to buffer enough input image data or coefficient data in advance so that the filter process can be performed. Similarly, in the case where the wavelet transform is performed in multiple stages, it is necessary to buffer the wavelet transform coefficients generated in the preceding stages so that the filter process can be performed.

As a specific example of this wavelet transform, a method using a 5x3 filter will be described. The method using this 5x3 filter is adopted also in the Joint Photographic Experts Group (JPEG) 2000 standard described above in the prior art, and is an excellent method in that wavelet conversion can be performed with a small number of filter taps.

The impulse response (Z transform representation) of the 5x3 filter is composed of a low pass filter H ₀ (z) and a high pass filter H ₁ (z), as represented by the following equations (1) and (2). From the equations (1) and (2), it was found that the low pass filter H ₀ (z) is 5 taps, and the high pass filter H ₁ (z) is 3 taps.

According to these formulas (1) and (2), coefficients of the low pass component and the high pass component can be calculated directly. Here, the calculation of the filter process can be reduced by using a lifting technique.

Next, this wavelet conversion method will be described in more detail. Fig. 7 shows an example in which the filter process by lifting the 5x3 filter is performed up to the decomposition level = 2. In FIG. 7, the part shown as an analysis filter on the left side of the figure is a filter of the wavelet transform unit 210 in the video signal encoder 120. In addition, the part shown as a synthesis filter on the right side of a figure is a filter of the wavelet inverse transform part in the video signal decoding part 136 mentioned later.

In the following description, for example, in a display device or the like, the pixel is scanned from the left end of the screen to the right end with the pixel in the upper left corner of the screen as a head, so that one line is formed, and scanning for each line is performed at the top of the screen. It is assumed that the first screen is configured from the bottom toward the lower end.

In Fig. 7, the left column shows pixel data at corresponding positions on a line of original image data arranged in a vertical direction. In other words, the filter processing in the wavelet converter 210 is performed by vertically scanning pixels on the screen using a vertical filter. The first to third columns from the left end represent the filter processing of division level = 1, and the fourth to sixth columns represent the filter processing of division level = 2. The second column from the left end shows the high pass component output based on the pixel of the left end original image data, and the third column from the left end shows the low pass component output based on the original image data and high pass component output. As shown in the fourth to sixth columns from the left end, the filter processing of dividing level = 2 is performed on the output of the filter processing of dividing level = 1.

In the filter processing of decomposition level = 1, as the filter processing of the first stage, coefficient data of the high frequency component is calculated based on the pixels of the original image data, and is calculated by the filter processing of the first stage as the filter processing of the second stage. The coefficient data of the low frequency component is calculated based on the coefficient data of the high frequency component and the pixels of the original image data. Filter processing of an example of decomposition level = 1 is shown in the first to third columns on the left side (analysis filter side) in FIG. 7. The calculated coefficient data of the high frequency component is stored in the coefficient rearrangement buffer unit 212 described with reference to FIG. 4. The calculated low pass component coefficient data is stored in the calculation buffer unit 211 on the way.

In FIG. 7, data enclosed by a dashed-dotted line is temporarily stored in the coefficient rearrangement buffer part 212, and data enclosed by a dotted line is temporarily stored in the calculation buffer part 211 on the way.

Based on the result of the filter process of decomposition level = 1 held in the calculation buffer part 211 on the way, the filter process of decomposition level = 2 is performed. In the filter processing with decomposition level = 2, the filter data calculated as the coefficient of the low pass component in the filter process with decomposition level = 1 is regarded as coefficient data including the low pass component and the high pass component, and the same filter processing as the decomposition level = 1 Is done. The coefficient data of the high pass component and the coefficient data of the low pass component calculated by the filter processing of decomposition level = 2 are stored in the coefficient rearrangement buffer unit 212 described in FIG. 4.

The wavelet converter 210 performs the above-described filter processing in the horizontal and vertical directions of the screen, respectively. For example, first, the filter processing of decomposition level = 1 is performed in the horizontal direction, and the generated coefficient data of the high frequency component and the low frequency component is stored in the calculation buffer unit 211 on the way. Next, the filter data of decomposition level = 1 is performed in the vertical direction with respect to the coefficient data stored in the calculation buffer part 211 on the way. By the processing in the horizontal and vertical directions of the decomposition level = 1, the region HH and the region HL, respectively, of the coefficient data obtained by decomposing the high frequency component into the high frequency component and the low frequency component, and the low frequency component into the high frequency component and the low frequency component again. Four areas of the area LH and the area LL are formed by each of the decomposed coefficient data.

Then, at decomposition level = 2, filter processing is performed on the coefficient data of the low pass component generated at decomposition level = 1 in each of the horizontal direction and the vertical direction. That is, at the decomposition level = 2, the region LL formed by dividing at the decomposition level = 1 is further divided into four, and the region HH, the region HL, the region LH and the region LL are formed in the region LL again.

The wavelet transform unit 210 divides the filter process by the wavelet transform into a process for each line in the vertical direction of the screen, and divides the process into a plurality of steps in a stepwise manner. In the example of FIG. 7, the first process, which is the process from the first line on the screen, filters the 7 lines, and the second and subsequent processes that become the process from the 8th line are filtered every 4 lines. The process is performed. The number of lines is based on the number of lines necessary for generating the lowest frequency component for one line after two decomposition into a high frequency component and a low frequency component.

In the following, a line block (or free) includes a collection of lines including other subbands necessary for generating one line of the lowest band component (coefficient data for one line of the subband of the lowest band component). Sint). Here, a line represents pixel data or coefficient data of one row formed in the picture or field corresponding to the image data before wavelet conversion, or in each subband. In other words, a line block (priint) is a group of pixel data for the number of lines necessary to generate coefficient data for one line of subbands of the lowest band component after the wavelet transform in the original image data before the wavelet transform, or The coefficient data group of each subband obtained by wavelet transforming the pixel data group is shown.

According to FIG. 7, the coefficient C5 obtained as a result of the filter processing at decomposition level = 2 is calculated based on the coefficient C4 and the coefficient C _a stored in the intermediate calculation buffer unit 211, and the coefficient C4 is the intermediate calculation buffer unit 211. ) it is calculated based on the stored coefficient C _a, C _b and the coefficient to the coefficient C _c. The coefficient C _c is calculated based on the coefficient C2 and the coefficient C3 and the pixel data of the fifth line stored in the coefficient rearrangement buffer unit 212. The coefficient C3 is calculated based on the pixel data of the fifth to seventh lines. In this way, in order to obtain the coefficient C5 of the low pass component at the division level = 2, the pixel data of the first to seventh lines is required.

On the other hand, in the second and subsequent filter processes, the coefficient data already calculated by the previous filter process and stored in the coefficient rearrangement buffer unit 212 can be used, thereby reducing the number of necessary lines.

That is, according to FIG. 7, coefficient C9 which is the next coefficient of coefficient C5 among the coefficients of the low-pass component obtained as a result of the filter process of decomposition level = 2 is coefficient C4 and coefficient C8, and coefficient C stored in middle calculation buffer part 211. _It is calculated based on _c . The coefficient C4 is already calculated by the first filter process described above and stored in the coefficient rearrangement buffer unit 212. Similarly, the coefficient C _c is already calculated by the first filter process described above, and is stored in the calculation buffer unit 211 on the way. Therefore, in this second filter process, only the filter process for calculating the coefficient C8 is newly performed. This new filter process uses the eighth to eleventh lines as well.

In this way, the filter processing after the second time is calculated by the filter processing up to the previous time, and data stored in the calculation buffer unit 211 and the coefficient rearrangement buffer unit 212 can be used. Processing will complete.

If the number of lines on the screen does not match the number of lines of encoding, the lines of the original image data are duplicated by a predetermined method, and the number of lines is matched with the number of lines of encoding to perform the filter process.

In this way, the filter processing as much as the coefficient data for one line of the lowest frequency component is obtained in stages, divided into a plurality of times (in line block units) for the entire line of the screen, so that decoding is performed with low delay when the encoded data is transmitted. It is possible to obtain an image.

In order to perform the wavelet transform, a first buffer used to execute the wavelet transform itself and a second buffer for storing coefficients generated during the processing up to a predetermined division level are required. The first buffer corresponds to the calculation buffer unit 211 on the way, and in Fig. 7, data enclosed by a dotted line is temporarily stored. In addition, the second buffer corresponds to the coefficient rearrangement buffer unit 212, and in FIG. 7, data enclosed by a dashed-dotted line is temporarily stored. Since the coefficients stored in the second buffer are used at the time of decoding, they are subjected to entropy encoding processing at a later stage.

The processing of the coefficient rearrangement unit 213 will be described. As described above, the coefficient data calculated by the wavelet transform unit 210 is stored in the coefficient rearrangement buffer unit 212, and the order is rearranged and read by the coefficient rearrangement unit 213, and the quantization unit It is sent to 214.

As described above, in the wavelet transform, coefficients are generated from the high pass component side to the low pass component side. In the example of FIG. 7, the coefficient C1, the coefficient C2, and the coefficient C3 of the high frequency component are sequentially generated by the pixel data of the original image in the filter processing of decomposition level = 1. Then, filter processing with decomposition level = 2 is performed on the coefficient data of the low pass component obtained by the filter processing with decomposition level = 1, and the coefficients C4 and coefficient C5 of the low pass component are sequentially generated. That is, in the first time, the coefficient data is generated in the order of the coefficient C1, the coefficient C2, the coefficient C3, the coefficient C4, and the coefficient C5. The generation order of the coefficient data is always in this order (from high to low) on the principle of wavelet transform.

On the other hand, on the decoding side, it is necessary to generate and output an image from a low pass component in order to decode immediately with low delay. Therefore, it is preferable to rearrange the coefficient data generated on the encoding side from the lowest pass component side toward the high pass component side and supply it to the decoding side.

It demonstrates more concretely using the example of FIG. The right side of Fig. 7 shows the synthesis filter side for performing inverse wavelet transform. The first synthesis process (wavelet inverse transform process) including the first line of the output image data on the decoding side includes coefficients C4 and coefficient C5 of the lowest frequency component generated by the first filter process on the encoding side, It is done using the coefficient C1.

That is, in the first synthesis process, coefficient data is supplied from the encoding side to the decoding side in the order of the coefficient C5, the coefficient C4, and the coefficient C1, and the processing side at the synthesis level = 2, which is the synthesis process corresponding to decomposition level = 2 , Coefficients C5 and C4 are synthesized to generate coefficients C _f and stored in a buffer. And in the process of synthesis level = 1 which is the synthesis process corresponding to decomposition level = 1, a synthesis process is performed with respect to this coefficient _Cf and the coefficient C1, and a 1st line is output.

As described above, in the first synthesis process, the encoding data generates coefficients C1, coefficients C2, coefficients C3, coefficients C4, and coefficients C5 and stores coefficient data stored in the coefficient rearrangement buffer unit 212 in the coefficients C5, Coefficient C4, coefficient C1,... It is rearranged in order of and supplied to the decoding side.

In addition, on the synthesis filter side shown in the right side of FIG. 7, the number of the coefficient in the encoding side is described in parentheses about the coefficient supplied from the encoding side, and the line order of the synthesis filter is described outside the parentheses. For example, the coefficient C1 (5) is the coefficient C5 on the analysis filter side on the left side of FIG. 7, and indicates that it is the first line on the synthesis filter side.

The decoding side synthesis processing by the coefficient data generated in the second and subsequent filter processings on the encoding side can be performed using the coefficient data supplied from the synthesis or encoding side at the time of the previous synthesis processing. In the example of FIG. 7, the 2nd synthesis process of the decoding side performed using the coefficient C8 and the coefficient C9 of the lowpass component produced | generated by the 2nd filter process of the encoding side is produced | generated by the 1st filter process of the encoding side. Coefficient C2 and Coefficient C3 are also required, and the second to fifth lines are decoded.

That is, in the second synthesis process, coefficient data is supplied from the encoding side to the decoding side in the order of the coefficient C9, the coefficient C8, the coefficient C2, and the coefficient C3. On the decoder side, processing of synthesis level = 2, and the coefficient C8 and coefficient C9, using the coefficient C4 supplied from the encoding side at the time of synthesizing the first process generate the coefficient C _g, and stored in the buffer. Using the coefficient C _g , the coefficient C4 described above, and the coefficient C _f generated by the first synthesis process and stored in the buffer, the coefficient C _h is generated and stored in the buffer.

And, it is shown in the process of synthesizing level = 1, a synthesis level = 2, the coefficient C _g and the coefficient C _h and encoding the coefficient C2 (in the synthesis filter coefficient C6 (2) supplied from the side is stored in is generated in the processing buffer of the And the coefficient C3 (indicated by the coefficient C7 (3) in the synthesis filter), the synthesis process is performed, and the second to fifth lines are decoded.

As described above, in the second synthesis process, the coefficient data generated in the order of coefficients C2, coefficients C3, (coefficients C4, coefficients C5), coefficients C6, coefficients C7, coefficients C8, and coefficients C9 are coefficients C9. , Coefficient C8, coefficient C2, coefficient C3,... It is rearranged in order of and supplied to the decoding side.

Similarly, in the synthesis processing after the third time, the coefficient data stored in the coefficient rearrangement buffer unit 212 is rearranged in a predetermined order and supplied to the decoding unit, and the lines are decoded by four lines.

In addition, in the encoding process on the decoding side corresponding to the filter process (hereinafter referred to as the last time) including the line at the bottom of the screen on the encoding side, it is necessary to output all the coefficient data generated in the above process and stored in the buffer. As a result, the number of output lines increases. In the example of Fig. 7, eight lines are output at the last time.

In addition, the rearrangement process of the coefficient data by the coefficient rearrangement unit 213 is performed by, for example, setting a read address when reading coefficient data stored in the coefficient rearrangement buffer unit 212 in a predetermined order. Is done.

Referring to Fig. 8, the above-described processing will be described in more detail. 8 shows an example in which a filter process by wavelet transform is performed to a decomposition level of 2 using a 5x3 filter. In the wavelet converting unit 210, as shown in FIG. 8A, the first filter processing is performed in the horizontal and vertical directions from the first line to the seventh line of input image data, respectively (Fig. In-1 of A of 8).

In the process at the decomposition level = 1 of the first filter process, coefficient data for three lines of the coefficient C1, the coefficient C2 and the coefficient C3 is generated, and as shown in B of FIG. 8, at an decomposition level = 1. It is arrange | positioned in each of the area | region HH formed, the area | region HL, and the area | region LH (WT-1 of FIG. 8B).

Further, the region LL formed at decomposition level = 1 is further divided into four in the horizontal and vertical filter processing at decomposition level = 2. The coefficient C5 and the coefficient C4 generated at the decomposition level = 2 are arranged in the region LL with the decomposition level = 1 in the region LL with one line with the coefficient C5 in each of the regions HH, HL and LH. One line by the coefficient C4 is arranged.

In the filter processing after the second time by the wavelet transforming unit 210, the filter processing is performed every four lines (In-2 in Fig. 8A), and coefficient data of two lines are generated at the decomposition level = 1. (WT-2 in B of FIG. 8), coefficient data for each line is generated at the decomposition level = 2.

In the second example of FIG. 7, coefficient data for two lines of the coefficient C6 and the coefficient C7 is generated in the filter processing at decomposition level = 1, and is formed at decomposition level 1 as shown in FIG. 8B as an example. The area HH, the area HL, and the area LH are arranged after the coefficient data generated by the first filter process. Similarly, within the region LL with decomposition level = 1, the coefficient C9 for one line generated by the filter process with decomposition level = 2 is disposed in the region LL, and the coefficient C8 for one line is the region HH, the region HL and the region. Are each disposed in the LH.

When decoding the wavelet transformed data as shown in B of FIG. 8, as shown in C of FIG. 8, the decoding side of the first filter processing by the first to seventh lines on the encoding side is shown. The first line by the first synthesis process in is outputted (Out-1 in Fig. 8C). Thereafter, four lines are output on the decoding side for the filter processing from the second to the last time on the encoding side (Out-2 in FIG. 8C). Then, 8 lines are output from the decoding side with respect to the last filter processing on the encoding side.

The coefficient data generated by the wavelet transform unit 210 from the high pass component side to the low pass component side is sequentially stored in the coefficient rearrangement buffer unit 212. When coefficient data is accumulated in the coefficient rearrangement buffer unit 212 until the above-mentioned coefficient data rearrangement becomes possible, the coefficient rearrangement unit 213 performs a synthesis process from the coefficient rearrangement buffer unit 212. Read the coefficient data by rearranging it in the required order. The read coefficient data is sequentially supplied to the quantization unit 214.

The quantization unit 214 quantizes the coefficient data supplied from the coefficient rearrangement unit 213. Any method may be used as the method of quantization, and for example, a general means, i.e., a method of dividing coefficient data W represented by Equation 3 below by the quantization step size Δ may be used.

The entropy encoding unit 215 controls the encoding operation so that the bit rate of the output data becomes the target bit rate based on the control signal supplied from the rate control unit 216 with respect to the coefficient data quantized and supplied as described above. Entropy coding is performed. Entropy coded encoded data is supplied to the decoding side. As the encoding method, half-man coding, arithmetic coding, or the like, which are known techniques, can be considered. Of course, it is not limited to these, If a reversible coding process is possible, you may use another coding system.

As described with reference to FIGS. 7 and 8, the wavelet converter 210 performs wavelet transform processing for each of a plurality of lines (per line block) of image data. Encoded data encoded by the entropy encoding unit 215 is output for each line block. That is, when the processing is performed up to the decomposition level = 2 using the above-described 5x3 filter, the first time is the first line, and the last time is 4 lines before the last time after the second time. Eight lines of output are obtained.

In addition, when entropy-encoding coefficient data after rearrangement by the coefficient rearrangement unit 213, for example, when entropy-encoding the line of the first coefficient C5 in the first filter process shown in FIG. There is no line at which the coefficient data has already been generated. Therefore, in this case, only one line is entropy encoded. On the other hand, when encoding the line of the coefficient C1, the line of the coefficient C5 and the coefficient C4 becomes a past line. Since these adjacent multiple lines are considered to be comprised of similar data, it is effective to combine these multiple lines and entropy-encode.

In addition, in the above description, the example in which the wavelet transform unit 210 performs the filter processing by the wavelet transform using a 5x3 filter has been described, but this is not limited to this example. For example, the wavelet converter 210 may use a filter having a longer tap number, such as a 9 × 7 filter. In this case, when the number of taps of the filter is long, the number of lines accumulated in the filter also increases, so that a delay time from input of image data to output of encoded data becomes long.

Incidentally, in the above description, the decomposition level of the wavelet transform is set to decomposition level = 2 for explanation, but this is not limited to this example, and the decomposition level can be raised further. The higher the decomposition level, the higher the compression ratio. For example, in general, in the wavelet transform, the filter process is repeated up to decomposition level = 4. In addition, as the decomposition level rises, the delay time also increases.

Therefore, when applying the present invention to an actual system, it is preferable to determine the number of taps of the filter and the resolution level according to the delay time required for the system, the quality of the decoded image, and the like. The number of taps and the resolution level of the filter are not fixed but can be selected adaptively.

The coefficient data transformed and rearranged as described above is quantized by the quantization unit 214 and encoded by the entropy encoding unit 215. The obtained encoded data is transmitted to the camera control unit 112 via the digital modulator 122, the amplifier 124, the video separation / synthesis unit 126, and the like. At that time, the encoded data is packetized by the packetizing unit 217 and transmitted as a packet.

9 is a schematic diagram for explaining an example of the mode of passing the encoded data. As described above, the image data is wavelet-converted for a predetermined number of lines for each line block (subband 251). When the predetermined wavelet transform resolution level is reached, the coefficient lines from the lowest subband to the highest subband are rearranged in the reverse order of the generated order, that is, from low to high frequency.

In the subband 251 of FIG. 9, portions divided into diagonal lines, vertical lines, and broken lines are different line blocks (as indicated by arrows), and the white portions of the sub band 251 are similarly line-by-line. Divided into). The coefficients of the line blocks after rearrangement are entropy coded as described above to generate encoded data.

Here, for example, when the transmission unit 110 transmits the encoded data as it is, it may be difficult for the camera control unit 112 to identify the boundary of each line block (or complicated processing is required). The packetizing unit 217 adds a header to the encoded data, for example, in line block units, generates a packet consisting of the header and the encoded data, and transmits the packet, thereby facilitating the process of receiving data. .

As shown in FIG. 9, when the transmission unit 110 generates encoded data (encoded data) of the first line block (Line block-1), the transmission unit 110 packetizes it and converts it into a camera as a transmission packet 261. It sends out to the control part 112. Upon receiving the packet (receive packet 271), the camera control unit 112 depackets the packet, extracts the encoded data, and decodes (decodes) the encoded data.

Similarly, when the transmission unit 110 generates encoded data of the second line block (Line block-2), the transmission unit 110 packetizes it and sends it to the camera control unit 112 as the transmission packet 262. When receiving the packet (receive packet 272), the camera control unit 112 decodes (decodes) the encoded data. Similarly, when the transmission unit 110 generates encoded data of the third line block (Line block-3), the transmission unit 110 packetizes it and sends it to the camera control unit 112 as the transmission packet 263. Upon reception of the packet (receive packet 273), the camera control unit 112 decodes (decodes) the encoded data.

The transmission unit 110 and the camera control unit 112 repeat the above process to the X-th last line block (Line block-X) (transmission packet 264, reception packet 274). The decoded image 281 is generated by the camera control unit 112 as described above.

10 shows an example of the configuration of the header. As described above, the packet is composed of a header 291 and encoded data, but the header 291 has a number (NUM) 293 of the line block and the code amount for each subband constituting the line block. A description of the coded data length (LEN) 294 indicating a symbol is included. In addition, a description of the quantization step sizes (Δ1 to ΔN) 292 for each subband constituting the line block is added as information on encoding (encoding information).

The camera control unit 112 that receives the packet can easily identify the boundary of each line block by reading these information included in the header added to the received encoded data, thereby reducing the load and processing time of the decoding process. You can. In addition, by reading the encoding information, the camera control unit 112 can perform dequantization for each subband, so that finer picture quality control can be performed.

The transmission unit 110 and the camera control unit 112 may execute each processing such as encoding, packetization, packet transmission and reception, and the like simultaneously in parallel (pipelined) for each line block.

By doing in this way, the delay time until the image output is obtained by the camera control part 112 can be reduced significantly. In FIG. 9, an example of operation | movement in an interlaced moving image (60 fields / sec) is shown as an example. In this example, the time of one field is 1 second ÷ 60 = about 16.7 msec. However, by simultaneously performing each process, the image output can be obtained at a delay time of about 5 msec.

Next, the data converter 138 of FIG. 3 will be described. 11 is a block diagram illustrating a detailed configuration example of the data conversion unit 138.

The data converter 138 has a video signal decoder 136 and a data controller 137 as described above. In addition, as illustrated in FIG. 11, the data conversion unit 138 includes a memory unit 301 and a packetization unit 302.

The memory unit 301 has a rewritable storage medium such as random access memory (RAM), stores information supplied from the data control unit 137, and stores the information based on the request of the data control unit 137. The information is supplied to the data control unit 137.

The packetizing unit 302 packetizes the encoded data for return supplied from the data control unit 137, and supplies the packet to the digital modulation unit 135. The configuration and operation of the packetizer 302 are basically the same as those of the packetizer 217 shown in FIG.

When the video signal decoding unit 136 acquires a packet of encoded data supplied from the digital demodulation unit 134, it depackets and extracts the encoded data. The video signal decoding unit 136 performs decoding processing of the encoded data, and supplies the coded data before performing the decoding processing to the data control unit 137 via the bus D15. The data control unit 137 supplies the encoded data to the memory unit 301 via the bus D26 and accumulates the encoded data, or acquires the encoded data stored in the memory unit 301 via the bus D27 and returns data. As a result, the packetization unit 302 is supplied to control the bit rate of the encoded data for return.

Although details of the processing related to the conversion of the bit rate will be described later, the data control unit 137 temporarily accumulates the coded data supplied sequentially from the low frequency component to the memory unit 301, and reaches a predetermined data amount. A part or all of the encoded data stored in the memory unit 301 is read out and supplied to the packetizing unit 302 as encoded data for return. That is, the data control unit 137 uses the memory unit 301 to extract and output a part of the supplied encoded data, and discards the rest, thereby reducing (changing) the bit rate of the encoded data. If the bit rate is not changed, the data control unit 137 outputs all of the supplied encoded data.

The packetizer 302 packetizes the encoded data supplied from the data controller 137 for each predetermined size, and supplies the packetized data to the digital modulator 135. At this time, the information regarding the header of the encoded data is supplied from the video signal decoding unit 136 which performs depacketizing. The packetizing unit 302 performs packetization by appropriately correlating the information on the supplied header with the content of the bit rate conversion processing performed by the data control unit 137.

In the above description, the bus D26 is used when the encoded data is supplied from the data control unit 137 to the memory unit 301, and the encoded data read out from the memory unit 301 is used when the data control unit 137 is supplied. Although the bus D27 to be described is described as two independent buses, the transfer of the encoded data may be performed by one bus that can transfer in both directions.

For example, data other than coded data, such as a variable when the data control unit 137 uses the bit rate for conversion, may also be stored in the memory unit 301.

12 is a block diagram illustrating a configuration example of the video signal decoding unit 136. The video signal decoder 136 is a decoder corresponding to the video signal encoder 120. As shown in FIG. 12, the depacketize unit 321, the entropy decoder 322, and the inverse quantizer 323 ), A coefficient buffer unit 324 and a wavelet inverse transform unit 325.

A packet of encoded data output from the packetizer 217 of the video signal encoder 120 is supplied to the depacketize unit 321 of the video signal decoder 136 through various processing units. The depacketizing unit 321 depackets the supplied packet and extracts encoded data. The depacketizing unit 321 supplies the encoded data to the entropy decoding unit 322 and also to the data control unit 137.

When the entropy decoding unit 322 acquires the encoded data, the entropy decoding unit entropy decodes the encoded data for each line, and supplies the obtained coefficient data to the dequantization unit 323. The inverse quantization unit 323 performs inverse quantization on the supplied coefficient data based on the information about the quantization obtained from the depacketizing unit 321, and supplies the obtained coefficient data to the coefficient buffer unit 324. Save it. The wavelet inverse transform unit 325 performs synthesis filter processing by the synthesis filter using the coefficient data stored in the coefficient buffer unit 324, and stores the result of the synthesis filter processing in the coefficient buffer unit 324 again. The wavelet inverse transform unit 325 repeats this process according to the decomposition level to obtain decoded image data (output image data). The wavelet inverse transform unit 325 outputs this output image data to the outside of the video signal decoding unit 136.

In the case of the general wavelet inverse transform method, the horizontal synthesis filtering process is performed first in the screen horizontal direction, and the vertical synthesis filtering process is performed in the screen vertical direction, for all coefficients of the decomposition level to be processed. That is, for each synthesis filtering process, it is necessary to hold the result of the synthesis filtering process in the buffer, but the buffer must maintain the synthesis filtering result of the decomposition level at that time and the total coefficients of the next decomposition level at that time. It requires a large amount of memory (large amount of data to maintain).

In this case, since image data output is not performed until all wavelet inverse transforms are completed in the picture (field in the case of the interlacing method), the delay time from input to output increases.

In contrast, in the wavelet inverse transform unit 325, since the vertical synthesis filtering process and the horizontal synthesis filtering process are continuously performed up to level 1 in line block units, the wavelet inverse transform unit 325 can be buffered at the same time (simultaneously) compared with the conventional method. Since the amount of data required is small, the amount of memory of the buffer to be prepared can be greatly reduced. Further, by performing the synthesis filtering process (wavelet inverse transform process) up to level 1, it is possible to sequentially output the image data before the entire image data in the picture is obtained (in line block units), which is delayed in comparison with the conventional method. The time can be greatly reduced.

The video signal decoding unit 121 (FIG. 3) of the transmission unit 110 also has the same configuration as that of the video signal decoding unit 136, and executes the same processing. Therefore, the description described above with reference to FIG. 12 can basically be applied to the video signal decoder 121. However, in the case of the video signal decoding unit 121, the output from the depacketizing unit 321 is only supplied to the entropy decoding unit 322, and the supply to the data control unit 137 is not performed.

The various processes executed by each unit shown in FIG. 3 as described above are appropriately executed in parallel as shown in FIG. 13, for example.

FIG. 13 is a diagram schematically showing an example of parallel operation of each element of a process executed by each unit shown in FIG. 3. This FIG. 13 corresponds to FIG. 8 mentioned above. The wavelet transform unit 210 (FIG. 4) performs the first wavelet transform WT-1 on the input In-1 (A in FIG. 13) of the image data (B in FIG. 13). As described with reference to Fig. 7, the first wavelet transform WT-1 is started at the time when the first three lines are input, and the coefficient C1 is generated. That is, a delay of three lines occurs from the input of the image data In-1 to the start of the wavelet conversion WT-1.

The generated coefficient data is stored in the coefficient rearrangement buffer unit 212 (FIG. 4). Subsequently, the wavelet transform is performed on the input image data, and when the first process is completed, the process is transferred to the second wavelet transform WT-2 as it is.

In parallel with the input of the image data In-2 for the second wavelet transform WT-2 and the processing of the second wavelet transform WT-2, the coefficient rearrangement unit 213 (FIG. 4) gives a value of 3; The rearrangement Ord-1 of the coefficients C1, C4, and C5 is executed (C in Fig. 13).

The delay from the end of the wavelet transform WT-1 to the start of the rearrangement Ord-1 may be, for example, a delay associated with the transmission of a control signal instructing the coefficient rearrangement unit 213 of the rearrangement process. The delay based on the apparatus or system configuration, such as the delay required for the start of processing of the coefficient rearrangement unit 213 for the control signal, the delay required for the program processing, and the like, is not an essential delay in the encoding process.

The coefficient data is read from the coefficient rearrangement buffer unit 212 in the order in which the rearrangement is completed, supplied to the entropy coding unit 215 (FIG. 4), and entropy coding EC-1 is performed (FIG. 13). D). This entropy coding EC-1 can be started without waiting for the completion | finish of all rearrangement of the three coefficients C1, the coefficient C4, and the coefficient C5. For example, when the rearrangement of one line by the coefficient C5 to be outputted first is finished, the entropy coding for the coefficient C5 can be started. In this case, the delay from the start of processing of rearrangement Ord-1 to the start of processing of entropy coding EC-1 is one line.

Encoded data in which entropy encoding EC-1 is completed by the entropy encoding unit 215 is transmitted to the camera control unit 112 through the triax cable 111 after predetermined signal processing is performed (E in FIG. 13). ). At this time, the encoded data is packetized and transmitted.

To the video signal encoder 120 of the transmission unit 110, image data is input sequentially to the bottom line on the screen, following image data input for seven lines by the first process. In the video signal encoder 120, the wavelet transform WT-n, the rearranged Ord-n, and the entropy encoding EC for every four lines as described above with the input In-n (n is two or more) of the image data. -n Rearrangement Ord and entropy encoding EC for the last processing in the video signal encoder 120 are performed for six lines. These processes are performed in parallel in the video signal encoder 120 as illustrated in FIGS. 13A to 13D.

A packet of the encoded data encoded by the entropy encoding EC-1 by the video signal encoder 120 is transmitted to the camera controller 112 and supplied to the video signal decoder 136 after predetermined signal processing is performed. do. The depacketizing unit 321 extracts the encoded data from the packet and supplies it to the entropy decoding unit 322. The entropy decoding unit 322 sequentially decodes the entropy code iEC-1 on the supplied encoded data encoded by the entropy coding EC-1 to restore the coefficient data (F in FIG. 13). The reconstructed coefficient data is dequantized by the inverse quantization unit 323 and then sequentially stored in the coefficient buffer unit 324. The wavelet inverse transform unit 325 reads the coefficient data from the coefficient buffer unit 324 if the coefficient data is stored in the coefficient buffer unit 324 so as to perform the inverse wavelet transform, and uses the read coefficient data. The wavelet inverse transform iWT-1 is performed (G in FIG. 13).

As described with reference to FIG. 7, the wavelet inverse transform iWT-1 by the wavelet inverse transform unit 325 may be started at a time when the coefficients C4 and C5 are stored in the coefficient buffer unit 324. Therefore, the delay from the start of decoding iEC-1 by the entropy decoding unit 322 to the start of wavelet inverse transform iWT-1 by the wavelet inverse transform unit 325 is two lines.

In the wavelet inverse transform unit 325, when the wavelet inverse transform iWT-1 for three lines by the first wavelet transform is completed, the output Out-1 of the image data generated by the wavelet inverse transform iWT-1 is performed. (H in Fig. 13). In output Out-1, as described with reference to FIGS. 7 and 8, the image data of the first line is output.

The video signal decoder 136 is input to the entropy coding EC-n (n is 2 or more) following the input of the coded coefficient data for three lines by the first process in the video signal encoder 120. The coefficient data encoded by the data is sequentially input. The video signal decoding unit 136 performs entropy decoding iEC-n and wavelet inverse transform iWT-n every four lines on the input coefficient data as described above, and reconstructs the image by wavelet inverse transform iWT-n. Output Out-n of data is performed sequentially. The entropy decoding iEC and the wavelet inverse transform iWT corresponding to the last time of the video signal encoder 120 are performed for 6 lines, and the output Out is 8 lines. These processes are performed in parallel in the video signal decoding unit 136 as illustrated in F to 13 in FIG. 13.

As described above, the image compression processing and the image decoding processing are performed by performing the respective processing in the video signal coding unit 120 and the video signal decoding unit 136 in parallel from the top to the bottom of the screen in order. It becomes possible to perform more low-latency.

With reference to FIG. 13, the delay time from image input to image output when a wavelet transform is performed to resolution level = 2 using a 5x3 filter is calculated. The delay time from when the image data of the first line is input to the video signal encoder 120 and then the image data of the first line is output from the video signal decoder 136 is described in the following elements. The sum of In addition, the delay which differs according to system structure, such as the delay in a transmission path and the delay accompanying the actual process timing of each apparatus part, is excluded here.

(1) Delay D_WT from the first line input to the end of wavelet conversion WT-1 for seven lines

(2) Time D_Ord accompanying coefficient reordering Ord-1 for three lines

(3) Time D_EC accompanying three lines of entropy coding EC-1

(4) Time D_iEC accompanying entropy decoding iEC-1 for three lines

(5) Time D_iWT associated with wavelet inverse transform iWT-1 for three lines

Referring to Fig. 13, the calculation of the delay by each element described above is attempted. The delay D_WT in (1) is time for 10 lines. The time D_Ord of (2), the time D_EC of (3), the time D_iEC of (4), and the time D_iWT of (5) are time for three lines, respectively. The video signal encoder 120 may also start entropy coding EC-1 one line after rearrangement Ord-1 is started. Similarly, in the video signal decoding unit 136, two lines after the entropy decoding iEC-1 is started, the wavelet inverse transform iWT-1 can be started. In addition, the entropy decoding iEC-1 can start the process at the time when the encoding of one line is finished in the entropy encoding EC-1.

Accordingly, in the example of FIG. 13, the first line of image data is input to the video signal encoder 120, and then the image data of the first line is output from the video signal decoder 136. The delay time is 10 + 1 + 1 + 2 + 3 = 17 lines.

For the delay time, consider a more specific example. When the input image data is an interlaced video signal of HDTV (High Definition Television), for example, one frame is composed of a resolution of 1920 pixels by 1080 lines, and one field is 1920 pixels by 540 lines. Therefore, when the frame frequency is set to 30 Hz, 540 lines of one field are input to the video signal encoder 120 at a time of 16.67 msec (= 1 sec / 60 fields).

Therefore, the delay time accompanying the input of image data for seven lines is 0.216 msec (= 16.67 msec x 7/540 lines), for example, a very short time for one field update time. Also, since the number of lines to be processed is small for the sum of the above-described delay D_WT of (1), time D_Ord of (2), time D_EC of (3), time D_iEC of (4), and time D_iWT of (5). The delay time is very short. It is possible to further shorten the processing time by hardwareizing the elements that perform each processing.

Next, the operation of the data control unit 137 will be described.

As described above, the image data is wavelet transformed by the video signal encoder 120 in line block units, and the coefficient data of each subband obtained is rearranged in order from low to high, and then quantized and encoded. The converter 138 is supplied.

For example, each subband obtained by performing a wavelet transform (wavelet transform when splitting level = 2) is performed by the video signal encoder 120 as shown in FIG. If LLL, LHL, LLH, LHH, HL, LH, and HH are set from the low range, the encoded data of these subbands is obtained from the low range to the high range for each line block, as shown in B of FIG. 14 and C of FIG. It is supplied to the data conversion part 138 in order. In other words, the depacketized encoded data is supplied to the data control unit 137 in the same order.

14B and 14C show the order (of the sub band) of the coded data supplied to the data control unit 137, and show that they are supplied in order from the left. That is, first, in FIG. 14A, encoded data of each subband of the first line block, which is the highest line block in the image, is indicated by diagonal lines from the upper right to the lower left in FIG. 14. As shown in B, the data is supplied to the data control unit 137 in the order of the low band to the high band.

In Fig. 14B, 1LLL represents the subband LLL of the first line block, 1LHL represents the subband LHL of the first line block, 1LLH represents the subband LLH of the first line block, and 1LHH The subband LHH of the first line block is shown, 1HL represents the subband HL of the first line block, 1LH represents the subband LH of the first line block, and 1HH is the subband HH of the first line block. Indicates. In the example of FIG. 14B, first, coded data of 1LLL (coded data obtained by encoding coefficient data of 1LLL) is supplied first, followed by coded data of 1LHL, coded data of 1LLH, coded data of 1LHH, and coded 1HL. Data, 1LH coded data is supplied, and 1HH coded data is supplied last.

Then, when all of the data of the first line block is supplied, a second block, which is a line block below one of the first line blocks in the image data of the base band, which is indicated by diagonal lines from the lower right to the left in A of FIG. 14. The coded data of each subband of the line block is supplied to the data control unit 137 in the order of the lowband subbands and the highband subbands as shown in FIG. 14C.

In Fig. 14C, 2LLL represents the subband LLL of the second line block, 2LHL represents the subband LHL of the second line block, 2LLH represents the subband LLH of the second line block, and 2LHH The subband LHH of the second line block is shown, 2HL represents the subband HL of the second line block, 2LH represents the subband LH of the second line block, and 2HH is the subband HH of the second line block. Indicates. In the example of FIG. 14C, similarly to the case of B of FIG. 14, the encoded data of each subband includes 2LLL (subband LLL of the second line block), 2LHL, 2LLH, 2LHH, 2HL, 2LH, and 2HH. It is supplied in order.

As described above, the coded data is supplied for each line block in order from the top line block of the image in the baseband image data. That is, the encoded data of each subband of each line block after the third line block is also supplied in the same order as in the case of B of FIG. 14 and C of FIG.

In addition, since the order for each subband may be low to high range, it may be supplied in order of LLL, LLH, LHL, LHH, LH, HL, HH, for example. In the case where the decomposition level is 3 or more, similarly, the low level subbands are supplied in the order of the high band subbands.

With respect to the encoded data supplied in this order, the data control unit 137 accumulates the encoded data in the memory unit 301 for each line block, counting the total sum of the code amounts of the accumulated encoded data, and the code. When the amount reaches the target value, the coded data up to the immediately preceding subband is read from the memory unit 301 and supplied to the packetizing unit 302.

Referring to the example of B of FIG. 14 and C of FIG. 14, first, as indicated by the arrow 331 of B of FIG. 14, the data control unit 137 is supplied with respect to the encoded data of the first line block. The coded data is stored in the memory unit 301, and the cumulative value which is the sum of the code amounts of the accumulated coded data is counted (calculated). That is, each time the data control unit 137 accumulates the encoded data in the memory unit 301, the code amount of the accumulated encoded data is added to the accumulated value.

The data control unit 137 accumulates the encoded data in the memory unit 301 until the cumulative value reaches the predetermined target code amount. When the cumulative value reaches the target code amount, the data control unit 137 ends the accumulation of the encoded data. The encoded data up to the immediately preceding subband is read from the memory unit 301 and output. This target code amount is set according to the desired bit rate.

In the example of B of FIG. 14, the data control unit 137 sequentially accumulates the supplied encoded data while counting its code amount as shown by arrow 331, and breaks the code stream at which the accumulated value reaches the target code amount. When accumulated up to P1, the accumulation of encoded data ends, and as indicated by arrow 332, the encoded data from the encoded data of the first subband to the immediately preceding subband of the currently stored subband (Fig. 14). In the example of B, 1LLL, 1LHL, 1LLH, 1LHH, and 1HL) are read and output, and the data from the point P2 to the point P1 (the part of 1LH in the example of B in FIG. 14) that is the head of the current subband is read. Destroy.

In this way, the data control unit 137 controls the data output in units of subbands so that the video signal decoding unit 121 can decode the data output. The entropy coding unit 215 encodes the coefficient data in a manner that can be decoded at least in subband units, and the encoded data is configured in a format that can be decoded by the video signal decoding unit 121. Therefore, the data control unit 137 performs the selection of the coded data in subband units so as not to change the format of the coded data.

In the case of the wavelet transform (wavelet inverse transform) performed in the line block unit, even if coefficient data of all subbands in the line block does not coincide, data of the baseband is interpolated by performing data interpolation or the like during the wavelet inverse transform. You can restore the data to some extent. That is, even in the example of FIG. 14A, even if only coefficient data of LLL, LHL, LLH and LHH, which are low-band subbands exist, and coefficient data of high-band subbands HL, LH, and HH do not exist, for example, By using coefficient data of LLL, LHL, LLH, and LHH, which are low-band subbands, by replacing coefficient data of high-band subbands HL, LH, and HH, an image before wavelet transformation can be restored to some extent. In this case, however, since the high frequency component of the image does not exist, the interpolation method is generally used, but in general, the image quality of the restored image is degraded (the resolution is lowered) compared with the original image. However, in the wavelet transform, as described with reference to FIG. 6, the energy of the image is basically concentrated in the low pass component. Therefore, the effect of the image quality deterioration by the loss of a high frequency component is small for a user who views an image.

The data control unit 137 controls the bit rate of the encoded data by using the supplied encoded data having such a property. That is, the data control unit 137 extracts, from the supplied encoded data, the encoded data from the head until the target code amount is reached in accordance with the supply order as the encoded data for return. When the target code amount is smaller than the code amount of the original coded data, that is, when the data control unit 137 lowers the bit rate, the coded data for return is constituted by the low frequency component of the original coded data. In other words, the one from which some high frequency components were removed from the original coded data is extracted as the coded data for return.

The data control unit 137 performs the above processing on each line block. That is, when the processing is finished for the first line block as shown in FIG. 14B, the data control unit 137 processes the same for the second line block to be supplied next as shown in C of FIG. 14. Until the target code amount is reached from the head as shown by the arrow 333, the accumulated value is counted while accumulating the supplied encoded data in the memory unit 301, and the code stream breakpoint P3 is reached. At this point in time, as indicated by the arrow 334, the encoded data of the currently stored subband (2HL in the case of C in FIG. 14) is discarded, and the encoded data from the head to the immediately preceding subband (C in FIG. 14). In this example, 2LLL, 2LHL, 2LLH and 2LHH are read from the memory unit 301 and output as encoded data for return.

The bit rate conversion process is similarly performed for each line block after the third line block that is the next line block of the second line block.

In addition, since the code amount of each subband is independent for each line block, as shown in B of FIG. 14 and C of FIG. 14, the positions of the code stream breakpoints P1 and P3 are also independent of each other. In other cases, they may coincide with each other). Therefore, the subbands to be discarded (that is, the positions of the point P2 and the point P4 in B of FIG. 14 and C of FIG. 14) are also independent of each other.

The target code amount may be a fixed value or may be variable. For example, if the resolution is extremely different between line blocks and frames in the same image, the difference in image quality is conspicuous (image quality deterioration for a user viewing an image). In order to suppress such a phenomenon, the target code amount (that is, bit rate) may be appropriately controlled based on, for example, the content of the image. Further, for example, based on any external conditions such as the band in the transmission path of the triax cable 111 or the like, the processing capacity or load situation of the transmission unit 110 at the transmission destination, or the image quality required as the return video image, for example. The target code amount may be appropriately controlled.

As described above, the data control unit 137 can generate the encoded data for return at a desired bit rate independent of the bit rate of the supplied encoded data without decoding the supplied encoded data. In addition, since the data control unit 137 can perform this bit rate conversion processing by a simple process of outputting the encoded data starting from the user in the order of supply thereof, the bit rate of the encoded data can be easily and at high speed. Can be converted.

That is, the data control unit 137 can further shorten the delay time from when the digital video signal of the main line is supplied to the returning digital video signal to the transmission unit 110.

FIG. 15: is a schematic diagram which shows the relationship of the timing of each process performed in each part of the digital triax system 100 shown in FIG. 3, and is a figure corresponding to FIG. Encoding processing in the video signal coding unit 120 of the transmission unit 110 shown in the uppermost stage of FIG. 15, or video of the camera control unit 112 shown in the second stage from the upper side of FIG. 15. Since the decoding processing in the signal decoding unit 136 is executed at the same timing as shown in Fig. 2, the delay time from the start of the encoding processing until the decoding processing result is outputted is P [msec. ]to be.

Thereafter, as shown in the third column from the top of FIG. 15, the data control unit 137 outputs the encoded data for return after T [msec] after the output of the decode result is started, thereby decoding the video signal. The unit 121 decodes the encoded coded data for the return and outputs an image after L [msec], as shown in the lowermost stage of FIG. 15.

That is, the time from the start of encoding of the video image for the main line to the start of output of the decoded image of the video image on the return line is (P + T + L) [msec], where the time of T + L is P If shorter, the delay time is shorter than that in FIG.

P [msec] is the sum of the time required for the encoding process and the decode process (the sum of the time until the minimum information required for the encoding process is collected and the time until the minimum information required for the encoding process is collected). L [msec] is a time required for decoding processing (time until the lowest information necessary for decoding processing is collected). In other words, if T [msec] is shorter than the time required for the encoding process, the delay time is shorter than in the case of FIG.

In the encoding process, as described with reference to FIG. 4 and the like, processes such as wavelet transform, coefficient rearrangement and entropy encoding are performed. In the wavelet transform, the division process is recursively repeated, and data is accumulated several times in the calculation buffer unit 211 during this time. The coefficient data obtained by the wavelet transform is held in the coefficient rearrangement buffer unit 212 until at least one line block of data is accumulated. In addition, entropy encoding is performed on the coefficient data. Therefore, the time required for the encoding process is obviously longer than the time for input image data being input for one line block.

In contrast, T [msec] is a time until the data control unit 137 extracts a part of encoded data and starts transmission. For example, if the encoded data on the main line is 150 Mbps and the returned encoded data is 50 Mbps, the supplied 150 Mbps data 50 Mbps data is accumulated from the beginning, and the output is started when 50 Mbps encoded data is accumulated. . The time for cooking selection of this data is T [msec]. In other words, T [msec] is shorter than the time for which 150 Mbps encoded data is supplied for one line block.

Therefore, T [msec] is clearly shorter than the time required for the encoding process, and the delay time from the start of encoding of the video image for the main line to the start of output of the decoded image of the video image on the return line is shown in FIG. In the case of FIG.

As described above, the processing of the data control unit 137 is easy, and the detailed configuration thereof will be described later. However, the circuit configuration can be made smaller in size than in the case of using the encoder as in the prior art shown in FIG. . That is, by applying this data control part 137, the circuit scale and cost of the camera control part 112 can also be reduced.

Next, the internal structure of the data control unit 137 which performs such a process is demonstrated. 16 is a block diagram illustrating a detailed configuration example of the data control unit 137. In FIG. 16, the data control unit 137 includes an accumulated value initialization unit 351, an encoded data acquisition unit 352, a line block determination unit 353, an accumulation value counting unit 354, an accumulation result determination unit 355, An encoded data accumulation control unit 356, a first encoded data output unit 357, a second encoded data output unit 358, and an end determination unit 359 are included. In the figure, the solid arrows indicate the relationships between the blocks including the moving direction of the encoded data, and the dashed arrows indicate the control relationships between the blocks not including the moving direction of the encoded data.

The accumulated value initialization unit 351 initializes the value of the accumulated value 371 counted by the accumulated value counting unit 354. The cumulative value is the sum of the code amounts of the coded data accumulated in the memory unit 301. When the accumulated value initialization unit 351 initializes the accumulated value, the encoded data acquisition unit 352 starts acquisition of the encoded data.

The coded data acquisition unit 352 is controlled by the accumulated value initialization unit 351 and the coded data accumulation control unit 356 to acquire coded data supplied from the video signal decoding unit 136, and the line block determination unit 353. ) To determine the line block.

The line block determination unit 353 determines whether or not the coded data supplied from the coded data acquisition unit 352 is the last coded data of the line block currently being acquired. For example, the depacketizing unit 321 of the video signal decoding unit 136 is supplied with some or all of the header information of the packet together with the encoded data. The line block determination unit 353 determines whether or not the supplied encoded data is the last encoded data of the current line block based on such information. If it is determined that it is not the last encoded data, the line block determination unit 353 supplies the encoded data to the cumulative value counting unit 354, and causes the cumulative value to be counted. On the contrary, when it determines with the last encoded data, the line block determination unit 353 supplies the encoded data to the second encoded data output unit 358 to start output of the encoded data.

The cumulative value counting unit 354 incorporates a storage unit (not shown), and holds a cumulative value that is a variable representing the total sum of the code amounts of the coded data stored in the memory unit 301. When the encoded data is supplied from the line block determination unit 353, the cumulative value counting unit 354 adds the code amount of the encoded data to the cumulative value, and supplies the addition result to the cumulative result determining unit 355.

The cumulative result determination unit 355 determines whether the cumulative value has reached the target code amount corresponding to the bit rate of the predetermined return coded data, and determines that the cumulative value has not reached. 354 is controlled to supply the encoded data to the encoded data accumulation control unit 356, and the encoded data accumulation control unit 356 is controlled to store the encoded data in the memory unit 301. When it is determined that the cumulative value has reached the target code amount, the cumulative result determination unit 355 controls the first coded data output unit 357 to start output of the coded data.

The coded data accumulation control unit 356 acquires the coded data from the accumulated value count unit 354, and supplies the coded data to the memory unit 301 to store the coded data. When the encoded data is accumulated, the encoded data accumulation control unit 356 causes the encoded data acquisition unit 352 to start acquiring new encoded data.

When the first coded data output unit 357 is controlled by the cumulative result determination unit 355, the coded data stored in the memory unit 301 may be used to determine the subbands immediately preceding the subband currently being processed from the first coded data. The encoded data is read out and output to the outside of the data control unit 137. When the coded data is output, the first coded data output unit 357 determines the end of the processing by the end determination unit 359.

When the encoded data is supplied from the line block determination unit 353, the second encoded data output unit 358 reads out all of the encoded data stored in the memory unit 301, and the data control unit 137 reads the encoded data. Output outside of When the coded data is output, the second coded data output unit 358 determines the end of the processing by the end determination unit 359.

The end determination unit 359 determines whether or not the input of the coded data has ended, and when determining that the input of the encoded data has not been terminated, controls the accumulated value initialization unit 351 to initialize the accumulated value 371. In addition, when it determines with being finished, the termination determination part 359 complete | finishes a bit rate conversion process.

Next, a specific example of the flow of the processing executed in each part of FIG. 3 will be described. FIG. 17 is a flowchart for explaining an example of the flow of main processing executed in the entire digital triax system 100 (the transmitting unit 110 and the camera control unit 112).

As shown in FIG. 17, the transmission unit 110 encodes the image data supplied from the video camera unit 113 in step S1, and modulates or signals the encoded data obtained by the encoding in step S2. It amplifies and supplies it to the camera control part 112.

When the camera control unit 112 acquires the coded data in step S21, the camera controller 112 performs a process such as signal amplification and demodulation, further decodes the coded data in step S22, and converts the bit rate of the coded data in step S23. In step S24, the coded data obtained by converting the bit rate is modulated or signal amplified and transmitted to the transmission unit 110.

The transmission unit 110 acquires the coded data in step S3. The transmission unit 110 which acquired the coded data then performs processing such as signal amplification and demodulation, further decodes the coded data, and displays an image on the display unit 151.

In addition, the detailed flow of the encoding process of the image data of step S1, the encoding data decoding process of step S22, and the bit rate conversion process of step S23 is mentioned later. In addition, in the transmission unit 110, you may make each process of step S1 thru | or step S3 perform in parallel with each other. In addition, the camera control unit 112 may similarly perform each of the processes in steps S21 to S24 in parallel with each other.

Next, with reference to the flowchart of FIG. 18, the example of the detailed flow of the encoding process performed by step S1 of FIG. 17 is demonstrated.

When the encoding process starts, the wavelet transform unit 210 sets the number A of the process target line block as the initial setting in step S41. Normally, the number A is set to "1". When the setting is completed, the wavelet converter 210 acquires the image data of the number of lines (that is, one line block) required to generate the first A line from the top in the lowest subband in step S42. The image data is subjected to vertical analysis filtering processing for performing analysis filtering on the image data arranged in the screen vertical direction in step S43, and the analysis filtering process for image data arranged in the screen horizontal direction in step S44. The horizontal analysis filtering process is performed.

In step S45, the wavelet transforming unit 210 determines whether or not the analysis filtering process has been performed to the final level, and if it is determined that the decomposition level has not reached the final level, the process returns to step S43 to determine the current decomposition level. For this, the analysis filtering process of step S43 and step S44 is repeated.

When it is determined in step S45 that the analysis filtering process has been performed to the final level, the wavelet transform unit 210 advances the process to step S46.

In step S46, the coefficient rearrangement unit 213 rearranges the coefficients of the line block A (the A-th line block from the top of the picture (field in the case of the interlace method)) in order from low to high. In step S47, the quantization unit 214 quantizes the rearranged coefficients using predetermined quantization coefficients. In step S48, the entropy coding unit 215 entropy encodes the coefficients for each line. When the entropy encoding ends, the packetizing unit 217 packetizes the coded data of the line block A in step S49, and transmits the packet (coded data of the line block A) to the outside in step S50.

The wavelet transforming unit 210 increments the value "A" in step S51 by "1" and makes the next line block to be processed. If it is determined whether there is an unprocessed image input line, and if it is determined to exist, the process returns to step S42 and subsequent processing is repeated for the new block to be processed.

As described above, the processing of steps S42 to S52 is repeatedly performed to encode each line block. When it is determined in step S52 that there is no unprocessed image input line, the wavelet converter 210 ends the encoding process for the picture. The encoding process is newly started for the next picture.

In this way, the wavelet transforming unit 210 continuously performs vertical analysis filtering processing and horizontal analysis filtering processing to the final level in line block units, and thus, the wavelet converter 210 maintains (buffers) at one time (simultaneously) compared with the conventional method. Since the amount of data required is small, the amount of memory of the buffer to be prepared can be greatly reduced. In addition, by performing an analysis filtering process to the final level, processing such as rearrangement of coefficient rearrangement and entropy encoding can also be performed (that is, coefficient rearrangement or entropy encoding can be performed in line block units). Therefore, the delay time can be significantly reduced as compared with the method of performing wavelet transform on the entire screen.

Next, with reference to the flowchart of FIG. 19, the example of the detailed flow of the decoding process performed by step S22 of FIG. 17 is demonstrated. This decoding process corresponds to the encoding process shown in the flowchart of FIG.

When the decoding process is started, in step S71, the depacketizing unit 321 depackets the packet obtained to obtain encoded data. In step S72, the entropy decoding unit 322 entropy decodes the encoded data for each line. In step S73, the inverse quantization unit 323 performs inverse quantization on coefficient data obtained by entropy decoding. In step S74, the coefficient buffer unit 324 holds the dequantized coefficient data. In step S75, the wavelet inverse transform unit 325 determines whether coefficients for one line block are stored in the coefficient buffer unit 324, and if not, returns the processing to step S71, and thereafter. Processing is executed, and waits until coefficients for one line block are accumulated in the coefficient buffer unit 324.

If it is determined in step S75 that coefficients for one line block are accumulated in the coefficient buffer unit 324, the wavelet inverse transform unit 325 advances the process to step S76, and the coefficients held in the coefficient buffer unit 324 Read for 1 line block.

The wavelet inverse transform unit 325 performs the vertical synthesis filtering process for performing the synthesis filtering process on the coefficients arranged in the screen vertical direction, in step S77, with respect to the read coefficients. A horizontal synthesis filtering process is performed to perform the synthesis filtering process on the coefficients to be arranged, and in step S79, whether the synthesis filtering process has ended to level 1 (the level at which the decomposition level value is "1"), that is, before the wavelet transform If it is determined whether the state has been inversely transformed, and if it is determined that the level 1 has not been reached, the process returns to step S77, and the filtering processes of steps S77 and S78 are repeated.

When it is determined in step S79 that the inverse transform process is completed up to level 1, the wavelet inverse transform unit 325 advances the process to step S80 to output the image data obtained by the inverse transform process to the outside.

In step S81, when the entropy decoding unit 322 determines whether or not the decoding process is to be completed, and inputting the encoded data through the depacketizing unit 321 continues, and determines that the decoding process is not to end, The processing returns to Step S71 and the subsequent processing is repeated. In addition, when it is determined in step S81 that the input of the encoded data is terminated and the decoding process ends, the entropy decoding unit 322 terminates the decoding process.

As described above, the wavelet inverse transform unit 325 continuously performs the vertical synthesis filtering process and the horizontal synthesis filtering process up to level 1 on a line block basis, so that the wavelet inverse transform unit 325 performs the inverse wavelet transform on the entire screen. Since the amount of data that needs to be buffered at a time (at the same time) is small, the amount of memory of the buffer to be prepared can be greatly reduced. In addition, by performing the synthesis filtering process (wavelet inverse transform process) up to level 1, it is possible to sequentially output the image data before the entire image data in the picture is obtained (in line block units), so that the inverse wavelet transform is performed for the entire screen. The delay time can be significantly reduced as compared with the method of performing the step.

Next, with reference to the flowchart of FIG. 20, the example of the flow of the bit rate conversion process performed by step S23 of FIG. 17 is demonstrated.

When the bit rate conversion process is started, the accumulated value initialization unit 351 initializes the value of the accumulated value 371 in step S101. In step S102, the encoded data acquisition unit 352 acquires the encoded data supplied from the video signal decoding unit 136. In step S103, the line block determination unit 353 determines whether or not it is the last encoded data in the line block. If it is determined that it is not the last encoded data, the process proceeds to step S104. In step S104, the cumulative value counting unit 354 counts the cumulative value by adding the cumulative value holding the code amount of the newly obtained encoded data.

In step S105, the cumulative result determination unit 355 determines whether the cumulative result which is the current cumulative value has reached the code amount previously assigned to the line block to be processed, that is, the allocated code amount to be the target code amount of the line block to be processed. Determine whether or not. If it is determined that the allocated code amount has not been reached, the process proceeds to step S106. In step S106, the coded data accumulation control unit 356 supplies the coded data obtained in step S102 to the memory unit 301, and stores the coded data. When the process of step S106 ends, the process returns to step S102.

In addition, when it is determined in step S105 that the cumulative result has reached the assigned code amount, the process proceeds to step S107. In step S107, the first coded data output unit 357 moves from the head subband of the coded data stored in the memory unit 301 to the subband immediately preceding the subband to which the coded data acquired in step S102 belongs. Reads and outputs encoded data. When the process of step S107 is complete | finished, a process progresses to step S109.

If it is determined in step S103 that the coded data obtained by the process of step S102 is the last coded data in the line block, the process proceeds to step S108. In step S108, the second coded data output unit 358 reads out all coded data in the line block to be processed stored in the memory unit 301 and outputs the coded data obtained by the process of step S102. . When the process of step S108 ends, the process proceeds to step S109.

In step S109, the termination determination unit 359 determines whether all line blocks have been processed. If it is determined that there is an unprocessed line block, the process returns to step S101, and subsequent processing is repeated for the next unprocessed line block. In addition, when it is determined in step S109 that all the line blocks have been processed, the bit rate conversion process is terminated.

By performing the bit rate conversion processing as described above, the data control unit 137 can easily convert the bit rate to a desired value without delaying the encoded data without delay. Thereby, in the flowchart of FIG. 17, the digital triax system 100 can easily reduce the delay time from the start of the process of step S1 to the completion of the process of step S3. In this way, it is not necessary to prepare an encoding for the encoded data for return, and the circuit scale and the cost of the camera control unit 112 can be reduced.

In FIG. 4, the rearrangement of the coefficients is performed immediately after the wavelet transform (before quantization). However, the encoded data may be supplied to the video signal decoder 136 in order from low to high frequency (i.e., low-band subbands). The encoded data obtained by encoding the coefficient data belonging to may be supplied in the order of the encoded data obtained by encoding the coefficient data belonging to the high frequency subband), and the timing of rearrangement may be other than immediately after wavelet transform.

For example, the order of the encoded data obtained by entropy encoding may be rearranged. 21 is a block diagram illustrating a configuration example of the video signal encoder 120 in that case.

In the case of FIG. 21, the video signal encoder 120, like the case of FIG. 4, includes a wavelet transform unit 210, an intermediate calculation buffer unit 211, a quantization unit 214, and an entropy encoder 215. Has a rate controller 216 and a packetizer 217, but instead of the coefficient rearrangement buffer unit 212 and the coefficient rearrangement unit 213 of FIG. 4, the code rearrangement buffer unit 401 and the code rearrangement unit Has 402.

The code rearrangement buffer unit 401 is a buffer for rearranging the output order of the encoded data encoded by the entropy encoding unit 215. The code rearrangement unit 402 is the code rearrangement buffer unit 401. By reading the encoded data stored in the data in a predetermined order, the output order of the encoded data is rearranged.

That is, in FIG. 21, the wavelet coefficients output from the wavelet transform unit 210 are supplied to the quantization unit 214 and quantized. The output of the quantizer 214 is supplied to the entropy encoder 215 and encoded. Each encoded data obtained by the encoding is sequentially supplied to the code rearrangement buffer unit 401, and temporarily stored for rearrangement.

The code rearrangement unit 402 reads the coded data written in the code rearrangement buffer unit 401 in a desired order and supplies it to the packetization unit 217.

In the case of the example of FIG. 21, the entropy encoding unit 215 encodes each coefficient data in the output order by the wavelet transform unit 210, and writes the obtained encoded data into the code rearrangement buffer unit 401. . That is, the code rearrangement buffer unit 401 stores the encoded data in the order corresponding to the output order of the wavelet coefficients by the wavelet transform unit 210. In general, when comparing coefficient data belonging to one line block, the wavelet converter 210 outputs coefficient data belonging to a higher subband first, and the coefficient data belonging to a lower subband. Print later. That is, in the code rearrangement buffer unit 401, encoded data obtained by entropy encoding coefficient data belonging to a low frequency subband from encoded data obtained by entropy encoding coefficient data belonging to a high frequency subband. It is remembered in order to face up.

In contrast, the code rearrangement unit 402 rearranges the coded data by reading each coded data stored in the code rearrangement buffer unit 401 in any order independently of this order.

For example, the code rearrangement unit 402 reads out the encoded data obtained by encoding coefficient data belonging to the low frequency subband first, and finally reads the encoded data obtained by encoding coefficient data belonging to the highest frequency subband. Serve In this way, by reading the encoded data from the low range to the high range, the code rearrangement unit 402 allows the video signal decoding unit 136 to decode each encoded data in the order of acquisition, and thereby the video signal decoding unit 136. It is possible to reduce the delay time caused by the decoding processing by).

The code rearrangement unit 402 reads the coded data stored in the code rearrangement buffer unit 401 and supplies it to the packetization unit 217.

The data encoded and output by the video signal encoder 120 shown in FIG. 21 is transferred from the video signal encoder 120 of FIG. 4 by the video signal decoder 136 described above with reference to FIG. 13. The decoding can be performed similarly to the case of output coded data.

In addition, the timing of performing rearrangement may be other than what was mentioned above. For example, as shown in FIG. 22, the video signal encoder 120 may be used. As shown in FIG. 23, the video signal decoder 136 may be used. do.

In the process of rearranging the coefficient data generated by the wavelet transform, a relatively large capacity is required as the storage capacity of the coefficient rearrangement buffer, and high processing capability is also required in the coefficient rearrangement process itself. Even in this case, no problem occurs when the processing capacity of the transmitting unit 110 is higher than a certain degree.

Here, the case where the transmission unit 110 is mounted in the apparatus with relatively low processing capability, such as what is called a mobile terminal, such as a mobile telephone terminal and a PDA (Personal Digital Assistant), is considered. For example, in recent years, the product which added the imaging function with respect to the portable telephone terminal is prevalent widely (it is called the portable telephone terminal with a camera function). It is conceivable to compress and encode image data captured by a mobile phone terminal having such a camera function by wavelet transform and entropy encoding, and to transmit the data via wireless or wired communication.

For example, a mobile terminal has a limited processing capacity of a CPU (Central Processing Unit) and also has a certain upper limit in memory capacity. Therefore, the load of the process accompanying the coefficient rearrangement as mentioned above becomes a problem which cannot be ignored.

Thus, as an example is shown in FIG. 23, by putting the rearrangement process into the camera control unit 112, the load of the transmission unit 110 becomes light, so that the transmission unit 110 can be relatively processed such as a mobile terminal. It becomes possible to mount in this low apparatus.

In the above description, the data amount control is performed on a line block basis. However, the data amount control is performed on a plurality of line blocks, for example. In general, in the case of performing data amount control in units of a plurality of line blocks, the image quality is higher than in the case of performing data amount control in line block units, but the delay time is longer.

Fig. 24 shows how data is counted in order from low to high frequency after buffering each subband in N line blocks (N is an integer). In FIG. 24A, portions indicated by the diagonal lines from the upper right to the lower left indicate respective subbands of the first line block, and portions denoted by the diagonal lines from the lower right to the upper left indicate each sub of the Nth line block. The band is shown.

The data control unit 137 may perform data control in such a manner that N consecutive line blocks are grouped as one group. At this time, the arrangement order of the coded data and N line blocks can be arranged in one group. An example of the arrangement order is shown in FIG. 24B.

As described above, the coded data is supplied to the data control unit 137 in order from the coded data corresponding to the coefficient data belonging to the low subband in line block units to the coded data belonging to the high subband. The data control unit 137 stores the encoded data in the N line blocks and the memory unit 301.

When the data control unit 137 reads the coded data for the N line blocks stored in the memory unit 301, as shown in the example of B of FIG. 24, first, the first line block to the Nth line are shown. The encoded data (1LLL, 2LLL, ..., NLLL) of the subband LLL of the lowest region (level 1) of the line block is read, and then the encoded data (1LHL, of the subband LHL of the first to Nth line blocks) is read. 2LHL, ..., NLHL), the encoded data (1LLH, 2LLH, ..., NLLH) of the subband LLH of the first line block to the Nth line block is read, and the sublines of the first line block to the Nth line block. The coded data of the band LHH (1LHH, 2LHH, ..., NLHH) is read.

When reading of the level 1 encoded data is finished, the data control unit 137 reads the coded data of level 2 above one next. That is, the data control unit 137 reads the encoded data (1HL, 2HL, ..., NHL) of the subband HL at the level 2 of the first line block to the Nth line block, as shown in B of FIG. Next, the encoded data (1LH, 2LH, ..., NLH) of the subband LH of the first line block to the Nth line block is read, and the encoded data of the subband HH of the first line block to the Nth line block ( 1HH, 2HH…, NHH).

As described above, the data control unit 137 sets N line blocks into one group, and reads encoded data of each line block in the group in order from the lowest subband to the highest subband in parallel. Serve

That is, the data control unit 137 stores the encoded data stored in the memory unit 301 (1LLL, 2LLL, ..., NLLL, 1LHL, 2LHL, ..., NLHL, 1LLH, 2LLH, ..., NLLH, 1LHH, 2LHH, ..., NLHH, 1HL, 2HL, ..., NHL, 1LH, 2LH, ..., NLH, 1HH, 2HH, ..., NHH, ...), and output.

The data control unit 137 reads the coded data of the N line blocks, counts the total of the code amounts, and when the target code amount is reached, ends the reading and discards the subsequent data. And when the process with respect to the N line blocks is complete | finished, the data control part 137 performs the same process with respect to the next N line blocks. In other words, the data control unit 137 controls the code amount for each of the N line blocks (converts the bit rate).

In this way, by controlling the code amount for each of the N line blocks, it is possible to reduce the difference in the image quality between the line blocks and to suppress a significant decrease in the local resolution in the display image, thereby improving the image quality of the display image. have.

25 shows an example in which the read order of encoded data is different. As shown in A of FIG. 25, the data control unit 137 processes the coded data for each of N line blocks (where N is an integer) as in the case of FIG. 24. In other words, also in this case, the data control unit 137 performs data control using N consecutive line blocks as one group. At this time, the arrangement order of the coded data and N line blocks can be arranged in one group. An example of the arrangement order is shown in FIG. 25B.

As described above, the coded data is supplied to the data control unit 137 in line block units from the coded data corresponding to the coefficient data belonging to the low band to the coded data belonging to the high band. The data control unit 137 stores the encoded data in the N line blocks and the memory unit 301.

When the data control unit 137 reads the coded data for the N line blocks stored in the memory unit 301, the data control unit 137 first performs the first line block to the Nth line block as in the case of B of FIG. The encoded data (1LLL, 2LLL ..., NLLL) of the subband LLL of the lowest band (level 1) is read.

Here, unlike in the case of B of FIG. 24, the data control unit 137 reads the encoded data LHL, LLH, and LHH of the remaining subbands of level 1 for each line block, as shown in B of FIG. 25. . That is, the data control unit 137 reads the coded data of the subband LLL, reads the coded data 1LHL, 1LLH, and 1LHH of the remaining subbands of level 1 of the first line block, and then reads the second line block. Similarly, the encoded data (2LHL, 2LLH, 2LHH) is read in the same manner, and then repeated until the encoded data (NLHL, NLLH, NLHH) of the Nth line block is read.

In the same order as described above, when all encoded data is read for the level 1 subbands of the first line block to the Nth line block, the data control unit 137 reads the coded data of the level 2 above one next. Make a bet. At this time, the data control unit 137 reads the coded data HL, LH, and HH of the remaining subbands of level 2 for each line block. That is, the data control unit 137 reads the encoded data 1HL, 1LH, 1HH of the remaining subbands of level 2 of the first line block, and then similarly encodes the data 2HL, 2LH, 2HH with respect to the second line block. ), And then repeats until the encoded data (NHL, NLH, NHH) of the Nth line block is read.

Similarly, the data control unit 137 reads the encoded data up to the subband of the highest band in the same order as described above.

That is, the data control unit 137 stores the encoded data stored in the memory unit 301 (1LLL, 2LLL, ..., NLLL, 1LHL, 1LLH, 1LHH, 2LHL, 2LLH, 2LHH, ..., NLHL, NLLH, NLHH, 1HL, 1LH, 1HH, 2HL, 2LH, 2HH, ..., NHL, NLH, NHH, ...

The data control unit 137 reads the coded data of the N line blocks, counts the total of the code amounts, and when the target code amount is reached, ends the reading and discards the subsequent data. And when the process with respect to the N line blocks is complete | finished, the data control part 137 performs the same process with respect to the next N line blocks. In other words, the data control unit 137 controls the code amount for each of the N line blocks (converts the bit rate).

By doing in this way, the bias of allocation for every subband can be suppressed, the visual discomfort of a display image can be reduced, and an image quality can be improved.

FIG. 26 shows a detailed configuration example of the data control unit 137 when the bit rate is changed for every N line blocks as described with reference to FIGS. 24 and 25.

In FIG. 26, the data control unit 137 includes an accumulated value initialization unit 451, an encoded data acquisition unit 452, an encoded data accumulation control unit 453, an accumulation determination unit 454, an encoded data reading unit 455, The group determining unit 456, the cumulative value counting unit 457, the cumulative result determining unit 458, the first coded data output unit 459, the second coded data output unit 460, and the end determination unit 461 are provided. Have

The accumulated value initialization unit 451 initializes the value of the accumulated value 481 counted by the accumulated value counting unit 457. When the accumulated value 481 is initialized, the accumulated value initialization unit 451 causes the encoded data acquisition unit 452 to start the acquisition of the encoded data.

The coded data acquisition unit 452 is controlled by the accumulated value initialization unit 451 and the accumulation determination unit 454 to acquire coded data supplied from the video signal decoding unit 136, and the coded data accumulation control unit 453 is obtained. To store the encoded data. The encoded data accumulation control unit 453 accumulates the encoded data supplied from the encoded data acquisition unit 452 in the memory unit 301, and notifies the accumulation determination unit 454 of the fact. The accumulation determination unit 454 determines whether or not the coded data for N line blocks has been accumulated in the memory unit 301 based on the notification from the encoded data accumulation control unit 453. When it is determined that the coded data for N line blocks has not been accumulated, the accumulation judgment unit 454 controls the coded data acquisition unit 452 to acquire new coded data. In addition, when it is determined that the coded data for N line blocks has been accumulated in the memory unit 301, the accumulation determination unit 454 controls the coded data reading unit 455 to store the coded data in the memory unit 301. Starts reading the encoded data.

The encoded data reading unit 455 is controlled by the accumulation determining unit 454 or the cumulative result determining unit 458 to read the encoded data stored in the memory unit 301, and the read encoded data is grouped. Supply to the government (456). At this time, the coded data reading unit 455 sets the coded data for N line blocks into one group, and reads the coded data for each group in a predetermined order. That is, when the coded data accumulation control section 453 stores one group of coded data in the memory unit 301, the coded data reading unit 455 sets the group as a processing target and selects the coded data of the group. Read in order.

The group determining unit 456 determines whether or not the encoded data read by the encoded data reading unit 455 is the last data of the last line block in the group to be processed. When it is determined that the supplied encoded data is not the encoded data read last from the group to which the encoded data belongs, the group determining unit 456 supplies the supplied encoded data to the accumulated value counting unit 457. . Further, when it is determined that the supplied encoded data is the encoded data read last from the group to which the encoded data belongs, the group determination unit 456 controls the second encoded data output unit 460.

The cumulative value counting unit 457 incorporates a storage unit (not shown), which counts the total of the code amounts of the coded data supplied from the group determining unit 456, and stores the counted value in the storage unit in the accumulated value 481. ), And the accumulated value 481 is supplied to the cumulative result determining unit 458.

The cumulative result determination unit 458 determines whether or not the cumulative value 481 reaches the target code amount corresponding to the bit rate of the coded data for return, and determines that the cumulative value 481 has not reached. The data reading unit 455 is controlled to read new encoded data. Further, when it is determined that the cumulative value 481 has reached the target code amount assigned to the group, the cumulative result determination unit 458 controls the first coded data output unit 459.

The first coded data output unit 459 is controlled by the cumulative result determination unit 458 to read all coded data belonging to the subband from the beginning to the previous of the coded data belonging to the group to be processed, and then the data control unit 137. Output to outside of

As described with reference to B in FIG. 24 or B in FIG. 25, the encoded data stored in the memory unit 301 is read out in units of subbands of each line block. Thus, for example, when it is determined that the accumulated value 481 has reached the target code amount when the encoded data belonging to the m-th subband to be read is read out, the first encoded data output unit 459 stores the memory. The coded data belonging to the subbands read first to the (m-1) th is read from the unit 301 and output to the outside of the data control unit 137.

When the coded data is output, the first coded data output unit 459 determines the end of the processing by the end determination unit 461.

The second coded data output unit 460 is controlled by the group determining unit 456 to read all the coded data of the group to which the coded data read by the coded data reader 455 belongs and the data control unit 137. Output to outside of When the coded data is output, the second coded data output unit 460 determines the end of the processing by the end determination unit 461.

The end determination unit 461 determines whether or not the input of the coded data is terminated. If it is determined that the input of the encoded data has not been terminated, the end determination unit 461 controls the accumulated value initialization unit 451 to initialize the accumulated value 481. In addition, when it determines with completion, the termination determination part 461 complete | finishes a bit rate conversion process.

Next, with reference to the flowchart of FIG. 27, the example of the flow of the bit rate conversion process by the data control part 137 shown in this FIG. 26 is demonstrated. This bit rate conversion process is a process corresponding to the bit rate conversion process shown in the flowchart of FIG. In addition, processing other than this bit rate conversion process is performed similarly to the case demonstrated with reference to FIGS. 17-19.

When the bit rate conversion process is started, the accumulated value initialization unit 451 initializes the value of the accumulated value 481 in step S131. In step S132, the encoded data acquisition unit 452 acquires the encoded data supplied from the video signal decoding unit 136. In step S133, the coded data accumulation control unit 453 stores the coded data obtained in step S132 in the memory unit 301. In step S134, the accumulation determination unit 454 determines whether or not the coded data for N line blocks has been accumulated. If it is determined in the memory unit 301 that N coded data has not been accumulated, the process returns to step S132, and the process thereafter is repeated. When it is determined in step S134 that the coded data for N line blocks has been accumulated in the memory unit 301, the process proceeds to step S135.

When the coded data for N line blocks is stored in the memory unit 301, in step S135, the coded data reading unit 455 sets the accumulated N line blocks into one group, and sets the grouped coded data as a predetermined group. Read in order.

In step S136, the group determining unit 456 determines whether the coded data read in step S135 is the coded data last read out of the group to be processed. If it is determined that it is not the last encoded data of the group to be processed, the process proceeds to step S137.

In step S137, the cumulative value counting unit 457 adds to the cumulative value 481 holding the code amount of the encoded data obtained in step S132, and counts the cumulative value. In step S138, the cumulative result determining unit 458 determines whether the cumulative result has reached the target code amount (allocation code amount) assigned to the group. If it is determined that the cumulative result has not reached the allocated code amount, the process returns to step S135 and the process after step S135 is repeated for the next new coded data.

In addition, when it is determined in step S138 that the cumulative result has reached the assigned code amount, the process proceeds to step S139. In step S139, the first encoded data output unit 459 reads out the encoded data up to the immediately preceding subband from the memory unit 301 and outputs it. When the process of step S139 ends, the process proceeds to step S141.

In addition, when it is determined in step S136 that the last encoded data in the group has been read, the process proceeds to step S140. In step S140, the second encoded data output unit 460 reads out all the encoded data in the group from the memory unit 301 and outputs it. When the process of step S140 is complete | finished, a process progresses to step S141.

In step S141, the termination determination unit 461 determines whether all line blocks have been processed. If it is determined that there is an unprocessed line block, the process returns to step S131, and subsequent processing is repeated for the next unprocessed line block. In addition, when it is determined in step S141 that all the line blocks have been processed, the bit rate conversion process is terminated.

By performing the bit rate conversion process as described above, the data control unit 137 can improve the image quality of the image obtained from the data after the bit rate conversion.

In FIG. 3, although the digital triax system 100 was demonstrated so that it may be comprised by one transmission unit 110 and one camera control part 112, the number of transmission units and a camera control part may each be sufficient.

FIG. 28 is a diagram illustrating another configuration example of the digital triax system to which the present invention is applied. In FIG. 28, the digital triax system is a system having X cameras (X is an integer) of camera heads (camera heads 511-1 to 511-X) and one camera control unit 512. 3 is a system corresponding to the digital triax system 100 of FIG.

In the digital triax system 100 of FIG. 3, the digital triax system of FIG. 28 is controlled by one camera control unit 112 and one transmission unit 110 (video camera unit 113). In the above, a plurality of camera heads (camera heads 511-1 to 511-X) are controlled by one camera control unit 512. That is, the camera heads 511-1 to 511-X correspond to the transmission unit 110 of FIG. 3, and the camera control unit 512 corresponds to the camera control unit 112.

The camera head 511-1 has a camera portion 521-1, an encoder 522-1, and a decoder 523-1, and captures image data (video) obtained by the camera portion 521-1. The encoder 522-1 encodes the encoded data, and supplies the encoded data to the camera control unit 512 through the main line D510-1, which is one system of the transmission cable. In addition, the camera head 511-1 decodes the encoded data supplied from the camera control unit 512 via the return line D513-1 by the decoder 523-1, and stores the obtained moving image for the return image. It displays in the return view 51-1 which is a display.

The camera heads 511-2 to 511-X also have the same configuration as the camera head 511-1, and perform the same processing. For example, the camera head 511-2 has a camera portion 521-2, an encoder 522-2, and a decoder 523-2, and image data obtained by photographing the camera portion 521-2. The (video) is encoded by the encoder 522-2, and the encoded data is supplied to the camera control unit 512 via the main line D510-2, which is one system of the transmission cable. In addition, the camera head 511-2 decodes the encoded data supplied from the camera control unit 512 via the return line D513-2 by the decoder 523-2, and stores the obtained moving image for the return image. It displays in the return view 531-2 which is a display.

The camera head 511-X also has a camera portion 521-X, an encoder 522-X, and a decoder 523-X, and image data obtained by shooting with the camera portion 521-X (video) Is encoded by the encoder 522-X, and the encoded data is supplied to the camera control unit 512 via the main line D510-X, which is one system of the transmission cable. In addition, the camera head 511-X decodes the encoded data supplied from the camera control unit 512 via the return line D513-X by the decoder 523-X, and stores the obtained moving image for the return image. It is displayed in the return view 531-X which is a display.

The camera control unit 512 has a switch unit (SW) 541, a decoder 542, a data control unit 543, a memory unit 544, and a switch unit (SW) 545. The coded data supplied through the main line D510-1 to the main line D510-X is supplied to the switch unit (SW) 541. The switch unit (SW) 541 selects a part of them and supplies the coded data supplied through the selected line to the decoder 542. The decoder 542 decodes the encoded data, and supplies the decoded video data to the main view 546, which is a display for the main line video, via a cable D511 to display an image.

Also, in order to confirm to the user of the camera head whether or not the camera control unit 512 has received an image transmitted from each camera head, the image data is resent to the camera head as a return video image. In general, the bandwidths of the return lines D513-1 to D513-X that transmit this return video image are narrower than those of the main lines D510-1 to D510-X.

Therefore, the camera control unit 512 supplies the coded data before being decoded by the decoder 542 to the data control unit 543, and converts the bit rate into a desired value. As in the case described with reference to FIG. 16 and the like, the data control unit 543 converts the bit rate of the supplied encoded data into a desired value by using the memory unit 544, and switches the encoded data after the bit rate conversion. The part (SW) 545 is supplied. In addition, the description about packetization is abbreviate | omitted here for simplicity of description. In other words, the packetizing unit (corresponding to the packetizing unit 302) for packetizing the encoded data for return will be described as being included in the data control unit 543.

The switch unit (SW) 545 connects a part of the lines among the return lines D513-1 to D513-X to the data control unit 543. That is, the switch unit (SW) 545 controls the transmission destination of the encoded data for return. For example, the switch unit (SW) 545 connects the return line connected to the camera head of the source of the encoded data to the data control unit 543, and returns the encoded data for the return to the camera head of the source of the encoded data. It is supplied as a return video image.

The camera head which acquired the coded data (return video video) decodes by the built-in decoder, supplies the decoded video data to a return view, and displays an image. For example, when the encoded data for return is supplied from the switch unit (SW) 545 to the camera head 511-1 through the return line D513-1, the decoder 523-1 is provided with the encoded data. Is decoded and supplied to the return view 531-1, which is a display for the return image, via the cable D514-1 to display an image.

The same applies to the case where the coded data is transmitted to the camera heads 511-2 to 511-X. In addition, below, when it is not necessary to distinguish camera head 511-1-camera head 511-X from each other, it is simply called camera head 511. FIG. Similarly, when the camera portions 521-1 to 521-X do not need to be distinguished from one another, the camera portions 521 are simply referred to as the encoders 522-1 to 522-X. Are simply referred to as the encoder 522 when it is not necessary to describe them separately, and simply referred to as decoder 523 when it is not necessary to describe the decoders 523-1 to 523-X separately. When the main line D510-1 to the main line D510-X do not need to be distinguished from each other and explained, the main line D510 is simply referred to as the main line D510-1 and the return line D513-1 to D513-X are mutually different from each other. When it is not necessary to describe separately, it is simply called the return line D513, and when it is not necessary to describe the return views 531-1 to 531-X separately from each other, the return view 531 is simply described. It is called.

As described above, the camera control unit 512 shown in FIG. 28 has the same configuration as the camera control unit 112 shown in FIG. 3, and the switch unit (SW) 541 and the switch unit (SW). By passing the coded data through 545), the camera head 511 that is the opponent of the coded data can be selected. That is, the user of the camera head 511 selected as the control target by the camera control unit 512, that is, the photographer, shoots, and the captured image is on the camera control unit 512 side (main view 546). You can see how it is displayed in.

Even in a system for controlling the plurality of camera heads 511 in this manner, the camera control unit 512 can easily control the bit rate of the return moving image data using the data control unit 543, thereby reducing the encoded data. You can send with a delay.

In the conventional digital triax system shown in FIG. 29, the camera control unit 561 has an encoder 562 instead of the data control unit 543, and the moving picture data obtained by being decoded by the decoder 542 is used for this encoder. It is encoded again by 562 and outputted. Therefore, the camera control unit 512 shown in FIG. 28 can convert the bit rate of the moving picture data into a desired value more easily than the camera control unit 561 shown in FIG. 29, and transmit the encoded data with low latency. Can be.

That is, in the case of the system of FIG. 28, since the delay time between the shooting and the returning moving image is displayed in the return view is shorter than that of the system of FIG. 29, the photographer who is the user of the camera head 511 The return video can be confirmed with low latency. Therefore, the photographer can easily perform the shooting operation while checking the return moving image. In particular, when the camera control unit 512 controls the plurality of camera heads 511 as in the case of the digital triax system shown in FIG. 28, switching of control objects occurs, so that the interval of the switching is If the delay time from the shooting until the return moving image is displayed is too long, most of the photographers may have to shoot without checking the moving image. That is, as shown in FIG. 28, it is more important that the camera control unit 512 shorten this delay time by easily controlling the bit rate of the encoded data.

In addition, the camera control unit 512 may control the plurality of camera heads 511 simultaneously. In that case, the camera control unit 512 may transmit the encoded data of each moving picture supplied from each camera head 511, that is, the different encoded data, to each of the supply sources, and is supplied from each camera head 511. The coded data of one video, i.e., the common coded data, which simultaneously display the respective moving images may be transmitted to all the sources.

In addition, as shown in FIG. 30, instead of the camera control unit 561, a camera control unit 581 having both the data control unit 543 and the encoder 562 may be used. The camera control unit 581 arbitrarily selects either the data control unit 543 or the encoder 562 and uses it for generation of encoded data for return. For example, the camera control unit 581 selects the data control unit 543 when the bit rate of the encoded data for return is lower than the bit rate of the encoded data on the main line, and supplies the encoded data before decoding to convert the bit rate. By doing so, the bit rate can be changed easily and at high speed. For example, the camera control unit 581 selects an encoder when the bit rate of the encoded data for return is higher than the bit rate of the encoded data of the main line, and then selects an encoder and supplies the decoded moving image data to convert the bit rate. The bit rate can be converted easily.

Such a digital triax system is used, for example, in broadcasting stations, or in relaying events such as sports or concerts. It can also be used as a system for centrally managing surveillance cameras installed in facilities.

In addition, the above-described data control unit may be applied to any system or device, and for example, the data control unit may be a unit. In other words, it may function as a bit rate converter. For example, in the image coding apparatus which encodes image data, a data control part may control the output bit rate of the coding part which performs an encoding process. For example, in the image decoding apparatus which decodes the encoded data in which image data is encoded, the data control unit may control the input bit rate of the decoding unit which performs the decoding process.

For example, as shown in FIG. 31, you may make it apply to the system which mutually transmits and receives a return image with the communication apparatuses which receive the image data of the main boat.

In the communication system shown in FIG. 31, the communication device 601 and the communication device 602 carry moving picture data. The communication device 601 supplies the moving picture data obtained by the camera 611 to the communication device 602 as the moving picture data of the main ship, and the moving picture data of the main ship supplied from the communication device 602 and the communication device ( 601) Returning moving picture data corresponding to the moving picture data of the main ship supplied by the ship is obtained, and those images are displayed on the monitor 612.

The communication device 601 includes an encoder 621, a main line decoder 622, a data control unit 623, and a return decoder 624. The communication device 601 encodes the moving picture data supplied from the camera 611, and supplies the obtained encoded data to the communication device 602. In addition, the communication device 601 decodes the coded data for the main ship supplied from the communication device 602 by the main line decoder 622 and displays the image on the monitor 612. In addition, the communication device 601 converts the bit rate of the encoded data before decoding supplied from the communication device 602 and supplies it to the communication device 602 as encoded data for return. In addition, the communication device 601 acquires the encoded data for return supplied from the communication device 602, decodes it in the return decoder 624, and displays the image on the monitor 612.

Similarly, the communication device 602 has an encoder 641, a main line decoder 642, a data control unit 643, and a return decoder 644. The communication device 602 encodes the moving image data supplied from the camera 631, and supplies the obtained encoded data to the communication device 601. In addition, the communication device 602 decodes the coded data for the main ship supplied from the communication device 601 by the main line decoder 622 and displays the image on the monitor 632. In addition, the communication device 602 converts the bit rate of the encoded data before decoding supplied from the communication device 601 and supplies it to the communication device 601 as encoded data for return. In addition, the communication device 602 obtains the return coded data supplied from the communication device 601, decodes it in the return decoder 644, and displays the image on the monitor 632.

The encoder 621 and the encoder 641 correspond to the video signal encoder 120 of FIG. 3, and the main line decoder 622 and the main line decoder 642 include the video signal decoder 136 of FIG. 3. ), The data control unit 623 and the data control unit 643 correspond to the data control unit 137 of FIG. 3, and the return decoder 624 and the return decoder 644 are the video signals of FIG. 3. It corresponds to the decoder 121.

That is, the communication apparatus 601 and the communication apparatus 602 both have the structure and the function of both the transmission unit 110 and the camera control part 112 of FIG. 3, and mutually cameras (camera 611 or The encoded data of the captured image obtained by the camera 631 is supplied to the counterpart, and the moving picture for the main ship, which is a captured image taken by the counterpart camera supplied from the counterpart, and the captured image transmitted by the self to the counterpart Obtains the encoded data of the moving picture for the dragon.

At this time, the communication apparatus 601 and the communication apparatus 602 use the data control part 623 or the data control part 643 similarly to the case of FIG. 3, making it easy and fast the bit rate of the encoded data for return. Can be controlled so that the encoded data for return can be transmitted more slowly.

In addition, arrows between the communication device 601, the communication device 602, the camera 611, the monitor 612, the camera 631, and the monitor 632 indicate the direction of data transmission, respectively, and the bus (or Cable) does not represent itself. That is, the number of buses (or cables) between the devices is arbitrary.

32 shows an example of displaying an image on the monitor 612 or the monitor 632. On the display screen 651 shown in FIG. 32, in addition to the moving picture 661 of the communication partner photographed by the camera 631, the own moving picture 662 taken by the camera 611 and the return moving picture 663. Is displayed. The moving image 662 is a moving image supplied to the communication apparatus as the communication partner for the main line, and the moving image 663 is a return moving image corresponding to the moving image 662. That is, the moving image 663 is an image for confirming how the moving image 662 is displayed on the monitor of the communication partner.

Therefore, the user at the communication device 601 side uses the camera 611 and the monitor 612, and the user at the communication device 602 side uses the camera 631 and the monitor 632 to communicate with each other (movie image). Can be performed). In addition, the voice is omitted for simplicity of explanation. Thereby, each user can see an image as shown in the example in Fig. 32, so that not only the photographed image on the other side, but also the photographed image photographed by the camera on his side and how the photographed image is displayed on the other side. The image for confirmation can be viewed simultaneously.

The moving picture 662 and the moving picture 663 are moving pictures having the same contents, but as described above, the moving picture data is compressed and transmitted in communication between communication devices. Therefore, in normal cases, the image (motion image 663) displayed on the other side is deteriorated in image quality than at the time of photographing (motion image 662), and the visible state may be different. There is a fear that the dialogue between them will not be established. For example, the picture which can be confirmed by the moving image 662 cannot be confirmed by the moving image 663, and there exists a possibility that users may not talk based on the image. Therefore, it becomes very important to be able to confirm how the moving picture is displayed on the other side.

At that time, if a long delay time occurs until the display of the moving image for confirmation (that is, if the delay time between the moving image 662 and the moving image 663 is too long), the user confirms the image and the conversation ( It is difficult to make a call). Therefore, it is more important that the communication device 601 and the communication device 602 can transmit the encoded data for return with a low delay even if it is necessary to conduct a call while confirming the moving image 663.

In addition, since the control of the encoded data for return can be easily performed, the band required for transmission of the encoded data for return can be easily reduced. That is, for example, it is possible to transmit the encoded data for return at an appropriate bit rate in accordance with the bandwidth limitation of the transmission path, the circumstances of the display screen on the monitor, and the like. Also in such a case, the encoded data can be transmitted with low latency.

Such a system can be used, for example, in a television conference system for receiving moving images between conference rooms separated from each other, or a telemedicine system for allowing a doctor to examine a remote patient. As described above, the system shown in Fig. 31 can transmit the encoded data for return in a low delay, so that, for example, the presentation and the instruction can be efficiently performed or the examination can be performed correctly.

In the above description, when the data control unit 137 controls the bit rate of the encoded data, the data control unit 137 has been described so as to count the code amount. However, for example, the video signal encoding unit 120, which is an encoder, transmits the data. The coded data may be marked by a predetermined method at a position where the target code amount corresponding to the bit rate after conversion is reached. That is, the video signal encoder 120 determines the code stream break point in the data controller 137. In this case, the data control unit 137 can easily specify the code stream breakpoint only by detecting the marking position. In other words, the data control unit 137 may omit the count of the code amount. This marking may be performed by any method. For example, flag information for indicating the location of the code stream breakpoint may be set in the header of the packet. Of course, other methods may be sufficient.

In the above description, the data control unit 137 explained that the encoded data is temporarily accumulated, but the data control unit 137 counts the code amount of the acquired coded data and outputs coded data for the required code amount. It is not necessary to temporarily store coded data. For example, the data control unit 137 acquires the coded data supplied sequentially from the low frequency component, counts the code amount of the acquired coded data, and outputs the coded data, and the count value is the target code amount. The output of the coded data may be stopped at the point of arrival of.

In each of the systems described above, the data transmission path such as a bus or a network may be wired or wireless.

As described above, the present invention can be applied in various forms, and it is also a great effect to be easily applied to various applications (that is, high in versatility).

By the way, in the above-mentioned digital triax system, orthogonal frequency division multiplexing (OFDM) is used for data transmission via a triax cable (coaxial cable). OFDM is a digital modulation scheme, in which a plurality of carriers are tightly arranged so as not to interfere with each other using orthogonality, and data is transmitted in parallel on a frequency axis. By using orthogonality, OFDM can improve frequency utilization efficiency, thereby realizing wideband transmission using a narrow frequency range efficiently. In the above-described digital triax system, a plurality of such OFDMs are used to perform data transmission by frequency-multiplexing each modulated signal, thereby realizing data transmission at a larger capacity.

33 shows an example of a frequency distribution of data transmitted in a digital triax system. As described above, the transmitted data is modulated in different frequency bands by a plurality of OFDM modulators. Therefore, the modulated data is divided into a plurality of OFDM channels (OFDM channel 1001, OFDM channel 1002, OFDM channel 1003, OFDM channel 1004, ...), which are different in band from each other, as shown in FIG. Distributed in. Arrow 1001A in FIG. 33 indicates the center of the band of the OFDM channel 1001. Similarly, arrows 1002A to 1004A represent centers of bands of the OFDM channels 1002 to OFDM channels 1004, respectively. The frequencies of arrows 1001A to 1004A (center of each OFDM channel) and the bandwidth of each OFDM channel are determined so as not to overlap each other in advance.

As described above, in a digital triax system, data is transmitted in a plurality of bands. However, in the case of data transmission on a triax cable, for example, the cable length, thickness, material of the triax cable, and the like, There is a characteristic that profit is easy to attenuate.

The graph shown in FIG. 34 shows an example of the attenuation of the gain by the cable length in the triax cable. In the graph of Fig. 34, the line 1011 shows a state of gain for each frequency when the cable length of the triax cable is short, and the line 1012 shows a gain for each frequency when the cable length of the triax cable is long. It shows the state. As shown by line 1011, when the cable length is short, the gain of the high pass component is approximately equal to the gain of the low pass component. On the other hand, as shown by the line 1012, when the cable length is long, the gain of the high frequency component is small compared with the gain of the low frequency component.

That is, when the cable length is long, the attenuation rate of the high frequency component is larger than that of the low frequency component, and the increase in the noise component increases the error rate of the symbol in data transmission, and as a result, the error occurrence rate in the decoding process may be high. In the digital triax system, since one data is allocated to a plurality of OFDM channels, if the decoding process fails in the high frequency component, there is a fear that the entire image cannot be correctly decoded (that is, the decoded image is degraded).

In the digital triax system, as described above, low-delay data transmission is required. Therefore, it is virtually impossible to reduce the symbol error rate by retransmission control, redundancy data buffering, or the like.

Therefore, in order to avoid the failure of the decoding process, it is necessary to increase the allocation amount of the error correction bits, lower the transmission rate, and perform data transmission more stably. If the gain is obtained, if the rate control is performed in accordance with the high frequency component, there is a risk of unnecessarily reducing the transmission efficiency. As described above, since the low-delay data transmission is required in the digital triax system, the higher the efficiency of data transmission is, the better.

Therefore, the OFDM control for rate control may be divided between the high frequency side and the low frequency side. 35 is a block diagram illustrating an exemplary configuration of a digital triax system in that case. The digital triax system 1100 illustrated in FIG. 35 is basically the same system as the digital triax system 100 illustrated in FIG. 3, and has the same configuration as the digital triax system 100. In Fig. 35, only parts necessary for explanation are shown.

The digital triax system 1100 has a transmission unit 1110 and a camera control unit 1112 connected to each other by a triax cable 1111. The transmission unit 1110 basically has the same configuration as the transmission unit 110 of FIG. 3, and the triax cable 1111 is basically the same coaxial cable as the triax cable 111 of FIG. 1112 basically has the same configuration as the camera control unit 112 of FIG. 3.

In FIG. 35, for convenience of description, the video unit supplied from the video camera unit (not shown) is encoded and modulated by the OFDM scheme, and the modulation signal is transmitted to the camera control unit via the triax cable 1111. Only the configuration related to the operation of transmitting to 1112, demodulating and decoding the modulated signal received by the camera control unit 1112, and outputting to the system of a later stage is shown.

That is, the transmission unit 1110 is the same as the video signal encoder 1120 of the video signal encoder 120 of the transmission unit 110 and the digital modulator 122 similar to the digital modulator 122 of the transmission unit 110. 1122, an amplifier 1124 similar to the amplifier 124 of the transmission unit 110 and a video separation / synthesis unit 1126 similar to the video separation / synthesis unit 126 of the transmission unit 110 are provided.

The video signal encoder 1120 compresses and encodes a video signal supplied from a video camera unit (not shown) in the same manner as the video signal encoder 120 described with reference to FIG. 4 and the like, and encodes the encoded data (encoded stream). ) Is supplied to the digital modulator 1122.

As shown in FIG. 35, the digital modulator 1122 has a low pass modulator 1201 and a high pass modulator 1202, and modulates the coded data in an OFDM scheme in two frequency bands, a low pass and a high pass ( Hereinafter, modulation by the OFDM method is called "OFDM." That is, the digital modulator 1122 divides the coded data supplied from the video signal encoder 1120 into two, using the low pass modulator 1201 and the high pass modulator 1202, with reference to FIG. 33. As described, each is modulated in a different band (OFDM channel) (of course, the low pass modulator 1201 performs OFDM at a lower frequency than the high pass modulator 1202).

Here, for the sake of convenience, the digital modulator 1122 has two modulators (low pass modulator 1201 and high pass modulator 1202) and is described as performing modulation on two OFDM channels. The number of modulators (that is, the number of OFDM channels) that the digital modulator 1122 has may be any number, as long as the number is modifiable.

The low pass modulator 1201 and the high pass modulator 1202 respectively supply modulation signals with OFDM coded data to the amplifier 1124.

The amplifier 1124 amplifies and modulates these modulated signals by frequency multiplexing as shown in FIG. 33, and supplies them to the video separation / synthesis unit 1126. The video separation / combining unit 1126 synthesizes a modulated signal of the supplied video signal with another signal transmitted together with the modulated signal, and combines the synthesized signal with the camera control unit 1112 through the triax cable 1111. Send to

The video signal thus OFDM is transmitted to the camera controller 1112 via the triax cable 1111.

The camera controller 1112 is a video separation / synthesis unit 1130 similar to the video separation / synthesis unit 130 of the camera control unit 112, an amplifier 1131 similar to the amplifier 131 of the camera control unit 112, The front end section 1133 similar to the front end section 133 of the camera control section 112, the digital demodulation section 1134 and the camera control section 112 similar to the digital demodulation section 134 of the camera control section 112, respectively. It has a video signal decoder 1136 similar to the video signal decoder 136.

When the video separation / synthesizing unit 1130 receives the signal transmitted from the transmission unit 1110, the video separation / combining unit 1130 separates and extracts a modulated signal of the video signal from the signal and supplies the same to the amplifier 1131. The amplifier 1131 amplifies the signal and supplies it to the front end portion 1133. The front end part 1133 has a gain control part which adjusts the gain of an input signal similarly to the front end part 133, and the filter part which performs predetermined filter process with respect to an input signal, and supplied from the amplifier 1131. Gain modulation, filter processing, or the like is performed on the modulated signal, and the processed signal is supplied to the digital demodulation unit 1134.

As shown in FIG. 35, the digital demodulator 1134 has a low pass demodulator 1301 and a high pass demodulator 1302, and using the low pass demodulator 1301 and the high pass demodulator 1302, Modulated signals OFDM in two frequency bands (OFDM channels) of low and high frequencies are demodulated by the OFDM scheme in each band (of course, the low frequency demodulator 1301 is a lower frequency OFDM channel than the high frequency demodulator 1302). Demodulate the modulated signal).

In addition, although the digital demodulator 1134 has two demodulators (low pass demodulator 1301 and high pass demodulator 1302) and demodulates on two OFDM channels, the digital demodulator 1134 is described. The number of demodulators (that is, the number of OFDM channels) of the? May be any number if it is equal to the number of modulators (that is, the number of OFDM channels) of the digital modulator 1122.

The low pass demodulation unit 1301 and the high pass demodulation unit 1302 respectively supply the demodulated coded data to the video signal decoding unit 1136.

The video signal decoding unit 1136 synthesizes the encoded data supplied from the low pass demodulator 1301 and the high pass demodulator 1302 into one in a method corresponding to the division method, and the encoded data is referred to FIG. 12 and the like. The decoding is performed in the same manner as the video signal decoding unit 136 described above. The video signal decoding unit 1136 outputs the obtained video signal to a processing unit at a later stage.

In addition, the digital triax system 1100, as shown in FIG. 35, for the system of data transmission between the transmission unit 1110 and the camera control unit 1112 via the triax cable 1111 as described above, It also has a rate control unit 1113 which performs control so that the data transmission is stably performed so that no breakage occurs (not causing the decoding process to fail).

As illustrated in FIG. 35, the rate controller 1113 includes a modulation controller 1401, an encoding controller 1402, a carrier to noise ratio measurement unit 1403, and an error rate measurement unit 1404. Has

The modulation control unit 1401 controls the constellation signal point distance and the error correction bit allocation amount of the modulation performed by the digital modulator 1122 (low pass modulator 1201 and high pass modulator 1202). In OFDM, digital modulation schemes such as Phase Shift Keying (PSK) (including Differential Phase Shift Keying (DPSK)) and Quadrature Amplitude Modulation (QAM) are employed. Constellation is one of the observation methods of such a digital modulation wave, and observes the spread of the trajectory of the signal drawn to and from the above signal points on I-Q coordinates orthogonal to each other. The constellation signal point distance represents the distance between signal points on this I-Q coordinate.

In the constellation, the larger the noise component included in the signal, the larger the trajectory of the signal. That is, in general, the shorter the signal point distance is, the more likely a symbol error is caused by the noise component, and the resistance to the noise component of the decoding process is weakened (the decoding process is likely to fail).

Therefore, the modulation control unit 1401 can suppress the excessive increase of the symbol error rate based on the respective attenuation rates of the high frequency component and the low frequency component, so that the low frequency modulator 1201 and the high frequency modulator 1202 can stably transmit data. By setting the modulation schemes for each, the length of the signal point distance in each modulation process is controlled. In addition, each modulation method in the case where the attenuation rate set by the modulation control part 1401 is small and large is predetermined.

In addition, the modulation control unit 1401 further suppresses excessive increase in the symbol error rate based on the respective attenuation rates of the high pass component and the low pass component, so that the low pass modulator 1201 and the high pass modulation can be more stably transmitted. For unit 1202, an allocation amount of error correction bits (error correction bit lengths allocated to data) for data is set, respectively. By increasing the allocation amount of the error correction bits (by lengthening the error correction bit length), the data transmission efficiency is reduced because the amount of unnecessary data is originally increased, but the symbol error rate due to the noise component can be reduced, thereby reducing the noise component of the decoding process. The resistance to can be made stronger. In addition, the allocation amount of each error correction bit in the case where the attenuation rate set by the modulation control unit 1401 is small and large is predetermined.

The coding control unit 1402 controls the compression rate of the compression coding performed by the video signal coding unit 1120. The encoding control unit 1402 controls the video signal encoding unit 1120 to set the compression rate. However, when the attenuation rate is large, the compression rate setting is made high to reduce the data amount of the encoded data and increase the data transfer rate. Reduce. In addition, each value of the compression rate when the damping rate set by the coding control part 1402 is small, and the compression rate when it is large is predetermined previously.

The C / N ratio measurement unit 1403 is received by the video separation / synthesis unit 1130 and measures the C / N ratio, which is the ratio of the carrier wave and noise, to the modulated signal supplied to the amplifier 1131. CN ratio (CNR) can be calculated | required, for example by following formula (4). The unit is [dB].

Where PN is noise power [W] and PC is carrier power [W]

The C / N ratio measurement unit 1403 supplies the measurement result (C / N ratio) to the measurement result determination unit 1405.

The error rate measuring unit 1404 is based on the processing result of the demodulation process by the digital demodulation unit 1134 (low pass demodulation unit 1301 and high pass demodulation unit 1302), and the error rate of the demodulation process (symbol error occurrence rate). Measure The error rate measuring unit 140 supplies the measurement result (error rate) to the measurement result determination unit 1405.

The measurement result determination unit 1405 measures the C / N ratio of the transmission data received by the camera control unit 1112 and the error rate measurement unit 1404 measured by the C / N ratio measurement unit 1403. The attenuation rates of the low pass component and the high pass component of the transmission data are respectively determined based on at least one of the error rates of the demodulation process, and the determination results are supplied to the modulation control unit 1401 and the encoding control unit 1402. The modulation control unit 1401 and the coding control unit 1402 each perform the control as described above based on the determination result (for example, whether or not the attenuation rate of the high frequency component is obviously larger than that of the low frequency component).

An example of the flow of the rate control processing executed by the rate control unit 1113 will be described with reference to the flowchart of FIG. 36.

The rate control process is executed at a predetermined timing, for example, at the start of data transmission of the transmission unit 1110 and the camera control unit 1112. When the rate control process is started, in step S201, the modulation control unit 1401 controls the digital modulation unit 1122 to set the allocation amount of the constellation signal point distance and the error correction bit when the attenuation rate is not large. It is set to a value common to all predetermined bands. In other words, the modulation control unit 1401 sets the same modulation scheme and the same error correction bit allocation amount for each of the low pass modulation unit 1201 and the high pass modulation unit 1202.

In step S202, the encoding control unit 1402 controls the video signal encoding unit 1120 to set the compression rate to a predetermined predetermined initial value to be set when the attenuation rate is not large.

In this way, in the state where the low and high ranges are set equal to each other, the modulation control unit 1401 and the encoding control unit 1402 control each unit of the transmission unit 1110 in step S203, and perform each process at the set value. And the predetermined predetermined compressed data is transmitted to the camera control unit 1112.

For example, the rate control unit 1113 (modulation control unit 1401 and encoding control unit 1402) inputs a predetermined predetermined video signal (image data) to the transmission unit 1110, and the video signal encoding unit 1120. The video signal is encoded by the digital video signal, the OFDM coded by the digital modulator 1122, the amplified modulation signal by the amplifier 1124, and the video signal is transmitted to the video separation / synthesis section 1126. The transmission data thus transmitted is transmitted through the triax cable 1111 and received by the camera controller 1112.

In step S204, the C / N ratio measurement unit 1403 measures the C / N ratio of the transmitted data transmitted in this manner for each OFDM channel, and supplies the measurement result to the measurement result determination unit 1405. In step S205, the error rate measuring unit 1404 measures the occurrence rate (error rate) of the symbol error of the demodulation process by the digital demodulation unit 1134 for each OFDM channel, and measures the measurement result in the measurement result determination unit 1405. To feed.

In step S206, the measurement result determination unit 1405 determines whether the measured data is transferred based on the C / N ratio supplied from the C / N ratio measuring unit 1403 and the error rate supplied from the error rate measuring unit 1404. It is determined whether or not the attenuation rate of the high frequency component is equal to or greater than a predetermined threshold value. When it is determined that the high band attenuation rate of the transmitted data is clearly larger than the low band attenuation rate, and the high band attenuation rate is determined to be greater than or equal to the threshold value, the measurement result determination unit 1405 advances the processing to step S207.

In step S207, the modulation control unit 1401 changes the modulation scheme of the high pass modulation unit 1202, widens the constellation signal point distance of the high pass component, and in step S208, the high pass modulation unit 1202 Change the setting to increase the error correction bit allocation.

In step S208, the encoding control unit 1402 controls the video signal encoding unit 1120 to increase the compression ratio.

As described above, when the setting is changed, the rate control unit 1113 ends the rate control process.

In addition, when it is determined in step S206 that the high band attenuation is about the same as the low band and the high band attenuation is less than the threshold value, the measurement result determination unit 1405 omits the processing of steps S207 to S209 and controls the rate. The process ends.

As described above, the transmission control unit 1110 and the camera control unit 1112 are more stable by the rate control unit 1113 controlling the signal point distance (modulation method) and the error bit allocation amount for each modulation unit (for each OFDM channel). In addition, data transmission can be performed more efficiently. As a result, a more stable, low-latency digital triax system can be realized.

In the above description, for convenience of explanation, the case where there are two OFDM channels (the digital modulator 1122 has two modulators of the low pass modulator 1201 and the high pass modulator 1202) has been described. The number of channels (the number of modulators) is arbitrary, for example, three or more modulators may exist. At this time, the modulators may be divided into two groups, a high band and a low band by the band of the OFDM channel, and rate control is performed for each group as described above with reference to the flowchart of FIG. However, the rate control described with reference to the flowchart of FIG. 36 may be performed for three or more modulators (or groups).

For example, when three modulators exist, the attenuation rate may be determined for each modulator. In this case, the C / N ratio and the error rate of the transmission data are measured for three of the low range, the middle region and the high region. As described above, each modulator is set to a value (method) common to all bands. When the high band only has a large attenuation rate, only the high band modulator is changed, and the high band and mid band have a large attenuation. In this case, only the high and medium region modulators are changed. In the setting of the compression rate of the video signal encoder 1120, the more the band with the larger attenuation rate, the larger the compression rate.

By performing control in a finer band as described above, it is possible to perform a control more suited to the attenuation characteristic of the triax cable, and to improve the efficiency of data transmission in a stable state.

The rate control may be any method that allows more suitable control of the attenuation characteristics of the triax cable. When the rate control is performed on three or more modulators as described above, the control method is, for example, For example, a method other than the above may be used, such as changing the allocation amount of the error correction bits for each band.

In the above description, the rate control is performed at a predetermined timing such as at the start of data transfer. However, the timing and the number of executions of the rate control are arbitrary. For example, even when the rate control unit 1113 actually transmits data. The attenuation rate (C / N ratio or error rate) may be measured and at least one of a modulation scheme, an allocation amount of error correction bits, and a compression rate may be controlled in real time (immediately).

In addition, although the C / N ratio and the error rate have been described as an index for determining the attenuation rate, it is arbitrary how and what parameter is used to calculate or determine the attenuation rate. Therefore, for example, parameters other than those described above, such as S / N ratio (Signal Noise Ratio), may be measured.

In FIG. 35, only the case where the rate control unit 1113 controls data transmission performed through the triax cable 1111 to the camera control unit 1112 from the transmission unit 1110 has been described. In the digital triax system, data transmission may be performed from the camera control unit 1112 toward the transmission unit 1110. The rate control unit 1113 may also perform rate control for such a transmission system. Also in this case, the direction of data transmission in the transmission system is changed, but since the method is basically the same as in the case of FIG. 35, the rate control unit 1113 can perform rate control in the same manner as described with reference to FIGS. 35 and 36. Can be.

In addition, although the rate control part 1113 was demonstrated so that it may be comprised separately from the transmission unit 1110 and the camera control part 1112, the structure method of each part of the rate control part 1113 is arbitrary, for example, the rate control part 1113 may be incorporated in either the transmission unit 1110 or the camera control unit 1112. In addition, for example, a modulation control unit 1401 and an encoding control unit 1402 are incorporated in the transmission unit 1110, and the C / N ratio measurement unit 1403, the error rate measurement unit 1404, and the measurement result determination unit 1405 ) May be incorporated in the camera control unit 1112, and the transmission unit 1110 and the camera control unit 1112 may be configured to incorporate different portions of the rate control unit 1113.

By the way, for example, as shown in FIG. 37, the digital triax system shown in FIG. 3 is actually implemented as a system larger than a combination of some camera and some CCU. For example, the digital triax system 1500 illustrated in FIG. 37 has a configuration in which three configurations illustrated in FIG. 3 are synthesized. That is, in the digital triax system 1500, the camera 1511 to the camera 1513 corresponding to the video camera unit 113 and the transmission unit 110 of FIG. 3 are each the triax cable 111 of FIG. 3. The transmission shown in FIG. 3 is connected to the CCU 1531 to the CCU 1533 corresponding to the camera control unit 112 of FIG. 3 by the triax cable 1521 to the triax cable 1523 corresponding thereto. Three transmission systems similar to the system are formed. In addition, data output from each of the CCU 1531 to CCU 1533 is integrated as data of one system by a selection operation by a switcher 1541.

For example, if the transmission system as described with reference to FIG. 3 is a single system of digital triax system, the delay from the image capturing (image data generation) by the camera to the output of the image data from the CCU is reduced. In order to achieve the delay, the encoder built into each camera and the decoder built into the CCU are operated on the basis of their own synchronization signals, and the encoder executes encoding processing if image data is obtained by imaging with the camera. If the decoder transmits the encoded data to the CCU, the decoder may decode the encoded data. However, in a system having a plurality of transmission systems as shown in FIG. 37, in order to integrate in the switcher 1541, it is necessary to match the timing (phase) of image data output from each CCU.

Therefore, as shown in FIG. 37, the reference signal 1551, which is an external synchronization signal, is supplied to each camera as well as each CCU via each CCU. That is, the operation of the encoder built in each camera and the decoder built in each CCU are synchronized with this reference signal 1551. By doing in this way, the data transfer of each system, ie, the output timing of the image data from each CCU, can be synchronized mutually, without performing unnecessary buffering etc .. In other words, the system can be synchronized while maintaining a low delay.

In general, however, data transmission from the camera to the CCU cannot be performed without delay. In other words, in order to avoid unnecessary buffering (i.e., to suppress the increase in delay), the execution timing of the decoding process by the decoder built into the CCU is slightly different from the execution timing of the encoding process by the encoder built into the camera. It is desirable to be delayed.

Since the appropriate delay time of this execution timing depends on the delay time of a transmission system, there exists a possibility that it may differ in each system by various factors, such as a cable length. Therefore, for example, an appropriate value of this delay time may be obtained for each system, and the synchronization timing of the encoder and the decoder may be set for each system based on the value. By setting the synchronization timing for each system in this way, it is possible to synchronize between systems based on the reference signal while maintaining the low delay.

The calculation of the delay time is performed by transmitting image data from the camera to the CCU as in the actual case. At this time, if the data amount of the image data to be transmitted is larger than necessary (for example, if the contents of the image are complicated), there is a fear that the delay time is set to be larger than the delay time required for actually performing data transfer. That is, there is a fear that unnecessary delay time occurs in data transmission.

FIG. 38 is a diagram showing an example of the state of data transmission in the digital triax system 1500 of FIG. 37. An example of the state of processing timing during each processing step when transferring image data from the camera to the CCU is shown. It is shown. In Fig. 38, T1 to T5 at each stage indicate the synchronization timing of the reference signal.

In FIG. 38, the uppermost stage shows the state of data when image data is obtained (image input) by imaging with a camera. As shown here, at each timing of T1 to T4, image data (image data 1601 to 1604) for one frame is input.

In FIG. 38, the 2nd stage from the top shows the state of the data when the encoding process by the encoder built in a camera is performed (encoding). As shown here, at timing T1, when the encoder built into the camera encodes the image data 1601 by the encoding method described with reference to Fig. 4 or the like, two packets of encoded data (packet 1611) And packet 1612). Here, the "packet" indicates that the encoded data is divided for each predetermined data amount (partial data of the encoded data). Similarly, at timing T2, five packets of encoded data (packets 1613 to 1661) are generated from the image data 1602, and at timing T3, two packets of encoded data (packets (from the image data 1603) are generated. 1618 and the packet 1619 are generated. At timing T4, one packet of encoded data (packet 1620) is generated from the image data 1604. Here, the packet 1611, the packet 1613, the packet 1618, and the packet 1620 enclosed with the rectangles represent the first packet of the image data of each frame.

In FIG. 38, the 3rd stage from the top shows the state of the data at the time of transmission (transmission) from a camera to a CCU. As shown here, in the transmission from the camera to the CCU, the upper limit of the transmission rate is determined, and a maximum of three packets can be transmitted at each timing, so that the timing T2 is surrounded by a dotted line in the second stage from the top. Packets (packet 1616 and packet 1617) are to be transmitted at the next timing T3. In other words, as indicated by arrow 1651, the transmission timing is shifted by one timing. As a result, as indicated by arrow 1652, the first packet 1618 is transmitted last at the timing T3, and the packet 1619 surrounded by the dotted line at the second stage from the top is transmitted at the next timing T4. .

The leading packet 1620 is transmitted at the end of timing T4, as indicated by arrow 1653.

As described above, when the code amount is large, it takes time for data transmission, and the data transmission may not be completed within one timing. In FIG. 38, the bottommost stage shows an example of the state of the data when decoding the encoded data transmitted by the decoder built in the CCU. When such a thing occurs, the packets 1613 to 1661 generated from the image data 1602 coincide on the CCU side at timing T3, so that decoding processing for them is performed at timing T3.

Therefore, the packet 1611 and the packet 1612 generated from the image data 1601 are decoded at timing T2 so that the decoding can be performed continuously, and the packet 1618 and the packet 1619 generated from the image data 1603. Is decoded at timing T4, and the packet 1620 generated from image data 1604 is decoded at timing T5.

As described above, when the delay time is measured using image data having a large amount of data such as the image data 1602, there is a fear that unnecessary delay time is measured. Therefore, when image data is transmitted for the measurement of the delay time, image data having a small amount of data such as a black image or a white image may be used.

39 is a block diagram showing an example of the configuration of a digital triax system in that case. The digital triax system 1700 illustrated in FIG. 39 is a system corresponding to a part of the digital triax system 1500 described with reference to FIG. 37, and is basically the same as that of the digital triax system 100 of FIG. 3. Has a configuration. In FIG. 39, only the structure required for description is shown.

As shown in FIG. 39, the digital triax system 1700 is, for example, a video camera unit 1713 and a transmission unit 1710 corresponding to the camera 1511 of the digital triax system 1500 (FIG. 37). ), For example the triax cable 1711 corresponding to the triax cable 1521 of the digital triax system 1500 (FIG. 37), for example the CCU (of the digital triax system 1500 (FIG. 37). And a camera controller 1712 corresponding to 1531. The video camera unit 1713 also corresponds to the video camera unit 113 of the digital triax system 100 (FIG. 3), and the transmission unit 1710 transmits the digital triax system 100 (FIG. 3). Corresponds to unit 110, triax cable 1711 also corresponds to triax cable 111 of digital triax system 100 (FIG. 3), and camera control unit 1712 is digital triax system 100. It also corresponds to the camera control part 112 of FIG.

The transmission unit 1710 has a video signal encoder 1720 which is equivalent to the video signal encoder 120 of the transmission unit 110, and the camera controller 1712 has a video signal decoder 136 of the camera controller 112. Has a video signal decoder 1736 equivalent to. The video signal encoder 1720 of the transmission unit 1710 encodes the image data supplied from the video camera unit 1713 in the same manner as the video signal encoder 120 described with reference to FIG. 4 and the like. The transmission unit 1710 also OFDMs the obtained coded data and transmits the obtained modulated signal to the camera control unit 1712 via the triax cable 1711. When the camera control unit 1712 receives the modulated signal, it demodulates it by the OFDM method. The video signal decoding unit 1736 of the camera control unit 1712 decodes the coded data obtained by demodulation, and outputs the obtained image data to a system (for example, a switcher) at a later stage.

In addition, an external synchronization signal 1751 is supplied to the camera controller 1712. The external synchronization signal 1751 is also supplied to the transmission unit 1710 via the triax cable 1711. The transmission unit 1710 and the camera control unit 1712 operate in synchronization with this external synchronization signal.

In addition, the transmission unit 1710 has a synchronization control unit 1719 for controlling synchronization timing with the camera control unit 1712. Similarly, the camera control unit 1712 has a synchronization control unit 1701 for controlling synchronization timing with the transmission unit 1710. Naturally, the external synchronization signal 1751 is also supplied to these synchronization control unit 1771 and the synchronization control unit 1771. The synchronization control unit 1771 and the synchronization control unit 1771 respectively control the camera control unit 1712 and the transmission unit 1710 so that the mutual synchronization timing is appropriate while synchronizing with the external synchronization signal 1751. .

An example of the flow of this control process is demonstrated with reference to the flowchart of FIG.

When the control process is started, the synchronization control unit 1701 of the camera control unit 1712 communicates with the synchronization control unit 1773 in step S301, and establishes command communication so that the control command can be received. Correspondingly, in step S321, the synchronization control unit 1771 of the transmission unit 1710 also communicates with the synchronization control unit 1701 to establish command communication.

When the control command can be received, the synchronization control unit 1701 causes the encoder to input a black pixel in which all the pixels are images for one picture of black, in step S302. The synchronization control unit 1773 has image data 1701 (hereinafter referred to as black pixel 1781) of black pixels (the entire pixel is an image for one picture of black) with a small amount of data. In step S322, the synchronization control unit When the instruction is received from the unit 1761, the black pixel 1781 is supplied to the video signal encoder 1720 (encoder) in step S323, and the video signal encoder 1720 is controlled in step S324. Similarly to the case of the image data supplied from the video camera unit 1713 (actual case), the black pixel 1781 is encoded. In addition, the synchronization control unit 1773 controls the transmission unit 1710 to start data transmission of the obtained encoded data in step S325. More specifically, the synchronization control unit 1771 controls the transmission unit 1710 to OFDM the encoded data in the same manner as in the actual case, and then obtains the modulated signal obtained through the triax cable 1711 through the camera control unit 1712. Send it to

After instructing the synchronization control unit 1771, the synchronization control unit 1701 waits for the modulation signal to be transmitted from the transmission unit 1710 to the camera control unit 1712 in steps S303 and S304. If it is determined in step S304 that the camera control unit 1712 has received data (modulation signal), then the synchronization control unit 1701 advances the processing to step S305 to control the camera control unit 1712 to transmit the modulated signal to the OFDM signal. The video signal decoding unit 1736 starts the decoding (decoding) of the obtained coded data. When the decoding is started, the synchronization control unit 1701 waits until the decoding is completed in steps S306 and S307. If it is determined in step S307 that the decoding is completed and the black pixel has been obtained, then the synchronization control unit 1701 advances the processing to step S308.

In step S308, the synchronization control unit 1701 determines the decoding start timing of the video signal decoding unit 1736 based on the time from the instruction in step S302 to the determination in step S307 that the decoding is completed as described above. Relative timing with respect to the encoding start timing of the video signal encoder 1720). Of course, this timing is synchronized with the external synchronization signal 1751.

In step S309, the synchronization control unit 1701 instructs the synchronization control unit 1771 to input the captured image from the video camera unit 1713 to the encoder. When the instruction is acquired in step S326, the synchronization control unit 1171 controls the transmission unit 1710 in step S327, so that the image data of the captured image supplied from the video camera unit 1713 at a predetermined timing is a video signal. The encoder 1720 is supplied.

The video signal encoder 1720 starts encoding of the captured image at a predetermined timing corresponding to the supply timing thereof. The video signal decoding unit 1736 starts decoding at a predetermined timing corresponding to the encoding start timing, based on the setting made in step S308.

As described above, the synchronization control unit 1771 and the synchronization control unit 1771 control the synchronization timing between the encoder and the decoder using the image data having a small amount of data, thereby increasing unnecessary delay time by setting the synchronization timing. Can be suppressed. As a result, the digital triax system 1700 can synchronize the output of the image data with other systems while maintaining the low latency and suppressing the increase in the buffer required for data transmission.

In the above description, the black pixel is used to control the synchronization timing. However, an image having a small data amount may be used. For example, any image such as a white pixel in which all pixels are white images may be used.

In the above description, the synchronization control unit 1701 embedded in the camera control unit 1712 has been described so as to give an instruction such as start of encoding to the synchronization control unit 1771 included in the transmission unit 1710, but the present invention is not limited thereto. 1177 may be mainly used to perform control processing, and to instruct an instruction such as start of decoding. In addition, the synchronization control unit 1701 and the synchronization control unit 1775 may be configured separately from the transmission unit 1710 and the camera control unit 1712. In addition, the synchronization control unit 1701 and the synchronization control unit 1775 may be configured as one processing unit, and the synchronization control unit 1701 and the synchronization control unit 1775 may be incorporated in the transmission unit 1710 at that time. It may be incorporated in the camera control unit 1712 or may be configured separately from them.

The series of processes described above may be executed by hardware or may be executed by software. When a series of processes are performed by software, the program which comprises the software can perform various functions by installing the computer or various programs which were built in the dedicated hardware, for example, general-purpose personals. It is installed in a computer or the information processing apparatus of the information processing system which consists of a some apparatus from a program recording medium.

41 is a block diagram illustrating an example of a configuration of an information processing system that executes the above-described series of processes by a program.

As shown in FIG. 41, the information processing system 2000 includes an information processing device 2001, a storage device 2003, and a plurality of video tapes connected by the information processing device 2001 and a PCI bus 2002. A system constituted by VTRs 2004-1 to VTRs 2004-S, which are recorders VTRs, a mouse 2005, a keyboard 2006, and an operation controller 2007 for a user to input operation to them. It is a system which performs the above-mentioned image coding process, image decoding process, etc. by the installed program.

For example, the information processing device 2001 of the information processing system 2000 stores the encoded data obtained by encoding the moving picture content stored in the large capacity storage device 2003 made of RAID (Redundant Arrays of Independent Disks). (2003) or the decoded image data (video image content) obtained by decoding the coded data stored in the storage device 2003, or the decoded image data (video image content). 2004-1) to VTR 2004-S to record on videotape. Further, the information processing apparatus 2001 is configured to supply the moving picture contents recorded on the video tapes mounted on the VTRs 2004-1 to VTR 2004-S to the storage device 2003. FIG. At that time, the information processing device 2001 may be configured to encode the moving image content.

The information processing device 2001 includes a microprocessor 2101, a graphics processing unit (GPU) 2102, an extreme data rate (RAM) -RAM 2103, a south bridge 2104, a hard disk drive (HDD) 2105. And a USB (Universal Serial Bus) interface (USB I / F (Interface)) 2106 and a sound input / output codec 2107.

The GPU 2102 is connected to the microprocessor 2101 via a dedicated bus 2111. The XDR-RAM 2103 is connected to the microprocessor 2101 via a dedicated bus 2112. The south bridge 2104 is connected to an I / O (In / Out) controller 2144 of the microprocessor 2101 via a dedicated bus. The south bridge 2104 is also connected with an HDD 2105, a USB interface 2106, and a sound input / output codec 2107. The speaker 2121 is connected to this sound input / output codec 2107. The display 2122 is connected to the GPU 2102.

In addition, the south bridge 2104 further includes a mouse 2005, a keyboard 2006, a VTR 2004-1 to a VTR 2004-S, a memory device 2003, and an operation controller via the PCI bus 2002. 2007) is connected.

The mouse 2005 and the keyboard 2006 receive a user's operation input, and supply a signal indicating the contents of the user's operation input to the microprocessor 2101 through the PCI bus 2002 and the south bridge 2104. . The storage device 2003 and the VTRs 2004-1 to VTR 2004-S are configured to record or reproduce predetermined data.

The drive bus 2008 is also connected to the PCI bus 2002, and removable media 2011, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is appropriately mounted and read from them. A computer program is installed in the HDD 2105 as necessary.

The microprocessor 2101 is a general-purpose main CPU core 2141 that executes basic programs such as an operating system (OS), and a plurality (in this case, eight connected to the main CPU core 2141 through an internal bus 2145). A sub CPU core 2142-1 to a sub CPU core 2142-8, which is a signal processing processor of the RISC (Reduced Instruction Set Computer) type, and an XDR-RAM having a capacity of 256 [MByte], for example. The memory controller 2143 which performs memory control for 2103 and the I / O controller 2144 which manages the input / output of data between the south bridge 2104 have a multi-core configuration in which one chip is integrated. For example, the operating frequency 4 [kHz] is realized.

The microprocessor 2101 reads out necessary application programs stored in the HDD 2105 based on the control program stored in the HDD 2105 at startup, and expands them to the XDR-RAM 2103, after which the application is executed. The necessary control processing is executed based on the program and operator operations.

In addition, the microprocessor 2101 executes software, for example, realizes the image encoding process and the image decoding process of the above-described embodiments, and transmits the encoded stream through which the result of encoding is obtained via the south bridge 2104. The playback video of the moving picture content obtained as a result of being supplied to the HDD 2105 and stored or decoded can be transferred to the GPU 2102 for display on the display 2122.

Although the use method of each CPU core in the microprocessor 2101 is arbitrary, for example, the main CPU core 2141 performs processing concerning the control of the bit rate conversion processing performed by the data control unit 137, and the eight sub-CPUs. A part or all of the cores 2142-1 to the sub CPU cores 2142-8 may be controlled so as to execute details of the bit rate conversion processing such as the count of the code amount. By using a plurality of CPU cores, for example, it is possible to perform a plurality of processes simultaneously and in parallel, so that a bit rate conversion process can be performed at a higher speed.

In addition, processing other than bit rate conversion may be performed by any CPU core in the microprocessor 2101, such as image encoding processing, image decoding processing, or processing related to communication. At this time, different processes may be executed simultaneously and in parallel in each CPU core, thereby improving the efficiency of each process, reducing the delay time of the whole process, and also loading, processing time, and memory required for processing. The capacity may be reduced.

For example, when an independent encoder, decoder, or codec processing apparatus is connected to the PCI bus 2002, the eight sub CPU cores 2142-1 to the sub CPU cores 2142-8 of the microprocessor 2101 are used. ) May control the processing executed by these devices via the south bridge 2104 and the PCI bus 2002. Further, when a plurality of these devices are connected or when these devices include a plurality of decoders or encoders, the eight sub CPU cores 2142-1 to the sub CPU cores 2142-8 of the microprocessor 2101 are used. ) May share and control the processing executed by a plurality of decoders or encoders.

At this time, the main CPU core 2141 manages the operations of the eight sub CPU cores 2142-1 to the sub CPU cores 2142-8, allocates processing to each sub CPU core 2142, or processes the results. Takes over or In addition, the main CPU core 2141 also performs processing other than those performed by the sub CPU cores 2142-1 to the sub CPU cores 2142-8. For example, the main CPU core 2141 receives a command supplied from the mouse 2005, the keyboard 2006, or the operation controller 2007 through the south bridge 2104, and executes various processes according to the command. .

The GPU 2102 displays the playback image of the moving image content and the still image of the still image content, in addition to the final rendering processing relating to the placement of the texture when moving the reproduced image of the moving image content displayed on the display 2122 or the like. It is responsible for the coordinate conversion calculation processing for displaying a plurality of times at a time, and the function of enlarging / reducing the reproduced video of the moving image content and the still image content of the still image content, thereby reducing the processing burden of the microprocessor 2101. It consists of

Under the control of the microprocessor 2101, the GPU 2102 performs predetermined signal processing on the image data of the supplied moving image content or the image data of the still image content, and displays the resulting image data or image data (2122). ), And the image signal is displayed on the display 2122.

By the way, the reproduced images of the plurality of moving picture contents decoded in parallel in parallel by the eight sub CPU cores 2142-1 to the sub CPU cores 2142-8 in the microprocessor 2101 are transferred via the bus 2111. Although transferred to the GPU 2102, the transfer speed at this time is, for example, at most 30 [Gbyte / sec], so that even a complex playback video with special effects can be displayed at high speed and smoothly.

In addition, the microprocessor 2101 performs a voice mixing process on the audio data among the video data and the audio data of the moving image content, and converts the resulting edited audio data into the south bridge 2104 and the sound input / output codec 2107. By transmitting the sound to the speaker 2121, the sound based on the sound signal can be output from the speaker 2121.

When the above-described series of processes are executed by software, a program constituting the software is installed from a network or a recording medium.

This recording medium is, for example, a magnetic disk (including a flexible disk) on which a program is recorded, which is distributed to distribute a program to a user, separately from the apparatus main body, as shown in FIG. 41, an optical disk. (Including compact disc-read only memory (CD-ROM), digital versatile disc (DVD)), magneto-optical disc (including mini-disk), or removable media (2011) And a HDD 2105, a storage device 2003, or the like in which a program is recorded, which is delivered to a user in a state of being pre-built in the apparatus main body. Of course, the recording medium may be a semiconductor memory such as a ROM or a flash memory.

In the above description, it has been described that eight sub CPU cores are configured in the microprocessor 2101. However, the present invention is not limited thereto, and the number of sub CPU cores is arbitrary. The microprocessor 2101 does not have to be constituted by a plurality of cores such as the main CPU core 2141 and the sub CPU cores 2142-1 to the sub CPU cores 2142-8. CPUs composed of two cores) may be used. Alternatively, a plurality of CPUs may be used instead of the microprocessor 2101, and a plurality of information processing apparatuses may be used (that is, a program for executing the process of the present invention is executed in a plurality of apparatuses operating in cooperation with each other). You may also do so.

In addition, in this specification, the step of describing the program recorded on the recording medium includes not only processing performed in time series according to the described order but also processing executed in parallel or separately even if not necessarily in time series. will be.

In addition, in this specification, a system shows the whole apparatus comprised by the some device (device).

In addition, the structure demonstrated as one apparatus may be divided and comprised as a some apparatus above. On the contrary, the configuration described as a plurality of devices may be integrated so as to be configured as one device. In addition, you may make it add the structure of that excepting the above to the structure of each apparatus. In addition, as long as the configuration and operation as the whole system are substantially the same, a part of the configuration of an arbitrary device may be included in the configuration of another device.

The present invention can be applied to, for example, a digital triax system.

Claims (16)

An information processing apparatus for generating encoded data by encoding image data, A plurality of subbands decomposed into frequency bands for each line block including image data for the number of lines necessary to generate coefficient data for each line of the subband of the lowest band component. Rearrangement means for rearranging in advance in order of performing a synthesizing process of synthesizing coefficient data to generate image data; Encoding means for encoding the coefficient data rearranged by the rearrangement means for each line block to generate encoded data; Storage means for storing encoded data generated by the encoding means; Calculation means for calculating a total of the code amounts of the coded data each time the coded data is stored in the plurality of line blocks by the storage means; Output means for outputting the coded data stored in the storage means when the total sum of the code amounts calculated by the calculation means reaches the target code amount An information processing apparatus having a. The method of claim 1, And said output means changes a bit rate of said encoded data. The method of claim 1, And the rearrangement means rearranges the coefficient data for each line block in the order of the low pass component to the high pass component. The method of claim 1, And control means for controlling the rearrangement means and the encoding means to operate in parallel for each line block. The method of claim 1, The rearrangement means and the encoding means perform each processing in parallel. The method of claim 1, And filter means for performing filter processing for each of the line blocks on the image data to generate a plurality of subbands composed of coefficient data decomposed for each frequency band. The method of claim 1, And decoding means for decoding the encoded data. The method of claim 1, Modulation means for modulating the encoded data in different frequency domains to generate a modulated signal; Amplifying means for frequency multiplexing and amplifying the modulated signal generated by the modulating means; And transmitting means for synthesizing and transmitting the modulated signal amplified by the modulating means. The method of claim 8, And an modulation control means for setting a modulation method of said modulation means based on the attenuation rate in the frequency domain. The method of claim 8, And a control means for setting a large signal point distance for the high frequency component when the attenuation ratio in the frequency domain is equal to or greater than the threshold value. The method of claim 8, And control means for setting a large amount of error correction bits for the high frequency component when the attenuation ratio in the frequency domain is equal to or greater than the threshold value. The method of claim 8, And a control means for setting a high compression ratio for the high frequency component when the attenuation ratio in the frequency domain is equal to or greater than the threshold value. The method of claim 8, The information processing apparatus modulates by an OFDM method. The method of claim 1, And a synchronization control section for controlling synchronization timing between the encoding means and the decoding means for decoding the encoded data, using image data having a smaller data amount than a threshold value. The method of claim 14, The image data in which the data amount is smaller than the threshold value is an information processing apparatus in which all pixels are images for one picture of black. An information processing method of an information processing apparatus that encodes image data to generate encoded data, A plurality of subbands decomposed into frequency bands for each line block including image data for the number of lines required to generate coefficient data for one line of the subband of the lowest band component. Rearranged in advance in the order of performing a combining process of synthesizing the coefficient data to generate image data, The rearranged coefficient data is encoded for each line block to generate encoded data, Storing the generated encoded data, Whenever the coded data is stored for the plurality of line blocks, the total sum of the code amounts of the coded data is calculated. And outputting the stored coded data when the total sum of the calculated code amounts reaches the target code amount.

Images