JP4559785B2 - Signal processing method and signal processing apparatus - Google Patents

Description

  The present invention relates to a signal processing method and apparatus, and more particularly to a block-unit encoding method and apparatus for which high speed is required.

  In the field of image compression, there are image encoding techniques represented by MPEG, JPEG and the like. These encoding technologies are image compression technologies that divide an image into a plurality of blocks and sequentially perform encoding in units of blocks. In a still image coding technique represented by JPEG, intra-frame predictive coding (hereinafter referred to as intra coding) that attempts to reduce (compress) the amount of information using spatial redundancy in this small area (block). In video coding represented by MPEG, in order to meet the demand for compressing a huge amount of information compared to still images, in addition to the intra coding, temporal direction prediction is performed. Coding using inter-frame predictive coding (hereinafter referred to as inter coding) that uses information to reduce the amount of information is used.

  Hereinafter, processing performed in common in these encoding techniques will be briefly described with reference to FIG. First, discrete cosine transform (hereinafter referred to as DCT) is performed on the image data divided into blocks by an orthogonal transform unit. Next, in order to further increase the coding efficiency (compression efficiency), in order to cut high-frequency components that are difficult to be visually identified by the human eye, the quantization means performs quantization on the DCT coefficients. A quantized DCT coefficient is obtained. The quantized DCT coefficients are temporarily stored in the storage unit in a writing order called raster scan. The quantized DCT coefficients temporarily stored in the storage unit as described above are then read in the reading order called zigzag scanning, and, for example, Huffman encoding is sequentially performed in the encoding means. As described above, in order to encode block units, the order in which the quantized DCT coefficients are generated differs from the order in which the quantized DCT coefficients are encoded, so data rearrangement (scan change) is performed. The above storage unit is indispensable.

  In order to realize high-speed encoding in the above-described manner, a high-speed technique for writing to and reading from a memory is known (see, for example, FIGS. 9 to 11 in Patent Document 1).

  Hereinafter, the technique disclosed in Patent Document 1 will be specifically described.

  A signal processing device 1001 in FIG. 23 is an example of a typical signal processing device disclosed in Patent Document 1, and is a device capable of performing data rearrangement as described above at high speed. FIG. 24A shows a write scan (raster scan) of the storage unit, and FIG. 24B shows a read scan (zigzag scan) of the storage unit.

  The signal processing apparatus 1001 in FIG. 23 includes input terminals 2 and 3 for inputting two pieces of data (here, quantized DCT coefficients, hereinafter referred to as coefficients) that are consecutive in the write scan (first scan) order, The input selectors 4 and 5 for distributing the coefficients input from the input terminals 2 and 3 to a plurality of memories, and the first memory 701 and the second memory 702, and a memory for temporarily storing the coefficients The unit 7, output selectors 9 and 10 for selecting one of the two coefficients read from the storage unit 7, and the two coefficients selected by the output selectors 9 and 10 are output. It comprises output terminals 11 and 12 and a memory control circuit 8 for controlling writing and reading of the storage unit 7.

  A state in which data rearrangement is performed at high speed using the signal processing apparatus 1001 as described above will be described below.

<Description of write control>
First, the two coefficients input from the input terminals 2 and 3 are input to the input selectors 4 and 5. The first input selector 4 is a coefficient to be written to the first memory 701 among the coefficients input in parallel from the input terminal 2 and the input terminal 3 based on the selection signal S1 input from the memory control circuit 8. And is output as write data WD1 to the first memory 701. On the other hand, the second input selector 5 writes the coefficient input in parallel from the input terminal 2 and the input terminal 3 to the second memory 702 based on the selection signal S2 input from the memory control circuit 8. A power coefficient is selected and output as write data WD2 to the second memory 702.

  Here, the selection signals S1 and S2 are pre-set with coefficients that are consecutive in the write scan order so that the coefficients that are consecutive in the read scan order can be read in parallel by two coefficients from the first memory 701 and the second memory 702. It is generated so as to be written into the first memory 701 and the second memory 702 by one coefficient.

  The first memory 701 sequentially writes the write data WD1 supplied from the first input selector 4 to the address A1 in accordance with the address A1 supplied from the memory control circuit 8 and the write enable signal WE1. . On the other hand, the second memory 702 sequentially writes the write data WD2 supplied from the second input selector 5 to the address A2 in accordance with the address A2 supplied from the memory control circuit 8 and the write enable signal WE2. .

  FIG. 25A and FIG. 25B show an example of a memory map when the coefficients are written in the first memory 701 and the second memory 702 by the control as described above. FIG. 26A shows a specific example of addressing at the time of writing in the first memory 701 and the second memory 702 having such a memory map.

<Description of read control>
Next, a description will be given of control for reading in parallel two quantized DCT coefficients that are consecutive in the reading scan order.

  First, the memory control circuit 8 reads the coefficients written in the first memory 701 and the second memory 702 in parallel by the above-described write control from the respective memories in parallel by the first memory 701 and the first memory 701. Addresses A1 and A2 as shown in FIG. 26B are sequentially supplied to the second memory 702, and at the same time, read enable signals RE1 and RE2 are supplied. As a result, the quantized DCT coefficients are sequentially read from the first memory 701 in the order of 0, 16, 9, 10,. On the other hand, the quantized DCT coefficients are sequentially read from the second memory 702 to the RD 2 in the order of 1, 8, 2, 3,.

  As described above, the two coefficients read out one by one from RD1 and RD2 are input to the output selectors 9 and 10, and the earlier coefficient in the reading scan (second scan) order is output to the output terminal 11. The memory control circuit 8 supplies the selection signal S4 to the first output selector 9 and the selection signal S5 to the second output selector 10 so that the slower coefficient is output to the output terminal 12. Supply.

That is, the quantized DCT coefficients are output to the output terminal 11 in the order of 0, 8, 9, 3,..., 62 and the output terminal 12 in the order of 1, 16, 2, 10,. By performing such control, the coefficients can be rearranged at high speed.
JP-A-11-252338

  As described above, the storage unit is composed of two memories, and in order to be able to simultaneously read two consecutive data in the reading scan order from different memories, the write scan ( If the consecutive data in the first scan) is written in different memories, the data can be read in parallel, and high-speed encoding can be realized.

  However, in the case where only the zigzag scan is present as the readout scan (second scan) as in JPEG encoding, it is possible to realize a high-speed scan change by using the above technique. When there are a plurality of read scans (horizontal priority scan, vertical priority scan, zigzag scan), as in MPEG-4, which is an image compression technique for realizing bit rate encoding, the above technique is used. It is difficult to respond.

  MPEG-4, which realizes the low bit rate encoding, has been applied to various devices such as mobile phones in recent years. However, as broadbandization further progresses in the future, encoding corresponding to higher resolution images and more It is easily expected that the demand for encoding corresponding to a high frame rate will increase.

  In such MPEG-4, various new technologies are adopted to realize a low bit rate, and one of them is improvement of encoding efficiency in intra-frame prediction encoding (intra encoding). There is predictive coding that aims to

  In MPEG-4, in order to improve coding efficiency in intra coding, one optimum prediction block is selected from a plurality of blocks adjacent to the block to be coded, and the difference from the prediction block is sequentially encoded. It will become. This will be described with reference to FIG.

  In FIG. 27, a block X is an encoding target block, and blocks A, B, and C are blocks adjacent to the block X, respectively. Blocks X, A, B, and C are each composed of a plurality of DCT coefficients.

The prediction block P of the block X is selected as in (Equation 1). That is,
(Formula 1)
if (| σA−σB | <| σB−σC |)
P = C
else
P = A
It is. here,
σA: DC coefficient of block A σB: DC coefficient of block B σC: DC coefficient of block C | z |: Absolute value of z. That is, the gradient of the DC coefficient of adjacent blocks A, B, and C is calculated, and the block with the larger gradient is selected as the prediction block P of the encoding target block X.

  For example, when the block C is selected as the prediction block P according to (Equation 1), the difference between the block X and the block C is encoded, and when the block A is selected, the block A Are encoded (coefficient predictive encoding).

  In addition, MPEG-4 supports two types of prediction methods such as DC prediction encoding and DC / AC prediction encoding as such coefficient prediction encoding.

  The DC predictive coding is a coding in which only DC coefficients are predicted among the coefficients of the prediction block selected as described above, and a zigzag scan (second scan) is used for the coefficient reading scan in the coding. ) Fixed.

  On the other hand, DC / AC predictive coding is predictive coding intended to improve the coding efficiency more than the above-described DC predictive coding, and includes not only the DC coefficient among the coefficients of the prediction block. Also, the AC coefficient is also a prediction target. For example, when the block C is selected as the prediction block P, all the coefficients at the upper end of the block C are the coefficients to be predicted, and when the block A is selected, all the coefficients at the left end of the block A are the coefficients to be predicted. It becomes. Also, with respect to the read scan in encoding, when block C is selected as the prediction block P, horizontal direction priority scan (third scan) is performed, and when block A is selected as the prediction block P, vertical direction priority scan is performed. (Fourth scan). That is, depending on the prediction block P selected, the readout scan in encoding also changes.

  Also, when it is determined that the DC / AC predictive coding can improve the coding efficiency compared to the DC predictive coding, the DC / AC predictive coding is used, and the coding efficiency is improved. In the MPEG-4 encoding, the DC predictive encoding and the DC / AC predictive encoding are adaptively controlled so that the DC predictive encoding is used. The reading scan in encoding needs to correspond to not only one type such as JPEG but also three types of read scan such as horizontal direction priority scan, vertical direction priority scan, and zigzag scan. In addition, when encoding using only DC prediction is performed, since the reading scan is fixed in a zigzag scan, parallel reading of coefficients can be realized using the technique of Patent Document 1, but DC / AC prediction is used. When the coding is performed, one read scan is not determined from the three types of read scans until the writing of DCT coefficients in the block is completed (AC prediction effect determination is completed). When allocating the two DCT coefficients to the two memories, the two consecutive coefficients in the selected reading scan order can be read in parallel regardless of which reading scan is selected. It is necessary to keep.

  Specific examples in which the technique disclosed in Patent Document 1 is applied to encoding that requires a plurality of read scans are shown in FIGS. 28 (a) to 28 (d). 28A shows a first scan (raster scan) corresponding to FIG. 24A, FIG. 28B shows a second scan (zigzag scan) corresponding to FIG. 24B, and FIG. (C) represents the third scan (horizontal priority scan), and FIG. 28 (d) represents the fourth scan (vertical scan priority).

  As is clear from FIGS. 28B to 28D, the storage unit is composed of two memories, and in the case of all three types of read scans, two coefficients that are consecutive in the read scan order are different. It is difficult to read from two memories simultaneously. In the example shown in the figure, in the third scan shown in FIG. 28C, the coefficients 2, 3 and the coefficients 6, 7 need to be read from the second memory 702, and these coefficients are stored in different addresses. Therefore, it is not possible to read two coefficients in parallel at a time. In the same scan, the coefficients 16 and 17 and the coefficients 10 and 11 need to be read from the first memory 701, and these coefficients are stored in different addresses. It is not possible.

  The example shown here is only an example of memory division when the storage unit is configured by two memories when reading two coefficients in parallel. However, which of the two coefficients is stored in two memories? Thus, it is difficult to solve the above-mentioned problem even if it is distributed.

  In order to solve the above-described problems, the object of the present invention is to provide a case where there are a plurality of coefficient readout scans in encoding and these readout scans are not determined in advance when the coefficients are written to the storage unit. It is another object of the present invention to provide a signal processing method and apparatus capable of continuously reading two coefficients that are consecutive in the reading scan order from different memories, and capable of rearranging data at low cost and at high speed.

  A first signal processing method according to the present invention is a signal processing method for processing two-dimensional block data composed of a plurality of data, and is a raster in a row direction or a column direction so as to store the block data. A step of sequentially writing two pieces of data in the scan order into m pieces of memory (m is an integer of 3 or more) and a plurality of pieces of data are read from the m pieces of memory so as to rearrange the data, and zigzag A step of sequentially selecting two pieces of data that are selected in the order of reading scan selected from among scan, horizontal direction priority scan, and vertical direction priority scan.

  In this way, it is possible to perform high-speed rearrangement (scan change) from two data that are consecutive in the write scan order to two data that are consecutive in the selected read scan order among the plurality of scan orders. Specifically, the data can be rearranged at high speed from the raster scan order in the row direction or the column direction to the zigzag scan order, the horizontal direction priority scan order, or the vertical direction priority scan order.

  According to the second signal processing method of the present invention, in the first signal processing method, a step of performing an encoding process on the selected two pieces of data, and after the encoding process is completed, Two consecutive data are sequentially selected from a plurality of data read out from m memories in the order of raster scanning in the row direction or the column direction, and between the frames in the moving image encoding for the selected two data And a step of performing a process for generating a reference image necessary for predictive encoding.

  In this way, it is possible to start encoding data that has been rearranged at high speed at an early stage, and further increase the speed of the encoding itself. Also, decoding (reference image generation) for data that has been rearranged at high speed can be started early, and it can be expected that the decoding itself will be accelerated.

  According to a third signal processing method of the present invention, in the first signal processing method, each of the m memories has at least one write port and a read port that can read data independently of each other. A memory having two or more, wherein data reading for rearranging the data is performed from a read port of each of the m memories, and selected from the zigzag scan, the horizontal priority scan, or the vertical priority scan A step of performing an encoding process on two consecutive data in the read scan order, and in parallel with the encoding process, the read data is read from another read port of each of the m memories. Two consecutive data items are sequentially selected from a plurality of data in the raster scan order in the row direction or the column direction, and the selected two data items are selected. It is characterized by further comprising a processing step of performing for generating a reference image needed when inter-frame prediction coding in the moving picture coding on data.

  In this way, a plurality of data can be rearranged at high speed, and the processing for generating a reference image necessary for the interframe predictive encoding in the encoding process and the moving image encoding can be executed in parallel. The speed can be expected to be higher than that of the second signal processing method.

A fourth signal processing method according to the present invention is a signal processing method for processing two-dimensional block data composed of a plurality of data, wherein zigzag scanning, horizontal direction priority scanning is performed so as to store the block data. Alternatively, in the vertical direction priority scan, the step of sequentially writing two pieces of data in the selected write scan order into m memories (m is an integer of 3 or more) and the rearrangement of the data so as to realize data rearrangement. A step of reading a plurality of data from one memory and sequentially selecting two pieces of data in the order of raster scanning in the row direction or the column direction.

  In this way, data can be rearranged at high speed from the selected write scan order to the read scan order among the plurality of scan orders.

The fifth signal processing method according to the present invention is characterized in that the fourth signal processing method further comprises a step of performing a decoding process on the two selected data.

  By doing so, it is possible to realize a decoding process using a high-speed data rearrangement result.

According to a sixth signal processing method of the present invention, data is rearranged using any one of the first to third signal processing methods in the first operation mode according to the set operation mode. In the second operation mode, data is rearranged using the fourth or fifth signal processing method.

  By doing this, the data is rearranged from the write scan order to the selected read scan order among the multiple scan orders, and the data is rearranged from the selected write scan order to the read scan order among the multiple scan orders. Is performed while switching appropriately according to the set operation mode, so that data can be rearranged at low cost and at high speed.

  A first signal processing apparatus according to the present invention is a signal processing apparatus that processes two-dimensional block data composed of a plurality of data, and m (m is 3 or more) for storing the block data. An integer) memory, a plurality of input selectors for selecting write data to the m memories from two consecutive data in the row- or column-direction raster scan order, A plurality of output selectors for selecting two consecutive data in the order of reading scan selected from among zigzag scan, horizontal direction priority scan, and vertical direction priority scan among a plurality of data read from m memories And writing and reading control of block data to the storage means, and for the plurality of input selectors and the plurality of output selectors It is characterized in that a memory control means for supplying a selection signal.

  With this configuration, it is possible to provide an apparatus that performs high-speed data rearrangement from the write scan order to the read scan order selected from among the multiple scan orders.

  According to a second signal processing device of the present invention, in the first signal processing device, each of the m memories constituting the storage means has at least one write port, and can be read independently of each other. The memory is characterized by having two or more possible read ports.

  According to this configuration, since data rearrangement from the write scan order to the plurality of read scan orders can be executed in parallel, it is possible to provide an apparatus that executes even faster than the first signal processing apparatus.

  A third signal processing apparatus according to the present invention further includes clock control means for individually controlling clocks supplied to the m memories in the first signal processing apparatus. It is a feature.

  With this configuration, it is possible to individually supply clocks to m memories, and to provide a signal processing device that realizes high speed and low power consumption.

  In a fourth signal processing device according to the present invention, in the third signal processing device, the clock control unit writes two consecutive data in the row scan direction or the column scan sequence in the m memories. In this case, the clock is supplied to the memory where writing occurs and the clock is stopped for the memory where writing does not occur, and the zigzag scan, the horizontal priority scan or the vertical priority scan is selected. In order to select two consecutive data in the read scan order, control is performed so that a clock is supplied to a memory in which a read cycle occurs and a clock is stopped for a memory in which no read cycle occurs. It is characterized by.

  In this way, it is possible to adaptively perform control such as supplying a clock to a memory in which a memory access necessary for data rearrangement occurs and stopping a clock supply to a memory in which no memory access occurs. Because of this, lower power consumption can be expected.

The first imaging system according to the present invention includes an image processing circuit that performs image processing including any of the first to fourth signal processing devices, a sensor that outputs an image signal to the image processing circuit, And an optical system for imaging light onto the sensor.

  With this configuration, it is possible to expect an increase in image processing speed as data rearrangement is performed at high speed.

  According to a second imaging system of the present invention, the first imaging system further includes a converter that converts an image signal obtained from the sensor into a digital signal and supplies the digital signal to the image processing circuit. Yes.

  In this way, the advantages of digital signal processing can be exhibited.

  According to the present invention, data can be rearranged at high speed as a whole.

  Embodiments of the present invention will be described below with reference to the drawings.

<< First Embodiment >>
FIG. 1 is a block diagram showing a configuration of a signal processing apparatus 1 according to the first embodiment of the present invention, and FIG. 2 is a flowchart showing a signal processing method of the present invention using the signal processing apparatus 1 shown in FIG. is there.

  The present embodiment is characterized in that the storage unit 7 is configured by m memories (m is an integer of 3 or more) so that two data (coefficients) that are consecutive in the reading scan order can be read in parallel. And Hereinafter, a case where m = 3 will be described as an example.

  The signal processing apparatus 1 shown in FIG. 1 has input terminals 2 and 3 for inputting two pieces of continuous data in the order of write scan (first scan), and the quantized DCT coefficient input from these input terminals 2 and 3 is m. It consists of input selectors 4, 5, 6 for distributing to (here, m = 3) memories, a first memory 701, a second memory 702, and a third memory 703 to temporarily store data. Storage unit 7 for storing data, output selectors 9 and 10 for selecting one data from each of m data read from the storage unit 7, and 2 selected by these output selectors 9 and 10 The DC terminals (σA, σB, σC) of the blocks (A, B, C) adjacent to the encoding target block necessary for selecting the reading scan and the output terminals 11 and 12 for outputting pieces of data are input. Prediction A number input terminal 13, a setting terminal 14 for setting mode switching between DC prediction encoding and DC / AC prediction encoding, and writing and reading of the storage unit 7 according to the encoding mode set by the setting terminal 14 The memory control circuit 8 supplies the selection signals S1, S2, S3, S4, and S5 to the selectors 4, 5, 6, 9, and 10, respectively.

  Hereinafter, the signal processing method of this embodiment will be described with reference to FIGS.

  As shown in FIG. 2, first, the selected readout is selected regardless of which of the plurality of readout scans (first scan, second scan, third scan, and fourth scan) is selected. Two coefficients that are consecutive in the write scan (first scan) order are stored in the first memory 701, the second memory 702, or the third memory 703 so that two coefficients that are consecutive in the scan order can be read in parallel. They are written separately (step 100). Next, it is determined whether or not writing of all the block coefficients necessary for encoding is completed (step 101). For example, in the case of MPEG-4, the coefficient of a block necessary for encoding is a coefficient for one macro block composed of a plurality of blocks. Step 100, until all coefficient writing is completed. Step 101 is repeated.

  FIG. 3A shows an example in which all the coefficients are written in the first memory 701, the second memory 702, and the third memory 703 as described above. In FIG. 3A, the hatched portion with the lower left diagonal line indicates the area of the first memory 701, the hatched portion with the lower right oblique line indicates the area of the second memory 702, and the non-hatched area indicates the area of the third memory 703. Each of the two coefficients shown in the order of writing scan (first scan) (0, 8, 16, 24,..., 55, 63) is written into each memory area as shown in FIG. It is done. In this case, the memory control circuit 8 uses the addresses shown in FIGS. 4A to 4C for the first memory 701, the second memory 702, and the third memory 703, respectively. A1, A2, and A3 are supplied. In addition, the memory control circuit 8 controls the input selectors 4, 5, and 6 in order to control the write data WD 1, WD 2, and WD 3 in the first memory 701, the second memory 702, and the third memory 703. The signals S1, S2, and S3 are supplied, and the input selectors 4, 5, and 6 are written based on the selection signals S1, S2, and S3, respectively, and write data WD1, WD2 as shown in FIGS. , WD3 are sequentially selected, and the first memory 701, the second memory 702, and the third memory 703 are assigned to each address of each memory in accordance with the write enable signals WE1, WE2, WE3 supplied from the memory control circuit 8. Write the coefficients.

  FIG. 5 shows the states of the addresses (A1, A2, A3), write data (WD1, WD2, WD3) and write enable signals (WE1, WE2, WE3) of each memory in the above write control. In FIG. 5, writing is performed when WE1, WE2, and WE3 are 1, and writing is not performed when 0.

  As described above, when all the two coefficients consecutive in the write scan (first scan) order are written to the first memory 701, the second memory 702, and the third memory 703, respectively, the next reading is performed. Scan determination is performed (step 102 in FIG. 2). The determination of the readout scan is made by the coding mode (DC prediction coding, DC / AC prediction coding) set from the setting terminal 14 and the adjacent blocks (A, B, C) inputted from the prediction coefficient input terminal 13. This is performed as follows using DC coefficients (σA, σB, σC).

  When the encoding mode is set to DC predictive encoding, the readout scan is determined to be a zigzag scan (second scan). When the encoding mode is set to DC / AC predictive encoding, The gradient of the DC coefficient (σA, σB, σC) of the adjacent block is calculated. When the prediction block P is determined from the calculated gradient result and the prediction block P is determined to be the block C, when the encoding efficiency can be improved as compared with the case of the DC prediction encoding, the read scan is performed. If it is determined that the scan is the priority scan in the horizontal direction (third scan) and the encoding efficiency cannot be improved, the scan is determined as the zigzag scan (second scan). Further, the prediction block P is determined from the calculated gradient result, and when the prediction block P is determined to be block A, reading is performed when the encoding efficiency can be improved as compared with the case of DC prediction encoding. The scan is determined to be a vertical direction priority scan (fourth scan), and when the encoding efficiency cannot be improved, it is determined to be a zigzag scan (second scan).

  As a result of the determination of the read scan as described above, when the read scan becomes a zigzag scan (second scan), as shown in FIG. Two consecutive coefficients in the zigzag scan order are read out in parallel from the second memory 702 and the third memory 703 (step 103). FIG. 6 shows the state of read control of each memory in step 103. In the case of zigzag scanning, the coefficient reading order is 0, 1, 8, 16, 9, 2,..., 47, 55, 62, 63, etc. , Read address A1, A2, A3 and read enable signals RE1, RE2, RE3 are supplied, these coefficients are read out from the three memories 701 to 703 as appropriate, and two coefficients that are consecutive in the zigzag scan order are output in the output selectors 9 and 10. By selecting, two coefficients are output to the output terminals 11 and 12 in parallel.

  That is, among the coefficients (0, 1) that are the first two coefficients to be output to the output terminals 11 and 12, the coefficient 0 is read from the first memory 701 (RD1), and at the same time, the coefficient 1 is the second coefficient. Read from the memory 702 (RD2). The memory control circuit is configured so that the earlier coefficient (0) in the zigzag scan order is output to the output terminal 11 and the later coefficient (1) is output to the output terminal 12 among the two coefficients read in this way. 8 supplies selection signals S4 and S5 to the output selectors 9 and 10.

  Of the two coefficients (8, 16) to be output next to the output terminals 11, 12, the coefficient 8 is read from the third memory 703 (RD3), and at the same time, the coefficient 16 is the second memory 702. (RD2). The memory control circuit is configured so that the earlier coefficient (8) in the zigzag scan order is output to the output terminal 11 and the later coefficient (16) is output to the output terminal 12 among the two coefficients read in this way. 8 supplies selection signals S4 and S5 to the output selectors 9 and 10. Thereafter, the coefficients are sequentially read out in parallel two by two in the same manner, and the first processing (encoding) is performed (step 106).

  As a result of the determination of the read scan, when the read scan becomes the horizontal direction priority scan (third scan), as shown in FIG. 3C, the first memory 701, the second memory 701, Two consecutive coefficients are read out in parallel from the memory 702 and the third memory 703 in the order of priority scanning in the horizontal direction (step 104). FIG. 7 shows the state of reading control of each memory in step 104. The manner in which two coefficients are read out in parallel and the first processing is sequentially performed is the same as in the case of the zigzag scan described above, and thus a detailed description thereof will be omitted.

  As a result of the determination of the read scan, when the read scan becomes the vertical direction priority scan (fourth scan), as shown in FIG. Two consecutive coefficients are read out in parallel from the memory 702 and the third memory 703 in the order of priority scanning in the vertical direction (step 105). FIG. 8 shows the state of reading control of each memory in step 105. The manner in which two coefficients are read out in parallel and the first processing is sequentially performed is the same as in the case of the zigzag scan described above, and thus a detailed description thereof will be omitted.

  As described above, all the coefficients are read out, and the processing from step 102 to step 106 is repeated until the encoding is completed (step 107). When all the coefficients are read and the first processing is completed, reading from each memory is performed in the same scan (first scan) order as the write scan (step 108), and the second processing (decoding) is sequentially performed. ) To generate a reference image (step 109). Steps 108 to 109 are repeated until the generation of the reference image is completed (step 110). The reason why the second process (decoding) is necessary is that, in the case of moving picture coding, in order to perform interframe predictive coding, image data (reference data) of the previous frame is required, and the current coding is performed. This is because it is necessary to perform decoding (inverse quantization, inverse DCT, etc.) of the target block.

  As is clear from the above, in order to realize parallel writing and reading of two pieces of data, if the storage unit 7 is composed of m memories (m is an integer of 3 or more), a plurality of readings are performed. Regardless of which scan is selected from among the scans, the two data (coefficients) that are consecutive in the read scan order are written in different memories, so that the two data (coefficients) that are consecutive in the read scan order can be sequentially read in parallel. As a result, scanning changes in writing and reading are performed at a high speed, so that high-speed encoding can be realized. In order to achieve this object, the memory configuration of the storage unit 7 has been devised, and there is no increase in capacity.

<< Second Embodiment >>
Next, a signal processing device 201 according to the second embodiment of the present invention will be described with reference to FIG. 9 and FIG.

  In this embodiment, m (here, m = 3) memories constituting the storage unit 7 are divided into at least one write port (port A) and at least two read ports that can be read independently of each other (port A). Port A, port B), and the m memories can be read in parallel from the plurality of read ports in different read scan orders, and the first process (encoding) and the second process (decoding) Is different from the first embodiment in that it can be executed simultaneously.

  Hereinafter, a different part from 1st Embodiment is demonstrated concretely.

  In FIG. 9, a first memory 701, a second memory 702, and a third memory 703 each include a port A capable of writing and reading independently and a port B dedicated to reading. Port A and port B are ports that can be read independently of each other. First Embodiment With reference to the flowchart shown in FIG. 10, the encoding method in the case where the storage unit 7 is constituted by m memories that can be read in parallel independently from each other from a plurality of ports as described above will be described. Different parts will be described below.

  First, as in the case of the first embodiment, any of the plurality of readout scans (first scan, second scan, third scan, fourth scan) is selected regardless of which scan is selected. The two coefficients that are consecutive in the write scan (first scan) order are the first memory 701, the second memory 702, or the third memory so that the two coefficients that are consecutive in the read scan order can be read in parallel. It is written in the memory 703. The difference from the first embodiment is that writing is performed using port A which is a writing / reading port among a plurality (two in this case) of ports (step 200).

  In the case of the first embodiment, after all the coefficients have been written in each memory in step 101, first processing (step 102 to step 107) is performed first, and then second processing (step 108 to step 108). Step 110) is performed, but in the present embodiment, the first processing (symbol) is performed by simultaneously performing reading by different reading scans using the port A and the port B that are capable of independent reading operations. And the second process (decoding) are to be executed in parallel.

  That is, the reading of the coefficients necessary for the first processing (step 203, step 204, step 205) is performed using port A, and the reading of the coefficients necessary for the second processing (step 208) is performed using port B. By using this method, each process can be executed independently and in parallel. Therefore, it is possible to realize higher-speed moving image encoding as compared with the case of the first embodiment. Incidentally, since the port A and the port B are read in parallel, the data (RDB1, RDB2, RDB3) read from the port B is input to the output selectors 209 and 210 and supplied from the memory control circuit 8 Two coefficients consecutive in the reading scan order are selected based on S6 and S7, and are output to the output terminals 211 and 212 for executing the second processing.

<< Third Embodiment >>
FIG. 11 is a block diagram showing a configuration of a signal processing device 301 according to the third embodiment of the present invention. A significant difference from the first and second embodiments is that a clock control circuit 30 for controlling the clocks of m memories (here, m = 3) constituting the storage unit 7 is newly provided.

  In the first and second embodiments, when writing and reading control to m memories are considered, it is not necessary for all the memories to operate in every clock cycle as shown in FIGS. In the write control, as shown in FIG. 5, when writing two coefficients to m (m> 2) memories in parallel, there is a memory that does not need to be written. In such a case, the clock control circuit 30 controls to stop the clock of the memory that does not need to be written. For example, in a cycle in which coefficient 0 (WD1 = 0) and coefficient 8 (WD3 = 8) are written, the memory control circuit is configured so that coefficient 0 is written to the first memory 701 and coefficient 8 is written to the third memory 703. 8, no write cycle occurs in the second memory 702. Therefore, the clock control circuit 30 supplies the clock stop signal CKE2 to the second memory 702 to stop the clock.

  Similarly, in the cycle in which writing does not occur in the first memory 701 and the third memory 703, clock stop signals CKE1 and CKE3 are supplied from the clock control circuit 30, and the first memory 701 and the third memory 703 The clock is stopped based on CKE1 and CKE3.

  As described above, in this embodiment, control is performed so that a clock is supplied to each memory only in a cycle in which writing and reading occur, so that it is possible to realize a signal processing device that further reduces power consumption. .

  Although the configuration in which the clock control circuit 30 is added to the configuration in FIG. 1 is shown in FIG. 11, a similar clock control circuit 30 may be added to the configuration in FIG.

<< Fourth Embodiment >>
FIG. 12 is a block diagram showing a configuration of a signal processing device 401 according to the fourth embodiment of the present invention. A significant difference from the first embodiment is that the storage unit 7 is not composed of a plurality of (m) memories, but is composed of one memory (first memory 701). The m memories in the first embodiment are memories having one read port, whereas the first memory 701 in this embodiment has at least one write port and is independent of each other. The difference is that it has at least two read ports capable of reading.

  The operation will be specifically described below with reference to the flowchart shown in FIG.

<Description of write control method>
In FIG. 12, the first memory 701 is a pair of two data (WDAU, WDAL) consecutive in the first scan order input from the input terminals 2 and 3 to form a WDA. One address is sequentially written. Write control to the first memory 701 at the port A is performed using the write address AA supplied from the memory control circuit 8 and the write enable signal WEA (step 400). Step 400 is repeated until all the block data is written (step 101). An example in which all data is written to the first memory 701 in this way is shown in FIGS. 14 (a) and 14 (b). FIG. 14A shows a memory map of the first memory 701 as a two-dimensional image, and it can be seen that two data are temporarily stored at one address. FIG. 14B shows a write address AA and write data WDA (WDAU, WDAL) in the first memory 701. FIG. 15 shows transitions of AA, WDAU, and WDAL in this case. As shown in FIG. 15, for example, the write control is performed in such a way that two data 0 and 8 consecutive in the order of write scan (first scan) are at address 0, the next data 16 and 24 are at address 8, and so on. It is memorized sequentially.

<Description of read control method>
The read scan is determined using the determination method described in the description of the first embodiment (step 102). If the read scan is confirmed, two data that are consecutive in the order of the read scan (second scan, third scan, or fourth scan) are read using port A and port B. Specifically, of the two pieces of data that are consecutive in the reading scan order, the earlier data is read from the port A, and the later data is read from the port B in parallel. This is shown in FIGS. FIG. 16 shows the read address AA and read data RDA of port A when the read scan is a zigzag scan, and the read address AB and read data RDB of port B. FIG. 17 shows the case where the read scan is a horizontal priority scan. FIG. 18 shows the read address AA and read data RDA of the port A, and the read address AB and read data RDB of the port B. FIG. 18 shows the read address AA and read data RDA of the port A when the read scan is the vertical priority scan. Show.

  More specifically, taking the case where the reading scan is a zigzag scan (FIG. 16) as an example, in order to read out two consecutive data in the zigzag scan order in parallel, (0, 1), (8, 16), ( 9, 2), (3, 10),..., (62, 63) need to be read in the order. In order to read out in this way, the read address AA required to read out the earlier data (0, 8, 9, 3,..., 62) out of the two data from the port A is 0, 0, 1, 3,..., 30 and so on, the data supplied from the memory control circuit 8 and stored in the respective addresses AA are (0, 8), (0, 8), (1, 9), (3, 11 ),..., (54, 62). Of the two data read out in this way, the earlier data (0, 8, 9, 3,..., 62) in the reading scan order is selected and output to the output terminal 11, so that the memory control circuit 8 Supplies a selection signal S4 to the first output selector 9, and the first output selector 9 sequentially selects the earlier data in the reading scan order based on S4 and outputs it to the output terminal 11. In parallel with the reading from the port A, the read address AB required for reading the later data (1, 16, 2, 10,..., 63) out of the two data from the port B is: The data supplied from the memory control circuit 8 such as 1, 8, 2, 2,... 31 and stored in the respective addresses AB are (1, 9), (16, 24), (2, 10 ), (2, 10),..., (55, 63). Of the two data read out in this way, the data (1, 16, 2, 10,..., 63) that is later in reading scan order is selected and output to the output terminal 12, so that the memory control circuit 8 Supplies a selection signal S5 to the second output selector 10, and the second output selector 10 sequentially selects the later data in the reading scan order based on S5 and outputs the selected data to the output terminal 12. The above operation is the parallel reading operation (second scan) in step 403. Since other operation control is the same as that in the first embodiment, detailed description thereof is omitted here.

  By performing the writing and reading control as described above, two pieces of data can be read in parallel even if the storage unit 7 is configured by one memory, and a high-speed scan can be changed. Further, unlike the first, second, and third embodiments, it is not necessary to configure the storage unit 7 with a plurality of (m) memories, and when the present invention is implemented in a semiconductor device or the like, In terms of the area occupied by the memory, a more inexpensive device can be realized.

<< Fifth Embodiment >>
FIG. 19 is a flowchart showing a signal processing method according to the fifth embodiment of the present invention. The signal processing apparatus is the same as that in the first embodiment (FIG. 1).

  In the present embodiment, a high-speed scan change is performed in the decoding of a moving image using the signal processing device 1, and will be specifically described below with reference to FIG.

  In the case of encoding, writing to the first memory 701, the second memory 702, and the third memory 703 is fixed to the first scan, and the reading scan is the first scan, the second scan, and the third scan In the case of decoding (this embodiment), the write scan is the second scan, the third scan, or the fourth scan, which is one of the scan and the fourth scan. Any one of them, and the reading scan is fixed to the first scan.

<Description of write control method>
First, DC coefficients (σA, σB, σC) of blocks (A, B, C) adjacent to a decoding target block necessary for selecting a write scan are input from the prediction coefficient input terminal 13, and a DC prediction code The setting of mode switching between normalization and DC / AC predictive coding is input from the setting terminal 14, and a write scan (second scan, third scan, or fourth scan) is determined (step 500). The determination method is the same as the method described in the description of the first embodiment.

  Next, in accordance with the write scan determined in step 500, two pieces of continuous data in the order of the write scan are written in m (here, m = 3) memories 701, 702, and 703 (step 503 to step 505). The writing is repeated until all the data is written (step 506). The write address and write data supplied to each memory in the write control are the same as those shown in FIGS.

<Description of read control method>
If it is determined in step 506 that all data has been written, the two consecutive coefficients in the first scan order are sequentially read from the m memories 701, 702, and 703, and the output selectors 9 and 10 By selecting two coefficients that are consecutive in one scan order, two data that are consecutive in the first scan order are sequentially output to the output terminals 11 and 12 (step 507). The address and read data supplied to each memory in the read control are the same as in FIG.

  Decoding processing is sequentially performed on the two data sequentially read in the first scan order in this way (step 508) until all data is read and decoded (step 509). Repeatedly.

  By using the signal processing method as described above, it is possible to increase the speed of data rearrangement (scan change) in the moving picture decoding process as in the case of the encoding process.

<< Sixth Embodiment >>
FIG. 20 is a flowchart showing a signal processing method according to the sixth embodiment of the present invention. In the present embodiment, the signal processing device 1 as shown in the first embodiment is shared between the case of encoding and the case of decoding, thereby attempting to realize low-speed and high-speed data rearrangement. Is.

  First, the set operation mode is determined (step 600). When the operation mode is encoding, in step 601, the same processing as in steps 100 to 110 shown in FIG. 2 is performed, and high-speed data rearrangement is performed. If the operation mode is decoding, the same processing as in steps 500 to 509 shown in FIG. 19 is performed in step 602, and high-speed data rearrangement is performed.

  As described above, high-speed data rearrangement in encoding and high-speed data rearrangement in decoding can be realized at low cost and at high speed by exclusively using one signal processing device.

<< Seventh Embodiment >>
FIG. 21 is a block diagram illustrating a configuration of an imaging system 501, for example, a digital still camera (DSC), according to the seventh embodiment of the present invention. A signal processing device 506 in FIG. 21 is one of the signal processing devices according to the first to sixth embodiments of the present invention.

  According to FIG. 21, the image light incident through the optical system 502 is imaged on the sensor 503. The sensor 503 is driven by the timing control circuit 509 to accumulate the imaged image light and photoelectrically convert it into an electrical signal. The electrical signal read from the sensor 503 is converted into a digital signal by an analog / digital converter (ADC) 504 and then input to an image processing circuit 505 including the signal processing device 506. The image processing circuit 505 performs image processing such as Y / C processing, edge processing, image enlargement / reduction, and image compression / expansion processing using the present invention. The image-processed signal is recorded or transferred to a medium in a recording / transfer circuit 507. The recorded or transferred signal is reproduced by the reproduction circuit 508. The entire imaging system 501 is controlled by a system control circuit 510.

  Note that the image processing in the signal processing device 506 according to the present invention is not necessarily applied only to a signal based on the image light imaged on the sensor 503 via the optical system 502, and is input as an electrical signal from an external device, for example. Needless to say, the present invention can also be applied to processing an image signal to be processed.

  The signal processing method and signal processing apparatus according to the present invention are inexpensive because they can easily realize high-speed data rearrangement (scan change) even when there are a plurality of write scans and a plurality of read scans. Therefore, it can be applied to an image encoding / decoding system that requires high-speed encoding and decoding.

  In particular, it is useful for camera-equipped mobile phones equipped with MPEG-4, which has been attracting attention as a low-bit-rate encoding technology in recent years, portable devices such as PDAs, etc. Furthermore, high-quality, long-time video at low bit rates It is also useful for DSCs for which the demand for recording is increasing, and AV devices that are linked with these portable devices.

It is a figure which shows the structure of the signal processing apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the signal processing method in the 1st Embodiment of this invention. (A)-(d) is the figure which showed the memory map of the some memory in the 1st Embodiment of this invention with a two-dimensional image, and showed four scanning orders. (A)-(c) is the figure which showed the memory map of the some memory in the 1st Embodiment of this invention with the one-dimensional image. It is the figure which showed the write-in control of the some memory in the 1st Embodiment of this invention about the case of a raster scan. It is the figure which showed the read-out control of the some memory in the 1st Embodiment of this invention about the case of a zigzag scan. It is the figure which showed the read-out control of the some memory in the 1st Embodiment of this invention about the case of a horizontal direction priority scan. It is the figure which showed the read-out control of the some memory in the 1st Embodiment of this invention about the case of a vertical direction priority scan. It is a figure which shows the structure of the signal processing apparatus which concerns on the 2nd Embodiment of this invention. It is a flowchart which shows the signal processing method in the 2nd Embodiment of this invention. It is a figure which shows the structure of the signal processing apparatus which concerns on the 3rd Embodiment of this invention. It is a figure which shows the structure of the signal processing apparatus which concerns on the 4th Embodiment of this invention. It is a flowchart which shows the signal processing method in the 4th Embodiment of this invention. (A) And (b) is the figure which each showed the memory map of the memory in the 4th Embodiment of this invention with the two-dimensional image and the one-dimensional image. It is the figure which showed the write-in control of the memory in the 4th Embodiment of this invention about the case of a raster scan. It is the figure which showed the reading control of the memory in the 4th Embodiment of this invention about the case of a zigzag scan. It is the figure which showed the reading control of the memory in the 4th Embodiment of this invention about the case of a horizontal direction priority scan. It is the figure which showed the read-out control of the memory in the 4th Embodiment of this invention about the case of a vertical direction priority scan. It is a flowchart which shows the signal processing method in the 5th Embodiment of this invention. It is a flowchart which shows the signal processing method in the 6th Embodiment of this invention. It is a figure which shows the structure of the imaging system in the 7th Embodiment of this invention. It is a figure which shows the structure of the conventional general image coding apparatus. It is a figure which shows the structure of the conventional signal processing apparatus. (A) And (b) is the figure which showed the memory map of the some memory in the conventional signal processing apparatus with a two-dimensional image, and showed two scanning orders. (A) And (b) is the figure which showed the memory map of the some memory in the conventional signal processing apparatus with the one-dimensional image. (A) And (b) is the figure which showed the write-in and read-out control of the some memory in the conventional signal processing apparatus, respectively. It is a figure explaining the conventional coefficient prediction method. (A)-(d) is a figure which shows concretely that the subject of this invention cannot be solved with the conventional signal processing method.

1, 201, 301, 401, 1001 Signal processing device 2, 3 Input terminals 4, 5, 6 Input selector 9, 10, 209, 210 Output selector 7 Storage unit 8 Memory control circuit 11, 12, 211, 212 Output Terminal 13 Prediction coefficient input terminal 14 Setting terminal 30 Clock control circuit 501 Imaging system 502 Optical system 503 Sensor 504 Analog to digital converter (ADC)
505 Image processing circuit 506 Signal processing device 507 Recording transfer circuit 508 Reproduction circuit 509 Timing control circuit 510 System control circuit 701 First memory 702 Second memory 703 Third memory

Claims (12)

A signal processing method for processing two-dimensional block data composed of a plurality of data,
Sequentially writing two pieces of data in the order of raster scanning in the row direction or the column direction to m memories (m is an integer of 3 or more) so as to store the block data;
In order to realize data rearrangement, a plurality of data are read from the m memories, and two consecutive data in order of the read scan selected in the zigzag scan, the horizontal priority scan or the vertical priority scan are sequentially And a selecting step. The signal processing method according to claim 1,
Performing an encoding process on the selected two pieces of data;
After the encoding process is completed, two consecutive data are sequentially selected from the plurality of data read from the m memories in the row- or column-direction raster scan order, and the selected two data And a step of performing a process for generating a reference image necessary for inter-frame predictive coding in moving picture coding. The signal processing method according to claim 1,
Each of the m memories is a memory having at least one write port and two or more read ports capable of reading independent from each other,
Data reading for rearranging the data is performed from a certain read port of each of the m memories, and continues in the order of the read scan selected from among the zigzag scan, the horizontal direction priority scan, and the vertical direction priority scan. Performing an encoding process on each piece of data;
In parallel with the encoding process, two consecutive data items in the order of raster scan in the row direction or the column direction are sequentially selected from a plurality of data read from the other one read port of each of the m memories. And a step of performing a process for generating a reference image necessary for inter-frame predictive coding in moving picture coding on the selected two pieces of data. . A signal processing method for processing two-dimensional block data composed of a plurality of data,
In order to store the block data, two consecutive data in the order of the write scan selected from zigzag scan, horizontal priority scan or vertical priority scan are sequentially transferred to m memories (m is an integer of 3 or more). A step to separate,
A step of reading a plurality of data from the m memories and sequentially selecting two consecutive data in a row scan or column scan order so as to realize data rearrangement. Signal processing method. The signal processing method according to claim 4 , wherein
The signal processing method further comprising the step of performing a decoding process on the two selected data. According to the set operation mode, data is rearranged using the signal processing method according to any one of claims 1 to 3 in the first operation mode, and charged in the second operation mode. 6. A signal processing method comprising rearranging data using the signal processing method according to item 4 or 5 . A signal processing apparatus for processing two-dimensional block data composed of a plurality of data,
Storage means composed of m (m is an integer of 3 or more) memories for storing the block data;
A plurality of input selectors for selecting data to be written to the m memories from two consecutive data in the row- or column-direction raster scan order;
A plurality of output selections for selecting two consecutive data in the order of the read scan selected from the zigzag scan, the horizontal direction priority scan, or the vertical direction priority scan among the plurality of data read from the m memories. And
Memory control means for writing and reading block data to and from the storage means and supplying selection signals to the plurality of input selectors and the plurality of output selectors, Signal processing device. The signal processing device according to claim 7 ,
Each of the m memories constituting the storage means is a memory having at least one write port and having two or more read ports capable of reading independently of each other. The signal processing device according to claim 7 ,
A signal processing apparatus, further comprising clock control means for individually controlling clocks supplied to the m memories. The signal processing device according to claim 9 , wherein
The clock control means supplies a clock to a memory in which writing occurs when writing two consecutive data in the row direction or column direction in the raster scan order to the m memories, and no writing occurs. The memory is controlled to stop the clock, and a read cycle is generated to select two consecutive data in the selected read scan order among zigzag scan, horizontal direction priority scan, or vertical direction priority scan. A signal processing apparatus, wherein a clock is supplied to a memory that performs a control, and a clock is stopped for a memory that does not generate a read cycle. An image processing circuit that includes the signal processing device according to any one of claims 7 to 10 and performs image processing;
A sensor for outputting an image signal to the image processing circuit;
An imaging system comprising: an optical system that focuses light on the sensor. The imaging system according to claim 11 , wherein
An imaging system, further comprising a converter that converts an image signal obtained from the sensor into a digital signal and supplies the digital signal to the image processing circuit.

Images