WO2015125960A1

WO2015125960A1 - Information processing device, digital camera, and processor

Info

Publication number: WO2015125960A1
Application number: PCT/JP2015/055050
Authority: WO
Inventors: 武昭杉村; 風見　一之
Original assignee: 株式会社ニコン
Priority date: 2014-02-24
Filing date: 2015-02-23
Publication date: 2015-08-27
Also published as: JPWO2015125960A1; JP6319420B2

Abstract

An application of an embodiment showing an example of the present invention is an information processing device equipped with a data memory (21), which temporarily stores a stream of data comprising a plurality of unit data; and a processor (43) that has a computation unit (220) and performs prescribed information processing on stream data that has been read from the data memory. This information processing device (200) is configured such that a matching memory (210), which has columns wherein a plurality of storage regions are consecutively provided, is provided between the data memory (21) and the computation unit (220).

Description

Information processing apparatus, digital camera and processor

The present invention relates to an information processing apparatus including a data memory that temporarily stores stream data including a plurality of unit data, and a processor that performs predetermined information processing on stream data read from the data memory. The present invention also relates to a digital camera equipped with this information processing apparatus.

There is an image processing apparatus as one of the information processing apparatuses as described above. Image processing apparatuses are widely used in digital cameras, video cameras, computers, printers, and the like that handle image data recorded on recording media of these cameras. In recent years, the volume of image data per image (one frame) has increased significantly due to demands for higher definition and higher image quality, and recording has been performed per unit time due to the demand for higher frames. The amount of image data to be processed has increased remarkably. In order to perform image processing on image data that has become enormous in this way without causing stress on the photographer or the like, the image processing apparatus is required to increase the processing speed.

Image processing apparatuses having various configurations have been proposed as means for speeding up image processing. For example, predetermined (for example, one image or one frame) image data is read from a plurality of image data recorded on an external recording medium such as a CompactFlash (registered trademark) card or an SD card (registered trademark). In an image processing apparatus including a data memory such as a DRAM that temporarily stores data and an image processing processor that performs predetermined image processing on image data temporarily stored in the data memory, between the data memory and the image processing processor, There is known an image processing apparatus provided with a read cache memory that reads a part of image data from a data memory and temporarily stores it, and a write cache memory that temporarily stores the part of the image data processed by the image processor. Yes.

Further, as a technique for speeding up the arithmetic processing for image data itself, an image processing apparatus that employs a single instruction multiple data stream (SIMD: Single Instruction Multiple Date stream) has been proposed (for example, Patent Document 1). reference). In this technique, a plurality of arithmetic units that execute arithmetic processing are provided in parallel, and these are simultaneously operated in parallel, thereby improving the processing speed.

JP 2002-358288 A

On the other hand, research on a new memory technology is progressing as a storage means for storing data, and one of them is a marching memory. Marching Memory is the 6th Workshop on Unique Chips and Systems held in conjunction with the 43rd International Annual Symposium (MICRO-43) of IEEE / ACM, an international symposium on semiconductor chip microarchitecture It was announced at UCAS-6. The contents of the announcement are described in the following documents which are the proceedings.
T. Watanabe and MJFlynn, "Marching Memory: designing computers to avoid the Memory Bottleneck", Workshop on Unique Chips and Systems, UCAS-6, December 2010 Atlanta, GA, pp44-47.

From the contents shown in the above document, the basic configuration and operation mode of the marching memory is outlined as follows. The marching memory is configured with a column in which a plurality of storage areas (referred to as cells) are continuously arranged as one unit. Both ends of the column become data (or instruction) input ports and output ports according to the setting, and data is sequentially input to the end of the column set as the input port. A plurality of data and the like input to the input port are sequentially transferred to adjacent cells and temporarily stored in each cell.

For example, in a marching memory having a column in which five cells (storage areas) are connected in the left-right direction, the left end side of the column is an input port, and five data of 1, 2, 3, 4, and 5 are stored in the input port. The case of inputting in order will be described. In this case, 1 is temporarily stored in the leftmost cell when 1 as the first data is input. The other cells are in a reset state (or a state in which the four data inputted immediately before these five data are inputted are temporarily stored). When the second data 2 is input, the data 1 that has been stored in the leftmost cell until then is sent to the second cell from the left end and moved to the second cell, and the leftmost cell contains 2 of the newly input data. Is temporarily stored. In the same manner, the input data is sequentially transferred and stored in each cell, and when all five data are input, 1, 2, 3, 4, and 5 are input to each cell from the right end side to the left end side of the column. Five data are temporarily stored.

Therefore, when the right end side of the column is set as an output port, it is temporarily stored when an output command for five data temporarily stored in the marching memory is issued or whenever new data is input. The data is output from the output port in the order of 1, 2, 3, 4, and 5. On the other hand, when the right end of the column is an input port and an output port, when an output command for five data temporarily stored in the marching memory is issued, the temporarily stored data is 5, 4, 3 , 2 and 1 in this order.

The object of the present invention is to provide a suitable application utilizing the characteristic operation mode of marching memory, which is such a novel memory technology.

An application according to an aspect exemplifying the present invention includes a data memory that temporarily stores stream data composed of a plurality of unit data (for example, image data in the embodiment) and a stream data read from the data memory having a computing unit. An information processing apparatus including a processor (for example, a digital signal processor in the embodiment) that performs information processing. In addition, the information processing apparatus according to the first aspect includes a marching memory as a register file of the processor. The marching memory is a unit marching memory that has a column in which a plurality of storage areas are connected in series, and that temporarily transfers a plurality of input unit data from one end of the column to adjacent storage areas and temporarily stores them in each storage area. A plurality are provided. The marching memory temporarily stores a plurality of input unit data in each storage area of the first unit marching memory, and the processor stores each storage area of the first unit marching memory based on the batch calculation processing instruction. A plurality of unit data temporarily stored in the storage unit are sequentially calculated by an arithmetic unit, and the processed unit data are temporarily stored in each storage area of the second unit marching memory.

Here, “stream data” in this specification is a data group composed of a plurality of unit data, and data in which adjacent unit data have a predetermined relationship with each other spatially and / or temporally. An aggregate of Specifically, image data of still images and moving images taken by digital cameras and digital video cameras, audio data recorded by these cameras and IC recorders, and temporal and spatial data measured by various measuring instruments An example of stream data is shown as an example of stream data.

For example, image data of a still image taken by a digital camera is an aggregate (data group) of unit data generated based on the detection signal of each pixel constituting the image sensor. The unit data constituting this data group is not random data having no relationship between data but data having a spatial relationship within a predetermined range. Based on this spatial relevance, adjacent unit data has a certain relevance with respect to feature quantities such as lightness and saturation, and the feature quantities change smoothly, for example. In addition, in the case of image data or still image data when still images are continuously shot, in addition to having the spatial relationship as described above with respect to unit data spatially adjacent within one frame, There is a temporal relationship between them. The same applies to audio data recorded by an IC recorder or the like, and is an aggregate of unit data whose frequency and intensity change smoothly in the time axis direction.

The information processing apparatus according to the second aspect includes a marching memory as a shared memory between the data memory and the plurality of processors. The marching memory is a unit marching memory that has a column in which a plurality of storage areas are connected in series, and that temporarily transfers a plurality of input unit data from one end of the column to adjacent storage areas and temporarily stores them in each storage area. A plurality of units are provided in parallel.

At this time, in the input port and / or the output port of each unit marching memory in the marching memory, an arrangement changing unit (for example, changing the arrangement of a plurality of unit data inputted and / or the arrangement of a plurality of unit data outputted) The ring register in the embodiment can be provided to constitute the information processing apparatus.

The arrangement changing means includes a ring register composed of a register group connected to an input port and / or an output port of each unit marching memory, and a sequencer for controlling the operation of the ring register (for example, read / write in the embodiment). The sequencer may be configured to control the operation of the ring register in accordance with the information processing mode executed by the information processing apparatus and change the arrangement of the plurality of unit data.

Here, the information processing mode can be appropriately set according to the system to which the information processing apparatus is applied. For example, as a typical example when the information processing apparatus is applied to a digital camera, an image compression mode, a vertical-horizontal conversion mode, a deconvolution mode, and the like are exemplified. As the image compression mode, a plurality of known modes can be considered.For example, image data compressed by thinning out the number of pixels from image data of all pixels stored in the data memory (stream data having a large data capacity) ( An example is a mode that changes to stream data with a reduced data capacity.

The information processing apparatus according to the third aspect includes a marching memory as a buffer memory of the processor. The marching memory is a unit marching memory that has a column in which a plurality of storage areas are connected in series, and that temporarily transfers a plurality of input unit data from one end of the column to adjacent storage areas and temporarily stores them in each storage area. A plurality of units are provided in parallel.

According to another aspect of the present invention, there is provided an information processing apparatus according to any one of the above, an image input system having an image sensor and inputting image data as stream data to the information processing apparatus, and processing by the information processing apparatus. A digital camera provided with an image output system for outputting the processed image data.

As described above, the marching memory uses a column in which a plurality of storage areas (cells) are connected as a unit, and unit data input to the input port is sequentially transferred to adjacent storage areas and temporarily stored in each cell. The At this time, the input and movement speed of the unit data can correspond to, for example, a CPU reference clock, and a high-speed reading operation is possible. The data group targeted by the information processing apparatus according to the aspect of the present invention is stream data composed of a plurality of unit data, and is a data group having a predetermined relationship between adjacent data. That is, unlike random data that requires addressing for each unit data to be read, it is a data group that matches the operation mode of the marching memory.

Therefore, according to the information processing apparatus of the aspect of the present invention and the digital camera including the information processing apparatus, it is possible to provide a suitable application utilizing the characteristic operation form of the marching memory.

It is a block diagram which illustrates the architecture of the signal processing system in a digital camera shown as an example of application of this invention. It is a block diagram of the outline | summary of the image processing apparatus of the 1st aspect in 1st Embodiment. It is a block diagram of the outline | summary of the image processing apparatus of the 2nd aspect in 1st Embodiment. It is a block diagram of the outline | summary of the image processing apparatus of 2nd Embodiment. It is explanatory drawing for demonstrating the effect | action of the ring register in the image processing apparatus of 2nd Embodiment. It is a block diagram which put together roughly the signal processing system in an image processing device for explaining other examples of application of the present invention.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. The present invention can be applied to an information processing apparatus that processes various types of information. In the present embodiment, a case where the present invention is applied to an image processing apparatus of a digital camera will be described as an example. A block diagram illustrating the architecture of a signal processing system in a digital camera is shown in FIG.

This signal processing system includes an image input / output unit 1, a data memory unit 2, a CPU core unit 3, a DSP array unit 4, a CPU / DSP synchronous communication mechanism unit 5, and buses 61 to 66 that connect the respective units. .

The image input / output unit 1 is a control unit that controls input / output of image data to / from an external recording medium such as an image input system, an image output system, a CF card, or an SD card (registered trademark) (not shown). It is. The image input / output unit 1 is provided with an I / O circuit 11 that exchanges image data with the above-described units, a function block (IP) 12 that executes codec and decoding of image data, and the like, and a stream I / O bus 61. Via an image input system, an image output system, and an external recording medium.

The image input system is a system for inputting original data of images (still images or moving images) taken in various formats by a digital camera. For example, an image sensor such as a CMOS or a CCD, and a subject on the image sensor. And an imaging lens that forms an image. The image output system is a system for outputting image data processed by the image processing apparatus to a liquid crystal display panel or an external output terminal (see FIG. 6).

The image input / output unit 1 is connected to the main bus 62 via a stream I / O bus 61 and a bus bridge. A data memory unit 2, a CPU core unit 3, and a DSP array unit 4 are connected to the main bus 62. That is, in addition to the data memory unit 2 being connected to the main bus 62, the CPU data bus 63 of the CPU core unit 3 is connected via a bus bridge, and the DSP data of the DSP array unit 4 is similarly connected via the bus bridge. A bus (stream bus) 64 is connected. The stream I / O bus 61 and the DSP data bus 64 are also directly connected via a bus bridge.

The data memory unit 2 includes a data memory 21 and a memory controller 22. The data memory 21 is a storage element that temporarily stores image data input via the image input / output unit 1 from an image input system or an external recording medium. For example, a DRAM (Dynamic Random Access Memory) having a storage capacity of the order of gigabytes. Is used. The memory controller 22 is a control circuit that controls the writing of image data to the data memory 21 and the reading of image data temporarily stored in the data memory 21.

The CPU core unit 3 is a control unit that controls the operation of the digital camera based on a program set and stored in advance, and FIG. 1 shows a configuration example of a parallel arithmetic module in which two processing circuits are provided in parallel. The CPU core unit 3 includes an instruction RAM (Instruction RAM) 31, a processing circuit connected in parallel to the instruction RAM 31 via a CPU instruction bus 65, and a DMAC (Direct that is connected to each processing circuit via a CPU data bus 63. It is configured to have a Memory (Access Controller) 35, SRAM (Static Random Access Memory) 36, etc. The instruction RAM 31 is a RAM (Random Access Memory) in which a program composed of a plurality of processing instructions is set and stored in advance.

Each processing circuit provided in parallel includes an instruction cache 32 that temporarily holds a processing instruction of each step of the program, a CPU 33 that executes the processing instruction, and a data cache that temporarily holds arithmetic data referred to when the processing instruction is executed. 34. The instruction cache 32 is connected to the instruction RAM 31 via the CPU instruction bus 65, and the processing instructions of the program stored in the instruction RAM 31 are read as the processing steps progress and are temporarily held in the instruction cache 32. The data cache 34 is connected to a CPU data bus 63, and arithmetic data to be referred to when the CPU 33 executes processing is read from the DMAC 35, SRAM 36, etc. connected to this bus as the processing steps progress. It is temporarily held in the cache 34. Based on the processing instruction temporarily held in the instruction cache 32, the CPU 33 refers to the operation data temporarily held in the data cache 34 and executes processing.

The DSP array unit 4 is an image processing apparatus that performs predetermined image processing based on a program set and stored in advance. FIG. 1 shows a configuration example of a parallel processing module in which a plurality of digital signal processors (DSPs) 43, 43... Are provided in parallel. 1 shows a configuration example in which a common shared memory 44 is provided for a plurality of

digital signal processors

43, 43...

The DSP array unit 4 includes an instruction RAM 41 in which a program including a plurality of processing instructions is set and stored in advance, an image processing circuit connected in parallel to the instruction RAM 41 via a DSP instruction bus 66, and a DSP data bus 64 connected to the image processing circuit. And a DMAC 45, an SRAM 46, and the like connected via each other.

Each of the image processing circuits provided in parallel has an instruction cache (I $) 42 that temporarily holds processing instructions, and a digital signal processor (hereinafter abbreviated as DSP) 43 that executes the processing instructions, A plurality of

DSPs

43, 43... Share processing and execute image processing in parallel. The

instruction caches

42, 42... Are connected in parallel to the instruction RAM 41 via the DSP instruction bus 66, and processing instructions to be shared by the DSPs 43 are read and temporarily stored in the instruction caches 42. .

DSP

43, 43... Are connected to a shared memory 44, and each DSP can read image data from the shared memory 44 or write processed image data to the shared memory 44.

The shared memory 44 is connected to the DSP data bus 64 and is connected to the main bus 62 and the stream I / O bus 61 via a bus bridge. Therefore, the shared memory 44 can exchange image data with the data memory 21, the DMAC 45, the SRAM 46, the image input system, and the like via these buses.

The shared memory 44 reads predetermined image data from the above-described units (for example, the data memory 21) based on the processing instruction read from the instruction RAM 41, and temporarily stores it in the memory. The

DSPs

43, 43... Read out a part of the image data shared by them from the shared memory 44 based on the processing instructions read from the instruction RAM 41 and temporarily held in the

instruction caches

42, 42. Execute the process. The image data processed by the

DSPs

43, 43... Is temporarily stored in the shared memory 44, and sequentially transferred to the data memory 21 for storage.

The CPU / DSP synchronous communication mechanism unit 5 is a mechanism that adjusts the timing of processing executed between the CPU core unit 3 and the DSP array unit 4, and is provided between the CPU 33 and the DSP 43. The control interrupt controller 51 includes a shared RAM 52 provided between the CPU data bus 63 and the DSP data bus 64. For example, when a process that requires image processing occurs during execution of the program, the CPU 33 writes data such as the address of image data to be processed and the processing content to the shared RAM 52, and the DSP 43 via the synchronization control interrupt controller 51. The interrupt processing execution command is output to. The DSP 43 reads the designated image data, executes image processing, writes the address of the processed image data into the shared RAM 52, and signals to the CPU 33 that the interrupt processing has been completed via the synchronous control interrupt controller 51. Is output. Thereby, the CPU core unit 3 and the DSP array unit 4 perform parallel processing efficiently.

In the signal processing system of the digital camera schematically structured as described above, image data recorded on an external recording medium (not shown) or an image read from the external recording medium and temporarily stored in the data memory 21 The data is a data group composed of a large number of unit data constituting an image, and is an aggregate of data in which adjacent unit data has a predetermined relationship spatially, that is, stream data. The DSP 43 that performs predetermined image processing on image data has an arithmetic unit such as an arithmetic logic unit (ALU) or a floating point number processing unit (FPU) in the processor. Configured.

In the signal processing system of the present digital camera, a plurality of storage areas are provided between an external recording medium (not shown) in which image data is temporarily stored, the data memory 21, and the arithmetic units of the

DSPs

43, 43. Is provided with a marching memory having columns connected to each other. Hereinafter, specific modes of the image processing apparatus using the marching memory will be described in detail.

The image processing apparatus according to the first embodiment includes a marching memory in the DSP 43, that is, the marching memory is built in the DSP 43 and is integrally configured. An image processing apparatus 100 according to a first aspect included in this embodiment will be described with reference to FIG. FIG. 2 is a block diagram mainly showing a DSP (digital signal processor) 43 in the image processing apparatus 100. The memory controller 22 in the data memory unit 2, the instruction RAM 41 in the DSP array unit 4, the instruction cache 42, and the sharing are shown in FIG. Description of the memory 44 and the like is omitted.

The image processing apparatus 100 includes a data memory 21 that temporarily stores image data, and a DSP 43 that performs predetermined information processing on the image data read from the data memory 21. The DSP 43 of this aspect includes a register file 110 that temporarily holds data necessary for arithmetic processing, an arithmetic unit 120 such as an ALU or FPU that executes arithmetic processing using the data held in the register file 110, and an arithmetic result. The accumulator 130 temporarily holds, and a buffer memory 150 including a load buffer 151 and a store buffer 152 provided between the data memory 21 and the register file 110. A marching memory is used as the load buffer 151 and the store buffer 152.

The load buffer 151 and the store buffer 152 are configured by providing a unit marching memory formed by a column in which a plurality of cells (storage areas) are connected in the row direction as one buffer, and providing a plurality of them in parallel. For example, a unit marching memory formed by a column in which 256 cells are connected in the row direction is used as one buffer, and 256 units are provided in parallel in the column direction so that a load buffer 151 including 256 buffers and a store buffer are provided. 152 is formed. Each cell of the marching memory temporarily stores unit data constituting image data, and the load buffer 151 and the store buffer 152 are each configured to be capable of temporarily storing (temporarily storing) 256 × 256 unit data.

The load buffer 151 accesses the data memory 21 in response to a DMA (Direct Memory Access) transfer command from the command RAM 41, reads a predetermined range of image data, and temporarily holds it. For example, image data for 256 pixels in the X direction and 256 pixels in the Y direction is read from image data of 4 megapixels in the horizontal direction (X direction) × 3 megapixels in the vertical direction (Y direction), Temporarily stored in 256 buffers. At this time, unit data for 256 pixels of the X1 line read first in the X direction is sequentially sent from the first cell of the B1 buffer to the 256th cell, temporarily held, and then read out for 256 pixels of the X2 line. Minute unit data is sequentially sent from the first cell of the B2 buffer to the 256th cell and temporarily held. Thereafter, the read operation is repeated in the same manner, and the B1 buffer and the B256 buffer are sequentially temporarily held. Each register temporarily holds unit data of 256 pixels adjacent to each other along the X direction line of the image data.

The register file 110 reads unit data from the load buffer 151 in response to a load instruction from the instruction RAM 41 and stores it in a predetermined register, for example, the R1 register. Similarly, data corresponding to the image processing to be executed is read from the DMAC 45 or the SRAM 46 and stored in a predetermined register, for example, the R2 register.

The arithmetic unit 120 executes arithmetic processing using the unit data stored in the R1 register and the data stored in the R2 register in accordance with the arithmetic instruction from the instruction RAM 41, and registers the arithmetic result via the accumulator 130. The data is output to the file 110 and stored in a predetermined register, for example, the R3 register.

The register file 110 outputs the unit data after the arithmetic processing in accordance with the store instruction from the instruction RAM 41 and writes the unit data of the arithmetic result in the store buffer 152.

At this time, the unit data arranged in the column of each buffer (unit marching memory) in the load buffer 151 is sequentially loaded into the register file 110 and processed by the calculator 120. Further, the unit data after the arithmetic processing is stored in the order in which the arithmetic processing is performed, and is temporarily held in the column of each buffer (unit marching memory) in the store buffer 152. For example, the unit data arranged in the column of the B1 buffer in the load buffer 151 is read into the register file 110 sequentially from the first cell and is arithmetically processed by the arithmetic unit 120. The unit data after the arithmetic processing is B1 in the store buffer 152. The data is sequentially sent from the first cell to the 256th cell and written in the buffer column. Thereafter, the same applies to the B2 buffer to the B256 buffer.

As the arithmetic processing by the arithmetic unit 120 proceeds, the image data temporarily stored in the load buffer 151 sequentially decreases, and the image data after the arithmetic processing (after image processing) sequentially increases in the store buffer 152. The load buffer 151 and the store buffer 152 perform DMA transfer of image data to and from the data memory 21 in parallel with the progress of the arithmetic processing, that is, in the background of execution of the arithmetic processing.

In the image processing apparatus 100 having such a configuration, image data for one line in the X direction is temporarily held from the first cell to the 256th cell of each buffer, and image data from the X1 line to the X256 line is transferred from the B1 buffer. The data is temporarily stored in the B256 buffer (from the first unit marching memory to the 256th unit marching memory). Then, the computing unit 120 uses the first cell to the 256th cell of the B1 buffer, the first cell to the 256th cell of the B2 buffer, the first cell to the 256th cell of the B3 buffer, and the first cell of the B256 buffer. As in the 256th cell, unit data temporarily held in the cells of each buffer is sequentially loaded and processed.

Image data is a collection of data in which unit data temporarily stored in adjacent cells are spatially related to each other (for example, detected by adjacent pixels), and is stream data. Therefore, unlike random data, it is not necessary to address and read / write each unit data, and unit data for one line is sequentially sent in the buffer of each unit marching memory and stored in the cell, or Read it out. In addition, data transfer between adjacent cells, that is, unit data write and read operations can be performed at high speed in synchronization with the clock pulse of the DSP 43.

Furthermore, since the load buffer 151 and store buffer 152 and the data memory 21 are physically separated from each other, it takes time to exchange image data, which may be one obstacle for speeding up and increasing the efficiency of image processing. However, in the image processing apparatus 100, the transfer of the image data between the load buffer 151 and the store buffer 152 and the data memory 21 is performed in the background in parallel with the period during which the arithmetic unit 120 executes the arithmetic processing. Done in

Therefore, according to the image processing apparatus 100, it is possible to execute image processing at high speed and high efficiency by utilizing the characteristic operation form of the marching memory.

The configuration in which the image data of X1 to X256 lines are temporarily stored in the B1 to B256 buffer is illustrated, but each of the B1 to B256 buffers can temporarily store image data of an arbitrary line and is freed according to processing. Can use the buffer. For example, the image data of the X1 line can be temporarily stored in the B35 buffer, and the image data of the X2 line can be temporarily stored in the B42 buffer.

Next, a second aspect of the image processing apparatus 200 included in the first embodiment will be described with reference to FIG. 3 is a block diagram mainly showing a DSP (digital signal processor) 43 in the image processing apparatus 200. Similar to FIG. 2, the memory controller 22 in the data memory unit 2, the instruction RAM 41 in the DSP array unit 4, and the instruction cache are shown. 42, the shared memory 44, etc. are omitted.

The image processing apparatus 200 includes a data memory 21 that temporarily stores image data, and a DSP 43 that performs predetermined information processing on the image data read from the data memory 21. The DSP 43 according to this aspect includes a register file 210 that temporarily holds data necessary for arithmetic processing, an arithmetic unit 220 such as an ALU or FPU that performs arithmetic processing using the data held in the register file 210, and an arithmetic result. An accumulator 230 that temporarily holds and an address register file 250 that temporarily holds address information of image data temporarily stored in each register of the register file 210 are configured. In the image processing apparatus 200 of this aspect, a marching memory is used as the register file 210.

The register file 210 includes a unit marching memory formed by a column in which a plurality of cells (storage areas) are connected in the row direction as one register, and a plurality of the unit marching memories are provided in parallel. In other words,

unit marching memories

211, 212, 213,..., 21N provided in parallel constitute R1, R2, R3,. For example, in the register file 210 in which 32 unit marching memories having 256 cells in the row direction are provided in parallel in the column direction, 32 registers each capable of holding 256 unit data are provided.

The address register file 250 is a register file that temporarily holds the address data of the image data temporarily held in each register of the register file 210. A1, A2, A2, A2, A2, A2, A2, A2, A2, A2, A3,..., AN address register. For example, it has 32 registers.

In the image processing apparatus 200 of this aspect, loading, storing, and arithmetic processing for one line of the register are executed by one batch instruction. That is, one line of image data consisting of 256 unit data is loaded into the register by one batch load instruction, and one line of image data is arithmetically processed by one batch operation instruction. The image data for one line is stored in the data memory 21 by the instruction. By using a marching memory for the register and executing such batch instructions, the instruction efficiency can be greatly increased. Specifically, batch instructions and processing are executed as follows.

The register file 210 accesses the data memory 21 in response to a batch load command from the command RAM 41, reads a predetermined range of image data, and stores it in each register. For example, image data of 256 pixels in the horizontal direction × 10 pixels in the vertical direction is read from the image data of 4 megapixels in the horizontal direction × 3 megapixels in the vertical direction, and stored in 10 registers. At this time, the unit data for 256 pixels of the X1 line loaded based on the first batch load instruction is sequentially sent from the first cell of the R1 register to the 256th cell and stored, and based on the next batch load instruction. The unit data for 256 pixels of the loaded X2 line is sequentially sent from the first cell of the R2 register to the 256th cell to be stored. Thereafter, similarly, the load operation based on the batch load instruction is repeated, and sequentially stored in the R1 register to the R10 register. Each register of R1 to R10 stores unit data for 256 pixels adjacent along the X-direction line of the image data.

At this time, the address register file 250 stores the address information of the image data read based on the batch load instruction and stored in each register. For example, when image data of the X1 line is stored in the first cell to the 256th cell of the R1 register, the address of the X1 line is stored in the A1 register. Further, when the image data of the X1 line after the arithmetic processing described below is stored in the R21 register, the address of the X1 line after the arithmetic processing is stored in the A21 register.

Also, in the register file 210, data corresponding to the image processing to be executed is read from the DMAC 45 or the SRAM 46 in response to the execution of the batch load instruction from the instruction RAM 41, and stored in each register. For example, 256 data corresponding to the X1 line image processing is read from the DMAC 45 according to the execution of the batch load instruction and stored in the R11 register, and the X2 line image processing is performed according to the execution of the batch load instruction. Corresponding 256 pieces of data are read out and stored in the R12 register. Similarly, 256 data corresponding to the image processing of the X10 line are read and stored in the R20 register.

The computing unit 220 performs batch processing on the image data stored in each register in response to a batch computation command from the command RAM 41. For example, when the batch operation instruction from the instruction RAM 41 is a batch operation instruction for adding the image data stored in the R1 register and the data stored in the R11 register and storing them in the R21 register, 220 executes arithmetic processing as follows.

The arithmetic unit 220 adds the unit data stored in the first cell of the R1 register based on the batch operation instruction and the data stored in the first cell of the R11 register, and stores the result in the first cell of the R21 register. To do. Next, the unit data stored in the second cell of the R1 register and the data stored in the second cell of the R11 register are added and stored in the second cell of the R21 register. Similarly, the unit data stored in the nth cell of the R1 register and the data stored in the nth cell of the R11 register are added and stored in the nth cell of the R21 register. This process is executed from the first cell to the 256th cell with one instruction.

When a processing instruction for the image data stored in the R2 register (X2 line) is issued as the next batch operation instruction from the instruction RAM 41, the arithmetic unit 220 performs processing performed on the image data stored in the R1 register. The arithmetic processing is executed in the same way as Thereafter, when the batch calculation is repeated, for example, the same calculation process is executed for the R3 to R10 registers.

Here, the unit data before the arithmetic processing forming the X1 line to the X10 line are sequentially stored in the first cell to the 256th cell of the R1 register to the R10 register, respectively. Each register constituting the register file 210 is a unit marching memory, and the data stored in the first cell to the 256th cell is moved forward while maintaining the data arrangement order. Therefore, the unit marching memory is simply forwardly operated at the time of execution of the arithmetic processing, so that the unit data stored in the first cell to the 256th cell can be sequentially sent to the arithmetic unit 220 and the arithmetic processing can be executed. The same applies to the case where the unit data after the arithmetic processing is written to the first cell to the 256th cell of the R21 register to the R30 register. The unit marching memory is sequentially operated to send the unit data to the first cell to the second cell. Unit data is stored in order in 256 cells.

As the arithmetic processing by the arithmetic unit 220 proceeds, the image data before the arithmetic processing stored in the R1 register to the R10 register and the data stored in the R11 register to the R20 register are sequentially reduced and stored in the R21 register to the R30 register. The stored image data after the arithmetic processing sequentially increases. The DSP 43 issues a DMA transfer of image data between the register file 210 and the data memory 21 by issuing a batch store instruction in parallel with the progress of the arithmetic processing by the arithmetic unit 220.

In the image processing apparatus 200 having such a configuration, image data including 256 unit data for one X direction line is stored in the first cell to the 256th cell of each register by one batch load instruction. Image data composed of unit data of 2560 from the X1 line to the X10 is stored from the R1 register to the R10 register by the batch load instruction. In the arithmetic unit 220, image data for one line composed of 256 unit data is processed by one batch operation instruction, and image data for 10 lines stored in the R1 register to R10 register by 10 batch operation instructions. It is calculated and stored in R21 to R30.

Image data is a collection of data in which unit data stored in adjacent cells are spatially related to each other (for example, detected by adjacent pixels), and is stream data. Therefore, unlike the random data, it is not necessary to perform addressing for each unit data one by one and read and write, and the time required for addressing, data search, etc. can be eliminated.

Further, in the image processing apparatus 200, the arithmetic unit 220 performs one instruction for each of the load, operation, and store processes of a unit data group (256 unit data groups in the above embodiment) corresponding to one X-direction line. Process in batch. Since the unit data group for one line is moved forward while maintaining the data arrangement order, the unit data group can be moved in order by simply operating the unit marching memory at the time of loading, calculation processing, or storing. It is possible to load, perform arithmetic processing, and store the data group of the arithmetic results in order. Data transfer movement between adjacent cells in the unit marching memory, that is, unit data write and read operations can be performed at high speed in synchronization with the reference clock pulse. Furthermore, load, store, and arithmetic processing for one register line are executed by one batch instruction, so that the instruction efficiency can be greatly improved.

Also, since the register file 210 and the data memory 21 are physically separated, it takes time to exchange image data, which can be an obstacle to speeding up and increasing the efficiency of image processing. However, in the image processing apparatus 200, the transfer of the image data between the register file 210 and the data memory 21 is performed in the background in parallel with the period during which the arithmetic unit 220 executes the arithmetic processing.

Therefore, according to the image processing apparatus 200, it is possible to execute image processing at high speed and high efficiency by utilizing the characteristic operation form of the marching memory.

In the above description, for simplification of explanation, the image data of the X1 to X10 lines are stored in the R1 to R10 registers in the register file 210, the processing data of the X1 to X10 lines are stored in the R11 to R20 registers, and R21 to R30. The configuration in which the image data after processing of the X1 to X10 lines is stored in the register is illustrated. However, the R1 to R32 registers can store image data, processed data, and processed image data of arbitrary lines, respectively, according to the allocation at that time, and an unused register can be allocated and used according to the processing. it can. For example, the image data of the X1 line and the X2 line are stored in the R8 register and the R13 register, the processing data of the X1 line and the X2 line are stored in the R2 register and the R25 register, and the X1 line and the X2 line are stored in the R5 register and the R32 register. The processed image data can be stored.

The same applies to the address register 250, and the A1 to A32 registers can each store the address of the image data of an arbitrary line and the address of the processed image data according to the assignment at that time, and are free according to the processing. Registers can be allocated and used. For example, the address of the X3 line image data stored in the R9 register can be stored in the A2 register, and the processed address of the X3 line stored in the R22 register can be stored in the A11 register.

Next, an image processing apparatus according to the second embodiment will be described. The image processing apparatus according to the second embodiment includes a data memory 21 and a plurality of DSPs (digital signal processors) 43, 43,... 43, and the data memory 21 and the

DSPs

43, 43,. A shared memory 44 using a marching memory is provided between the two.

The image processing apparatus 400 of the present embodiment will be described with reference to FIG. FIG. 4 is a block diagram mainly showing the shared memory 44 in the image processing apparatus 400, and the description of the memory controller 22 in the data memory unit 2, the instruction RAM 41, the instruction cache 42, etc. in the DSP array unit 4 is omitted. .

The image processing apparatus 400 includes a data memory 21 that temporarily stores image data, a DSP 43 that performs predetermined information processing on image data read from the data memory 21, and a shared memory 44. The shared memory 44 includes a data storage unit 401 that temporarily stores image data and the like, and a data control unit 402 that controls the flow of image data and the like input to and output from the data storage unit 401.

The data storage unit 401 includes an MM array 410 that temporarily stores image data, and an MM label management controller 420 that temporarily stores information of image data temporarily stored in the MM array (marching memory array) 410. The The data control unit 402 is a read / write adjustment circuit 430 provided between the data memory 21 and

DSPs

43, 43,... 43 and the MM array 410, and a read that controls the operation of the read / write adjustment circuit 430. / Write sequencer 440.

The MM array 410 includes a unit marching memory formed by a column in which a plurality of cells (storage areas) are connected in the row direction as a single memory, and a plurality of them are provided in parallel. In other words,

unit marching memories

411, 412, 413,..., 41N provided in parallel constitute C1, C2, C3,. The capacity of the MM array is set according to the data capacity of the image data captured by the digital camera. For example, 256 × m unit marching memories having 256 × n cells in the row direction are provided in parallel in the column direction. Configured. In the present embodiment, a configuration in which one end of a column of each unit marching memory is used as a data input / output port is illustrated.

The read / write adjustment circuit 430 reads and writes image data between the MM array 410 and the data memory 21, and transfers image data between the MM array 410 and the

DSPs

43, 43,. This is a circuit for adjusting the flow of image data when it is performed. The read / write adjustment circuit 430 shown in FIG. 1 includes load /

store units

431, 431,... 431 provided corresponding to the

DSPs

43, 43,. A ring register 433 in which first to Nth registers corresponding to 41N are interconnected in a ring shape, and a port connection controller 432 that controls connection between the load / store unit 431 and the ring register 433 Is done.

The load / store unit 431 has one end connected to the

DSPs

43, 43,..., 43 and the data memory 21 via the global memory transfer bus 68 and the other end connected to the port connection controller 432. The ring register 433 has one end of each of the first to Nth registers connected to an input / output port of the corresponding C1 to CN memory, and the other end connected to a port connection controller 432.

The read / write sequencer 440 controls the operations of the load / store unit 431, the port connection controller 432, and the ring register 433 of the read / write adjustment circuit 430 according to the contents of the image processing executed by the image processing apparatus 400. There are various known image processes executed by the image processing apparatus 400, and the control mode (processing mode) of the read / write adjustment circuit 430 controlled by the read / write sequencer 440 differs depending on the processing contents. Become. Here, (1) a copy mode in which image data stored in the data memory 21 is temporarily stored in the MM array 410 as it is, and (2) X in which the horizontal direction and the vertical direction of the image data stored in the data memory 21 are reversed. A -Y conversion mode and (3) a compression mode for reducing the number of image data stored in the data memory 21 will be described as representative examples of processing modes.

(1) Copy mode The copy mode is a processing mode in which image data stored in the data memory 21 is temporarily stored in the MM array 410 as it is. In the copy mode, the read / write sequencer 440 controls the read / write adjustment circuit 430 as follows. Now, it is assumed that the first load / store unit 431 reads the image data of the X1 line to the X5 line and temporarily stores (writes) them in the C1 memory to the C5 memory of the MM array 410.

At this time, the read / write sequencer 440 first causes the first load / store unit 431 to specify the address of the X1 line and read the image data for one X direction line. The port connection controller 432 is connected to the first load / store unit 431 and a first register that is a register corresponding to the C1 memory in the ring register 433. The ring register 433 is set to output data input from the port connection controller 432 to the memory without performing data movement between the registers. As a result, the image data of the X1 line read from the data memory 21 passes through the first registers of the first load / store unit 431 to the port connection controller 432 to the ring register 433, and is temporarily stored in the C1 memory of the MM array 410. Remembered.

Next, the read / write sequencer 440 causes the first load / store unit 431 to specify the address of the X2 line and read the image data for one X direction line. The port connection controller 432 is connected to the first load / store unit 431 and the second register of the ring register 433. The ring register 433 maintains a setting that does not move data between registers. As a result, the image data of the X2 line read from the data memory 21 passes through the second registers of the first load / store unit 431 to the port connection controller 432 to the ring register 433, and is temporarily stored in the C2 memory of the MM array 410. Remembered. The same applies to the X3 line to X5 line. By sequentially switching the connection settings of the port connection controller 432, the image data of the X3 line to X5 line is temporarily stored in the C3 memory to C5 memory of the MM array 410.

In the MM label management controller 420, address information of image data temporarily stored in the C1 memory to the C5 memory of the MM array 410 is temporarily stored as a label. For example, the image data temporarily stored in the C1 memory temporarily stores a label indicating that it is the X1 line image data in the original image data stored in the data memory 21. The same applies to the C2 memory to C5 memory.

Note that image data of an arbitrary line can be temporarily stored in the C1 to CN memories of the MM array 410. For example, the X1 line image data can be temporarily stored in the C3 memory, and the X2 line image data can be temporarily stored in the C6 memory.

The above has described the case where the image data stored in the data memory 21 is temporarily stored in the MM array 410. However, the image data temporarily stored in the MM array 410 is transferred to the DSP 43 or written into the data memory 21. The same applies to time. That is, the read / write sequencer 440 sequentially switches the connection between the memory of the MM array 410 to be read and the load / store unit 431 that performs transfer to the DSP 43, and transfers image data in a predetermined range to the DSP 43.

(2) XY Conversion Mode In the XY conversion mode, the horizontal direction (X direction) and the vertical direction (Y direction) in the image data stored in the data memory 21 are reversed and temporarily stored in the MM array 410. This is the processing mode. In the XY conversion mode, the read / write sequencer 440 controls the read / write adjustment circuit 430 as follows. Now, a case will be described in which image data of the X1 line to X5 line is read out to the first load / store unit 431 and temporarily stored in the MM array 410 as the Y1 line to Y5 line.

At this time, the read / write sequencer 440 first causes the first load / store unit 431 to read the image data for one line in the X direction by designating the address of the X1 line. The port connection controller 432 is connected to the first load / store unit 431 and the first register in the ring register 433. The ring register 433 is set so that data is sent and moved between registers each time unit data constituting the X1 line is input.

FIG. 5 shows the operation of the ring register 433 when one X-direction line is composed of four unit data. At this time, the four unit data of the X1 line read from the data memory 21 are sequentially moved by the ring register 433, and the first unit data of the X1 line is transferred to the fourth register corresponding to the C4 memory. The unit data is moved to the third register corresponding to the C3 memory, the third unit data is moved to the second register corresponding to the C2 memory, and the fourth unit data is moved to the first register corresponding to the C1 memory. Then, after the image data for one line in the X direction is stored in the fourth to first registers of the ring register 433, the unit data stored in each register is written into the MM array 410. As a result, the X1 line is XY-converted to the Y1 line.

Next, the read / write sequencer 440 causes the first load / store unit 431 to specify the address of the X2 line and read the image data for one X direction line. The settings of the port connection controller 432 and the ring register 433 are the same. The four unit data of the X2 line read from the data memory 21 are sequentially moved by the ring register 433, the first unit data of the X2 line is transferred to the fourth register, the second unit data is transferred to the third register, The third unit data is moved to the second register, and the fourth unit data is moved to the first register. Then, after the image data for one line in the X direction is stored in the fourth to first registers of the ring register 433, the unit data stored in each register is written into the MM array 410. As a result, the X2 line is XY converted to the Y2 line. At this time, each unit data of the Y1 line temporarily stored in the first cell of the C4 memory to the C1 memory is sent and moved to the second cell together with the writing operation of each unit data of the Y2 line.

XY lines are similarly converted for the X3 line to X5 line and temporarily stored in the MM array 410 as the Y3 line to Y5 line.

In the MM label management controller 420, information of image data temporarily stored in the C1 memory to the C4 memory of the MM array 410 is temporarily stored as a label. For example, the label indicating that the image data temporarily stored in the C1 memory is the image data corresponding to the Y4 line in the original image data stored in the data memory 21 is temporarily stored. The same applies to the C2 memory to C4 memory.

In the above description, the case where the image data stored in the data memory 21 is XY-converted and temporarily stored in the MM array 410 has been described. However, when the image data temporarily stored in the MM array 410 is transferred to the DSP 43. The same applies to XY conversion when writing to the data memory 21. For example, when the XY-converted image data is temporarily stored in the C1 memory to the C4 memory as described above, the C1 memory to the C4 memory are simultaneously sent to operate the unit data in each register of the ring register 433. If the four unit data are moved and output from one load / store unit 431, the Y-direction line can be XY-converted into the X-direction line and transferred to the DSP 43 or the like.

(3) Compression mode The compression mode is a processing mode in which image data in which the number of data is reduced by thinning out the image data stored in the data memory is temporarily stored in the MM array 410. In the compression mode, the read / write sequencer 440 controls the read / write adjustment circuit 430 as follows. Now, a case where the first load / store unit 431 reads the image data of the X1 line to X5 line and temporarily stores the image data in which the number of data is compressed to ¼ in the MM array 410 will be described.

At this time, the read / write sequencer 440 first causes the first load / store unit 431 to specify the address of the X1 line and read the image data for one X direction line. The port connection controller 432 is connected to the first load / store unit 431 and the first register in the ring register 433. When the unit data of the X1 line input from the port connection controller 432 to the ring register 433 is 4n−2 to 4n (n is an integer of 1 or more), the unit data is input to the ring register 433. Each time it is set to send and move data between registers.

Specifically, when the unit data constituting the X1 line is input, the same operation as in the XY conversion mode described above is performed on the first to fourth (n = 1) unit data. That is, when the second to fourth unit data is input, the data is sequentially moved between the registers by the ring register 433, the first unit data is transferred to the fourth register, and the second unit data is transferred to the third register. The third unit data is moved to the second register, and the fourth unit data is moved to the first register. Then, after these four unit data are stored in the fourth register to the first register of the ring register 433, the unit data stored in each register is written into the MM array 410. As a result, the first unit data in the X1 line is temporarily stored in the C4 memory, the second unit data is the C3 memory, the third unit data is the C2 memory, and the fourth unit data is temporarily stored in the C1 memory.

Similarly, when the fifth to eighth (n = 2) unit data is input, when the sixth to eighth unit data is input, the ring register 433 sequentially moves the data between the registers. The unit data is moved to the fourth register, the sixth unit data is moved to the third register, the seventh unit data is moved to the second register, and the eighth unit data is moved to the first register. Then, after these four unit data are stored in the fourth register to the first register of the ring register 433, the unit data stored in each register is written into the MM array 410. As a result, the first and fifth unit data in the X1 line are the C4 memory, the second and sixth unit data are the C3 memory, the third and seventh unit data are the C2 memory, the fourth and eighth unit data. Is temporarily stored in the C1 memory.

When such an operation is performed for one X1 line, the 1,5, 9,..., 4mth unit data in the X1 line is stored in the C4 memory of the MM array 410, and the 2,6 in the X1 line is stored in the C3 memory. , (4m + 1) th unit data, 3,7,11, ..., (4m + 2) th unit data in the X1 line in the C2 memory, 4,8,12, in the X1 line in the C1 memory ..., (4m + 3) th unit data is temporarily stored. The image data temporarily stored in the C1 memory to the C4 memory is all X1 line image data, but is compressed image data in which the number of data is reduced to ¼ by skipping four data.

The X direction lines after the X2 line can also be executed simultaneously in synchronism with the X1 line. For example, the read / write sequencer 440 causes the second load / store unit 431 to specify the address of the X2 line and read the image data for one X direction line. The port connection controller 432 is connected to the second load / store unit 431 and the fifth register in the ring register 433. For the ring register 433, if the unit data of the X2 line input from the port connection controller 432 to the ring register 433 is 4n-2 to 4n (n is an integer of 1 or more), the unit constituting the X2 line It is set so that data is sent and moved between registers each time data is input.

With this setting, the 1st to 4th unit data in the X2 line are the first unit data in the 8th register, the 2nd unit data in the 7th register, and the 3rd unit data in the 6th register. The fourth unit data is moved to the fifth register. Then, after these four unit data are stored in the eighth to fifth registers of the ring register 433, the unit data stored in each register is written into the MM array 410. As a result, the first unit data in the X2 line is temporarily stored in the C8 memory, the second unit data is the C7 memory, the third unit data is the C6 memory, and the fourth unit data is temporarily stored in the C5 memory.

When the fifth to eighth (n = 2) unit data is input, the same processing is performed. The fifth unit data is stored in the eighth register, the sixth unit data is stored in the seventh register, and the seventh unit data. Are moved to the sixth register and the eighth unit data are moved to the fifth register. Then, after these four unit data are stored in the eighth to fifth registers of the ring register 433, the unit data stored in each register is written into the MM array 410. As a result, the first and fifth unit data in the X2 line are the C8 memory, the second and sixth unit data are the C7 memory, the third and seventh unit data are the C6 memory, the fourth and eighth unit data. Is temporarily stored in the C5 memory.

When such an operation is executed for one X2 line, the C8 memory of the MM array 410 has 1,5, 9,..., 4mth unit data in the X2 line, and the C7 memory has 2,6,6 in the X2 line. ,..., (4m + 1) th unit data, 3,6,7,11,..., (4m + 2) th unit data in the X2 line in the C6 memory, 4,8,12, in the X2 line in the C5 memory ..., (4m + 3) th unit data is temporarily stored. The image data temporarily stored in the C5 memory to the C8 memory is all X2 line image data, but is compressed image data in which the number of data is reduced to ¼ by skipping four pieces of data.

The read / write sequencer 440 performs the same processing for the X3 line to X5 line as well as the same setting as the X1 line and X2 line. In the MM array 410, the 1st, 5th, 9th,..., 4mth unit data in the X3, X4, and X5 lines are temporarily stored in the C12 memory, C16 memory, and C20 memory, and the C11 memory, C15 memory, and C19 memory. , 4m + 1-th unit data in the X3, X4, and X5 lines are temporarily stored.

Through the above operation, the MM array 410 has the X1-line compressed image data in which the number of data is reduced to 1/4 in the C1 to C4 memories, and the number of data in the C5 to C8 memories is reduced to 1/4. X2 line compressed image data, C9 to C12 memory each reduced the number of data to X3 line compressed image data, ... C17 to C20 memory each reduced the number of data to 1/4 X5 line compressed image data is temporarily stored.

Then, the port connection controller 432 causes the first load / store unit 431 and the C1 memory, the second load / store unit 431 and the C5 memory, the third load / store unit 431 and the C9 memory,. The load / store unit 431 and the C17 memory are connected to each other, and the unit marching memory of the C1, C5, C9,..., C17 memory is operated in order and written to the data memory 21, for example, to the original image data. On the other hand, compressed image data in which the number of data is reduced to ¼ can be created.

In the present embodiment, one end of a plurality of unit marching memory columns arranged in parallel is an input / output port, but one end may be an input port and the other end may be an output port, or both ends may be input / output ports. Also good. In these cases, the load / store unit 431 and the port connection controller 432 are provided on both sides of the MM array 410, but the ring register 433 may be either one.

In the above, the case where the marching memory is applied to the image processing apparatus has been described with some application examples. However, the marching memory is not limited to these application examples, and can be widely applied to other parts of the image processing apparatus, other information processing apparatuses having different processing forms, and the like. Application to other parts of the image processing apparatus will be briefly described with reference to FIG. FIG. 6 is a block diagram schematically showing a signal processing system in the image processing apparatus.

The illustrated signal processing system includes an image input system 510, an image processing system 520, an image output system 530, a DRAM 540 connected to the image processing apparatus, an external memory (MM) 550, a storage 560, an external processing system 570, and the like. The

The image processing system 520 includes a CPU 521, a GPU (Graphics Processing Unit) 522, a codec 523, a DRAM controller 524, an external memory controller 525, a storage IP 526, an external processing system IP 527, and the like. System on Chip) configuration. In comparison with the signal processing system described with reference to FIG. 1, the CPU 521 generally corresponds to the CPU core unit 3, the GPU 522 corresponds to the DSP array unit 4, and the DRAM 540 corresponds to the DRAM 21.

In such a signal processing system, the marching memory is suitably applied to the image input system 510, the CPU 521 and the external processing system IP 527 in the image processing system 520, the image output system 530, the external memory 550, the storage 560, the external processing system 570, and the like. can do. That is, the marching memory is suitably applied to the image input system 510 for temporary storage of image data captured by the image sensor and the image output system 530 for temporary storage of image data output from the image processing system 520. can do. In the CPU 521 in the image processing system 520, for example, a marching memory can be suitably applied to temporary storage of image data when performing face recognition or tracking of a moving subject. The same applies to the external memory 550, the storage 560, the external processing system 570, and the like, and the marching memory can be suitably applied to temporary storage of image data.

As described above, the data group targeted by the present invention is stream data, and has a predetermined relationship between adjacent data. Therefore, unlike the random data, it is not necessary to perform addressing for each unit data one by one and read and write, and the time required for addressing, data search, etc. can be eliminated. In the marching memory, a column in which a plurality of cells are connected is used as a unit, and unit data input to the cell is sequentially transferred and temporarily stored in each cell. At this time, the input and movement speed of the unit data can be made to correspond to the reference clock of the CPU, and the writing and reading operations can be performed at a high speed. Therefore, according to the present invention, it is possible to provide a suitable application utilizing the characteristic operation form of the marching memory.

21 Data memory 43 Digital signal processor (DSP)
44 Shared Memory 100 Image Processing Device 120 of First Mode in First Embodiment Operation Unit 150 Buffer Memory 151 Load Buffer (Marching Memory)
152 Store buffer (marching memory)
200 Image Processing Device 210 of Second Mode in First Embodiment Register File (Marching Memory)
211, 212, 213,..., 21N Unit marching memory 220 Calculator 400 Image processing apparatus 410 according to the second embodiment MM array (marching memory)
411, 412, 413,..., 41N Unit marching memory 433 Ring register (array changing means)
440 Read / write sequencer (sequencer)
510 Image Input System 520 Image Processing System 530 Image Output System

Claims

A data memory for temporarily storing stream data composed of a plurality of unit data;
An information processing apparatus comprising a processor and a processor that performs predetermined information processing on stream data read from the data memory;
A marching memory is provided as a register file of the processor,
The marching memory has a column in which a plurality of storage areas are arranged in series, and a plurality of unit data that are input are sequentially moved from one end of the column to adjacent storage areas and temporarily stored in each storage area. A plurality of memories are provided in parallel,
The marching memory temporarily stores a plurality of input unit data in each storage area of the first unit marching memory,
The processor is configured to sequentially calculate the plurality of unit data temporarily stored in the storage areas of the first unit marching memory based on a batch calculation processing instruction, and sequentially process the unit data using the calculator. Is temporarily stored in each storage area of the second unit marching memory.
A second marching memory is provided as a shared memory between the data memory and the processor,
The information processing apparatus according to claim 1, wherein the second marching memory temporarily stores the input unit data.
3. The information processing apparatus according to claim 1, further comprising a third marching memory as a buffer memory of the processor.
A data memory for temporarily storing stream data composed of a plurality of unit data;
An information processing apparatus comprising a plurality of processors having a computing unit and performing predetermined information processing on stream data read from the data memory,
A marching memory is provided as a shared memory between the data memory and the plurality of processors,
The marching memory has a column in which a plurality of storage areas are continuously arranged, and a unit for temporarily storing a plurality of input unit data by sequentially moving the input unit data from one end of the column to an adjacent storage area An information processing apparatus configured by providing a plurality of marching memories in parallel.
The input port and / or the output port of each unit marching memory in the marching memory is provided with an arrangement changing means for changing an arrangement of a plurality of unit data inputted and / or a plurality of unit data outputted. Item 5. The information processing apparatus according to Item 4.
The arrangement changing means has a ring register composed of a register group connected to an input port and / or an output port of each unit marching memory, and a sequencer for controlling the operation of the ring register,
6. The information processing apparatus according to claim 5, wherein the sequencer controls the operation of the ring register in accordance with an information processing mode executed by the information processing apparatus, and changes the arrangement of the plurality of unit data.
A data memory for temporarily storing stream data composed of a plurality of unit data;
An information processing apparatus comprising a plurality of processors having a computing unit and performing predetermined information processing on stream data read from the data memory,
A marching memory is provided as a buffer memory of the processor,
The marching memory has a column in which a plurality of storage areas are continuously arranged, and a unit for temporarily storing a plurality of input unit data by sequentially moving the input unit data from one end of the column to an adjacent storage area An information processing apparatus configured by providing a plurality of marching memories in parallel.
An information processing apparatus according to any one of claims 1 to 7,
An image input system having an image sensor and inputting image data as the stream data to the information processing apparatus;
A digital camera comprising: an image output system that outputs image data processed by the information processing apparatus.
A processor that performs predetermined information processing on stream data composed of a plurality of unit data,
A register file composed of marching memory,
An arithmetic unit that performs arithmetic processing using the data held in the register file,
The marching memory has a column in which a plurality of storage areas are continuously arranged, and a unit for temporarily storing a plurality of input unit data by sequentially moving the input unit data from one end of the column to an adjacent storage area A processor configured by providing a plurality of marching memories in parallel.
10. The processor according to claim 9, further comprising a buffer memory composed of a marching memory.