JP2014174689A - Data processing device and data processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a data processing device configured to improve performance by reducing a bus band and improving access efficiency.SOLUTION: A data processing device performs reading data from a memory 113 having a plurality of storage areas, and writing processed data on the storage areas. A first image processing unit 104 causes a data transfer control unit 106 to process the data read from the memory 113, and outputs the processed data to the data transfer control unit 106. A second image processing unit 105 causes the data transfer control unit 106 to acquire the processed data written on the storage areas, for filter processing. The memory 113 stores overlap data necessary for the filter processing and non-overlap data other than the overlap data in different storage areas. The data transfer control unit 106 causes an address control unit 109 to control access by switching the storage area for the overlap data and the storage area for the non-overlap data.

Description

  The present invention relates to a data transfer control technique for reading data in a predetermined unit, performing filter processing, and outputting the data.

  In an imaging apparatus (digital camera or the like), a large capacity and low cost Dynamic Random Access Memory (DRAM) is used to store processed image data. In a DRAM, when accessing different row addresses (different pages) in the same bank, it is necessary to issue a precharge command (page close) and an active command (page open). During execution of this command, access to the same bank cannot be performed. For this reason, if the number of accesses to different pages in the same bank increases, it is necessary to issue many precharge commands and active commands, and the time until data transfer to the DRAM is completed becomes longer. Therefore, bank interleave control is performed as DRAM access control. According to the bank interleaving control, an active command is issued to another bank after an active command is issued to one bank until data transfer is completed, thereby requiring data transfer to the DRAM. Time can be shortened.

  In the imaging apparatus, processing such as pixel interpolation, white balance, sharpness, noise reduction, reduction / enlargement, and the like is performed. For example, in the case of noise reduction processing, a plurality of types of image data generated by dividing image data into a plurality of frequency bands are stored in the DRAM (first image processing). Thereafter, a technique is known in which each image data is subjected to appropriate filter processing, and the image data processed for each frequency band is frequency-synthesized again (second image processing). In the filter processing, for example, a pixel located at the center of a window composed of n × n pixels is processed using (n × n) −1 pixel values at peripheral positions. When a line memory (SRAM or the like) is used for image processing, the circuit scale increases as the number of pixels increases. In view of this, Patent Documents 1 to 3 propose methods for dividing and processing image data.

  Patent Document 1 discloses a technique for realizing high-performance image processing without increasing the circuit scale. Specifically, when the horizontal pixel width of the input image is larger than the line buffer, the input image is equally divided vertically so that the size of the divided area is smaller than the horizontal pixel width of the line buffer. . Then, the input data transfer means is controlled to sequentially transfer the pixel data of the input image to the line buffer for each equally divided area. The pixel processing means sequentially processes pixel data of the input image temporarily stored in the line buffer and outputs output pixel data. Further, the image combining unit controls the output data transfer unit, and generates output images by combining the output pixel data sequentially output for each divided region.

Further, in the filtering process in the dividing process, processing is performed by adding extra pixels (hereinafter referred to as overlap pixels) necessary for the filtering process to the pixels at the upper, lower, left, and right ends of the divided image. Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique for reading overlapping pixel data necessary for performing filter processing using a band buffer having a small capacity and performing image processing without disturbing pipeline processing. Specifically, the band buffer has first to third bands. In the pipeline processing, input image data is written to the first band, and stored image data is read from the remaining second and third bands and supplied to the image processing unit. The image processing unit can read the divided image data stored in the third band and the overlap pixel data stored in the second band without being affected by writing.
Patent Document 3 discloses a technique for reducing the memory capacity at the time of division processing and shortening the delay time. The writing side and the reading side notify each other of the completion of writing of a certain amount of data and the completion of reading of a certain amount of data, and a handshake is performed to control that reading does not write or writing does not overtake reading. Done.

JP 2006-186717 A JP-A-2005-250534 JP 2008-172410 A

However, in the technique disclosed in Patent Document 1, when pipeline control requiring filter processing is performed, it is necessary to write the overlap pixel data necessary for the second image processing in each buffer. That is, in the first image processing, there is a possibility that the overlap pixel data necessary for the second image processing is written in the respective buffers and processed twice.
In addition, since the technique disclosed in Patent Document 2 has a configuration of three band buffers (storage areas), overlapping pixels for the upper end and the lower end (or the left end and the right end) of the divided image are necessary for the divided image. May not be filtered correctly. Furthermore, since the data of pixels other than the overlapping pixels are held in the band buffer, there is a possibility that a period for holding unused image data occurs, and the band buffer cannot be used effectively.

Further, in the technique disclosed in Patent Document 3, overlap pixel data is included in a write side or a read side bank on a DRAM only by controlling so that reading does not write or writing does not overtake reading. Will be placed. With such a method, there is a possibility that improvement in memory access efficiency by bank interleave control cannot be expected.
Therefore, various measures have been desired for any of the techniques.

  An object of the present invention is to improve performance in a data processing apparatus by reducing bus bandwidth and increasing access efficiency.

  In order to solve the above-described problems, an apparatus according to the present invention includes a storage device that stores data and an address control unit that switches an access destination of a storage area of the storage device, and reads data from the storage device. And a data transfer control means for controlling writing, a first processing means for processing the data in the storage device in a batch or divided, and a first processed data as a processing result of the first processing means. Second processing means is provided. The storage device includes a first storage area for storing data to be input to the first processing means, an overlap area in which the second processing means refers to the first processed data at least twice, and The second processing means has a second storage area including a non-overlapping area that refers to the first processed data only once, and the data transfer control means uses the address control means to When the storage area or the second storage area is accessed and a specific address is reached, the address is jumped to switch the access destination of the storage area, thereby dividing the first processed data and A process of writing in the wrap area and the non-overlap area is controlled.

  According to the present invention, the performance can be improved by reducing the bus bandwidth and improving the access efficiency.

FIG. 7 is a block diagram illustrating a configuration example of a main part of the imaging apparatus in order to explain the first embodiment of the present invention in conjunction with FIGS. 2 to 6. It is a block diagram which shows the structural example of an address control part. It is a figure explaining the structural example of division | segmentation image data. It is a figure explaining the pipeline process of division | segmentation image data. It is a flowchart explaining the example of address control of WRDMAC. It is a flowchart explaining the example of address control of RDDMAC. It is a figure explaining the pipeline process of the division | segmentation image data in the modification of 1st Embodiment of this invention. FIG. 11 is a block diagram illustrating a configuration example of a main part of an imaging device in order to describe a second embodiment of the present invention together with FIGS. 9 and 10. It is a block diagram which shows the structural example of the catch-up monitoring part of DMAC. It is a figure explaining the pipeline process of division | segmentation image data. It is a figure explaining the pipeline process of the division | segmentation image data in the modification of 2nd Embodiment of this invention. It is a figure explaining the example of a process of the divided image data in 3rd Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Note that a digital camera will be described as an example in which the data processing apparatus is applied to an imaging apparatus. However, the present invention can be applied to an imaging apparatus such as a digital video camera, a mobile phone with a camera, and an in-vehicle camera, and various image processing apparatuses.

[First Embodiment]
FIG. 1 is a block diagram illustrating a configuration example of a digital camera according to the first embodiment.
The imaging element 100 is a CCD sensor, a CMOS sensor, or the like that photoelectrically converts a subject image received through an imaging optical system and outputs an imaging signal. The A / D converter 101 converts the analog output signal of the image sensor 100 into a digital signal.
The image processing unit 102 includes an imaging processing unit 103, a first image processing unit 104, and a second image processing unit 105. The image processing unit 102 has an image buffer memory (not shown), and performs image processing other than the processing performed by the first and second image processing units. For example, a compression / expansion unit that performs compression processing and decompression processing of image data, a display control unit that performs display control such as a monitor, and a recording processing unit that records data on a recording medium are provided. The imaging processing unit 103 acquires the image data converted by the A / D converter 101 and performs pixel correction, black level correction, shading correction, and the like.

  The first image processing unit 104 performs predetermined image processing (first processing) on the image data processed by the imaging processing unit 103 or image data stored in an image buffer memory (not shown) or the memory 113. . Specifically, there are processing for generating image data by dividing it into a plurality of frequency bands as preprocessing for flaw correction, magnification chromatic aberration correction, image data format conversion processing, and noise reduction processing. In the preprocessing of the noise reduction process, it is possible to generate image data of a single frequency without being divided into a plurality of frequency bands (resizing process). The first image processing unit 104 outputs the processing result to the memory 113, the second image processing unit 105, or an image data processing unit (not shown).

  The second image processing unit 105 performs predetermined image processing (second processing) on the image data processed by the first image processing unit 104 or image data stored in an image buffer memory (not shown) or the memory 113. )I do. Specifically, an image processed for each frequency band after appropriate filter processing is performed on image data divided in a plurality of frequency bands as post-processing of image data format conversion processing or noise reduction processing There is a process to synthesize the data again. In addition, processing such as development processing and distortion correction can be cited. In post-processing of noise reduction processing, it is possible to perform noise reduction processing by performing appropriate filter processing on image data other than image data divided in a plurality of frequency bands. The second image processing unit 105 outputs the processing result to the memory 113 or an image data processing unit (not shown).

The data transfer control unit 106 includes a WRDMAC 107 and an RDDMAC 108 which are a plurality of Direct Memory Access controllers that perform data transfer. “WR” indicates a writing process, and “RD” indicates a reading process. In FIG. 1, WRDMAC (1) is shown for data output from the imaging processing unit 103. WRDMAC (2) and RDDMAC (2) are provided for the first image processing unit 104, and WRDMAC (3) and RDDMAC (3) are provided for the second image processing unit 105. The image data is output to the bus 111 by each WRDMAC 107 and temporarily stored in the memory 113 via the memory control unit 112. The image data temporarily stored in the memory 113 is read out by each RDDMAC 108. At that time, the image data is output from the memory 113 to the bus 111 via the memory control unit 112.
A bus 110 is a system bus, and a bus 111 is an image data bus.

  The memory control unit 112 controls processing for writing data into the memory 113 and processing for reading data from the memory 113 in accordance with instructions from the system control unit 116 or the data transfer control unit 106 having a CPU (central processing unit). Note that output data from the A / D converter 101 may be directly written into the memory 113. The memory 113 is a storage device that stores a predetermined number of still images, data such as moving images and sound over a predetermined time, and stores constants and programs for operating the system control unit 116. The nonvolatile memory control unit 114 controls processing for writing data to the nonvolatile memory 115 and processing for reading data from the nonvolatile memory 115 in accordance with instructions from the system control unit 116. The nonvolatile memory 115 is an electrically erasable and recordable memory. For example, an EEPROM (Electrically Erasable Programmable Read-Only Memory) is used. The nonvolatile memory 115 stores constants, programs, and the like for the operation of the system control unit 116.

  A system control unit 116 that controls the operation of the digital camera performs various control processes by giving various instructions to each functional block. The system control unit 116 controls the image processing unit 102, the data transfer control unit 106, the memory control unit 112, and the nonvolatile memory control unit 114 connected via the bus 110. The system control unit 116 controls the image sensor 100 and the like according to an operation instruction from the operation unit 117. The CPU of the system control unit 116 implements each process of the present embodiment by executing a program recorded in the nonvolatile memory 115. The operation unit 117 includes switches and buttons operated by the user, and is used for power ON / OFF operation, shutter ON / OFF operation, and the like.

  FIG. 2 is a block diagram illustrating a configuration example of the address control unit 109 provided in the WRDMAC 107 and the RDDMAC 108. The WRDMAC 107 and RDDMAC 108 have an offset function that jumps the data storage address value to a specific address after transferring a certain amount of data. This is a so-called address jump function, and a plurality of storage areas can be accessed by using this function. The system control unit 116 sets a start address, an offset data transfer length, an address offset value, and a burst length for the WRDMAC 107 and the RDDMAC 108. Thereby, the start address of the storage area in the memory and the storage area size are designated. At this time, when there are a plurality of storage areas to be accessed, for example, three areas, three sets of offset data transfer length and address offset value are set. That is, set values corresponding to the number of storage areas to be accessed are prepared and the setting process is performed, thereby enabling access to a plurality of storage areas.

  The address selector 200 selects the start address set by the system control unit 116 as an address value at the start of data access. Each time data is transferred, the address selector 200 selects and outputs the value output from the address counter 203 as the current address value. The transfer length counter 201 counts the transfer data length and outputs an offset timing signal to the offset value calculator 202 every time data of the offset data transfer length set in the system control unit 116 is transferred. When the offset value calculator 202 receives the offset timing signal from the transfer length counter 201, the offset value calculator 202 outputs the address offset value set by the system control unit 116, and the burst length set by the system control unit 116 at other timings. Output. The address counter 203 adds the value output from the offset value calculator 202 to the current address value, thereby generating and outputting an address value for storing the next data.

FIG. 3 is an explanatory diagram when the image processing unit 102 divides and processes one screen of image data 300, and shows an example of divided image data held in a plurality of storage areas in the memory 113. Image data 301 to 305 are divided image data obtained by dividing the image data 300 into strips, and indicate respective ranges of data when the second image processing unit 105 performs division processing. The divided image data 306 to 314 obtained by dividing the image data 300 into strips indicate the results obtained by the first image processing unit 104 to acquire and process the data (hereinafter referred to as first processed data). It is stored in a plurality of storage areas in the memory 113. Note that the memory 113 has a first storage area for storing data input to the first image processing unit 104 and a second storage area for storing first processed data.
The divided image data 306 and 307 are image data (processing target data) when the second image processing unit 105 processes the divided image data 301. The divided image data 307 is image data necessary when the second image processing unit 105 performs the filtering process at the right end of the divided image data 301, and the second image processing unit 105 at the left end of the divided image data 302. This is image data necessary for performing the filtering process. That is, the divided image data 307 includes overlap pixel data (overlap data) used when the second image processing unit 105 performs the filtering process on the divided image data 301 and 302.

The divided image data 308 to 314 are the same as described above, and the relationship between the image data is shown in Table 1 below.

Of the first processed data 306 to 314, data indicated by an even code (non-overlapping data) constitutes a first data group and is stored in a storage area in the memory 113 (hereinafter referred to as a non-overlapping area). The The non-overlap area is an area in which the second image processing unit 105 refers to the first processed data only once. In addition, the overlap data indicated by the odd symbols constitutes a second data group and is stored in a storage area in the memory 113 (hereinafter referred to as an overlap area). The overlap region is a region where the second image processing unit 105 refers to the first processed data at least twice. Note that the non-overlap area and the overlap area in the memory 113 are not fixed areas but dynamically changed.
FIG. 3 shows an example in which the first image processing unit 104 divides the image data 300. However, the image data 300 is batch-processed without being divided, or the number of divisions is larger than the number of division processes of the second image processing unit 105. May be stored in the memory 113 after processing. Furthermore, as shown in FIG. 3, the image data 300 is not limited to a method of dividing the image data 300 in a vertically long manner, and a method of dividing the image data 300 in a horizontally long manner or a method of performing block division may be adopted. At the time of horizontal division, the image data at the upper end and the lower end of the divided image becomes overlap data. In the case of block division, the image data at the top, bottom, left and right ends of each block image is overlap data. When performing horizontal division, block division, or the like, for example, overlap data and non-overlap data can be stored in separate storage areas.

FIG. 4 illustrates the data arrangement and the periods 405 to 410 when the first image processing unit 104 and the second image processing unit 105 divide the image data 300 and perform pipeline processing. Storage areas 400 to 404 indicate areas in which the divided image data 306 to 314 are stored in the memory 113, respectively. The divided image data 306 to 314 represent the arrangement in each storage area. Furthermore, the period (holding period) for holding the image data 306 to 314 in the memory 113 is shown.
In the period 405, processing for writing the first processed data 306 to the storage area 401 of the memory 113 by the WRDMAC 107 and processing for writing the first processed data 307 to the storage area 402 of the memory 113 by the WRDMAC 107 are executed. Next, in a period 406, processing for writing the first processed data 308 to the storage area 403 of the memory 113 by the WRDMAC 107 and processing for writing the first processed data 309 to the storage area 404 of the memory 113 by the WRDMAC 107 are executed. . In the period 406, the second image processing unit 105 reads out the divided image data 306 and 307 from the storage areas 401 and 402 of the memory 113 by the RDDMAC 108 and performs processing.

Table 2 summarizes the data writing status and the data reading status for each of the storage areas in each period. “W” represents a writing process, “R” represents a reading process, and the reference numerals attached to the divided image data to be processed are shown in parentheses.

Note that the storage areas 400 to 404 are assigned to different banks in the memory 113. Bank interleave control can be performed during pipeline processing by the first image processing unit 104 and the second image processing unit 105 using the address jump function of the address control unit 109. Thus, access to the memory 113 can be performed efficiently.
Next, the data write process in the period 405 in FIG. 4 will be described with reference to the flowchart in FIG. The divided image data 306 and 307 processed by the first image processing unit 104 in the period 405 are written in the storage areas 401 and 402 of the memory 113 by the WRDMAC 107 (see FIG. 1: WRDMAC (2)), respectively.

  In S500, the system control unit 116 sets a start address, an offset data transfer length (2 sets), an address offset value (2 sets), and a burst length for the WRDMAC 107, and starts data transfer. In step S <b> 501, the data transfer control unit 106 sends an instruction to write the divided image data 306 (first processed data) to the storage area 401 of the memory 113 for each burst length to the memory control unit 112. Thereby, the process of writing the divided image data 306 in the memory 113 is executed.

  S502 is a determination process as to whether or not the writing of one line of the divided image data 306 has been completed based on the offset data transfer length. In this embodiment, when writing is completed after image data for one line is transferred, it is determined that a specific address corresponding to a value obtained by adding the data transfer amount to the start address value has been reached. If the result of this determination is that writing of data for one line has not been completed, processing returns to S501 and the writing process is continued. If it is determined that the writing of data for one line has been completed, the process proceeds to S503. In step S <b> 503, processing for changing the access destination to the memory 113 to the storage area 402 is performed using the address jump function of the address control unit 109. In next step S504, the data transfer control unit 106 sends an instruction to write the divided image data 307 (first processed data) to the storage area 402 of the memory 113 for each burst length to the memory control unit 112. Thereby, the process of writing the divided image data 307 in the memory 113 is executed. S505 is a determination process as to whether or not the writing of one line of the divided image data 307 has been completed based on the offset data transfer length. As a result of the determination, when it is determined that the data writing for one line is not completed, the process returns to S504. If it is determined that data writing for one line has been completed and a specific address has been reached, the process proceeds to S506.

In S506, it is determined whether or not the data transfer of all lines has been completed. As a result of the determination, if it is determined that the data transfer of all lines has not been completed, the process proceeds to S507. If it is determined that the data transfer for all lines has been completed, the process is terminated. As a result, the divided image data 306 and 307 are written in the storage areas 401 and 402 of the memory 113, respectively (see FIG. 4).
In S507, the address jump function of the address control unit 109 is used to change the access destination to the memory 113 to the storage area 401, and the process returns to S501 and continues. Although not described, each data writing process is performed in the periods 406 to 408 as in the flowchart of FIG. In the period 409, one set of offset data transfer length and address offset value is set, and the divided image data 314 is written in the storage area 401 by normal WRDMAC transfer.

Next, data read processing in the period 407 in FIG. 4 will be described with reference to the flowchart in FIG. In the period 407, the second image processing unit 105 reads out the divided image data 307, 308, and 309 from the storage areas 402, 403, and 404 of the memory 113 by the RDDMAC 108 (see RDDMAC (3) in FIG. 1). Run.
In S600, the system control unit 116 sets a start address, an offset data transfer length (3 sets), an address offset value (3 sets), and a burst length for the RDDMAC 108, and starts data transfer. In step S <b> 601, the data transfer control unit 106 sends an instruction to read data from the storage area 402 of the memory 113 to the memory control unit 112 for each burst length so that the second image processing unit 105 processes the divided image data 307. . Thereby, the read processing of the divided image data 307 is executed. S602 is a determination process as to whether or not reading of one line of the divided image data 307 has been completed based on the offset data transfer length. If the result of determination is that reading of data for one line has not been completed, processing returns to S601 and processing continues. If it is determined that the reading of data for one line has been completed, the process proceeds to S603.

In step S <b> 603, the data transfer control unit 106 uses the address jump function of the address control unit 109 to execute processing for changing the access destination to the memory 113 to the storage area 403. In next step S604, in order for the second image processing unit 105 to process the divided image data 308, the data transfer control unit 106 instructs the memory control unit 112 to read data from the storage area 403 of the memory 113 for each burst length. send. Thereby, the read processing of the divided image data 308 is executed.
S605 is a determination process as to whether or not reading of one line of the divided image data 308 has been completed based on the offset data transfer length. If the result of determination is that reading of data for one line has not been completed, processing returns to S604 and processing continues. If it is determined that the reading of data for one line has been completed, the process proceeds to S606. In S606, the address jump function of the address control unit 109 is used to change the access destination to the memory 113 to the storage area 404. In step S <b> 607, the data transfer control unit 106 sends an instruction to read data from the storage area 404 of the memory 113 to the memory control unit 112 for each burst length so that the second image processing unit 105 processes the divided image data 309. Thereby, the process which reads the division | segmentation image data 309 is performed.

S608 is a determination process as to whether or not reading of one line of the divided image data 309 has been completed based on the offset data transfer length. If the result of determination is that reading of data for one line has not been completed, processing returns to S607 and processing is continued. If it is determined that the reading of data for one line has been completed, the process proceeds to S609. In S609, it is determined whether the reading process for all lines has been completed. If it is determined that the reading process has not been completed, the process proceeds to S610. In S610, the address jump function of the address control unit 109 is used to change the access destination to the memory 113 to the storage area 402, and the process returns to S601 and continues.
If it is determined in step S609 that all the lines have been processed, the series of processes ends. In this way, the divided image data 307, 308, and 309 are read from the storage areas 402, 403, and 404 of the memory 113, respectively. Although not described, each data read process is executed in the periods 408 and 409 as in the flowchart of FIG. However, for the periods 406 and 410, two sets of offset data transfer length and offset value are set, and steps S606 to S608 are not necessary in the flowchart of FIG.

As described above, in the first embodiment, the overlap data storage area (overlap area) required for the filtering process is based on the image data divided by the second image processing unit 105, and the non- Overlapping areas are used. By using the address jump function of the address control unit 109, the first image processing unit 104 does not need to process the overlap data necessary for the second image processing unit 105 twice. Since the first image processing unit 104 does not need to read / write the overlap data twice, it is possible to reduce the bus bandwidth and improve the performance. Furthermore, by arranging each divided image data in a storage area of a different bank, the memory access efficiency can be increased by bank interleaving (memory interleaving) control.
In the example illustrated in FIG. 4, the storage areas 400 to 404 of the memory 113 are described as being different bank memories, but five banks are required to hold the divided image data separately. Therefore, a modification for reducing the number of banks will be described below.

[Modification of First Embodiment]
FIG. 7 shows an example of data arrangement that is used by switching banks based on the period processed by the first image processing unit 104. Since the arrangement of the divided image data in each period is the same as that in the case of FIG. 4, the difference will be described below.
In the period 405 illustrated in FIG. 7, the storage areas 401 and 402 are arranged in the bank 0. In the period 406, the storage areas 401 and 402 are arranged in the bank 0, and the storage areas 403 and 404 are arranged in the bank 1. In the period 407, the storage areas 400, 401, and 402 are arranged in the bank 0, and the storage areas 403 and 404 are arranged in the bank 1. In the period 408, the storage areas 400 and 401 are arranged in the bank 0, and the storage areas 402, 403, and 404 are arranged in the bank 1. In the period 409, the storage areas 400 and 401 are arranged in the bank 0, and the storage areas 402 and 403 are arranged in the bank 1. In the period 410, the storage area 401 is arranged in the bank 0 and the storage area 402 is arranged in the bank 1. The arrangement of each storage area is performed using the setting of the WRDMAC 107 and the address jump function of the address control unit 109.

  With such an arrangement, a plurality of overlap data are arranged in the same bank by the writing process of the processing result of the first image processing unit 104 and the reading process of the second image processing unit 105. However, when the number of pixels related to overlap data is sufficiently smaller than the number of pixels related to non-overlap, the effect of bank interleave control can be obtained. Therefore, the memory access efficiency can be improved while saving the number of banks.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. Note that the differences from the first embodiment described above will be mainly described, and the same components as those in the first embodiment will be omitted by using the same reference numerals already used. The way of omitting such description is the same in other embodiments described later.

  FIG. 8 is a block diagram illustrating a configuration example of a digital camera according to the second embodiment. The difference from the configuration according to the first embodiment is the WRDMAC 800 and the RDDMAC 801 of the data transfer control unit 106. The WRDMAC 800 has a configuration in which a catch-up monitoring unit 802 is added to the WRDMAC 107, and exchanges information on the amount of data transfer or the number of lines with each other via a signal line with the RDDMAC 801. The RDDMAC 801 has a configuration in which a catch-up monitoring unit 802 is added to the RDDMAC 108, and exchanges information on the amount of data transfer or the number of lines with each other via a signal line with the WRDMAC 800.

  In the second embodiment, by controlling the writing process so as not to catch up with the reading process, it is possible to control the divided image data in the overlap area necessary for the second image processing unit 105 without being overwritten by the pipeline process. . In the present embodiment, an example in which the data transfer control unit 106 has one WRDMAC 800 and one RDDMAC 801 will be described. For other DMACs, WRDMAC 800 and RDDMAC 801 may be implemented. In this case, WRDMAC 800 and RDDMAC 801 are not directly connected to each other by signal lines in order to transmit and receive address or line number information, but a relay is provided so that a plurality of WRDMAC 800 and RDDMAC 801 connection destinations can be freely selected. It may be configured.

FIG. 9 is a block diagram illustrating the configuration of the catch-up monitoring unit 802. The catch-up monitoring unit 802 receives the data transfer amount or line number of its own DMAC (DMAC having the catch-up monitoring unit 802) and the data transfer amount or line number of another DMAC as input signals. The comparison unit 900 compares these input signals and outputs a signal for temporarily stopping the control of the own DMAC. Specifically, the comparison unit 900 subtracts the data transfer amount or line number of the own DMAC from the input data transfer amount or line number of another DMAC, and compares the subtraction result with a threshold value. A control signal for temporarily stopping the processing of the own DMAC is output according to the comparison result. The threshold and the determination condition based on the threshold may be set by the system control unit 116 or may be held by the DMAC itself. In this example, the comparison unit 900 determines the magnitude relationship between the subtraction result and the threshold value and outputs a pause control signal. However, the present invention is not limited to this, and the pause condition may be determined using another means. Good.
FIG. 10 exemplifies data arrangement and periods 1004 to 1009 when the first image processing unit 104 and the second image processing unit 105 divide the image data 300 and perform pipeline processing. Storage areas 1000 to 1003 indicate areas in which the divided image data 306 to 314 are stored in the memory 113, respectively. Further, a holding period for holding the image data 306 to 314 on the memory 113 is shown.

In the period 1004, processing for writing the first processed data 306 to the storage area 1000 by the WRDMAC 800 and processing for writing the first processed data 307 to the storage area 1001 by the WRDMAC 800 are executed.
In the next period 1005, processing for writing the first processed data 308 to the storage area 1002 by the WRDMAC 800 and processing for writing the first processed data 309 to the storage area 1003 by the WRDMAC 107 are executed. Further, the second image processing unit 105 reads out the divided image data 306 and 307 from the storage areas 1000 and 1001 by the RDDMAC 801 and performs processing.

  In the next period 1006, processing for writing the first processed data 310 to the storage area 1000 by the WRDMAC 800 and processing for writing the first processed data 311 to the storage area 1001 by the WRDMAC 800 are executed. At this time, the status of data writing to the storage area 1001 is monitored by the catch-up monitoring unit 802. That is, the second image processing unit 105 monitors the situation where the divided image data 307 in the storage area 1001 is read, and writing control is performed for each fixed data transfer amount or line so that writing cannot catch up with reading. In this case, the comparison unit 900 of the catch-up monitoring unit 802 subtracts the transfer amount or line number of the own DMAC from the transfer amount or line number of the RDDMAC 801 in the second image processing unit 105, and the subtraction result is less than the threshold value. The control signal for temporarily stopping the writing is output. Further, the second image processing unit 105 reads out the divided image data 308 and 309 from the storage areas 1002 and 1003 by the RDDMAC 801 and performs processing.

In the next period 1007, processing for writing the first processed data 312 to the storage area 1002 by the WRDMAC 800 and processing for writing the first processed data 313 to the storage area 1003 by the WRDMAC 800 are executed. At this time, the status of data writing to the storage area 1003 is monitored by the catch-up monitoring unit 802. The situation in which the second image processing unit 105 reads the divided image data 309 from the storage area 1003 is monitored, and writing control is performed for each fixed data transfer amount or line so that writing cannot catch up with reading. Further, the second image processing unit 105 reads out the divided image data 310 and 311 from the storage areas 1000 and 1001 by the RDDMAC 801 and performs processing.
In the next period 1008, a process of writing the first processed data 314 to the storage area 1000 by the WRDMAC 800 is executed. Further, the second image processing unit 105 reads out the divided image data 311, 312, and 313 from the storage areas 1001, 1002, and 1003 by the RDDMAC 801 and performs processing. In the period 1009, the second image processing unit 105 reads the divided image data 313 and 314 from the storage areas 1003 and 1000 of the memory 113 and performs processing.

  The storage areas 1000 to 1003 are all arranged in different banks using the address jump function of the address control unit 109. Therefore, bank interleave control can be performed for access to the memory 113 during the pipeline processing of the first image processing unit 104 and the second image processing unit 105. A part of the overlap data, specifically, the divided image data 311 and 307 in the period 1006 and the divided image data 313 and 309 in the period 1007 are stored in the same bank. Thus, by performing access control using the catch-up monitoring unit 802 of the RDDMAC 801 so that the read and write addresses are substantially the same, the access efficiency can be increased by reducing the precharge command and the active command.

  As described above, according to the second embodiment, the address jump function and the catch-up monitoring unit 802 of the address control unit 109 are used. Further, the number of storage areas for storing data in the memory 113 at the same time can be reduced (four in the example of FIG. 10). Since the first image processing unit 104 does not need to read / write the overlap data twice, it is possible to reduce the bus bandwidth and the use amount of the storage area and improve the performance. Furthermore, by arranging the divided image data in different banks, the access efficiency to the memory can be improved by the bank interleave control.

[Modification of Second Embodiment]
Next, a modification of the second embodiment will be described. In the above example, the divided image data storage areas are arranged in different banks, so that four banks are used. Hereinafter, in order to reduce the number of banks, an example will be described in which banks are switched and arranged for each of a plurality of storage areas as shown in FIG.
Storage areas 1000 and 1001 shown in FIG. 11 are arranged in bank 0, and storage areas 1002 and 1003 are arranged in bank 1. The arrangement of each storage area is performed using the setting of the WRDMACs 107 and 800 and the address jump function of the address control unit 109.
With such an arrangement, overlap data of the divided image data is arranged in the same bank between the writing process by the first image processing unit 104 and the reading process by the second image processing unit 105. It will be. However, when the number of pixels related to the overlap data is sufficiently smaller than the number of pixels related to the non-overlap data, the effect of the bank interleave control can be obtained. Therefore, the memory access efficiency can be improved while saving the number of banks.

[Third Embodiment]
Next, a third embodiment of the present invention will be described. The difference from the second embodiment is the arrangement method of the divided image data in the memory 113 and the control method of the WRDMAC 800 and the RDDMAC 801. Below, it demonstrates centering around difference.
FIG. 12 shows data arrangement and periods 1202 to 1206 when the first image processing unit 104 and the second image processing unit 105 divide and process the image data 300. Storage areas 1200 and 1201 indicate areas in which the divided image data 306 to 314 are stored in the memory 113, respectively. Further, a holding period for holding the image data 306 to 314 on the memory 113 is shown. Here, it is assumed that non-overlap data is held in the storage area 1200 and overlap data is held in the storage area 1201.

  In the period 1202, processing for writing the first processed data 306 to the storage area 1200 by the WRDMAC 800 and processing for writing the first processed data 307 to the storage area 1201 by the WRDMAC 800 are executed. The second image processing unit 105 reads the divided image data 306 and 307 from the storage areas 1200 and 1201 of the memory 113 and performs processing. At this time, the RDDMAC 801 monitors the data write status by the catch-up monitoring unit 802. The comparison unit 900 compares the transfer amount or line number when the first image processing unit 104 writes data into the storage area and the transfer amount or line number when the second image processing unit 105 reads data. For example, the comparison unit 900 subtracts the transfer amount or line number of the RDDMAC 801 in the second image processing unit 105 from the transfer amount or line number of the WRDMAC 800 in the first image processing unit 104. The catch-up monitoring unit 802 monitors that the first image processing unit 104 has written the data for the transfer amount or the number of lines for which the subtraction result is equal to or greater than the threshold, and the second image processing unit 105 performs the divided image data 306. And 307 are read out respectively. When the subtraction result is less than the threshold value, the reading process is temporarily stopped so that the reading cannot catch up with the writing.

  In the next period 1203, processing for writing the first processed data 308 to the storage area 1200 by the WRDMAC 800 is executed. At this time, the write start timing of the storage area 1200 is determined based on the number of lines from the RDDMAC 801 by determining the end of processing of the divided image data 307 in units of lines. Next, a process of writing the first processed data 309 into the storage area 1201 by the WRDMAC 800 is executed. The divided image data 309 and 307 are held in the same storage area 1201. However, when the divided image data 308 is written, it is assumed that the divided image data 307 has been read in units of lines. For this reason, the divided image data 309 is directly written. The second image processing unit 105 reads out the divided image data 307, 308, and 309 from the storage areas 1200 and 1201, respectively, and performs processing. At this time, the RDDMAC 801 monitors the data write status by the catch-up monitoring unit 802. While monitoring that the first image processing unit 104 is writing data corresponding to the transfer amount or the number of lines in which the subtraction result in the comparison unit 900 is equal to or greater than the threshold value to the storage areas 1200 and 1201, the divided image data Processing to read out 308 and 309 is performed.

  In the periods 1204 and 1205, the same processing as that in the period 1203 is executed. Therefore, detailed description is omitted, and only differences are briefly described. In the period 1204, the first processed data 310 is stored in the storage area 1200, and the first processed data 311 is stored in the storage area 1201. The second image processing unit 105 reads out the divided image data 309, 310, and 311 from the storage areas 1200 and 1201, respectively, and performs processing. In the next period 1205, the first processed data 312 is stored in the storage area 1200, and the first processed data 313 is stored in the storage area 1201. The second image processing unit 105 reads the divided image data 311, 312, and 313 from the storage areas 1200 and 1201 and performs processing.

In the period 1206, the first processed data 314 is stored in the storage area 1200 by the WRDMAC 800. At this time, the write start timing of the storage area 1200 is determined based on the number of lines from the RDDMAC 801 by determining the end of processing of the divided image data 313 in units of lines. Further, the second image processing unit 105 reads the divided image data 313 and 314 from the storage areas 1201 and 1200 and performs processing. At this time, similarly to the above, control is performed so that reading cannot catch up with writing.
Note that the storage areas 1200 and 1201 are arranged in the same bank using the address jump function of the address control unit 109. By controlling using the catch-up monitoring unit 802 of the RDDMAC 801 so that reading and writing have substantially the same row address, it is possible to reduce precharge and active commands and increase access efficiency.

As described above, according to the third embodiment, in addition to the effects of the second embodiment, the number of storage areas that hold data on the memory 113 at the same time can be reduced as compared with the case of the second embodiment. it can. Furthermore, as compared with the first and second embodiments, the period from the start time of the first image processing to the end time of the second image processing can be shortened. Therefore, when the processed image data is displayed on a screen such as a monitor (live view display or the like), the display delay time can be shortened.
As mentioned above, although this invention was demonstrated based on each embodiment, this invention is not necessarily limited to the said embodiment, A various change and deformation | transformation are possible in the range which does not deviate from the summary.

104 First image processing unit 105 Second image processing unit 106 Data transfer control unit 109 Address control unit 113 Memory 116 System control unit 802 Catch-up monitoring unit

Claims (8)

A storage device for storing data;
An address control unit that switches an access destination of a storage area of the storage device, and a data transfer control unit that controls reading and writing of data with respect to the storage device;
First processing means for processing the data in the storage device in a batch or divided;
A second processing means for processing first processed data that is a processing result of the first processing means;
The storage device
A first storage area for storing data to be input to the first processing means;
The second processing means includes an overlap area in which the first processed data is referred to at least twice, and the second processing means includes a non-overlap area in which the first processed data is referred to only once. 2 storage areas,
The data transfer control means accesses the first storage area or the second storage area by the address control means, and jumps to an address to switch the access destination of the storage area when a specific address is reached. Thus, the data processing apparatus is characterized in that the processing of dividing the first processed data and writing to the overlap area and the non-overlap area is controlled.   2. The data processing apparatus according to claim 1, wherein the second processing unit processes the first processed data read from the overlap region and the non-overlap region by the data transfer control unit. .   When the writing and reading of the first processed data are performed in the same storage area in the second storage area, the data transfer control means compares the transfer amount of writing and reading to store the data. 3. The data processing apparatus according to claim 1, wherein monitoring is performed and control is performed such that writing does not catch up with reading or reading does not catch up with writing.   The data transfer control unit monitors writing and reading in line units when writing and reading of the first processed data is performed in the same storage area in the second storage area, and writing is read. 3. The data processing apparatus according to claim 1, wherein control is performed such that reading does not catch up or reading does not catch up with writing.   5. The data processing device according to claim 1, wherein the data transfer control unit changes a start address and a storage area size of the second storage area by the address control unit. 6. .   6. The data processing apparatus according to claim 1, wherein the data transfer control means arranges the overlap area and the non-overlap area in different banks by the address control means. .   When the data transfer control unit controls writing and reading of the first processed data, the address control unit uses the same overlap region and non-overlap region for writing the first processed data. Arrange the overlapping area and the non-overlapping area, which are arranged in a bank and read the first processed data, in the same bank, and the storage area for reading and writing the first processed data The data processing apparatus according to claim 1, wherein the data processing apparatus is arranged in a different bank. A data transfer control step for controlling reading and writing of data with respect to the storage device by address control for switching an access destination of a storage area of the storage device for storing data;
A first processing step for processing the data in the storage device in a batch or divided;
A second processing step of processing first processed data that is a processing result of the first processing step;
The data transfer control step includes:
A first storage area of the storage device that stores data input in the first processing step; an overlap area that references the first processed data at least twice in the second processing step; and A step of accessing a second storage area including a non-overlapping area that refers to the first processed data only once in the second processing step; and jumping an address when a specific address is reached A data processing method comprising a step of controlling a process of dividing the first processed data and writing to the overlap area and the non-overlap area by switching an access destination of the storage area.

Images