JP2013175074A - Execution method for a plurality of processes - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an execution method for a plurality of processes, capable of reducing deterioration of system performance due to a handshake or a waiting time of processing in a system executing the plurality of processes.SOLUTION: An execution method for a plurality of processes in a data processing system comprising an external memory for storing data, a processor having a plurality of process divisions and a DMA part including a plurality of channels, and a host CPU for instructing execution of a process by each of the process divisions equipped in the processor includes: a read DMA step for starting reading of next data after reading of data corresponding to the process division instructed to execute by the host CPU is all completed; a step for executing the plurality of processes, making the process division instructed to execute by the host CPU execute the process in parallel with read-out of next data; and a write DMA step for starting writing of the resultant data for which process is completed by the process division instructed to execute by the host CPU.

Description

  The present invention relates to a multiple processing execution method. In particular, the present invention relates to a method for executing a plurality of processes in a processor controlled by a host CPU that does not have an interrupt control mechanism.

  In general, when a predetermined process is executed, if the host CPU does not have sufficient performance and the desired system performance cannot be satisfied, the process is supplementarily performed by using a processor having high processing performance. A method may be taken to satisfy the performance. In such a method, when there are a plurality of predetermined processes to be processed, the priority of the process executed by the host CPU is determined, and an instruction is given to the processor to execute the process with the highest priority. . When the instructed process is completed, the processor notifies the host CPU that the process has been completed. When the host CPU receives a notification of completion of processing from the processor, the host CPU instructs the processor to execute processing with the next highest priority. While repeating such handshaking between the host CPU and the processor, a plurality of predetermined processes are executed in the system (see Non-Patent Document 1).

Shogo Kishida, “Outline of Service Call (1) —Generation and Termination of Tasks”, Interface Special Issue Practical Application of μITRON Compliant TOPPERS Chapter 15, CQ Publisher, p. 181-198, issued March 1, 2007

  However, in the method using the conventional technique, handshaking between the host CPU and the processor occurs by the number of processes executed in the system. In general, handshaking is performed by interrupt processing. Therefore, when the host CPU receives a notification of completion of processing from the processor, the host CPU must interrupt the currently executing processing to perform handshaking processing. Don't be. For this reason, the performance of the entire system is reduced by the processing time required for handshaking.

  Further, when the processor performs processing, the processor acquires (transfers) data necessary for calculation (processing), and temporarily stores the acquired data in a built-in memory provided in the processor. Then, the processor executes a calculation instructed by the host CPU using the data stored in the built-in memory, and outputs (transfers) the calculated data. However, it is assumed that the capacity of the built-in memory provided in the processor is not so large. This is to prevent an increase in the circuit scale of the processor by increasing the capacity of the built-in memory. For this reason, the processor transfers the data used for the calculation or the data after the calculation in a plurality of times. At this time, depending on the contents of the processing executed by the processor, for example, the time for transferring the data used for the operation is more dominant than the time for the entire processing to be executed this time than the time for actually executing the operation. It may be. In such a case, until the transfer of the data used for the operation is completed, the actual operation in the processor cannot be performed, and the operation of the processor becomes inefficient.

  On the other hand, depending on the contents of the processing executed by the processor, for example, the time for actually executing the operation dominates the time for the entire processing to be executed this time, rather than the time for transferring the data used for the operation. It can be the target. In such a case, data used for the next calculation cannot be transferred until the actual calculation in the processor is completed, and the data transfer of the processor becomes inefficient.

  The relationship between the handshake and data transfer by the method using the conventional technique and the processing time of the system will be described below. FIG. 8 is a diagram schematically showing an example of the relationship between handshaking and data transfer in the conventional system and the processing time of the system. FIG. 8 shows a system that performs two processes of process A and process B. In FIG. 8, process A is a process having a calculation time longer than the data transfer time, and process B is a process having a calculation time shorter than the data transfer time. Further, the process A is a process having a higher priority than the process B. Accordingly, the host CPU first causes the processor to execute the process A, and after the process A is completed, causes the processor to execute the process B. Moreover, in each process shown in FIG. 8, the data used for a process are acquired in 3 steps, and the calculation of each process is performed based on the acquired data. In FIG. 8, data transfer (output) after calculation is omitted.

  First, when a process start signal for starting a predetermined process in the system is input to the host CPU, the host CPU instructs the processor to execute the process A. When an instruction to start execution of process A is input from the host CPU, the processor starts transfer (acquisition) of data used for process A. Then, when the data transfer that can execute the calculation of the first process A is completed, the processor starts the calculation of the first process A based on the transferred data.

  As described above, the process A is a process in which the calculation time is longer than the data transfer time. For this reason, for example, there is a case where the third data transfer cannot be started following the second data transfer because the calculation of the first process A is not completed. In this case, as shown in FIG. 8, the third data transfer is started after the calculation of the first process A is completed. That is, in the process A, the third data transfer cannot be started until the calculation of the first process A is completed, and a data transfer waiting time (transfer waiting time) occurs.

  After that, when all the data transfer and calculation in the process A are completed, the processor outputs a notification that the process A is completed to the host CPU.

  Subsequently, when the host CPU receives a notification that the process A has been completed from the processor, the host CPU performs a handshake process, and then instructs the processor to execute the next process B. When an instruction to start execution of process B is input from the host CPU, the processor starts transfer (acquisition) of data used for process B. When the data transfer sufficient to execute the first process B operation is completed, the processor starts the first process B operation based on the transferred data.

  As described above, the process B is a process in which the calculation time is shorter than the data transfer time. For this reason, for example, after the calculation of the first process B is completed, there is a case where the data transfer that can execute the calculation of the second process B is not completed. In this case, as shown in FIG. 8, the calculation of the second process B is started after the data transfer that can execute the calculation of the second process B is completed. That is, in the process B, the calculation of the second process B cannot be started until the second data transfer is completed, and a calculation waiting time (calculation waiting time) occurs.

  Thereafter, when all the data transfer and calculation of the process B are completed, the processor outputs a notification that the process B is completed to the host CPU. When the host CPU receives a notification that the process B is completed from the processor, the host CPU completes a predetermined process in the system.

  As described above, when a system is configured by a method using a conventional technique, there is a time during which data is not actually processed, such as handshaking or waiting time. There is a problem that the time during which data is not actually processed becomes a factor that degrades the performance of the system.

  The present invention has been made on the basis of the above-mentioned problem recognition, and in a system that executes a plurality of processes, the system performance deteriorates due to frequent handshaking and the occurrence of waiting time within each process. It is an object of the present invention to provide a multi-process execution method that can reduce the above-described problem.

  In order to solve the above problems, a multiple processing execution method of the present invention includes an external memory that stores data, a plurality of processing units that perform processing on the data, and a plurality of processing units corresponding to the plurality of processing units, respectively. A processor that includes a plurality of channels and a DMA unit that accesses the external memory for each channel, and a host CPU that instructs execution of processing by each of the processing units included in the processor, A plurality of processing execution methods in the data processing system provided, wherein all of the reading of the data from the external memory is performed by the data reading channel of the DMA unit corresponding to the processing unit instructed to be executed by the host CPU. After completion, the next data read channel from the external memory is transferred to the data read channel. In parallel with the read DMA step for starting the data read and the next reading of the data from the external memory by the data read channel, the processing unit instructed to execute by the host CPU is sent from the external memory. The processing unit has completed processing on a plurality of processing execution steps for executing processing on the data that has been read and the data write channel of the DMA unit corresponding to the processing unit that is instructed to execute by the host CPU. A write DMA step for starting writing of the result data to the external memory.

  Further, the read DMA step of the present invention may be configured so that each of the processing units is based on an execution cycle corresponding to each of the processing units, which represents a cycle of whether or not the processing unit executes processing for the data. A read DMA execution determination step for determining whether or not to start reading the next data in the corresponding data read channel, and the data read channel determined to start reading the next data Next, the next read DMA execution determination step is started after reading of the next data from the external memory by the data read channel that has started reading the data is completed. The multiple processing execution step is executed by the read DMA execution determination step. Before determining whether or not to start reading of the next data, the processing unit corresponding to the data reading channel that has started reading of the data is processed with respect to the data that has been read from the external memory. The write DMA step writes the result data, which has been processed by the processing unit, to the external memory in the data write channel corresponding to the processing unit that has performed the processing on the data. It is characterized by starting.

  In the read DMA execution determination step of the present invention, the DMA execution state corresponding to each data read channel, which indicates whether or not the data read is executed by the data read channel, and the processing unit The process execution state corresponding to each of the processing units, which indicates whether or not to execute the process on the data, and before determining whether to start reading the next data, By copying the DMA execution state of the data to the processing execution state, the processing unit corresponding to the data reading channel for executing the data reading is set to a state in which processing for the data is executed, and the next data In determining whether to start reading, the execution period is determined by the processing unit. The DMA execution state corresponding to the data read channel determined to start reading the next data and to start reading the next data when it is time to execute processing on the data Is updated to a state in which the data is read, and when the execution cycle is a timing at which the processing unit does not execute the process on the data, it is determined that reading of the next data is not started, and Whether or not to update the DMA execution state corresponding to the data read channel determined not to start reading the data to a state in which the data reading is not executed, and to start reading the next data After the determination, the DMA execution state is a state in which reading of the data is executed. The next data read from the external memory by the data read channel in which the DMA execution state is a state in which the data read is executed. Is completed, the next read DMA execution determination step is executed, and the multiple-process execution step sends the processing execution state from the external memory to the processing unit in which the process for the data is executed. The write DMA step performs processing on the data write channel corresponding to the processing unit in which the processing execution state is a state of executing processing on the data. Start writing the result data to the external memory after the processing is completed It is characterized by that.

  Further, the multiple processing execution method of the present invention further includes a processing time required for the processing unit measured in advance to execute processing on the data, and the data read channel corresponding to the processing unit from the external memory. An execution process determining step in which the host CPU determines a combination of the processing units to instruct the processor to execute based on a difference from a data transfer time required for reading the data, and execution is instructed by the host CPU The data read by the data read channel corresponding to the processing unit, which is required when each of the combined processing units executes processing, is temporarily stored in accordance with the combination of the processing units that has been set. A buffer securing step for dynamically securing a capacity of the built-in memory, Steps are combined by the execution processing determination step, and each of the processing units instructed to be executed by the host CPU are processed with respect to each of the data that has been read from the external memory by the corresponding data read channel. The processing is executed, and the write DMA step is combined by the execution processing determination step, and the processing unit is connected to the data write channel corresponding to the processing unit instructed to be executed by the host CPU. The writing of each result data that has been processed to the external memory is started.

  According to the present invention, in a system that executes a plurality of processes, it is possible to reduce the deterioration in system performance due to frequent handshaking and the occurrence of waiting time in each process. .

1 is a block diagram illustrating a schematic configuration of an imaging apparatus to which a multiple processing execution method according to a first embodiment of the present invention is applied. It is the block diagram which showed schematic structure in the imaging device concerning execution of the multiple processing execution method of the 1st embodiment. It is the flowchart which showed the process sequence of the multiple process execution method in the 1st embodiment. It is the figure which showed typically an example of the relationship between the handshaking and the transfer of data in the multiple processing execution method of the 1st embodiment, and the processing time of a system. It is the figure which showed typically an example of the data division | segmentation method in the imaging device to which the multiple processing execution method of 2nd Embodiment of this invention is applied. It is the flowchart which showed the process sequence of the multiple process execution method in the 2nd embodiment. It is the flowchart which showed the process sequence of the multiple process execution method in the 3rd Embodiment of this invention. It is the figure which showed typically an example of the relationship between the handshaking and the transfer of data in the conventional system, and the processing time of a system.

<First Embodiment>
Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the first embodiment, an example in which the multi-processing execution method of the first embodiment is applied to an imaging apparatus such as a digital still camera will be described. FIG. 1 is a block diagram illustrating a schematic configuration of an imaging apparatus to which the multiple processing execution method according to the first embodiment is applied. An imaging apparatus 10 illustrated in FIG. 1 includes a lens 1, an imager 2, a capture unit 3, a DRAM 4, an image processor 5, a host CPU 6, an image processor 7, a recording unit 8, and a memory card 9. And.

The imager 2 includes a solid-state imaging device that photoelectrically converts an optical image of a subject formed by the lens 1, and outputs an image signal corresponding to the subject light to the capture unit 3.
The capture unit 3 captures the image signal output from the imager 2 and stores the captured image signal in the DRAM 4 as input image data. The capture unit 3 also outputs an interrupt signal (hereinafter referred to as “capture completion signal”) indicating that the capture of the image signal output from the imager 2 is completed when the input image data is completely stored in the DRAM 4. To the host CPU 6.

  The host CPU 6 controls the entire imaging apparatus 10 by controlling each component in the imaging apparatus 10. Further, after the capture completion signal is input from the capture unit 3, the host CPU 6 sends an instruction (hereinafter referred to as “execution instruction”) for causing the image processor 5 to execute a predetermined task (image processing). Output to the processor 5. Further, the host CPU 6 receives the interrupt signal indicating that the image processing is completed from the image processor 5 (hereinafter referred to as “end interrupt signal”), and then performs image processing to be executed by the image processor 5 next. In this case, an execution instruction for executing the next task (image processing) is output to the image processing processor 5.

  The image processor 5 reads image data (for example, input image data) necessary for the task (image processing) to be executed this time from the DRAM 4 in response to the execution instruction input from the host CPU 6, Image processing instructed by the host CPU 6 is performed. Then, the image processing processor 5 writes the image processing result after the image processing back to the DRAM 4 again. Further, the image processor 5 outputs an interrupt signal (hereinafter referred to as “end interrupt signal”) indicating the completion of the image processing to the host CPU 6 when the writing back of the image processing result to the DRAM 4 is completed. .

  The image processor 5 uses DMA (Direct Memory Access) in accessing the DRAM 4. That is, reading of image data from the DRAM 4 by the image processor 5 and writing back of the image processing result to the DRAM 4 are performed by DMA. Further, the image processor 5 can perform a plurality of types (for example, N types) of different image processing on the image data at the same time (in parallel). Therefore, the image processor 5 can read out image data corresponding to each image processing from the DRAM 4 by DMA at the same time (in parallel), and each image processing result can be read from the DRAM 4 by DMA at the same time ( Can be written back in parallel). A detailed description of the configuration of the image processor 5 and the execution procedure of the image processing by the host CPU 6 and the image processor 5 will be described later.

The image processing unit 7 reads out the image processing result stored in the DRAM 4 in accordance with the execution instruction input from the host CPU 6 and performs various image processing on the read image processing result. Then, the image processing unit 7 outputs the image processing result after the image processing to the recording unit 8 as output image data.
The recording unit 8 performs compression processing for storing in the memory card 9 on the output image data input from the image processing unit 7 to generate image data for recording. Then, the recording unit 8 writes the generated image data for recording into the memory card 9.

  Next, a configuration for executing the multiple processing execution method of the first embodiment in the imaging apparatus 10 will be described. FIG. 2 is a block diagram showing a schematic configuration of the imaging apparatus 10 related to the execution of the multiple processing execution method of the first embodiment. As shown in FIG. 2, the components of the imaging apparatus 10 related to the execution of the multiple processing execution method of the first embodiment are a DRAM 4, an image processing processor 5, and a host CPU 6.

  Further, as described above, the image processor 5 can perform access to the DRAM 4 by DMA and image processing on image data read from the DRAM 4 at the same time (in parallel). For this reason, as shown in FIG. 2, the image processor 5 accesses a plurality of channels of DMA units 51 that access the DRAM 4 by DMA, and each image data read from the DRAM 4 by the DMA unit 51 in each channel. A plurality of types (for example, N types) of image processing calculation units 52 that perform the respective image processing are provided.

The DMA unit 51 accesses the DRAM 4 by DMA for each channel. When reading the image data stored in the DRAM 4, the DMA unit 51 temporarily stores the image data read by the DMA in a built-in memory (not shown) provided in the image processor 5. The DMA unit 51 stores the image processing result temporarily stored in the built-in memory in the DRAM 4 by DMA when the image processing result after the image processing is written back to the DRAM 4.
Each of the image processing arithmetic units 52 performs image processing on the image data temporarily stored in the built-in memory, and temporarily stores again the image processing result after the image processing is performed in the built-in memory.

  With this configuration, in the image processor 5, the DMA unit 51 reads image data from the DRAM 4 by the DMA of the corresponding channel according to the execution instruction input from the host CPU 6, and the corresponding image processing arithmetic unit 52 Then, image processing is performed on the read image data. Then, the image processor 5 writes back the image processing result after the DMA unit 51 performs the image processing by the DMA of the corresponding channel to the DRAM 4, and sends an end interrupt signal indicating the completion of the image processing to the host CPU 6. Output.

  Next, a processing procedure of the multiple processing execution method of the first embodiment in the imaging apparatus 10 will be described. FIG. 3 is a flowchart showing a processing procedure of the multiple processing execution method according to the first embodiment. The flowchart shown in FIG. 3 shows operations of the host CPU 6 and the image processor 5 in the multiple processing execution method of the first embodiment. Note that, as described above, the image processor 5 can perform a plurality of types (for example, N types) of different image processing at the same time (in parallel). In the following description, a processing procedure when two kinds of different image processing (processing A and processing B) are performed on input image data at the same time (in parallel) will be described.

  When the operation of the imaging apparatus 10 is started, the host CPU 6 starts a process for executing a plurality of processes. In the multi-process execution process, the host CPU 6 first executes an interrupt process (step S1). In the interrupt process of step S1, the host CPU 6 receives an interrupt signal from each component in the imaging device 10. Thereafter, when the capture unit 3 stores the input image data obtained by capturing one frame of the image signal output from the imager 2 in the DRAM 4, the capture unit 3 outputs a capture completion signal to the host CPU 6.

  When the host CPU 6 receives an interrupt signal from each component in the imaging device 10, the cause of the received interrupt is determined (step S2). In step S2, if the cause of the interrupt is a capture completion signal from the capture unit 3, the host CPU 6 activates the image processor 5 (step S3). When the host CPU 6 activates the image processing processor 5, the host CPU 6 outputs an execution instruction to the image processing processor 5, and which image processing is the two types of image processing to be executed this time, that is, a plurality of types (for example, , N types) of the image processing calculation units 52 provided, information indicating which image processing calculation unit 52 is to be executed is output to the image processing processor 5.

  When the host CPU 6 instructs activation, that is, when an execution instruction and information on image processing to be executed are input from the host CPU 6, the image processor 5 starts execution of the two types of designated image processing. Since the image processor 5 has only a small-capacity built-in memory, the input image data captured by the capture unit 3 for one frame is divided into a plurality (for example, M), and the divided input image data The image processing for one frame is completed by repeating the image processing for each of the divided number (for example, M times).

  First, the image processor 5 reads the first divided (first) input image data (1 / M input image data) necessary for the two types of image processing from the DRAM 4 (first DMA). Read DMA) is started (step S5). Next, the image processor 5 waits for the end of the first read DMA in the first loop of the divided processing (step S6). When the first read DMA is completed, the image processor 5 reads the next (second) input image data (1 / M input image data) from the DRAM 4 in advance (second read DMA). ) Is activated (step S7).

  The image processor 5 reads out the second input image data from the DRAM 4 by the second read DMA, and performs two types of image processing on the first input image data (calculation of processing A and processing B). Are executed at the same time (in parallel) (steps S8 and S9). Note that the flowchart shown in FIG. 3 shows an example in which the calculation of process A (step S8) and the calculation of process B (step S9) are executed sequentially.

  Thereafter, when the two types of image processing are completed, the image processing processor 5 determines whether or not the current processing loop is the first loop (step S10). In step S10, when the current processing loop is the first loop, the image processor 5 performs image processing on the first input image data (1 / M input image data). A write DMA (first write DMA) for writing back each of the subsequent first image processing results to the DRAM 4 is activated (step S12).

  In the flowchart shown in FIG. 3, the calculation of the process A (step S8) and the calculation of the process B (step S9) are sequentially executed, and the write DMA is activated after the two types of image processing are completed (step S12). ) Explained the case. However, the procedure for executing the image processing and writing back the image processing result after the image processing to the DRAM 4 is not limited to the processing procedure from step S8 to step S12 shown in FIG. For example, the load of the determination process for starting the write DMA (step S10) and the process for starting the write DMA (step S12) is small, and it takes a lot of time to write back the image processing result by the write DMA to the DRAM 4. It may not be necessary. In this case, every time the calculation of the process A (step S8) and the calculation of the process B (step S9) are completed, the processes from step S10 to step S12 are performed, that is, each time the respective image processes are completed. In addition, it is possible to use a procedure for writing back the image processing result after the image processing to the DRAM 4.

  Then, the image processor 5 waits for the end of the second read DMA in the second loop of the divided processing (step S6). When the second read DMA is completed, the image processor 5 reads the next (third) input image data (1 / M input image data) from the DRAM 4 in advance (third read DMA). ) Is activated (step S7).

  While the third input image data is being read from the DRAM 4 by the third read DMA, the image processor 5 performs two types of image processing on the second input image data (calculation of processing A and processing B). Are executed at the same time (in parallel) (steps S8 and S9).

  Thereafter, when the two types of image processing are completed, the image processing processor 5 determines whether or not the current processing loop is the first loop (step S10). If the current processing loop is not the first loop in step S10, the image processor 5 waits for the end of the first write DMA (step S11). When the first write DMA is completed, the image processor 5 performs each second image processing result after performing image processing on the second input image data (1 / M input image data). Is started to write back to the DRAM 4 (second write DMA) (step S12).

  Thereafter, similarly, the image processor 5 repeats the processing from step S6 to step S12 by the number of times (for example, M times) obtained by dividing the input image data. Thereafter, when the image processor 5 repeats the processing from step S6 to step S12 as many times as the number of divided input image data, that is, 2 for the input image data for one frame captured by the capture unit 3. When the type of image processing is completed, the process loops out and waits for the end of the last write DMA (step S13). When the last write DMA ends, that is, when the image processing result for one frame captured by the capture unit 3 is written back to the DRAM 4, the image processor 5 outputs an end interrupt signal to the host CPU 6 (step S14).

  In the interrupt process, the host CPU 6 receives an end interrupt signal from the image processor 5 (step S1). When the host CPU 6 receives an end interrupt signal from the image processor 5, it determines the cause of the received interrupt (step S2). Here, in step S2, since it is determined that the cause of the interruption is the end interruption signal from the image processor 5, the host CPU 6 activates the image processor 7, and the image processor 5 writes back to the DRAM 4. Various image processing is performed on the image processing result (step S4).

  Here, the relationship between the handshake and data transfer in the multiple processing execution method of the first embodiment and the processing time of the system will be described. FIG. 4 is a diagram schematically illustrating an example of a relationship between handshaking and data transfer and system processing time in the multiple processing execution method of the first embodiment. FIG. 4 shows the case of a system that performs two image processes of process A and process B. In FIG. 4, process A is an image process whose calculation time is longer than the data transfer time, and process B is an image process whose calculation time is shorter than the data transfer time. The host CPU 6 causes the image processor 5 to execute two image processes of process A and process B at the same time (in parallel). Further, the image processor 5 processes the input image data necessary for each image processing by dividing it into a plurality (for example, M). FIG. 4 shows only the read DMA time for reading out the input image data (1 / M input image data) necessary for the two types of image processing from the DRAM 4 and the calculation time in each image processing. The write DMA for writing back the image processing result after the image processing to the DRAM 4 is omitted.

  First, when a capture completion signal from the capture unit 3 is input to the host CPU 6, the host CPU 6 outputs an execution instruction for processing A and processing B to the image processor 5 (steps S1 to S3 in FIG. 3). reference). The image processor 5 activates the read DMA in response to the execution instruction input from the host CPU 6, and reads the first (first) divided input image data necessary for the processes A and B from the DRAM 4. Then, the next (second) input image data read DMA (second read DMA) is activated (see steps S5 to S7 in FIG. 3).

  After that, the image processor 5 reads out the second input image data from the DRAM 4 by the second read DMA, and performs the same processing A and B processing on the first input image data. It is executed at the time (in parallel) (see step S8 and step S9 in FIG. 3). FIG. 4 shows an example in which the process A and the process B are sequentially executed. At this time, as can be seen from FIG. 4, since the process A and the process B are continuously executed, the waiting time of one process can be used in the other process.

  More specifically, since the process A is an image process in which the calculation time is longer than the data transfer time, if the process A is continuously executed, the next data transfer is started until the previous calculation is completed. Data transfer waiting time (transfer waiting time) due to the inability to do so occurs, and this transfer waiting time can be used for DMA (data transfer) of process B. Further, since the process B is an image process whose calculation time is shorter than the data transfer time, if the process B is continuously executed, the next calculation cannot be started until the previous data transfer is completed. The calculation waiting time (calculation waiting time) occurs, but this calculation waiting time can be used for the calculation of the process A.

  In the example shown in FIG. 4, the first (first) input image data required for the process A and the process B is read from the DRAM 4, that is, the read DMA of the channel corresponding to the process A and the process B. An example is shown in which the operation of the process A is started after the read DMA of the corresponding channel is completed. This is because the read DMA of the channel corresponding to the process A and the read DMA of the channel corresponding to the process B are performed at the same time (in parallel), that is, when the read DMA of the two channels is completed. The case where the input image data necessary for each calculation of the processing A and the processing B is prepared is shown. However, in the DMA control by the system, for example, the read DMA of the channel corresponding to the process A may occupy the reading of the input image data from the DRAM 4 in some cases. In this case, when the read DMA of the channel corresponding to the process A is finished first and the read DMA of the channel corresponding to the process A is finished, the input image data necessary for the calculation of the process A is prepared. . Therefore, in such a case, after the read DMA of the channel corresponding to the process A is completed, the calculation of the process A is started first, and then the read DMA of the channel corresponding to the process B is completed. The calculation of B can be started. In this way, the time until all the processes A and B are completed, that is, the period during which the processes A and B are executed can be shortened.

  As described above, in the imaging apparatus 10 to which the multiple processing execution method of the first embodiment is applied, the host CPU 6 performs multiple types (for example, N types) of different image processing on the image processor 5. Give it to the time (in parallel). As a result, the image processor 5 starts a plurality of types (for example, N types) of read DMA and write DMA corresponding to a plurality of types (for example, N types) of image processing at the same time (in parallel). ) Is executed at the same time (in parallel). That is, in the multiple processing execution method of the first embodiment, a plurality of types (for example, N types) of different image processing are grouped together, and corresponding data transfer (DMA) and calculation of each image processing are continuously performed. Run it. As a result, in the multiple processing execution method of the first embodiment, in the conventional processor processing, by executing each processing alone, data transfer, which has been a factor of reducing system performance, The calculation wait time can be used as a time for executing another process that is executed at the same time (in parallel). As a result, in the multiple processing execution method of the first embodiment, the waiting time in the processing of the entire system can be reduced, and the system performance can be prevented from deteriorating.

  Further, in the multiple processing execution method of the first embodiment, a plurality of types (for example, N types) of different image processing are grouped and performed at the same time (in parallel). Can be reduced. That is, in the multi-process execution method of the first embodiment, in the conventional processor process, the number of handshake processes performed each time each process is executed alone is set at the same time (in parallel). ) It can be reduced once for each of a plurality of types (for example, N types) of processing to be executed. As a result, in the multiple processing execution method of the first embodiment, the processing time required for handshaking in the processing of the entire system can be reduced, and deterioration in system performance can be further prevented.

  Note that the multi-processing execution method of the first embodiment is when the image data required for each image processing to be executed at the same time (in parallel) in the image processor 5 is divided into the same number of divisions. It is effective for. However, in the imaging device 10, the image data used in all image processing is not necessarily divided into the same number of divisions. It is also conceivable that the image data to be processed in each image processing executed at the same time (in parallel) is image data generated by different components.

<Second Embodiment>
Next, a multiple processing execution method according to the second embodiment of the present invention will be described. In the following description, similarly to the first embodiment, a case where the multi-processing execution method of the second embodiment is applied to the imaging apparatus 10 will be described. The schematic configuration of the imaging apparatus 10 to which the multiple processing execution method of the second embodiment is applied and the imaging apparatus 10 related to the execution of the multiple processing execution method of the second embodiment are shown in FIGS. 1 and 2. Since it is the same as that of the imaging device 10 to which the multiple processing execution method of the first embodiment shown is applied, detailed description thereof is omitted. Therefore, in the following description, it is assumed that the imaging apparatus 10 to which the multiple processing execution method of the first embodiment shown in FIGS. 1 and 2 is applied executes the multiple processing execution method of the second embodiment. Give an explanation. The multi-processing execution method of the second embodiment is effective when the image processor 5 provided in the imaging apparatus 10 processes image data used for each image processing by dividing it into different numbers of divisions.

  First, a method for dividing image data in the image processor 5 will be described. FIG. 5 is a diagram schematically illustrating an example of a data dividing method in the imaging apparatus 10 to which the multiple processing execution method according to the second embodiment is applied. FIG. 5A shows a case where the number of divisions of image data is the same in different image processing executed at the same time (in parallel), and FIG. 5B shows that the number of divisions of image data is the same. The case where it differs by the different image processing performed in time (in parallel) is shown.

  In the example shown in FIG. 5A, the image data to be processed in the process A and the reduced image data to be processed in the process B are both processed by being divided into horizontal strips. The number of strips in one frame subjected to image processing by processing B is the same. Thus, when the image data used for image processing is divided into the same number of divisions, each image processing is performed even if the size of the image data processed in each image processing is different. It can end at the same time. That is, the number of executions of the processing loop from step S6 to step S12 in the processing procedure of the multiple processing execution method of the first embodiment shown in FIG. 3 is equal to the number of divided image data.

  On the other hand, in the example shown in FIG. 5B, the image data is processed in units of blocks in the process A, whereas the reduced image data is processed in units of horizontal strips in the process B. The processing units when the processing A and the processing B perform image processing on one frame of image data are different between the processing A and the processing B. For example, if the block of process A shown in FIG. 5B is obtained by dividing the strip of process B into four in the horizontal direction, the ratio of the number of divisions of one frame between process A and process B is the process A : Processing B = 4: 1

  As described above, when the image data used for processing is divided into different numbers of divisions, the image to be processed in each image processing is the multi-processing execution method of the first embodiment shown in FIG. Regardless of the size of the data, each image processing cannot be completed at the same time. That is, from step S6 to step S12 in the processing procedure of the multiple processing execution method of the first embodiment shown in FIG. 3 according to the number of times of the number of divisions of the image data in the processing A shown in FIG. When the process loop is executed, an unnecessary process is executed in the process B. Conversely, steps S6 to S6 in the processing procedure of the multiple processing execution method of the first embodiment shown in FIG. 3 are matched with the number of times of the number of divisions of the image data in the processing B shown in FIG. When the processing loop up to step S12 is executed, the image processing by the processing A does not end, that is, the processing A is not executed for all the image data for one frame.

  Therefore, in the multiple processing execution method of the second embodiment, the processing corresponding to the processing loop from step S6 to step S12 in the processing procedure of the multiple processing execution method of the first embodiment shown in FIG. In the loop, whether or not each image processing is executed is switched according to the number of divisions of the image data in different image processing executed simultaneously (in parallel).

  Next, the processing procedure of the multiple processing execution method of the second embodiment in the imaging apparatus 10 will be described. FIG. 6 is a flowchart showing the processing procedure of the multiple processing execution method in the second embodiment. The flowchart shown in FIG. 6 shows the operations of the host CPU 6 and the image processor 5 in the multiple processing execution method of the second embodiment. Note that, as described above, the image processor 5 can perform a plurality of types (for example, N types) of different image processing at the same time (in parallel). In the following description, a processing procedure in the case of performing two types of different image processing (processing A and processing B) for processing input image data with different division numbers at the same time (in parallel) will be described.

  Here, process A is a process in which image data is processed in units of blocks, process B is a process in which image data is processed in units of horizontal strips, and a block of process A is obtained by dividing the strip of process B into two in the horizontal direction. That is, it is assumed that the ratio of the number of divisions of one frame in process A and process B is process A: process B = 2: 1 (see FIG. 5B).

  When the operation of the imaging apparatus 10 is started, the host CPU 6 starts a process for executing a plurality of processes. In the multi-process execution process, the host CPU 6 first executes an interrupt process (step S101). In the interrupt process of step S101, the host CPU 6 receives an interrupt signal from each component in the imaging device 10. Thereafter, when the capture unit 3 stores the input image data obtained by capturing one frame of the image signal output from the imager 2 in the DRAM 4, the capture unit 3 outputs a capture completion signal to the host CPU 6.

  When the host CPU 6 receives an interrupt signal from each component in the imaging device 10, the cause of the received interrupt is determined (step S102). In step S102, when the cause of the interruption is a capture completion signal from the capture unit 3, the host CPU 6 activates the image processor 5 (step S103). When the host CPU 6 activates the image processing processor 5, the host CPU 6 outputs an execution instruction to the image processing processor 5, and which image processing is the two types of image processing to be executed this time, that is, a plurality of types (for example, , N types) of the image processing calculation units 52 provided, information indicating which image processing calculation unit 52 is to be executed is output to the image processing processor 5.

  When the host CPU 6 instructs activation, that is, when an execution instruction and information on image processing to be executed are input from the host CPU 6, the image processor 5 starts execution of the two types of designated image processing. Since the image processor 5 has only a small internal memory, the input image data captured by the capture unit 3 for one frame is divided into a plurality of pieces according to the respective processes, and the divided input image data The image processing for one frame is completed by repeating the image processing for each of the number of times divided according to each processing. In the following description, the division number Ma of the input image data in the image processing of the process A is the division number Ma, and the division number of the input image data in the image processing of the process B is the division number Mb. The maximum value of the number of divisions of input image data in A and process B) is defined as the number of divisions MAX. Further, it is assumed that the division number Ma of the process A is the division number MAX.

  First, the image processor 5 obtains divided first (first) input image data (1 / Ma input image data and 1 / Mb input image data) necessary for each of the two types of image processing. In order to read from the DRAM 4, the read DMA (first read DMA) of all channels corresponding to each of the two types of image processing is started (step S105). Next, the image processor 5 turns on all the DMA execution state flags (hereinafter referred to as “pre-flags”) indicating whether or not the read DMA of the channel corresponding to each of the processes A and B has been activated (read DMA). Is activated) (step S106).

  Subsequently, the image processor 5 performs processing in which the pre-flag is ON in the first processing (i = 1) of the loop (i) that is repeated by the number of divisions MAX (assuming “i”) (here). Then, the end of the first read DMA of the channel corresponding to the processing A and the processing B) is waited (step S107). When the first read DMA is completed, the image processor 5 corresponds to each of the processing A and the processing B, and indicates a processing execution state flag (hereinafter referred to as “post flag”) indicating whether or not the processing operation is executed. ), The state of the pre-flag is copied (step S108).

  Subsequently, the image processing processor 5 performs the following (2) for each image processing in a loop (j) that repeats only for the type of image processing to be executed at the same time (in parallel) (assuming “j”). It is determined whether or not to activate a read DMA (second read DMA) for reading the input image data from the DRAM 4 in advance (step S109). In this determination, a remainder is calculated when the count value of the loop (i) that repeats the number of times i corresponding to the number of divisions MAX is divided by a period for executing each image processing (hereinafter referred to as “execution period T”). When the remainder is “0”, a read DMA (second read DMA) for reading from the DRAM 4 in advance is activated, and when the remainder is other than “0”, the read DMA for reading from the DRAM 4 in advance. It is determined not to start (second read DMA). The execution period T is a value representing how many times each image processing operation is executed in the loop (i) that repeats the number of times i corresponding to the number of divisions MAX. This is a quotient divided by the number of divisions of input image data in image processing.

  For example, consider a case where the division number Ma of process A = 10 and the division number Mb of process B = 5. In this case, the division number MAX = 10. At this time, the execution cycle T (hereinafter referred to as “execution cycle Ta”) corresponding to the process A is the number of divisions MAX = the number of divisions Ma, so the execution cycle Ta = 10/10 = 1. That is, the process A is executed once at a time, that is, executed every time. Further, the execution cycle T (hereinafter referred to as “execution cycle Tb”) corresponding to the process B is the division number MAX ≠ the division number Mb, and the execution cycle Tb = 10/5 = 2. That is, the process B is executed once every two times.

  In the first loop (j = 1), the next (second) input image data (1 / Ma input image data) necessary for the process A, which is the first (j = 1) image processing. ) Is read from the DRAM 4 in advance, it is determined whether or not to activate the read DMA (second read DMA) of the channel corresponding to the process A. Here, since the remainder when the count value of loop (i) (here, i = 1) is divided by the execution cycle Ta = 1 is “0”, when the read DMA of the channel corresponding to the process A is activated. judge.

  In the second loop (j = 2), the next (second) input image data (1 / Mb input image data) necessary for the second (j = 2) image processing B ) Is read from the DRAM 4 in advance, it is determined whether to activate the read DMA (second read DMA) of the channel corresponding to the process B. Here, since the remainder when the count value of loop (i) (here, i = 1) is divided by the execution cycle Tb = 2 is other than “0”, the read DMA of the channel corresponding to the process B is activated. Judge that not.

  If it is determined in step S109 that the read DMA of the channel corresponding to the process is to be activated, the corresponding pre-flag is turned ON (read DMA is activated) (step S110), and the read DMA of the channel corresponding to the process is activated. If it is determined not to do so, the corresponding pre-flag is turned OFF (read DMA is not activated) (step S111).

  In the loop (j) from step S109 to step S111, it is determined whether or not to start the read DMA for reading the next input image data from the DRAM 4 in advance, and changing the preflag corresponding to each process. Repeat for the types of image processing to be executed at the same time (in parallel). When the determination for each image processing executed at the same time (in parallel) is completed, the loop (j) is exited.

Subsequently, the image processor 5 activates the read DMA (second read DMA) of the channel corresponding to the process in which the pre-flag is turned on, that is, the read DMA for reading from the DRAM 4 in advance is activated. (Step S112).
Here, only the read DMA of the channel corresponding to the process A is activated according to the results of the processes in steps S109 to S111.

  In the image processor 5, while the second input image data is being read from the DRAM 4 by the second read DMA of the channel corresponding to the process A, the post flag is ON (the state in which the calculation is executed). The respective image processing operations on the first input image data of the processing are executed at the same time (in parallel). Here, while the second input image data (1 / Ma input image data) necessary for the process A is being read from the DRAM 4, the first input image data (1 / Ma input image data, and Two types of image processing (calculation of processing A and processing B) for 1 / Mb input image data) are executed at the same time (in parallel) (step S113 and step S114). Note that the flowchart shown in FIG. 6 shows an example in which the calculation of process A (step S113) and the calculation of process B (step S114) are executed sequentially.

  Thereafter, when the two types of image processing are completed, the image processor 5 determines whether or not the current loop (i) is the first loop (i = 1) (step S115). In step S115, when the current loop (i) is the first loop (i = 1), the image processor 5 is one of the processes in which the post flag is ON (the state in which the calculation is executed). A write DMA (first write DMA) for writing back the first image processing result after image processing is performed on the first input image data to the DRAM 4 is started (step S117). Here, since processing A and processing B are performed on the first input image data (1 / Ma input image data and 1 / Mb input image data), processing A and processing B The write DMA (first write DMA) of the channel corresponding to each is activated.

  In the flowchart shown in FIG. 6, the calculation of process A (step S113) and the calculation of process B (step S114) are sequentially executed, and after two types of image processing are completed, the channels corresponding to the respective processes are executed. The case where the write DMA is activated (step S117) has been described. However, the procedure for executing the image processing and writing back the image processing result after the image processing to the DRAM 4 is not limited to the processing procedure from step S113 to step S117 shown in FIG. For example, in the same manner as the processing from step S8 to step S12 in the processing procedure of the multiple processing execution method of the first embodiment shown in FIG. A procedure for writing back the image processing result to the DRAM 4 may be employed.

  The image processor 5 corresponds to a process in which the pre-flag is ON (here, only the process A) in the second process (i = 2) of the loop (i) that repeats the number i of the number of divisions MAX. Wait for the end of the second read DMA of the channel (step S107). When the second read DMA is completed, the image processor 5 copies the pre-flag state (here, only process A is ON) to the post flag (step S108).

  Subsequently, the image processor 5 inputs the next (third or second) input for each image processing in the loop (j) that repeats the type of image processing to be executed at the same time (in parallel). It is determined whether or not to activate read DMA (third or second read DMA) for reading image data from the DRAM 4 in advance (step S109).

  Here, since it is the second loop (j = 2), the next (third) input image data (1 / Ma) required for the process A which is the first (j = 1) image processing. In order to read the input image data) from the DRAM 4 in advance, it is determined whether or not to activate the read DMA (third read DMA) of the channel corresponding to the process A. Further, in order to read out the next (second) input image data (1 / Mb input image data) necessary for the second (j = 2) image processing B in advance from the DRAM 4, the processing is performed. It is determined whether or not to activate the read DMA (second read DMA) of the channel corresponding to B.

  For example, if the division number Ma of process A = 10, the division number Mb of process B = 5, and the division number MAX = 10, in step S109, the count value of the second loop (i) (here, i = 2) ) Is divided by the execution cycle Ta = 1, the remainder is “0”, and it is determined that the read DMA of the channel corresponding to the process A is activated. Since the remainder when the count value (here i = 2) of the second loop (i) is divided by the execution cycle Tb = 2 is “0”, the read DMA of the channel corresponding to the process B is also performed. Determine to start.

  In step S109, the pre-flag corresponding to process A and process B determined to activate the read DMA of the channel corresponding to the process is set to ON (read DMA is activated) (step S110). When the determination for each image processing executed at the same time (in parallel) is completed, the loop (j) is exited.

  Subsequently, the image processor 5 reads the channel whose pre-flag is ON, that is, the read DMA (third read DMA) of the channel corresponding to the processing A that has been determined to activate the read DMA for reading from the DRAM 4 in advance. Then, the read DMA (second read DMA) of the channel corresponding to the process B is activated (step S112).

  The image processing processor 5 performs two processes in which the post flag is ON (a state in which the operation is executed) while the input image data is being read from the DRAM 4 by the read DMA of the channels corresponding to the processes A and B. Each image processing operation on the input image data of the eye is executed at the same time (in parallel). Here, the third input image data (1 / Ma input image data) required for the process A and the second input image data (1 / Mb input image data) required for the process B are stored in the DRAM 4. During the reading, the processing A is performed on the second input image data (1 / Ma input image data) (step S113). Note that the calculation of the process B (step S114) is not executed.

  Thereafter, when the image processing of process A is completed, the image processor 5 determines whether or not the current loop (i) is the first loop (i = 1) (step S115). In step S115, when the current loop (i) is not the first loop (i = 1), the image processor 5 performs the first process in which the post flag is ON (the state in which the calculation is executed). The end of the write DMA is awaited (step S111). Here, since only the process A is ON, the end of the first write DMA of process A is awaited.

  When the first write DMA of the process A is completed, the image processor 5 performs the image process on the second input image data of the process whose post flag is ON (a state in which the calculation is executed). The write DMA (second write DMA) for writing back the subsequent second image processing result to the DRAM 4 is activated (step S117). Here, since the post flag is ON only for process A, the second image processing result of process A obtained by performing image processing on the second input image data (1 / Ma input image data). Is written back into the DRAM 4, the write DMA (third write DMA) of the channel corresponding to the process A is started.

  Thereafter, similarly, the image processor 5 performs the processing of the loop (i) from step S107 to step S117 as many as the maximum number of divisions of the input image data in all the image processing executed this time (the number of divisions MAX). I) is repeated. Thereafter, when the image processor 5 repeats the processing of the loop (i) from step S107 to step S117 for the same number i as the division number MAX, that is, the input image for one frame captured by the capture unit 3. When the two types of image processing for the data are completed, the processing loop is exited. Then, the image processor 5 waits for the end of the last write DMA of the process in which the post flag is ON (the state in which the calculation is executed) (step S118). When the last write DMA of the process in which the post flag is ON ends, that is, when the image processing result for one frame captured by the capture unit 3 is written back to the DRAM 4, the image processor 5 sends an end interrupt signal. The data is output to the host CPU 6 (step S119).

  In the interrupt process, the host CPU 6 accepts an end interrupt signal from the image processor 5 (step S101). When the host CPU 6 receives the end interrupt signal from the image processor 5, it determines the cause of the received interrupt (step S102). Here, since it is determined in step S <b> 102 that the cause of the interruption is an end interruption signal from the image processor 5, the host CPU 6 activates the image processor 7 and the image processor 5 writes back to the DRAM 4. Various image processing is performed on the image processing result (step S104).

  As described above, in the imaging apparatus 10 to which the multiple processing execution method of the second embodiment is applied, the image processor 5 performs multiple types (for example, N types) of image processing at the same time (in parallel). Based on the ratio of the number of divisions of the image data in the different image processing to be executed, the activation of the DMA and the execution of the operation are controlled. That is, in the multiple processing execution method of the second embodiment, activation of read DMA and write DMA of channels corresponding to multiple types (for example, N types) of image processing, and execution of respective image processing operations are performed. Control is performed according to the number of divisions of image data in each image processing. In other words, in the multiple process execution method of the second embodiment, while executing the loop (i) of the process A having the maximum number of divisions, the process B having the maximum number of divisions is executed. Similar to the multiple processing execution method of the first embodiment, the host CPU 6 causes the image processor 5 to perform multiple types (for example, N types) of different image processing at the same time (in parallel). As a result, in the multiple processing execution method of the second embodiment, multiple types (for example, N types) of different image processing for processing image data with different division numbers are executed at the same time (in parallel). However, the corresponding data transfer (DMA) and the respective image processing operations can be executed continuously. As a result, the multi-process execution method of the second embodiment is a factor that degrades the performance of the system in the conventional processor processing, as in the multi-process execution method of the first embodiment. The waiting time for data transfer and computation can be used for the time to execute other processes executed at the same time (in parallel), reducing the waiting time for the entire system processing and reducing the system performance. Can be prevented.

  Also, in the multi-process execution method of the second embodiment, as in the multi-process execution method of the first embodiment, in the conventional processor process, each time each process is executed alone. The number of handshake processes can be reduced to once for each of a plurality of types (for example, N types) of processes executed simultaneously (in parallel). As a result, in the multi-process execution method of the second embodiment, as in the multi-process execution method of the first embodiment, the processing time required for handshaking in the entire system process is reduced, and the system performance is reduced. Further, it can be prevented.

  In the processing procedure of the multiple processing execution method according to the second embodiment, two different types of image processing (processing A and processing B) for processing the input image data with different division numbers are performed at the same time (in parallel). Explained the processing procedure for execution. However, as described above, in the multiple processing execution method of the second embodiment, it is possible to control the start of DMA and the execution of computations according to the number of divisions of image data when executing image processing. . For this reason, in the multiple processing execution method of the second embodiment, any one of the image processes executed at the same time (in parallel) is an image process for executing the image process without dividing the image data. However, by considering the number of divisions as “1”, it is possible to similarly control the activation of the DMA and the execution of the calculation.

  In the multiple processing execution method of the first embodiment and the multiple processing execution method of the second embodiment, the image processor 5 performs two different types of image processing (Process A and Process B) at the same time. The case of applying (in parallel) has been described. However, the number of image processes that the image processor 5 executes at the same time (in parallel) is not limited to two types, and the capacity of the built-in memory used when executing the image process is not limited to the image process. As long as the capacity of the built-in memory included in the processor 5 is not exceeded, more types of image processing can be combined and executed at the same time (in parallel).

<Third Embodiment>
Next, a multiple processing execution method according to the third embodiment of the present invention will be described. In the following description, similarly to the first embodiment and the second embodiment, a case where the multiple processing execution method of the third embodiment is applied to the imaging apparatus 10 will be described. The schematic configuration of the imaging apparatus 10 to which the multiple processing execution method of the third embodiment is applied and the imaging apparatus 10 related to the execution of the multiple processing execution method of the third embodiment are shown in FIGS. 1 and 2. Since it is the same as that of the imaging device 10 to which the multiple processing execution method of the first embodiment shown is applied, detailed description thereof is omitted. Therefore, in the following description, it is assumed that the imaging apparatus 10 to which the multiple processing execution method of the first embodiment shown in FIGS. 1 and 2 is applied executes the multiple processing execution method of the third embodiment. Give an explanation. In the multiple processing execution method of the third embodiment, the host CPU 6 provided in the imaging apparatus 10 considers the capacity of the built-in memory provided in the image processor 5 for the image processing combination that the image processor 5 executes. It is a method to change dynamically.

  In the multiple processing execution method of the third embodiment, data transfer time and calculation are performed for multiple types (for example, N types) of image processing executed by all the image processing calculation units 52 included in the image processing processor 5. Each time is measured in advance, and each measured value is stored as program data of the host CPU 6 in, for example, a program memory. The host CPU 6 determines a combination of image processings to be instructed to be executed this time by the image processor 5 based on the data transfer time and calculation time measured in advance. The host CPU 6 determines the combination of image processings for instructing the image processing processor 5 to execute based on the difference between the data transfer time and the calculation time. Therefore, the value stored in advance as the program data may be a difference time between the data transfer time and the calculation time.

  In the multiple processing execution method of the third embodiment, the maximum number of processes executed in the image processor 5 at the same time (in parallel), that is, executed in combination is “the maximum number of combined processes”. As previously defined. The maximum number of combined processes is such that the image processor 5 has a capacity of a built-in memory used when executing image processing when a plurality of types of image processing executed in accordance with the operation mode of the imaging apparatus 10 are combined. It must be set within the range that does not exceed the capacity of the internal memory.

  Next, a processing procedure of the multiple processing execution method of the third embodiment in the imaging apparatus 10 will be described. FIG. 7 is a flowchart showing the processing procedure of the multiple processing execution method according to the third embodiment. The flowchart shown in FIG. 7 shows the operations of the host CPU 6 and the image processor 5 in the multiple processing execution method of the third embodiment. Note that, as described above, the image processor 5 can perform a plurality of types (for example, N types) of different image processing at the same time (in parallel). In the following description, a processing procedure when four types of different image processing (processing A, processing B, processing C, and processing D) are performed on input image data will be described.

  Here, the processing A and the processing B are image processing whose calculation time is longer than the data transfer time, and the processing C and the processing D are image processing whose calculation time is shorter than the data transfer time. Shall. In the processing A and the processing B, the time of the difference between the data transfer time and the calculation time is larger in the processing A, and in the processing C and the processing D, the time of the difference between the data transfer time and the calculation time. Suppose that process C is larger. Further, it is assumed that “2” is preset as the maximum number of combined processes in the host CPU 6. Therefore, in the following description, a case will be described in which the image processor 5 executes four types of image processing in two steps twice.

  When the operation of the imaging device 10 is started, the host CPU 6 selects image processing to be executed according to the operation mode of the imaging device 10 (step S201). Here, as described above, four different image processes (Process A, Process B, Process C, and Process D) are selected.

  Subsequently, the host CPU 6 calculates a time difference diff [i] between the data transfer time and the calculation time in each selected image processing (step S202). In the calculation of the time difference diff [i] in step S202, for each image processing (process A, process B, process C, and process D), the time difference diff [ i] is calculated. Here, “i” distinguishes the time difference diff corresponding to each image processing. In the following description, the time differences diff corresponding to the processes A, B, C, and D are respectively the time difference diff [A], the time difference diff [B], the time difference diff [C], and the time difference diff [ D]. Note that if the time difference between the data transfer time and the calculation time, that is, the time difference diff [i] is stored in advance as program data, step S202 is not necessary.

  Here, as described above, processing A and processing B have a calculation time longer than the data transfer time, and processing C and processing D have a calculation time shorter than the data transfer time. Therefore, the time difference diff [A] and the time difference diff [B] are positive values, and the time difference diff [C] and the time difference diff [D] are negative values. Further, as described above, in the processing A and the processing B, the processing A has a larger time difference between the data transfer time and the calculation time. In the processing C and the processing D, the processing C is more data. The difference between the transfer time and the calculation time is large. Therefore, the time difference diff [A] and the time difference diff [B] have a larger time difference diff [A], and the time difference diff [C] and the time difference diff [D] have the absolute value of the time difference diff [C]. Is big.

  Subsequently, the host CPU 6 repeats the loop that repeats the number of times of the maximum number of combined processes as many times as the number of processes (hereinafter referred to as “the number of processes TASK_NUM”) for which the time difference diff [i] is a positive value, 5 determines a combination of image processings instructed to be executed at the same time (in parallel). As described above, the processes in which the time difference diff [i] is a positive value are the processes A and B. Therefore, the processing number TASK_NUM is “2”.

  First, in a first process (here, a process corresponding to process A) that is repeated by the number of processes TASK_NUM, a loop that is repeated by the number of times of the maximum number of combined processes is executed. In the loop that repeats the number of times of the maximum number of combined processes, first, search for other processes to be combined with the current target process (step S203). In the process search in step S203, a process is searched for in which the absolute value of the time difference diff [i] of each process is a negative value closest to the value of the inner ring of the time difference diff [i] of the current process. . Here, since the target process is the process A, the absolute value of the time difference diff [C] closest to the inner ring value of the time difference diff [A] of the process A or the absolute value of the time difference diff [D] of the process D Detect value. Then, either process C or process D is selected as a process to be combined with process A. Note that the process B is not a process to be combined with the search target, that is, the process A because the time difference diff [B] is a positive value.

  Subsequently, the host CPU 6 determines whether or not there is a combination process (step S204). In step S204, if there is a process to be combined, the host CPU 6 adds the time difference diff [i] of the selected process to the time difference diff [i] of the process that is the current target, and a new one after combining the processes. The time difference diff [i] is calculated (step S205). The value of the new time difference diff [i] calculated here is a positive value obtained by subtracting the value of the time difference diff [i] of the selected process from the value of the time difference diff [i] of the current process. .

  Thereafter, the host CPU 6 replaces the time difference diff [i] of the current target process with the calculated new time difference diff [i], and performs the loop process from step S203 to step S205 to the maximum number of combined processes. Repeat as many times. Thereafter, when the host CPU 6 repeats the loop processing from step S203 to step S205 for the same number of times as the maximum number of combined processes, the host CPU 6 exits the processing loop. In step S204, if there is no combination process, that is, before repeating the same number of times as the maximum number of combined processes, the negative value closest to the value of the inner ring of the time difference diff [i] of the target process is a negative value. If there is no process, the process loops from step S203 to step S205 is exited.

  Subsequently, the host CPU 6 similarly executes a loop process that repeats the number of times of the maximum number of combined processes in the second process (here, the process corresponding to the process B) that is repeated by the number of processes TASK_NUM.

  Thereafter, similarly, the host CPU 6 repeats the loop processing that is repeated the number of times of the maximum number of combined processings as many times as the processing number TASK_NUM. After that, the host CPU 6 repeats the loop process that repeats the number of times of the maximum number of combined processes as many times as the number of processes TASK_NUM, that is, for all processes in which the time difference diff [i] is a positive value. When the image processing processor 5 determines another combination of image processing to be executed at the same time (in parallel), the processing loop is exited.

  Here, since the number of processes TASK_NUM = 2, the host CPU 6 repeats the loop process repeated twice as many times as the number of processes TASK_NUM. Since the maximum number of combined processes = 2, the host CPU 6 combines two types of image processes at maximum. In the following description, it is assumed that process A is combined with process A, and process D is combined with process B.

  Subsequently, the host CPU 6 resets (initializes) a count value task_cnt indicating the number of combinations of image processing executed by the image processor 5, that is, sets it to “0”. Then, the host CPU 6 starts a process for executing a plurality of processes.

  In the multi-process execution process, the host CPU 6 first executes an interrupt process (step S207). In the interrupt process of step S207, the host CPU 6 receives an interrupt signal from each component in the imaging device 10. Thereafter, when the capture unit 3 stores the input image data obtained by capturing one frame of the image signal output from the imager 2 in the DRAM 4, the capture unit 3 outputs a capture completion signal to the host CPU 6.

  When the host CPU 6 accepts an interrupt signal from each component in the imaging device 10, the cause of the accepted interrupt is determined (step S208). In step S208, if the cause of the interruption is a capture completion signal from the capture unit 3, the host CPU 6 activates the image processor 5 (step S211). When the host CPU 6 activates the image processing processor 5, the host CPU 6 outputs an execution instruction to the image processing processor 5, and the combination of image processing to be executed first (here, first, processing A and processing C 2 Information representing the combination of types is output to the image processor 5. For the information representing the combination of image processing to be executed, a flag representing each of a plurality of types (for example, N types) of image processing arithmetic units 52 provided in the image processing processor 5 with different bits is used. In this case, the host CPU 6 outputs a flag with the bit corresponding to the image processing to be executed this time turned ON to the image processing processor 5.

  When the activation is instructed by the host CPU 6, that is, when an execution instruction and a flag indicating information of image processing to be executed are input from the host CPU 6, the image processing processor 5 starts executing the image processing of the instructed combination. . First, the image processor 5 dynamically secures the capacity of the built-in memory required for image processing by the image processing arithmetic unit 52 corresponding to the bit turned ON in the input flag (step S214). ).

  The image processor 5 divides the input image data captured by the capture unit 3 for one frame into a plurality (for example, M) corresponding to each process, and performs image processing on each of the divided input image data. Is repeated for the number of times divided according to each processing, thereby completing the image processing for one frame. Therefore, the capacity of the built-in memory that is dynamically secured in correspondence with each image processing calculation unit 52 is a small capacity necessary for performing image processing on the divided input image data. In the following description, the number of divisions of input image data in each of the image processing of process A, process B, process C, and process D is represented as division number Ma, division number Mb, division number Mc, and division number Md, respectively. To do.

  Subsequently, the image processor 5 splits the first (first) input image data (1 / Ma input image data) necessary for each of the two types of image processing (processing A and processing C) to be executed this time. , And 1 / Mc input image data) is read from the DRAM 4, the read DMA (first read DMA) corresponding to the bit whose flag is ON is activated (step S 215). Next, the image processor 5 waits for the end of the first read DMA corresponding to the bit whose flag is ON in the first loop of the divided processing (step S216). When the first read DMA is completed, the image processor 5 reads the next (second) input image data (1 / Ma input image data and 1 / Mc input image data) from the DRAM 4 in advance. Then, the read DMA (second read DMA) corresponding to the bit whose flag is ON is activated (step S217).

  The image processor 5 corresponds to the bit for which the flag is ON for the first input image data while the second input image data is being read from the DRAM 4 by the second read DMA. Two types of image processing (operations of processing A and processing C) are executed at the same time (in parallel) (steps S218 and S219). Note that the flowchart shown in FIG. 7 shows an example in which the calculation of the first process (step S218) and the calculation of the second process (step S219) are sequentially executed. Here, process A corresponds to the first process, and process C corresponds to the second process.

  Thereafter, when the two types of image processing corresponding to the bit whose flag is ON are completed, the image processing processor 5 determines whether or not the current processing loop is the first loop (step S220). . In step S220, when the current processing loop is the first loop, the image processor 5 determines that the first input image data (1 / Ma input image data and 1 / Mc input image data). ) To write back each first image processing result after image processing to DRAM 4 to activate the write DMA (first write DMA) corresponding to the bit whose flag is ON. (Step S222).

  In the flowchart shown in FIG. 7, the calculation of the first process (process A) (step S218) and the calculation of the second process (process C) (step S219) are sequentially executed, and two types of image processing are performed. A case has been described in which the write DMA corresponding to the bit whose flag is ON is started after step S <b> 222 is completed (step S <b> 222). However, the procedure of executing the image processing and writing back the image processing result after the image processing to the DRAM 4 is not limited to the processing procedure from step S218 to step S222 shown in FIG. For example, in the same manner as the processing from step S8 to step S12 in the processing procedure of the multiple processing execution method of the first embodiment shown in FIG. A procedure for writing back the image processing result to the DRAM 4 may be employed.

  Then, the image processor 5 waits for the end of the second read DMA corresponding to the bit whose flag is ON in the second loop of the divided processing (step S216). When the second read DMA is completed, the image processor 5 reads the next (third) input image data (1 / Ma input image data and 1 / Mc input image data) from the DRAM 4 in advance. Then, the read DMA (third read DMA) corresponding to the bit whose flag is ON is activated (step S217).

  While the third input image data is being read from the DRAM 4 by the third read DMA, the image processor 5 corresponds to the bit for which the flag is ON for the second input image data. Two types of image processing (operations of processing A and processing C) are executed at the same time (in parallel) (steps S218 and S219).

  Thereafter, when the two types of image processing corresponding to the bit whose flag is ON are completed, the image processing processor 5 determines whether or not the current processing loop is the first loop (step S220). . If the current processing loop is not the first loop in step S220, the image processor 5 waits for the end of the first write DMA corresponding to the bit whose flag is ON (step S221). When the first write DMA corresponding to the bit whose flag is ON is completed, the image processor 5 reads the second input image data (1 / Ma input image data and 1 / Mc input image data). ) Is started, the write DMA corresponding to the bit whose flag is ON (second write DMA) is started in order to write back each second image processing result after the image processing to the DRAM 4. (Step S222).

  Thereafter, similarly, the image processor 5 repeats the processing from step S216 to step S222 by the number of times (for example, M times) obtained by dividing the input image data. Thereafter, when the image processor 5 repeats the processing from step S216 to step S222 as many times as the number of divided input image data, that is, for one frame of input image data captured by the capture unit 3 When the two types of image processing corresponding to the bit for which the flag is ON are completed, the processing loop is exited. Then, the image processor 5 waits for the end of the last write DMA corresponding to the bit whose flag is ON (step S223). The last write DMA corresponding to the bit whose flag is ON ends, that is, corresponding to the bit whose flag is ON executed for one frame of input image data captured by the capture unit 3 When the two types of image processing results (image processing results of processing A and processing C) are written back to the DRAM 4, the image processing processor 5 outputs an end interrupt signal to the host CPU 6 (step S224).

  In the interrupt process, the host CPU 6 accepts an end interrupt signal from the image processor 5 (step S207). When the host CPU 6 receives the end interrupt signal from the image processor 5, it determines the cause of the received interrupt (step S208). Here, in step S208, it is determined that the cause of the interruption is the end interruption signal from the image processor 5.

  Subsequently, the host CPU 6 adds “1” to the count value task_cnt, that is, advances the value of the count value task_cnt by one (step S209). Then, the host CPU 6 determines whether or not all image processing (processing A, processing B, processing C, and processing D) to be performed on the input image data has been completed (step S210). In the determination in step S210, it is determined whether or not all image processing has been completed depending on whether or not the count value task_cnt has reached the value of the processing number TASK_NUM. When the count value task_cnt has reached the value of the processing number TASK_NUM, it is determined that all image processing has been completed, and when the count value task_cnt has not reached the value of the processing number TASK_NUM. It is determined that all image processing has not been completed. In the multi-process execution method of the third embodiment, the process in which the time difference diff [i] is a positive value is combined with the process in which the time difference diff [i] is a negative value. Therefore, image processing can be combined by the value of the processing number TASK_NUM, which is the number of processes for which the time difference diff [i] is a positive value. For this reason, when the image processing processor 5 executes image processing that is combined by the number of processing times TASK_NUM, all image processing is completed.

  Here, since it is determined in step S210 that all image processing has not been completed, the host CPU 6 activates the image processing processor 5 again (step S211). When the host CPU 6 activates the image processing processor 5, the host CPU 6 outputs an execution instruction to the image processing processor 5 and a combination of image processing to be executed next (here, first, processing B and processing D are 2 Information representing the combination of types is output to the image processor 5.

  When the activation is instructed by the host CPU 6, the image processor 5 starts executing the image processing of the instructed combination, and performs processing B and processing D corresponding to the bit turned ON in the input flag. The two types of image processing arithmetic units 52 dynamically secure the capacity of the built-in memory necessary for image processing (step S214).

  Subsequently, the image processor 5 splits the first (first) input image data (1 / Mb input image data) necessary for each of the two types of image processing (processing B and processing D) to be executed this time. , And 1 / Md input image data) is read from the DRAM 4, the read DMA (first read DMA) corresponding to the bit whose flag is ON is activated (step S 215).

  Thereafter, the two types of image processing of the processing B and the processing D are executed in the same procedure as the two types of image processing of the processing A and the processing C described above. Note that the two types of image processing, processing B and processing D, have been described above except that processing B corresponds to the first processing (step S218) and processing D corresponds to the second processing (step S219). Since it is executed in the same procedure as the two types of image processing of processing A and processing C, detailed description is omitted.

  Thereafter, two types of image processing (processing B and processing D) corresponding to the bit whose flag is ON are completed, that is, executed for one frame of input image data captured by the capture unit 3, When the two types of image processing results (image processing results of processing B and processing D) corresponding to the bit whose flag is ON are written back to the DRAM 4, the image processing processor 5 outputs an end interrupt signal to the host CPU 6. (Step S224).

  In the interrupt process, the host CPU 6 accepts an end interrupt signal from the image processor 5 (step S207). When the host CPU 6 receives the end interrupt signal from the image processor 5, it determines the cause of the received interrupt (step S208). Here, in step S208, it is determined that the cause of the interruption is the end interruption signal from the image processor 5.

  Subsequently, the host CPU 6 adds “1” to the count value task_cnt and advances the count value task_cnt by one (step S209). Then, the host CPU 6 determines whether or not all image processing (processing A, processing B, processing C, and processing D) to be performed on the input image data has been completed (step S210). Here, since it is determined in step S210 that all image processing has been completed, the host CPU 6 activates the image processing unit 7 and performs various processing on the image processing results written back to the DRAM 4 by the image processing processor 5. Image processing is performed (step S212). Then, the host CPU 6 resets (initializes) the count value task_cnt, that is, sets it to “0”.

  As described above, in the imaging device 10 to which the multiple processing execution method of the third embodiment is applied, the image processing processor 5 includes a combination of image processing that the host CPU 6 causes the image processing processor 5 to execute. Dynamically determined in consideration of the capacity of the built-in memory. As a result, like the multiple processing execution method of the first embodiment, the host CPU 6 causes the image processor 5 to perform multiple types (for example, N types) of different image processing at the same time (in parallel). . As a result, in the multi-process execution method of the third embodiment as well, as in the multi-process execution method of the first embodiment, data that has been a factor of reducing the system performance in the conventional processor processing The waiting time for transfer and computation can be used for the time to execute other processes executed at the same time (in parallel), reducing the waiting time for the entire system processing and preventing the system performance from degrading. can do.

  In the imaging apparatus 10 to which the multiple processing execution method of the third embodiment is applied, when the host CPU 6 determines combinations of image processing to be executed by the image processor 5, more types of images than necessary are required. It is possible not to combine the processes. The image processor 5 dynamically secures a capacity of a built-in memory for temporarily storing input image data required when executing image processing instructed by the host CPU 6. Thereby, in the multiple processing execution method of the third embodiment, the capacity of the built-in memory provided in the image processor 5 is not compressed.

  In the imaging apparatus 10 to which the multiple processing execution method of the third embodiment is applied, the host CPU 6 determines a combination of image processing so that the image processing to be executed by the image processing processor 5 is not more than necessary. For this reason, the number of handshaking processes may be greater than the multiple process execution method of the first embodiment and the multiple process execution method of the second embodiment. However, even in the multi-process execution method of the third embodiment, a plurality of types (for example, N types) of different image processes are grouped and performed at the same time (in parallel). The number of handshake processes can be reduced from the number of handshake processes performed each time each process is executed alone. As a result, even in the multiple processing execution method of the third embodiment, it is possible to reduce the processing time required for handshaking in the processing of the entire system and further prevent the system performance from being deteriorated.

  In the multiple processing execution method of the third embodiment, the case has been described where the processing procedure of the image processing processor 5 is substantially the same as the multiple processing execution method of the first embodiment. The processing procedure 5 can be substantially the same as the multiple processing execution method of the second embodiment.

  As described above, according to the embodiment for carrying out the present invention, in the image processor, the input / output unit (DMA unit 51) that performs data transfer and the calculation unit (image processing calculation unit) that performs calculation of image processing. 52), the control of each input / output unit and the control of each calculation unit are collectively controlled. Thereby, in the form for implementing this invention, the data transfer in an image processor and the calculation of image processing can be performed at the same time (in parallel). As a result, in the mode for carrying out the present invention, the waiting time for data transfer and calculation, which has been a factor of degrading the system performance in the processing of the conventional processor, is set at the same time (in parallel). It can be devoted to the time to execute other image processing being executed. In the embodiment for carrying out the present invention, a plurality of different types of image processing can be combined (collectively) and executed at the same time (in parallel). As a result, in the mode for carrying out the present invention, the number of handshaking processes performed each time each process is executed alone in the conventional processor process is set at the same time (in parallel). The number of image processing combinations to be executed can be reduced. As a result, in the embodiment for carrying out the present invention, in a system that executes a plurality of processes, reduction of system performance due to frequent handshaking and waiting time in each process is reduced. can do.

  In the present embodiment, the case where the image processor 5 executes two types of different image processes at the same time (in parallel) has been described. However, the number of image processes that the image processor 5 executes at the same time (in parallel) is not limited to two types, and more types of image processing are executed at the same time (in parallel). Even in this case, the multi-processing execution method of the present invention can be similarly applied.

  In the present embodiment, the case where the multiple processing execution method of the present invention is applied to the imaging apparatus 10 has been described. However, the system to which the multiple processing execution method of the present invention can be applied is not limited to the mode for carrying out the present invention, and does not have a host CPU and an interrupt control mechanism controlled by the host CPU. Any system can be applied as long as the system includes a processor and the processor performs a plurality of processes.

  The embodiment of the present invention has been described above with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes various modifications within the scope of the present invention. It is.

10 ... Imaging device (data processing system)
DESCRIPTION OF SYMBOLS 1 ... Lens 2 ... Imager 3 ... Capture part 4 ... DRAM (external memory)
5. Image processing processor (processor)
51... DMA unit 52... Image processing calculation unit (processing unit)
6 ... Host CPU
7: Image processing unit 8: Recording unit 9: Memory card

Claims (4)

External memory for storing data,
A processor comprising: a plurality of processing units that process the data; and a DMA unit that has a plurality of channels corresponding to each of the plurality of processing units and accesses the external memory for each channel. When,
A host CPU that instructs execution of processing by each of the processing units included in the processor;
A method for executing multiple processes in a data processing system comprising:
After all the data read from the external memory is completed by the data read channel of the DMA unit corresponding to the processing unit instructed to be executed by the host CPU, the external memory is connected to the data read channel. A read DMA step for starting reading of the next data from
In parallel with the reading of the next data from the external memory by the data reading channel, the processing unit instructed to execute by the host CPU performs processing on the data that has been read from the external memory. Multiple process execution steps to be executed;
A write DMA step for starting writing the result data that the processing unit has completed processing to the external memory into the data write channel of the DMA unit corresponding to the processing unit that is instructed to be executed by the host CPU;
including,
A method for executing multiple processes. The read DMA step includes:
Based on the execution cycle corresponding to each of the processing units, which represents the cycle of whether or not the processing unit executes the processing on the data, the data reading channel corresponding to each of the processing units is set to the following A read DMA execution determination step for determining whether to start reading the data;
Including
In the data read channel determined to start reading the next data, start reading the next data,
After all the next reading of the data from the external memory by the data reading channel that has started reading the data is completed, a next read DMA execution determination step is executed,
The multiple processing execution step includes:
Before the read DMA execution determination step determines whether or not to start reading the next data, the processing unit corresponding to the data read channel that has started the data read from the external memory Causing the data that has been read to be processed,
The write DMA step includes:
Causing the data writing channel corresponding to the processing unit that has performed processing on the data to start writing the result data that the processing unit has completed to the external memory;
The multiple processing execution method according to claim 1, wherein: The read DMA execution determination step includes:
A DMA execution state corresponding to each of the data read channels, which indicates whether or not to perform the data read by the data read channel;
A process execution state corresponding to each of the processing units, which indicates whether or not to execute the process on the data by the processing unit;
Set
Before determining whether to start reading the next data,
By copying the current DMA execution state to the processing execution state, the processing unit corresponding to the data read channel for executing the data reading is in a state of executing processing on the data,
In determining whether to start reading the next data,
When the execution cycle is the timing at which the processing unit executes processing on the data, the data read is determined to start reading the next data and is determined to start reading the next data Updating the DMA execution state corresponding to the communication channel to a state in which the data is read out;
When the execution cycle is a timing at which the processing unit does not execute the process on the data, the data read is determined not to start reading the next data and not to start reading the next data Updating the DMA execution state corresponding to the communication channel to a state in which the reading of the data is not executed,
After determining whether to start reading the next data,
The DMA execution state starts reading the next data to the data read channel in which the data read is executed,
The DMA execution state is a state in which reading of the data is executed. After the next reading of the data from the external memory by the data reading channel is completed, the next read DMA execution determination step is executed. ,
The multiple processing execution step includes:
The processing execution state is a state of executing processing on the data, causing the processing unit to execute processing on the data that has been read from the external memory,
The write DMA step includes:
The processing execution state causes the data writing channel corresponding to the processing unit in a state of executing processing on the data to start writing the result data that has been processed by the processing unit to the external memory. ,
The multiple processing execution method according to claim 2, wherein: The multiple processing execution method further includes:
A processing time required for the processing unit measured in advance to execute processing on the data, and a data transfer time required to read the data from the external memory by the data reading channel corresponding to the processing unit. An execution process determining step for determining a combination of the processing units that the host CPU instructs the processor to execute based on the difference;
According to the combination of the processing units instructed to be executed by the host CPU, each of the combined processing units read by the data reading channel corresponding to the processing unit required when executing the processing. A buffer securing step for dynamically securing a capacity of an internal memory for temporarily storing data;
Including
The multiple processing execution step includes:
The processing for each piece of data that has been read from the external memory by the corresponding data reading channel is executed on each processing unit that is combined by the execution processing determining step and instructed to be executed by the host CPU. Let
The write DMA step includes:
Combined by the execution process determination step, the data write channel corresponding to each of the processing units instructed to be executed by the host CPU has the processing data of each of the result data that has been processed by each of the processing units. Start writing to external memory,
The multiple processing execution method according to claim 1, wherein:

Images