JP2009295163A - Image processing device and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing device and image processing method for easily enhancing the processing efficiency of imaging processing having dependency between pixels. <P>SOLUTION: This image processing device includes a common memory 3 for storing an image, a plurality of processors 6 that are connected to the common memory and execute image processing, and a controller 5 for dividing the image stored in the common memory into a plurality of regions, setting the divided regions to optional processors, respectively, and parallel-processing the set processors while sequentially shifting their processing start timings. The processing region in which a precedent processor precedingly starting operation performs the image processing includes a pixel for satisfying the dependency between pixels firstly processed by a following processor subsequently starting operation. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus having a plurality of general-purpose processor cores, and relates to a technique for speeding up image processing of a type in which pixel processing results are determined depending on other adjacent pixels.

  Conventionally, when a large amount of image processing is to be realized in real time, dedicated hardware such as ASIC (Application Specific Integrated Circuit) is used and the image processing algorithm is fixed to the dedicated hardware. 2. Description of the Related Art In recent years, an image processing technique using a SIMD (Single Instruction Multiple Data) type arithmetic unit and hardware for performing an operation between slots (between adjacent pixels) of a SIMD type data register is known. In this image processing technique, parameters and algorithms can be flexibly changed by rewriting programs that run on these hardware.

  As a technique for disclosing such SIMD type technology, the technology described in Patent Document 1 is known. The technique described in this Patent Document 1 decomposes error diffusion type image processing into pixel row processing in two directions, horizontal and vertical directions, the SIMD type arithmetic unit in the vertical direction and the sequential processing in the horizontal direction. High-speed processing is realized by processing using a circuit.

  However, dedicated hardware is required to implement the technique described in Patent Document 1. When further improving the performance, it is not possible to increase the processing efficiency simply by adding dedicated hardware. Therefore, new dedicated hardware is required to further improve the performance. That is, the technique described in Patent Document 1 has a problem that it is difficult to easily improve the processing efficiency.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide an image processing apparatus and an image processing method capable of easily realizing improvement in processing efficiency of image processing having a dependency relationship between pixels. To do.

  In order to solve the above problems, the present invention provides a shared memory for storing an image, a plurality of processors connected to the shared memory for executing image processing, and an image stored in the shared memory divided into a plurality of areas. A processing area in which the preceding processor that starts the operation in advance has a controller that sets the divided areas in arbitrary processors, and sequentially shifts the processing start timings of the set processors to perform parallel processing. The image processing apparatus includes a pixel that satisfies the inter-pixel dependency of the pixel that is subsequently processed by the subsequent processor that starts the operation.

  The present invention also provides a shared memory for storing an image, a plurality of processors connected to the shared memory for executing image processing, an image stored in the shared memory being divided into a plurality of areas, Each group is set as an arbitrary processor, a plurality of groups in which at least a plurality of processors are grouped into one group is set, and a plurality of processors belonging to the one group are started to operate at the same time. A controller that sequentially processes start timings and performs parallel processing, and a processor belonging to the preceding group that starts the operation in advance performs processing within the processing area for all subsequent members that start the operation. The image processing apparatus includes a pixel that satisfies the inter-pixel dependency of a pixel that is first processed by the processor.

  The present invention is also directed to an image processing system in which a plurality of image processing apparatuses according to the above-described inventions are connected to each other via a communication path, wherein a controller of one image processing apparatus is a part of the divided processing area. Image processing system for transmitting and processing the image area to another image processing apparatus and controlling a timing signal transmission / reception operation for starting the image processing accompanying pixel dependence with the other image processing apparatus It is.

  According to another aspect of the present invention, there is provided an image processing method of an image processing apparatus including a shared memory for storing an image and a plurality of processors connected to the shared memory to execute image processing. In the processing area where the divided processor sets each divided area to an arbitrary processor, shifts the processing start timing of the set processor sequentially and performs parallel processing, and the preceding processor that starts the operation in advance performs image processing. Is an image processing method including a pixel that satisfies the inter-pixel dependency of a pixel that is subsequently processed by a subsequent processor that starts the operation.

  According to another aspect of the present invention, there is provided an image processing method of an image processing apparatus including a shared memory for storing an image and a plurality of processors connected to the shared memory to execute image processing. Dividing into areas, setting the divided areas as arbitrary processors, setting a plurality of groups in which at least a plurality of processors are grouped into one group, and operating a plurality of processors belonging to the one group simultaneously In the processing area in which the processor belonging to the preceding group that starts the operation in advance performs image processing in the processing area that sequentially performs the processing start timing for each group, the succeeding group that starts the operation subsequently An image containing a pixel that satisfies the inter-pixel dependency of the pixel that is first processed by all processors belonging to It is a management method.

  The present invention also provides an image processing method of an image processing system in which a plurality of image processing apparatuses according to the above-described inventions are connected to each other via a communication path, wherein one image processing apparatus An image that transmits a part of the region to another image processing apparatus to be processed, and controls an operation of sending and receiving a timing signal for starting the image processing with pixel dependency with the other image processing apparatus It is a processing method.

  According to the image processing apparatus and the image processing method of the present invention, it is possible to easily realize improvement in processing efficiency of image processing having a dependency relationship between pixels.

1 is a diagram illustrating an image processing apparatus according to a first embodiment. 1 is a diagram illustrating an image processing system to which an image processing apparatus according to a first embodiment is applied. The figure explaining an example of the image process in which the process of the pixel made into object depends on the process result of another pixel. The figure which shows the process area | region of each processor. The figure which shows the synchronous timing between cores. The figure which shows the synchronous timing between cores. The time chart which shows operation | movement of a general purpose processor core. The figure which shows the allocation of the general purpose processor core when an image area is large. The flowchart which shows the outline process of the processor core for control. The flowchart which shows the general | schematic process of a general purpose processor core. The figure which shows the area | region allocated to a general purpose processor core. The figure showing the area | region which two processor cores which perform image processing first process. The figure showing the area | region which two processor cores which perform image processing next process. The figure which shows the timing of the synchronous process of a general purpose processor core. The figure which shows the synchronous timing between general purpose processor cores in detail. The figure which shows the image data processed with a general purpose processor core, and the processed data stored in the shared memory. The flowchart which shows the outline process of the processor core for control. The flowchart which shows the general | schematic process of a general purpose processor core. The time chart which shows operation | movement of a general purpose processor core. The figure which shows the synchronous timing between a processing area and a general purpose processor core. The figure explaining the process in case the number of image processing apparatuses connected to the external bus is one. The figure explaining the process in case there are a plurality of image processing apparatuses connected to the external bus. The flowchart which shows the outline process of the processor core for control. The flowchart which shows the general | schematic process of a general purpose processor core. The flowchart which shows the general | schematic process of a general purpose processor core.

[First embodiment]
FIG. 1 is a diagram illustrating an image processing apparatus according to the first embodiment.
The image processing apparatus 1 includes an external interface 2 that transmits / receives data to / from an external device (not shown), a shared memory 3, a memory interface 4 that transmits / receives data to / from the shared memory, a control processor core 5, and a plurality of processors that execute image processing. A general-purpose processor core 6 and a bus 7 serving as a data communication path between these pieces of hardware are provided.

  The general-purpose processor core 6 includes a local memory 8 that can be accessed faster than the shared memory 3. Further, the general-purpose processor core 6 performs an operation start timing between an arithmetic unit (not shown) that executes image processing according to a program, data transfer between the local memory 8 and the shared memory 3, and other general-purpose processor cores 6. A DMA transfer unit (not shown) that realizes synchronization processing for matching the two is provided. Each of these general-purpose processor cores 6 can be controlled by a program and can be executed in parallel.

  As a data communication method between the general-purpose processor cores 6, there are a method of directly transmitting / receiving between the local memories 8 and a method of transmitting / receiving via the shared memory 3. Any of these methods can be used. However, when a large amount of data such as image data is transmitted / received, high-speed processing can be performed by using a method of communicating via the shared memory 3.

  As for the synchronization processing for controlling the activation between the general-purpose processor cores, the time required for the synchronization processing exponentially increases as the number of general-purpose processor cores performing the synchronization processing increases. For this reason, in the present image processing apparatus 1, it is necessary to realize predetermined processing with a smaller number of synchronizations.

  The control processor core 5 controls the overall operation of the image processing apparatus 1. For example, the control processor core 5 communicates processing data generated by the general-purpose processor core 6 to the outside via the external interface 2. Further, the control processor core 5 receives an instruction from the outside via the external interface 2 and controls the general-purpose processor core 6.

  The image processing apparatus 1 shown in FIG. 1 includes the shared memory 3. However, the shared memory 3 may be an external device that can be arbitrarily connected, and may not always include the image processing apparatus. good.

  FIG. 2 is a diagram illustrating an image processing system to which the image processing apparatus according to the first embodiment is applied. This image processing system is configured as a part of an MFP (Multi Functional Peripheral) having a fax, printer, copy, and scan functions.

  That is, the image processing apparatus 1 is managed by the control unit 10 on the MFP body side as one piece of hardware constituting the MFP. Then, the image processing apparatus 1 receives the image processing program and the image data via the MFP main body side and the external interface 2, performs the image processing according to the processing program, and sends the processed image again to the MFP main body side. Output.

Next, the flow of image processing will be described with reference to FIG.
First, the control unit 10 on the MFP side takes out the image processing program from the storage area 11 in the MFP, copies it to the shared memory 3, and instructs the image processing apparatus 1 according to a predetermined procedure to perform image processing. Is started. At this time, the image data to be subjected to image processing is stored in the storage area 11 of the MFP main body, like the image processing program.

  In the image processing apparatus 1, the control processor core 5 causes each general-purpose processor core 6 to read an image processing program. In addition, the control processor core 5 acquires image data from the storage area 11 on the MFP side and stores it in the shared memory 3.

When the start of processing is instructed from the control means 10 on the MFP main body side, the control processor core 5 instructs the general-purpose processor core 6 to start image processing. Each general-purpose processor core 6 executes image processing. The image data subjected to image processing is temporarily stored in the shared memory 3. When the image processing is completed, the image data stored in the shared memory 3 is transmitted to and stored in the storage area 11 on the MFP side.
In FIG. 2, the flow of image data is indicated by arrows. The storage area 11 on the MFP side shows that unprocessed image data and processed image data are stored.

  Here, the image processing executed by the image processing apparatus 1 is processing that takes in the processing result of a certain pixel and processes other pixels. In other words, the processing of the target pixel is not performed independently of the processing result of the other pixels, but the processing of the target pixel depends on the processing result of the other pixels.

  FIG. 3 is a diagram illustrating an example of such image processing. In the image processing shown in FIG. 3, in the calculation of the target pixel, as shown in FIG. 3, the image processing results of the pixels adjacent to the left, upper left, and upper of the target pixel are used. In the image processing apparatus 1 of the first embodiment, high-speed processing is realized by processing such dependent image processing in parallel by a plurality of general-purpose processor cores 6. Note that image processing having the dependency relationship as described above includes error diffusion processing, dither processing, and the like.

  Subsequently, an image processing method in the plurality of general-purpose processor cores 6 will be described. FIG. 4 is a diagram showing areas allocated to the respective general-purpose processor cores 6.

  In parallel processing of image processing, the calculation results of the pixels on the left, upper left, and upper side are necessary as shown in FIG. 3, so the image area is divided in an oblique direction as shown in FIG. The general-purpose processor core 6 is assigned to the image processor to execute image processing. FIG. 4 clearly shows the general-purpose processor core 6 that executes processing for each area among the plurality of general-purpose processor cores 6.

  With respect to processing in each general-purpose processor core, efficient parallel processing with less overhead of synchronization processing becomes possible by performing processing while shifting the execution timing in time by a synchronization method described later.

  Details of image processing by processing in each general-purpose processor core 6 and synchronization processing between the general-purpose processor cores will be described. FIG. 5 is an enlarged view showing an area allocated to the general-purpose processor core 6 on the left side of the image area.

  The first process is executed by the general-purpose processor core (1) assigned to the leftmost area in FIG. 5. For the other general-purpose processor cores, the area located on the left side of the area in charge of the process is processed. Waiting for a synchronization signal from the general-purpose processor core (waiting for synchronization).

  The image processing of the general-purpose processor core (1) in the processing state is executed sequentially from the left pixel to the right pixel in the region. During this image processing, image data necessary for the processing is taken out from the shared memory 3 and transferred to the local memory 8. The transfer size of the image data is a size determined in advance between the shared memory 3 and the local memory 8 or a multiple of the size.

Further, when image processing is executed for one row in the area, the processing result of the previous row is required. Therefore, the processing result of the previous line is continuously held.
The image processing of the general-purpose processor core (1) is continuously performed up to the pixels for which the synchronization processing with the general-purpose processor core (2) that processes the right region is performed.

The cancellation of the synchronization processing with the adjacent area will be described with reference to FIG. 6 in which the first two lines in FIG. 5 are enlarged. In FIG. 6, a portion denoted by “A” represents a memory block that is a DMA transfer unit between the shared memory 3 and the local memory 8. When expressed as A _{i, j} , the subscript i represents the number of rows, and the subscript j represents the block number. Up (x), Upleft (x), and Left (x) represent the values of the upper, upper left, and left pixels on which the pixel x depends, respectively. A dotted line in FIG. 6 represents a boundary line of a region processed by the general-purpose processor core.

  The pixel from which the general-purpose processor core (2) starts processing is a pixel corresponding to the left end of the first row (= 1 row) in the region to be processed, and is indicated by alm in FIG. Consider the calculation dependency with other pixels to execute image processing of this pixel. The pixels Up (alm) and Upleft (alm) depending on the calculation belong to the upper row and are outside the image area. Accordingly, the only other pixel that affects the image processing of the pixel alm is the adjacent left pixel Left (alm). That is, the general-purpose processor core (2) can start processing if the processing result of the pixel Left (alm) is obtained.

In image processing, an original image and a processed image are transferred to and from a shared memory at a size optimized for the transfer size. In order for the general-purpose processor core (2) to acquire data necessary for processing, the data of the memory block A _{l, n} including the adjacent left side pixel necessary for starting the processing of the general-purpose processor core (2) is stored in the general-purpose processor core. Must be after being written by (1).

In the present embodiment, the synchronization cancellation is not the timing at which the processing of the adjacent left pixel is completed, but is a later timing, that is, the memory in which the general-purpose processor core (1) is the terminal pixel of the second row that is the next row After completing the processing of the pixels belonging to the right end of the blocks A _{2 and n−1} , the synchronization is released. This is to ensure a stable operation by providing a time margin between the operation of the general-purpose processor core (1) and the start operation of the general-purpose processor core (2).

After this synchronization release, the general-purpose processor core (1) subsequently performs image processing up to the last pixel in the area without performing synchronization processing. Similar to the general-purpose processor core (1), the general-purpose processor core (2) performs the synchronization release processing of the adjacent right side area with respect to the general-purpose processor core (3), and then continuously performs image processing up to the end of the area.
Thereafter, the processing of the general-purpose processor core 6 is similarly performed up to the last image area.

  FIG. 7 is a time chart showing the operation of the general-purpose processor core 6. In FIG. 7, the timings of the internal processing of each general-purpose processor core, the data transfer between the shared memories, and the synchronization release processing with the adjacent general-purpose processor core are developed on the time axis. Hereinafter, a description will be given with reference to FIG. It is assumed that the processing program is loaded in advance on the control processor core 5 and a plurality of general-purpose processor cores 6 that execute processing before the processing shown in FIG.

  First, the control processor core 5 notifies each general-purpose processor core 6 of the start of processing via the bus 7. After the notification, the general-purpose processor core (1) assigned with the processing in the left end area starts image processing, and the other general-purpose processor cores 6 wait for a synchronization signal from the left-side area.

The general-purpose processor core (1) first reads the data of the block A ₁ , ₁ from the shared memory 3 by the DMA transfer size. When the reading is completed, the general-purpose processor core (1) executes the data reading operation of the next blocks A1 and ₂ while sequentially executing the image processing of the read area from the left pixel.

Next, the processing of the blocks A ₁ and ₂ is performed in parallel with the writing of the data of the first block A _1,1 to the shared memory 3 and the reading of the data of the next processing block A _1,3. Execute.
As described above, the general-purpose processor core (1) continues the processing up to the pixel for which the synchronization is canceled while performing the image processing in parallel with the reading / writing to the shared memory 3. The general-purpose processor core (1) executes a synchronization process with the general-purpose processor core (2). After executing the synchronization processing, the general-purpose processor core (1) performs image processing to the end of the region without synchronizing with the general-purpose processor core (2).

  After executing synchronization processing with the general-purpose processor core (1), the general-purpose processor core (2) performs processing up to the block including the last pixel in the second row, and executes synchronization processing with the general-purpose processor core (3). . Thereafter, the same processing is performed for the general-purpose processor core (3) and the general-purpose processor core (4).

  As described above, the processing assigned to the general-purpose processor core 6 includes synchronization with the general-purpose processor core 6 that processes the left adjacent area, image processing of the area, synchronization with the general-purpose processor core 6 that processes the right adjacent area, and the area. This is image processing up to the end, and these processes are executed sequentially.

  If the image area is large, as shown in FIG. 8, the image area is divided into a larger number of areas than the number of general-purpose processor cores, and after sequentially assigning general-purpose processor cores, Image areas are allocated in order from the processor core, and one image is processed repeatedly.

Next, control of each unit and data handling in the image processing apparatus 1 will be described.
FIG. 9 is a flowchart showing a schematic process of the image processing apparatus 1. The control processor core 5 executes the process shown in FIG.

  In Act A01, the control processor core 5 receives an instruction regarding processing from the outside. That is, the type of image data to be processed is received. The data type here indicates whether the entire image is received and processed at once or whether band data is received and processed sequentially. At this time, a processing type indicating what kind of processing is performed on the image is also received. In Act A02, a program for executing the designated image processing is sent to the general-purpose processor core 6 that executes processing in the image processing apparatus.

  In Act A03, the image data to be processed is received and its image type is determined. As a result of the determination, if the received image data is the first / last band of the band data or a single image, in Act A04, the data storage part of the area outside the image area is initialized with the initial value corresponding to the previously received process. . If not, that is, if it is band data other than the head, in Act A05, a part of the image stored in the memory is copied to the outside area.

  After completion of this process, in Act A06, image data is received from the outside. Subsequently, in Act A07, a part of the received image data is copied to the shared memory 3 directly connected to the image processing apparatus. After the completion of copying, in Act A08, the general-purpose processor core 6 is instructed to start processing. Thereafter, each general-purpose processor core 6 performs image processing in parallel, and in Act A09, the control processor core 5 waits for reception of processing end transmitted from these general-purpose processor cores 6.

  After receiving the processing end, in Act A10, the image data is copied from the shared memory 3 storing the processing result to the transfer memory, and the result of the band or one image is notified to the processing request source. To do. In Act A12, it is checked whether or not all the bands have been processed. If the image type is a band and the processing has not been completed (No in Act A12), the above processing is repeated again, Alternatively, if the processing of one image has been completed (Yes in Act A12), the image processing is completed here.

FIG. 10 is a flowchart showing a schematic process of the general-purpose processor core 6.
The processing executed by the general-purpose processor core 6 is instructed from the processing request side. That is, a program for executing a process is transmitted from the process requesting side, and the process on the general-purpose processor core 6 is started by executing the program.

  In Act P01, the general-purpose processor core 6 receives an area on the image to be processed. Subsequently, in Act P02, image data of an independent image area is received. Here, an independent image area is an image area that is a processing target and that does not use the processing result of the adjacent area.

  Next, in Act P03, a synchronization signal from the adjacent general-purpose processor core 6 is waited. That is, in the adjacent area, the process waits for the image processing of the synchronized pixel (synchronized pixel) to be completed. In Act P04, when receiving the synchronization signal, that is, that the processing of the synchronized pixel is completed, in Act P05, the general-purpose processor core 6 starts processing. In Act P06, the processing result is copied to the shared memory 3.

  When the process of the synchronized pixel is completed in the own area, in Act P07, the general processor core 6 that processes the adjacent area is notified of the completion of the process up to the synchronized signal, that is, the synchronized pixel. After this process, in Acts P08 to P09, the general-purpose processor core 6 repeatedly executes the process and the copy of the process result up to the last pixel in the allocated area. After the processing is completed, the control processor core 5 is notified of the processing completion in Act P10.

  In this way, even for image processing that is difficult to process simply by parallelization due to dependency relationships, by adopting processing control that considers the dependency relationship between pixels by dividing the image in an oblique direction, data transfer, Parallelization suitable for general-purpose processor processing can be performed. Therefore, it is possible to achieve high speed without performing acceleration by adding additional hardware.

[Second Embodiment]
The image processing apparatus according to the second embodiment of the present invention is different from the first embodiment in the image processing method. Accordingly, the same parts as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

  A parallel image processing method implemented by the image processing apparatus 1 according to the second embodiment will be described. FIG. 11 is a diagram showing an area allocated to the general-purpose processor core 6. In this parallel image processing method, the area processed by each general-purpose processor core 6 is an area obtained by dividing an image in the vertical and horizontal directions. In the horizontal direction, a pixel area divided in line (row) units is assigned to the general-purpose processor core 6, and in the vertical direction, a pixel area related to the calculation of the vertical area + the vertical area is assigned to the general-purpose processor core 6. Each general-purpose processor core 6 performs calculations in parallel.

FIG. 12 is a diagram illustrating an area processed by two processor cores that first execute image processing. In this method, two processor cores always start processing simultaneously.
The processing areas of the general-purpose processor core (1) are the shaded area (1) + (2) and the right hatched area (1) in FIG. Similarly, the processing area of the general-purpose processor core (2) is a shaded area (1) + (2) and a left hatched area (2). The size of the area allocated to the general-purpose processor core (1) that processes the area divided in the horizontal direction is one line of the image. The size of the area allocated to the general-purpose processor core (2) that processes the area divided in the vertical direction is a trapezoidal area in which the upper side is the DMA transfer block size × 2 and the lower side is the DMA transfer block size.

Following the processing in this area, the general-purpose processor cores (3) and (4) start processing at the same time for the inner area.
FIG. 13 is a diagram illustrating an area to be processed by two processor cores that next execute image processing. Compared to the area shown in FIG. 12, the area processed by the general-purpose processor core (3) is smaller than the area processed by the general-purpose processor core (1) by the block size column of DMA transfer, and the general-purpose processor core (4) The area to be processed is an area having a size smaller by one row and one column than the area processed by the general-purpose processor core (2).

FIG. 14 is a diagram illustrating the timing of synchronization processing for the general-purpose processor cores (3) and (4) to start processing after the processing of the general-purpose processor cores (1) and (2) is started.
In order for the general-purpose processor cores (3) and (4) to process the pixel at the start point, the processing results of the upper pixel, the upper left pixel, and the left pixel are required with reference to the start point pixel. is there.

  As shown in FIG. 12, the area of the general-purpose processor core (2) for processing the vertically divided area includes a part of the general-purpose processor core (1), and the general-purpose processor cores (3), (4 ) Includes all the pixels on which the starting point depends. Therefore, the processing of the general-purpose processor core (2) is processed up to the pixel located on the left side of the start point of the general-purpose processor cores (3) and (4) and is written to the shared memory 3, and then the general-purpose processor core (3) , (4) can start processing.

FIG. 15 is a diagram showing in detail the synchronization timing between general-purpose processor cores.
Block A _1,1 and block A _1,2 are areas processed by the general-purpose processor cores (1), (2), and correspond to the shaded areas (1) + (2) in FIG. Block A _2,1 and block A _3,1 are areas processed by the general-purpose processor core (2).
Blocks A _{2 and 2} are areas processed by the general-purpose processor cores (3) and (4), and a part of the area is processed by the general-purpose processor core (2). Blocks A _{2 and 3} are areas processed by the general-purpose processor core (4), and a part of the area is processed by the general-purpose processor core (2).

Here, the pixel a2m at the start point where the general-purpose processor cores (3) and (4) start processing is the pixel at the left end of the blocks A2 and ₂ . In order for the pixel a2m at the start point to execute image processing, it is necessary that the processing of the pixel Up (a2m), Upleft (a2m), and Left (a2m) is completed. Accordingly, when the processing of Left (a2m), which is the left adjacent pixel, is completed, the general-purpose processor cores (3) and (4) can start the processing. However, as described above, a stable operation is ensured by providing a time margin between the operation of the general-purpose processor core (2) and the start operations of the general-purpose processor cores (3) and (4). Therefore, the timing when the general-purpose processor core (2) processes the rightmost pixel of the block A _3,1 is set as the synchronization timing.

Therefore, when the processing of the rightmost pixel in the block A ₃ , ₁ is completed, the synchronization wait state is released and the general-purpose processor cores (3) and (4) start processing. Since the pixel at the processing start point requires the processing results of the upper, upper left, and left pixels, the stored data is read from the shared memory 3 and processed before the synchronization processing. The general-purpose processor core (2) does not perform the synchronization process after this synchronization process, and proceeds to the end of the area. Thereafter, similarly, processing is performed in parallel toward the lower right direction of the image.

  As described in the first embodiment, when the synchronization processing of the general-purpose processor cores 6 is performed via the bus 7, the time required for the synchronization processing increases as the number of general-purpose processor cores 6 to be synchronized increases. When the area is simply divided vertically and horizontally and processed, the general-purpose processor cores (3) and (4) must perform synchronization processing with the general-purpose processor cores (1) and (2). That is, it is necessary to perform synchronous processing with four general-purpose processor cores, and processing time increases.

  FIG. 16 is a diagram showing image data processed by the general-purpose processor core 6 and processed data stored in the shared memory 3.

  In the second embodiment, all the pixels necessary for synchronization are included in the processing area of one general-purpose processor core using the dependency of image processing. In addition, by configuring so that the synchronization timing is determined based on the operation of the general-purpose processor core having a large amount of processing, at the timing when the general-purpose processor core that has started processing accesses the shared memory 3, a large amount of processed data is stored. It exists on the shared memory 3.

Next, control of each unit and data handling in the image processing apparatus 1 will be described.
FIG. 17 is a flowchart showing a schematic process of the image processing apparatus 1. The processing shown in FIG. 17 is executed by the control processor core 5.
The process shown in the flowchart of FIG. 17 is the same as the process shown in the flowchart of FIG.

FIG. 18 is a flowchart showing a schematic process of the general-purpose processor core 6.
The processing executed by the general-purpose processor core 6 is instructed from the processing request side. That is, a program for executing a process is transmitted from the process requesting side, and the process on the general-purpose processor core 6 is started by executing the program.

  In Act P101, the general-purpose processor core 6 receives an area on the image to be processed. Subsequently, in Act P102, image data of an independent image area is received. Here, an independent image area is an image area that is a processing target and that does not use the processing result of the adjacent area.

  Next, in Act P103, a synchronization signal from the adjacent general-purpose processor core 6 is waited. That is, in the adjacent area, the process waits for the image processing of the synchronized pixel (synchronized pixel) to be completed. In Act P104, when receiving the synchronization signal, that is, that the processing of the synchronized pixel is completed, in Act P105, the general-purpose processor core 6 starts processing. In Act P106, the processing result is copied to the shared memory 3.

In Act P107, the general-purpose processor core 6 determines whether it is necessary to communicate the synchronization release to other general-purpose processor cores 6 that process adjacent areas.
When it is necessary to communicate the release of synchronization (Yes in Act P107), when processing of the synchronized pixel is completed in its own region, the adjacent region is processed in Act P108 to indicate that the processing up to the synchronized signal, that is, the synchronized pixel is completed. Notify the general-purpose processor core 6. When it is not necessary to communicate the synchronization release (No in Act P107), the next Act P109 is executed.

  In Acts P109 to P110, the general-purpose processor core 6 continues processing and copying of processing results up to the last pixel in the allocated area. After the process is completed, the process completion is notified to the control processor core 5 in Act P111.

  Here, whether or not the general-purpose processor core 6 needs to perform synchronization release communication with other general-purpose processor cores 6 that process adjacent areas is determined according to the area to be processed by the general-purpose processor core 6. . As shown in FIG. 11, when the target processing area is divided into vertical and horizontal directions when the image band is divided into vertical and horizontal areas, synchronization processing is performed, and when the target processing area corresponds to horizontal division areas, it is determined that synchronization processing is not performed. To do.

  FIG. 19 is a time chart showing the operation of the general-purpose processor core 6. FIG. 19 shows the timings of the internal processing of each general-purpose processor core, the data transfer between the shared memories, and the synchronization release processing with the adjacent general-purpose processor core on the time axis. Before the process illustrated in FIG. 19, it is assumed that a program is loaded in advance into the plurality of general-purpose processor cores 6 for processing by the control processor core 5.

  First, the control processor core 5 notifies each general-purpose processor core 6 of the start of processing via the bus 7. After the notification, the general-purpose processor cores (1) and (2) start image processing, and the other general-purpose processor cores 6 wait for a synchronization signal from the adjacent area.

  The general-purpose processor core (1) processes the upper side of the image area stored in the image processing apparatus 1, that is, the first line of the image data. The general-purpose processor core (2) processes the left end of the image and several sequences including it. Here, the number of columns is a size optimized for data transfer between the shared memory 3 connected to the image processing apparatus 1 and the local memory 8.

  The general-purpose processor cores (1) and (2) acquire image data from the storage area 11 connected to the image processing apparatus 1 via the shared memory 3 as an initial data load. The general-purpose processor cores (1) and (2) acquire and process the pre-processed image data stored in the shared memory 3 in a certain amount sequentially from the left end, and process the processed results by a certain amount. Write to 3.

  Image transfer and image processing between the shared memory 3 connected to the image processing apparatus 1 and the local memory 8 are processed in parallel by the calculation processing unit in the general-purpose processor core 6 and the DMA transfer unit. . That is, the general-purpose processor core 6 acquires a certain amount of data and performs image processing on this data. At the timing of this processing, the processing data group is written and the next processing data group is read in parallel. To do.

  The time chart shown in FIG. 19 describes the details of the synchronization processing timing and the synchronization processing method between the general-purpose processor cores 6. The time chart is already described with reference to FIG. Therefore, detailed description thereof is omitted.

  The general-purpose processor core (2) performs the synchronization process with the general-purpose processor cores (3) and (4), and then proceeds to the end of the allocated area. The general-purpose processor core (4) continues the image processing and writes the processed data to the shared memory 3 and performs the synchronization processing when processing is performed on the synchronized pixels for synchronizing the adjacent regions. Execute. After that, the processing is performed up to the end of the allocated area in the same manner as the general-purpose processor core (2).

  If the image area is large, as shown in FIG. 20, the image area is divided into a number larger than the number of general-purpose processor cores, and after sequentially assigning general-purpose processor cores, the general-purpose processor cores assigned first are again assigned. An internal image area is allocated in order, and one image is processed repeatedly.

  In the parallel image processing method according to the second embodiment described above, subsequent to the image processing of the allocated area, the processing is executed using the inner area as the allocated area. As described above, when the image area is divided in the vertical and horizontal directions and processed in parallel, the number of synchronization processes can be reduced by including pixels (synchronous pixels) necessary for synchronization with the next parallel processing in one divided area. This reduces parallel processing with less overhead.

[Third embodiment]
An image processing apparatus according to the third embodiment of the present invention has a configuration in which the first embodiment and the second embodiment are combined. Accordingly, the same parts as those in the first and second embodiments are denoted by the same reference numerals, and detailed description thereof is omitted.

  In the third embodiment, a mechanism for connecting a plurality of image processing apparatuses 1 to an external bus is provided. The number of image processing apparatuses 1 connected to the external bus is detected manually or automatically by hardware having an automatic recognition function.

When only one image processing apparatus 1 is connected to the external bus, as shown in FIG. 21, image processing is performed according to the image processing method described in the second embodiment of the present invention.
When there are a plurality of image processing apparatuses 1 connected to the external bus, as shown in FIG. 22, the image data to be processed is an image according to the image processing method according to the first embodiment of the present invention. It is processed.

  That is, in the image processing method shown in FIG. 22, the image area is divided in accordance with the first embodiment of the present invention in accordance with the number of image processing apparatuses 1. In each image processing apparatus 1, the divided area is further divided in the form according to the first embodiment. Further, pixels that have a dependency relationship in each image processing apparatus 1 and need to exchange information with each other are processed while being shared using the storage area 11 connected to the external bus.

Next, control of each unit and data handling in the image processing apparatus 1 will be described.
FIG. 23 is a flowchart showing a schematic process of the image processing apparatus 1. The process shown in FIG. 23 is executed by the control processor core 5.

  In Act A201, the control processor core 5 receives a processing instruction from the outside. That is, the type of image data to be processed is received. The data type here indicates whether the entire image is received and processed at once or whether band data is received and processed sequentially. At this time, a processing type indicating what kind of processing is performed on the image is also received.

  In Act A202, the number of image processing apparatuses 1 connected to the external bus is checked. When the number of the image processing apparatuses 1 is one, in Act A203, a program for executing the image processing of the second embodiment is loaded into each general-purpose processor core 6.

  When the number of the image processing apparatuses 1 is two or more, in Act A204, the image processing of the first embodiment is executed, and the pixel synchronization process having the dependency relationship between the image processing apparatuses 1 and the processing are performed. A program for executing the communication of the result is loaded into each general-purpose processor core 6.

  Then, the processing from Act A205 to Act A214 is executed using this loaded program. Note that the processing from Act A205 to Act A214 is the same as the processing from Act A03 to Act A12 shown in FIG.

24 and 25 are flowcharts showing a schematic process of the general-purpose processor core 6. 24 and 25 are processing flows when the number of image processing apparatuses 1 is two as described above.
The processing executed by the general-purpose processor core 6 is instructed from the processing request side. That is, a program for executing a process is transmitted from the process requesting side, and the process on the general-purpose processor core 6 is started by executing the program.

  In Act P01, the general-purpose processor core 6 receives an area on the image to be processed. Subsequently, in Act P02, image data of an independent image area is received. Here, an independent image area is an image area that is a processing target and that does not use the processing result of the adjacent area.

  In Act P203, the general-purpose processor core 6 determines whether or not to wait for a synchronization signal from another image processing apparatus 1. When the processing result of the general-purpose processor core 6 of another image processing apparatus 1 is used (Yes in Act P203), the processing result of the external general-purpose processor core 6 is waited in Act P204. When the processing result of the general-purpose processor core 6 of another image processing apparatus 1 is not used (No in Act P203), the process waits for a synchronization signal from the general-purpose processor core 6 of the same image processing apparatus 1 in Act P205.

In Act P206, when the synchronization signal, that is, the completion of the processing of the synchronized pixel is received, in Act P207, the general-purpose processor core 6 starts the process, and performs the synchronization processing based on the dependency on the adjacent region (synchronization pixel). Image processing is performed up to (pixel).
In Act P208, it is determined whether or not to send a synchronization signal to another image processing apparatus 1. When the synchronization target is another image processing apparatus 1 (Yes in Act P208), in Act P209, the processing result is written on the external memory connected to the external bus. When the synchronization target is the general-purpose processor core 6 of the same image processing apparatus 1 (No in Act P208), the processing result is written in the shared memory in Act P210.

  Then, in Act P211, the corresponding general-purpose processor core 6 is notified of the completion of the processing up to the synchronization signal, that is, the synchronization pixel. After this processing, the image processing is continued in Act P212.

In Act P213, it is determined whether or not to send the processing result to another image processing apparatus 1. When the transmission destination is another image processing apparatus 1 (Yes in Act P213), in Act P214, the processing result is written on the external memory connected to the external bus. When the transmission destination is the general-purpose processor core 6 of the same image processing apparatus 1 (No in Act P213), the processing result is written in the shared memory in Act P215.
Then, after the process is completed, the control processor core 5 is notified of the process completion in Act P216.

  Here, the case where the number of image processing apparatuses is 2 has been described in the above flow. However, even in the case where the number of image processing apparatuses is 3 or more, only the number of divisions and the area are different, and the processing is performed in the same manner. Is called.

  As described above, the image processing apparatus according to the third embodiment improves performance by using a plurality of image processing apparatuses and applying a processing method that considers data transfer between general-purpose processor cores and synchronization processing overhead. Realize.

  According to the image processing apparatus of each embodiment described above, in an image processing system having a plurality of processor cores, image processing having a dependency relationship between pixels can be processed at high speed by parallelization.

  According to the image processing method of the first embodiment of the present invention, in an apparatus composed of a plurality of calculation cores, image processing having a dependency such that processing of a certain pixel uses the result of another pixel is performed by a plurality of processors. Parallel processing can be performed by the core, and high-speed processing can be realized.

  With the image processing method according to the second embodiment of the present invention, even if the overhead of synchronization processing is large to some extent in a device composed of a plurality of computing cores, parallel processing taking this into consideration is possible, so there are hardware limitations Even in an image processing environment, high-speed processing can be realized.

  According to the image processing method of the third embodiment of the present invention, it is possible to further add a plurality of parallel processing processors composed of a plurality of cores to realize parallel processing considering synchronization and communication overhead suitable for a device connection form. it can. Therefore, even when the performance of a processor with a plurality of cores is insufficient, it is possible to realize image processing with higher performance by using the plurality of processors and configuring as in the present embodiment.

Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage.
Further, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, you may combine suitably the component covering different embodiment.

  INDUSTRIAL APPLICABILITY The present invention can be used in an industry for manufacturing an image processing apparatus that can easily realize improvement in processing efficiency of image processing having a dependency relationship between pixels.

      DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus, 2 ... External interface, 3 ... Shared memory, 4 ... Memory interface, 5 ... Control processor core, 6 ... General-purpose processor core, 7 ... Bus, 8 ... Local memory, 10 ... Control means, 11 ... Storage area.

JP 2007-87417 A

Claims (10)

Shared memory to store images,
A plurality of processors connected to the shared memory and executing image processing;
A controller that divides the image stored in the shared memory into a plurality of areas, sets the divided areas in arbitrary processors, and shifts the processing start timing of the set processors sequentially to perform parallel processing. ,
The processing area where the preceding processor that starts the operation performs image processing includes a pixel that satisfies the inter-pixel dependency of the pixel that the subsequent processor that starts the operation first processes. A featured image processing apparatus.   The controller starts the operation of the succeeding processor after the image processing is completed for all the pixels that satisfy the inter-pixel dependency with the succeeding processor included in the processing region by the controller. The image processing apparatus according to claim 1, wherein: Shared memory to store images,
A plurality of processors connected to the shared memory and executing image processing;
The image stored in the shared memory is divided into a plurality of areas, each divided area is set in an arbitrary processor, a plurality of groups in which at least a plurality of processors are grouped into one group unit is set, A plurality of processors belonging to one group, having a controller that simultaneously starts an operation and sequentially shifts the processing start timing for each group to perform parallel processing;
In the processing area where the processor belonging to the preceding group that starts the operation first performs image processing, the inter-pixel dependency of the pixel that is first processed by all the processors belonging to the succeeding group that starts the operation is satisfied. An image processing apparatus comprising a pixel.   The controller performs the post-processing after the image processing is completed for all the pixels satisfying the inter-pixel dependency with the processors belonging to the succeeding group included in the processing area. The image processing apparatus according to claim 3, wherein an operation of a processor belonging to a row group is started. In an image processing system in which a plurality of image processing apparatuses according to claim 1 are connected to each other via a communication path,
The controller of one of the image processing devices transmits a part of the divided processing region to another image processing device for processing, and the pixel dependency with the other image processing device An image processing system for controlling a timing signal transmission / reception operation for starting image processing. In an image processing method of an image processing apparatus comprising a shared memory for storing an image and a plurality of processors connected to the shared memory and executing image processing,
Dividing the image stored in the shared memory into a plurality of areas;
Set the divided areas for each processor,
Parallel processing by sequentially shifting the processing start timing of the set processor,
The processing area where the preceding processor that starts the operation performs image processing includes a pixel that satisfies the inter-pixel dependency of the pixel that the subsequent processor that starts the operation first processes. A featured image processing method.   The operation of the succeeding processor starts after the image processing is completed for all the pixels satisfying the inter-pixel dependency with the succeeding processor included in the processing area. The image processing method according to claim 6. In an image processing method of an image processing apparatus comprising a shared memory for storing an image and a plurality of processors connected to the shared memory and executing image processing,
Dividing the image stored in the shared memory into a plurality of areas;
Set the divided areas for each processor,
A plurality of groups in which at least a plurality of processors are grouped into one group unit are set,
The plurality of processors belonging to the one group are started to operate simultaneously,
The processing start timing for each group is sequentially shifted in parallel,
In the processing area where the processor belonging to the preceding group that starts the operation first performs image processing, the inter-pixel dependency of the pixel that is first processed by all the processors belonging to the succeeding group that starts the operation is satisfied. An image processing method comprising a pixel.   The processor belonging to the preceding group belongs to the succeeding group after the image processing is completed for all the pixels satisfying the inter-pixel dependency with the processors belonging to the succeeding group included in the processing region. The image processing method according to claim 8, wherein the operation of the processor is started. In an image processing method of an image processing system in which a plurality of image processing apparatuses according to claim 6 are connected to each other via a communication path,
One of the image processing devices transmits a part of the divided processing region to another image processing device for processing, and the image processing accompanying pixel dependency with the other image processing device An image processing method for controlling an operation of transmitting and receiving a timing signal for starting the operation.

Images