JP2006293632A - Semiconductor integrated circuit device, image processing system, and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a semiconductor integrated circuit device, an image processing system and an image processing method, capable of easily packaging an effective algorithm for image processing. <P>SOLUTION: A semiconductor integrated circuit 20 is provided with feature quantity extracting device 32 which performs statistical processing for digital image data and extracts the feature quantity, and a plurality of processing units 41. Each processing unit 41 comprises a high parallel processor 40 which performs processing regarding the image data using the feature quantity. Each processing unit 41 is equipped with a holding circuit 42 which holds the feature quantity extracted by the feature quantity extracting device 32 and performs processing using the feature quantity held in the holding circuit 42. The feature quantity extracting device 32 performs operations independently of the parallel processor 40. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a semiconductor integrated circuit device, an image processing system, and an image processing method. For example, the present invention relates to an algorithm for processing image data captured by a camera using a highly parallel processing unit.

  In recent years, as the operation of a CPU (Central Processing Unit) increases, there is a remarkable improvement in the performance of an image processing apparatus.

  In conventional image processing, a method has been proposed in which feature amounts are extracted from image data, and which image processing is to be performed based on the extracted feature amounts (see, for example, Patent Document 1). According to this method, the input image is divided into a plurality of regions, and texture feature amounts are extracted in the respective regions. Then, image processing means for processing the area is selected according to the extracted feature amount.

However, the above-described method has a problem that it is difficult for the image processing means to implement an image processing algorithm using the feature value itself.
Japanese Patent No. 3047952

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a semiconductor integrated circuit device, an image processing system, and an image processing method capable of easily implementing an efficient algorithm for image processing. is there.

  In order to achieve the above object, a semiconductor integrated circuit device according to an aspect of the present invention includes a feature quantity extraction device that performs statistical processing on digital image data and extracts a feature quantity, and a plurality of processing units. Each of the processing units includes a parallel processing device that performs processing related to the image data using the feature amount, and each of the processing units holds the feature amount extracted by the feature amount extraction device. And the processing is performed using the feature amount held in the holding circuit, and the feature amount extraction device performs an operation independently of the parallel processing device.

  An image processing system according to an aspect of the present invention includes the semiconductor integrated circuit device and an imaging device that captures an image to acquire the digital image data. The semiconductor integrated circuit is connected to the imaging device from the imaging device. The image processing apparatus further includes a buffer circuit that transfers image data and temporarily stores the image data, and a storage device that stores the image data processed by the parallel processing device. The feature amount is extracted for the transferred image data.

  Furthermore, an image processing method according to an aspect of the present invention is an image processing method for processing image data using a parallel processing device including a plurality of processing units, and the step of receiving the image data; A step of calculating a feature amount for image data by a feature amount extraction device and a step of transferring the feature amount to each of the processing units, using each of the transferred feature amounts, And a process for processing data.

  According to the present invention, it is possible to provide a semiconductor integrated circuit device, an image processing system, and an image processing method capable of easily mounting an efficient algorithm for image processing.

  Embodiments of the present invention will be described below with reference to the drawings. In the description, common parts are denoted by common reference symbols throughout the drawings.

  A semiconductor integrated circuit device, an image processing system, and an image processing method according to a first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram of an image processing system according to the present embodiment.

  As shown in the figure, the image processing system 1 according to the present embodiment includes a digital video camera 10 and an image processing LSI 20. An image photographed by the digital video camera 10 is processed by the image processing LSI 20.

  The image processing LSI 20 includes a data input device 30, a highly parallel processing device 40, a storage device 50, a DMA (Direct Memory Access) controller 60, and a data bus 70. The data input device 30 includes an input buffer 31 and a feature amount extraction device 32. Image data is input from the digital video camera 10 to the input buffer 31. The feature amount extraction device 32 is a block dedicated to statistical processing that performs statistical processing on the input buffer 31, and extracts a feature amount of the image data. The feature quantity extraction device 32 is separated from the highly parallel processing device 40 and can execute processing independently of the highly parallel processing device 40.

  The highly parallel processing device 40 includes a plurality of (for example, 128) processing units (CPUs) 41. Each of the processing units 41 reads out the image data held in the input buffer 31 and executes processing on the image data in parallel with each other. All the processing units 41 may perform the same process, or may perform different processes. Each processing unit 41 includes a memory (register) 42. The processing unit 41 reads the feature amount from the feature amount extraction device 32 to the memory 42. Then, using the feature amount read out to the memory 42, processing relating to the image data for which it is responsible is performed.

  The storage device 50 is, for example, an eDRAM (embedded DRAM) that holds image data input to the input buffer 31 and image data processed by the highly parallel processing device 40.

  The DMA controller 60 controls DMA transfer of data between the input buffer 31 and the processing unit 41, between the storage device 50 and the processing unit 41, and between the input buffer 41 and the storage device 50. These data are transferred via the data bus 70.

  Next, an image processing method by the image processing system 1 will be described with reference to the flowchart of FIG. First, the digital video camera 10 captures an imaging target and acquires image data (step S10). The camera 10 includes an A / D converter, and image data digitized by the camera 10 is input to the LSI 20. Of course, the LSI 20 may perform A / D conversion.

  The image data acquired by the camera 10 is input to the input buffer 31 (step S11). As the input buffer 31, for example, a double buffer can be used. When the front buffer becomes full, switch to the back buffer. Of course, the form of the buffer is various, and is not limited to this method.

  Next, the feature quantity extraction device 32 performs statistical processing on the image data on the input buffer 31 to extract the feature quantity. The image data is transferred to the storage device 50 (step S12). The feature amount is extracted before the processing of the highly parallel processing device 40 regarding the image data is started.

  When the feature amount extraction is completed, the feature amount extraction device 32 transfers the extracted feature amount to the processing unit 41 (step S13). At this time, since the feature amount is used in each of the processing units 41, access is likely to be concentrated. Therefore, it is desirable that a copy of the feature amount is created and each processing unit 41 has its own feature amount.

  Next, each of the processing units 41 in the highly parallel processing device 40 reads the image data in the storage device 50. And the process regarding this image data is performed using the feature-value in the memory 42 (step S14).

Thereafter, the processed image data is stored in the storage device 50.
As described above, the following effects (1) and (2) can be obtained with the LSI, the image processing system, and the image processing method according to the first embodiment of the present invention.

(1) An efficient algorithm for image processing can be easily implemented in a highly parallel processing device.
With the configuration according to the present embodiment, the feature quantity extraction device 32 that performs statistical processing is separated and independent from the highly parallel processing device 40. Therefore, it is not necessary to consider statistical processing in the processing program in the highly parallel processing device. Each processing unit 41 holds a feature amount in the memory 42. Then, processing can be performed with reference to the feature quantity freely. As a result, the processing program in the highly parallel processing device can be simplified, that is, the efficient algorithm using the feature amount can be easily mounted on the highly parallel processing device. In particular, the hardware cost to be implemented is low in the statistic calculation exemplified as the feature quantity. For example, for the average value calculation, an address generation unit for data acquisition, a counter, an adder, and a divider may be mounted. For the divider, a shifter may be used if the divisor for the average value is a power of 2. Of course, a divider may be implemented. In addition to the above, the variance calculation needs only to add a multiplier and an inverse root calculator. In order to generate a histogram, a storage unit and a counter for holding a value as a histogram may be used. In creating the histogram, entries of the number of values that the pixel value P can take are secured in the storage unit. For example, when the pixel value P is 8 bits, the value can be 0 to 255, and 256 entries are secured. Then, a pixel value P is assigned to each entry, and the number of pixels taking each pixel value P is stored in the corresponding entry. The example of FIG. 3 shows that the number of pixels at P = 0 is 1, the number of pixels at P = 1 is 2, the number of pixels at P = 2 is 2, and the number of pixels at P = 255 is 0. Show. Such an entry may be held in the storage device 50, for example. Furthermore, median calculations and order statistics can also be implemented with histograms and adders. More details are omitted, but the cost of hardware required to calculate statistics is low.

  When this statistic calculation is performed by a highly parallel processing device, the cost for the calculation becomes high. The calculation of the average value is often used by the pyramid algorithm. The pyramid algorithm is performed as follows. For example, if there is luminance data having a size of (512 × 512), first, an average value of four adjacent pixels in the original image D0 is calculated, and the vertical and horizontal directions are each ½ (number of pixels is ¼). An image D1 is created. Next, an average value of four adjacent pixels in the image D1 is calculated, and an image D2 in which the vertical and horizontal directions are further halved is created. If this is repeated, the number of pixels finally becomes one, and the value of that pixel becomes an average value. Since the image gradually gets smaller, it is named the pyramid algorithm. Similarly, the mean square for variance calculation can be calculated by the pyramid algorithm. The pyramid algorithm is efficient because the degree of parallelism can be utilized when the image is large, but when the processing proceeds and the image becomes small, a processing unit that does not move is generated in the parallel processing device. In particular, when the degree of parallelism is high, there is a problem that the number of useless units (in short, there is no data and not operating) increases. As another method, there is a method of calculating by a normal method using only one unit of the highly parallel processing device, but other units may be idle during that time, and similarly wasteful.

  Similarly, histogram processing cannot be executed efficiently in a highly parallel processing device. As in the case of the average value and the variance, it is necessary to merge the processing results in each unit of the highly parallel processing device, and it is difficult in principle to process the merge processing without waste in the highly parallel processing device.

(2) Image processing efficiency can be improved.
With the configuration according to the present embodiment, feature amounts are extracted by the feature amount extraction device 32 separated from the highly parallel processing device 40. In this way, statistical processing that is not compatible with parallel processing is performed by a dedicated device, and is not performed at all by the highly parallel processing device 40, so that the extraction efficiency of feature quantities can be improved and the processing efficiency in the high parallel processing device 40 can be improved. Can improve.

  Further, by separating the feature amount extraction from the highly parallel processing device 40, the feature amount extraction processing and the processing in the highly parallel processing device 40 can be processed in a pipeline manner. This point will be described with reference to FIG. FIG. 4 is a timing chart of each process during image processing.

  As shown in the figure, it is assumed that data A to D are sequentially input from the camera 10 to the LSI 20 (step S11). Then, at times t1 to t4, the feature quantity extraction device 32 extracts the feature quantities of the data A to D, respectively (step S12). Focusing on data A, when the transfer of the feature value is completed at time t3 (step S13), the highly parallel processing device 40 can start the processing regardless of the situation of the feature value extraction device 32 (step S14). Since the feature amount extraction device 32 also operates independently of the highly parallel processing device 40, the feature amount extraction processing for the data C is performed during the period t3 to t4 when the highly parallel processing device 40 performs the processing for the data A. Can be executed. As described above, the processes in steps S12 to S14 can be overlapped with each other. Therefore, the processing efficiency of the LSI 20 can be improved. Note that FIG. 4 shows a case where the time required for the processing of steps S11 to S14 is the same for simplification. However, as described above, the time required for the processing is not particularly limited as long as the processing in the highly parallel processing device 40 is started after the feature amount extraction.

  Next, a semiconductor integrated circuit device, an image processing system, and an image processing method according to a second embodiment of the invention will be described. This embodiment corresponds to a specific example of the first embodiment described above, and is used when template matching is performed in the highly parallel processing device 40. First, template matching will be described with reference to FIG. FIG. 5 is a diagram showing the concept of template matching.

  Template matching refers to searching for the same image as a specific image T (template) from other images In + 1. For example, an image T including (64 × 64) pixels is scanned in an image In + 1 including (1024 × 1024) pixels, and a region matching the image T is included in the image In + 1. It is detected whether it exists in. This image T is usually a part of the image In having the same size as the image In + 1. In other words, this template is used when detecting a position where a certain part of the image In has moved in the image In + 1. Think about what matching uses.

  An image processing method in the case where the highly parallel processing device 40 performs such template matching will be described with particular attention to the processing in steps 12 and S13 in FIG. First, digital image data In and In + 1 are input from the camera 10 to the LSI 20. The digital image data is represented by a set of pixels, and each pixel has a gradation indicating a brightness level (shading). This gradation is hereinafter referred to as “pixel value P”. Digital images include a binary image having only two levels of pixel values and a multi-value image having pixel values divided into a number of levels. For example, when the pixel value is represented by 8 bits, the pixel value has 256 gradations.

First, in step S12, the feature quantity extraction device 32 calculates the average value P˜ of the pixel values P of the image data In and the average value P ′ of the pixel values P ′ of the image data In + 1 as feature quantities. 6 and 7 are conceptual diagrams of the image data In and In + 1, respectively. As shown in the figure, (1024 × 1024) pixels included in the image data In each have a pixel value Pij (i = 0 to 1023, j = 0 to 1023). The (1024 × 1024) pixels included in the image data In + 1 have pixel values P′kl (k = 0 to 1023, l = 0 to 1023), respectively. When the number of pixels is N, the average value P′˜ is obtained by the following equation.
P '~ = ΣP'kl / N
P ~ = ΣPij / N
That is, the average value P′˜ of the pixel values of the image In + 1 is obtained by adding the pixel values of all the pixels included in the image I and dividing the result by the total number N of pixels. P˜ is also obtained by the same calculation. The average values P˜ and P′˜ obtained as described above are transferred to the processing unit 41 (step S13).

  Next, the highly parallel processing device 40 performs template matching (step S14). That is, in order to detect whether the image T shown in FIG. 5 is included in the image I, the difference A between the pixel values Pij and P′kl of the pixels is calculated. If the difference A is smaller than a certain threshold Ath, it is determined that the image T is included in the area. For example, assume that the pixel values Pij and P′kl for the images T and I are as shown in FIGS. FIG. 7 is a graph in which the vertical axis represents the pixel value P and the horizontal axis represents (64 × 64) pixels. FIG. 8 is a graph in which the vertical axis represents the pixel value P ′ and the horizontal axis represents (1024 × 1024) pixels.

The highly parallel processing device 40 performs the following calculation for each region including (64 × 64) pixels in the image I, and obtains the difference A between the images T and I.
A = Σ | (Pij−P˜) − (P′kl−P′˜) |
That is, the average value P˜ is subtracted from the pixel values Pij of all the pixels included in the image T. Further, the average value P′˜ is subtracted from all the pixel values P′kl included in the region to be compared with the image T in the image In + 1. The difference between the two is defined as a difference value A. This process is template matching. This template matching is repeated for all regions of the image In + 1. Since the template matching process can be performed independently, the search for the image T in each region in the image I is performed in parallel in each of the processing units 41. Then, as shown in FIG. 8, since the difference value A is lower than the threshold value Ath in the area AREA1 in the image In + 1, it can be seen that the image T exists in the area AREA1. Template matching is completed as described above.

In the first and second embodiments, the case where the difference between the pixel values Pij and Pkl obtained by subtracting the average value is calculated has been described. However, image variances σ and σ ′ may be calculated, and the result of subtracting the average value from the pixel value may be divided by the variance. That is, there is also a method for calculating the difference A by the following equation (second embodiment).
A = Σ | (Pij−P˜) / σ− (P′kl−P′˜) / σ ′ |
By using dispersion in this way, the difference A becomes more robust against the influence of the surrounding environment. The calculation of the variances σ and σ ′ may be performed by the highly parallel processing device 40 in step S14, or may be performed by the feature amount extraction device 32 in step S12, treating the variance as a feature amount.

With the configuration and method according to the present embodiment, the following effect (3) can be obtained in addition to the effects (1) and (2) described in the first embodiment.
(3) The accuracy of template matching can be improved.
In the method according to the present embodiment, an average value of pixel values is calculated as a feature amount in each image data. The difference value A is calculated after subtracting the average value from the actual pixel value. Therefore, it is possible to remove the influence of extra background and improve the accuracy of template matching.

  Specifically, for example, the influence of ambient light can be reduced. For example, when the camera 10 captures images T and I, it is assumed that there is a difference in ambient brightness. In the case shown in FIGS. 8 and 9, when the image I is captured more brightly than when the image T is captured, and P′˜ is much larger than P˜, all the regions in the image I are captured. Thus, a phenomenon may occur in which the difference value A exceeds the threshold value Ath or the difference value A falls below the threshold value Ath in an area other than the area AREA1. In this case, the image T in the image I is not correctly recognized.

  However, in this embodiment, the average values P˜ and P′˜ are subtracted from the pixel values Pij and Pkl of the images T and I in advance. That is, the component in the background of each image is removed. Therefore, the net image data in the images T and I can be obtained by subtracting the average value. Therefore, even if there is a difference in ambient brightness at the time of shooting, the ambient brightness component can be removed from the images T and I, so that accurate template matching is possible.

  As for the effects (1) and (2), in the present embodiment, the highly parallel processing device 40 does not require an algorithm for calculating the average value of the pixel values, so that the program can be simplified. In general, the process of obtaining an average value is not suitable for parallelization. By causing the feature amount extraction apparatus 32 to perform this process, the highly parallel processing apparatus 40 only needs to perform the template matching process. Efficiency can be improved.

  Next explained is a semiconductor integrated circuit device, an image processing system and an image processing method according to the third embodiment of the invention. This embodiment corresponds to a specific example of the first embodiment described above, and is used when a part of the image encoding process is performed in the highly parallel processing device 40.

  FIG. 10 is a block diagram of the LSI 20 according to the present embodiment. As shown in the figure, the LSI 20 according to the present embodiment further includes an MPEG (Moving Picture coding Experts Group) decoder 80 in the configuration described in the first embodiment. The configuration of the MPEG decoder 80 will be described with reference to FIG. FIG. 11 is a block diagram of the MPEG decoder 80.

The MPEG decoder 80 encodes the moving image input to the input buffer 31 to the MPEG standard. As shown in the figure, an MPEG decoder 80 includes adders / subtracters 81 and 82, a DCT (Discrete Cosine Transform) unit 83, a quantization unit 84, a variable length coding unit 85, buffers 86 and 87, a rate control unit 88, and an inverse quantization unit. 89, an inverse DCT unit 90, and a motion compensation prediction unit 91.
The DCT unit 83 orthogonally transforms the input image data into the frequency domain. The quantization unit 84 divides the sample value obtained by the DCT unit 83 with a constant step width. In other words, the data rate is reduced by dividing the sample value by the step width and rounding the value. This step width is determined by the rate control unit 88. The variable length coding unit 85 performs, for example, Huffman coding on the image data whose data has been reduced by the quantization unit. Thereby, MPEG data is obtained and stored in the buffer 86. The inverse quantization unit 89 and the inverse DCT unit 90 decode the image data encoded by the DCT unit 83 and the quantization unit 84. The decoded image data is stored in the buffer 87 and further used by the motion compensation prediction unit 91 for motion estimation (ME). In the present embodiment, each processing unit 41 in the highly parallel processing device 40 functions as the motion compensation prediction unit 91.

  One image at an individual time constituting a moving image is called a frame. While DCT and quantization compress one frame as space, motion prediction performs data compression across two consecutive frames. In motion prediction, two consecutive frames are delta-analyzed to determine whether or not each area has changed between frames. If a certain image area is the same as the previous frame, it may be displayed in the same manner as that frame. When the image area moves in any direction, the image to be displayed is the same as the previous frame, and it is only necessary to move it by a certain amount in a specific direction. This is actually performed by generating a motion vector (MV) in the motion compensation prediction unit 91. In this manner, redundant data can be greatly reduced by obtaining a motion vector.

  In order to perform motion prediction, the motion compensation prediction unit 91 performs the template matching described in the second embodiment between the entire region of the previous frame and the entire region of the current frame. As a result, it can be determined how much and in which direction the predetermined area in the previous frame has moved in the next frame. FIG. 12 shows how an object moves between two frames. For example, it is assumed that the object displayed at time t1 (frame 1) moves as indicated by a motion vector MV as illustrated at time t2 (frame 2). Then, the feature quantity extraction device 32 calculates the average values P˜ and P′˜ of the pixel values P and P ′ of the frames 1 and 2 (step S12), and uses the obtained average values P˜ and P′˜ as the feature quantities. The data is transferred to the storage device 42 (step S13). Each region of frame 2 is transferred from the input buffer 31 to each of the processing units 42, and each region of frame 1 reproduced by the inverse quantization unit 89 and the inverse DCT unit 90 is transferred from the buffer 87 to each of the processing units 42. (Step S12). Then, each of the processing units 42 performs template matching between the frame 1 and the frame 2 (step S14). A state of template matching is shown in FIG.

  In step S14, as in the second embodiment, the average value P˜ is subtracted from the pixel value P of frame 1, and the average value P′˜ is subtracted from the pixel value P ′ of frame 2. Then, a difference value A between the two pixel values is calculated. The object motion vector MV can be calculated by searching for a region where the difference value A is equal to or less than the threshold value Ath.

  With the configuration and method according to the present embodiment, the effects (1) to (3) described in the first and second embodiments can be obtained. Also, the effect of (3) can improve the MPEG encoding accuracy.

  Next explained is a semiconductor integrated circuit device, an image processing system and an image processing method according to the fourth embodiment of the invention. This embodiment corresponds to a specific example of the first embodiment, and is for performing binarization processing in the highly parallel processing device 40. Therefore, the configuration of the image processing system is the same as that in the first embodiment, and a description thereof will be omitted.

  The binarization process is a process for changing the pixel value of an image to “0” or “1”, and is set to “0” or “1” depending on whether the pixel value exceeds a certain threshold value. It is decided whether to do. In the normal embodiment, the highly parallel processing device 40 performs binarization processing, and statistical processing hardware creates a histogram (histgram) in order to determine a threshold value. As described in the first embodiment, the histogram is obtained by counting the appearance frequency (number of pixels) of each pixel value for the entire area (or a part) of one image. A method for determining the threshold will be described with reference to FIGS. 14 and 15 are graphs in which the number of pixels is plotted on the vertical axis and the pixel value P is plotted on the horizontal axis. For example, as shown in FIG. 14, a pixel value from the largest pixel value to x% of the total number of pixels can be set as the threshold value Pth. Further, as shown in FIG. 15, when two peaks and a valley between them are clear, the position of the valley can be set as the threshold value Pth. Binarization processing is usually used as preprocessing for image recognition.

  Next, the binarization processing method according to the present embodiment will be described with reference to the flowchart of FIG. In the following, description will be given with particular attention to the processing in step S14. After step S11, the feature quantity extraction device 32 creates a histogram of the pixel values Pij of the image data and transfers it to the feature quantity storage device 42 (steps S12 and S13). The method for creating the histogram is as described in the first embodiment. Further, the image data is transferred to each processing unit 41 of the highly parallel processing device 40 (step S12).

  Next, the highly parallel processing device 40 starts binarization processing (step S14-1). First, each processing unit 41 calculates a threshold value Pth from the histogram (step S14-2). For example, the threshold value Pth is determined using the method described with reference to FIG. Alternatively, a method of performing calculation up to the threshold value Pth by the collection amount extraction device 32 is also conceivable. This is desirable from the viewpoint of facilitating programming in a highly parallel processing device.

  Next, each processing unit 41 compares the pixel value (Pij−P˜) of each pixel with the threshold value Pth (step S14-3). The pixel value Pij used here is, of course, a value after subtracting the average value P˜. When Pij ≧ Pth, Pij = “1” (step (S14-4), and when Pij <Pth, Pij = “0”) (step S14-5) This process is performed for all pixels. The binarization process ends.

In the configuration and method according to the present embodiment, a histogram is further used as a feature amount. Then, binarization processing is performed by using this histogram. Therefore, the effects (1) and (2) described in the first embodiment can be obtained.
As described above, in the LSI, the image processing system, and the image processing method according to the first to fourth embodiments of the present invention, the feature quantity extraction device 32 that performs statistical processing is separated from the highly parallel processing device 40. being independent. Each processing unit 41 can freely read the feature value from the memory 42. Therefore, it is not necessary to consider statistical processing in the processing program in the highly parallel processing device 40, and the feature amount can be read from the memory 42 and processed as necessary. Therefore, the processing program in the highly parallel processing device 40 can be simplified, and the burden on the programmer can be greatly reduced. In addition, since it is not necessary for the highly parallel processing device 40 to perform statistical processing, the feature extraction efficiency can be improved and the processing efficiency in the highly parallel processing device 40 can be improved. Furthermore, the processing accuracy can be improved by applying the above configuration to template matching and MPEG decoding.

  In the above-described embodiment, a case has been described in which all the processing units 41 in the highly parallel processing device 40 perform the same type of processing. However, as shown in FIG. 17, each of the processing units 41 may read one of the programs 51-1, 51-2,... From the storage device 50 to the main memory 100 and perform processing according to the program as appropriate. At this time, which program is to be executed can be designated by the program counter 110 or the like, for example. Therefore, it is also possible for each of the processing units 41 to execute different processes based on different programs.

  In the third embodiment, the DCT and quantization processes are performed using the dedicated hardware 80. However, these processes may be performed by the highly parallel processing device 40 using software. Furthermore, in the fourth embodiment, the threshold value Pth may be treated as a feature value, and the feature value extraction device 32 may perform histogram creation and threshold value determination in step S12.

  The configuration of the feature quantity extraction device 32 is not particularly limited as long as the feature quantity extraction device 32 can operate independently from the highly parallel processing device 40 and the processing unit can freely read the feature quantity. The feature amount calculation may be processed by hardware or may be processed by software. If necessary, statistical processing may be performed by firmware using a scalar processor.

That is, the semiconductor integrated circuit device according to the embodiment of the present invention is
1. A feature extraction device that performs statistical processing on digital image data and extracts features;
A plurality of processing units, and each processing unit includes a parallel processing device that performs processing on the image data using the feature amount, and each of the processing units is extracted by the feature amount extraction device. A holding circuit for holding the feature quantity is provided, and processing is performed using the feature quantity held in the holding circuit, and the feature quantity extraction device performs an operation independently of the parallel processing device.
2. In the above item 1, the feature amount is an average value of pixel values of a region including a plurality of pixels in the image data.
3. In the above 1 or 2, a storage device that holds a plurality of programs for operating the processing unit;
A main memory from which the program is read, and the processing unit executes a process based on any of the programs read to the main memory.

The image processing system according to the above embodiment is
4). 4. The semiconductor integrated circuit device according to any one of 1 to 3 above;
An image pickup device for taking an image and acquiring the digital image data, and the semiconductor integrated circuit receives the image data from the image pickup device, and temporarily holds the image data;
And a storage device for storing the image data processed by the parallel processing device, wherein the feature amount extraction device extracts the feature amount of the image data transferred to the buffer circuit.

Furthermore, the image processing method according to the above embodiment is as follows.
5. An image processing method for processing image data using a parallel processing device having a plurality of processing units, the step of receiving the image data;
A feature amount extraction device calculating a feature amount for the received image data;
Transferring the feature quantity to each of the processing units;
Each of the processing units performs processing related to the image data using the transferred feature quantity. Further, the semiconductor integrated circuit device comprises:
6). In 2 above, the parallel processing device subtracts the feature amount from the pixel values of the pixels included in the two image data, and calculates a difference between the subtraction results between the two image data.
7). In the above item 6, the parallel processing device divides the subtraction result by the variance of the pixel values of the image data, and calculates a difference between the division results between the two image data.
8). In the above item 2, the parallel processing device binarizes the image data based on the feature amount from the pixel value of the pixel included in the image data, and the feature amount is a histogram.

Furthermore, the image processing method
9. In the above item 5, the feature amount is an average value of pixel values of a region including a plurality of pixels in the image data.
10. In step 9, the step of performing the process relating to the image data includes subtracting the feature amount from the pixel values of the pixels included in the two image data;
Calculating a difference between the two image data of the subtraction result obtained in the subtracting step.
11. In step 9, the step of performing the process relating to the image data includes subtracting the feature amount from the pixel values of the pixels included in the two image data;
Dividing the subtraction result obtained in the subtracting step by the variance of the pixel values in the respective image data;
A step of calculating a difference between the two image data based on the division result obtained in the dividing step.
12 9. In the above 9, the processing relating to the image data includes the step of creating a histogram,
And binarizing the image data based on the histogram.

Furthermore, the image processing system
13. In 4 above, the parallel processing device subtracts the feature amount from the pixel values of the pixels included in the two image data, and calculates a difference between the subtraction results between the two image data.
14 In the above 13, the parallel processing device divides the subtraction result by the variance of the pixel values of the image data, and calculates a difference of the division result between the two image data.
15. In the above item 4, the parallel processing device binarizes the image data based on the feature amount from the pixel value of the pixel included in the image data.

  Note that the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the scope of the invention in the implementation stage. Furthermore, the above embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, the problem described in the column of the problem to be solved by the invention can be solved, and the effect described in the column of the effect of the invention Can be extracted as an invention from which this configuration requirement is deleted.

1 is a block diagram of an image processing system according to a first embodiment of the present invention. 1 is a flowchart of an image processing method according to the first embodiment of the present invention. 1 is a conceptual diagram of a memory space in a storage device included in an image processing apparatus according to a first embodiment of the present invention. 4 is a timing chart of each step in the image processing method according to the first embodiment of the present invention. The conceptual diagram of the template matching in the image processing method which concerns on 2nd Embodiment of this invention. The schematic diagram of the image I which is for demonstrating the calculation method of the feature-value in the image processing method which concerns on 2nd Embodiment of this invention, and contains N pixels. The schematic diagram of the image In + 1 containing N pixels for demonstrating the calculation method of the feature-value in the image processing method which concerns on 2nd Embodiment of this invention. The graph for demonstrating the template matching in the image processing method which concerns on 2nd Embodiment of this invention, and shows the relationship between a position and a pixel value. The graph for demonstrating the template matching in the image processing method which concerns on 2nd Embodiment of this invention, and shows the relationship between a position and a pixel value. The block diagram of the image processing apparatus which concerns on 3rd Embodiment of this invention. The block diagram of the MPEG decoder with which the image processing apparatus which concerns on 3rd Embodiment of this invention is provided. The conceptual diagram of the motion compensation prediction in the image processing method which concerns on 3rd Embodiment of this invention. The graph for demonstrating the motion compensation prediction in the image processing method which concerns on the 3rd Embodiment of this invention, and shows the relationship between a position and a pixel value. The graph which is for demonstrating the binarization process in the image processing method which concerns on 4th Embodiment of this invention, and shows the relationship between a pixel value and the number of pixels. The graph which is for demonstrating the binarization process in the image processing method which concerns on 4th Embodiment of this invention, and shows the relationship between a pixel value and the number of pixels. The flowchart of the image processing method which concerns on 4th Embodiment of this invention. The block diagram of the image processing apparatus which concerns on the modification of the 1st thru | or 4th embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Image processing system 2, 20 ... LSI, 3 ... Host processor, 4 ... I / O processor, 5 ... Main memory, 6 ... Graphic processor, 30 ... Data input device, 31 ... Input buffer, 32 ... Feature amount extraction 40: highly parallel processing device, 41: processing unit, 42: feature quantity storage device, 50 ... storage device, 51-1, 51-2 ... program, 60 ... DMA controller, 70 ... data bus, 80 ... MPEG decoder , 81 ... subtractor, 82 ... adder, 83 ... DCT section, 84 ... quantization section, 85 ... variable length coding section, 86, 87 ... buffer, 88 ... rate control section, 89 ... inverse quantization section, 90 ... Inverse DCT unit, 91 ... Motion compensation prediction unit, 100 ... Main memory, 110 ... Program counter

Claims (5)

A feature extraction device that performs statistical processing on digital image data and extracts features;
A plurality of processing units, and each processing unit includes a parallel processing device that performs processing on the image data using the feature amount, and each of the processing units is extracted by the feature amount extraction device. A holding circuit for holding the feature amount, and performing processing using the feature amount held in the holding circuit;
The feature quantity extraction device performs an operation independently of the parallel processing device. A semiconductor integrated circuit device, wherein: The semiconductor integrated circuit device according to claim 1, wherein the feature amount is an average value of pixel values of a region including a plurality of pixels in the image data. A storage device that holds a plurality of programs for operating the processing unit;
The semiconductor integrated circuit according to claim 1, further comprising: a main memory from which the program is read, wherein the processing unit executes a process based on any one of the programs read into the main memory. Circuit device. A semiconductor integrated circuit device according to any one of claims 1 to 3,
An image pickup device for taking an image and obtaining the digital image data, and the semiconductor integrated circuit receives the image data from the image pickup device and temporarily holds the image data;
A storage device that stores the image data processed by the parallel processing device, and the feature amount extraction device extracts the feature amount of the image data transferred to the buffer circuit. Processing system. An image processing method for processing image data using a parallel processing device including a plurality of processing units,
Receiving the image data;
A feature amount extraction device calculating a feature amount for the received image data;
Transferring the feature quantity to each of the processing units;
Each of the processing units performs processing related to the image data using the transferred feature quantity. An image processing method comprising:

Images