JP2011086263A - Data processing apparatus, and processing method therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To shorten the wait time due to parameter loading, when collating frame data of a moving image. <P>SOLUTION: In a data processor is disclosed which performs collation processing on data representing an input image by parameters held in a parameter holding means, the data processor holds the result of collation processing, which determines whether parameters for use in the next collation processing are to be prefetched, according to the held result of collation processing and acquires parameters for use in the next collation processing, on the basis of the result of the determination. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a data processing apparatus that repeatedly processes input data and a processing method thereof.

  2. Description of the Related Art Conventionally, there has been proposed a technique for detecting a specific subject such as a person or a face in an input image in a digital camera or a printer, and performing processing suitable for the detected subject. As an example of detecting this specific subject, there is a “face detection process” for performing a skin color correction process on the face.

  Various methods have been proposed for this face detection process. For example, a method of detecting a human face by using a Viola & Jones method, a symmetrical feature of a human face, template matching, a neural network, etc. is shown. ing.

  In addition, when performing face detection processing, as a conventional technique for speeding up processing while suppressing the circuit scale of the internal memory, there are a cache method and a prefetch method (a method for preparing parameters required next in advance). Widely used. The cache method is a method in which parameters are stored in a cache memory and deleted from the oldest parameters when the memory becomes full. On the other hand, the prefetch method is a method in which parameters required next are acquired in a parameter holding unit.

P. Viola and M. Jones, “Robust Real-time Object Detection”, SECOND INTERNATIONAL WORKSHOP ON STATISTICAL AND COMPUTATIONAL THEORIES OF VISION, July 13 2001.

  However, the conventional cache method and refetch method for increasing the processing speed while suppressing the circuit scale of the internal memory have the following problems.

  In the cache method that erases from the oldest dictionary data, if the process proceeds to a step process that has more parameters than the number of saved parameters and an erroneous determination is made, the next input data process must be reloaded with all dictionary data. Met. Therefore, there is a problem that the cache effect is not at all.

  Further, the prefetch method performs prefetching by predicting that the next required dictionary data is the dictionary data for the next step, but the recognition process has a feature that the prediction may be lost. Therefore, in the prefetch method, for example, the dictionary data 3 is loaded during the process of step 2, but if the processing result of step 2 is erroneously determined, the loading of the dictionary data 3 is wasted. Furthermore, since the next required dictionary data 1 is deleted, there is a problem that it takes time to load the dictionary data 1 again.

  An object of the present invention is to reduce useless prefetch and reduce waiting time due to parameter loading by determining whether or not to perform prefetch using a past processing history.

The present invention is a data processing apparatus that performs a collation process using data held in a parameter holding unit for data indicating an input image,
Processing history holding means for holding the result of the matching process;
A determination unit that determines whether to prefetch a parameter used for the next matching process, according to the held result of the matching process;
Parameter acquisition means for acquiring a parameter used for the next matching process based on the result of the determination;
It is characterized by having.

  According to the present invention, it is possible to reduce useless prefetch and reduce waiting time due to parameter loading by determining whether or not to perform prefetch using past processing history. Therefore, it is possible to realize a data processing apparatus capable of increasing the processing speed while suppressing the circuit scale of the internal memory.

The block diagram which shows the structure of the data processor in 1st Embodiment. The figure which shows the pattern matching process in a recognition process. The figure for demonstrating a pattern collation process. The figure for demonstrating the recognition process in 1st Embodiment. The figure which shows the processing time of the recognition process in 1st Embodiment. The flowchart which shows the recognition process of the data processor in 1st Embodiment. The block diagram which shows the structure of the data processor in 3rd Embodiment.

  Hereinafter, embodiments for carrying out the invention will be described in detail with reference to the drawings. In the present embodiment, a description will be given of a recognition process for recognizing a human face in an input image by a pattern matching process, for example, a Viola & Jones technique in a digital camera or a printer.

  The data processing apparatus is mounted on a digital camera or printer, repeatedly performs pattern matching processing on the input image, and determines whether or not to perform the next pattern matching processing according to the processing result.

[First Embodiment]
Here, processing for determining whether to prefetch a parameter to be collated with an input image using a past processing history and prefetching the parameter according to the determination result will be described with reference to FIGS. The parameter is a feature amount necessary for pattern matching processing, a threshold value used as a criterion for correct / incorrect determination (determination of face or non-face), and is hereinafter referred to as dictionary data. In addition, a processing unit for performing correctness determination for each verification process is referred to as a step. In addition, the recognition process may be applied to a moving image. In this case, the recognition process is performed on each frame of the moving image.

  FIG. 1 is a block diagram illustrating a configuration of a data processing device according to the first embodiment. In FIG. 1, a calculation unit 101 receives a rectangular image of a frame currently being processed and dictionary data necessary for the step currently being processed, performs a matching process, and outputs a processing result. Here, as shown in FIG. 2, the collation process is performed in step 0 (201), step 1 (202), step 2 (203),... As shown in FIG. 3, in step 0, the feature quantity 0 (301) is collated with the rectangular image (303), and in step 1, the feature quantity 1 (302) is collated with the rectangular image (304).

  Returning to FIG. 1, the processing history holding unit 102 holds the processing result output from the calculation unit 101. The parameter acquisition unit 103 detects the next necessary dictionary data by receiving the processing result output from the calculation unit 101. Here, when the necessary dictionary data is not held in either of the parameter holding units 104 and 105, the required dictionary data is loaded. In order to perform the loading process, the parameter acquisition unit 103 outputs a parameter input request to a control unit (not shown). When receiving a parameter input response to the parameter input request, the parameter acquisition unit 103 outputs a parameter load permission to the parameter holding unit 104 or 105. Thus, the parameter holding unit 104 or 105 that has received the parameter load permission signal loads the input parameter.

  When loading of necessary dictionary data is completed or when necessary dictionary data is already held in the parameter holding unit 104 or 105, the parameter acquisition unit 103 instructs the parameter switching unit 106 to switch parameters. The parameter switching instruction is a select signal for switching the parameter holding unit 104 or 105 so that the calculation unit 101 can perform a calculation using a necessary parameter. When the parameter switching is completed, the calculation unit 101 starts processing.

  Simultaneously with instructing the start of processing, the parameter acquisition unit 103 outputs the number of the step currently being processed to the prefetch determination unit 107. On the other hand, the prefetch determination unit 107 determines whether to prefetch dictionary data required in the next step based on the input step number, and returns the determination result to the parameter acquisition unit 103. Specifically, determination processing is performed by receiving address information indicating an area currently being processed from the calculation unit 101 and referring to the processing result of the previous frame held in the processing history holding unit 102.

  Next, when the parameter acquisition unit 103 receives the determination result that prefetching is performed, in order to request loading of parameters necessary for the next step, a parameter input request for the next step is issued. When a parameter input response to the parameter input request is received, a parameter load permission is output to the other parameter holding unit that is not used in the current step, and the input parameter is loaded into the parameter holding unit. .

  When the parameter acquisition unit 103 receives the processing result from the calculation unit 101 and makes a positive determination, the collation processing of the next step is performed. Here, if the parameter of the next step is already held in the parameter holding unit by the above-described prefetch, the collation process of the next step can be performed only by issuing a switching instruction to the parameter switching unit 106.

  Next, information held in the processing history holding unit 102 will be described in detail. The processing history holding unit 102 holds the processing result output from the calculation unit 101. Here, the processing result is the address information of the area where the collation processing has been performed and the number of the last step where the collation processing of that area has been determined to be positive. The area address information includes information for identifying a frame and information for identifying a rectangular image area in the frame. For example, a frame number can be used as information for identifying a frame. As information for identifying the rectangular image area in the frame, the address of the upper right pixel in the rectangular image area can be used.

  Next, step numbers held in the processing history holding unit 102 will be described. If the processing result in a certain region is erroneously determined at a certain step, the processing result up to that step is considered to be a correct determination. Therefore, the number before the step is held in the processing history holding unit 102. For example, when the collation process is erroneously determined in step 3, step 2 is held in the process history holding unit 102. Similarly, if the collation process is erroneously determined in step 1, step 0 is held in the process history holding unit 102. Therefore, when the collation process is positive in the final step, the final step number is held in the process history holding unit 102.

  Next, the determination process in the prefetch determination unit 107 will be described in detail. The prefetch determination unit 107 receives address information indicating an area currently being processed from the calculation unit 101. Based on the received address information, with reference to the processing result of the previous frame of the currently processed rectangular image area held in the processing history holding unit 102, the last step in which the collation process of the previous frame is determined to be positive Get the number. Further, the prefetch determination unit 107 receives the number of the step currently being processed from the parameter acquisition unit 103, and compares the number of the next step with the step number acquired from the processing history holding unit 102. As a result of the comparison, if the number of the next step is less than or equal to the number of the step acquired from the processing history holding unit 102, it is determined that prefetching is performed. Then, the determination result is output to the parameter acquisition unit 103.

  Here, in the case of recognition processing using a moving image, the processing result of the previous frame is very close to the processing result of the currently processed frame. This is because the moving speed of the recognition target object such as the face or the human body is sufficiently slow with respect to the frame rate of the moving image. That is, there is a high possibility that the processing result of the currently processed frame is also positively determined until the step in which the processing result of the previous frame is positively determined. Therefore, it is possible to reduce useless prefetching by prefetching dictionary data up to the step in which a positive determination is made in the previous step.

  Next, the processing time of the recognition process in the first embodiment will be described with reference to FIGS. Before explaining the processing time of this recognition process, terms necessary for the explanation are defined. First, the step processing time is the time required for the collation processing of each step. The parameter loading time is a time required for loading dictionary data stored in the external memory into the parameter holding unit. Here, an example is shown in which the parameter loading times of S1, S2, and S3 are gradually increased to 1, 2, and 4 times. This is because it is assumed that the number of collation processes increases as the step goes to the subsequent stage. All step processing times show the same example. This is because the hardware can be created so that a plurality of collation processes can be executed in parallel. Further, it is assumed that the step processing time and the parameter loading time of S1 are the same.

  FIG. 4 is a diagram for explaining the recognition processing in the first embodiment. The horizontal direction shown in FIG. 4 is the line direction, and the vertical direction is the depth of the matching processing. In this example, for the leftmost rectangular image area A1 in a certain line direction, the collation process of S1 is a positive determination, and the next collation process of S2 is an erroneous determination. Further, for the rectangular image area A2, the collation process of S1 and S2 is a positive determination, and the collation process of S3 is an erroneous determination. As described above, the collation process in each step for the rectangular image area is performed until the result is erroneously determined or all the collation processes are positively determined.

  FIG. 5 is a diagram illustrating the processing time of the recognition process in the first embodiment. In FIG. 5, the uppermost processing cycle is a time during which the calculation unit 101 is performing the matching process. Further, A1 (S1) below is a cycle related to the collation process in step 1 for the rectangular image area A1. S1, S2, and S3 below are dictionary data load cycles. Here, S1 indicates a load cycle of dictionary data used in Step 1, and S2 indicates a load cycle of dictionary data used in Step 2. The bottom row shows dictionary data held in the parameter holding units 104 and 105.

  In FIG. 5, first, parameters necessary for S1 are loaded. The prefetch determination unit 107 determines the step to which prefetch should be performed for A1 that is currently being processed, at the same time that the arithmetic unit 101 performs S1 collation processing for A1. The prefetch determination unit 107 refers to the processing result in the previous frame of A1 from the processing history holding unit 102. Here, since the processing result in the previous frame is a positive determination until S1, it is determined to prefetch dictionary data up to S1 for A1 currently being processed. Then, the prefetch determination unit 107 receives the number (S1) of the step currently being processed from the parameter acquisition unit 103, and determines whether or not to prefetch dictionary data in the next step (S2). Here, since the received number is the same as the step determined to prefetch, it is determined not to prefetch the next dictionary data in S2. The result of the determination is returned to the parameter acquisition unit 103, whereby the parameter acquisition unit 103 does not prefetch the dictionary data in S2.

  Subsequently, when the result of A1 (S1) is erroneously determined in the calculation unit 101, A2 (S1) as the next process is performed. The prefetch determination unit 107 determines to which step prefetching should be performed for A2 as in A1. In A2, since the collation process is positive until S2 in the previous frame, it is determined that prefetching is performed until S2. Further, the prefetch determination unit 107 receives the number (S1) of the step currently being processed from the parameter acquisition unit 103, and determines whether or not to prefetch the next dictionary data of S2. Here, since the received number is not the same number as the step determined to perform prefetching, it is determined that the dictionary data in S2 is to be prefetched. The determination result is returned to the parameter acquisition unit 103, and the parameter acquisition unit 103 prefetches the dictionary data in S2.

  Subsequently, the result of A2 (S1) becomes a positive determination, and since the dictionary data prefetching of S2 is still performed at this time, the process waits until the prefetching of dictionary data of S2 is completed. When dictionary data prefetch is completed, parameters are switched and the process of A2 (S2) is performed. Since the determination result of prefetch for A2 is up to S2, the parameter acquisition unit 103 does not prefetch dictionary data in S3. At this time, the parameter holding units 104 and 105 hold the parameters S1 and S2. If the processing result of A2 (S2) is erroneously determined, the next A3 (S1) is performed. However, since the parameter of S1 is held in the parameter holding unit 104, the matching process can be continuously started.

  As described above, when the processing result matches the result of the prefetch determination, the processing stop due to the dictionary data load waiting does not occur unlike the cache method. Further, there is no need to reload the next necessary dictionary data, which is caused by overwriting the necessary dictionary data while the processing is stopped during the loading time of unnecessary dictionary data for prefetching.

  Therefore, according to the first embodiment, it is 11 cycles until the processing of A3 (S3) is completed, and it can be seen that the processing is fast.

  Next, the recognition processing of the data processing apparatus in the first embodiment will be described in detail with reference to FIG. The recognition process is performed in F601 to F615. When the recognition process starts in F601, data of the cut-out (or addressed) rectangular image area is input in F602. In F603, prefetch determination value calculation processing is performed on the input data (input data). In the prefetch determination value calculation process, it is determined to what step prefetching is performed for input data. Here, it is assumed that prefetching is performed up to the processing of the step in which the processing of data input one frame before becomes the last positive determination.

  When the prefetch determination value calculation processing is completed, step collation processing is sequentially executed in F604 to F611 and F613 to F615. In the collation process at each step, first, at F605, it is determined whether there is dictionary data for the next step. If there is dictionary data, the process proceeds to F606, where the parameter acquisition unit 103 issues a parameter switching instruction to the parameter switching unit 106 to switch parameters. By this switching, the calculation unit 101 can perform pattern matching processing using dictionary data necessary for matching processing in the next step.

  On the other hand, if there is no dictionary data in F605, the process proceeds to F613 to determine whether prefetch processing is in progress. If prefetch processing is in progress, the process advances to F615 to wait for completion of the prefetch processing. If the prefetch process is not in progress, the process proceeds to F614, the parameter acquisition process is performed, and the process waits until the parameter acquisition process is completed in F615. The parameter acquisition process in F614 is not a parameter prefetch process but a normal parameter load process.

  In F615, when the parameter acquisition processing is completed by prefetch or parameter loading processing, the processing proceeds to F606, and the parameter acquisition unit 103 issues a parameter switching instruction to the parameter switching unit 106 to switch the parameters. By this switching, the calculation unit 101 can perform pattern matching processing using dictionary data necessary for matching processing in the next step.

  Next, in F607, it is determined whether or not to prefetch dictionary data in the next step. In this determination, if the number of the next step is a number equal to or smaller than the determination value acquired in F603, it is determined that prefetching is performed. If it is determined that prefetching is not performed, the process proceeds to F609. If it is determined that prefetching is performed, the process proceeds to F608, and prefetch processing of parameters necessary for the next step is started. The parameter acquisition unit 103 issues a parameter (input) request. After issuing this parameter (input) request, collation processing is started in F609.

  When the collation process ends, the result of the collation process is determined in F610. Here, in the case of a positive determination, the process proceeds to F611, and it is determined whether or not the collation process of the final step is completed. If the result of determination is that the collation processing for the final step has not been completed, the process returns to F604 to perform collation processing for the next step. Further, when the final step collation process is completed in F611, or if the process result is an erroneous determination in F610, the process proceeds to F612 and whether or not the recognition process has been performed up to the data of the final rectangular image area. Make a decision. If the recognition process has not been completed up to the data of the final rectangular image area, the process returns to F601 to perform the recognition process for the data of the next rectangular image area. On the other hand, when the process for the data of the final rectangular image area is completed, the recognition process is terminated.

  According to the first embodiment, it is possible to reduce useless prefetch and reduce waiting time due to parameter loading by determining whether or not to perform prefetch using the past processing history.

[Second Embodiment]
Next, a second embodiment according to the present invention will be described in detail with reference to the drawings. The configuration of the data processing device in the second embodiment is the same as the configuration of the data processing device described in the first embodiment, but the operation of the prefetch determination unit is different.

  In the first embodiment, the processing result of the previous frame of the data of the rectangular image area currently being processed is referred to from the processing history holding unit 102, and the processing result is directly output to the parameter acquisition unit 103 as a determination result. It was.

  In the second embodiment, the prefetch determination unit 107 refers to the processing result of the previous frame in the vicinity of the currently processed rectangular image area, obtains the average value of the processing results, and uses the average value as the determination result. And output to the parameter acquisition unit 103.

  Here, a case will be described in which the processing result in the vicinity of one pixel of the rectangular image region currently being processed is used as the determination result. First, assume that the address of the rectangular image area currently being processed is (X, Y). Here, X represents the position in the line direction of the image, and Y represents the position in the column direction of the image.

  The prefetch determination unit 107 refers to the processing result of the rectangular image area indicated by the following nine addresses of the previous frame from the processing history holding unit 102.

(X-1, Y-1), (X, Y-1), (X + 1, Y-1),
(X-1, Y), (X, Y), (X + 1, Y),
(X-1, Y + 1), (X, Y + 1), (X + 1, Y + 1)
In order to obtain the average of the processing results in the vicinity of one pixel, the sum of the nine processing results referred to is obtained and divided by nine. Note that there is a possibility that the processing result for nine pixels may not be obtained near the edge of the image. In that case, an average value can be obtained by taking only the sum of the obtained processing results and dividing by the number of the obtained processing results. In addition, the result of division may not be divisible. In that case, the obtained result may be rounded off, rounded down or rounded up.

  The prefetch determination unit 107 outputs the obtained average value to the parameter acquisition unit 103, and the parameter acquisition unit 103 performs prefetching of parameters until the step of the average value.

[Third Embodiment]
Next, a second embodiment according to the present invention will be described in detail with reference to the drawings. The configuration of the data processing apparatus according to the third embodiment will be described with reference to FIG. The configuration of the data processing apparatus of the third embodiment is obtained by adding a motion vector detection unit 108 to the configuration of the data processing apparatus of the first embodiment. Hereinafter, the detection process of the motion vector detection unit 108 and the determination process of the prefetch determination unit 107 that determines whether or not to perform prefetching with reference to the result will be described.

  The motion vector detection unit 108 detects the motion vector of the object at each pixel between frames. As a result of the detection, the direction of the motion vector and the magnitude of the vector at each pixel are obtained. For example, if the motion vector is +2 in the X direction and -1 in the Y direction, the result is obtained in the form (+2, -1).

  The prefetch determination unit 107 refers to the motion vector of the rectangular image area currently being processed from the motion vector detection unit 108. Further, when referring to the processing result of the previous frame held in the processing history holding unit 102, an offset is added to the reference destination address using the referenced motion vector.

  Here, a case will be described in which the address of the rectangular image area currently being processed is (X, Y) and the motion vector in the area is given by (+2, -1). In this case, the address to which the offset is added is (X-2, Y + 1). When referring to the processing result of the previous frame from the processing history holding unit 102, the address (X-2, Y + 1) with an offset is used. .

  The prefetch determination unit 107 outputs the obtained processing result of the address with offset to the parameter acquisition unit 103, and the parameter acquisition unit 103 performs prefetching of the parameter until the step of the processing result of the address with offset.

  In this way, by adding an offset to the reference destination address using a motion vector, it is possible to correct the motion of the object, and it is possible to perform prediction with higher accuracy. In particular, when processing is performed with thinning out frames or when an object with a high moving speed is recognized, the correlation between frames becomes low, so that motion correction is effective.

  In the above-described embodiment, the parameter holding units 104 and 105 have been explicitly described separately, but may be implemented by distinguishing areas of the same parameter holding unit.

[Other embodiments]
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (7)

A data processing apparatus for performing a collation process on data indicating an input image using a parameter held in a parameter holding unit,
Processing history holding means for holding the result of the matching process;
A determination unit that determines whether to prefetch a parameter used for the next matching process, according to the held result of the matching process;
Parameter acquisition means for acquiring a parameter used for the next matching process based on the result of the determination;
A data processing apparatus comprising:   The data processing apparatus according to claim 1, wherein the input image is a moving image, and the matching process is performed on data in a frame of the moving image.   The determination unit determines that the parameter used for the verification process for the next frame is to be prefetched when the result of the verification process for the previous frame of the moving image is held in the processing history holding unit. The data processing apparatus according to claim 2.   The said determination means determines whether the parameter used for the said next collation process is prefetched using the average value of the process result in the several area | region of the front frame, The said determination means determines whether the parameter used for the said next collation process is prefetched. Data processing device. Motion vector detecting means for detecting motion between frames of the moving image;
3. The determination unit according to claim 2, wherein when the processing result stored in the processing history storage unit is referred to, the determination unit adds a correction to a reference destination address using the result of the motion vector detection unit. Data processing equipment. A processing method of a data processing apparatus for performing a collation process on data indicating an input image using a parameter held in a parameter holding unit,
A process history holding step in which a processing history holding unit of the data processing apparatus holds a result of the matching process;
A determination step of determining whether or not the determination unit of the data processing device prefetches a parameter used for the next verification process according to the stored result of the verification process;
A parameter acquisition step of acquiring a parameter used for the next matching process based on the result of the determination, by the parameter acquisition means of the data processing device;
A processing method for a data processing apparatus, comprising:   The program for making a computer perform each process of the processing method of the data processor of Claim 6.

Images