JP5836724B2 - Image recognition method, image recognition apparatus, and program - Google Patents

Description

  The present invention relates to an image recognition method, an image recognition apparatus, and a program, and more particularly to a technique suitable for specifying an individual from face image data.

  In the recognition of individuals using face image data (hereinafter referred to as face recognition), the determination of the position of a facial organ or a characteristic part corresponding to it (hereinafter referred to as a feature point) is an important task. I often do it. However, highly accurate feature point position determination requires a high processing load, and may limit the time of the entire recognition process.

  Therefore, Patent Document 1 discloses a technique for reducing the number of feature points to be extracted from a processing target frame using a recognition result of a previous frame when an individual is recognized from moving image data. That is, when a target person is recognized once (tracking state), the speed is increased by reducing the number of feature points extracted in the next frame.

JP 2009-75999 A

Beumer, G.M .; Tao, Q .; Bazen, A.M .; Veldhuis, R.N.J. “A landmark paper in face recognition” Automatic Face and Gesture Recognition, 2006. FGR 2006. 7th International Conference, pp. 73-78

  However, according to the technique disclosed in Patent Document 1, with the reduction of feature points to be extracted, subsequent processing using the positions of the feature points (processing such as feature amount extraction or comparison of similarity to registered data). The contents are different. Therefore, it is necessary to prepare in advance a plurality of types of registration data corresponding to the number of feature points. At that time, registration data generated based on feature points with low position accuracy, such as a small number of feature points, is likely to have low reliability. For this reason, when data with low reliability is mixed in a part of registered data, the recognition accuracy is greatly lowered.

  In view of the above-described problems, an object of the present invention is to reduce a reduction in recognition accuracy even when recognition processing is executed at high speed.

The image recognition method of the present invention includes an image data acquisition step of acquiring image data, a first feature point position detection step of detecting a position of a feature point from the acquired image data, and the first of the acquired image data. a second feature point position detection step of detecting a position of a number of feature points than the first feature point position detection step, the process according to the first feature point position detection step or the second feature point position detection step, A selection step for selecting, a feature amount calculation position determining step for determining a feature amount calculation position based on the selected result, and a feature amount calculation for calculating a feature amount based on the result of the feature amount calculation position determining step step and a determination step of determining the processing mode, the determining if the step is the processing mode determined by the registration process is to select the processing by the second feature point position detection step, the second characteristic point position Inspection If the result and registration step of generating registration data from the feature amount calculated based on the results of the feature quantity calculating positioning process step, the determination processing mode determined by the process of the recognition process, the first feature select processing by the point position detecting step, recognizing the image data on the basis of the calculated feature amount based on the result of the first feature point position detection process result and the feature amount calculating positioning step and the registration data And the feature amount calculation position determination step is the first feature point position detection step when the processing by the first feature point position detection step is selected in the selection step. The detected feature point and the position of the feature point that is detected in the second feature point position detection step but not detected in the first feature point position detection step are represented by predetermined coordinate values. Based on the feature points obtained by replacement, and determines the calculated position of the feature quantity.

  According to the present invention, it is possible to suppress a reduction in recognition accuracy even when the position of a feature point is detected at high speed during recognition processing.

It is a figure explaining the outline | summary of operation | movement of 1st Embodiment. It is a block diagram which shows the structural example of the image recognition apparatus of 1st Embodiment. It is a figure explaining the example of cut-out of a face image. It is a figure explaining the example of a feature point. It is a figure explaining the example of a feature point position detection means. It is a figure explaining the example of correction of a feature point. It is a figure explaining the process example of a feature point position correction means. It is a flowchart explaining operation | movement of 1st Embodiment. It is a flowchart explaining operation | movement of 2nd Embodiment. It is a flowchart explaining operation | movement of 3rd Embodiment. It is a block diagram which shows the structural example of the image recognition apparatus of 3rd and 4th embodiment. It is a flowchart explaining operation | movement of 4th Embodiment.

(First embodiment)
The operation of the first embodiment of the present invention will be described below.
FIG. 1 is a diagram for explaining the outline of the processing of this embodiment.
In FIG. 1, reference numeral 109 denotes an outline of processing during recognition processing. Here, the recognition process is a process for identifying an individual from the face image data acquired in the face image data acquisition process 100. Reference numeral 110 denotes an outline of processing at the time of registration processing. Here, the registration process is a process of generating registration data to be used during the recognition process from the face image data acquired in the face image data acquisition process 100. In the recognition process 109 and the registration process 110, processes having the same number are configured as a common process. The selection of the recognition process and the registration process is based on the result of the process mode determination process 111.

  During the recognition process, the feature point position is detected from the face image data acquired in the face image data acquisition process 100. Here, the feature points are positions of characteristic image shapes related to facial organs such as eyes, nose and mouth. In this embodiment, two feature point position detection processes are performed. The first feature point position detection process 102 is a high-speed feature point position detection process that is the first feature point position detection. On the other hand, the second feature point position detection process 103 is a highly accurate feature point position detection process which is the second feature point position detection. For example, in the first feature point position detection process 102, the number of feature points to be calculated is smaller than that in the second feature point position detection process 103.

  The recognition operation mode setting process 101 is a setting process for instructing whether the recognition process is operated with high accuracy or at high speed. The selection process 104 selects a feature point position detection process according to the recognition operation mode setting process 101. When it is desired to select a high-speed mode for operating recognition processing at high speed, the first feature point position detection process 102 is selected. When a high-precision mode for operating with high accuracy is selected, second feature point position detection processing is selected. 103 is selected.

  In the feature amount calculation position determination processing 105, a position serving as a reference for calculating the feature amount is determined based on the result of the first feature point position detection processing 102 or the second feature point position detection processing 103 with different detection accuracy. To do. Here, a common feature amount calculation position reference is determined regardless of which feature point position detection process is selected.

  In the feature amount calculation process 106, a predetermined feature amount is calculated based on the result of the feature amount calculation position determination process 105. In this embodiment, since the common reference position is calculated by the feature amount calculation position determination processing 105, the common feature amount calculation processing 106 is used regardless of the type of feature point position detection processing (number of feature points, etc.). be able to. In the determination processing 107, the facial image of the recognition processing target is obtained by comparing the feature amount of the recognition target facial image obtained by the feature calculation processing 106 with the registration data generated by the registration data generation processing 108. It is determined whether or not.

  In the registration process 110, the feature point position is detected from the face image data input from the face image data acquisition process 100. The second feature point position detection process 103 detects the position of the feature point with high accuracy. In the feature amount calculation position determination processing 105, a reference position for calculating the feature amount is determined according to the result of the second feature point position detection processing 103. In the feature amount calculation process 106, a predetermined feature amount is calculated based on the result of the feature amount calculation position determination process 105. In the registered data generation process 108, the feature quantity for the face image data calculated in the feature quantity calculation process 106 is recorded together with information for identifying an individual corresponding to the face image.

  In the present embodiment, the feature point position detection process is switched at high speed or with high accuracy according to the recognition operation mode at the time of recognition processing, and registration data is generated by the highly accurate feature position detection process at the time of registration processing. That is, at the time of recognition processing, recognition processing is executed using common registration data generated based on high-precision feature position detection processing regardless of the type of feature point position detection processing. In the present embodiment, the detection accuracy of the feature point position detection process may be variably set according to the mode.

  FIG. 2 is a block diagram illustrating a configuration example of the image recognition apparatus according to the present embodiment. In the image recognition apparatus of this embodiment, first, face image data is extracted from image data. Then, a plurality of feature point positions are detected from the obtained face image data, and an individual is identified from the feature amount calculated based on the feature point positions.

  In FIG. 2, an image input unit 201 includes an optical system device, a photoelectric conversion device, a driver circuit that controls a sensor, an AD converter, a signal processing circuit that controls various image corrections, a frame buffer, and the like. Reference numeral 202 denotes a preprocessing unit that executes various types of preprocessing in order to effectively perform various types of subsequent processing. Specifically, image data conversion such as color conversion processing and contrast correction processing is performed on the image data acquired by the image input unit 201 by hardware.

  The face image data cutout processing unit 203 performs face detection processing on the image data corrected by the preprocessing unit 202. Various conventionally proposed methods can be applied to the face detection method. The face image data cutout processing unit 203 normalizes and cuts out face image data to a predetermined size for each detected face.

  FIG. 3 is a diagram illustrating an example of face image data cutout processing. The face area 32 is determined from the image 31 corrected by the preprocessing unit 202, and the face image 33 normalized to a predetermined size is cut out. That is, the size of the face image 33 is constant regardless of the face. The cut face image 33 is stored in a RAM (Random Access Memory) 209 via a DMAC (Direct Memory Access Controller) 205. Hereinafter, the position of the feature point is defined as the coordinate of the feature point in the face image 33, and the coordinate here is expressed by a coordinate system (x coordinate, y coordinate) with the upper left corner of the face image 33 as the origin. And

  A CPU (Central Processing Unit) 207 executes main processing according to the present embodiment and controls the operation of the entire image recognition apparatus. A bridge 204 provides a bus bridge function between the image bus 210 and the CPU bus 206. Reference numeral 208 denotes a ROM that stores instructions that define the operation of the CPU 207. A RAM 209 is a work memory necessary for the operation of the CPU 207. The RAM 209 is configured by a memory having a relatively large capacity such as a DRAM (Dynamic RAM), and is connected to the CPU bus 206 via a memory controller (not shown). Devices on the image bus 210 and devices on the CPU bus 206 operate simultaneously.

  A user interface unit 211 provides a physical interface (switch, touch panel, etc.) for the user to input a processing mode. The CPU 207 executes recognition processing on the face image 33 stored in the RAM 209.

FIG. 8 is a flowchart showing an example of the operation procedure of the face recognition process of the present embodiment. The flowchart shows the operation of the CPU 207.
First, the CPU 207 determines a processing mode in step S800. The processing mode here is recognition processing or registration processing, and is determined based on setting information of the user interface unit 211. In the following description, it is assumed that the face image data is already stored in the RAM 209.

  First, the recognition process will be described. In step S801, the recognition operation mode is determined. The recognition operation modes here are a high-accuracy mode and a high-speed mode. In the high accuracy mode, face recognition is performed with priority given to recognition accuracy over processing speed. On the other hand, in the high-speed mode, a high-speed recognition process is executed while allowing a decrease in accuracy. For example, whether the high-accuracy mode or the high-speed mode is determined is determined based on whether or not a recognition target person (person registered as a recognition target) has been recognized in the previous frame. That is, the CPU 207 determines the recognition operation mode with reference to the recognition result of the previous frame stored in the RAM 209. Specifically, when a person registered as a recognition target is recognized in the previous frame, the current frame is processed in the high-speed mode.

  The CPU 207 selects and executes the corresponding feature point position detection process according to the determination result of step S801. If the high-speed mode is selected, a high-speed feature point position detection process is executed in step S802. If the high accuracy mode is selected, a highly accurate feature point position detection process is executed in step S803.

  Here, a case where the number of feature points detected in the high-speed mode and the high-accuracy mode is different will be described. FIG. 4 is a diagram for explaining an example of the position of the feature point to be detected. FIG. 4A shows the positions of feature points to be detected in the high accuracy mode, and 15 feature points 401 to 415 related to the facial organ are detected. On the other hand, FIG. 4B shows the positions of feature points detected in the high speed mode, and six feature point positions 417 to 421 are detected. When a large number of feature points are detected, the reliability of the position data obtained in step S804 for determining the calculation position of the feature amount described later is improved, and the influence of false detection is mitigated by using the position information of a plurality of feature points Is done.

  FIG. 5 is a diagram for explaining a specific example of steps S802 and S803 which are feature point position detection processing. In this embodiment, the feature point position is detected by CNN (Convolutional Neural Networks). FIG. 5 shows a configuration in the case of detecting two feature point positions for explanation.

  The CNN is configured by hierarchical feature extraction processing, and in the example illustrated in FIG. 5, an example of a two-layer CNN in which the number of features in the first layer 506 is 3 and the number of features in the second layer 510 is two. Reference numeral 501 denotes face image data, which corresponds to the face image 33 described above. Reference numerals 503 a to 503 c denote characteristic surfaces of the one layer 506. The feature plane is an image data plane that stores a result of calculation while scanning data of the previous layer with a predetermined feature extraction filter (cumulative sum of convolution calculations and nonlinear processing). Since the feature plane is a detection result for raster-scanned image data, the detection result is also represented by a plane. The feature planes 503a to 503c are calculated with reference to 501 using different feature extraction filters. The characteristic surfaces 503a to 503c are generated by calculation corresponding to the two-dimensional convolution filters 504a to 504c and non-linear conversion of the calculation results, respectively. Reference numeral 502 denotes a reference image area necessary for the convolution calculation. For example, a convolution filter operation with a filter size (length in the horizontal direction and height in the vertical direction) of 11 × 11 is processed by a product-sum operation as shown in Equation (1) below.

  Here, input (x, y) indicates a reference pixel value at coordinates (x, y), and output (x, y) indicates a calculation result at coordinates (x, y). weight (column, row) indicates a weight coefficient at coordinates (x + column, y + row), and columnSize = 11 and rowSize = 11 indicate a filter size (number of filter taps).

  The convolution filters 504a to 504c have different coefficients. The coefficient of the convolution filter is determined in advance by learning. Also, the size of the convolution filter varies depending on the feature plane. In the CNN operation, a product-sum operation is repeated while scanning a plurality of filter filters in units of pixels, and a final product-sum result is nonlinearly transformed to generate a feature plane. Note that a sigmoid function or the like is applied to the nonlinear transformation. When calculating the feature plane 503a, since the number of connections with the previous layer is 1, one convolution filter 504a is used.

  On the other hand, when calculating the feature plane 507a and the feature plane 507b, since the number of connections with the previous layer is 3, the calculation results of the three convolution filters corresponding to 508a to 508c and 508d to 508e, respectively, are cumulatively added. That is, one feature value of the feature surface 507a is obtained by accumulating all the outputs of the convolution filter operations 508a to 508c and finally performing a nonlinear conversion process. Reference numerals 505 a to 505 c denote reference image areas necessary for the convolution filter calculation 508. CNN calculation is known as a powerful feature extraction technique, but is a technique that requires a large processing load and requires a high processing load.

  In step S802 or step S803, the center of gravity of the feature surfaces 507a and 507b, which is the CNN calculation result, is used as the feature point position coordinates. In actual processing, the calculation is performed on a limited region in consideration of the possibility of the existence of feature points to be detected in the image. Reference numerals 509a and 509b denote two-layer feature plane regions that are actually calculated. The center of gravity of the result obtained for the limited region is used as the feature point position coordinate.

  In FIG. 5, the case where two features are extracted has been described for the sake of explanation. However, in step S803 in which high-precision feature point position detection processing is performed, a network configuration capable of detecting 15 feature point positions is processed. In this case, the number of features of the two layers is 15. On the other hand, in step S802 in which high-speed feature point position detection processing is performed, a network configuration capable of detecting six feature point positions is processed. In step S802 in which high-speed feature point position detection processing is performed, the amount of processing calculations can be greatly reduced by reducing the number of feature points to be detected.

  In step S804 in which the feature amount calculation position determination process is performed, a geometric correction process is executed on the result of step S802 or S803 to determine a position serving as a reference for calculating the feature amount. FIG. 6 is a diagram illustrating an example of the correction process. 402a is a feature point that characterizes the corner of the eye but is erroneously determined as the position of the end of the eyebrows. In step S804, the position is corrected by statistical processing based on the arrangement relationship of human face features. That is, 402a is corrected to the position 402b.

  FIG. 7 is a diagram for explaining the processing content of step S804. In step S700, the number of feature points is adjusted. Here, the number of different feature points is adjusted for each recognition operation mode (high-speed mode or high-accuracy mode). In the case of the high-speed mode, nine feature point positions that are not detected in the high-accuracy mode are replaced with predetermined average coordinate values. The average value here is a corresponding element of the average vector A in FIG. That is, the processes in steps S701 to S708 are performed in common regardless of the number of feature points.

In step S701, each feature point position coordinate is simply connected to generate one vector. In the present embodiment, a 30-dimensional feature vector V is generated from the position coordinates of 15 feature points. A data string obtained by simply connecting the position coordinate data (x _i , y _i ) of each feature point (where i is a feature point number 1 to 15) is a feature point vector V (element v _j : j = 1 to 30). . Feature point numbers 1 to 15 correspond to feature points 401 to 415 in this embodiment. Therefore, for example, elements v ₁ and v ₂ of the feature point vector correspond to the x coordinate value and the y coordinate value of the feature point 401, respectively. The feature vector V is defined by the following equation (2). Hereinafter, T indicates transposition, and f indicates the number of feature points.
V = (v ₁ , v ₂ , v ₃ ,..., V ₂ × _f ) ^T (2)

In step S702 for subtracting the average vector and in step S703 for performing the projection calculation process, the projection vector P is calculated using the average vector A707 and the projection matrix E708, respectively. The projection vector P is calculated by the following equations (3) to (5) using a vector obtained by subtracting the average vector A from the feature point vector V and the projection matrix E. Note that the projection matrix E and the average vector A are previously calculated by principal component analysis using feature point vectors (learning feature vectors) for a large number of face images. Therefore, the projection matrix E here is composed of eigenvectors. The learning feature vector is a vector generated by concatenating correct feature point position coordinates of a face image in the same manner.
P = E ^T (V−A) (3)
A = (a ₁ , a ₂ , a ₃ ,..., A ₂ × _f ) ^T (4)
E = (u ₁ , u ₂ ,..., U _p ) (5)

u ₁ , u ₂ ,..., u _p are 2 × f-dimensional orthonormal vectors (eigenvectors) obtained by principal component analysis. In the case of this embodiment, it is a 30-dimensional vector. p is the dimension of the projection vector, and is 8 in this embodiment. That is, a projection matrix E is a matrix in which eight vectors having large corresponding eigenvalues among orthogonal vectors obtained by principal component analysis are selected. Assume that the projection matrix E and the average vector A are stored in advance in the ROM 208, the RAM 209, or the like. In Steps S702 and S703, the 2 × f-dimensional feature point vector is reduced to a p-dimensional projection vector by the operations of Expressions (3) to (5). That is, it projects onto a p-dimensional subspace.

In steps S704 and S705 for restoring the dimensions, the original feature point vector dimension data (that is, the coordinate position) is restored from the projection vector P. The restoration vector V ′ is calculated by the equation (6) using the projection matrix E and the average vector A.
V ′ = EP + A (6)

  In step S706, corrected coordinate data is extracted from the back-projected restored vector V ′. Through the processing from step S701 to step 706 described above, a statistical outlier can be corrected by projecting a vector obtained by concatenating all the feature point position data onto a subspace having a reduced dimension and then performing reverse projection. That is, outliers (incorrect detection) that cannot be expressed in the projected subspace are corrected. As a result, the geometrical arrangement relation is corrected based on the arrangement relation of each feature point, and the erroneous detection as shown in FIG. 6 is corrected. This processing is also disclosed in Non-Patent Document 1 and the like.

  As described above, the difference in the content of the feature point position detection process is absorbed in step S804 in which the feature value calculation position is determined. In other words, a position reference that is common to subsequent processing is provided regardless of the number of input feature points.

  Next, in steps S805 to S807, face recognition processing is executed based on the position obtained in step S804. For recognition processing, various conventionally proposed methods may be applied. First, in step S805, a plurality of local regions are cut out based on the processing result in step S804, dimensionally compressed by orthogonal transformation or the like, and the dimensionally compressed data is used as a feature amount. For example, a plurality of feature amounts are calculated from a local region centered on the corners of the eyes and the eyes.

  Next, in step S806, a plurality of registration data is read from the RAM 209, and the similarity to the registrant is calculated by a correlation operation between the registration data and the feature amount obtained in step S805 in step S807. A plurality of feature amounts are calculated based on the position of the feature point. Then, a final similarity is calculated by integrating a plurality of correlation values obtained for each feature amount, and whether or not the image data to be processed is a registrant by performing threshold processing on the final similarity. judge.

  In step S807, the plurality of registration data (registration data group) read in step S806 is compared with the processing result in step S805. If the final similarity is the highest and is equal to or greater than a predetermined threshold, the face of the recognition target image Is determined to be the registrant. Therefore, if even one piece of registered data is mixed in the registered data group (for example, when the reliability of the feature point position is low), the overall recognition accuracy may be lowered.

  In step S808, the determination result is recorded in the RAM 209. In the case of the present embodiment, the number and type of local regions for calculating feature amounts are the same even when the recognition operation modes are different. That is, in steps S805 to S807, recognition is always performed by the same process regardless of the recognition operation mode.

  Next, the registration process will be described. If registration processing is selected in step S800, the CPU 207 executes high-precision feature point position detection processing in step S809. This processing step is the same as step S803 in the recognition processing, and 15 feature point positions are detected from the face image. Hereinafter, step S810 and step S811 are the same as step S804 and step S805, respectively, during the recognition process.

  In step S812, the feature quantity obtained together with the personal information (name etc.) corresponding to the face image data to be processed is stored in the RAM 209 or the like. Alternatively, it is stored in a recording device such as a hard disk (not shown). As described above, since there are few processing time restrictions during registration processing (there is often less necessity for real-time processing), a large number of feature point positions are always detected by highly accurate feature point position detection processing. As described above, by detecting a large number of feature points, the reliability of the position data obtained in step S804 is improved, and accordingly, the reliability of the registered data is also improved.

  In the present embodiment, feature point positions are detected by feature point position detection processing with different processing loads depending on the recognition operation mode, and feature amounts calculated based on a common reference and high-precision feature point positions regardless of the recognition operation mode. A recognition process is executed using the generated registration data. That is, the difference in the feature point position detection process is absorbed by the feature amount calculation position determination process, so that registration data is always generated with high accuracy.

  As described above, according to the present embodiment, the feature point position is always detected with high accuracy at the time of registration, so that a reduction in recognition performance when the recognition process is accelerated by a simple feature point position detection process is reduced. The

(Second Embodiment)
FIG. 9 is a flowchart for explaining the operation of the present embodiment. This embodiment also operates with the apparatus shown in FIG. 2, and the flowchart shown in FIG. 9 shows the operation of the CPU 207. In the following description, it is assumed that face image data is already stored in the RAM 209.

  In this embodiment, the feature point position detection method differs according to the processing mode, and the CPU 207 determines the processing mode in step S900. Then, during the recognition process, a high-speed feature point position detection process is executed in step S901. That is, the recognition process is always operated at high speed. Note that the content of step S901 is the same as that of step S802 in FIG. Here, the positions of six feature points are detected.

  Next, in step S902, a feature amount calculation position is determined according to the processing result of step S901. Thereafter, a feature amount calculation process (step S903), a registered data read process (step S904), a determination process (step S905), and a recognition result recording process (step S906) are sequentially executed. Steps S902 to S906 here are the same as steps S804 to S808, respectively. That is, 15 feature point positions are determined from the detected 6 feature points in step S902, and recognition processing is executed based on the positions. That is, the difference in the number of feature points detected in step S902 is absorbed.

  On the other hand, at the time of registration processing, 15 feature point positions are detected in step S907, and thereafter, feature amount calculation processing (step S908), registration data reading processing (step S909), and registration data generation processing (step S910) are performed. Run. Steps S908 to S910 here are the same as steps S810 to S812, respectively.

  As described above, according to the present embodiment, feature points are detected by the feature point position determination process having different processing loads depending on the processing mode, and the recognition process is executed using the common feature amount / registered data. At the time of registration, the recognition performance is improved by detecting the feature point position with higher accuracy.

(Third embodiment)
FIG. 10 is a flowchart for explaining the operation of the present embodiment. This embodiment operates with the image recognition apparatus shown in FIG. 11, and the flowchart shown in FIG. 10 shows the operation of the CPU 207. In the image recognition apparatus shown in FIG. 11, a network interface 212 is added to the image recognition apparatus shown in FIG. Further, it is connected to a server device 213 on the network via the network. 11 are the same as those in FIG.

  In the present embodiment, the recognition operation is processed by the image recognition device, and the registration data generation is processed by the server device 213. In the following description, it is assumed that the face image data is already stored in the RAM 209.

  The operation of this embodiment will be described below with reference to FIG. In this embodiment, the feature point position detection method differs depending on the processing mode. First, the CPU 207 determines a processing mode in step S1000. At the time of recognition processing, high-speed feature point position detection processing is executed in step S1001. The contents of this processing step are the same as those in step S802 in FIG. 9, and the positions of six feature points are detected.

  Next, in step S1002, the feature amount calculation position is determined according to the processing result of step S1001. Thereafter, a feature amount calculation process (step S1003), a registered data read process (step S1004), a determination process (step S1105), and a recognition result recording process (step S1006) are executed. Steps S1002 to S1006 here are the same as steps S804 to S808, respectively. That is, 15 feature point positions are determined in step S1002 from the detected 6 feature points, and recognition processing is executed based on the positions. That is, the difference in the number of feature points detected in step S1002 is absorbed.

  On the other hand, at the time of registration processing, face image data and information for identifying an individual with respect to the face image are transferred to the server device 213 via the network interface 212 in step S1007. When the server apparatus 213 receives the face image data in step S1008, the server apparatus 213 starts a highly accurate feature point position detection process for the data in step S1009. Here, detection processing is executed for 15 feature points as in step S809.

  Next, in step S1010, a reference position determination process for calculating a feature amount is executed from the feature point position obtained in step S1009. The processing contents of steps S1010 to S1011 are the same as the processing contents of steps S810 to S811. In step S1012, the obtained feature amount is transmitted as registration data to the image recognition apparatus. The image recognition apparatus receives the registration data generated by the server apparatus 213 via the network interface 212 in step S1013. In step S1014, it is stored in the RAM 209 as registered data. At the time of recognition processing, determination processing (step S1005) is executed using the registration data stored here.

  In the present embodiment, feature point positions are detected by feature point position detection processing with different processing loads depending on the processing mode. At that time, the server apparatus 213 performs feature point position detection processing with a high processing load. As described above, according to the present embodiment, the recognition performance is improved by detecting the feature point position with higher accuracy during registration. In this case, the processing load during registration can be prevented from increasing.

(Fourth embodiment)
FIG. 12 is a flowchart for explaining the operation of the present embodiment. In this embodiment, the image recognition apparatus shown in FIG. 11 operates. In the present embodiment, only differences from the third embodiment will be described. The recognition processing operation is the same as in the third embodiment. That is, steps S1201 to S1206 are the same as steps S1001 to S1006.

  In the registration process, the face image data is transferred to the server device 213 via the network interface 212 in step S1207. When the server apparatus 213 receives the face image data in step S1208, the server apparatus 213 starts a highly accurate feature point position detection process for the data in step S1209. Here, detection processing is executed for 15 feature points as in step S809. In step S1210, the detected 15 feature point data are transmitted to the image recognition apparatus via the network.

  In step S1211, the image recognition apparatus is detected by the server apparatus 213 via the network interface 212 and receives the feature point position. In step S1212, a position determination process for calculating a feature amount is executed from the feature point position obtained in step S1209. Steps S1212 to S1213 are the same as steps S810 to S811. In step S1204, the obtained feature amount is stored in the RAM 209 as registered data. At the time of recognition processing, determination processing (step S1205) is executed using the registration data stored here.

  In the present embodiment, feature point positions are detected by feature point position detection processing with different processing loads depending on the processing mode. At that time, the server apparatus 213 performs feature point position detection processing with a high processing load. Furthermore, in the present embodiment, the processing load of the server device 213 is reduced by processing the feature amount calculation on the terminal device side. As described above, according to the present embodiment, the recognition performance is improved by detecting the feature point position with higher accuracy during registration. In this case, the processing load during registration can be prevented from increasing.

(Other embodiments)
In the first embodiment, the case where the recognition operation mode (high speed mode / high accuracy mode) is selected according to the recognition result of the previous frame has been described. However, the present invention is not limited to such a case. For example, the high accuracy mode and the high speed mode are switched depending on the number of faces in the target image 31. Various cases are conceivable, such as a user setting a high-precision mode and a high-speed mode via a predetermined user interface. For example, when setting via a user interface, step S801 determines the recognition operation mode based on information designated by the user.

  In the above-described embodiment, the case where a specific person is recognized from the face image in the image has been described, but the present invention is not limited to this. The present invention can be used for various image recognition apparatuses that identify a predetermined object based on the position of a feature point. In the above-described embodiment, the case where the number of feature points detected by the feature point position detection process is different has been described, but the detection method may be different. Furthermore, in the above-described embodiment, the case where the processing related to the present invention is realized by software has been described, but it is also possible to configure with dedicated hardware.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

100 face image data acquisition processing 102 first feature point position detection processing 103 second feature point position detection processing 104 selection processing 105 feature amount calculation position determination processing 106 feature amount calculation processing 111 processing mode determination processing

Claims (5)

An image data acquisition process for acquiring image data;
A first feature point position detecting step for detecting a position of a feature point from the acquired image data;
A second feature point position detecting step for detecting positions of more feature points than the first feature point position detecting step from the acquired image data;
A selection step of selecting processing by the first feature point position detection step or the second feature point position detection step;
A feature amount calculation position determining step for determining a calculation position of the feature amount based on the selected result;
A feature amount calculation step for calculating a feature amount based on the result of the feature amount calculation position determination step;
A determination step of determining a processing mode ;
When the processing mode determined by the determination step is a registration process, the processing by the second feature point position detection step is selected, and the result of the second feature point position detection step and the feature amount calculation position determination step A registration step of generating registration data from the feature amount calculated based on the result of
When the processing mode determined by the determination step is recognition processing, the processing by the first feature point position detection step is selected, and the result of the first feature point position detection step and the feature amount calculation position determination step and a recognizing step the image data on the basis of the calculated feature amount based on the result and the said registration data,
When the processing by the first feature point position detection step is selected in the selection step, the feature amount calculation position determination step includes the feature point detected in the first feature point position detection step, Based on a feature point obtained by substituting a predetermined coordinate value for the position of a feature point that is detected in the second feature point position detection step but not detected in the first feature point position detection step An image recognition method characterized by determining a calculation position of a quantity . The image recognition method according to claim 1, wherein the predetermined coordinate value is an average coordinate value corresponding to an average vector calculated by principal component analysis. 3. The feature amount calculation position determination step according to claim 1, wherein a geometric correction process is performed on the position of the feature point to determine the calculation position of the feature amount. Image recognition method. Image data acquisition means for acquiring image data;
First feature point position detecting means for detecting the position of the feature point from the acquired image data;
Second feature point position detecting means for detecting positions of more feature points than the first feature point position detecting means from the acquired image data;
Selecting means for selecting processing by the first feature point position detecting means or the second feature point position detecting means;
Feature amount calculation position determining means for determining the calculation position of the feature amount based on the selected result;
Feature quantity calculation means for calculating a feature quantity based on a result of processing by the feature quantity calculation position determination means;
Determining means for determining a processing mode;
When the processing mode determined by the determination unit is a registration process, the process by the second feature point position detection unit is selected, the result of the process by the second feature point position detection unit, and the feature amount calculation position Registration means for generating registration data from the feature amount calculated based on the result of processing by the determination means ;
When the processing mode determined by the determination unit is a recognition process, a process by the first feature point position detection unit is selected, and a result of the process by the first feature point position detection unit and the feature amount calculation position determination Recognizing means for recognizing image data based on the feature amount calculated based on the result of processing by the means and the registered data;
When the processing by the first feature point position detection unit is selected by the selection unit, the feature amount calculation position determination unit includes the feature point detected by the first feature point position detection unit, Based on a feature point obtained by substituting a predetermined coordinate value for the position of a feature point that is detected in the process by the second feature point position detection unit but not detected in the process by the first feature point position detection unit. An image recognition apparatus characterized by determining a calculation position of a feature amount . The program for making a computer perform each process of the image recognition method of any one of Claims 1-3.

Images