JP4427714B2 - Image recognition apparatus, image recognition processing method, and image recognition program - Google Patents

Description

  The present invention relates to an image recognition apparatus, an image recognition processing method, and an image recognition program for capturing, for example, a logo mark posted on a person or a store and identifying an individual or a corporation from the target image.

  Face recognition technology that identifies a person by identifying who is the face in the image captured by the camera is used in the security field to identify and authenticate an individual, for example, imitating an animal such as a dog It is also used for the human interface of pet robots with external shapes.

  As a face recognition technique used for a human interface, for example, a support vector machine (SVM) is used to detect a face by patterning a shading pattern corresponding to a human face from a captured image, and an input face obtained thereby A difference value between the image data and registered face image data for each person registered in the database in advance is calculated, and a person associated with the registered face image data having the smallest difference value is captured based on the calculation result. A technique for identifying a person appearing inside is known, and details thereof are disclosed in Patent Document 1, for example.

Japanese Patent Publication No. 2002-157596

  By the way, in the conventional face recognition technology described above, if the number of reference image samples registered in advance in the database for storing and managing registered face image data increases, the number of similar faces inevitably increases, and thus erroneous recognition is likely to occur. There is a problem that invites a decrease in recognition rate.

  Therefore, the present invention has been made in view of such circumstances, and an image recognition apparatus, an image recognition processing method, and image recognition that can improve the recognition rate even when the number of reference image samples registered in the database is large. The purpose is to provide a program.

In order to achieve the above object, the invention according to claim 1 depends on the familiarity, an imaging means for photographing a target image to be recognized, an estimation means for estimating a familiarity with a user for the target image, and the familiarity. A plurality of attribute-specific image database means for storing a reference image classified into a plurality of attributes, each of which is classified into each attribute, and a recognition ID uniquely assigned to each reference image; A selection means for selecting an attribute-specific image database means corresponding to the familiarity estimated by the estimation means from among the attribute-specific image database means, and an attribute-specific image database means selected by the selection means. reference image with reference to the to and a target specifying means for specifying a recognition ID corresponding to the target image captured by the imaging means, said estimating means, before The target image based on the occupation area ratio of the target image with respect to the entire region imaged by the imaging unit, the number of times the target image is specified by the target specifying unit in the past, or the combination of the occupation area ratio and the number of times The intimacy with the user is estimated .

Invention according to claim 2, in addition to claim 1, further comprising a sundial number means for counting the current time, the attribute-based image database means includes a degree of intimacy with the user with respect to the target image, the The selection unit is categorized into attributes depending on the time taken by the imaging unit, and the selection unit is configured to select the plurality based on the familiarity estimated by the estimation unit and the current date and time counted by the date and time counting unit. The attribute-specific image database means corresponding to the user's intimacy with the target image currently captured by the imaging means and the current date and time is selected from the attribute-specific image database means .

Invention of claim 3, in addition to claim 1, further comprising a positioning means for positioning a current position, the attribute-based image database means includes a degree of intimacy with the user with respect to the target image, the image pickup means are classified by attributes that depend on the position to be imaged by said selection means, said a parent density estimated by the estimating means, on the basis of the positioning has been in the current position by the positioning means, by said plurality of attributes An attribute-specific image database means corresponding to the user's intimacy with the target image currently captured by the imaging means and the current position is selected from the image database means .

Invention of claim 4, in addition to claim 1, a sundial number means for counting the current time and date, further comprising a positioning means for positioning a current position, the attribute-based image database means, wherein and closeness of the user with respect to the target image, the time to be captured by the imaging unit, and is classified by the attributes depending on the position, the selection means includes a closeness estimated by said estimating means, said sundial number Based on the current date and time counted by the means and the current position measured by the positioning means, the user for the target image currently picked up by the image pickup means is selected from the plurality of attribute-specific image database means. The attribute-specific image database means corresponding to the intimacy, the current date and time, and the current position is selected .

The invention according to claims 5 and 9 includes an imaging step of capturing a target image to be recognized, an estimation step of estimating a closeness of the user with respect to the target image, and a plurality of attributes depending on the closeness. are categorized, and the reference images classified into each attribute, each corresponding uniquely granted recognition ID and from the plurality of demographic image database for storing the respective reference images, now, the process of the estimating step A selection process step of selecting an attribute-specific image database corresponding to the intimacy with the user estimated by the above, and referring to a reference image stored in the attribute-specific image database selected by the process of the selection process step, ; and a target specifying process step of identifying the recognition ID corresponding to the target image captured by the process of the imaging step, the process of the estimation step, Based on the occupation area ratio of the target image with respect to the entire region imaged by the processing of the imaging step, the number of times the target image has been identified in the past by the processing of the target identification step, or a combination of the occupation area ratio and the number of times Then, the closeness with the user for the target image is estimated .

In addition to the fifth and ninth aspects, the invention described in claims 6 and 10 further includes a date and time counting processing step for counting the current date and time, and the attribute-specific image database is a parent to the user for the target image. and density, are classified to the attributes that depend on the time taken by the processing of the image pickup step, said processing of the selected process step, the parent density estimated by the processing of the estimation processing step, the sundial number of processing steps based on the current date and time counted by the processing, from among the plurality of attribute-based image database, now the closeness of the user with respect to the captured target image by the processing of the imaging step, on the current date and time A corresponding attribute-specific image database is selected .

In addition to the fifth and ninth aspects, the invention according to the seventh and eleventh aspects further includes a positioning processing step for positioning a current position, and the attribute-specific image database includes a closeness with the user for the target image. are classified by attribute that depends on the position taken by the processing of the image pickup step, said processing of the selected process step, the parent density estimated by the processing of the estimation process step, positioning the processing of the positioning processing step based on the has been the current position, from the plurality of demographic image database, now and familiarity for the captured target image by the processing of the image pickup step, the attribute-based image database corresponding to the current position It is characterized by selecting .

In addition to claims 5 and 9 , the invention described in claims 8 and 12 further comprises a date and time counting processing step for counting the current date and time, and a positioning processing step for positioning the current position, and each attribute is classified. image database, and closeness to the user with respect to the target image, the time taken by the processing of the image pickup step, and is classified by the attributes depending on the position, the processing of the selected processing steps of the estimation process step Based on the familiarity estimated by the process , the current date and time counted by the process of the date and time counting process step , and the current position measured by the positioning process step, the plurality of attribute-specific image databases from now, the closeness to the target image captured by the process of the imaging step, and the current date, and the current position And selects the corresponding attribute-based image database.

  According to the present invention, for example, when a logo mark or the like posted on a person or a store is imaged and an individual or a corporation is identified from the target image, the date and place where the recognition is performed, the familiarity with the target image An image database with an attribute corresponding to the situation at the moment is automatically selected, and a reference image stored in the image database with the selected attribute is referenced to identify an individual or a corporation from the captured image. As a result, even if the total number of reference images registered in the image database increases, the image database is segmented by attribute, and the number of reference images registered in the image database corresponding to one attribute is determined. The image database with the most appropriate attributes is automatically selected according to the situation such as date, place, familiarity, etc. Without diagrammatically perform selection operation, with the number of the reference image is narrowed, since the reference image is narrowed by the additional information that situation, it is possible to improve the recognition rate.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

(1) Outline FIG. 1A is an external view showing an external appearance of a sewing packet 100 according to an embodiment of the present invention. The stitch wrap 100 is a character doll that imitates a cat, and a face recognition device 20 (described later) is built therein. The face recognition device 20 operates as a “shouldering mode” in a state in which the stitching 100 is placed on the user's shoulder (see FIG. 5B), while the user's face recognition device 20 operates as shown in FIG. It operates as “knee-mounting mode” while it is placed on the knee or desk. These modes are automatically switched by a mode switch 9 described later.

  In “shoulder-mounting mode”, the person in the captured image is identified by identifying who the face is, and the name of the person is taught to the user in a cat (scream) or captured in the captured image. Processing for newly registering a face in the database and sequentially recording captured images as images to be displayed in an album browsing process described later is executed.

  On the other hand, in the “knee-mounting mode”, as shown in FIG. 1C, the cable CA of the external monitor M is connected to the video output port 10 (described later) provided in the tail portion of the sewing wrap 100. A process of browsing a series of face images taken under the “shouldering mode” as an album on the external monitor M is executed. The processing operation in each mode will be described in detail later.

(2) Configuration of Face Recognition Device 20 Next, the configuration of the face recognition device 20 will be described with reference to FIG. In FIG. 2, the CPU 1 executes a control program (BIOS) stored in the ROM 2 to establish an input / output interface for each part of the device, and then loads an OS program stored in the HDD 4 (hard disk device) into the RAM 3 and starts it. Let After the OS (operating system) program is started, the CPU 1 reads an application program instructed to be executed by a user operation from the HDD 4, loads it into the RAM 3, and executes it.

  The application program referred to here includes a main routine including “database selection processing”, “person registration processing”, “name notification processing”, and the like, which will be described later.

  The RAM 3 includes a program area for storing various program data, a work area for temporarily storing the calculation result of the CPU 1, and an image data area for temporarily storing an image captured by the CCD camera 7. The HDD 4 stores various programs and various databases, table data, and audio files.

  The various databases stored in the HDD 4 indicate a plurality of image database IDBs and closeness databases FDB that are referred to during face recognition. The plurality of image database IDBs are databases that are provided for each of a plurality of attributes corresponding to a situation where face recognition is performed, and store a reference face image of a person included in each attribute.

  Specifically, as in the example illustrated in FIG. 3, for example, an image database IDB1 in which reference face images of persons belonging to a company where the user works (such as a boss, a colleague, and a subordinate) are registered, or reference faces of friends / acquaintances This is a database in which a person's public / private human relationship is classified by attribute and registered as a reference face image of a person, such as an image database IDB2 in which images are registered.

  These image database IDBs are automatically selected according to the situation where face recognition is performed, as will be described later. Each image database IDB stores and manages a plurality of records including at least a recognition ID and a reference face image data of a person associated therewith. In order to avoid a decrease in the recognition rate, it is preferable to register reference face image data of about 10 people per image database.

  The intimacy database FDB is a relational database linked to each image database IDB, and the intimacy for each person's recognition ID registered in each image database IDB, the target for all areas imaged by the CCD camera 7 described later The occupation area ratio of the image and the number of times of recognition identified as the same target image in the past are stored and managed, and an example is shown in FIG.

  The familiarity referred to here is a value determined according to the size of the face area of the person whose face is recognized and the number of times of recognition. For example, as shown in FIG. 11 (a), if the recognized face area A1 is large and the number of times of recognition as the same person is large, the closeness is high because it is intimately related to the user. As shown in (b), if the face area A2 is small and the number of times of recognition as the same person is small, it is defined that the intimacy is low because it is not so close to the user.

  Here, the familiarity defined by the face recognition device 20 will be described more specifically. The face area A1 shown in FIG. 11A is an area calculated when a face is detected based on a luminance pattern described later, and substantially includes eyes, nose, and mouth in the face necessary for specifying an individual. Means a rectangular area. The large face area A1 means that the ratio of the face area A1 (target image) to the entire imaged area, that is, the occupied area ratio is large. In this case, the distance from the target person Is close, and it is estimated that the intimacy is higher.

  On the other hand, as shown in FIG. 11B, the small face area A2 means that the ratio of the face area A2 (target image) to the entire imaged area, that is, the occupied area ratio is small. In this case, it is estimated that the distance to the target person is far and the intimacy is lower.

Based on the above estimation, the familiarity R is calculated based on the following calculation formula (1), for example.
R = a × (N / Nmax) + (1−a) × (N / Nfull) (1)

  Here, a is a weighting constant arbitrarily set within the range of 0.0 to 1.0, N is the number of recognitions recognized as the same person in the past with respect to the currently recognized face area A1, and Nmax is stored in the familiarity database FDB. The maximum number of times of recognition (in the example shown in FIG. 4, 10 times), N is the area of the currently recognized face area A1, and Nfull is the area of the entire area to be imaged. The intimacy R shown in FIG. 4 is calculated by the calculation formula (1) described above, and the intimacy database FDB is updated each time a new intimacy R is calculated. Note that the face area A1 is not a rectangular area that substantially includes the eyes, nose, and mouth in the face, but parameters that change according to the distance from the target person, such as the area of the rectangular area that the face inscribes. Of course, you can use it.

  The table data stored in the HDD 4 refers to the database selection table DST and the name notification table NIT. The database selection table DST is table data that specifies which of the plurality of image database IDBs described above is selected according to the current date and time and the current position of the user.

  In this database selection table DST, the user can arbitrarily register a specified value in correspondence with the current date and time and the current position. For example, when the date and time is a weekday day and the current position is a company Is registered with a designated value for selecting the above-mentioned image database IDB1, and when the date and time is Saturday or Sunday and the current position is not specified, the designated value for selecting the above-mentioned image database IDB2 is registered.

  The name notification table NIT is table data in which a recognition ID of a person identified by face recognition is associated with an audio file, and is used when selecting an audio file corresponding to the identified ID of the person. It is done.

  The sound system 5 reads PCM waveform data from the audio file that the CPU 1 instructs to reproduce, D / A converts it, and outputs the audio. The mouse 6 generates a pointing signal or a switch event corresponding to a user operation, and is provided in the right hand portion RH of the sewing package 100 (see FIG. 1). The CCD camera 7 is provided in the left eye part LE of the stitching 100 and takes an image under the control of the CPU 1 to generate image data. The position detection unit 8 receives a GPS (Global Positioning System) signal under the control of the CPU 1, measures the current position, and generates position result position data.

  The mode changeover switch 9 is provided at the waist of the sewn wrap 100, and generates a mode changeover event corresponding to the bending and stretching of the waist. That is, as shown in FIG. 1B, when the stitching 100 is placed on the shoulder of the user, a switch event indicating “shouldering mode” is generated, and as shown in FIG. When it is placed on the user's knee or desk, a switch event indicating the “knee-mounting mode” is generated. A video output port (VGA connector) 10 is provided at the tail of the stitching 100 and outputs a display control signal.

  The constituent elements other than the mouse 6, the CCD camera 7, the mode switch 9 and the video output port 9 are built in the body portion of the sewn wrap 100 as the apparatus body.

(3) Details of Face Recognition Algorithm Details of the face recognition algorithm used in the above-described face recognition device 20 are disclosed in Patent Document 1 (Patent Publication 2002-157596 (patent publication No. 2002-157596) previously proposed by the present applicant). 2003/0059092)).

  That is, in the face recognition device 20, face recognition is realized by the following three techniques.

(I) Face detection from complex scenes (ii) Real-time tracking of faces (iii) Face identification

  Face detection methods can be broadly classified into those that use colors, movements, and patterns to identify objects, but using the face pattern is the most powerful method for accurately extracting faces from complex scenes. is there. However, searching for faces of all scales throughout the scene is very heavy and so far this technique has been used only for still images.

  On the other hand, most systems that detect faces in real time detect skin color. However, the color changes depending on the illumination conditions, and the skin color also has races and individual differences. Therefore, simple skin color recognition alone cannot be an effective means.

  Therefore, a method is adopted in which real-time tracking of the face is performed based on the color distribution included in the detected face pattern, and face detection is adapted to the dynamic change. In addition, the face pattern is searched only for the face area obtained from the estimated color distribution. This shortens the calculation time in face detection.

  Furthermore, the face is identified using a face image cut out by pattern search. Then, while tracking is successful, it is possible to make a comprehensive judgment from a plurality of identification results by treating it as the identification result of the same face.

  For example, the processing for face identification is as follows: (i) face detection from a complex scene is performed by face detection (face recognition) using a luminance pattern, and (ii) real-time tracking of a face is performed using face detection by color. Tracking (face tracking) is performed, and (iii) face identification is performed by person identification using a differential face.

  For example, each process in the face recognition apparatus 20 is realized as a module or an object. That is, the face recognition device 20 includes a face tracking module, a face detection module, and a face identification module. Here, the face tracking module functions as a face tracking unit that tracks a face that changes in an image captured by the CCD camera 7, and the face detection module is based on the face tracking information by the face tracking module. The face identification module functions as face data detection means for detecting face data of a face in the image captured by the CCD camera 7, and the face identification module identifies a specific face based on the face data detected by the face detection module. It functions as a face identification means.

  Here, in the face detection based on the luminance pattern, processing for detecting (recognizing) the face from the input image is performed. Specifically, in this face detection, a face or non-face is identified by a support vector machine (SVM). This processing is usually characterized by being resistant to environmental changes, requiring a large amount of calculation, and being vulnerable to posture changes. Here, as an environmental change, the change of ambient illumination is mentioned, for example.

  In the face tracking by color, a process for tracking the face in the input image is performed. Specifically, in this face tracking, estimation of the face color distribution and estimation of the face area are performed. This processing is usually characterized by being weak against environmental changes, having a small amount of calculation, and being strong against posture changes.

  In the person identification, the face recognized by the above-described face detection is identified as a specific face. Specifically, in this person identification, positioning (morphing) is performed from the position identification of eyes and nose, and the same person is determined from the difference face.

  In the face identification system, the processing described above is appropriately shared as each step in face identification, and a mutually complementary relationship is achieved, thereby enabling face detection with high accuracy. For example, each process complements each other as follows.

  For example, the tracking of a face by color is weak against environmental changes, but the detection of a face by a luminance pattern is complemented by utilizing the fact that it is strong to the environment. On the other hand, the detection of a face using a luminance pattern requires a large amount of calculation and is vulnerable to changes in posture. However, the use of the fact that tracking of a face by color has a small amount of calculation and is strong against posture change is supplemented.

  In summary, the following can be said. It is difficult to detect in real time a face that is originally a process with a large amount of calculation. However, if it is performed for a certain period at a predetermined timing, the burden of calculation amount is reduced. On the other hand, if it is detected every time from the input image to the face position at each timing, the burden is large.

  Therefore, using a process that requires a small amount of computation and is resistant to posture changes, the face change in the input image is tracked in real time, and the face is detected only for the estimated face position in the input image obtained from the tracking result. If the detection process is performed, the face can be detected with the face position specified. In other words, by combining rough but fast processing with high reliability but slow processing and sharing roles, the entire system is complemented between each processing, so that in real time in cooperation Face detection is possible.

  As a result, many face detection results can be acquired in a short time, face identification is performed based on the acquired face detection results, and such processing is processed statistically, thereby enabling highly accurate face identification. I have to.

  The face recognition device 20 uses such a face identification system to find a person in the scene (face detection process), gaze at it (face tracking process), and identify the face using the information obtained therefrom. All processes are automatically performed until the person is identified (face identification process) by, thereby realizing highly reliable face identification.

(4) Operation of Face Recognition Device 20 Next, the operation of the face recognition device 20 configured as described above will be described with reference to FIGS. In the following, the operation of the main routine will be described first, and then the operations of database selection processing, person registration processing, and name notification processing that constitute the main routine will be described.

[1] Operation of Main Routine When the user turns on the apparatus power and causes the main routine shown in FIG. 5 to be executed, the face recognition apparatus 20 proceeds to step SA1 and determines whether or not it is under “shouldering mode”. Determine whether. Here, as illustrated in FIG. 1B, when the stitching 100 is placed on the shoulder of the user, the mode change switch 9 generates a switch event indicating “shouldering mode”. "YES" is determined, and the process proceeds to Step SA2.

  In step SA2, the CCD camera 7 is instructed to execute imaging, and in the subsequent step SA3, a gray pattern corresponding to a human face is identified from the captured image to detect the face. Next, in step SA4, more specifically, whether or not a face image capturing condition (person photographing condition) is satisfied, that is, whether or not the face area detected from the previously captured image exceeds a predetermined size is determined. Determines whether the ratio of the occupied area calculated as the ratio of the face area A1 (target image) to the entire imaged area is larger than a predetermined value.

  If the face area detected from the captured image does not exceed a predetermined size, the determination result is “NO” because the imaging condition is not satisfied, and the process returns to step SA2. Thereafter, imaging and face detection are repeated as needed until the face area detected from the captured image exceeds a predetermined size.

  If the face area detected from the captured image exceeds a predetermined size and satisfies the image capturing condition, the determination result in step SA4 is “YES”, and the CCD camera 7 is imaged to acquire the face image. Instruct. The face image data thus obtained is temporarily stored in the image data area of the RAM 3 once.

  In step SA4, whether or not the face area detected from the captured image exceeds a predetermined size is set as an imaging condition. However, the present invention is not limited to this. It does not matter as a mode for capturing a human face image.

  Now, when the face image data is acquired in this way, the face recognition device 20 advances the process to step SA6 and executes the database selection process. In the database selection process, face recognition is performed from among a plurality of image databases stored in the HDD 4, that is, from the plurality of image databases in which the human relations between the user's public and private are divided according to attributes and the face images of persons included in each attribute are respectively registered. Select an image database with attributes that best suit the scene you want to play. Specifically, the corresponding image database is selected from the database selection table DST described above according to the current date and time and the current position of the user.

  Next, in step SA7, the person registration process is executed when the user clicks the right button of the mouse 6 provided in the right hand portion RH of the stitching 100 to generate a registration instruction event. In this process, a recognition ID is assigned to the face image data imaged in step SA5 and newly registered in the image database selected in step SA6, or the voice corresponding to the new recognition ID in the name notification table NIT. Assign a file.

  In step SA8, a difference value between the registered face image data for each person registered in the image database selected in step SA6 and the face image data newly obtained by imaging is calculated. A face recognition process is performed for identifying an individual associated with the registered face image data having the smallest value as a person shown in the captured image.

  In step SA9, the contents of the familiarity database FDB described above are updated based on the face recognition result. That is, the familiarity corresponding to the recognition ID of the person identified by face recognition is updated according to the number of recognitions and the size of the face area. In step SA9, the face image data for which face recognition has been completed is read from the image data area of the RAM 3 and stored in the album folder of the HDD 4.

  Note that the manner of updating the intimacy is not limited to the process of step SA9, and the concept of time can be adopted. That is, in the familiarity database FDB shown in FIG. 4, the date and time of face recognition are also stored and managed as database items, the recognition interval is obtained from the date and time of previous face recognition and the date and time of current face recognition, and the obtained recognition interval. It is possible to increase the intimacy if the length is short, and decrease the intimacy if the length is long.

  Next, in step SA10, name notification processing is executed when the user clicks the left button of the mouse 6 provided in the right hand portion RH of the stitching 100 to generate a name notification instruction event. In this process, referring to the name notification table NIT, an audio file corresponding to the recognition ID of the person identified by face recognition is selected and reproduced.

  Thereafter, the processing is returned to the above-described step SA1, and thereafter, in the state where the user puts the stitching 100 on the shoulder, the operation in the shoulder mode composed of steps SA2 to SA10 is repeated.

  Then, as shown in FIG. 1C, the sewn packet 100 is connected to the video output port (VGA connector) 10 provided at the tail portion of the sewn packet 100 while being connected to the cable CA of the external monitor M. When the user is placed on the knee or placed on the desk, the mode change switch 9 generates a switch event indicating “knee-mounting mode”, so that the determination result in step SA2 is “NO”, the process proceeds to step SA11, and the album browsing process is performed. Execute.

  In the album browsing process, an album browsing window W shown in FIG. 6 is generated and displayed on the external monitor M. This album browsing window W is an index display (thumbnail display) of face image data stored in the album folder of the HDD 4, and a frame (screen frame) is added to the face image data selected from the index display. In addition, it has a function to display one screen.

  For example, when the date designation button 20 in the album browsing window W is clicked by a mouse operation, a list of shooting dates corresponding to all face image data stored in the album folder of the HDD 4 is displayed in the date list window 21.

  On the other hand, when the person designation button 22 in the album browsing window W is clicked by a mouse operation, the person recognition IDs (for example, 001, 002, 003...) Corresponding to all the face image data stored in the album folder of the HDD 4 are clicked. Etc.) is displayed in the person list window 23.

  When an arbitrary date displayed in the date list window 21 or a recognition ID of an arbitrary person displayed in the person list window 23 is specified by clicking with the mouse, the corresponding button is clicked with the mouse. The face image data corresponding to the date or person ID is read from the HDD 4 and displayed as a list in the thumbnail display area 25 as index images P1, P2, P3, P4.

  Further, when an arbitrary image is clicked and designated from among the index images P1, P2, P3, and P4 displayed as a list in the thumbnail display area 25, an image obtained by adding a frame to the selected face image data is selected. indicate.

  At that time, the closeness of the corresponding person is searched from the closeness database FDB based on the recognition ID of the face image data displayed on the screen, and a frame (screen frame) corresponding to the searched closeness is selected. For example, when face image data of a person with a high degree of closeness is displayed on the screen, a gorgeous frame is added as shown in FIG. 12A, while face image data of a person with a low degree of closeness is displayed on the screen. In such a case, as shown in FIG. 12B, entertainment such as providing a dark frame is provided.

[2] Operation of Database Selection Process Next, the operation of the database selection process will be described with reference to FIG. When this process is executed via step SA6 described above, the face recognition apparatus 20 advances the process to step SB1 shown in FIG. 7, and acquires the current date and time data from the operating OS program side. Next, the process proceeds to step SB2 to determine whether or not there is a plan corresponding to the current date and time. That is, it is determined whether or not a schedule corresponding to the current date and time is registered in a schedule book (schedule management software) operating as resident software.

  If a schedule corresponding to the current date and time is registered, the determination result is “YES”, the process proceeds to the next step SB3, and an image database corresponding to the database designated value set in the schedule book is selected. .

  On the other hand, if the schedule corresponding to the current date and time is not registered in the schedule book, the determination result in step SB2 is “NO”, and the flow advances to step SB4. In step SB4, the current location is determined from the GPS position information generated by the position detector 8. In a state where the GPS signal cannot be received and the current location cannot be specified, the current location is determined based on GPS position information at the time when the GPS signal is lost (for example, when entering the indoors).

  Subsequently, in step SB5, the corresponding image database is selected from the database selection table DST described above according to the current date and time and the current position of the user. Thereby, the image database having the attribute most suitable for the scene for face recognition is selected. Specifically, for example, if the date is a weekday and the place is a company, the image database IDB1 is selected, and if the date is Saturday or Sunday and the location is not specified, the image database IDB2 is selected. The

[3] Operation of Person Registration Process Next, the operation of the person registration process will be described with reference to FIG. When this process is executed via step SA7 described above, face recognition device 20 advances the process to step SC1 shown in FIG. 8, and determines whether there is a registration instruction event. If there is no registration instruction event, the determination result is “NO”, and the processing returns to the main routine (see FIG. 3) without performing any processing.

  On the other hand, when the user clicks the right button of the mouse 6 provided in the right hand portion RH of the sewing wrap 100 to generate a registration instruction event, the determination result is “YES”, and the process proceeds to the next step SC2. In step SC2, a new recognition ID is assigned to the face image data captured in step SA5 of the main routine, and the face image data is newly registered in the image database selected in the database selection process.

  Next, in step SC3, the process waits until a registration end instruction event occurs. When the user clicks again on the right button of the mouse 6 provided on the right hand portion RH of the sewn wrap 100 to generate a registration end instruction event, the determination result is “YES”, and the flow proceeds to step SC4.

  In step SC4, a new recognition ID is registered in association with an unused voice file in the name notification table NIT. In step SC5, an audio file associated with the new recognition ID is reproduced. As a result, the user is notified of the contents of the audio file that informs the name of the person newly registered in the image database (for example, the cat's cry of “Nyan”).

[4] Operation of Name Notification Process Next, the operation of the name notification process will be described with reference to FIG. When this process is executed via step SA10 described above, face recognition device 20 advances the process to step SD1 shown in FIG. 9, and determines whether there is a name notification instruction event. If there is no name notification instruction event, the determination result is “NO”, and no processing is performed, and the process returns to the main routine (see FIG. 5).

  On the other hand, when the user clicks the left button of the mouse 6 provided in the right hand portion RH of the sewing packet 100 to generate a name notification instruction event, the determination result is “YES”, the process proceeds to step SD2, and the name notification table NIT Referring to FIG. 4, an audio file corresponding to the recognition ID of the person identified by face recognition is selected and reproduced. As a result, the stitching 100 informs the name of the person identified by the face recognition in the cat language (scream).

  As described above, according to the present embodiment, a plurality of attributes corresponding to the situation in which face recognition is performed, in other words, the personal relationship between the user's public and private is divided according to attributes such as date / time or place, and the persons included in each attribute A plurality of image databases each of which are registered face images are selected, an image database having an attribute most suitable for the face recognition scene is selected from these image databases, and the selected image database is referred to in the captured image scene. A person is identified by identifying who the face is in.

  For this reason, even if there are many people registered in the database, the image database is subdivided by attribute, so the number of people registered per image database can be optimized, and it is most suitable for face recognition situations. As a result of selecting an image database of attributes and performing face recognition based on the selected image database, it is possible to improve the recognition rate.

(5) Modification In the embodiment described above, the face recognition device 20 is incorporated in the stitching 100 to identify the person who is the face in the captured image scene, and inform the name of the identified person. However, the gist of the present invention is not limited to such an embodiment, and various modifications are possible.

  For example, as shown in FIG. 10, the main body portion of the face recognition device 20 is accommodated in the shoulder back B in place of the sewing wrap 100, and the mouse 6 and the CCD camera 7 are arranged on the shoulder belt SB of the back. You can also.

  In the present embodiment, the image database most suitable for the face recognition scene is selected according to the date and time or the place. However, the present invention is not limited to this, and the above-described familiarity database FDB (see FIG. 4) is used. It is also possible to create an image database suitable for scenes that are used for face recognition.

  That is, based on the size of the face area obtained at the time of face detection, a recognition ID of a person having a closeness corresponding to the size of the face area is searched from the closeness database FDB, and the registered face corresponding to the searched recognition ID If image data is extracted from each image database to create a new image database and face recognition is performed using it, it is limited to persons with closeness corresponding to the size of the face area detected in the captured image. Therefore, the recognition rate can be improved.

  In the above-described embodiment, the case where an individual is specified from the face of a person has been described as an example. However, the present invention is not limited thereto, and for example, a logo mark posted at a store or the like is imaged. Applicable to image recognition devices that identify store names, corporations, etc. from logo mark images, images with attributes corresponding to the current situation, such as date and time of recognition, familiarity with the target logo mark image, etc. A database may be automatically selected, and a store name, a corporation, or the like may be specified from the captured logo mark image with reference to a reference image stored in the image database of the selected attribute.

  Further, the gist of the present invention can be applied not only to the above-described embodiment, but also to a mobile phone having an imaging function or a GPS position detection function or a mobile terminal having an imaging function, a GPS position detection function, and a wireless communication function. is there. In that case, since many mobile phones or mobile terminals do not have sufficient CPU processing capability, the image captured at the terminal side and the imaging position are sent to the server side that performs face recognition processing via the network. The recognized result may be returned to the terminal side.

It is a figure for demonstrating the external appearance and the operation mode of the sewing packet 100 which are one Embodiment by this invention. 2 is a block diagram illustrating a configuration of a face recognition device 20. FIG. It is a conceptual diagram which shows the concept of image database IDB1, IDB2. It is a conceptual diagram which shows the concept of the familiarity database FDB. It is a flowchart which shows operation | movement of a main routine. It is a figure which shows an example of the GUI screen displayed on a screen by an album browsing process. It is a flowchart which shows operation | movement of a database selection process. It is a flowchart which shows operation | movement of a person registration process. It is a flowchart which shows operation | movement of a name notification process. It is a figure which shows a modification. It is a figure for demonstrating the definition of intimacy. It is a figure which shows an example of the flame | frame added to the face image data displayed on a screen by an album browsing process.

Explanation of symbols

  1 CPU, 2 ROM, 3 RAM, 4 HDD, 5 Sound system, 6 Mouse, 7 CCD camera, 8 Position detection unit, 9 Mode switch, 10 Video output port, 20 Face recognition device, 100 Sewing

Claims (12)

Imaging means for imaging a target image to be recognized;
Estimating means for estimating a closeness with a user for the target image;
A plurality of attribute-specific image databases each storing a reference image classified into a plurality of attributes depending on the familiarity, and a recognition ID uniquely assigned to each reference image. Means,
A selection means for selecting an attribute-specific image database means corresponding to the intimacy estimated by the estimation means from the plurality of attribute-specific image database means;
With reference to a reference image stored in the attribute-specific image database means selected by the selection means, and a target specifying means for specifying a recognition ID corresponding to the target image imaged by the imaging means,
The estimation means is based on the occupation area ratio of the target image with respect to the entire region imaged by the imaging means, or the number of times the target image has been identified by the target identification means in the past, or a combination of the occupation area ratio and the number of times. An image recognition apparatus characterized by estimating a closeness with a user for the target image . It further comprises a date and time counting means for counting the current date and time,
The attribute-specific image database means is classified into attributes depending on a closeness with the user for the target image and a time when the image is picked up by the imaging means, and the selecting means is a closeness estimated by the estimating means. And, based on the current date and time counted by the date and time counting means, from among the plurality of attribute-specific image database means, the familiarity with the user for the target image currently captured by the imaging means, the image recognition apparatus according to claim 1, wherein the selecting the demographic image database means corresponding to the date and time. A positioning means for positioning the current position;
The attribute-specific image database unit is classified according to an attribute depending on a closeness with a user with respect to the target image and a position captured by the imaging unit, and the selection unit is determined by the closeness estimated by the estimation unit If, on the basis of the current position as the positioning by said positioning means, from among the plurality of attribute-based image database unit, now the closeness of the user with respect to the captured target image by the imaging means, the current the image recognition apparatus according to claim 1, wherein the selecting demographic image database means corresponding to the position. A date and time counting means for counting the current date and time;
A positioning means for positioning the current position;
The attribute-based image database means includes a degree of intimacy with the user with respect to the target image, are classified by the attributes that depend on the time and location to be imaged by said imaging means, said selection means is estimated by said estimating means Based on the familiarity, the current date and time counted by the date and time counting means , and the current position measured by the positioning means, the imaging means is currently selected from the plurality of attribute-specific image database means. image recognition apparatus familiarity and the current date and time, and claim 1, wherein the selecting demographic image database means corresponding to the current position of the user with respect to the captured target image by. An imaging step of imaging a target image to be recognized ;
An estimation step for estimating a closeness with the user for the target image;
A plurality of attribute-specific image databases each storing a reference image classified into a plurality of attributes depending on the familiarity, and a recognition ID uniquely assigned to each reference image. from among a selection processing step of selecting the current, the attribute-image database corresponding to closeness of the user estimated by the processing of said estimating step,
A target identification processing step of identifying a recognition ID corresponding to the target image captured in the processing of the imaging step with reference to a reference image stored in the attribute-specific image database selected by the processing of the selection processing step; Equipped ,
The process of the estimation step is the ratio of the occupied area of the target image to the entire region imaged by the process of the imaging step, or the number of times the target image is specified by the process of the target specifying step in the past, or the occupied area ratio And the intimacy with the user for the target image based on the combination of the number of times and the number of times
Image recognition processing method characterized by. A date and time counting step for counting the current date and time,
The attribute-specific image database is categorized into attributes that depend on a closeness with the user for the target image and a time imaged by the imaging step process, and the selection process step process includes the estimation process step . Based on the closeness estimated by the process and the current date and time counted by the process of the date and time counting process step , the image is currently captured by the process of the imaging step from the plurality of attribute-specific image databases. 6. The image recognition processing method according to claim 5 , wherein an attribute-specific image database corresponding to the closeness with the user for the target image and the current date and time is selected . It further includes a positioning processing step for positioning the current position,
The attribute-specific image database is classified according to an attribute depending on a closeness with a user with respect to the target image and a position imaged by the imaging step processing, and the selection processing step processing is performed by the estimation processing step . a closeness estimated by processing, on the basis of the the process the current position as the positioning by the positioning process step, from among the plurality of attribute-based image database, currently taken by the processing of the imaging step object 6. The image recognition processing method according to claim 5 , wherein an attribute-specific image database corresponding to the closeness to the image and the current position is selected . A date and time counting process step for counting the current date and time;
A positioning process step for positioning the current position;
The demographic image database, and closeness to the user with respect to the target image, the time taken by the processing of the image pickup step, and is classified by the attributes depends on the position, the processing of the selection processing step, the estimated Based on the closeness estimated by the processing of the processing step , the current date and time counted by the processing of the date and time counting processing step , and the current position measured by the positioning processing step, the plurality of attribute-specific images from among the database, the current, and familiarity for the subject image picked up by the processing in the imaging step, claim 5, characterized in that to select the current date and time, and the attribute-based image database corresponding to the current position The image recognition processing method described. An imaging step of imaging a target image to be recognized ;
An estimation step for estimating a closeness with the user for the target image;
A plurality of attribute-specific image databases each storing a reference image classified into a plurality of attributes depending on the familiarity, and a recognition ID uniquely assigned to each reference image. from among a selection processing step of selecting the current, the attribute-image database corresponding to closeness of the user estimated by the processing of said estimating step,
A target identification processing step of identifying a recognition ID corresponding to the target image captured in the processing of the imaging step with reference to a reference image stored in the attribute-specific image database selected by the processing of the selection processing step; Equipped ,
The process of the estimation step is the ratio of the occupied area of the target image to the entire region imaged by the process of the imaging step, or the number of times the target image is specified by the process of the target specifying step in the past, or the occupied area ratio And the intimacy with the user for the target image based on the combination of the number of times and the number of times
An image recognition program characterized by that . A date and time counting step for counting the current date and time,
The attribute-specific image database is categorized into attributes that depend on a closeness with the user for the target image and a time imaged by the imaging step process , and the selection process step process includes the estimation process step . Based on the closeness estimated by the process and the current date and time counted by the process of the date and time counting process step , the image is currently captured by the process of the imaging step from the plurality of attribute-specific image databases. The image recognition program according to claim 9 , wherein an attribute-specific image database corresponding to a user's familiarity with the target image and the current date and time is selected . It further includes a positioning processing step for positioning the current position,
The attribute-specific image database is classified according to an attribute depending on a closeness with a user with respect to the target image and a position imaged by the imaging step processing , and the selection processing step processing is performed by the estimation processing step . a closeness estimated by processing, on the basis of the the process the current position as the positioning by the positioning process step, from among the plurality of attribute-based image database, currently taken by the processing of the imaging step object The image recognition processing program according to claim 9 , wherein an image database classified by attribute corresponding to a user's familiarity with an image and a current position is selected . A date and time counting process step for counting the current date and time;
A positioning process step for positioning the current position;
The demographic image database, and closeness to the user with respect to the target image, the time taken by the processing of the image pickup step, and is classified by the attributes depends on the position, the processing of the selection processing step, the estimated The plurality of attributes based on the closeness estimated by the processing of the processing step , the current date and time counted by the processing of the date and time counting processing step , and the current position measured by the processing of the positioning processing step from among the different image database, now to the closeness to the target image captured by the process of the imaging step, and selects the current date and time, and the attribute-based image database corresponding to the current position according Item 10. The image recognition program according to Item 9 .

Images