JP4643872B2 - Image recognition method and apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image recognition method and apparatus for detecting an image such as a photograph belonging to a specific category, and a program for realizing them.
[0002]
[Prior art]
Japanese Patent Application Laid-Open No. 11-25271 discloses a method for extracting an in-image object from a photograph. In this method, an in-image object is obtained by gradually enlarging a gradation edge image using gradation continuity. If this method is used, an image similar to a specific image can be automatically searched. That is, an object having some meaning can be appropriately extracted from a photographic image or the like, and an image similar to a target specific image (key image) can be automatically searched from a large number of photographic images.
[0003]
[Problems to be solved by the invention]
However, the method disclosed in the above publication cannot accurately detect an image belonging to a specific category such as a cocoon image. That is, a category has an abstract and ambiguous conceptual spread, and the criteria for whether or not it belongs to a specific category may not always be clear, and the criteria may change with the times. Whether or not an image belongs to a specific category cannot be automatically and appropriately determined. In addition, when the target of the determination is an image of a living organism or the like, diversity exists unavoidably in the image, such as changes in posture, background object type / presence / absence, but a systematic category that can handle such diversity There is no judgment method yet.
[0004]
Accordingly, an object of the present invention is to provide an image recognition method and the like that can automatically extract objects belonging to a specific category.
[0005]
It is another object of the present invention to provide an image recognition method and the like that can automatically detect images belonging to a specific category.
[0006]
[Means for Solving the Problems]
In order to solve the above problems, an image recognition method of the present invention includes a plurality of stored reference patterns, It is a collection of various images From the learning image In a state where components that contribute to the determination of whether an image belongs to an abstract predetermined category or not The object pattern of the geometric shape automatically cut out and extracted Excluding the rotation angle element for the shape In comparison, for example, the relationship between the reference pattern and a specific object pattern Is a numerical value of Adding the specific object pattern to the contents of the plurality of reference patterns when the value exceeds a predetermined value, extracting the object pattern from the inspection image, extracting the object pattern from the inspection image, and the A step of comparing a reference pattern, and a relationship between the object pattern extracted from the inspection image and the reference pattern Is a numerical value of Is greater than or equal to There is something that An image recognition method comprising: determining that the inspection image is an image belonging to a predetermined category, wherein an object pattern having a predetermined shape out of the geometrically-shaped object pattern extracted by automatic clipping is provided. The object pattern is removed before the comparison with the reference pattern, and among the object patterns of the geometric shape extracted by automatic cutout, the remaining object patterns that are not removed are ranked, and the ranking result The object pattern determined to be higher in ranking based on the above can be replaced with a lower one of the reference templates, and when it is replaced, it is stored as a new reference template .
[0007]
In the above method, when the relationship between the reference pattern and the specific object pattern becomes equal to or greater than a predetermined value, the specific object pattern is added to the contents of the plurality of reference patterns. The relevance with the object pattern extracted from the inspection image can be determined while updating the image. Therefore, it is possible to automatically detect objects belonging to a specific category that is abstract and ambiguous with high accuracy. Here, “adding” an object pattern includes adding an object pattern to the reference pattern to increase the reference pattern and replacing any of the reference patterns with an additional object pattern.
[0008]
In a specific aspect of the above method, when the relationship between the object pattern extracted from the inspection image and the reference pattern is equal to or greater than a predetermined value, the inspection image is determined to be an image belonging to a predetermined category. The method further includes a step. In this case, an image belonging to a predetermined category such as a bag image can be automatically detected from a large number of images with a certain degree of accuracy.
[0009]
In another specific aspect of the above method, for example, when the relationship between the reference pattern and a specific object pattern is greater than or equal to a predetermined value, the learning image is a typical image belonging to the predetermined category. If it is determined, the specific object pattern is added to the contents of the plurality of reference patterns. In this case, the above-described plurality of reference patterns can be updated to a more appropriate one using the typical image.
[0010]
In another specific aspect of the above method, the object pattern extracted from the additional image when it is determined that the object pattern extracted from the additional image belonging to the predetermined category is a cause of belonging to the predetermined category. Is further added to the contents of the plurality of reference patterns. In this case, an important object pattern in the typical image can be selected, and the plurality of reference patterns described above can be updated to a more appropriate one.
[0011]
In another specific aspect of the above method, the step of storing the object pattern extracted from the inspection image when the similarity between the object pattern extracted from the inspection image and the reference pattern exceeds a predetermined value Is further provided. In this case, object patterns belonging to a predetermined category such as portraits from a large number of images can be automatically saved with a certain degree of accuracy.
[0012]
In another specific aspect of the above method, the similarity is determined using a relative arrangement relationship between a plurality of object patterns extracted from the inspection image as an element. In this case, an object pattern belonging to a predetermined category such as a portrait can be recognized with higher accuracy.
[0013]
The image recognition apparatus of the present invention includes a plurality of stored reference patterns, It is a collection of various images From the learning image In a state where components that contribute to the determination of whether an image belongs to an abstract predetermined category or not The object pattern of the geometric shape automatically cut out and extracted Excluding the rotation angle element for the shape Compare the reference pattern with a specific object pattern Is a numerical value of Means for adding the specific object pattern to the contents of the plurality of reference patterns, means for extracting an object pattern from the inspection image, and an object pattern extracted from the inspection image A means for comparing the reference pattern, and an association between the object pattern extracted from the inspection image and the reference pattern Is a numerical value of Is greater than or equal to There is something that An image recognition apparatus comprising: a means for determining that the inspection image is an image belonging to a predetermined category, wherein the predetermined shape is selected from the geometrically-shaped object patterns extracted by automatic clipping. The object pattern is removed before the comparison with the reference pattern, and among the object patterns of the geometric shape extracted by automatic cutout, the remaining object patterns that are not removed are ranked, and the ranking result The object pattern determined to be higher in ranking based on the above can be replaced with a lower one of the reference templates, and when it is replaced, it is stored as a new reference template .
[0014]
In the above apparatus, when the relationship between the reference pattern and the specific object pattern becomes equal to or greater than a predetermined value, the specific object pattern is added to the contents of the plurality of reference patterns. The relevance with the object pattern extracted from the inspection image can be determined while updating the image. Therefore, it is possible to automatically detect objects belonging to a specific category that is abstract and ambiguous with high accuracy.
[0015]
The computer program of the present invention includes a computer and a plurality of stored reference patterns. It is a collection of various images From the learning image In a state where components that contribute to the determination of whether an image belongs to an abstract predetermined category or not The object pattern of the geometric shape automatically cut out and extracted Excluding the rotation angle element for the shape Compare the reference pattern with a specific object pattern Is a numerical value of Means for adding the specific object pattern to the contents of the plurality of reference patterns, means for extracting an object pattern from the inspection image, and an object pattern extracted from the inspection image A means for comparing the reference pattern, and an association between the object pattern extracted from the inspection image and the reference pattern Is a numerical value of Is greater than or equal to There is something that A computer program for functioning as a means for determining that the inspection image is an image belonging to a predetermined category, wherein a predetermined one of the geometrically shaped object patterns extracted by automatic cut-out The object pattern is removed before comparison with the reference pattern, and among the object patterns of the geometric shape extracted by automatic cutout, the remaining object patterns that are not removed are ranked, and the ranking is performed. Result of The object pattern determined to be higher in ranking based on the above can be replaced with a lower one of the reference templates, and when it is replaced, it is stored as a new reference template .
[0016]
In the computer in which the computer program is incorporated, when the relationship between the reference pattern and the specific object pattern becomes a predetermined value or more, the specific object pattern is added to the contents of the plurality of reference patterns. The relevance with the object pattern extracted from the inspection image can be determined while updating the reference pattern to an appropriate one. Therefore, it is possible to automatically detect objects belonging to a specific category that is abstract and ambiguous with high accuracy.
[0017]
DETAILED DESCRIPTION OF THE INVENTION
[First Embodiment]
FIG. 1 shows the overall structure of a network system for realizing the image recognition apparatus and method according to the first embodiment.
[0018]
This network system automatically monitors the Internet IN which is a communication network, a large number of WEB servers NS that are connected to the Internet IN and transmit various types of information including images, and image information provided by the large numbers of WEB servers NS. A monitoring system MS that can
[0019]
The WEB server NS is a computer system provided with a communication device for connecting directly or indirectly to the Internet IN, and publishes various homepages via the Internet IN. The home page provided by the WEB server NS includes images belonging to various categories.
[0020]
The monitoring system MS is also a computer system provided with a communication device for connecting directly or indirectly to the Internet IN, and can access home pages provided by a large number of WEB servers NS. Note that the monitoring system MS is an image detection device, that is, an image recognition device for automatically monitoring whether, for example, a spider image is included in a homepage provided by a large number of WEB servers NS.
[0021]
FIG. 2 is a block diagram conceptually illustrating the structure of the monitoring system MS shown in FIG. The illustrated monitoring system MS includes a CPU 21, an input device 22, a display device 23, a storage device 24, and a communication control device 25, similarly to a general computer.
[0022]
The CPU 21 can exchange data with the display device 23, the storage device 24, and the communication control device 25 via the bus 100. Further, the CPU 21 reads a predetermined program or data from the storage device 24 or the communication control device 25 based on an instruction from the input device 22 and executes various processes based on the program and data.
[0023]
More specifically, the CPU 21 uses the object pattern appropriately extracted from the learning image input via the input device 22, the storage device 24, or the like based on an instruction from the input device 22 in the pre-learning program. In addition to the reference template, which is a pattern, the reference template can be optimized. Further, the CPU 21 appropriately extracts an object pattern from an image included in the homepage data obtained via the communication control device 25 based on an instruction from the input device 22 in the homepage monitoring program, and learns this object pattern and learning. The inertia of the image is automatically determined by comparing with the updated reference template.
[0024]
The input device 22 is composed of a keyboard and the like, and outputs a command signal reflecting the intention of the operator who operates the monitoring system MS to the CPU 21 by a GIU operation using the display device 23.
[0025]
The display device 23 includes a display drive circuit that generates a drive signal based on data input from the CPU 21, a CRT that performs necessary display based on the drive signal input from the display drive circuit, and the like. Necessary display is performed based on the command signal.
[0026]
The storage device 24 includes a ROM that stores a plurality of basic programs for operating the monitoring system MS, and a RAM such as a work memory that temporarily stores application programs, input instructions, input data, processing results, and the like. Furthermore, the storage device 24 includes a drive for driving a recording medium that can hold an application program and data by a magnetic or optical technique. The driven recording medium is stored in the storage device 24. It can be fixedly mounted or detachably mounted. The application program includes a pre-learning program and a homepage monitoring program, and the data includes a database including various images.
[0027]
The communication control device 25 is configured by a LAN adapter or the like, and enables communication with the Internet IN using the TCP / IP protocol via a firewall (not shown).
[0028]
Hereinafter, main operations of the monitoring system MS shown in FIG. 1 will be described. FIG. 3 is a flowchart for explaining pre-learning, and FIGS. 4 to 6 are flowcharts for explaining the flowchart of FIG. 3 more specifically.
[0029]
In the monitoring system MS, first, the CPU 21 prepares an initial template from the data stored in the storage device 24 based on an instruction from the input device 22 (step S2). The initial template is a fixed template that does not require user judgment, and a typical bag object can be used as the initial template. Specifically, a partial image of each part of the human body such as a chest, a crotch, and a buttocks that is considered to constitute a heel image is stored in advance in the storage device 24 as an initial template, thereby forming an initial template. An initial template can be prepared by the user. In this case, a haze image is prepared in advance, an object is cut out from this haze image, and this is used as an initial image.
[0030]
Next, the CPU 21 captures a plurality of learning sample images including a typical spider image via the storage device 24 and the communication control device 25 based on an instruction from the input device 22 (step S4). Note that the plurality of learning sample images to be captured do not include only a general wrinkle image, but generally constitute a collection of various images including images that are not recognized as wrinkles.
[0031]
Next, the CPU 21 automatically cuts out an obscene object from each learning sample image, and adds the obscured object to the initially stored initial template while taking the user's judgment, and saves it in the storage device 24. Then, the template is corrected (step S6). In this way, by correcting the initial template using various learning sample images and user judgments, it is possible to obtain a reference template, that is, a reference pattern, adapted to changes in countries, regions, and times. That is, through the above-described steps, a basic reference template for detecting an image that is recognized as a wrinkle on the user's basis can be learned and prepared.
[0032]
The above steps S4 and S6 can be repeated a plurality of times. Thereby, the reference template can be updated a plurality of times, and a reference template that enables a determination closer to the user's determination can be obtained.
[0033]
In addition, in step S4, the plurality of learning sample images that are captured may include only a general eyelid image. In this case, the reference template can be accurately limited to those related to the haze image. On the other hand, there is also a problem that the user's judgment is likely to be affected by the range and tendency of the learning sample image. As a solution, it is conceivable to increase the number of learning sample images and increase the number of reference templates.
[0034]
FIG. 4 is a flowchart for explaining the template correction processing in step S6 of FIG. First, the CPU 21 reads an arbitrary unevaluated specific image among the learning sample images from the storage device 24 or the like (step S61).
[0035]
Next, the CPU 21 determines the intensity of the specific image read in step S61 using the initial template obtained in step S2 of FIG. 3 (step S62). As will be described in detail later, in determining the degree of accuracy, an object is automatically cut out from the target specific image, and the degree of correlation between the cut-out object and the initial template, that is, the degree of association, is output.
[0036]
Next, the CPU 21 determines whether or not the degrees of determination have been determined for all of the learning sample images (step S63), and steps S61 and S62 are repeated until the degrees of evaluation have been completed for all of the learning sample images. .
[0037]
Next, the CPU 21 ranks the degrees of the group of learning sample images based on the degrees obtained for each learning sample image, and displays the ranking result on the display device 23 (step S64).
[0038]
Next, the CPU 21 creates a list of worst objects obtained at the time of ranking the learning sample images, and displays the list and corresponding objects on the display device 23 (step S65).
[0039]
Thereafter, the CPU 21 replaces the worst object corresponding to the wrinkle image determined to be higher in the ranking with the object constituting the initial template, and stores it in the storage device 24 as a reference template (step S66). For example, when the initial template is 10 and five worst objects are added, the remaining five initial templates are left. The initial templates are given a rank, and the initial templates that are deleted when the worst object is added are in the lower rank.
[0040]
Next, the CPU 21 causes the user to appropriately select what is particularly recognized as a wrinkle from all the learning sample images regardless of the ranking via the display device 23 (step S67). Specifically, using the input device 22 and the display device 23, the user selects, for example, the top five that are considered to be a habit by his / her own judgment from the sample images for ranking displayed.
[0041]
When the user selects the cocoon image in step S67, the CPU 21 adds or replaces the object corresponding to the selected cocoon image with the object constituting the basic template, and re-saves it in the storage device 24 as the reference template (step S68). ). For example, if the reference plate whose initial template is updated in step S66 is 10, five objects constituting the eyelid image are added in step S68.
[0042]
If the user does not select the eyelid image in step S67, the CPU 21 stores the reference template obtained in step S66 as it is as the final reference template, and ends the process.
[0043]
FIG. 5 is a flowchart for explaining the process of determining the degree in step S62 of FIG. First, the CPU 21 automatically cuts out an object from a specific image to be determined (step S621). Various known techniques can be used for cutting out the object. For example, as disclosed in the publication cited in the “Prior Art” column, an edge image obtained by gradationizing a target image is extracted, and an area where gradation is continuously increased from an edge image having low gradation is expanded and integrated. The object can be cut out.
[0044]
Next, the CPU 21 removes a meaningless object from the object obtained in step S621 (step S622). Specifically, small objects and objects that are considered to constitute the background are removed from the detected group of objects. For example, when the exclusive area ratio of an object is less than or equal to a predetermined threshold, or even if the exclusive area ratio is greater than or equal to a predetermined threshold, Therefore, it is not used for the determination to improve the processing speed.
[0045]
Next, the CPU 21 calculates an object characteristic for each object remaining in step S622 (step S623). This object characteristic is expressed as a multidimensional vector composed of various parameters such as color, shape, texture, and size. In addition, regarding a shape, a rotation angle cannot be included as an element. Thereby, it is possible to prevent elements such as the posture of an object that is a part of a male or female body from being included in the object characteristics.
[0046]
Next, the CPU 21 performs a comprehensive evaluation of similarity for each object whose characteristics are calculated in step S623 while correlating with all the reference templates, thereby calculating the degree of the image as a whole (step S624).
[0047]
FIG. 6 is a flowchart for explaining the process of determining the degree in step S624 in FIG. First, the CPU 21 calculates the degree of correlation for the specific object for which the object characteristic has been calculated (step S624a). Specifically, the difference between the characteristic of the specific object and the characteristic of the specific reference template is obtained linearly or nonlinearly in consideration of the weight of each parameter, and the degree of correlation, that is, the similarity is comprehensively evaluated. Note that the object characteristics of the reference template are calculated in advance by the same method as described in step S623.
[0048]
Next, the CPU 21 determines whether or not the calculation of the degree of correlation with all the reference templates has been completed for the specific object subjected to the calculation in step S624a (step S624b).
[0049]
If it is determined in step S624b that the correlation degree calculation has not been completed for all the reference templates, the correlation degree calculation in step S624a is sequentially repeated for the remaining reference templates.
[0050]
If it is determined in step S624b that the correlation calculation for all the reference templates has been completed, the CPU 21 determines whether the calculation of the correlation has been completed for all target objects cut out from the determination target image. Is determined (step S624c).
[0051]
If it is determined in step S624b that the correlation calculation for all target objects has not been completed, the correlation calculation in step S624a is sequentially repeated for the remaining target objects.
[0052]
If it is determined in step S624b that the correlation calculation for all target objects has been completed, the CPU 21 calculates the correlation with respect to the reference object obtained for each target object linearly or in consideration of size and arrangement. By further non-linearly adding, an average index relating to relevance, that is, a degree indicating the degree of inertia as a whole specific image is obtained (step S624d). It should be noted that in non-linear processing, if there is a thing with a high degree of correlation, processing that shows a higher degree of accuracy than mere addition, and if there is a thing with a low degree of correlation, processing that shows a lower degree of accuracy than mere addition. For example, it is possible to perform an evaluation such that a value obtained by increasing the correlation value to the third power or fourth power is added.
[0053]
FIG. 7 is a flowchart for explaining monitoring of the WEB site. First, the CPU 21 automatically connects to an appropriate WEB site via the communication control device 25 (step S11).
[0054]
Next, the CPU 21 downloads the image data (that is, the inspection image) of the website of the WEB site via the communication control device 25 and stores the image file in the storage device 24 (step S12). Image files can be collected on a WEB site basis, and a predetermined number of homepage image data can also be collected in a batch. For example, a root directory is defined for a specific WEB site, and a list of image files in which the full path of the image file such as jpg, gif, etc. is recorded is created, and the file format is unified with the specific directory provided in the storage device 24. Can be stored.
[0055]
Next, the CPU 21 determines the degree of the image file using the reference template obtained in step S6 of FIG. 3 (step S13). The determination of the degree of brightness is almost the same as that shown in FIG. 5 except that the degree of the degree is determined by a similar process for a large number of image files instead of a single one.
[0056]
Next, the CPU 21 determines whether or not a habit image exists in these image files (step S14). Specifically, it is determined whether or not the degree of error for each image file exceeds a certain threshold value, and if the degree of error exceeds a predetermined threshold value in any of the image files, a defect image exists. to decide.
[0057]
If it is determined in step S14 that a haze image has been detected, the CPU 21 displays a warning on haze image detection on the display device 23 (step S15). At this time, the CPU 21 can display the detected haze image as a thumbnail or a file list on the display device 23 together with the address of the WEB site and the like, and stores the display information in the storage device 24. This result is automatically notified to a manager who operates another computer system at a remote location by using means such as e-mail.
[0058]
Next, the CPU 21 determines whether or not the degree determination has been completed for all the monitoring target WEB sites, and if it is determined that the degree determination has not been completed for all the target WEB sites, the steps S11 to S15 are performed. Repeat the process.
[0059]
As described above, it is possible to automatically monitor whether or not a cunning image is included in the monitoring target WEB site. At this time, according to the method of the present embodiment, the influence of the posture and background of the object constituting the eyelid image is canceled, and high detection accuracy can be obtained.
[0060]
To date, it has been determined by the human eye whether an image published on the Internet is a spider image. This is a very labor-intensive work, and there is a limit to checking the contents of the Internet that change every day. There is an urgent need for an Internet content provider that aims to provide sound services for users who want to surf the Internet with peace of mind. Recently, businesses that provide IPS (Internet Provider Service) for general consumers, including minor users, are required to check for spider images. The monitoring system MS of the present embodiment is an alternative to the conventional identification method with the naked eye, and enables automatic detection of a haze image on a WEB site by a computer. That is, the present monitoring system MS can realize the identification of the eyelid image, which can only be determined by the human naked eye, as automatic detection and automatic recognition by a computer. This can greatly reduce labor and cost, and can greatly contribute to the automation and management of Internet content. Furthermore, a more rational, low-cost, and flexible system can be constructed by automating image content management and monitoring. Furthermore, there is a benefit for the IPS trader that the possibility of receiving administrative guidance or business suspension by leaving the bag image is further reduced.
[0061]
FIG. 8 is a diagram for visually explaining the flow of data in the operations shown in FIGS. 3 to 7. FIG. 8 (a) shows the pre-learning process of the monitoring system MS, and FIG. The homepage monitoring process of the monitoring system MS is shown.
[0062]
In the pre-learning process shown in FIG. 8 (a), the automatic learning unit classifies the learning sample image into a wrinkle image and a non-wrinkle image using an initial template, and corrects the result by user judgment. Get a reference template.
[0063]
In the homepage monitoring process shown in FIG. 8B, the automatic determination unit automatically separates the monitoring target image into a habit image and a non-haze image based on the reference template obtained in FIG.
[0064]
In the above embodiment, the cocoon image is detected on the WEB site. However, if a cocoon image file exists in another network, for example, an in-house computer network, this can be detected easily and reliably.
[0065]
[Second Embodiment]
The image recognition apparatus and method according to the second embodiment is a modification of the apparatus and method according to the first embodiment for portrait extraction.
[0066]
The pre-learning process is basically the same as that of the first embodiment. Therefore, the overall process is the same as that shown in FIG. 3, the process shown in FIG. 9 is executed as the process corresponding to FIG. 4, and the process shown in FIG. 10 is executed as the process corresponding to FIG. As a process corresponding to FIG. 6, the process shown in FIG. 11 is performed.
[0067]
The initial portrait template can be an object appropriately cut out from the portrait and includes a face, a neck, a hair, and the like. The reason why the hair is included in the face image template in this way is that the actually used face photograph includes the hair. The learning sample image may be, for example, a face photograph of a frontal style of a person, and several patterns of images are prepared for typical ones such as women, men, and thin or dark hair.
[0068]
In addition, when re-adding images after ranking of learning sample images, it is possible to consider the gender, ethnicity, etc. of the portrait to be cut out. In addition, the reference template obtained by correcting the template is a composite of a group of objects (for example, a skin color object including a face and a neck, a hair object) obtained from one learning sample image, and a combination of these positional relationships. It has become a thing. The contours of the objects constituting such a reference template can be manually adjusted in some cases. A plurality of composite reference templates obtained as a result are used for automatic identification and segmentation of portrait images, that is, face image objects.
[0069]
FIG. 12 is a flowchart for explaining portrait cutout. First, the CPU 21 reads image data in the WEB site or the storage device 24 (step S201).
[0070]
Next, the CPU 21 performs a process related to selection of a cut-out processing target range (step S202). Specifically, a preview image corresponding to the image data is displayed on the display device 23. This preview image can be enlarged / reduced in several stages, and a range can be selected in the preview image using an appropriate tool. At this time, the CPU 21 causes the display device 23 to warn that the image quality is not good when the selection range or the original image is smaller than a certain size.
[0071]
Next, the CPU 21 checks the size of the target image (step S203). If the size of the target image does not exceed a certain value, the subsequent processing is performed on the original image as it is. If the size of the target image exceeds a certain value, (1) an ultra-high resolution mode for performing subsequent processing without changing the size of the target image, and (2) a general mode for reducing the size of the target image. To select as appropriate.
[0072]
Next, the CPU 21 determines the portrait degree of the target image obtained from the image file using the reference template obtained in the process of FIG. 9 (step S204). The determination of the portrait degree is almost the same as that shown in FIG.
[0073]
Next, the CPU 21 determines whether or not a portrait exists in these target images (step S205). Specifically, it is judged whether or not the degree of portrait obtained for the target image exceeds a certain threshold value, the layout element of the object, etc., and if the degree of portrait satisfies the predetermined criteria, the portrait exists. Judge that.
[0074]
When calculating the degree of portrait and determining the existence of a portrait, if the reference template that maximizes the correlation, that is, the degree of similarity is the template object with the skin color including the face and neck, it is recognized as a face candidate object. To do. Further, if the reference template having the highest similarity is a hair template object, it is recognized as a hair candidate object. At this time, the similarity is weighted and corrected based on the distance from the center area of the image. If all face candidate objects are adjacent to one or more hair candidate objects and the sum of the contour lines of the adjacent parts is not within a predetermined range, the face candidate object is calculated for portrait Exclude from the target. The same exclusion process is also performed on all hair candidate objects, and automatic discrimination is performed in consideration of the arrangement relationship of the cut out objects. At this time, the number of face candidate objects and the number of hair candidate objects do not necessarily match. Note that if the number of face candidate objects exceeds a predetermined maximum value, those having low similarity and hair candidate objects adjacent thereto are further excluded.
[0075]
Although the above description means extraction of the head object, clothes objects can be extracted in the same manner. In this case, an object within a predetermined expected range is extracted as a clothing candidate object on the opposite side of the hair candidate object adjacent to the face candidate object. The expected range is calculated from the size of the face candidate object, for example.
[0076]
When determining in step S204 that the portrait has been detected, the CPU 21 displays an object obtained from the portrait on the display device 23 (step S206). At this time, all the cutout objects can be displayed, but only some objects within the specified range of the target image can be displayed. The object display method can display an area where the object exists in the target image as a range with an ellipse or the like, and display the object in a tree display without overlapping. If there is an unnecessary one of the cut out objects, any of the tree display objects as described above can be designated by the input device 22 and removed from the object group.
[0077]
Next, the CPU 21 performs processing for accepting contour correction of the cut out object (step S207). The CPU 21 displays a contour line with a clear color on the preview of the target image displayed on the display device 23 corresponding to the clipped object. This contour line can be corrected based on an instruction from the input device 22. For example, a point at which the sharpness is extreme along the contour line is automatically detected at a certain density or less, and an operation handle for the contour line is obtained by connecting these points. And between adjacent points, a point can be compulsorily formed based on the instruction | indication from the input device 22, and an unnecessary point can also be thinned out conversely. The operation handle thus obtained can be corrected by using the input device 22, and appropriate contour detection with the user's judgment can be detected.
[0078]
Next, the CPU 21 performs a process of geometrically cutting out the object whose contour has been determined and saving the cut out image in the storage device 24 (step S208). The object can be cut out along the contour obtained as described above, but can be cut out as an image in which the contour fits in an arbitrary rectangular area, elliptical area, or circular area.
[0079]
As described above, portraits can be automatically cut out from an appropriate image file and stored and stored. The portrait file thus obtained can be attached as a face photo file to an e-mail sent between a mobile phone or a personal computer. Such a portrait file makes it easy to create various images. For example, the portrait file as described above is used for various purposes such as commemorative face photo writing and face photo stickers.
[0080]
Although the present invention has been described based on the above embodiments, the present invention is not limited to the above embodiments. For example, in the above-described embodiment, the eyelid image or portrait is automatically detected, but the method of the present invention can also be applied to the case of automatically detecting an image of another abstract category.
[Brief description of the drawings]
FIG. 1 shows an overall structure of a network system in which an image recognition apparatus and method according to a first embodiment are utilized.
FIG. 2 is a block diagram conceptually illustrating the structure of the monitoring system shown in FIG.
FIG. 3 is a flowchart illustrating pre-learning.
FIG. 4 is a flowchart illustrating a part of prior learning.
FIG. 5 is a flowchart illustrating a part of prior learning.
FIG. 6 is a flowchart illustrating a part of prior learning.
FIG. 7 is a flowchart for explaining monitoring of a WEB site.
FIG. 8A shows the pre-learning process of the monitoring system, and FIG. 8B shows the homepage monitoring process of the monitoring system.
FIG. 9 is a flowchart illustrating a part of pre-learning in the image recognition apparatus according to the second embodiment.
FIG. 10 is a flowchart illustrating a part of pre-learning in the image recognition apparatus according to the second embodiment.
FIG. 11 is a flowchart illustrating a part of pre-learning in the image recognition apparatus according to the second embodiment.
FIG. 12 is a flowchart for explaining portrait image clipping in the image recognition apparatus according to the second embodiment;
[Explanation of symbols]
22 Input device
23 Display device
24 storage devices
25 Communication control device
21 CPU
IN Internet
MS monitoring system

Claims (4)

A plurality of stored reference patterns and a learning image that is a collection of various images are automatically converted into parts that contribute to the determination of whether the image belongs to an abstract predetermined category or not. The geometric pattern of the object pattern cut out and extracted is compared with the shape excluding the element of the rotation angle , and the value obtained by quantifying the relationship between the reference pattern and the specific object pattern has become a predetermined value or more. A step of adding the specific object pattern to the contents of the plurality of reference patterns;
Extracting an object pattern from the inspection image;
Comparing the object pattern extracted from the inspection image with the reference pattern;
A step of determining that the inspection image is an image belonging to a predetermined category when there is an object whose value obtained by quantifying the relationship between the object pattern extracted from the inspection image and the reference pattern is a predetermined value or more. An image recognition method comprising:
Among the object patterns of the geometric shape extracted by automatic cutout, a predetermined object pattern is removed before the comparison with the reference pattern,
Among the object patterns of the geometric shape extracted by automatic cutout, ranking of the object patterns that remain without being removed is performed, and the object pattern determined as the top ranking based on the ranking result is used as the reference template. An image recognition method characterized in that it can be replaced with a lower one of the ranks, and is stored as a new reference template when it is replaced . 2. The image recognition method according to claim 1, wherein a value obtained by quantifying the relationship between the plurality of reference patterns and the specific object pattern is determined by weighting parameters including color and shape. A plurality of stored reference patterns and a learning image that is a collection of various images are automatically converted into parts that contribute to the determination of whether the image belongs to an abstract predetermined category or not. The geometric pattern of the object pattern cut out and extracted is compared with the shape excluding the element of the rotation angle , and the value obtained by quantifying the relationship between the reference pattern and the specific object pattern has become a predetermined value or more. A means for adding the specific object pattern to the contents of the plurality of reference patterns;
Means for extracting an object pattern from the inspection image;
Means for comparing the object pattern extracted from the inspection image with the reference pattern;
Means for determining that the inspection image is an image belonging to a predetermined category when there is an object whose value obtained by quantifying the relationship between the object pattern extracted from the inspection image and the reference pattern is greater than or equal to a predetermined value An image recognition device comprising:
Among the object patterns of the geometric shape extracted by automatic cutout, a predetermined object pattern is removed before the comparison with the reference pattern,
Among the object patterns of the geometric shape extracted by automatic cutout, ranking of the object patterns that remain without being removed is performed, and the object pattern determined as the top ranking based on the ranking result is used as the reference template. The image recognition apparatus can be replaced with a lower one of the ranks, and is stored as a new reference template when replaced . Computer
A plurality of stored reference patterns and a learning image that is a collection of various images are automatically converted into parts that contribute to the determination of whether the image belongs to an abstract predetermined category or not. The geometric pattern of the object pattern cut out and extracted is compared with the shape excluding the element of the rotation angle , and the value obtained by quantifying the relationship between the reference pattern and the specific object pattern has become a predetermined value or more. A means for adding the specific object pattern to the contents of the plurality of reference patterns;
Means for extracting an object pattern from the inspection image;
Means for comparing the object pattern extracted from the inspection image with the reference pattern;
Means for determining that the inspection image is an image belonging to a predetermined category when there is a value whose numerical value is a predetermined value or more of the relationship between the object pattern extracted from the inspection image and the reference pattern A computer program for functioning as
Among the object patterns of the geometric shape extracted by automatic cutout, a predetermined object pattern is removed before the comparison with the reference pattern,
Among the object patterns of the geometric shape extracted by automatic cutout, ranking of the object patterns that remain without being removed is performed, and the object pattern determined to be higher in ranking based on the ranking result is used as the reference template. A computer program characterized in that it can be replaced with a lower one of the ranks, and is stored as a new reference template when it is replaced .

Images