JP2003099778A - Image recognition method and device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image recognition method and the like capable of automatically detecting an image belonging to a specific category. SOLUTION: In previous learning processing shown in a figure 8a, an automatic learning part sorts learning sample images between indecent images and decent images by using an initial template. When the result is corrected according to determination by the user, a reference template is obtained. In home page monitoring processing shown in a figure 8b, an automatic determining part sorts monitoring objective images between the indecent images and the decent images on the basis of the reference template obtained in the figure 8a.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image recognition method and apparatus for detecting an image such as a photograph belonging to a specific category, and a program for realizing these.

[0002]

2. Description of the Related Art Japanese Unexamined Patent Publication No. 11-25271 discloses a method for extracting an in-image object from a photograph. In this method, the gradation edge image is gradually enlarged by utilizing the continuity of the gradation to obtain the object in the image. By using this method, an image similar to a specific image can be automatically searched. That is, it is possible to appropriately extract a meaningful object from a photographic image or the like, and it is also possible to automatically search for an image similar to a target specific image (key image) from a large number of photographic images or the like.

[0003]

However, the method disclosed in the above publication cannot accurately detect an image belonging to a specific category, such as an obscene image. In other words, categories have an abstract and ambiguous conceptual spread, and the criteria for whether or not they belong to a particular category are often not always clear, so that criteria may change over time. It has not been possible to automatically and appropriately determine whether or not an image belongs to a specific category. In addition, when the target of judgment is an image of a living thing, there is inevitably a variety of images such as changes in its posture and the type / presence / absence of background objects, but a systematic category that can handle such diversity. There is no judgment method yet.

Therefore, an object of the present invention is to provide an image recognition method and the like which can automatically extract objects belonging to a specific category.

Another object of the present invention is to provide an image recognition method and the like which can automatically detect images belonging to a specific category.

[0006]

In order to solve the above-mentioned problems, the image recognition method of the present invention compares a plurality of stored reference patterns with an object pattern extracted from a learning image, for example, the above-mentioned reference Adding the specific object pattern to the contents of the plurality of reference patterns when the relationship between the pattern and the specific object pattern exceeds a predetermined level; extracting the object pattern from the inspection image; And a step of comparing the object pattern extracted from the inspection image with the reference pattern.

In the above method, when the relation between the reference pattern and the specific object pattern exceeds a predetermined level, the specific object pattern is added to the contents of the plurality of reference patterns. Can be updated to an appropriate one and the relevance to the object pattern extracted from the inspection image can be determined.
Therefore, it is possible to automatically detect an object that belongs to an abstract and ambiguous specific category with high accuracy. Here, "adding" with respect to the object pattern includes adding the object pattern to the reference pattern to increase the reference pattern, and replacing any one of the reference patterns with the additional object pattern.

In a specific aspect of the above method, the inspection image is an image belonging to a predetermined category when the relationship between the object pattern extracted from the inspection image and the reference pattern is equal to or more than a predetermined value. And a step of determining In this case, an image belonging to a predetermined category such as an obscene image can be automatically detected from a large number of images with a certain degree of certainty.

In another specific mode of the above method, for example, when the relationship between the reference pattern and a specific object pattern exceeds a predetermined value, the learning image is further classified into a typical image belonging to the predetermined category. If it is determined that the specific object pattern is added, the specific object pattern is added to the contents of the plurality of reference patterns. In this case, the typical image can be used to update the plurality of reference patterns described above to more appropriate ones.

In another specific mode of the above method, when it is determined that the object pattern extracted from the additional image belonging to the predetermined category causes the belonging to the predetermined category, the object pattern is extracted from the additional image. The method further comprises the step of adding the created object pattern to the contents of the plurality of reference patterns. In this case, an important object pattern in the typical image can be selected, and the plurality of reference patterns described above can be updated to more appropriate ones.

In another specific aspect of the above method, when the similarity between the object pattern extracted from the inspection image and the reference pattern is more than a predetermined value, the object pattern extracted from the inspection image is The method further includes a step of storing. In this case, object patterns belonging to a predetermined category such as portraits can be automatically saved from a large number of images with a certain degree of certainty.

In another specific mode of the above method, the similarity is judged by using a relative arrangement relationship between a plurality of object patterns extracted from the inspection image as an element. In this case, object patterns belonging to a predetermined category such as portraits can be recognized with higher accuracy.

The image recognition apparatus of the present invention compares a plurality of stored reference patterns with an object pattern extracted from the learning image, and determines whether the reference pattern and the specific object pattern have a predetermined relationship or more. In this case, means for adding the specific object pattern to the contents of the plurality of reference patterns, means for extracting the object pattern from the inspection image, object pattern extracted from the inspection image and the reference pattern And means for comparing.

In the above apparatus, when the relation between the reference pattern and the specific object pattern exceeds a predetermined level, the specific object pattern is added to the contents of the plurality of reference patterns. Can be updated to an appropriate one and the relevance to the object pattern extracted from the inspection image can be determined.
Therefore, it is possible to automatically detect an object that belongs to an abstract and ambiguous specific category with high accuracy.

According to the computer program of the present invention, a computer compares a plurality of stored reference patterns with an object pattern extracted from a learning image, and the relation between the reference pattern and a specific object pattern is more than a predetermined value. In this case, the specific object pattern is added to the contents of the plurality of reference patterns, the object pattern is extracted from the inspection image, the object pattern extracted from the inspection image, and the reference pattern. It functions as a means to compare and.

In the computer incorporating the above computer program, the specific object pattern is added to the contents of the plurality of reference patterns when the relationship between the reference pattern and the specific object pattern exceeds a predetermined level. , It is possible to determine the relevance to the object pattern extracted from the inspection image while updating the plurality of reference patterns to appropriate ones. Therefore,
It is possible to automatically detect an object that belongs to an abstract and ambiguous specific category with high accuracy.

[0017]

BEST MODE FOR CARRYING OUT THE INVENTION [First Embodiment] FIG. 1 shows the overall structure of a network system for realizing an image recognition apparatus and method according to the first embodiment.

This network system automatically uses the Internet IN, which is a communication network, a large number of WEB servers NS connected to the Internet IN to transmit various information including images, and image information provided by the large number of WEB servers NS. And a monitoring system MS that can monitor the target.

The WEB server NS is the Internet IN
Is a computer system equipped with a communication device for connecting directly or indirectly to the Internet IN
Various homepages are published via. In addition, WE
Images belonging to various categories are included in the home page provided by the B server NS.

The monitoring system MS is also the Internet IN
Is a computer system equipped with a communication device for directly or indirectly connecting to the web site, and is capable of accessing home pages provided by a large number of WEB servers NS. The monitoring system MS is an image detecting device, that is, an image recognizing device for automatically monitoring whether or not, for example, an obscene image is included in homepages provided by a large number of WEB servers NS.

FIG. 2 is a block diagram conceptually explaining the structure of the monitoring system MS shown in FIG. The monitoring system MS shown in the figure is a CPU similar to a general computer.
21, an input device 22, a display device 23, a storage device 24, and a communication control device 25.

The CPU 21 can exchange data with the display device 23, the storage device 24, and the communication control device 25 via the bus 100. Also, CPU2
1 is a storage device 2 based on an instruction from the input device 22.
4 and the communication control device 25 to read a predetermined program and data, and execute various processes based on these programs and data.

More specifically, the CPU 21 appropriately extracts an object from the learning image input via the input device 22, the storage device 24, etc. based on an instruction from the input device 22 in the pre-learning program. Add the pattern to the reference template, which is the reference pattern,
The reference template can be optimized. Also,
CPU21, in the homepage monitoring program,
Based on the instruction from the input device 22, the communication control device 25
An object pattern is appropriately extracted from the image included in the homepage data obtained through, and the object pattern is compared with the reference template updated by learning to automatically determine the obscenity of the image.

The input device 22 is composed of a keyboard or the like, and outputs a command signal reflecting the intention of the operator who operates the monitoring system MS to the CPU 21 by GIU operation using the display device 23.

The display device 23 is composed of a display drive circuit which generates a drive signal based on the data input from the CPU 21, and a CRT which performs a necessary display based on the drive signal input from the display drive circuit. Required display is performed based on the command signal from the CPU 21.

The storage device 24 includes a ROM storing a plurality of basic programs for operating the monitoring system MS, and the like.
Application program, input instruction, input data,
A RAM such as a work memory for temporarily storing processing results and the like is provided. Further, the storage device 24 includes a drive for driving a recording medium capable of holding an application program and data by a magnetic or optical method, and the driven recording medium is the storage device 2.
4 can be fixedly provided or can be detachably mounted. The application programs include a pre-learning program and a homepage monitoring program, and the data includes a database including various images.

The communication control device 25 is composed of a LAN adapter or the like and is connected to the Internet IN via TCP / I via a firewall (not shown).
It enables communication by the P protocol.

The main operation of the monitoring system MS shown in FIG. 1 will be described below. FIG. 3 is a flowchart for explaining prior learning, and FIGS. 4 to 6 are flowcharts for more specifically explaining the flowchart of FIG.

In the monitoring system MS, first, the CPU 21
However, based on an instruction from the input device 22, the storage device 24
An initial template is prepared from the data stored in (step S2). The initial template is a fixed one that does not require the user's judgment, and a typical obscene object can be used as the initial template. Specifically, partial images of various parts of the human body, such as the chest, crotch, and hips, which are considered to constitute an obscene image, are stored in advance in the storage device 24 as the initial template to form the initial template. The initial template may be prepared by the user. In this case, an obscene image is prepared in advance, an object is cut out from this obscene image, and this is used as an initial image.

Next, the CPU 21 takes in a plurality of learning sample images including a typical obscene image via the storage device 24 and the communication control device 25 based on an instruction from the input device 22 (step S4). It should be noted that the plurality of sample images for learning that are captured do not include only general obscene images, but generally form a collection of various images including images that are not recognized as obscene.

Next, the CPU 21 automatically cuts out an obscene object from each learning sample image, adds this obscene object to the initially stored initial template while taking user judgment, and saves it in the storage device 24. By doing so, the template is corrected (step S6). In this way, by correcting the initial template using various learning sample images and user judgments, it is possible to obtain a reference template, that is, a reference pattern, which matches the transition of the country, region, or era. That is, through the above steps, a basic reference template for detecting an image recognized as obscene by the user's reference.
It is possible to learn and prepare (image data that is a component that contributes to an obscene image).

The steps S4 and S6 can be repeated a plurality of times. As a result, the reference template can be updated a plurality of times, and a reference template that enables a determination closer to the user's determination can be obtained.

In addition, in step S4, the plurality of learning sample images to be taken in may include only general obscene images. In this case, the reference template can be accurately limited to those related to the obscene image. On the other hand, there is also a problem that the judgment of the user is likely to be influenced by the range and tendency of the learning sample image. One possible solution to this problem is to increase the number of sample images for learning and increase the number of reference templates.

FIG. 4 is a flow chart for explaining the template correction processing in step S6 of FIG.
First, the CPU 21 reads out an unspecified arbitrary image out of the learning sample images from the storage device 24 or the like (step S61).

Next, the CPU 21 executes step S2 in FIG.
Using the initial template obtained in step S61, the degree of indecency of the specific image read in step S61 is determined (step S6).
2). As will be described in detail later, in determining the degree of indecency, the object is automatically cut out from the target specific image, and the degree of indecency that represents the correlation between the cut out object and the initial template, that is, the relation, is output.

Next, the CPU 21 determines whether or not the indecent degree has been determined for all the learning sample images (step S63), and the indecent degree evaluation for all the learning sample images is completed in step S61, Repeat S62.

Next, the CPU 21 ranks the obscenity of the group of learning sample images based on the obscenity obtained for each learning sample image, and displays the ranking result on the display device 23 (step S6).
4).

Next, the CPU 21 creates a list of the worst objects obtained in the obscenity ranking of each learning sample image, and displays the list and the corresponding objects on the display device 23 (step S65).

After that, the CPU 21 replaces the worst object corresponding to the obscene image determined to be in the higher ranking with the object forming the initial template and stores it in the storage device 24 as the reference template (step S66). For example, if the initial template is 10
And if you add 5 worst objects,
There are 5 remaining initial templates. Note that the initial templates are ranked, and the initial templates that are deleted when the worst object is added are below the ranking.

Next, the CPU 21 appropriately causes the user to select, through the display device 23, a sample image for all learning, which is recognized as obscene, regardless of the ranking (step S67). Specifically, using the input device 22 and the display device 23, the user selects, for example, the top five items that are considered to be obscene in his / her own judgment from the learning sample images displayed in the ranking.

When the user selects the obscene image in step S67, the CPU 21 adds or replaces the object corresponding to the selected obscene image with the object forming the basic template and re-saves it in the storage device 24 as the reference template. (Step S68). For example, if the number of reference plates for which the initial template has been updated in step S66 is 10, five objects that make up an obscene image will be added in step S68.

When the user does not select an obscene image in step S67, the CPU 21 saves the reference template obtained in step S66 as it is as a final reference template, and ends the process.

FIG. 5 is a flow chart for explaining the indecency degree determination processing in step S62 of FIG. First, the CPU 21 automatically cuts out an object from the specific image to be determined (step S621). Various publicly known techniques can be used to cut out the object. For example, as disclosed in the publication cited in the column of "Prior Art", an edge image in which a target image is gradation is taken out, and an area in which the gradation is continuously increased is integrated from a low-gradation edge image. Objects can be cut out by

Next, the CPU 21 removes meaningless objects from the objects obtained in step S621 (step S622). Specifically, small objects or objects that are considered to make up the background are removed from the set of detected objects. For example, if the occupied area ratio of an object is less than or equal to a predetermined threshold value, or if the occupied area ratio is greater than or equal to a predetermined threshold value, the object detected biased to the corner portion of the image is Therefore, it is not used for the determination to improve the processing speed.

Next, the CPU 21 calculates the object characteristics for each of the objects remaining in step S622 (step S623). This object characteristic is expressed as a multidimensional vector composed of various parameters such as color, shape, texture, size and the like. Note that the shape may not include the rotation angle as an element. As a result, it is possible to prevent elements such as the posture of an object that is a part of the body of a male or a female from being included in the object characteristics.

Next, the CPU 21 calculates the degree of indecency as the entire image by performing a comprehensive evaluation of the similarity of each object whose characteristics are calculated in step S623 while taking a correlation with all the reference templates (step). S624).

FIG. 6 is a flow chart for explaining the obscene degree determination processing in step S624 of FIG. First, the CPU 21 calculates the degree of correlation for the specific object for which the object characteristic has been calculated (step S62).
4a). Specifically, the difference between the characteristic of the specific object and the characteristic of the specific reference template is obtained linearly or non-linearly in consideration of the weighting of each parameter, and the degree of correlation, that is, the similarity is comprehensively evaluated. The object characteristic of the reference template is calculated in advance by the same method as described in step S623.

Next, the CPU 21 executes step S624a.
It is determined whether or not the calculation of the degree of correlation with all the reference templates has been completed for the specific object that was the target of calculation in step S624b.

If it is determined in step S624b that the correlation degree calculation has not been completed for all reference templates, the correlation degree calculation in step S624a is sequentially repeated for the remaining reference templates.

When it is determined in step S624b that the correlation degree calculation has been completed for all the reference templates, the CPU 21 ends the correlation degree calculation for all target objects cut out from the image to be determined. It is determined whether or not (step S624c).

When it is determined in step S624b that the correlation degree calculation has not been completed for all target objects, the correlation degree calculation in step S624a is sequentially repeated for the remaining target objects.

When it is determined in step S624b that the correlation degree calculation has been completed for all target objects, the CPU 21 considers the correlation degree with respect to the reference object obtained for each target object in consideration of size and arrangement. By adding further linearly or non-linearly,
An average index relating to relevance, that is, an indecent degree indicating the degree of indecentness of the entire specific image is obtained (step S624).
d). In addition, in the non-linear processing, if there is a high degree of correlation, it shows a higher degree of obscenity than mere addition, and if there is a low correlation degree, a degree of obscenity lower than that of mere addition For example, it is possible to perform an evaluation in which a value obtained by raising the value of the degree of correlation to the third power or fourth power is added.

FIG. 7 is a flow chart for explaining the monitoring of the WEB site. First, the CPU 21 automatically connects to an appropriate WEB site via the communication control device 25 (step S11).

Next, the CPU 21 downloads the image data of the home page of the WEB site (that is, the inspection image) via the communication control device 25, and saves the image file in the storage device 24 (step S12). The image files can be collected on a WEB site basis, and it is also possible to collectively collect image data of a predetermined number of home pages. For example, a root directory is set for a specific WEB site, a list of image files in which the full paths of image files such as jpg and gif are recorded is created from the root directory, and a file format unified in the specific directory provided in the storage device 24 is created. Can be stored.

Next, the CPU 21 executes step S6 of FIG.
The indecency degree of the image file is judged using the reference template obtained in step S13. The determination of the degree of indecentness is almost the same as that shown in FIG. 5, except that the degree of indecentness is determined by a similar process for a large number of image files instead of a single image.

Next, the CPU 21 determines whether or not an obscene image exists in these image files (step S).
14). Specifically, it is determined whether the obscenity degree for each image file exceeds a certain threshold value, and if the obscenity degree exceeds a predetermined threshold value in any of the image files, an obscene image is present. to decide.

When it is determined in step S14 that an obscene image is detected, the CPU 21 displays a warning about the detection of the obscene image on the display device 23 (step S15). At this time, CP
The U21 can display the detected obscene image as a thumbnail or a file list on the display device 23 together with the address of the WEB site, and saves the display information in the storage device 24. This result is automatically notified to an administrator who operates another computer system at a remote place by means of mail or the like.

Next, the CPU 21 determines whether or not the indecentness determination has been completed for all of the monitored WEB sites, and if it is determined that the indecentness has not been completed for all the target WEB sites, The processes of S11 to S15 are repeated.

As described above, it is possible to automatically monitor whether the WEB site to be monitored contains an obscene image. At this time, according to the method of the present embodiment, the influence of the posture and the background of the object forming the obscene image is canceled, and high detection accuracy can be obtained.

Until now, whether or not an image published on the Internet is an obscene image has been judged by the human naked eye. This is a very labor-intensive task, and there is a limit to the daily checking of Internet contents. For users who want to provide safe service for users who want to surf the internet with peace of mind, there is an urgent need to develop a technology that can automatically detect obscene images by means other than naked eye work. Recently, businesses that provide IPS (Internet Provider Service) to general consumers including minor users are obliged to check for obscene images. The monitoring system MS of the present embodiment is an alternative to the conventional naked eye identification method, and enables automatic detection of an obscene image on a WEB site by a computer. That is, the present monitoring system MS can realize the conventional obscure image identification, which could only be judged by the human naked eye, by automatic detection and automatic recognition by the computer.
This can save a great deal of labor and cost, and can greatly contribute to automation of Internet content management and monitoring. In addition, by automating the management and monitoring of image contents, a more rational, low cost, and flexible system can be constructed. Furthermore, for IPS providers,
There is a benefit that the possibility of being subject to administrative guidance and the disposition of business suspension due to the liberation of obscene images is further reduced.

FIG. 8 is a diagram for visually explaining the flow of data in the operations shown in FIGS. 3 to 7, and FIG.
8 shows a pre-learning process of the monitoring system MS, and FIG.
(B) shows the homepage monitoring process of the monitoring system MS.

In the pre-learning process shown in FIG. 8A, the automatic learning unit classifies the learning sample image into the obscene image and the non-obscene image using the initial template, and corrects the result according to the user's judgment. To obtain a reference template.

In the homepage monitoring process shown in FIG. 8B, the automatic determination unit automatically classifies the monitoring target image into an obscene image and a non-obscene image based on the reference template obtained in FIG. 8A. To do.

In the above embodiment, an obscene image is detected on the WEB site. However, if an obscene image file exists in a network other than that, such as an in-house computer network, it can be detected easily and reliably. You can also

[Second Embodiment] The image recognition apparatus and method according to the second embodiment is a modification of the apparatus and method of the first embodiment for extracting a portrait.

The pre-learning process is basically the same as that of the first embodiment. Therefore, as the overall processing, the same processing as in FIG. 3 is performed, the processing shown in FIG. 9 is performed as the processing corresponding to FIG. 4, and the processing shown in FIG. 10 is performed as the processing corresponding to FIG. The process shown in FIG. 11 is performed as the process corresponding to FIG.

The initial template of the portrait can be an object appropriately cut out from the portrait, and the face,
Including neck and hair. In this way, the reason why hair is included in the facial image template is that the actually used facial photograph also includes hair. The sample image for learning can be, for example, a frontal portrait of a person, a woman,
Prepare some patterns of images for typical things such as men, light or dark hair.

When re-adding the learning sample images after ranking, the sex, ethnicity, etc. of the portrait to be cut out can be taken into consideration. In addition, the reference template obtained by modifying the template is a group of objects (for example, a skin color object including a face and a neck, a hair object) obtained from one learning sample image, and a complex combination of these positional relationships. It has become. In some cases, the contours of the objects constituting such a reference template can be manually adjusted. The resulting composite reference templates are used in automatic identification and segmentation of portrait images, i.e. facial image objects.

FIG. 12 is a flow chart for explaining the cutout of a portrait. First, the CPU 21 reads the image data in the WEB site or the storage device 24 (step S
201).

Next, the CPU 21 carries out a process relating to the selection of the processing target range for clipping (step S202).
Specifically, the preview image corresponding to the image data is displayed on the display device 23. This preview image can be enlarged / reduced in several stages, and the range can be selected in the preview image using an appropriate tool. At this time, the CPU 21 causes the display device 23 to warn that the image quality is not good when the selection range and the original image are smaller than a certain size.

Next, the CPU 21 checks the size of the target image (step S203). If the size of the target image does not exceed the predetermined value, the subsequent processing is performed on the original image as it is. When the size of the target image exceeds a certain value, the operator is made to appropriately select the ultra-high resolution mode in which the subsequent processing is performed without changing the size of the target image and the general mode in which the size of the target image is reduced.

Next, the CPU 21 determines the portrait degree of the target image obtained from the image file by using the reference template obtained in the process of FIG. 9 (step S204).
The determination of the portrait degree is almost the same as that shown in FIG.

Next, the CPU 21 determines whether or not there is a portrait in these target images (step S20).
5). Specifically, whether or not the portrait degree obtained for the target image exceeds a certain threshold value, the layout element of the object, etc. are judged, and if the portrait degree satisfies a predetermined criterion, the portrait exists. Then decide.

In calculating the portrait degree and determining the presence / absence of the portrait, if the reference template having the largest correlation degree, that is, the similarity degree with respect to the clipped object is a skin-colored template object including the face and neck, a face candidate is selected. Qualify as an object. In addition, if the reference template having the highest degree of similarity is the template object for hair, it is recognized as a hair candidate object. At this time, the degree of similarity is weighted and corrected by the distance from the central region of the image. For all face candidate objects, adjacent to one or more hair candidate objects, if the total of the contour lines of the adjacent parts is not within a predetermined range of the whole, the face candidate objects are calculated for the portrait degree. Exclude from the target. The same exclusion processing is performed for all hair candidate objects, and automatic discrimination is performed in consideration of the arrangement relationship of the cut out objects. At this time, the numbers of face candidate objects and hair candidate objects do not necessarily match. In addition,
If the number of face candidate objects exceeds a predetermined maximum value, the one having a low degree of similarity and the hair candidate objects adjacent thereto are further excluded.

Although the above description means the extraction of the object of the head, the object of the clothing can be similarly extracted. In this case, an object within a predetermined prediction range is extracted as a clothing candidate object on the opposite side of the hair candidate object adjacent to the face candidate object. This expected range is calculated from the size of the face candidate object, for example.

When it is determined that the portrait is detected in step S204, the CPU 21 displays the object obtained from the portrait on the display device 23 (step S206). At this time, it is possible to display all the cut-out objects, but it is also possible to display only some of the objects in the specified range of the target image. As a method of displaying the objects, a region in which the object exists in the target image is displayed in a range of an ellipse and the objects in the region are displayed in a tree without overlapping. If any of the cut out objects is unnecessary, any one of the above tree-displayed objects can be designated by the input device 22 and removed from the group of objects.

Next, the CPU 21 performs a process for accepting the contour correction of the cut out object (step S2).
07). The CPU 21 overlays the preview of the target image displayed on the display device 23 and displays the contour line in a clear color corresponding to the cut-out object. This contour line can be modified based on an instruction from the input device 22. For example, points along which the sharpness has an extreme value are automatically detected at a certain density or less, and these points are connected to obtain an operation handle for the contour line. Then, it is possible to forcibly form points between adjacent points based on an instruction from the input device 22, and conversely, unnecessary points can be thinned out. The operation handle obtained in this way can be corrected by using the input device 22, and appropriate contour detection can be performed with the user's judgment added.

Next, the CPU 21 performs a process of geometrically cutting out the contour-determined object and saving the cut-out image in the storage device 24 (step S208). The object can be cut out along the contour obtained as described above, but can be cut out as an image such that the contour fits within an arbitrary rectangular area, elliptical area, or circular area.

As described above, it is possible to automatically and simply cut out a portrait from an appropriate image file and save and store it. The portrait file obtained in this way can be attached as a facial photograph file to an email sent between a mobile phone and a personal computer. In addition, such a portrait file simplifies the creation of various images. For example, the portrait file as described above is used for various purposes such as a memorial face photo message and a face photo sticker.

Although the present invention has been described with reference to the embodiments, the present invention is not limited to the above embodiments.
For example, in the above embodiment, an obscene image or a portrait is automatically detected, but the method of the present invention can also be applied to the case of automatically detecting an image in another abstract category.

[Brief description of drawings]

FIG. 1 shows an overall structure of a network system in which an image recognition device and method according to a first embodiment are utilized.

FIG. 2 is a block diagram conceptually explaining the structure of the monitoring system shown in FIG.

FIG. 3 is a flowchart illustrating prior learning.

FIG. 4 is a flowchart illustrating a part of prior learning.

FIG. 5 is a flowchart illustrating a part of prior learning.

FIG. 6 is a flowchart illustrating a part of prior learning.

FIG. 7 is a flowchart illustrating monitoring of a WEB site.

FIG. 8A shows a pre-learning process of the monitoring system, and FIG. 8B shows a homepage monitoring process of the monitoring system.

FIG. 9 is a flowchart illustrating a part of pre-learning in the image recognition device according to the second embodiment.

FIG. 10 is a flowchart illustrating a part of pre-learning in the image recognition device according to the second embodiment.

FIG. 11 is a flowchart illustrating a part of pre-learning in the image recognition device according to the second embodiment.

FIG. 12 is a flowchart illustrating a cutout of a portrait image in the image recognition apparatus according to the second embodiment.

[Explanation of symbols]

22 Input device 23 Display 24 storage 25 Communication control device 21 CPU IN Internet MS monitoring system

Claims (8)

[Claims] 1. When a plurality of stored reference patterns are compared with an object pattern extracted from a learning image, and when the relationship between the reference pattern and a specific object pattern exceeds a predetermined level, A step of adding a specific object pattern to the contents of the plurality of reference patterns; a step of extracting an object pattern from an inspection image; and a step of comparing the object pattern extracted from the inspection image with the reference pattern. Image recognition method provided. 2. The method further comprises the step of determining that the inspection image is an image belonging to a predetermined category when the relationship between the object pattern extracted from the inspection image and the reference pattern is a predetermined value or more. The image recognition method according to claim 1, wherein: 3. The specific object pattern is added to the contents of the plurality of reference patterns when it is determined that the learning image is a typical image belonging to the predetermined category. 2. The image recognition method described in 2. 4. When the object pattern extracted from the additional image belonging to the predetermined category is determined to be a cause of belonging to the predetermined category, the object pattern extracted from the additional image is set to the plurality of reference patterns. The image recognition method according to claim 2, further comprising a step of adding to the contents of the above. 5. The method further comprises the step of storing the object pattern extracted from the inspection image when the similarity between the object pattern extracted from the inspection image and the reference pattern becomes a predetermined value or more. The image recognition method according to claim 1. 6. An element is a relative positional relationship among a plurality of object patterns extracted from the inspection image,
The image recognition method according to claim 5, wherein the similarity is determined. 7. A plurality of stored reference patterns are compared with an object pattern extracted from a learning image, and when the relationship between the reference pattern and a specific object pattern exceeds a predetermined value, Means for adding a specific object pattern to the contents of the plurality of reference patterns; means for extracting an object pattern from the inspection image; and means for comparing the object pattern extracted from the inspection image with the reference pattern. An image recognition device provided. 8. When a computer compares a plurality of stored reference patterns with an object pattern extracted from a learning image, and the relationship between the reference pattern and a specific object pattern exceeds a predetermined level. A means for adding the specific object pattern to the contents of the plurality of reference patterns; a means for extracting the object pattern from the inspection image; and a comparison between the object pattern extracted from the inspection image and the reference pattern. A computer program that functions as a means.