JP2007102362A - Automatic classification category forming device, automatic digital image content classification device and digital image content management system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an automatic classification category forming device automatically forming a category for classifying a digital content (a video or an image), to provide an automatic digital image content classification device performing the automatic classification of the digital image content by the generated category, and to provide a digital image content management system automatically forming the classification category, and performing the automatic classification of the digital image content by the formed classification category. <P>SOLUTION: When a user supplies a group of a plurality of digital image contents to a system 100, the category universal to the contents is investigated, one classification is defined on the basis of the category (112), an attribute of the optionally supplied digital image content is compared with an attribute (121) related to the definition of the classification, and the classification is automatically imparted (123, 124). <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a classification category automatic forming apparatus, a digital image content automatic classification apparatus, and a digital image content management system for automatically classifying a plurality of digital image contents.

In recent years, digital cameras and digital video cameras capable of high-quality shooting at low prices have become widespread, and in addition, the use of large-capacity memory cards and hard disk drive devices has become common, Even general users can hold a large amount of digital image content (images, videos).
As a previously proposed method for classifying and managing objects (contents) such as a large number of images, there is a method by adding metadata. In this method, bibliographic information (text data) is assigned to each object such as an image and managed. Therefore, it is very expensive to add metadata, and it is difficult to recall the added metadata. Diverges by subjectivity, making it difficult to reuse.

  Another proposed method is a method using feature amounts. In this method, since the feature amount is an objective amount, problems such as lack of objectivity and difficulty in reusability as in the case of adding metadata can be avoided. However, there is a problem in that an object that is semantically similar to an object and an object that is similar in feature quantity are not necessarily equivalent for the user. For example, in the similar image search, images having similar colors and shapes are listed, but this does not necessarily give an image that the user feels is semantically similar.

  A technology has already been proposed to deal with the above-mentioned problems and to classify, organize, and manage a large number of objects efficiently and at low cost by combining the method of assigning metadata and the method of using object features. Has been. In this proposal, a plurality of objects are classified according to the feature amount of the object, the classified object set is managed together with the hierarchical relationship of metadata assigned to the objects included in the object set, and the metadata The main feature is to display one or both of the hierarchical relationship and the object set (see, for example, Patent Document 1).

In the technique described in this document, when a user designates an object to be searched from the displayed metadata or the object itself, a similar search is performed on an object set based on the feature amount of the specified object, and a part of the similar search result Displays the hierarchical relationship of metadata held by the set. When the user designates an image that matches the classification viewpoint from the similar search result, the image is registered together with the metadata. In addition, when the user designates the source of classification and the target of classification destination from the displayed metadata and registers the sample object in the target, the object set of the source is automatically classified based on the feature amount of the target sample object.
JP 2002-259410 A (paragraphs 0002 to 0010, FIG. 1)

  However, the technique proposed in Patent Document 1 described above leaves the following problems. That is, searching for a desired digital image content by inputting metadata from the stored digital image content, or performing a search while displaying a large number of content is not enough for general users. This is a difficult and unfamiliar task. Therefore, designing the hierarchical structure of metadata is an extremely difficult task for general users. Furthermore, when a universal metadata structure is used, there is a problem that a large amount of information unnecessary for the user exists, resulting in an inefficient management method.

  On the other hand, a normal user usually desires to classify content based on personal preference and necessity. For example, when shooting around a person, the classification is subdivided based on the presence or absence of the person and the identification of the person itself as the subject, or when shooting with emphasis on composition, etc. Is a natural requirement. However, it is more complicated and difficult for general users to perform these operations by themselves.

As defined in MPEG-7, similar image retrieval based on image feature amounts of digital image content is also widely performed. However, digital image content that the user wants to manage as “similar” is not necessarily an image. It cannot be expressed only by the feature amount.
The present invention has been made in view of the circumstances as described above. A classification category automatic forming apparatus that automatically forms a category for classifying digital content (video, image), and digital by the generated category. An object of the present invention is to provide a digital image content automatic classification apparatus that automatically classifies image content, and a digital image content management system that automatically forms a classification category and automatically classifies the digital image content according to the formed classification category. It is said.

  In the following description of the present invention, the term “digital image content” is intended to encompass both cases where the content is a still image and a moving image (video).

In order to solve the above-described problems, the present application proposes the following techniques.
(1) A designated content group recognizing unit for recognizing a plurality of digital image contents designated as being classified into the same category by the user as one designated content group related to the digital image content, and the designated content group recognizing unit. The category to which the digital image content including the plurality of digital image contents should belong is attributed to one of the universal attributes for all of the plurality of digital image contents recognized as the designated content group. An automatic classification category forming apparatus comprising: category forming means for automatically defining.

  In the classification category automatic forming apparatus of (1) above, when a user supplies a plurality of digital image contents to this apparatus in order to automatically form a universal category, the designated content group recognition means performs these. Recognized as one designated content group related to digital image content. For the group of digital image contents recognized in this way by the designated content group recognition means, the category forming means uses the relative attributes among the universal attributes for all of the plurality of digital image contents recognized as one designated content group. The category to which the digital image content including the plurality of digital image contents should belong is automatically defined by one attribute having a small variance. This category is one classification (a group defined by the category) to which the corresponding one of a large number of unclassified digital image contents should belong. The user simply supplies this device to automatically form a universal category for classifying a group of digital image content as a group, and the automatic classification category forming device is as described above. A category (classification) can be automatically formed. Therefore, on the user side, it is possible to obtain a desired classification method based on one's own preference and necessity without requiring complicated recognition such as construction of metadata and its hierarchical structure.

(2) Category holding means for holding a plurality of categories defined based on attributes of digital image content assumed as classification targets for classifying digital image content, and attributes of the supplied digital image content A category to which the supplied digital image content should belong, based on a comparison between the attribute extracting unit to extract, the attribute extracted by the attribute extracting unit and the attribute relating to the category definition held by the category holding unit; An automatic digital image content classification apparatus comprising: category selection means for selecting; and registration means for classifying and registering the supplied digital image content into the category selected by the category selection means.

  In the digital image content automatic classification apparatus of (2) above, the attribute extraction unit extracts the attribute of the digital image content supplied with the intention of giving the classification from the user. For the extracted attribute, the category selection means selects a category to which the supplied digital image content should belong based on a comparison between the attribute and the attribute related to the category definition held by the category holding means, Further, the registration unit classifies and registers the supplied digital image content into the category selected by the category selection unit. Therefore, when viewed from the user side, it is automatically classified (registered as belonging to the classification) simply by supplying the digital image content automatic classification device to the digital image content intended to be classified. Without requiring complicated recognition such as the construction of metadata and its hierarchical structure, it is possible to receive a classification attached to the digital image content based on its own preference and necessity.

(3) A designated content group recognizing unit for recognizing a plurality of digital image contents designated as being classified into the same category by the user as one designated content group related to the digital image content, and the designated content group recognizing unit. The category to which the digital image content including the plurality of digital image contents should belong is attributed to one of the universal attributes for all of the plurality of digital image contents recognized as the designated content group. Category forming means for automatically defining, category holding means for holding a plurality of categories defined by the category forming means, attribute extracting means for extracting attributes of the supplied digital image content, and the attribute extracting means Attributes extracted by Category selection means for selecting a category to which the supplied digital image content should belong based on a comparison with an attribute relating to the definition of the category held by the category holding means, and the supplied digital image content as the category A digital image content management system comprising: registration means for classifying and registering in a category selected by the selection means.

  In the digital image content management system of (3) above, when the user supplies a plurality of pieces of digital image content to this apparatus in order to automatically form a universal category, the designated content group recognition means performs these. Recognized as one designated content group related to digital image content. For the group of digital image contents recognized in this way by the designated content group recognition means, the category forming means uses the relative attributes among the universal attributes for all of the plurality of digital image contents recognized as one designated content group. The category to which the digital image content including the plurality of digital image contents should belong is automatically defined by one attribute having a small variance.

  A plurality of categories automatically defined in this way are held in the category holding means. The attribute extraction means extracts the attribute of the digital image content supplied with the intention of giving a classification from the user. For the extracted attribute, the category selection means selects a category to which the supplied digital image content should belong based on a comparison between the attribute and the attribute related to the category definition held by the category holding means, Further, the registration unit classifies and registers the supplied digital image content into the category selected by the category selection unit.

  Thus, based on their own preferences and needs, the user can simply supply multiple batches of digital image content to the device to automatically form a universal category into the category ( Classification) can be formed, and a category of a desired classification can be formed and held in the apparatus without requiring complicated recognition such as construction of metadata and its hierarchical structure.

  In addition, by simply supplying digital image content that is desired to be classified to a device in which such a state has been created, it is automatically classified (registered as belonging to that category), so metadata and its It is possible to receive a classification attached to the digital image content based on one's preference and necessity without requiring complicated recognition such as construction of a hierarchical structure.

(4) Representative content display means for displaying a predetermined digital image content belonging to each category held by the category holding means as representative content representing each category, and displayed by the representative content display means (3) further comprising digital image content search means for searching for and outputting digital image content belonging to the category represented by the representative content selected by the user from the representative content selected. Digital image content management system.

  In the digital image content management system of (4) above, particularly in the operation of the digital image content management system of (3), the digital image content belonging to each category automatically defined by the category forming means and held by the category holding means. A predetermined one of them is displayed as representative content representing each category by the representative content display means.

  The digital image content belonging to the category represented by the representative content selected by the user among the representative contents displayed by the representative content display means is searched for and output by the digital image content search means. The user can easily and clearly recognize what the category (classification by the category) automatically formed on the device side is, and such simple displayed by the representative content display means. It becomes possible to search and output digital image content belonging to a category represented by a clear display.

(5) It further comprises upper hierarchy category setting means for automatically setting an upper hierarchy category of the category represented by the representative content selected by the user among the representative contents displayed by the representative content display means, When the representative content is selected by the user, the digital image content search means performs a search from the category of the upper hierarchy set by the upper hierarchy category setting means with respect to the category represented by the selected representative content, (4) The digital image content management system according to (4), characterized in that the search is executed for a category in a lower hierarchy of the category of the hierarchy in which the search is performed.

  In the digital image content management system of (5) above, the digital image content management system of (4) is represented by the representative content selected by the user among the representative content displayed by the representative content display means. The category of the upper hierarchy of the category is automatically set by the upper hierarchy category setting means. Furthermore, when the representative content is selected by the user, the digital image content search means performs a search from the category of the upper hierarchy set by the upper hierarchy category setting means with respect to the category represented by the selected representative content, Next, a search is performed for a category in a lower hierarchy of the category of the hierarchy in which the search is performed. Hierarchical classification is automatically constructed, and an efficient search can be performed using this.

(6) The category selecting unit is supplied based on a comparison between the attribute extracted by the attribute extracting unit and the attribute related to the category definition held by the category holding unit for the supplied digital image content. The digital image content is configured to select one or more categories to which the digital image content should belong, and the registration unit classifies the supplied digital image content into one or more categories selected by the category selection unit. The digital image content management system according to (3), which is configured to be registered.

  In the digital image content management system of (6) above, the supplied digital image content is held by the attribute extraction means and the category holding means, particularly in the operation of the digital image content management system of (3). A plurality of categories to which the supplied digital image content should belong can be selected based on a comparison with an attribute relating to the definition of the category, so that a digital image corresponding to both one category and another category The content can be searched by any of those categories (classification), and natural classification or search can be performed according to the actual situation.

(7) The representative content display means is configured to display one of the digital image contents belonging to each category held by the category holding means as the representative content or a thumbnail thereof, and the digital image The content search unit searches for and outputs the representative image selected by the user or the digital image content belonging to the category represented by the thumbnail from the digital image content as the representative content displayed by the representative content display unit. (4) The digital image content management system according to (4), wherein

  In the digital image content management system of (7) above, particularly in the operation of the digital image content management system of (4), the representative content displayed by the representative content display means belongs to each category held by the category holding means. Because it is the digital image content itself or its thumbnail, the user can easily and clearly recognize what category (classification by it) automatically formed on the device side, and It becomes possible to search and output digital image content belonging to the category represented by such a simple and clear display displayed by the representative content display means.

(8) The representative content display means is an order having a typical property having a relatively high degree of attribution with respect to the attribute relating to the definition of the category among the digital image contents belonging to each category held by the category holding means. (4) The digital image content management system according to (4), which is configured to perform display in a format emphasized in this order.

  In the above (8) digital image content management system, particularly in the operation of the digital image content management system of (4), among the digital image content belonging to each category held by the category holding means, the attribute relating to the definition of the category. It is displayed by the representative content display means as a display having a typical property with a relatively high degree of attribution, or a display in a format emphasized along this order. For this reason, the user can easily grasp the features (attributes) for each category intuitively.

(9) The digital image content management according to (3), wherein the category forming unit handles a generation location of the digital image content as one attribute as an attribute of the digital image content applied to the definition of the category. system.
In the digital image content management system of (9) above, particularly in the operation of the digital image content management system of (3), digital image content obtained by a digital camera, a digital video camera, etc. is extremely common. Since the generation location (photographing location) is handled as one attribute, versatility is expected to be applied to the classification of digital image content acquired by digital image content acquisition means that are widely used like these cameras.

(10) The digital image content management system according to (3), wherein the category forming means handles the composition of the digital image content as one attribute as the attribute of the digital image content applied to the definition of the category. .
In the digital image content management system according to (10) above, the digital image content management system according to (3) particularly in the digital image content acquired by a digital camera, a digital video camera, etc. Since such a composition is handled as one attribute, versatility is expected to be applied to classification of digital image content acquired by digital image content acquisition means that are widely used like these cameras.

(11) The category forming means treats that the digital image content is a person image as one attribute as an attribute of the digital image content applied to the definition of the category. Image content management system.
In the digital image content management system of (11), the digital image content management system of (3) is an extremely common phenomenon particularly in digital image content acquired by a digital camera, a digital video camera, or the like. Since the image content is a human image as one attribute, versatility is expected to be applied to the classification of digital image content acquired by digital image content acquisition means that are widely used like these cameras.

(12) The digital signal according to (3), wherein the category forming means handles, as one attribute, that the digital image content is an image of a specific person as an attribute of the digital image content applied to the definition of the category. Image content management system.
In the digital image content management system of (12) above, in particular, in the operation of the digital image content management system of (3), the greatest concern for many users regarding the digital image content acquired by a digital camera, a digital video camera, or the like. In order to treat that the digital image content is an image of a specific person as one attribute, it is applied to the classification of digital image content acquired by digital image content acquisition means that are widely spread like these cameras. Versatile is expected.

(13) The digital image content according to (3), wherein the category forming means handles the image similarity of the digital image content as one attribute as an attribute of the digital image content applied to the definition of the category. Management system.
In the digital image content management system of (13) above, the classification relating to the digital image content is inherently dependent on some similarity as an image, particularly in the operation of the digital image content management system of (3). Since the universal premise that there is one is handled as one attribute, versatility is expected to be applied to classification of digital image content obtained by various means.

(14) The digital image content management according to (3), wherein the category forming unit treats digital image content metadata as one attribute as an attribute of the digital image content applied to the definition of the category. system.
In the digital image content management system of (14), in particular, in the operation of the digital image content management system of (3), the metadata representing the attribute on which the classification related to the digital image content depends is handled as one attribute. The present invention is expected to be useful for application to classification of digital image content obtained by various means having a function of attaching metadata to the user.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings to be referred to below, for the sake of convenience, the main part that is the subject of the description is exaggerated as appropriate, and other than the main part is appropriately simplified or omitted.
FIG. 1 is a conceptual diagram showing the present invention. A digital image content management system 100 according to an embodiment of the present invention is a classification category automatic forming apparatus that automatically forms universal categories for classifying a group of a plurality of digital image contents as a group. 110, a digital image content automatic classification device 120 that automatically classifies (registers as belonging to the classification) a digital image content that is arbitrarily supplied, and a classification category automatic as an embodiment. The image forming apparatus 110 and the digital image content automatic classification device 120 are configured to include an arithmetic processing device 130 and a storage device 140 that can be considered to be included as functional units thereof.

  The classification category automatic forming apparatus 110 recognizes a plurality of digital image contents designated as those classified into the same category by a user as a designated content group recognizing unit 111 for recognizing as a designated content group related to the digital image content, and the designated content Digital image content including a plurality of digital image contents by one attribute having a relatively small variance among all of the plurality of digital image contents recognized as one designated content group by the group recognition unit 111 Category forming means 112 that automatically defines the category to which the user belongs, and these means function in cooperation with the arithmetic processing unit 130 and the storage device 140 in this block diagram. It is configured as follows. The arithmetic processing device 130 is configured by, for example, an arithmetic processing function unit of a computer, and the storage device 140 is a large-capacity storage device such as a hard disk drive device and stores a plurality of digital image contents.

  Further, the digital image content automatic classification device 120 includes a category holding unit 121 that holds a plurality of categories defined depending on the attributes of the digital image content assumed as a classification target in order to classify the digital image content, and a supply The attribute extraction means 122 for extracting the attribute of the digital image content that has been extracted, and the comparison based on the attribute extracted by the attribute extraction means 122 and the attribute relating to the category definition held by the category holding means 121 Category selection means 123 for selecting a category to which the digital image content should belong, and registration means 124 for classifying and registering the supplied digital image content into the category selected by the category selection means 123, and Each of these means And it is configured to function by cooperation with processor 130 and storage device 140 in diagram.

  In the digital image content management system 100 of this embodiment, the category held by the category holding means 121 of the digital image content automatic classification apparatus 120 is automatically set by the category formation means 112 of the above-described classification category automatic formation apparatus 110. The category (data representing this) is supplied and held through the arithmetic processing unit 130. Note that the classification category automatic forming apparatus 110 and the digital image content automatic classification apparatus 120 can be configured to have separate arithmetic processing devices (130) and storage devices (140) as independent devices, In this case, the category (data representing this) held by the category holding unit 121 of the digital image content automatic classification device 120 is common to the classification category automatic formation device 110 and the digital image content automatic classification device 120, for example. It can be supplied off-line from the arithmetic processing unit (130) through a transmission line.

  In the digital image content management system 100 configured as described above, when a user arbitrarily selects a plurality of digital image contents to automatically form a universal category and supplies them to this apparatus, a classification category is obtained. The designated content group recognition unit 111 on the automatic forming apparatus 110 side recognizes these as one designated content group related to the digital image content.

  For the group of digital image contents recognized in this way by the designated content group recognition unit 111, the category forming unit 112 has universal attributes for all of the plurality of digital image contents recognized as one designated content group. A category to which a digital image content including the plurality of digital image contents should belong is automatically defined by one attribute having a relatively small variance.

A plurality of categories automatically defined in this way are transmitted to the digital image content automatic classification device 120 side through the arithmetic processing device 130 in the expression on the block diagram, and held in the category holding means 121 of the same device.
After the state as described above is prepared, if the digital image content is supplied through the arithmetic processing unit 130 (operation thereof) in the expression on the block diagram with the intention of giving a classification from the user, the supplied digital image content The attribute extracting unit 122 extracts the attribute. For the extracted attribute, the category selection means 123 determines whether the supplied digital image content is based on a comparison between the attribute of the digital image content and the attribute related to the category definition held in the category holding means 121 in advance. The category to be attributed is selected, and the registration unit 124 further classifies and registers the supplied digital image content into the category selected by the category selection unit 122.

  Therefore, the user simply supplies a group of digital image contents to the apparatus to automatically form a universal category based on his / her preference and necessity, and the classification category automatic forming apparatus. 110 automatically forms a category (classification), so that a category of a desired classification is formed and held in the apparatus without requiring complicated recognition such as construction of metadata and its hierarchical structure. Is possible.

  In addition, by simply supplying digital image content that is desired to be classified to a device in which such a state has been created, it is automatically classified (registered as belonging to that category), so metadata and its It is possible to receive a classification related to the digital image content based on its own preference and necessity without requiring complicated recognition such as construction of a hierarchical structure.

  In the digital image content management system 100 according to the present embodiment, the digital image content belonging to each category held by the category holding unit 121 is a predetermined one (usually one of them). Representative content display means 125 that displays the representative content representing each category), and representative content selected by the user among the representative contents displayed by the representative content display means 125. Digital image content search means 126 for searching and outputting digital image content belonging to the category is further provided.

For this reason, a predetermined one of the digital image contents belonging to each category automatically defined by the category forming means 112 and held by the category holding means 121 represents each category by the representative content display means 125. Displayed as representative content.
The digital image content belonging to the category represented by the representative content selected by the user among the representative contents displayed by the representative content display means 125 is searched for and output by the digital image content search means 126. . By viewing the representative content displayed by the representative content display means 125, the user can easily and clearly recognize what the category (classification based on the category) automatically formed on the device side is. The digital image content belonging to the category represented by such a simple and clear display displayed by the representative content display means 125 can be searched and output.

  The representative content displayed by the representative content display means 125 is a predetermined one of the digital image contents belonging to each category held by the category holding means 121. Therefore, in this embodiment, classification category automatic formation is performed. This means that the digital image content belongs to each category formed by the category forming unit 112 of the apparatus 110 and held by the category holding unit 121. Therefore, when the representative content display unit 125a similar to the above-described representative content display unit 125 is provided on the classification category automatic forming apparatus 110 side and the category forming unit 112 automatically forms a category (classification definition), A predetermined one of the plurality of digital image contents that are the basis of the formation may be configured to be displayed as a representative by the representative content display means 125a.

  On the other hand, in the digital image content management system 100 of the present embodiment, the category in the upper hierarchy of the category represented by the representative content selected by the user among the representative content displayed by the representative content display means 125 is automatically selected. Further, the digital image content search unit 126, when the representative content is selected by the user, the upper layer category setting unit 127 regarding the category represented by the selected representative content. It searches from the category of the upper hierarchy set by (3), Then, it searches for the category of the lower hierarchy of the category of the hierarchy which performed the said search.

According to such a configuration, the upper hierarchy category of the category represented by the representative content selected by the user among the representative contents displayed by the representative content display means 125 is automatically set by the upper hierarchy category setting means 127. Is set.
Further, when the representative content is selected by the user, the digital image content search unit 126 searches for the category represented by the selected representative content from the upper layer category set by the upper layer category setting unit 127. Then, the search is executed for the category in the lower hierarchy of the category of the hierarchy in which the search is performed. Hierarchical classification is automatically constructed, and an efficient search can be performed using this.

The user side intuitively understands the category represented by the simple and clear display displayed by the representative content display means 125 without requiring complicated recognition such as construction of metadata and its hierarchical structure, and the category. It is possible to search for and output digital image content belonging to the classification by.
It should be noted that an upper hierarchy category setting means 127a similar to the above described upper hierarchy category setting means 127 is provided on the classification category automatic forming apparatus 110 side, and a category (classification definition) is automatically formed by the category forming means 112. It may be configured to automatically set the upper hierarchy category. In this way, the upper category is automatically set on the classification category automatic forming apparatus 110 side, and the set category is supplied to the digital image content automatic classification apparatus 120 side through the arithmetic processing unit 130 in the representation on the block diagram. As a result, the digital image content automatic classification apparatus 120 can be configured to realize a state in which an upper hierarchy category is set. In this case, the burden on the digital image content automatic classification device 120 side can be relatively reduced.

  In the digital image content management system 100 described above, the category selection unit 123 compares the attribute extracted by the attribute extraction unit 122 with respect to the attribute related to the category definition held by the category holding unit 121 for the supplied digital image content. And the registration unit 124 selects one or more categories to which the supplied digital image content should belong, and the registration unit 124 selects the supplied digital image content by the category selection unit 123. It is configured to be classified into a plurality of categories and registered.

  As described above, a plurality of categories to which the supplied digital image content should belong are not necessarily limited to a single category, so that the category corresponds to either a certain category or another category. Such digital image contents can be classified and registered into a plurality of corresponding categories, and can be searched by any of those categories (classifications), and natural classification and search according to the actual situation can be performed. Can be.

  Further, in the digital image content management system 100 in the present embodiment, the representative content display means 125 (substantially the same for 125a) further includes a digital image belonging to each category held by the category holding means 121 as the representative content. One of the contents or a thumbnail thereof is configured to be displayed, and the digital image content search means 126 is a representative selected by the user among the digital image contents as the representative contents displayed by the representative content display means 125. The digital image content belonging to the category represented by the content or its thumbnail is searched and output.

  With this configuration, the representative content displayed by the representative content display unit 125 is the digital image content itself or its thumbnail belonging to each category held by the category holding unit 121. It is possible to easily and clearly recognize what the categories (classifications) automatically formed on the system) side are, and such simple displayed by the representative content display means 125. It becomes possible to search and output digital image content belonging to a category represented by a clear display.

  Furthermore, in the digital image content management system of the present embodiment, consideration is given to improving the convenience of display of the representative content as described above. In other words, the representative content display means 125 includes a category. Of the digital image content belonging to each category held by the holding means 121, an order having a typical property having a relatively high degree of attribution with respect to the attribute relating to the definition of the category, or emphasizing along this order It is configured to display in the specified format.

  In the above, “the order having a typical property with a relatively high degree of attribution with respect to the attribute relating to the definition of the category” means, for example, that the definition of the category relates to “Mountain of Japan” and the category The multiple digital image contents that were relied on in the formation process as the basis of formation are some of the “mountains in Japan” with a profile similar to that of Mt. Fuji called “XX Fuji” throughout Japan. In such a case, the order in which the profile resembles “Mt. Fuji” itself corresponds to this in some cases.

  In such a case, the profile of “Mount Fuji” itself is most similar (if Mount Fuji itself is included in the classification, it is an image of Mount Fuji itself). , The mountain thumbnail) is displayed by the representative content display means 125. For this reason, the user can easily grasp the features (attributes) for each category intuitively. The displayed image (content) is not necessarily limited to one.

In the digital image content management system as an embodiment of the present invention as described above, the following events can be considered as the events handled by the category forming unit 112 as the attributes of the digital image content applied to the definition of the category. That is:
(A) Generation location of digital image content (b) Composition of digital image content (c) Digital image content is a person image (d) Digital image content is an image of a specific person (e) Digital image content Image similarity (f) Digital image content metadata (content of the data)
In the case of (a) and (b) described above, the generation location (photographing location) and composition of the digital image content, which is very common in digital image content acquired by a digital camera, digital video camera, etc., have one attribute. Therefore, versatility is expected to be applied to the classification of digital image content acquired by digital image content acquisition means that are widely spread like these cameras.

  In the case of the above (c), it is treated as one attribute that the digital image content, which is a very common phenomenon in digital image content acquired by a digital camera, a digital video camera, or the like, is a person image. Therefore, even if divided, in the case of (d), since the digital image content that is the greatest concern for many users is an image of a specific person, it is handled as one attribute. Generality is expected to be applied to the classification of digital image content acquired by popular digital image content acquisition means.

Furthermore, in the case of (e), various means are used to treat the universal premise that the classification relating to the digital image content inherently relies on some similarity as an image as one attribute. The versatility is expected to be applied to the classification of digital image content acquired in (1).
Furthermore, in the case of (f), in order to handle the metadata representing the attribute on which the classification related to the digital image content is relied on as one attribute, various means having a function to attach metadata that has become popular in recent years. It is expected to be useful for applying to the classification of digital image content acquired in (1).

The essential points of the digital image content management system 100 as one embodiment of the present invention have been described above. Next, the digital image content management system 100 or the essential components thereof will be described in more detail and specifically. The part will be described.
The automatic classification performed in the digital image content management system 100 (category extraction / selection processing applied to define the classification and processing of classifying the supplied digital image content according to the classification criteria depending on the category) is as follows. It is configured to include three processes.
(1) A user designates a plurality of digital image contents classified as the same category (hereinafter referred to as an instance category) as samples.
(2) Extract attributes of a plurality of designated digital image contents and calculate a classification standard for each instance category.
(3) The digital image content to be newly classified is classified into the instance category closest to the attribute.

In addition, the instance category is automatically associated with a class category, which is a superordinate concept (described above as a function related to the upper hierarchy category setting means 127 and 127a). The two categories (class category and instance category) allow the user to search and display with flexibility. That is, when a user searches for digital image content,
A method of directly selecting a desired instance category.
-Method of selecting an instance category included in a class category after selecting a higher class category.
The following two methods can be selected.

  The category forming unit 112 in the representation on the block diagram of FIG. 1 includes a classification criterion determining unit (functional unit) that generates a classification criterion for instance categories related to a plurality of digital image contents as samples specified by the user. The processing in this functional part is expressed by generalization as in the following Equation 1.

In Equation 1, N is the number of digital image contents designated by the user to generate a category, and k represents an attribute for determining an instance category. For example, k = 1 is a vector representing the composition. That is, d ^k _i is a feature vector of the attribute K of the digital image content, and d ^k _ave is an average vector of the feature vectors. Furthermore, W _k is a weighting coefficient for comparing the variance values of the attributes. W _k is determined empirically to allow comparison. V _k (variance value of feature vector) is calculated for each attribute of the designated digital image content using Equation 1, and the attribute with the smallest value of V _k is estimated as the classification criterion of the instance category desired by the user. To do.

In the above, the method of calculating the feature vector corresponding to each attribute will be described later.
Next, the functions of the category holding unit 121, the attribute extracting unit 122, and the category selecting unit 123 in the expression on the block diagram of FIG. 1 are held in the attribute and category holding unit 121 extracted by the attribute extracting unit 122. The category to which the supplied digital image content should belong is selected on the basis of the comparison (similarity) with the attribute, and this is a process generalized by the following Equation 2. It can be seen that this is a function provided in the similarity determination means to be executed.

In Equation 2, for the existing instance category, the distance D _k between the feature vector corresponding to the attribute k used as the classification reference for the digital image content classified in the instance category and the feature vector of the digital image content to be classified is _represented by Equation 2 And is classified into the instance category having the smallest D _k .
Instead of the processing of Formula 2, the following processing of Formula 3 may be performed.

In Equation 3, d ^k _ave is an average vector of feature vectors of digital image content belonging to the instance category.
Depending on the value of D _k ,
Used when the digital image content with the smallest D _k is displayed on behalf of the instance category (for example, an icon image corresponding to the instance category).
When displaying the digital image content classified into the instance category, the display is made easier for the user to recognize in the descending order of _Dk .
In the above description, the digital image content is described as belonging to one instance category, but may be included in a plurality of instance categories.

  On the other hand, as described above, the six events of (a) to (f) are listed as examples of events handled by the category forming unit 112 as attributes of digital image content applied to the definition of the category. These events naturally relate to the categories held in the category holding means 121, and are also events related to the data that the attribute extraction means 122 and the digital image content search means 126 rely on in their processing.

The explanation will be supplemented with respect to the events (a) to (f) listed as examples. That is, in the embodiment of the present invention, a generation location extracting unit, a composition calculating unit, a person detecting unit, a person recognizing unit, and an image similarity calculation are performed by paying attention to these events (a) to (f). It can be seen that means and metadata extraction means are provided.
Among these means, a specific example will be given of the generation place extracting means. In other words, the “digital still camera image file format standard Exif 2.2” defines a standard for recording the captured latitude and longitude together with digital image content. This information is extracted to generate a feature vector (x, y), which is data representing the generation location with the latitude x and longitude y in the feature vector.

Next, specific examples of composition calculation means will be given. That is, the composition determination method is performed by calculating the degree of attention in the same digital content (for example, a still image). This constitutes a feature vector (f ₁ , f ₂ ,..., F _n ) from the attention level of each pixel constituting the digital content and the position of the pixel, where the dimension of the feature vector is the number of pixels. Yes, f ₁ , f ₂ ,..., F _n are the attention levels of each pixel. “Attention degree” is disclosed in Japanese Patent Laid-Open No. 2004-228993 (referred to in Japanese Patent Application Laid-Open No. 2001-122070 “Attention area extraction device and automatic composition determination device using the same”). Is a synonym for what has already been disclosed in “Detailed Disclosure”), and in short, a region of interest specified by physical characteristics, identified by performing evaluation in a manner that matches human subjectivity. It is.

  FIG. 2 is a diagram for explaining the concept of the person detecting means. The person detection means in this example is specified by specifying a group of instance categories by the user in order to specify an object (person) included in the digital image content for which a category (basis of classification definition) is to be detected. Then, the universal and relatively small attribute of the specified digital image content is recognized.

  Part (a) of FIG. 2 is an example when the user designates the contents 1 to 4 as belonging to the same instance category. In this example, since a person is shown in all, it is determined as an instance category of a person photo. Various methods for detecting a human face are known. For example, a method disclosed in Japanese Patent Application Laid-Open No. 8-63597 can be used. In this method, a human face is detected based on the degree of matching between the contour and the template. According to the detection result of the human face, the feature vector is (n), which is a one-dimensional vector. Here, n is the number of human faces to be detected.

  Part (b) of FIG. 2 is an example when the user designates the contents 5 to 8 as belonging to the same instance category. In this example, since the same person is shown in all, it is determined as the instance category of the specific person. Various methods for recognizing a person's face are also known. For example, the method disclosed in Japanese Patent Laid-Open No. 5-20442 can be used. In this method, a feature pattern is extracted from a face image (Fourier spectrum), and personal identification is performed using an identification dictionary file.

The feature vector is (h ₁ , h ₂ ,..., H _n ), which is an n-dimensional vector. h ₁ , h ₂ ,..., h _n correspond to persons included when the instance category is generated, and are set to 1 if included in the digital image content and set to 0 if not included.
FIG. 3 is a diagram for explaining the concept of the image similarity calculation means. Part (a) of FIG. 3 is a photograph of mountainous landscapes at various places, and part (b) of FIG. 3 is a photograph of coastal scenery at various places. (A), (b) It can be judged that the similarity of each group of images is strong. Image similarity is determined by generating a feature vector representing the visual impression of an image as follows. That is, the target image is divided into predetermined blocks, for example, 16 × 16 blocks, and then the average value of the color of each block is calculated. Further, a feature vector is generated using the calculated color and block position. That is, the feature vector is:
(R ¹ , g ¹ , r ² , g ² ,..., R ²⁵⁶ , g ²⁵⁶ ), which is a 512 (16 × 16 × 2) dimensional vector. Here, when R, G, and B are R components, G components, and B components, r and g are average values in each block of values obtained by dividing R and G by R + G + B. These average values are generalized and expressed by the following equations 4 to 5.

Here, m ^k is the number of pixels of the block k. Of course, R, G, and B may be used instead of r and g.
On the other hand, with regard to the metadata extraction means, since the metadata itself is more general rather than unique to the image, the following document with more general disclosure regarding its handling is cited and explained. Incorporated as. In other words: Concept Based Query Expansion, Y.Qiu, Proc.SIGIR-93,16 th ACM International Conference on Research and Development in
Information Retrieval, 1993 discloses that a feature vector indicating the similarity of words can be defined as follows using a large number of documents.

For the feature vectors (d ₁ , d ₂ ,..., D _n ), this is n-dimensional (the number of documents referred to), and d _k is a weight obtained from the frequency of appearance in each document. . As disclosed in the same document, the above vector may be normalized and used.
FIG. 4 is a diagram illustrating the relationship between the instance category and the class category. Regarding the example described above, the class category is a generation location, composition, person, metadata, and image similarity, and the instance category belongs to any class category. In the example of FIG. 2 described above, (a) creates an instance category with digital content including a person, and (b) creates an instance category including a specific person. Both instance categories can be considered to belong to the superordinate concept, namely the person's class category.

When searching for a person photo, the user can search for desired digital content by specifying the class category of the person photo and then specifying the instance category belonging to the class category of the person photo.
FIG. 5 is a flowchart showing a process of extracting an attribute as a determination criterion for an instance category from digital content designated by the user. In FIG. 5, first, in order to create an instance category, a plurality of sample digital contents are designated (step S501). Next, the content attribute of the digital content is extracted (step S502). Further, the classification criterion determination means calculates the determination criterion of the instance category from the content attribute (step S503). The class attribute is determined from the content attribute based on the determination criterion calculated in step S503.

  FIG. 6 is a flowchart showing processing for automatically classifying digital contents. First, digital content to be classified is read (step S601). Next, content attributes are extracted (step S602). Further, the above-mentioned similarity determination means determines which instance category is closest to the classification (step S603).

It is a conceptual diagram showing this invention. It is a figure for demonstrating the concept of a person detection means. It is a figure for demonstrating the concept of an image similarity calculation means. It is a figure which shows the relationship between an instance category and a class category. It is a flowchart which shows the process which extracts the attribute used as the determination criterion of an instance category from the digital content designated by the user. It is a flowchart showing the process which classifies a digital content automatically.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Digital image content management system 110 ... Classification category automatic formation apparatus 111 ... Designated content group recognition means 112 ... Category formation means 120 ... Digital image content automatic classification apparatus 121 ... Category holding means 122 ... Attribute extraction means 123 ... Category selection means 124 ... Registration means 125 ... Representative content display means 125a ... Representative content display means 126 ... Digital image content search means 127 ... Higher hierarchy category setting means 127a ... Higher hierarchy category setting means 130 ... Operation processing device 140 ... Storage device

Claims (14)

  A designated content group recognizing means for recognizing a plurality of digital image contents designated as those to be classified into the same category by the user as one designated content group related to the digital image content, and one designated content by the designated content group recognizing means. The category to which the digital image content including the plurality of digital image contents should belong is automatically attributed to one of the universal attributes of all of the plurality of digital image contents recognized as a group by the relatively small attribute. A category category automatic forming apparatus comprising: category forming means for defining.   Category holding means for holding a plurality of categories defined based on attributes of digital image content assumed as classification targets for classifying digital image content, and attributes for extracting the attributes of the supplied digital image content A category for selecting a category to which the supplied digital image content should belong based on a comparison between the extraction means and the attribute extracted by the attribute extraction means and the attribute relating to the category definition held by the category holding means An automatic digital image content classification device comprising: selection means; and registration means for classifying and registering the supplied digital image content into the category selected by the category selection means.   A designated content group recognizing means for recognizing a plurality of digital image contents designated as those to be classified into the same category by the user as one designated content group related to the digital image content, and one designated content by the designated content group recognizing means. The category to which the digital image content including the plurality of digital image contents should belong is automatically attributed to one of the universal attributes of all of the plurality of digital image contents recognized as a group by the relatively small attribute. Category forming means for defining, category holding means for holding a plurality of categories defined by the category forming means, attribute extracting means for extracting attributes of the supplied digital image content, and extracted by the attribute extracting means Attribute and the above Category selection means for selecting a category to which the supplied digital image content should belong based on comparison with the attribute relating to the definition of the category held by the category holding means, and the category selection of the supplied digital image content A digital image content management system comprising: registration means for classifying and registering in a category selected by the means.   Representative content display means for displaying a predetermined digital image content belonging to each category held by the category holding means as representative content representing each category, and a representative displayed by the representative content display means The digital image content search means for searching for and outputting digital image content belonging to the category represented by the representative content selected by the user among the content, further comprising: Digital image content management system.   The digital image content further comprises upper hierarchy category setting means for automatically setting an upper hierarchy category of the category represented by the representative content selected by the user among the representative contents displayed by the representative content display means. When the representative content is selected by the user, the search means performs a search from the category of the upper hierarchy set by the upper hierarchy category setting means with respect to the category represented by the selected representative content, and then the search 5. The digital image content management system according to claim 4, wherein the search is performed for a category in a lower hierarchy of the category of the hierarchy that has been performed.   The category selection unit is configured to provide the supplied digital image content based on a comparison between the attribute extracted by the attribute extraction unit and the attribute related to the category definition held by the category holding unit. The registration means classifies and registers the supplied digital image content into one or more categories selected by the category selection means. The digital image content management system according to claim 3, wherein the digital image content management system is configured as follows.   The representative content display means is configured to display one of digital image contents belonging to each category held by the category holding means as the representative content or a thumbnail thereof, and the digital image content search means Is configured to search for and output the representative image selected by the user or the digital image content belonging to the category represented by the thumbnail among the digital image content as the representative content displayed by the representative content display means. 5. The digital image content management system according to claim 4, wherein the digital image content management system is configured.   The representative content display means has an order having a typical property with a relatively high degree of attribution with respect to the attribute relating to the definition of the category among the digital image contents belonging to each category held by the category holding means. 5. The digital image content management system according to claim 4, wherein the digital image content management system is configured to perform display in a highlighted format along this order.   4. The digital image content management system according to claim 3, wherein the category forming unit handles a generation location of the digital image content as one attribute as an attribute of the digital image content applied to the definition of the category. .   4. The digital image content management system according to claim 3, wherein the category forming unit handles the composition of the digital image content as one attribute as an attribute of the digital image content applied to the definition of the category.   4. The digital image according to claim 3, wherein the category forming unit handles, as one attribute, that the digital image content is a person image as an attribute of the digital image content applied to the definition of the category. Content management system.   4. The digital image according to claim 3, wherein the category forming unit handles, as one attribute, that the digital image content is an image of a specific person as an attribute of the digital image content applied to the definition of the category. Content management system.   4. The digital image content management according to claim 3, wherein the category forming unit treats the image similarity of the digital image content as one attribute as an attribute of the digital image content applied to the definition of the category. system. 4. The digital image content management system according to claim 3, wherein the category forming unit handles metadata of digital image content as one attribute as an attribute of digital image content applied to the definition of the category. .

Images